Manual annotation revision (corpus 1)

Excluding any errors in manual annotation is hard to be achieved especially when the annotation structure is complex and data size is not small as ours. But the quality of manual annotation is one of the core factors in sequence learning because the estimation ability of a statistical model considerably depends on learning data extracted from the annotation.

With the progress of our experiments, we go on correcting any small errors in terms of both manual annotation and original input string. An example of the latter is inserting unnecessary space in URL when typing online a reference that provokes a wrong tokenization in Bilbo system. Since the first attempt of revision, which was intended to modify directly learning and test data, we have continued to correct dispersed tiny errors. After several attempt, we found that correcting them directly in our corpus was more practical in case we change feature-extracting strategy even though searching errors in the corpus is more annoying than that in learning and test data.

These little changes in the corpus always influence final annotation result and we currently have the following accuracies with the same settings on feature and field types with the previous experiments.

 There is a light difference by comparison with the first data revision. Especially, total number of tokens and the number of punctuation marks decrease (6632 vs. 6569 and 2090 vs. 2024) that comes mostly from the change of tokenization. The correction also allows us automatically remove several unintended field labels such as ‘author’, ‘name’ and ‘region’, whereas we keep ‘ref’ and ‘genname’ in spite of their week frequency (because they occur often in corpus 2). With this revised version of corpus 1, we try out a series of experiments for the incorporation of external resources such as proper noun list.