Note annotation on Corpus level 2

We continue our experiments with the corpus 2. In the previous post, we showed that a well-defined set of three different feature types improves note classification accuracy. Now we verify the usefulness of note classification on automatic annotation of bibliographical reference in notes.

 Two CRF models are constructed on two different sets respectively. First set is ‘NotesCL’ that consists of the classified notes into positive category after applying a SVM with the strategy 10. Second set is ‘NotesOR’ that contains all 1532 notes without classification. We randomly selected 70% of a set as learning data and the remaining 30% as test data just as the previous experiments.

 The above figure shows the annotation result of these two models with some selected fileds. The gray cells are three most important fields, surname, forename and title of article. Embolden value means that the corresponding CRF model better estimates on the field than the other model. Total accuracy of our approach is 87.28% and it outperforms the baseline (85.16%). Considering the baseline model is learned on a larger number of notes (1532 vs. 1185), this result is quite encouraging because in general classification result improves with number of instances.

Consequently, our approach of sequence classification then applying a CRF significantly improved the final annotation accuracy.