Class Incremental Learning for Character String Recognition

Character string recognition (CSR) has drawn much attention for document intelligence, but its performance is limited by the pre-defined character set without the ability to recognize new characters. To overcome this issue, class incremental learning (CIL) can be adopted where the model learns from new data instances incrementally over time. However, it is challenging to directly apply existing CIL methods in CSR because CSR is a typical sequence recognition problem. Without accurate alignment, the recognition error of new characters will affect the recognition of other characters in the same sequence. Moreover, the new characters are usually much fewer than the old ones, resulting in a data imbalance issue for learning new classes. To tackle the misalignment issue, we decouple the learning of feature alignment and classifiers during the incremental process in CSR. To handle the data imbalance issue, we propose a Prototype Incremental Learning framework for CSR, namely PIL-CSR. In the PIL-CSR framework, we propose a prototype-centered loss (PCL) to aid the model in facilitating better class separation, and we further propose a prototype separation and feature alignment (PSFA) strategy, allowing the model to adapt and learn new classes seamlessly. Finally, we collect a CSR dataset to evaluate CIL performance (github.com/tambourine666/Doc-CIL). Experimental results demonstrate the effectiveness of our proposed sequence CIL method, obtaining a significant improvement in both line-level and character-level accuracy.

Duke Scholars

Author Kaizhu Huang DKU Faculty