guwenbert

Why we included this project

Classical Chinese is a hard target for modern NLP models, since almost all pre-trained BERTs are built for contemporary language. GuwenBERT is a RoBERTa-style model trained on about 1.7 billion characters from more than 15,000 ancient books, with a vocabulary drawn from the high-frequency characters of that corpus. It loads through Hugging Face Transformers, so you can drop it into a fine-tuning pipeline without much setup. The authors report a 6.3% gain over the most popular modern Chinese RoBERTa on Classical Chinese named-entity recognition, and say the model reaches comparable results in about 300 training steps, which is a real advantage when your annotated corpus is small. For sentence segmentation, punctuation restoration, and entity labeling on historical documents, it saves you from adapting a modern-language model to a very different register.