Jiayan

Why we included this project

Most Chinese NLP libraries are trained on modern Mandarin and handle Classical Chinese poorly, which leaves people working with 文言文 without much tooling. Jiayan targets that gap directly. It is a Python toolkit for Classical Chinese that can build a lexicon from a corpus, tokenize, tag parts of speech, and add sentence segmentation and punctuation. The lexicon builder works unsupervised, using PMI and neighboring entropy to pull vocabulary out of raw text, and the tagging and segmentation models are CRF- or HMM-based, so they run fine on a laptop without a GPU. That makes it a reasonable fit for digital humanities projects and corpus annotation. One caveat: it targets simplified characters, so traditional-script input needs converting first。