nlp_xiaojiang

Why we included this project

This Chinese-language NLP toolkit packs a lot of common text-processing jobs into one codebase, so you can try out several techniques without wiring them together yourself. It handles sentence similarity with BERT and XLNet embeddings, text classification through several Keras-based models, named entity recognition with a BERT + BiLSTM + CRF pipeline, and data augmentation approaches like back-translation, EDA, and seq2seq generation. A retrieval-style chatbot matches questions by sentence vectors instead of literal strings, and there are utilities for text distances and extracting sentence main parts. If you work on Chinese text classification, similarity matching, or want to enrich training data, the ready-to-run examples are a good starting point. Just be aware that some pretrained models and word vectors need to be downloaded separately, and the project targets Python 3.6.