Chinese-BERT-wwm

Why we included this project

Chinese-language NLP teams have relied on these pre-trained BERT-family weights for years, and for good reason. The project provides a range of whole-word-masking models, from BERT-wwm and its extended version to RoBERTa-wwm-ext variants including a large checkpoint, plus the compact RBT3 and RBTL3, so you can match model size to your latency and accuracy targets. All checkpoints load directly through the transformers library, which makes fine-tuning for downstream tasks such as classification and NER straightforward. The smaller models also work well for tokenization and embeddings, or as teacher models. Since the work comes with a peer-reviewed paper and documented pre-training details, you get a dependable baseline instead of a black box.