nlp_chinese_corpus

Why we included this project

Getting Chinese text in a form you can actually train on has long been the bottleneck, and this repo gathers several substantial datasets into one download point. Inside are a structured Wikipedia dump with over a million entries, about 2.5 million news articles with titles and keywords, question-answer collections from Baike and community sources, and millions of Chinese-English sentence pairs, all packaged as JSON. That covers the common ground for pretraining a language model, building word vectors, or training QA and classification systems without spending weeks scraping and cleaning. The author started it after hunting for a large Chinese corpus and finding only small, outdated, or awkwardly formatted options, so the appeal is as much convenience as volume.