#98 · Primary category： NLP Tools & Text Processing
CLUECorpus2020
Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料
Project last updated：02/06/26
GitHub Stars
1.0K
Forks
83
Contributors
4
License
MIT
Why we included this project
Most teams that train Chinese models end up stitching together their own cleaned text from raw web crawls. CLUECorpus2020 removes that step: it filters the Chinese portion of Common Crawl down to roughly 100GB of text ready for pretraining, and ships a smaller 14GB variant, CLUECorpusSmall, split into news, community, wiki, and comment subsets. The maintainers also publish a compact simplified-Chinese vocabulary that runs smaller than the standard Google one, which matters if you want to trim tokenizer size. Reported BERT-base baselines give you a rough sense of what different data volumes and vocabulary choices actually change in downstream Chinese tasks. Just know the large corpus is distributed by email application rather than direct download, so plan for that gated access.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
flair
A very simple framework for state-of-the-art Natural Language Processing (NLP)
compromise
modest natural-language processing
tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
CoreNLP
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
Chinese-Word-Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量