gpt2-ml

Why we included this project

Chinese text generation is a corner where large open pretrained models have always been scarce, and gpt2-ml stands out because it actually published a 1.5B-parameter GPT-2 checkpoint trained on Chinese. The weights come in two versions, one using a CLUE vocabulary and one using a BERT-style 21k-token vocabulary, so you can match the tokenizer your downstream work already expects. Training scripts based on Grover support TPUs for anyone who wants to continue pretraining on their own data, and the Colab notebook gets the 1.5B model generating text in a couple of clicks, which makes it easy to judge output quality before building anything around it. Just keep in mind the project has been quiet since 2023 and targets the TensorFlow-era toolchain, so it suits people who are comfortable adapting older code rather than teams looking for a maintained dependency.