#78 · Primary category: Foundation Models

Baichuan-7B

artificial-intelligence ceval chatgpt chinese gpt-4 huggingface large-language-models llama mmlu natural-language-processing

A large-scale 7B pretraining language model developed by BaiChuan-Inc.

Project last updated:07/18/24

GitHub Stars

5.6K

Forks

501

Contributors

11

License

Apache-2.0

Why we included this project

Baichuan-7B is a 7B parameter pretraining model from Baichuan Inc., released under Apache 2.0, so commercial use is genuinely on the table. It was trained on roughly 1.2 trillion tokens with a 4,096 token context window, and the repo ships the tokenizer, evaluation scripts, and enough detail on data filtering and mixing to reproduce the published C-Eval, MMLU, Gaokao, and AGIEval scores. A Chinese-optimized tokenizer and the familiar LLaMA-style Transformer make it a reasonable base for teams working with Chinese and English text. Just remember this is a base model, not an instruction-tuned assistant, so most end-user products will need fine-tuning or chat-style adaptation on top. Hugging Face integration is straightforward and the reproduction commands are published, which keeps it practical for research and downstream experiments.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category