#78 · Primary category: Foundation Models
Baichuan-7B
A large-scale 7B pretraining language model developed by BaiChuan-Inc.
Project last updated:07/18/24
GitHub Stars
5.6K
Forks
501
Contributors
11
License
Apache-2.0
Why we included this project
Baichuan-7B is a 7B parameter pretraining model from Baichuan Inc., released under Apache 2.0, so commercial use is genuinely on the table. It was trained on roughly 1.2 trillion tokens with a 4,096 token context window, and the repo ships the tokenizer, evaluation scripts, and enough detail on data filtering and mixing to reproduce the published C-Eval, MMLU, Gaokao, and AGIEval scores. A Chinese-optimized tokenizer and the familiar LLaMA-style Transformer make it a reasonable base for teams working with Chinese and English text. Just remember this is a base model, not an instruction-tuned assistant, so most end-user products will need fine-tuning or chat-style adaptation on top. Hugging Face integration is straightforward and the reproduction commands are published, which keeps it practical for research and downstream experiments.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities