Chinese-Mixtral-8x7B

Why we included this project

Chinese-Mixtral-8x7B takes Mistral's Mixtral-8x7B mixture-of-experts model and reworks it for Chinese. The team extended the tokenizer's vocabulary and ran incremental pretraining on large-scale Chinese corpora, which lets the model encode and decode Chinese with noticeably fewer tokens per sentence than the original. That efficiency is the main draw for anyone building Chinese-language applications on a MoE backbone. The repo releases the merged weights for direct use, plus the LoRA adapter and the full vocabulary-expansion and QLoRA pretraining pipeline, so it also works as a reference for reproducing similar language-adaptation work. It's a base model, not an instruction-tuned one, so plan to fine-tune it for your task; the project points to its sibling instruction-tuned release for that.