Chinese-Mixtral

Why we included this project

Chinese-Mixtral takes the Mixtral-8x7B sparse mixture-of-experts model and continues its training on a large corpus of Chinese text, producing a base model plus an instruction-tuned chat variant that handle Chinese natively rather than leaning on a general-purpose checkpoint. The 32K native context window (tested up to 128K in practice) is a good fit for long-document work, and the released pretraining and fine-tuning scripts let you keep adapting the weights for your own domain. Quantized GGUF builds run through llama.cpp on as little as 16GB of memory, and the models drop into existing stacks via transformers, vLLM, or LangChain. A technical report on the adaptation approach accompanies the release, which helps if you are weighing whether continued pretraining.