ChatLM-mini-Chinese

Why we included this project

ChatLM-mini-Chinese starts from a simple premise: most large language models are impractical on consumer hardware, so it sets out to build a useful Chinese dialogue model from scratch on a modest setup. The result is a 0.2B model that loads in around 512MB under float16 and can be pretrained on a single 4GB GPU. Every step of the pipeline is open: the data sources, the cleaning and mini-hash deduplication code, tokenizer training, pretraining, SFT, and DPO preference optimization, with custom trainers that support single- and multi-GPU runs and resume from any checkpoint. A worked example shows fine-tuning for relation-triple extraction while keeping the chat ability intact, so the repo works both as a deployable small model and as a study reference for how a Chinese LLM is built end to end.