AngelSlim

Why we included this project

AngelSlim is worth a look if your team has to squeeze large models into realistic GPU budgets before serving them. It bundles quantization, speculative decoding, and distillation into one framework rather than forcing you to wire together several separate tools, and it works across current model families like Qwen, DeepSeek, Hunyuan, and GLM. You pick a model, choose a method such as FP8 or INT4 quantization, and get runnable weights plus scripts that slot into serving stacks like vLLM or llama.cpp. The toolkit goes deep on low-bit compression down to 1.25-bit and includes the decoding-speed tricks that keep compressed models responsive, which matters when you are pushing frontier-scale checkpoints onto limited GPU memory. It is also a handy place to compare competing compression approaches before you commit one for production.