mlx-lm

Why we included this project

mlx-lm is a Python package that turns an Apple silicon Mac into a practical LLM workstation. It pulls models directly from the Hugging Face Hub, so one command can fetch and run thousands of public models, often in 4-bit quantized form that fits in typical memory. Beyond generation, you get a chat REPL, streaming output, prompt caching for reusing long contexts, and rotating key-value caches to keep RAM use predictable. Fine-tuning is also on the table: low-rank or full training on the same hardware, with quantization and re-upload handled through a small CLI or Python API. For teams that want to keep data on-device and skip cloud GPU costs, this covers the whole workflow from download to adapted model.