bitsandbytes

Why we included this project

Training or serving a large model on one GPU often comes down to how little memory you can compress the weights into, and bitsandbytes provides the quantized math that makes that practical for PyTorch. It offers 8-bit optimizers, LLM.int8() inference, and 4-bit QLoRA-style training, so models run in a fraction of the memory full precision would need. Rather than a turnkey app, it ships low-level primitives such as Linear8bitLt and Linear4bit plus the optim module, which is why it sits underneath tools like QLoRA. If you already work inside the PyTorch ecosystem and want a tested foundation for quantized training or inference instead of another abstraction layer, this is a solid choice.