BitNet

Why we included this project

BitNet is Microsoft's official inference framework for 1-bit LLMs, built around CPU kernels that let models like the 2.4B-parameter BitNet-b1.58 run several times faster than their full-precision counterparts while using far less energy. It ships ready-to-use kernels for x86 and ARM, plus a GPU path, and it has grown beyond chat models to include 1-bit embedding models for retrieval-style workloads. The concrete speedup and power figures are useful for teams on edge devices or laptops, or anyone watching a cloud bill, who want to know whether 1-bit quantization fits their hardware. It is a well-documented way to serve or evaluate these models locally instead of through a hosted API.