Model-Optimizer

Why we included this project

Serving LLMs or diffusion models on NVIDIA hardware usually comes down to how much of the model fits in memory while keeping latency acceptable; that is the difference between a deployable service and an unaffordable one. Model Optimizer attacks that problem with one set of Python APIs covering the main compression levers, including post-training and quantization-aware training to FP8 or NVFP4, structural pruning, distillation, sparsity, and speculative decoding, and it ends with an exported checkpoint you can deploy as is. The export step is where it pays off, because the optimized model drops straight into TensorRT-LLM, TensorRT, vLLM, and SGLang, so you optimize once and serve wherever you already run. Teams that want to shrink a large model before serving can use the Minitron-style prune-and-distill recipes, which have powered real deployments. For production inference on NVIDIA GPUs, it is the practical route from a model too big to serve to a fast, memory-lean one.