speculators

Why we included this project

Speculative decoding is a neat way to cut LLM serving latency: a small draft model proposes tokens ahead of time and the larger base model verifies them in a single pass, so responses get faster without the output changing. This library turns that idea into something you can actually run in production. It provides an end-to-end pipeline for training the draft models, packages them in a Hugging Face-compatible format, and they drop straight into vLLM with minimal wiring. It supports several modern drafting algorithms, including EAGLE-3, DFlash, DSpark, and P-EAGLE, and covers MoE, dense, and vision-language verifiers, so it fits a decent range of model families. For inference engineers who would rather not hand-roll draft-model training, this is a maintained, standardized path to faster serving.