FastDeploy

Why we included this project

FastDeploy comes from the PaddlePaddle team itself, so it reads less like a wrapper around someone else's engine and more like the official route for serving Paddle-trained models in production. It has grown beyond a simple toolkit into a full serving solution, covering load-balanced prefill and decode disaggregation, context caching, speculative decoding, quantization options, and several hardware backends. The supported model list leans toward Paddle-adjacent families such as ERNIE, DeepSeek, Qwen3, and PaddleOCR-VL, and the OpenAI-compatible API means you can often slot it in behind existing clients without much rework. Teams already building on PaddlePaddle, or targeting the architectures it supports, will get the most out of it, but the production-focused features make it worth weighing against general-purpose serving tools.