#105 · Primary category： Inference & Local Deploy
lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Project last updated：05/28/26
GitHub Stars
3.8K
Forks
326
Contributors
67
License
Apache-2.0
Why we included this project
Most teams that fine-tune models end up paying for a separate server per variant, and that adds up fast. LoRAX sidesteps the cost: you run one base model and it loads LoRA adapter weights on demand, per request, from Hugging Face or your own storage, so thousands of tuned variants can share a single GPU. It keeps throughput up by batching requests for different adapters together and moving adapters between GPU and CPU memory as it needs them. Quantized base models and token streaming are supported, and the OpenAI-compatible API makes it easy to fit into an existing serving stack. If your job is serving a shared base model with many tuned variants, this is a practical way to cut the bill without giving up speed.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.