llama-swap

Why we included this project

Keeping several local models on one machine usually means running separate servers and configs. llama-swap sits in front of any OpenAI- or Anthropic-compatible backend and loads the model you ask for, then unloads it to free VRAM when you're done. One binary and one config file replace the mix of llama.cpp, vLLM, tabbyAPI, or stable-diffusion.cpp instances you'd otherwise manage. If you rotate between models during the day, your clients keep hitting the same endpoint while the upstream server changes underneath. A small web UI and a handful of admin endpoints show what's loaded and let you force-unload models manually.