api-for-open-llm

Why we included this project

Self-hosting an open LLM usually means wiring it into your stack yourself, but this project gives you a ready-made OpenAI-compatible endpoint instead. It serves LLaMA, Qwen, ChatGLM, Baichuan, Yi and other checkpoints through the familiar chat-completions format, so apps and LangChain tools can switch from a commercial API to a local model by changing environment variables rather than rewriting code. It also covers the practical bits: streaming responses, text embeddings for retrieval, loading your own LoRA-tuned weights, and optional vLLM acceleration when you need higher concurrency. If you want to test open models as a cheaper or more controllable backend, this is a fast way to get them behind a standard interface.