llm-server-docs

Why we included this project

Anyone who wants to run their own LLM server on hardware they control will find this guide unusually thorough. It walks through the whole job on Debian, from GPU drivers and Docker hardening to boot-time scripts, then shows how to assemble the pieces: Ollama, llama.cpp, or vLLM for inference, llama-swap to manage models, Open WebUI as the chat interface, SearXNG for web search, and MCP proxies for tool integration. Image generation with ComfyUI and text-to-speech with Kokoro are covered too, along with secure remote access over Tailscale, including SSH and firewall setup. This is documentation, not software you clone and run, so expect to follow it as a reference and adapt the steps to your own box. Because the stack leans on standard interfaces like OpenAI-compatible endpoints and MCP, you can swap components without redoing the whole setup.