#150 · Primary category： Inference & Local Deploy
LlamaEdge
The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge
Project last updated：02/08/26
GitHub Stars
1.7K
Forks
149
Contributors
11
License
Apache-2.0
Why we included this project
Serving open-source LLMs from your own hardware usually means wrestling with a Python runtime and its dependency tree. LlamaEdge skips that: the inference engine and an OpenAI-compatible API server ship as a single WasmEdge binary, so a chat or embedding service you build with something like LangChain gets a local endpoint without the usual setup cost. The whole runtime stays around 30MB and pulls in no Python packages, which matters on edge devices and in small containers. Because your own Rust or JavaScript code can be compiled to portable Wasm and run on CPUs or GPUs, a prototype that works on a laptop can move to a server unchanged. For teams that want a lightweight local alternative to hosted or PyTorch-based serving, that is the appeal.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.