#176 · Primary category： Inference & Local Deploy
llama2-webui
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Project last updated：03/22/24
GitHub Stars
1.9K
Forks
199
Contributors
10
License
MIT
Why we included this project
Llama2-webui bundles a Gradio chat interface and an OpenAI-compatible API around a local Llama 2 backend, so you get a working endpoint without assembling model loading, prompt handling, and a front end yourself. It runs on Linux, Windows, and macOS and supports the main Llama 2 variants, 7B through 70B, in Hugging Face, GPTQ, GGML, and GGUF formats. Backend choice (transformers, bitsandbytes 8-bit, AutoGPTQ 4-bit, or llama.cpp) is set through an env file, which makes it easy to move a model between a beefy GPU and a CPU-only laptop. There's also a Code Llama completion UI for people who want code filling rather than chat. The pip package, llama2-wrapper, lets an app or agent talk to the same local model, and because the API mirrors OpenAI's, swapping in a hosted model later takes minimal changes.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.