#155 · Primary category： Inference & Local Deploy
llama3.java
Llama 3+ inference in pure Java
Project last updated：04/24/26
GitHub Stars
816
Forks
94
Contributors
6
License
MIT
Why we included this project
Java teams that want to run Llama 3, 3.1, or 3.2 models usually end up wiring a native C++ runtime through JNI. This project skips that layer: the entire inference engine lives in a single Java file with no dependencies, reads GGUF weights directly, and handles the common quantizations from Q4_0 through Q8_0. A small CLI offers chat and instruct modes, so you can point it at a model file and start generating. The author also uses the code to test JVM compiler optimizations, which shows up in the Vector API matrix routines and GraalVM Native Image support for quick startup. It descends from Karpathy's llama2.c, so it doubles as a readable introduction to how transformer inference works, all without leaving the JVM.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
ollama
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
llama.cpp
LLM inference in C/C++
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
LocalAI
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.