Jlama

Why we included this project

Java teams that want LLM inference without leaving the JVM can pull Jlama in as a Maven dependency and get a full inference engine written in pure Java. It loads Hugging Face safetensors models, covering the Llama, Mistral, Gemma, Qwen, and BERT families, and its Q4/Q8 quantization keeps models usable on modest hardware. Besides plain text generation it implements paged attention, mixture-of-experts routing, tool calling, and embedding and classifier inference, so it is more than a demo. A CLI and an OpenAI-compatible REST API let you prototype locally and then embed the same engine in production, and the optional distributed inference mode helps when a single node is not enough. Teams already using LangChain4j get a ready integration.