Llama-2-Open-Source-LLM-CPU-Inference

Why we included this project

GPU costs and data-residency rules are the usual reasons teams end up wanting their own document Q&A, and this project shows a workable version running on CPU. It takes a quantized Llama 2 chat model through C Transformers and GGML, embeds documents with Sentence-Transformers into a FAISS vector store, and ties the pieces together with LangChain so you can ask natural-language questions over your own files. The code is small and split into focused files, which makes it a solid reference for teams checking whether CPU-only inference is realistic for private retrieval. The step-by-step guide linked in the README explains each piece along the way, so it works as a learning resource for anyone still getting comfortable with how retrieval and generation fit together.