CAG

Why we included this project

Teams building knowledge-grounded LLM apps usually hit the same three walls with classic RAG: retrieval adds latency, the retriever sometimes picks the wrong document, and the extra pipeline means more moving parts to keep running. CAG sidesteps all of that. It preloads the relevant material into the model's context window, caches the KV parameters, and answers questions directly at inference with no retrieval step in between. The repo includes working Python scripts for both the CAG path and a conventional RAG baseline, benchmarked on SQuAD and HotPotQA, with swappable models and similarity metrics. That makes it a solid reference implementation if you want to test whether a retrieval-free setup keeps your accuracy. It's research code rather than a turnkey service, and the README is upfront about the real constraint: the whole knowledge source has to fit in the context window, which is worth weighing before you commit to the approach.