GPTCache

Why we included this project

GPTCache cuts repeated LLM API calls by caching answers semantically rather than by exact text match, so a follow-up question phrased differently can reuse an earlier response instead of hitting the model again. That makes it a practical fit for chatbots, support tools, or retrieval pipelines that send many similar prompts to OpenAI and comparable services. Setup means choosing an embedding function, picking a store such as SQLite, FAISS, Redis, or Milvus, and setting a similarity threshold; the library then handles exact and near-duplicate queries automatically and plugs into LangChain and llama_index through adapters. A bundled server image lets non-Python services share the same cache. One caveat: the maintainers no longer add adapters for each new model API and point users at the generic get/set interface instead.