#16 · Primary category： Prompt Engineering Tools
LLMLingua
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
Project last updated：04/08/26
GitHub Stars
6.6K
Forks
419
Contributors
17
License
MIT
Why we included this project
LLMLingua attacks a cost problem most LLM apps eventually hit: long documents and large retrieved contexts that make every API call slower and pricier. It compresses the prompt before the model sees it, cutting redundant tokens while trying to keep the facts that matter, with compression ratios reported up to 20x at minimal quality loss. You don't run it as a separate service; it drops into an existing pipeline through ready-made integrations for LangChain, LlamaIndex, and Prompt flow, and it can also compress the KV cache during inference. The project comes out of Microsoft Research with papers at EMNLP 2023 and ACL 2024, a good sign if you want to understand how the compression works before trusting it. For teams paying per token or fighting context-window limits, it's worth testing against your own workloads before reworking anything else in your stack.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
awesome-nano-banana-pro-prompts
🍌 World's largest Nano Banana Pro prompt library — 10,000+ curated prompts with preview images, 16 languages. Google Gemini AI image generation. Free & open source.
prompt-master
A Claude skill that writes the accurate prompts for any AI tool. Zero tokens or credits wasted. Full context and memory retention
chatgpt_system_prompt
A collection of GPT system prompts and various prompt injection/leaking knowledge.
awesome-nanobanana-pro
🚀 An awesome list of curated Nano Banana pro prompts and examples. Your go-to resource for mastering prompt engineering and exploring the creative potential of the Nano banana pro(Nano banana 2) AI image model.
awesome-gpt-image-2
🚀 World's largest GPT Image 2 prompt library, updated daily — 2000+ curated prompts with preview images, 16 languages. OpenAI's next-gen image model with pixel-perfect text rendering, cross-image consistency, and commercial-grade illustration. Free & open source.