#123 · Primary category: LLM Application Frameworks

lotus

agentic-data-processing ai-data-processing data llm llm-data-processing llm-document-processing pandas python semantic-operators semantic-search unstructured-data

Optimized Agentic and LLM Bulk Processing Over Your Data

Project last updated:07/03/26

GitHub Stars

1.7K

Forks

152

Contributors

14

License

Apache-2.0

Why we included this project

LOTUS is a good fit when your data pipeline needs LLM reasoning applied across thousands of records rather than one prompt at a time. It packages LLM calls as familiar dataframe-style primitives like map, filter, and reduce, so you describe what you want in natural language and let its optimizer decide how to batch and schedule the model calls. Teams doing codebase sweeps, deep research over a corpus, document extraction, or agent-trace failure analysis will find the agentic mode especially useful, since it can spawn sandboxed tool-using agents per shard and fold their findings into a single answer. It sits naturally alongside pandas-style workflows and keeps model-API costs in mind, reporting higher accuracy at lower cost by batching and optimizing calls. The project comes out of Stanford and UC Berkeley and stays actively maintained, which makes it a credible pick for Python teams building LLM-powered data processing.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category