duckdb

Why we included this project

DuckDB runs inside your own process rather than as a separate database server, which is what makes it so easy to drop into existing data work. You can point it at CSV, Parquet, or JSON files, or at pandas DataFrames, and run columnar SQL over them directly from Python, R, or Java. That makes it a practical tool for the data-prep side of AI work, like cleaning and chunking documents for RAG pipelines or shaping training data. Analysts also use it to answer ad-hoc questions with plain SQL instead of spinning up infrastructure. For teams that would otherwise loop through rows in Python, the speed difference is usually the first thing they notice.