quokka

Why we included this project

Teams stuck with large, timestamped datasets know the usual choice: either move everything into a warehouse, or watch the analytics engine choke on windowed, stateful queries. Quokka takes a different route. It's a Python-native, push-based distributed query engine built on Ray, DuckDB, Polars, and Arrow that reads Parquet files and Iceberg tables directly from the data lake, so you can skip the ETL step into a separate store. It's happiest on time-series work most SQL engines handle poorly, like asof and range joins, pattern recognition, and stateful computation such as building a limit order book. The project ships concrete evidence too: a tick-level backtest that replays four years of SIP trade data in about ten minutes, and TPC-H results where it runs several times faster than SparkSQL on many queries. And since it's Python-native, hooking in a custom data source or operator is fairly straightforward when you need bespoke analytics.