#41 · Primary category: Business Intelligence & Analytics

sail

apache-iceberg apache-spark arrow artificial-intelligence big-data data-engineering datafusion delta-lake distributed-computing iceberg lakehouse machine-learning parquet pyspark python rust spark spark-connect spark-sql sql

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Project last updated:08/29/26

GitHub Stars

3.3K

Forks

211

Contributors

45

License

Apache-2.0

Why we included this project

Teams stuck paying the JVM overhead of a big Spark cluster have a simpler way out now. Sail is a Rust compute engine that implements the Spark Connect protocol, so PySpark code and DataFrame queries run against it without rewrites. SQL analytics, batch ETL, and streaming all run on the same runtime, which also handles compute-heavy AI jobs. For teams that keep lakehouse tables in Delta Lake or Iceberg, that means holding onto the familiar API while cutting infrastructure cost and startup latency. It is worth piloting on a real workload before committing to it.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category