openlake

Why we included this project

Most LLM deployments hit their ceiling on the I/O path long before the GPUs run out of compute, and that is the bottleneck OpenLake is built around. It is a distributed storage engine written in Rust that exposes an S3-compatible object store handling the small, high-frequency reads inference and training generate, so accelerator nodes are not left waiting on data. The standout piece is KV cache offload: it can keep petabyte-scale KV state across GPU hosts and let vLLM workers pull prefixes back in milliseconds, cutting prefill work for repeated and long prompts. Vector index building and serving, checkpoint storage, and large context or agent memory are also covered. Platform and infrastructure teams dealing with GPU idle time or storage-driven scaling limits will find it slots in as a storage layer for existing stacks.