kedro

Why we included this project

Moving a data-science notebook into a scheduled production pipeline is where many ML projects stall, and Kedro is built to remove that step. It is a Python framework that imposes software-engineering structure on data work: a standard project layout, a data catalog with connectors for local and cloud filesystems, and automatic dependency resolution between pipeline nodes. Because each pipeline node is a modular, testable function, the same codebase can go from a Jupyter prototype to scheduled runs on Airflow, Kubeflow, Databricks, or SageMaker without a rewrite. Kedro-Viz adds a visual layer for data lineage, node status, and execution time, which helps when explaining the workflow to stakeholders or hunting down a failing step. For data scientists and ML engineers who are tired of fragile glue code, it is a practical foundation for reproducible, standardized work.