whylogs

Why we included this project

Teams shipping machine learning models rarely have a lightweight way to answer the most basic reliability question: is the data coming in today still shaped like the data the model was trained on? whylogs answers that by producing compact statistical summaries, called profiles, of any dataset you pass it, capturing distributions, missing values, and custom metrics without storing the raw data. Those profiles merge together, so you can aggregate them across distributed or streaming workloads and compare distributions over any time window, which makes feature drift, training-serving skew, and concept drift detection a routine check rather than a bespoke project. You can also define data constraints to fail fast on malformed inputs, and the same profiles work for exploratory analysis on very large datasets. For data scientists, ML engineers, and data engineers who want observability in a pipeline without adopting a heavy monitoring platform, this is a sensible starting point, and the Python and Java APIs keep the learning curve low.