DataProfiler

Why we included this project

DataProfiler answers a question every data team hits early: before you can trust a dataset, you need to know its shape and what sensitive fields it carries. Point it at a file and it auto-detects the format, loads CSV, Parquet, JSON, or AVRO into a pandas DataFrame, and returns a profile covering the schema, per-column statistics, and recognized entities. The real differentiator is the pre-trained deep learning labeler, which flags names, email addresses, phone numbers, and other PII or NPI in structured and unstructured text alike. You can also retrain the recognizer or drop in a custom entity pipeline, so you are not stuck with the default labels. For data engineering, governance, and pipeline monitoring, that combination of quick profiling and sensitive-data detection saves a lot of guesswork.