langkit

Why we included this project

Teams that ship LLM features in production rarely see what those models actually produce once real users start sending prompts. LangKit closes that gap by turning raw prompt and response text into concrete, measurable signals like readability and complexity scores, relevance to the prompt, sentiment and toxicity levels, and checks for prompt injection, jailbreak attempts, hallucinations, and refusals. It plugs into whylogs, so the same profiling pipeline that logs your structured data can capture these text metrics without a separate monitoring stack. That makes it useful for anyone who needs guardrails and a baseline read on model behavior before rolling out a feature more broadly. You get numeric output you can track, alert on, and compare over time instead of eyeballing individual chat logs.