langtest

Why we included this project

LangTest is a testing framework for NLP and LLM projects that catches problems before they reach users. You can generate and run dozens of test types with a single line of code, covering robustness, bias, representation, fairness, and accuracy across tasks like NER, translation, text classification, question answering, and summarization. It works with classic Hugging Face and Spark NLP pipelines, and also tests hosted models from OpenAI, Cohere, and Azure OpenAI for issues like toxicity, factuality, sycophancy, and clinical or legal support. The workflow is simple: create a Harness, generate test cases, run them, and read the report. That makes it easy to wire evaluation into your normal development loop instead of treating it as a one-off audit. If you're responsible for model quality and want a concrete way to check for bias or failure modes before release, this gives you a structured starting point.