TrustLLM

Why we included this project

TrustLLM packages the evaluation methodology from the ICML 2024 paper of the same name into a pip-installable Python toolkit, so you can measure how a model behaves on dimensions that accuracy benchmarks never touch: truthfulness, safety, fairness, robustness, privacy, and machine ethics. It bundles more than 30 datasets and lets you run generation and scoring against your own model, whether it runs locally or behind an API, then compare the results with the published leaderboard. That makes it a practical fit for model selection, red-teaming, and compliance-style checks where you want reproducible, dimension-by-dimension numbers rather than a single accuracy figure. The documentation and bundled dataset download keep the setup light enough that a meaningful evaluation on your own stack is within reach.