nlg-eval

Why we included this project

For any project that generates text, whether it is a machine translation system, a dialogue agent, or a summarizer, there comes a point where you have to measure the output against reference sentences. nlg-eval bundles the standard metrics for that job into one tool: BLEU, METEOR, ROUGE, CIDEr, and SPICE all run from the same hypothesis and reference files, and it adds embedding-based similarities like SkipThought cosine and Greedy Matching. That means you can report a comparable set of scores across experiments without maintaining several incompatible packages. The setup is not free, it downloads pretrained embeddings and requires Java, but the tradeoff is that every metric uses the same preprocessing, so the numbers stay consistent across runs and papers. Researchers evaluating NLG systems and engineers who want a quick scoreboard during development should find both the command line and the Python API easy to fit into an existing pipeline.