beir

Why we included this project

Choosing a retrieval model usually means guessing how well it will hold up on data you did not train on. BEIR removes some of that guesswork by bundling more than 15 preprocessed datasets into a single evaluation harness, so dense, sparse, and reranking models all go through the same pipeline and come back with standard metrics like NDCG@k, MAP, and recall without per-dataset glue code. Zero-shot scores on BEIR have become a common shorthand when comparing embedding and dense retriever models, so a strong run here tends to carry real weight in model selection. The library also keeps pace as the stack changes: it accommodates LLM-based embeddings with LoRA and vLLM inference, which matters once you move past classic sentence-transformers setups.