text-embeddings-inference

Why we included this project

Deploying embedding models for retrieval or RAG workloads usually means wiring a transformer library into a web server and hoping it holds up under real traffic. Text Embeddings Inference (TEI) removes that step: it serves open models like FlagEmbedding, GTE, E5, and ModernBERT with token-based dynamic batching, so requests are processed efficiently without a hand-built serving layer. The project skips the model graph compilation step, keeps Docker images small, and boots quickly, which makes it workable for serverless setups and for local development on a Mac via Metal. It also covers rerankers and sequence classification models, and it exposes Prometheus metrics and OpenTelemetry tracing, so you can drop it into your monitoring stack before pointing production traffic at it.