model2vec

Why we included this project

Model2Vec distills an existing sentence transformer into a compact static embedding model that runs in pure Python without a GPU. Anyone who has served embeddings at scale knows how quickly transformer encoders eat into latency and memory budgets; this library shrinks an encoder by up to 50x and makes it up to 500x faster, at the cost of a modest accuracy dip. You can pull prebuilt distilled models from the HuggingFace hub in one line, or distill your own transformer with the provided API. It also includes a trainable classifier wrapper and supports both sequence-level and token-level embeddings, which covers semantic search, RAG pipelines, and text classification without a separate setup for each. For teams running text embeddings on commodity hardware, that is a practical shortcut that stays competitive with much larger models.