sie

Why we included this project

Most self-hosted inference servers are built around one large model per instance, which gets awkward when an agent actually needs several smaller models for different steps. SIE flips that: a single cluster pulls whatever model a task needs out of a catalog of over 100 options, loading it on demand and using LRU eviction to keep the ones you are actively using warm on the same hardware. Its OpenAI-compatible endpoints for embeddings and chat completions cover the standard agent calls, while the native encode, score, and extract primitives handle retrieval jobs directly, so it slots into LangChain, LlamaIndex, Haystack, and DSPy pipelines without adding a new abstraction layer. The same image runs on a laptop and in Kubernetes, so a small team can validate on one node and then move to a load-balanced, autoscaled setup without rewriting the serving layer. If you are tired of standing up a separate model server per task and paying per-token API costs on top, this is one central endpoint worth evaluating.