OpenArc

Why we included this project

For teams running open-source models on Intel hardware who care about keeping data off cloud APIs, OpenArc is a practical single service. It exposes OpenAI-compatible endpoints for language models, vision models, speech recognition, text-to-speech, embeddings, and reranking, all accelerated through OpenVINO on CPU, GPU, and NPU. Instead of wiring separate servers together, you point existing OpenAI clients at one async service that can load multiple models concurrently, and it also handles speculative decoding, multi-GPU parallelism, and CPU offload. The project is under active development, but the feature list is concrete and the included benchmarks are useful for comparing latency and throughput across device setups.