Vision-Agents

Why we included this project

Building a conversational agent that watches a live video feed and talks back usually means gluing together WebRTC streams, speech models, and vision detection yourself. Vision Agents packages those pieces into a single Python framework, so a few dozen lines can give you a working prototype that pairs a fast detector like YOLO with a realtime LLM. It keeps the native calling conventions of each major model provider, so you can use current models from OpenAI, Gemini, or Claude without a proprietary wrapper. The framework includes pluggable vision pipelines, turn detection, phone and tool-calling support, and runs on Stream's edge network or other WebRTC providers, which makes it a reasonable base for live coaching, monitoring, and other low-latency use cases.