#103 · Primary category: MLOps & Evaluation
future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
Project last updated:08/29/26
GitHub Stars
1.9K
Forks
561
Contributors
64
License
Apache-2.0
Why we included this project
Most teams building LLM agents end up stitching together separate tools for tracing, evals, and prompt tuning. future-agi folds those jobs into one self-hosted platform built around a single feedback loop: run simulations against synthetic users before launch, score outputs with built-in or custom metrics, trace live traffic to catch failures, then feed those observations back into optimization. Because every feature reads from the same traces, datasets, and scores, something caught in production becomes evaluation data and then a target for improvement, so the work compounds instead of scattering across tools. It starts from a single line of code, works with the stacks you already run, and supports a broad range of providers, which keeps it practical for engineers and product managers who want one source of truth. If keeping agents trustworthy is your real bottleneck, this is the kind of platform worth trying before you invest in more disjointed tooling.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
unsloth
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, Kimi K3, MiniMax-H3, Gemma 4, FLUX and more.
LlamaFactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
langfuse
🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
netron
Visualizer for neural network, deep learning and machine learning models