#468 · Primary category: AI Coding Assistants
agentic-harness-engineering
Observability-driven framework for automatic evolution of coding-agent harnesses, reaching 84.7% pass@1 on Terminal-Bench 2 and boosting GPT-5.4 from 69.7% to 77.0%.
Project last updated:08/03/26
GitHub Stars
863
Forks
99
Contributors
3
License
MIT
Why we included this project
Iterative improvement of a coding agent usually means retraining the model. This project takes a different approach: it keeps the base model fixed and automatically improves the harness around it, covering system prompts, tool descriptions and implementations, middleware, skills, sub-agents, and long-term memory. The mechanism is observability. The harness is broken into git-tracked components, raw execution traces are distilled into layered reports, and those reports drive each change, making every edit evidence-based, auditable, and reversible. In testing, the approach lifted GPT-5.4 from 69.7% to 77.0% pass@1 over ten iterations and reached 84.7% on Terminal-Bench 2 with GPT-5.5, and the harness transferred to SWE-bench-Verified, a sign it encodes reusable engineering practice rather than benchmark memorization.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
superpowers
An agentic skills framework & software development methodology that works.
ECC
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
skills
Skills for Real Engineers. Straight from my .agents directory.
opencode
The open source coding agent.
andrej-karpathy-skills
A single CLAUDE.md file to improve Claude Code behavior, derived from Andrej Karpathy's observations on LLM coding pitfalls.