#468 · Primary category: AI Coding Assistants

agentic-harness-engineering

Observability-driven framework for automatic evolution of coding-agent harnesses, reaching 84.7% pass@1 on Terminal-Bench 2 and boosting GPT-5.4 from 69.7% to 77.0%.

Project last updated:08/03/26

GitHub Stars

863

Forks

99

Contributors

3

License

MIT

Why we included this project

Iterative improvement of a coding agent usually means retraining the model. This project takes a different approach: it keeps the base model fixed and automatically improves the harness around it, covering system prompts, tool descriptions and implementations, middleware, skills, sub-agents, and long-term memory. The mechanism is observability. The harness is broken into git-tracked components, raw execution traces are distilled into layered reports, and those reports drive each change, making every edit evidence-based, auditable, and reversible. In testing, the approach lifted GPT-5.4 from 69.7% to 77.0% pass@1 over ten iterations and reached 84.7% on Terminal-Bench 2 with GPT-5.5, and the harness transferred to SWE-bench-Verified, a sign it encodes reusable engineering practice rather than benchmark memorization.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category