Meta-Harness

!Meta-Harness

Meta-Harness is a framework for automated search over task-specific model harnesses: the code around a fixed base model that decides what to store, retrieve, and show while the model works. This repo contains the framework and two reference experiments from the paper.

The paper is Meta-Harness: End-to-End Optimization of Model Harnesses.

**If you end up building something cool with Meta-Harness, please let us know!** We would be happy to showcase it here in the main README and link to your repository, artifact, blog post, paper, or whatever else is most useful.

Contents

• The reusable framework and onboarding flow for applying Meta-Harness to a new domain.
• Two paper reference experiments under `reference_examples/`:
• `reference_examples/text_classification/`: memory-system search for text classification.
• `reference_examples/terminal_bench_2/`: scaffold evolution for Terminal-Bench 2.0.
• The optimized Terminal-Bench 2 harness from the paper lives in the separate artifact repo: stanford-iris-lab/meta-harness-tbench2-artifact.

Quick Start

Text classification:

cd reference_examples/text_classification

uv sync

uv run python meta_harness.py –iterations 1

Terminal-Bench 2 smoke task:

cd reference_examples/terminal_bench_2

uv sync

uv run bash scripts/run_eval.sh agents.baseline_kira:AgentHarness full 1 1 -i extract-elf

Use the subdir READMEs for setup details, expected runtime, and additional commands.

Applying Meta-Harness To A New Domain

Start by pointing your coding assistant to `ONBOARDING.md` and having a conversation with it.

This should produce a `domain_spec.md` file with concrete details on how to proceed with implementing Meta-Harness for yo