The 36-Point Problem
In the rush to deploy AI agents, teams often obsess over the model while treating the “harness”—the code that wraps the model and manages the loop—as an afterthought. This is a mistake that shows up in the metrics. Data indicates that a harness can swing an agent’s performance score by as many as 36 points. This means the difference between a system that looks like a “frontier” intelligence and one that looks fundamentally broken is often the quality of the wrapper, not the weights.
As a Senior Architect, your first job is to know when not to build. Before you open a door, apply the short-circuit logic: If the workflow can be hardcoded, write a script. If the task requires only one or two turns, call the raw API. A harness is an investment reserved exclusively for open-ended, multi-step problems.
Once you’ve confirmed you actually need a harness, you face three specific doors: Adopt a commodity SDK, Extend an existing core, or Build your own custom loop. Moving past “vibes” requires moving toward architecture.
Architecture is Not a Personality Quiz: The Six-Axis Scorecard
Choosing a harness based on whether you “feel” like a builder is a failure of technical strategy. Instead, score your requirements against six specific axes to see which door naturally opens:
Control: How much visibility do you need into the internal mechanics of the loop?
Capability Ceiling: Do you need sub-agent orchestration or “frontier” features today that you can’t out-engineer a vendor’s dedicated team to produce?
Maintenance Tax: Who fixes the harness at 2 a.m.? Choosing “Build” means volunteering for a fourth on-call rotation.
Model Coupling: How painful is it to switch providers or avoid lock-in?
Governance: What are your requirements for audit trails and hard permissioning?
Team Shape: Does the harness need to fit in one head, or is it a shared liability across ten skulls?
The logic follows a flipped polarity. For a solo practitioner, the “Bus Factor” is already one; a custom harness adds no new single point of failure. For them, “Build” offers maximum control and speed. Conversely, an enterprise team prioritizing Governance and low Maintenance Tax will find that same custom code to be a toxic liability, making “Adopt” the only responsible path.
“A build-versus-adopt debate that hasn’t produced a scored table is a taste debate wearing an architecture costume.”
The “Resurrection” Feature: Why Your Harness Needs a Memory
We’ve seen agent scripts “golfed” down from 70 lines to 34, but a production-grade “middleweight” harness usually lands around 200 lines. That extra code isn’t bloat; it’s custody.
One of the highest-value engineering investments in an agentic loop is the “persist” verb. By appending every message to a session log in real-time, you enable a “resume” function. If a process is killed mid-task, you can restart with resume=True, and the agent resurrects exactly where it left off.
However, you must distinguish between the “summary” and the “durable truth.” Following the Anthropic long-running lesson, high-performing harnesses pair compaction with write_note offloading. In this architecture, a model-generated summary is treated as a “rumor”—prone to hallucination—while the PROGRESS.md file and session logs serve as the durable record. High-value engineering focuses on these “boring” features because they ensure the agent’s state remains transparent and recoverable.
Sequence is Safety: The Anatomy of a Loop
The “run method” is the week’s engineering in miniature. In our architecture, we utilize Bash as the primary action channel, and the sequence of the loop is the safety design, not decoration:
Compaction: This runs first so you never call the model with a bloated, expensive context window.
Call: The model is engaged only after the context is optimized.
Persist: The response is immediately saved to the session log.
Stop-check: The harness checks if the agent is finished.
Gate: If the agent wants to act, it hits a permission gate.
Execute: The action is performed only after policy clearance.
By placing the stop-check before the gate, we ensure a finished agent isn’t asked for redundant permissions. By placing the gate before execution, we ensure no tool—whether it’s a domain-specific registry or a raw Bash command—operates without oversight.
The Only Asset That Doesn’t Deprecate: The Measurement Rig
The most important component of your stack is the Measurement Rig (the “ruler”). Without it, you cannot prove your version earns its keep. The strategy is simple: Adopt the commodity organs; build only where your rig proves your version moves the numbers.
“The ruler... is what turns ‘I feel like we should build this’ into ‘our version moves our numbers, here’s the delta.’”
The middle door—Extend—is often the sweet spot. It allows you to stay current without forking the loop. By following the Portability Rule—”Translate at the boundary, keep your shapes plain”—your custom tools and safety hooks live in your files while the core lives in the vendor’s. This way, upstream improvements arrive without merges, and you wake up to a better system rather than a broken one.
Shipping Without the Drama: The Production Checklist
Production readiness requires answering a “colder” set of questions. Before shipping, satisfy this checklist:
Sandbox the Blast Radius: Run execution in a container or throwaway VM. The gate handles mistakes; the sandbox handles everything else.
Meter the Costs: Implement hard budget stops and track cost-per-run.
Posturing for Hostile Text: Assume hostile text in every tool output. Keep credentials out of the agent’s reach and never let untrusted content authorize actions.
Separate Tracing: Maintain a human-readable observability trail with timestamps and verdicts, distinct from the session log used for resumption.
Versioned Configurations: Pin the model, prompts, tools, and policy together as a single versioned unit.
In production, “boring” is what 2 a.m. respects.
Conclusion: The Five-Day Probation
To choose and validate a harness, put it through a five-day ritual:
Run Attended: Watch the agent perform three real backlog tasks; record every failure.
Create Rig Tasks: Convert those failures into programmatic checks in your measurement rig.
Prune: Use traces to remove one unnecessary tool or source of complexity.
Write One Extension: Test the seams by adding a custom tool or safety hook.
Record a Baseline: Establish a performance baseline in your CI/CD pipeline.
In a world of rapidly changing models, the specific code you write today is likely to be deprecated. However, your literacy in the “loop” and your ownership of the “ruler” are the only assets you truly own. In the end, the goal isn’t just to deliver an agent—it’s to own the loop and the ruler.
