hebel. today · 4 November 202637 / 69

Technology · AI engineering

Build a custom harness to control your agent's execution environment

Frameworks like LangGraph provide the orchestration logic for your agents, but a harness acts as the operating environment that manages file access, command execution, and context tracking. By building a custom harness rather than…

Frameworks like LangGraph provide the orchestration logic for your agents, but a harness acts as the operating environment that manages file access, command execution, and context tracking (Daily Dose of Data Science). By building a custom harness rather than relying on generic tools, you gain the ability to enforce specific repository instructions and access boundaries, which significantly improves task success rates. For instance, LangChain improved its coding agent's benchmark performance from 53% to 67% by replacing its harness while keeping the underlying model identical (Decoding AI Magazine). This shift allows you to define exactly what counts as done for your specific application, ensuring that the agent's intermediate actions remain aligned with your production requirements.

  • Ramp now merges roughly half of its pull requests using a custom-built agent harness, demonstrating the efficiency gains of moving away from off-the-shelf tools (Decoding AI Magazine).
  • PostHog found that subagents often increase costs and trigger unnecessary retries, leading them to favor reliability and visible reasoning over adding new agentic capabilities (The AI Corner).
  • A job is worth passing to an AI feature if it takes more than 30 seconds to perform, requires a language users do not know, or is repeated at least 20 times (The AI Corner).
  • Production-grade agents require a self-healing watchdog module and clear separation between cloud and state to ensure they remain always-on and resilient to failures (Linas's Newsletter).

Try today · 30 min

Create a simple harness script that logs every tool call and file change into a dedicated artifact file.

Start by wrapping your current agent loop in a function that captures input and output before and after each model call.

You need

  • Existing agent codebase
  • Logging library
  • JSON file for artifact storage

Steps

  1. Define a log structure to record the agent's current state and the tool it intends to call.
  2. Insert a hook into your agent loop that writes the state to a JSON file before execution.
  3. Capture the tool output and append it to the same file after execution.
  4. Verify that the file persists the sequence of actions even if the agent process crashes.
LinkedIn X

Sources

  1. Daily Dose of Data ScienceExplains the distinction between frameworks for orchestration and harnesses for environment control, emphasizing inspectable progress artifacts.

  2. Decoding AI MagazineDetails the performance benefits of custom harnesses and provides a roadmap for building one from scratch.

  3. Ruben Dominguez, The AI Corner ✎ Written by a personSummarizes PostHog's lessons on agent reliability and why they moved away from subagents to reduce cost and complexity.

  4. Linas's NewsletterOutlines a five-module architecture for production-grade agents, including the need for a self-healing watchdog.

Get this every morning, for your goal.

Tell Hebel your goals and when you want it. Every morning it reads 20,000 newsletters and writes your paper, with one small thing to do today. Free during beta.

Start my paper →

More in Technology

Automated patching versus code verification in an AI-driven threat landscape

Treat AI agents as managed identities with individual badges

Turn recurring work into persistent Claude skills

Beijing consolidates its role as a deep tech and AI powerhouse

Delegating security to AI agents creates new points of failure

Design systems move from Figma to active code repositories

Webhook systems move from simple queues to state machines

AI engineering yesterday

Replace agent loops with durable execution runs

Sign in to Hebel first

Likes, your collection and done Hebel need an account. We email you a six digit code; no password.