hebel. today · 14 November 202623 / 80

Agentic Engineering · Agent Evals & Reliability

Production agents stay safe with rigid action catalogs

Horizon3 limited its AI agent to a finite, pre-approved menu of deterministic tools to ensure it stayed within bounds during 420 deployments. This architecture allowed a smaller Claude Haiku 4.5 model to perform as…

Horizon3 limited its AI agent to a finite, pre-approved menu of deterministic tools to ensure it stayed within bounds during 420 deployments (Ruben Dominguez from The AI Corner). This architecture allowed a smaller Claude Haiku 4.5 model to perform as reliably as a larger one, as the system restricted the agent to specific, validated actions regardless of the model's internal reasoning. The company machine-checked system stability using a formal proof in Lean 4 and verified that the agent cleaned up after itself and logged all activity. This approach shifted the security conversation from avoiding disruption to proving impact, enabling autonomous EDR policy tuning without operational incidents.

  • Uber Engineering enforces policy decisions with Agent Guard hooks that deny tool calls if they violate an agent's declared intent, even when credentials are valid (Resilient Cyber).
  • Harness Arena tests allow developers to compare the cost, execution time, and success rate of different harness and model combinations side by side (Daily Dose of Data Science).
  • Cemri et al. identified 14 recurring failure modes in multi-agent systems, finding that most stem from system design rather than model capability (O'Reilly Radar The latest on the trends…).
  • AgentCore uses OpenTelemetry to generate spans for every LLM and tool call, allowing teams to trigger alarms based on failed turns or high latency (Sarthak Rastogi from AI Agent Engineering).

Try today · 45 min

Define a fixed action catalog for your agent that restricts it to a finite, pre-approved list of deterministic tools.

List every external API or file system operation your agent currently performs and create a wrapper that validates these against a hardcoded schema.

One email instead of your newsletter pile.

Forward the newsletters you already get (a Gmail filter does it) and tell Hebel your goals. Every morning it reads them with 20,000 others and sends one short email, with one small thing to do today. Fewer emails, not more. Free during beta.

Start my paper →
LinkedIn X

Sources

  1. Ruben Dominguez, The AI Corner on hebel. → ✎ Written by a personThis report details how Horizon3 used a constrained action catalog to maintain agent stability and security across hundreds of deployments.

  2. Resilient Cyber on hebel. →Uber Engineering explains its use of Agent Guard hooks to enforce policy constraints at runtime before tool calls proceed.

  3. Daily Dose of Data Science on hebel. →Harness Arena provides a framework for benchmarking agent harnesses, showing how different configurations change costs and success rates.

  4. O'Reilly Radar The latest on the trends… on hebel. →The MAST taxonomy categorizes 14 common multi-agent failure modes, highlighting how system design errors lead to quiet, fluent agent failures.

  5. Sarthak Rastogi, AI Agent Engineering on hebel. → ✎ Written by a personAgentCore outlines a method for using OpenTelemetry spans and CloudWatch alarms to monitor agent performance and trace failures.

More in Agentic Engineering

Autonomous agent reliability hinges on orchestration versus human oversight

Building reliable agent harnesses

Engineers pivot from prompt engineering to structured component manifests

LinkedIn automates 40,000 weekly pull requests

Agent Evals & Reliability yesterday

Production agents require evidence loops instead of green dashboards

Sign in to Hebel first

Likes, your collection and done Hebel need an account. We email you a six digit code; no password.