Why we don't build autonomous agents for production

"Agent" has become a word that means whatever the person saying it wants it to mean. That's bad for vendors and worse for buyers. When we say we don't build autonomous agents for production, we mean something specific: we don't ship systems where a language model is left to decide, on its own, which actions to take against your data, your customers, or your systems of record.

That's not a stylistic preference. It's a response to eighteen months of production incidents, a growing body of research on why those incidents happen, and explicit guidance from the people who build the models we rely on.

The most useful guidance on this question comes from Anthropic's own engineering team in the Building Effective Agents publication. The opening recommendation, unedited:

"When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all. Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense."

And further down:

"When more complexity is warranted, workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale."

This is the company that makes the models we build on. Their advice, in short: start with the simplest thing that could possibly work, prefer workflows when the task is well-defined, and only reach for autonomous agents when you actually need the flexibility.

Insurance submission intake is well-defined. Healthcare document triage is well-defined. Benefits verification is well-defined. Credentialing is well-defined. These are workflows, not agent problems. Building them as autonomous agents is complexity for its own sake, the exact failure mode Anthropic is warning against.

In a HIPAA-regulated workflow, "the model decided" is not an acceptable answer to "why did this happen?" A compliance officer asking about a PHI disclosure wants to see the specific code path that produced the disclosure, the inputs to that path, and the logged decisions made along the way. Autonomous agents, by construction, don't produce that artifact. They produce a reasoning trace in natural language, which is a different thing and not a legally defensible one.

The same logic applies to insurance, where state regulators ask carriers to show how rating, underwriting, and claims decisions were made, and to financial services, where every action taken against a customer account needs a reviewable audit trail that holds up in a regulatory exam.

Autonomous agents don't fail on these workflows because the engineering is hard. They fail because the core architectural property (the model decides) is incompatible with the core compliance property (the decision must be explainable, reproducible, and bounded).

You can bolt audit logging onto an autonomous agent. You cannot bolt on determinism after the fact.

Aren't you just behind the curve? Autonomous agents are the future. +

We might be. The honest answer is that we're building for regulated operations in 2026, not general AI in 2030. In the current state of the art, and based on documented incidents from Anthropic, Amazon, Google, Replit, and Cursor over the last eighteen months, autonomous agents are not ready for production use in environments where a mistake has compliance, legal, or financial consequences. When the reliability curve shifts, our position will shift with it. The UMM approach is designed to absorb that: the micro-agents inside a UMM get smarter over time, and the orchestration layer stays the same.

Doesn't Claude Code / Cursor / Cowork prove agents work? +

They prove agents work for coding in developer environments, which satisfies all four of the conditions above: outputs are automatically verifiable, the environment is a sandbox, the blast radius is a git branch, and a human reviews before merge. We use these tools ourselves. They are not evidence that the same architecture works for processing protected health information or handling insurance submissions against a book of record.

What about orchestrator-worker patterns? Isn't that an agent? +

It's a workflow pattern with a judgment step, and we use it constantly. An orchestrator LLM that decomposes a bounded problem into sub-tasks, delegates to specialist agents, and synthesizes their outputs is fine, especially when the orchestrator's action space is itself bounded (it can only route to specific workers, not invoke arbitrary tools). What we don't build is the version where the orchestrator can also open shells, call APIs we didn't predefine, or decide on the fly to take destructive actions. Autonomy in the decomposition is fine; autonomy in the execution is where production agents break.

How do you handle tasks that genuinely need open-ended reasoning? +

By asking whether the workflow actually needs it. Most of the workflows we're asked to automate don't. They look open-ended only because they've never been decomposed. When a workflow does need open-ended reasoning, we isolate that reasoning to a specific step, bound its action space, require structured output, and surface the result to a human before anything with side effects happens. The reasoning is real; the autonomy is constrained.

Does this position apply to your Discovery engagements too? +

No, and that's worth distinguishing. In Discovery, we often use agent-style tools ourselves to analyze workflows, read documentation, and generate hypotheses. That's our working environment: sandboxed, human-reviewed, with outputs that get sanity-checked before anything ships to you. What we don't do is take the same tools and deploy them against your production systems.