A while-loop that calls an LLM until it stops asking for tools is not a reliable agent. It looks like one in a demo: one tool call, perfect output, the loop terminates cleanly. You ship it. Then it calls the same tool fifteen times. Then it deletes a row you didn't ask it to delete.
The fix is not a longer system prompt. The fix is control flow.
Here's the loop that most people start with:
while (true) {
const response = await llm.call(messages, tools);
if (response.stop_reason === "end_turn") break;
// ... process tool calls
}This works fine in demos. In production it spirals. A few failure modes you'll hit:
Infinite tool-call loops. The model calls searchDatabase and gets back zero results. So it calls searchDatabase again with a slightly different query. Then again. The model is pattern-matching on "tool returned nothing, try again", and it will keep doing that until you kill the process.
Wrong sequence. You expect the model to fetch data, then classify it, then write results. Instead it decides to write results first, discovers it has no data, then fetches. Or it skips the classification step entirely because it thinks it already knows the labels.
Retry thrash. A downstream API is down. The model retries the tool call. Gets an error. Retries again. Gets the same error. There's no backoff, no circuit breaker, no "stop and tell the user" path, because those are code decisions, not prompt decisions.
Scope creep. You ask the agent to label open tickets. While doing that it notices some tickets have duplicate descriptions, so it starts merging them. You didn't ask for that. The model decided it was helpful.
Side effects without gates. The model calls deleteRecord, sendEmail, or deployToProduction. These go through immediately because there's nothing in the loop that says otherwise. Maybe the prompt says "be careful with destructive actions." The model agreed. Then it did it anyway, because it decided the action was warranted.
None of these problems are prompt problems. More careful wording does not fix an infinite loop. An iteration cap does. A fixed sequence does. A gate before destructive actions does.
Reliable agents are mostly ordinary control-flow engineering with models inserted at judgement points. AI agent orchestration is the name for that code: the part that picks the next step, holds the state, and stops the run.
There are three levels. Most teams ship Level 1 and wonder why things are unreliable. The move up to Level 2 is fast. Level 3 is where serious reliability lives.
The loop above. The model owns the entire sequence. It decides what to call, in what order, and when it's done. You get maximum flexibility and minimum predictability.
This is fine for prototyping. It is not fine for production workflows that touch real data.
Add two things: an iteration cap, and a structured output for the done/continue decision.
import { z } from "zod";
const StepResult = z.object({
done: z.boolean(),
reasoning: z.string().optional(),
});
const MAX_ITERATIONS = 5;
for (let i = 0; i < MAX_ITERATIONS; i++) {
const response = await llm.call(messages, tools);
const result = StepResult.parse(parseStructuredOutput(response));
if (result.done) break;
if (i === MAX_ITERATIONS - 1) {
throw new Error(`Agent exceeded iteration limit of ${MAX_ITERATIONS}`);
}
// ... process tool calls, update messages
}The Zod schema does two things. It forces the model to make an explicit done/continue decision instead of deciding implicitly. And it gives you a hard failure mode when the model doesn't terminate. The parse throws, and you handle it like any other error.
Five iterations is often enough. If your task genuinely needs more, that's a signal it should be broken into smaller steps. The cap is not a performance optimization. It is a correctness constraint.
This gets you pretty far. Most "agent gone wrong" stories are solved at Level 2.
At some point you realize that you, not the model, know the correct sequence of operations. Fetch, classify, label. That order. Always. The model is good at the classification step; it is not better than you at deciding that classification comes before labelling.
Own the sequence:
type Step = "fetch" | "classify" | "label" | "done";
type AgentState = {
tickets: Ticket[];
labels: Map<string, string>;
step: Step;
};
async function runWorkflow(state: AgentState): Promise<AgentState> {
switch (state.step) {
case "fetch":
return { ...state, tickets: await fetchOpenTickets(), step: "classify" };
case "classify":
return {
...state,
labels: await classifyTickets(state.tickets),
step: "label",
};
case "label":
await applyLabels(state.labels);
return { ...state, step: "done" };
case "done":
return state;
}
}classifyTickets calls the model. The model is doing what it's good at: reading ticket text and returning a structured label. But it does not decide that this is the next step. Your switch decides that. The model handles judgement inside a state; your code handles the sequence between states.
This is a workflow, not an agent. That distinction matters. I'll come back to it.
The loop version of an agent keeps all its context in the message history. The model infers where it is from the conversation so far. This works until it doesn't. Message history grows long, the model loses track, or you need to restart from a checkpoint after a failure.
Typed state fixes this. Define what the run knows:
interface AgentState {
runId: string;
step: "classify" | "label" | "done";
input: Ticket[];
labels?: Map<string, string>;
error?: string;
}Three categories:
If you can persist this state to a database between steps, you get resumability for free. The run crashes, you load the last AgentState, and runWorkflow picks up exactly where it left off. The full architecture post goes deeper on run stores, memory, and observability.
Max iterations. Already covered. Pick a number and throw when you hit it. Don't swallow the error.
Zod-validated structured outputs. Every LLM call that needs to produce structured data should have a schema. Treat it as a runtime assertion, not as documentation. If the model returns malformed output, you want an error, not a silent wrong result.
const ClassificationResult = z.object({
labels: z.record(z.string(), z.enum(["bug", "feature", "question", "spam"])),
confidence: z.number().min(0).max(1),
});
async function classifyTickets(
tickets: Ticket[],
): Promise<Map<string, string>> {
const response = await llm.call(classifyPrompt(tickets));
const { labels } = ClassificationResult.parse(JSON.parse(response.content));
return new Map(Object.entries(labels));
}Idempotency keys on side-effectful tools. Before any tool that has side effects, generate a key scoped to the run and step. Pass it to the tool. The tool uses it to deduplicate. If the step runs twice (retry, crash recovery), the effect happens once.
async function applyLabels(
labels: Map<string, string>,
runId: string,
): Promise<void> {
const idempotencyKey = `apply-labels-${runId}`;
await labelingApi.apply({
labels: Object.fromEntries(labels),
idempotencyKey,
});
}Human approval before destructive actions. This is just an if statement. Not a feature of an agent orchestration framework. Not a plugin:
if (requiresApproval(action)) {
await waitForHumanApproval(action);
}
await executeTool(action);requiresApproval is a function you write that checks action type, risk level, whatever your rules are. waitForHumanApproval suspends the run and resumes when someone clicks approve, or throws if they reject. The model does not know this gate exists. It just sees that the tool was eventually executed (or not). The gate is yours.
Human-in-the-loop is control flow, not a feature you add on top of an agent framework.
Models are good at extraction, summarisation, classification, and structured judgement inside a step. Reading a block of text and deciding if it's a bug report. Pulling fields out of an unstructured email. Ranking a list by relevance to a query.
They are not better than your code at deciding which step comes next. They are not better than your code at remembering that a side effect already happened. They are not better than your code at enforcing an iteration bound.
"Let the model decide" is good advice for the judgement calls inside a step. It is bad advice for the sequence of steps, the state tracking, and the safety gates.
Your control flow calls tools. If you want those tools defined portably (shareable across agents, composable with other systems), MCP is how you do that. But the orchestration that calls them is yours to write.
Stop calling it an agent if it's actually a workflow. Both are fine. Naming the thing tells you what tools and reliability budget it deserves.
A workflow has a fixed sequence of steps. You know them at write time. The model handles judgement inside steps, but the sequence itself is code. This is most of what people actually ship, and it should be. Workflows are auditable, testable, and resumable.
An agent plans its own sequence. It needs to adapt based on what it finds. It might take a branch you didn't anticipate. This is genuinely harder to make reliable, and it needs the full apparatus: typed state, iteration caps, structured outputs, approval gates, a run store, and observability. Don't reach for it unless you actually need the flexibility.
Most of what's being called an "AI agent" right now is a workflow with a free-form loop at its core. Add a bounded loop and a typed state object and you've done most of the real work.
The prompt is not the problem.
Occasional notes on software, tools, and things I learn. No spam.
Unsubscribe anytime.