The Messages API is a state machine

The problem

A developer implements a conversation loop for the Messages API by inspecting the response content string with regular expressions, searching for patterns resembling tool requests in free-form prose. It works during manual smoke tests. In production, runtime behavior turns erratic: occasionally the loop loops endlessly when it should terminate, or halts prematurely mid-task. The bug does not stem from a flawed regular expression — it stems from treating the API as an unstructured text generator rather than what it actually is: a deterministic state machine emitting explicit control signals.

stop_reason is a control signal, not text to parse

Every Messages API response payload includes an explicit stop_reason property declaring why the model ceased generation. This field is not cosmetic telemetry — it is the exact control signal governing the application's next state transition:

• tool_use: The model emitted one or more structured tool call requests. The host runtime must execute the requested tools and append the results as tool_result content blocks before continuing the conversation turn. This should never be parsed from raw text — the SDK delivers typed tool_use content blocks containing the tool name, unique id, and pre-validated input arguments.
• end_turn: The model finished its conversational turn naturally. This represents the primary happy path, but it is not the only valid terminal state.
• max_tokens: Generation halted because the response hit the maximum token limit before completing its thought. Treating this as a normal end_turn silently delivers truncated, corrupted completions downstream.
• Additional stop reasons encompass custom stop sequences and safety filters. A production-grade implementation must handle all valid states distinctly rather than assuming binary branching between text and tools.

Treating stop_reason as a structured state machine signal rather than inferring intent from natural language strings is the difference between an enterprise-grade agent loop and brittle script automation.

Preserving conversation state across requests

The Messages API is strictly stateless across HTTP invocations — each request must supply the cumulative conversation history up to that point. The host application is responsible for preserving the structured schema of previous turns: the assistant message initiating tool_use, followed immediately by user turns containing matching tool_result blocks bound to the exact corresponding tool_use_id.

A destructive anti-pattern is "summarizing" past assistant outputs into plain text strings before resubmitting them to save tokens. This destroys the structured content blocks expected by the API, stripping tool identifiers and degrading downstream conversational coherence. If token compression is required, summarize durable facts into external persistent memory or system context (as covered in the context management lesson) — never compress the API's required structural protocol.

Errors, rate limits, and cancellation as first-class states

An architecture that only implements the happy path — receiving completions where stop_reason is cleanly end_turn or tool_use — is unready for production workloads. Robust designs handle three operational failure states as first-class transitions:

• Network and API errors: Timeouts, 5xx server errors, and 429 rate limits. Each demands a distinct strategy — rate limits require exponential backoff with jitter, whereas 400 client validation errors must not be blindly retried without payload correction.
• Truncated outputs (max_tokens): When responses hit token limits mid-thought, the system must trigger continuation handling or flag an error rather than treating the truncated payload as complete.
• Cancellation: Real-world production systems require explicit mechanisms to cancel in-flight API requests and agent loops when users navigate away or client timeouts expire.

Where intuition fails

• "I can detect tool calls by scanning response text for keywords." The API natively provides structured tool_use blocks paired with stop_reason: tool_use — string regex parsing is an error-prone reinvention of existing core functionality.
• "Summarizing past turns into unstructured text saves tokens without downsides." It breaks required block schemas and destroys tool call provenance, degrading reasoning fidelity.
• "Handling end_turn and tool_use is enough; other states are edge cases." Network drops, rate limits, token truncations, and client cancellations occur continuously at scale — ignoring them defers systemic outages.

Put it into practice

In this lesson's lab, you will implement a Messages API lifecycle state machine: given simulated API responses (including tool_use, end_turn, token limit truncations, and network faults), it executes state transitions deterministically, backed by test suites verifying comprehensive state coverage.