The problem
An autonomous agent is tasked with updating a configuration setting across four locations in a 400,000-line monorepo. It locates the four code points, but discovers conflicting values: one configuration file specifies a 30-second timeout, while another specifies 90 seconds. The agent arbitrarily picks one value, applies the edit silently without surfacing the contradiction, and completes the run. The pull request passes CI checks (as no test covers this specific timeout divergence) and silently ships inconsistent runtime behavior to production that only surfaces under peak load.
The agent did not hallucinate. It made a plausible assumption under incomplete information — and the core architectural failure was failing to flag that the decision was made under unresolvable ambiguity. This highlights the guiding theme of this lesson: in ultra-long contexts where exhaustive human verification is infeasible, the central question is not merely "how do I process what I see," but "should the agent resolve this autonomously or escalate to human review?"
Data provenance: every fact carries source and timestamp metadata
In workloads spanning multiple files, external services, or schema revisions, every extracted fact should record its origin and observation timestamp — not merely its raw scalar value:
{
"fact": "timeout = 90s",
"source": "config/prod.yaml",
"observed_at": "2026-08-15",
"conflicts_with": ["config/staging.yaml: timeout = 30s"]
}
Capturing provenance makes conflicts transparent and citable rather than silently resolved. The difference between an agent that tracks provenance and one that does not is the difference between "I detected a conflict, here are the two contradicting sources" versus "I picked one value silently and nobody knows an alternative existed."
Observation timestamps matter for identical reasons: a configuration extracted three months ago may be obsolete. Without tracking observation freshness, an agent cannot differentiate active ground truth from stale historical artifacts.
Make large contexts navigable instead of expanding window limits
The common engineering reflex when confronting massive codebases is dumping entire directories into the context window. This rarely succeeds: even when tokens fit within theoretical limits, model attention suffers from degradation on facts situated in the middle of long sequences (the "lost in the middle" effect). The robust architectural patterns are structural:
- Scratchpads: External persistent scratch spaces where agents record verified intermediate findings, avoiding redundant re-discovery passes within the prompt turn.
- Manifest indices: Structured metadata catalogs (file maps, domain boundaries, configuration loci) that replace naive full-text scans with targeted, indexed retrieval.
- Context pruning: Deliberately compacting and purging obsolete conversational tokens as milestones complete.
- Isolated subagents: Delegating deep subsystem investigations to isolated subagents that return compact structured findings rather than dumping thousands of exploratory tokens into the primary session.
The core heuristic: massive context challenges are not solved by expanding window limits, but by structuring information so only immediate essentials enter active context while everything else remains queryable on demand.
Calibrating autonomous resolution versus escalation
Not every ambiguity warrants pausing for human intervention, and not every ambiguity should be resolved autonomously. The operational decision framework:
- Resolve autonomously: When the blast radius of an error is negligible and trivial to reverse (e.g., code formatting choices, non-functional variable naming conventions).
- Escalate to human review: When contradictory facts impact production runtime behavior, when decisions carry high blast radius or irreversibility, or when confidence is low on critical-path changes.
The opening scenario failed because the agent autonomously resolved a production configuration discrepancy — a high-impact ambiguity that should have triggered an escalation gate.
Put it into practice
In this lesson's lab, you will build a provenance tracking engine in Python: it processes assertions from multiple conflicting sources, detects contradictions automatically, and routes each conflict between automated resolution and human escalation based on impact and reversibility matrices.