Agent research becomes dangerous when a plausible sentence survives longer than the evidence behind it. A summary cites an earlier summary, a planning document treats the sentence as established, and an implementation decision eventually rests on a claim nobody can still reproduce.
For research that will drive architecture, migration, cleanup, or publication, citations scattered through prose are not enough. I want a claim ledger that records source, freshness, observation type, and the gap between what was seen and what was concluded.
- Pattern
- Claim-level provenance
- Output
- Research ledger
- Use when
- Agent research will influence an expensive or hard-to-reverse decision
One unsupported bug was the success case
In a migration investigation from the source sessions, an earlier summary proposed three regression candidates. The follow-up required commit hashes, actual diffs, and tests for each one. Two candidates were supported by repository history. The third could not be established from the code or commits and was marked unsupported.
Dropping that third candidate was not a failure to finish the story. It was the most important result of the research. The process prevented an attractive explanation from acquiring a second generation of authority.
The weak version of agent research asks, "Does this answer sound consistent with the available material?" The stronger version asks, "For each material claim, what was directly observed, what was inferred, and what would falsify it?"
Store claims as first-class records
A useful ledger is compact enough to maintain while still exposing uncertainty.
| Claim | Source | Freshness | Observation | Status | Open gap |
|---|---|---|---|---|---|
| The rewrite changed error handling in the parser | Commit abc123 and diff | Historical, immutable | Direct | Supported | None |
| The change caused the reported production symptom | Incident timeline plus matching test | Current at incident date | Inferred | Plausible | No runtime trace links the exact request |
| A second module has the same regression | Prior summary only | Unknown | Reported by another agent | Unsupported | Inspect module history and add a reproducer |
I use four statuses:
- Supported: the cited source directly establishes the claim.
- Plausible: evidence is consistent, but at least one causal or ownership link is inferred.
- Contested: authoritative sources disagree or changed over time.
- Unsupported: the available evidence does not establish the claim.
Confidence scores can be useful within a calibrated system, but a naked 0.9 is usually false precision. Status plus open gap tells the reviewer what to do next.
Provenance needs lineage, not just links
A URL or file reference proves very little by itself. The ledger should capture enough location and time context to retrieve the same evidence:
claim_id: parser-regression-01
claim: The parser began discarding nested errors after the rewrite.
status: supported
observation: direct
sources:
- kind: git_diff
repository: service-a
revision: abc123
path: src/parser.ts
lines: 84-112
- kind: regression_test
repository: service-a
revision: def456
path: test/parser-errors.test.ts
freshness_checked_at: 2026-07-11T09:30:00Z
inference: null
open_gap: nullFor live sources, record when they were checked and, where possible, preserve a stable snapshot or query. "The dashboard was green" without time range, filters, and environment will not survive the next deploy.
The ledger should also distinguish primary evidence from agent-produced artifacts. A prior agent report is useful context, but unless it embeds reproducible source evidence, it remains a secondary source.
Review the decision, not every sentence
A full ledger for every paragraph would make research unreadable and expensive. I create records for decision-bearing claims: statements that change scope, ownership, risk, cost, sequence, or the recommended action.
That yields a practical review flow:
- Read the recommendation and identify the claims it depends on.
- Inspect unsupported, contested, or stale claims first.
- Sample the strongest supported claims to calibrate source quality.
- Decide whether the remaining gaps change the decision.
This is a staff-level advantage because it moves review away from prose fluency. A concise report with three visible uncertainties is more useful than a seamless report whose uncertainty has been edited out.
The ledger can become stale too
Provenance is not permanence. Owners change, production state drifts, documentation is superseded, and links disappear. A detailed ledger can also create bureaucratic overhead or expose sensitive source locations.
Use immutable identifiers for code and artifacts, freshness windows for live state, neutral descriptions for confidential sources, and an owner for research that remains operationally important. Archive the ledger with the decision it supported so later readers know its scope.
Most importantly, do not let the ledger become a substitute for judgment. It can show that a claim is supported; it cannot decide whether the evidence is sufficient for the risk being taken.
The standard is simple: when an agent's research will move real engineering work, every decision-bearing claim should have a path back to evidence or an explicit label saying that path does not exist.