Mining Huge AI Session Transcripts Without Drowning in Them

I was staring at a Claude Code JSONL file that was roughly 68 MB, which is small by database standards and large enough to punish every lazy editorial instinct I had. The job sounded harmless: mine a prior coding session for facts that could support a homelab blog series. The useful conclusion was not "ask the model for a summary." For AI transcript mining, the transcript has to be treated like evidence, and the agent needs a retrieval protocol before it earns the right to write.

The operational tension is familiar if you use coding agents in real delivery work. The transcript contains the facts, but it also contains every false start, tool error, abandoned hypothesis, and confident sentence that turned out to be adjacent to reality. Loading the whole thing invites the agent to compress noise into a tidy story. Reading none of it invites fiction with better grammar.

The workable middle is narrower: search for specific anchors, read small excerpts, and extract only claims that remain attached to source lines. That sounds primitive. Good. Primitive is often what keeps the system honest.

AI transcript mining needs a protocol, not a bigger context window

The incident was a homelab write-up. A prior Claude Code session had set up a personal home server on a spare laptop. The future article needed concrete details: hostname, services, network choices, process manager behavior, and the odd little constraints that make a setup real rather than brochure-shaped.

The source was a large JSONL transcript, about 67 to 68 MB. The instruction was explicit: do not cat the file. Use grep or rg with targeted terms, then read narrow line ranges. There was also a sandbox quirk: regexes that looked like IP address patterns could get blocked. A search for dotted numeric patterns with escapes or character classes was not reliable. The fallback was to search for surrounding words, or use fixed-string search for literal dotted values when a literal was already known.

That detail matters because it changes the job from summarization to evidence handling. The agent was not being asked to remember a conversation. It was being asked to interrogate a file without flooding itself, tripping the environment, or laundering an inference into a fact.

A reasonable team might be tempted to solve this with a larger model context or a one-shot digest prompt. I would reject that for source material. It is acceptable when the output is disposable orientation. It is not acceptable when the output will become a technical article, runbook, architecture note, or migration record. Once the summary becomes the source of truth, every downstream paragraph inherits its omissions.

The failed move was trying to make the transcript convenient

The tempting move is to turn the transcript into a digest first. Ask for "everything important," let the model compress it, then write from the digest. That feels efficient because the file becomes smaller.

It also changes the unit of trust. You are no longer trusting line-backed observations. You are trusting a synthetic artifact whose errors are hard to locate. A repeated keyword might mean a real pattern, or it might mean the user corrected the agent five times. A command appearing in the transcript might mean it ran successfully, or it might mean it was proposed and rejected. JSONL logs are not prose. They are event streams with status, role, tool calls, and local context folded together in inconvenient ways, which is how you can tell they came from a real system.

The consequence in this case was not a dramatic outage. It was more mundane and more likely: the blog series could have shipped with technically plausible but unverifiable claims about what the server ran and how it was configured. That is enough damage. A reader can forgive a narrow article. They should not have to debug invented infrastructure.

The review contract I now want before writing from transcripts

Here is the reusable artifact I would put in front of any agent asked to mine a large coding transcript. It is deliberately operational. It tells the agent what it may claim, how to search, and when to stop.

cd ~/agent-sessions/project
grep -n "caffeinate" session.jsonl
grep -n "launchctl" session.jsonl
grep -n "hostname" session.jsonl

grep -n -F "192.0.2.10" session.jsonl

The important part is not the exact commands. It is the contract between retrieval and prose. Writing starts only after the agent has a table of claims whose status is visible.

The tradeoff is slower first contact and cleaner later review

This approach gives up speed at the beginning. A one-shot digest produces something readable in a minute. Evidence-first transcript mining produces a thin table first, and thin tables are not emotionally satisfying. They make the uncertainty visible before anyone has enjoyed the narrative.

That is the cost I would pay for anything that will be cited, published, or used to guide engineering work. It lets a reviewer challenge a claim without replaying the whole agent session. If the note says a process was kept awake with caffeinate, the reviewer can ask for the anchor. If the table says stated rather than observed, the prose should not quietly upgrade it into implementation fact.

The rejected alternative is full indexing of every transcript into a vector store. I would consider that if the team had many transcripts, stable schemas, and recurring questions across sessions. For a single large source, it adds a retrieval system before proving the retrieval discipline. Worse, semantic search can be very good at finding nearby language and less good at preserving whether the nearby language was true.

The recommendation stops being correct when the question is broad orientation rather than source-backed publication. If a staff engineer asks, "What is probably in this session so I know where to look tomorrow?" a rough digest is fine, provided nobody mistakes it for evidence. The stricter protocol is for claims that will survive beyond the chat window.

The sandbox quirk was not incidental

The IP-shaped regex issue is the kind of constraint that separates a real operating note from a generic appeal to verification. The environment did not merely prefer careful search. It punished a reasonable search pattern. Regexes containing dotted numeric shapes, backslashes, or certain character classes could be blocked, which meant the agent needed a different tactic.

That changed the protocol in two ways. First, search on semantic neighbors: host, listen, bind, port, caddy, ssh, or the service name. Second, when a literal value is already known, avoid clever regex and use fixed-string matching.

cd ~/agent-sessions/project
grep -n "listen" session.jsonl
grep -n "bind" session.jsonl
grep -n -F "192.0.2.10" session.jsonl

There is a small humility in that. The more constrained the environment, the less room there is for elegant retrieval theater. You either find the line or you do not.

The sharp edge

The limitation is that this protocol can miss facts that are real but poorly named. If the session used generic labels, renamed services halfway through, or buried a decision in conversational text with no stable anchor, targeted search may not find it. The remedy is not to abandon the protocol. It is to label the resulting gap honestly and, when the source warrants it, run a second pass with broader but still bounded terms.

There is also a human cost. Someone has to decide which claims deserve the slower treatment. Applying this contract to every casual recap would turn useful agents into very polite filing clerks. The boundary I use is persistence: if the output will be reused as source material, make the evidence table; if it is just a map for the next hour, keep it lightweight.

What changes in review

The review conversation becomes less literary and more technical. Instead of arguing about whether the article "feels accurate," the reviewer can inspect the claim table. Did this sentence come from an observed command result, a user statement, or an inference? Does the excerpt support the degree of certainty? Did the agent avoid reading the whole transcript and still gather enough context?

That is the staff-level decision: do not ask the agent for a better summary when what you need is a narrower chain of custody. The transcript is not raw material to be melted down into prose. It is evidence with awkward edges, and the edges are where the useful facts tend to live.