I was looking at an internal board of AI-mined story candidates, the kind of review surface editors use to decide what deserves human attention, and the newest candidate was not at the top. It had been scored, categorized, and politely filed into the middle of the page. That was the moment the product problem became obvious: in an AI editorial workflow, a quality score is not a queue order. The board was behaving like a ranked taxonomy when the operator needed a triage inbox.
- Pattern
- Freshness-first triage for AI-mined candidates
- Output
- Agent operating note
- Use when
- An AI system produces review candidates that humans must inspect, accept, reject, or defer
The useful decision was small and slightly irritating because it made the previous design look more sophisticated than useful. listStories now sorts by createdAt DESC. Score remains visible, but it only breaks ties among candidates from the same mining run. groupStories was removed from the board. Pillar, topic, sources, score, signal count, and mined date moved onto each card as filterable tags.
That is not a cosmetic preference. It is an operating constraint: when an AI system creates editorial candidates, freshness, quality, and actionability are separate dimensions. If the interface collapses them into one ranking, the human reviewer starts doing archaeology instead of triage.
Where the old board lied politely
The old design had a reasonable shape. Candidates had scores. Candidates belonged to pillars and topics. A board grouped them under those categories and ranked groups by their strongest candidate. That sounds like information architecture. It also sounds like the sort of thing one builds after staring at tidy sample data for too long.
The failure mode appeared when a fresh mining run produced new candidates. A newly mined story could land below older candidates because its score was lower. Worse, pillar and topic grouping made metadata act like navigation. The page invited the reviewer to browse a taxonomy rather than inspect the newest work the system had just produced.
No outage. No dramatic correctness bug. Just a product surface that made the next human action less obvious.
That distinction matters for agentic systems because the board is not a library. It is a handoff point. The mining job has done work and is asking a person to make decisions. The primary question is not, “What is theoretically best in this corpus?” It is, “What did the system just add, and what needs review now?”
AI editorial workflow sorting should match the job
A review board usually has at least three competing dimensions:
| Dimension | What it answers | Why it should not own the whole sort |
|---|---|---|
| Freshness | What arrived most recently? | It says nothing about strength or relevance. |
| Quality | How promising is this candidate? | It can bury new work if treated as the global order. |
| Actionability | What can a reviewer decide now? | It depends on context, filters, and current editorial goals. |
The mistake was letting the quality score impersonate actionability.
Scores are useful evidence. A high score tells me the candidate may have stronger signals, cleaner source support, or a better fit for the editorial system’s criteria. But a score is not the same as a queue position. Once a candidate is mined, the reviewer needs to see it, compare it with adjacent arrivals, and either move it forward or discard it. Hiding it halfway down the board because an older candidate scored better makes the system feel stale, even when the data is fresh.
The first attempted adjustment was to keep grouping but rank pillars and topics by their most recently mined candidate. That improved the top of the page: the newest story could bubble upward. It did not remove the deeper confusion. The grouping still told reviewers that pillar and topic were the main way to traverse the work. They were not. They were metadata for filtering.
So the board became flat.
The contract I would use before changing a review surface
The reusable artifact here is not a sorting function. It is a review contract. Before a team lets an agent or mining job populate a human review queue, I want the team to write down which dimension owns ordering, which dimensions remain evidence, and which filters are allowed to narrow the work without hiding arrivals.
Candidate review contract
Surface
- Surface name: Stories board
- Producer: Mining job
- Reviewer role: Editor or operator
- Primary job: Inspect newly mined candidates and decide whether to advance, defer, or reject them
Ordering rule
| Rule | Decision |
|---|---|
| Primary order | Newest mined first, using createdAt DESC |
| Same-run tiebreaker | Higher score first |
| Hidden ordering inputs | None |
| Reviewer-visible ordering evidence | Mined date and score |
Metadata rule
| Field | Role on the board |
|---|---|
| Pillar | Filterable tag |
| Topic | Filterable tag |
| Sources | Filterable tag or count |
| Score | Visible evidence and same-run tiebreaker |
| Signal count | Visible evidence |
| Mined date | Visible evidence and primary ordering explanation |
Rejected design
| Alternative | Reason rejected |
|---|---|
| Global score ranking | Fresh candidates can land mid-list, which hides new work from the reviewer |
| Pillar/topic grouping | Metadata becomes navigation, and reviewers browse categories instead of triaging arrivals |
| Group ranking by newest candidate | Improves freshness at the top but keeps the wrong page model |
Stop condition
This contract stops being correct if the surface is no longer a triage inbox. For retrospective planning, quality analysis, or corpus exploration, provide a separate ranked or grouped view rather than changing this queue.
This artifact is deliberately plain. It forces the team to name the board’s job before debating components. Without that, “sort by score” and “group by topic” both sound defensible, because both are defensible for a different product.
What we gave up
The flat board loses a kind of editorial overview. A grouped page can make the corpus feel organized. It can help a reviewer see that one pillar is suddenly rich with candidates or that a topic is empty. There is real value in that, especially for planning and coverage analysis.
But that value belongs in a different mode. If the same surface is both inbox and taxonomy, the inbox usually loses. Fresh items become less visible, and the reviewer has to infer whether the absence of a candidate near the top means “nothing new” or “new, but not highly scored enough.” That is a bad little ambiguity. It teaches people to distrust the board.
The chosen design still has a sharp edge: newest-first can overemphasize mediocre recent candidates. A reviewer may spend time on fresh but weak material while an older high-scoring candidate waits below. That is why score did not disappear. It stayed on the card and remained the same-run tiebreaker. The board should not pretend quality is irrelevant. It should keep quality in the right lane.
The recommendation stops being correct when the review job changes. If the operator is asking, “What are the strongest candidates this month?” then score should probably lead. If the operator is auditing coverage by pillar, grouping may be the right first interaction. But for a board fed by recurring AI mining, where the next action is to review what just arrived, newest-first is the honest default.
The small implementation detail that mattered
The implementation change was modest: listStories orders by createdAt DESC, score breaks ties inside the same run, and groupStories is no longer part of the board. The product change was larger than the diff.
Each card now carries the information the grouping used to own: pillar, topic, sources, score, signal count, and mined date. Those fields are not removed from the user’s mental model. They are demoted from architecture to controls. A reviewer can still filter down to a pillar or topic, but the board no longer starts by asserting that the taxonomy is the main path through the work.
That demotion is often the right move in AI-assisted review tools. Model outputs arrive with many plausible labels: confidence, category, source, owner, risk, novelty, urgency. The interface must decide which one is the queue order and which ones are evidence. If every label becomes structure, the screen starts to look considered while quietly avoiding the one decision that matters.
Here, the decision was that arrival order owns the queue. The rest helps the reviewer judge.
The staff-level review question
The question I would ask in review is not “Is this sorted correctly?” That is too local. The better question is: “What human action does this order optimize for?”
If the answer is triage, freshness probably deserves to lead. If the answer is selection, quality may lead. If the answer is coverage, grouping may lead. A single board can expose all three dimensions, but it should not let them fight invisibly inside one default order.
The quiet failure of the old board was that it had all the right metadata and the wrong posture. The fix was to make the first screen admit what job it was doing.