Batch Intake Triage β Evaluating Stories Together, Not One at a Time
Part of the AI Software House series.
In short: Per-story triage could say whether one article was acceptable, but not whether it was the best story waiting. Batch triage holds incoming items briefly, lets editors compare them together, and keeps the tracker-specific work behind a
TrackerAdapter.
The isolation problem
editorial triage's triage solved the worst noise problem, but every discussion still saw only one story. The editors could ask βis this publishable?β They could not ask βis this worth doing before the other five stories waiting today?β
That difference showed up quickly. A middling story could pass on a quiet day even though the same editors would drop it immediately beside several stronger stories.
It also meant paying for a separate discussion for every item. To compare stories properly, I needed to let a small queue form first.
The TrackerAdapter abstraction
GitHub Issues are the current inbox, but I did not want batch triage itself to become GitHub code. I expect to use the same idea for other intake systems later, possibly JIRA.
@dataclass
class TriageItem:
id: str # issue number, ticket ID, etc.
title: str
body: str # full content
url: str
created_at: datetime
metadata: dict # tracker-specific extras
class TrackerAdapter(ABC):
@abstractmethod
def list_pending(self) -> list[TriageItem]: ...
@abstractmethod
def approve(self, item: TriageItem, notes: str) -> None: ...
@abstractmethod
def skip(self, item: TriageItem, reason: str) -> None: ...
@abstractmethod
def is_approved(self, item_id: str) -> tuple[bool, str]: ...
TriageItem is the tracker-agnostic record. Everything the triage logic needs is in the dataclass; the metadata dict carries tracker-specific extras (GitHub labels, Jira status fields) without polluting the interface.
The GitHubTrackerAdapter maps this interface to GitHub Issues: list_pending() queries issues with the triage-pending label, approve() removes that label and adds triage-approved + the pipeline trigger label, skip() posts a comment and closes the issue. The is_approved() method is the bridge to the per-story triage fast-pass β more on that shortly.
A future JiraTrackerAdapter implements the same four methods using JQL and Jira transitions. The intake_triage.py script never changes.
Four triggers
The batch triage script is cron-runnable. It needs to decide when to actually fire. Four triggers are evaluated in order, and any one is sufficient:
1. --run flag (manual) β unconditional. Useful for testing or when you want to drain the pending queue immediately.
2. min_count β fire when β₯ N items are pending. Default is 5. Waits until there's a meaningful batch to compare.
3. max_age_hours β fire when the oldest pending item has been waiting β₯ N hours. Default is 6. Prevents stories from sitting in limbo too long even if volume is low.
4. schedule β fire if the cron expression matched within the last 65 minutes. Default is "0 9 * * *" (9am daily). The 65-minute window handles the case where the cron job runs slightly after the scheduled time.
def _should_fire(cfg, items, force=False, _now=None) -> bool:
if not items:
return False
if force:
return True
if trigger.min_count is not None and len(items) >= trigger.min_count:
return True
if trigger.max_age_hours is not None:
oldest = min(items, key=lambda x: x.created_at)
age = (now - oldest.created_at).total_seconds() / 3600
if age >= trigger.max_age_hours:
return True
if trigger.schedule:
cron = croniter(trigger.schedule, now - timedelta(minutes=65))
if cron.get_next(datetime) <= now:
return True
return False
With an empty queue, the script exits before creating a discussion or making any LLM call.
The batch context and discussion
When a trigger fires, items are sorted oldest-first, sliced to max_batch_size (default 10), and formatted into a single context block:
## Pending Items for Editorial Review
Triage scope: Focus areas: AI, software development tools, cybersecurity...
Item count: 5
--- ITEM 1 ---
Title: Apple releases iOS 18.5 with new AI features
URL: https://github.com/owner/ai-it-press/issues/42
Summary: Apple has released iOS 18.5 today, introducing on-device inference...
--- ITEM 2 ---
...
Body is truncated to body_preview_chars (default 300) per item. The editors see enough to evaluate each story, not so much that the context window fills before item 5.
The discussion (discussions/intake-triage.yaml) uses the same participants as per-story triage β editorial director, audience specialist, news editor β but the prompt context now shows all items simultaneously. Editors can explicitly compare: "Item 3 covers the same ground as Item 1 but with less depth β SKIP Item 3."
The moderator's final synthesis message emits per-item verdicts:
ITEM 1: PUBLISH
NOTES: Focus on the on-device inference angle for HK enterprise users.
ITEM 2: SKIP
NOTES: Acquisition too US-centric, no HK/Asia angle.
ITEM 3: PUBLISH
NOTES: Lead with privacy implications for Cantonese readers.
Parsed by _parse_batch_verdicts(text, item_count) into a list of (verdict, notes) tuples. Two regex passes: one looking for the full ITEM N: VERDICT\nNOTES: ... block, one fallback looking for bare ITEM N: VERDICT lines. Any item the parser can't match defaults to ("PUBLISH", "") β fail-open again, same principle as per-story triage.
What happens to max_batch_size overflow
If 14 items are pending and max_batch_size is 10, the oldest 10 are processed. The remaining 4 stay in triage-pending and will be included in the next run. Items don't expire or get auto-approved β they wait.
This keeps one busy period from producing an enormous prompt. If the remaining items are already old, max_age_hours may trigger another batch almost immediately, which is fineβthe point is to avoid leaving stale stories behind.
The fast-pass integration
Batch triage and per-story triage coexist. When the main pipeline processes a story that was already batch-triaged, the _stage_news_triage() method checks first:
def _stage_news_triage(self, result: PipelineResult) -> None:
if self._intake_triage_approved(result):
log.info("news_triage: batch-approved, fast-pass")
return # editorial_verdict/notes already set
# Original per-story triage (unchanged)
...
def _intake_triage_approved(self, result: PipelineResult) -> bool:
adapter = self._get_tracker_adapter() # None if intake_triage disabled
if not adapter:
return False
approved, notes = adapter.is_approved(str(result.issue_number))
if approved:
result.editorial_verdict = "PUBLISH"
result.editorial_notes = notes
return approved
The is_approved() method checks the issue's labels for triage-approved, then fetches the most recent [INTAKE TRIAGE] comment to extract the editorial notes. The notes are the same ones the batch editors wrote β they carry forward to the writer just like per-story triage notes do.
When intake_triage.enabled: false (the default), _get_tracker_adapter() returns None, the fast-pass never fires, and per-story triage runs exactly as before. Zero impact on existing pipelines.
Human override
I did not add a separate override system. A person can add triage-approved in GitHub, and the ordinary fast-pass handles it. Manual and automated decisions use the same state.
Same logic for skipping: a human closes the issue manually. The pipeline watcher only picks up open issues, so a closed issue is never processed regardless of its labels.
The CLI
# Normal cron run β respects all trigger conditions
python intake_triage.py
# Manual trigger β fires regardless of count/age/schedule
python intake_triage.py --run
# Dry run β shows what would happen, no write/state-changing API calls
python intake_triage.py --dry-run
The --dry-run flag is the most useful for debugging. It logs the full batch context that would be sent to the discussion agents β you can see exactly what the editors will see before paying for the LLM calls.
Scope beyond news
The TrackerAdapter abstraction and the intake_triage.py script were designed with a second use case in mind from the start: client project intake for the ai-software-house build pipeline. Future client requests arriving by email (or API, or form) get issues created with a triage-pending label. A partner team β different roles, different scope description, same infrastructure β discusses the batch and approves or rejects each request before the expensive build pipeline runs.
That future use case is why I kept trigger logic, batch formatting, and verdict parsing separate from the editorial roles and labels. The news desk is one configuration of the intake system, not the intake system itself.
Related reading: Editorial Triage