Editorial Triage β€” Stop Wasting Agent Cycles on Bad Stories

πŸ“ Computer

Part of the AI Software House series.

In short: The RSS watcher treated every story as equally worth writing. I put three editor agents at the front of the pipeline so weak or irrelevant stories stop early, while useful stories continue with a clear angle for the writer.


The problem with processing everything

The first version began every story at discuss_news_analysis. If an RSS item existed, it received two discussion rounds, a writer, an editor, and two translationsβ€”roughly eight to ten LLM calls before a human ever saw it.

RSS feeds are not an editorial desk. A minor version bump can arrive beside a major architectural change. A US press release with no local relevance can arrive beside something genuinely useful to Hong Kong developers. The pipeline wrote them all, and I ended up rejecting the weak ones from the PR queue.

Changing feeds would only move the problem around. What I needed was an editorial decision before the expensive work began.


Three editors, two rounds

The triage stage runs a standard discussion preset (discussions/news-triage.yaml) with three participants:

editorial_director β€” evaluates strategic importance, story substance, and topic relevance against a configurable scope. This role's prompt includes {triage_scope} substitution, so scope changes happen in config, not code.

audience_specialist β€” evaluates cultural fit for the Hong Kong Cantonese-speaking tech professional audience. Does this story have a local angle? Will readers care?

news_editor β€” evaluates source credibility and whether there's enough raw material to write a complete article.

# discussions/news-triage.yaml
participants:
  - editorial_director
  - audience_specialist
  - news_editor
homework_round: true
max_rounds: 2
early_exit: true

Before the shared discussion, each editor forms an opinion independently. I kept the homework round here because it reduces anchoring: an editor who has already judged the story is more likely to challenge the first speaker instead of simply agreeing.

max_rounds: 2 caps cost. Triage is cheap by design β€” it's the gate before the expensive stages, not a thorough analysis.

early_exit: true stops the discussion early if all editors reach consensus. No point running round two if everyone agrees this is a SKIP.


The verdict format trick

Getting structured output out of a multi-participant discussion requires some care. The verdict isn't parsed from a dedicated output field β€” it's parsed from the moderator's final synthesis message. The discussion YAML instructs every participant to end their final message in a specific format:

verdict_format: |
  End your final message with:
  VERDICT: PUBLISH|SKIP
  EDITORIAL_NOTES: <one sentence: angle for writer, or reason for skip>

The orchestrator then extracts the verdict with a regex:

def _parse_triage_verdict(text: str) -> dict:
    # VERDICT: (PUBLISH|SKIP) β€” case-insensitive
    # EDITORIAL_NOTES: (.+) β€” rest of line
    ...
    return {"verdict": "PUBLISH", "notes": ""}  # fail-open on any parse error

On a parse failure, the default is PUBLISH. That choice is intentionally cautious: a weak story that slips through can still be rejected in the PR, while a good story silently discarded by a broken regex may never be noticed.


Aborting on SKIP

When the verdict is SKIP, two things happen: the GitHub issue gets a comment explaining why, and the issue is closed. Then the pipeline halts.

The halt mechanism reuses the existing PipelineStage.stop_if field β€” no new infrastructure:

"news_triage": PipelineStage(
    fn=self._stage_news_triage,
    stop_if=lambda r: r.editorial_verdict == "SKIP",
    stop_message="🚫 Editorial triage: story skipped β€” pipeline aborted.",
),

The run() loop checks stop_if after each stage. If it returns True, execution stops and subsequent stages are never called. The Cantonese translation stage, the Traditional Chinese translation stage, the PR stage β€” all skipped. The issue is closed, the run is logged, and nothing is committed to the press repo.

A skipped story now costs two or three calls instead of eight to ten. The saving is useful, but the cleaner PR queue has been more valuable to me.


Passing editorial notes to the writer

PUBLISH isn't just "not SKIP". The triage discussion also produces EDITORIAL_NOTES β€” a one-sentence brief for the writer on angle or focus.

VERDICT: PUBLISH
EDITORIAL_NOTES: Lead with the on-device inference implications for enterprise
  mobile deployments; HK tech teams are evaluating this stack right now.

These notes are stored in result.editorial_notes and injected at the top of the news writer's prompt:

if result.editorial_notes:
    prompt = f"[EDITORIAL NOTES]\n{result.editorial_notes}\n\n" + prompt

This means the three editors who discussed the story β€” and read the source β€” have already told the writer what angle to take before a single word is drafted. The writer doesn't have to guess what the audience wants. The editorial judgement from triage carries forward through the entire pipeline.


Tuning the net without code changes

The triage_scope string is the key tuning knob:

press:
  triage:
    scope: |
      Focus areas: AI, software development tools, cybersecurity, Hong Kong tech scene,
      enterprise software, open-source.
      Audience: HK Cantonese-speaking tech professionals.
    min_score: 2

The scope value is substituted into the editorial_director and audience_specialist role prompts at instantiation time. Changing the editorial focus β€” say, adding "fintech" as a new area, or narrowing scope to AI-only β€” is a config edit, not a code change. The min_score parameter (minimum PUBLISH votes out of 3 editors) lets you tune strictness: 1 is very permissive (any editor can greenlight), 3 requires unanimity.

For deeper tuning β€” changing the evaluation criteria rather than the scope β€” the role files editorial_director.md and audience_specialist.md are the right place. The news editor role wasn't changed at all; it was already evaluating source quality, which is exactly what triage needs from it.


The updated pipeline

stages:
  - news_triage             # ← new β€” discussion + SKIP/PUBLISH logic
  - discuss_news_analysis
  - news_writer
  - discuss_news_draft
  - news_editor
  - translate_cantonese
  - translate_zh_traditional
  - news_article_pr

The triage discussion wraps the full _stage_discuss() machinery β€” same presets, same homework rounds, same synthesis injection that every other discussion stage uses. The only new code is _stage_news_triage() (which calls it and parses the output), _parse_triage_verdict() (20 lines of regex), and two new role files.

I originally thought of triage as a cost-control feature. In practice it improved the output too: the stories that continue already have an audience-aware angle before the writer starts.


Related reading: Multilingual by Default