The Two-Model Split: Fast Debate, Slow Research

πŸ“ Computer

Part of the AI Software House series.

In short: Debate needs speed; research needs tools. homework_llm lets each discussion participant use a capable tool-calling model to research first, then switch to a faster model for the actual conversation.


The tension

I first ran every discussion participant on one model. That was simple, but the two parts of a discussion have very different needs. During the debate, agents mostly read one another's messages and respond. A fast model is good enough.

Homework is different. Before arguing about an architecture, an agent may need to inspect the current code, old design decisions, or a similar change from last month.

Without that research, the conversation sounds convincing but stays generic.

I could run a large tool-capable model for every message, but most of those calls would be paying for abilities the debate round does not use.


Splitting the two jobs

Each participant can have two models: one for homework, one for debate.

participants:
  - role: analyst
    persona_file: roles/analyst.md
    llm: "fast-model"           # debate rounds β€” pure reasoning
    homework_llm: "slow-model"  # homework round β€” has RAG tool access
Phase Model Tools
Homework (round 0) homework_llm βœ… search_codebase, search_memory, search_docs
Debate (rounds 1+) llm ❌ pure reasoning only

During homework, the participant searches the codebase, retrieves relevant memory, and reads documentation before forming its initial analysis. During debate, it reasons from what it learned β€” fast, without tool overhead.


What it looks like in practice

With a fast model (no tools) doing all phases, the analyst might write:

The webhook retry logic looks reasonable in principle. Without seeing the current notification service implementation I can't say whether the existing queue mechanism would need to be extended or replaced.

With homework_llm enabled, the same analyst first searches the codebase:

[Tool: search_codebase("notification queue retry")]
β†’ Found: services/notifications/queue.py β€” uses Redis LPUSH/BRPOP, max 3 retries, no DLQ
β†’ Found: services/notifications/DESIGN.md β€” "retry policy is intentionally simple for MVP"

[Tool: search_memory("webhook DLQ decision")]
β†’ Found: discussion_2024-11-03 β€” "we explicitly deferred DLQ to v2 due to Redis complexity"

Then writes:

The current notification queue (services/notifications/queue.py) uses Redis with a max-3-retry policy and no dead-letter queue. The prior design discussion explicitly deferred DLQ to v2. The webhook retry proposal would need to either extend this infrastructure or introduce a parallel queue β€” the second option risks split queue semantics.

The analysis is grounded. It references real files and real decisions. The debate that follows is about concrete tradeoffs, not hypotheticals.


When to use homework_llm

I use the split when knowing the existing system could change the answer:

Use case homework_llm?
Architecture debate on existing codebase βœ… Yes β€” search is essential
Code review debate βœ… Yes β€” need to see the actual code
Feature brainstorm on existing system βœ… Yes β€” context matters
Requirements debate (new project) ❌ No β€” no codebase to search
Greenfield spec discussion ❌ No β€” reasoning is sufficient

For a greenfield spec, the participants only have the requirement in front of them, so a separate research model usually adds cost without improving the conversation.


Fallback safety

If homework_llm is set but the model doesn't support tool calling (some Ollama models, for example), the homework call falls back to a plain LLM call automatically. The participant loses the tool access but the pipeline doesn't break.

That lets me change models without turning an unsupported tool call into a failed pipeline.


How it fits in the LLM pool

In a typical local setup with one fast machine and some slower machines:

# discussions/brainstorm.yaml
participants:
  - role: analyst
    persona_file: roles/analyst.md
    llm: "ollama/qwen3.6"          # fast local β€” debate rounds
    homework_llm: "opencode-go/qwen3.5-plus"  # capable remote β€” homework research

The capable model gets one research call per participant. The faster model handles the repeated back-and-forth. In practice this gives me grounded opening positions without making every round slow.


Implementation note

Under the hood, homework_llm creates a fresh BaseAgent instance with the specified model, then calls call_with_tools() with the RAG tool_registry. The registry exposes the same three tools available to the architect and engineers: search_codebase, search_memory, and search_docs.

The discussion transcript includes the homework output from each participant. Downstream agents receive both the homework analyses and the debate rounds, so they see the full reasoning chain β€” including what each participant found during research.


Related reading: Debate Before You Write: Pre-Spec Discussion

Related reading: The Round Table: Multi-Agent Discussion Stages