When Your AI Agents Write Broken Code: A Four-Layer Accuracy System

Code: github.com/wanleung/ai-dev-team

In short: The pipeline’s own AI agents opened a pull request with 7 critical bugs — code that couldn’t run at all. This post dissects exactly why it happened and describes the four-layer system we’ve built to prevent, detect, learn from, and proactively seed context for agent mistakes: context injection, validation gates, a feedback loop that teaches agents from their own failures, and a bootstrap layer that gives new repos day-zero protection.

The incident

The pipeline’s own agents submitted a recent PR to add a PR/marketing campaign feature. The engineer agent wrote the code, the PM agent wrote the spec, and the PR was opened for human review.

I found 7 critical bugs. Not style issues. Not performance concerns. Bugs that meant the code couldn’t run at all:

API hallucination — All three new agents called self.llm.generate(system_prompt=..., user_prompt=...). That method doesn’t exist. BaseAgent exposes self.call(user_message). The agent invented a plausible-sounding API.
Config file wiped — repos.yaml configures every pipeline watcher in the system. The agent rewrote it from scratch, deleting all existing watchers, using a completely wrong schema.
Wrong YAML format — The pipeline definition used rich dict-style stage entries. The orchestrator expects plain strings. ["pr_analyst"] not [{"name": "pr_analyst", "agent": "...", "description": "..."}].
Missing registry wiring — New pipeline stages must be registered in _make_stage_registry(). The agent wrote the stage methods but never registered them. The pipeline would have raised ConfigurationError: Unknown stage 'pr_analyst' on first run.
Fragile relative path — open("roles/pr_creative.md") opens relative to the current working directory, not the file’s location. Works when run from the repo root. Fails everywhere else.
Redundant BaseAgent reimplementation — All three agents re-implemented _load_system_prompt(), a method BaseAgent already provides. The agent didn’t know it was already there.
Wrong class instantiation — GitHubClient() called with no arguments, but it requires repo and token. The agent didn’t know the constructor signature.

The PR was well-intentioned. The structure was right, the logic was coherent, the docs were good. But none of it could run.

Why it happened

Blaming the model isn’t useful. These bugs have structural causes — and structural causes have structural fixes.

The agent never read the code it was extending

When the engineer agent was asked to subclass BaseAgent, it never fetched base_agent.py. It wrote the subclass from memory — and its memory of “how LLM wrappers work” produced self.llm.generate(), a pattern that exists in dozens of open-source libraries but not in this one.

Same for repos.yaml. The agent generated the config from scratch based on what it thought the format should be, without reading the existing file.

RAG was bypassed for the new agents

The system has a RAG layer. The EngineerAgent, ArchitectAgent, and QAEngineerAgent all receive a tool_registry that lets them query the indexed codebase. But the new PR campaign agents were created by the engineer agent, which built the wiring without passing tool_registry to the new stage methods. The new agents couldn’t query anything.

Role files describe intent, not implementation patterns

The engineer.md role file says “write clean, idiomatic code.” It says nothing about: - “The correct way to call the LLM is self.call(user_message), not self.llm.generate()” - “Do NOT modify repos.yaml unless you have read its current contents first” - “New pipeline stages must be registered in _make_stage_registry() — see existing entries for the pattern”

The agent had no in-context API reference. It filled the gap with plausible guesses.

No gate between “code written” and “PR opened”

The pipeline went: engineer writes code → PR opened. No validation step. A python -m py_compile check would have caught the self.llm.generate() error in under a second.

The four-layer solution

The bugs divide cleanly into three categories: things that shouldn’t have been written (prevention), things that should have been caught before shipping (detection), and patterns that should be remembered to avoid next time (learning). A fourth layer — bootstrap — ensures new repos don’t have to wait for failures to accumulate before getting protection. One layer per concern.

Issue arrives
     │
     ▼
┌─────────────────────────────────────────────────────┐
│ LAYER 1 — PREVENTION                                │
│  Role file cheatsheets: concrete API patterns       │
│  Relevant source files auto-attached to prompt      │
│  RAG query before writing (method signatures)       │
└────────────────────┬────────────────────────────────┘
                     │ Agent writes code/content
                     ▼
┌─────────────────────────────────────────────────────┐
│ LAYER 2 — DETECTION (validation_gate stage)         │
│  Code:    py_compile → ruff → pytest -x             │
│  Content: JSON schema → required field checks       │
│  On fail: re-prompt agent with error (max 2×)       │
│  Still fails: draft PR, label needs-human-fix       │
└────────────────────┬────────────────────────────────┘
                     │ Passes gate
                     ▼
┌─────────────────────────────────────────────────────┐
│ LAYER 3 — LEARNING (LearningAgent)                  │
│  Triggered by: gate failures + PR review rejects    │
│  Writes anti-patterns → role file + repo-patterns/  │
│  Builds institutional memory over time              │
└────────────────────┬────────────────────────────────┘
                     │ Patterns accumulate
                     ▼
┌─────────────────────────────────────────────────────┐
│ LAYER 4 — BOOTSTRAP (BootstrapPatternsAgent)        │
│  Triggered on new repo onboarding                   │
│  Reads: base classes, configs, existing conventions │
│  Writes: .github/copilot-instructions.md            │
│  Seeds Layer 1 protection on day zero               │
└─────────────────────────────────────────────────────┘

Each layer ships independently. Layer 1 alone would have prevented 5 of the 7 bugs. Layer 2 would have caught the remaining 2. Layer 3 ensures this specific class of failure never recurs. Layer 4 gives newly onboarded repos the same prevention context from the start.

Layer 1: Prevention

Prevention means reducing what the agent has to guess. The less it has to infer, the less it can get wrong.

Cheatsheet sections in role files

The role file for engineer.md gets a new ## Codebase Patterns section:

## Codebase Patterns

### Calling the LLM
ALWAYS use `self.call(user_message: str) -> str`.
NEVER use `self.llm.generate()` — this method does not exist.

### Subclassing BaseAgent
Required class attributes: `role_name = "your_role_name"`
BaseAgent.__init__ loads `roles/{role_name}.md` automatically.
Do NOT implement `_load_system_prompt()` — it is already provided.

### Adding a new pipeline stage
1. Add a `_stage_yourname(self, result: PipelineResult)` method to Orchestrator
2. Register it in `_make_stage_registry()` — follow existing entries exactly
3. Stage names in pipelines/*.yaml must match registry keys exactly (plain strings)

### Modifying configuration files
NEVER rewrite repos.yaml or config files from scratch.
Always read the current file first. Add only the new entry.

This costs nothing. It’s just text. But it puts the correct API directly in front of the agent when it’s writing code.

Auto-attaching relevant source files

When the orchestrator dispatches an engineer task, it now inspects the task description and attaches related source files. Extending BaseAgent? Receive the first 100 lines of base_agent.py. Modifying repos.yaml? Receive the current file contents as context.

def _build_engineer_context(self, task: str) -> str:
    """Attach relevant source files based on task content."""
    context_parts = []

    if "BaseAgent" in task or "agent" in task.lower():
        context_parts.append(self._read_signature_block("agents/base_agent.py"))

    if "repos.yaml" in task or "watcher" in task.lower():
        context_parts.append(f"## Current repos.yaml\n```yaml\n{Path('repos.yaml').read_text()}\n```")

    if "pipeline" in task.lower() or "_make_stage_registry" in task:
        context_parts.append(self._read_signature_block("orchestrator.py", focus="_make_stage_registry"))

    return "\n\n".join(context_parts)

The agent can’t guess the wrong API if it’s looking at the real one.

RAG grounding for all new agents

The three new PR campaign agents now receive tool_registry=rag_registry in their orchestrator stage methods:

def _stage_pr_analyst(self, result: PipelineResult) -> None:
    from agents.pr_analyst import PRAnalystAgent
    agent = PRAnalystAgent(
        model=self.model,
        github_token=self._github_token,
        ollama_url=self.ollama_url,
        tool_registry=self._rag_registry,   # was missing
    )

This lets the agent query the codebase before writing: “What methods does BaseAgent expose?” becomes a RAG query rather than a hallucination.

Layer 2: Detection

Prevention reduces errors. Detection catches what slips through.

A new validation_gate stage runs after any code-writing stage and before PR creation. For code pipelines:

def _stage_validation_gate(self, result: PipelineResult) -> None:
    """Validate generated files before opening a PR."""
    errors = []

    for path, content in result.all_files.items():
        if path.endswith(".py"):
            errors.extend(self._check_syntax(path, content))
            errors.extend(self._check_lint(path, content))

    if errors and result.validation_attempts < 2:
        # Re-prompt the generating agent with the errors
        result.validation_attempts += 1
        result.validation_errors = errors
        self._re_prompt_engineer(result, errors)
        return

    if errors:
        # Still failing after 2 retries — open a draft PR with a flag
        result.pr_draft = True
        result.add_label("needs-human-fix")
        result.add_error(f"Validation failed after 2 retries: {errors}")

The re-prompt is targeted — it shows the agent exactly what broke:

def _re_prompt_engineer(self, result: PipelineResult, errors: list[str]) -> None:
    error_report = "\n".join(f"- {e}" for e in errors)
    revision_task = f"""
Your previous code has validation errors:

{error_report}

Here is the code that failed:
{self._format_files(result.all_files)}

Fix only the errors listed above. Do not change anything else.
"""
    # Re-run the engineer stage with the error context
    result.requirement = revision_task
    self._stage_engineer(result)

The key design decision: the agent gets to try again with the actual error message, not a vague “something went wrong.” This is how a junior developer learns from a compiler.

For content pipelines (JSON output agents like pr_analyst), the gate validates schema:

def _check_content_output(self, stage_key: str, result: PipelineResult) -> list[str]:
    output = getattr(result, f"{stage_key}_output", None)
    schema = STAGE_OUTPUT_SCHEMAS.get(stage_key)
    if schema and output:
        return validate_schema(output, schema)
    return []

Layer 3: Learning

Detection makes the system reliable. Learning makes it smarter over time.

The LearningAgent triggers in two situations: when the validation gate catches an error, and when a human marks a PR with needs-revision or rejects it with review comments. It receives the error, reads the relevant role file and memory-bank pattern docs, and writes targeted updates.

class LearningAgent(BaseAgent):
    role_name = "learning_agent"

    def run(self, failure: FailureRecord) -> None:
        """Update role files and memory bank from a failure event."""
        # Read the role file that generated the bad code
        role_path = Path("roles") / f"{failure.agent_role}.md"
        current_role = role_path.read_text()

        # Ask the LLM to derive the anti-pattern
        anti_pattern = self.call(
            f"An agent with this role file:\n{current_role}\n\n"
            f"Produced this error:\n{failure.error}\n\n"
            f"The correct code was:\n{failure.fix}\n\n"
            f"Write a concise 'DO NOT' rule to add to the role file's "
            f"## Anti-patterns section to prevent this exact mistake."
        )

        # Append to role file
        self._append_antipattern(role_path, anti_pattern)

        # Update memory bank
        self._update_system_patterns(failure, anti_pattern)

After a recent batch of agent failures, the learning agent would have appended to engineer.md:

## Anti-patterns (learned from failures)

- DO NOT call `self.llm.generate()` — use `self.call(user_message)` instead.
  BaseAgent does not expose a `.llm` attribute directly. (2026-05-17)

- DO NOT rewrite configuration files from scratch. Read the existing file
  first, then add only the new entry. (2026-05-17)

- DO NOT instantiate GitHubClient without arguments. Always pass `repo` and
  `github_token`, or receive an existing client as a parameter. (2026-05-17)

The next engineer agent that runs will have these rules in its system prompt. The failure is remembered even after the human fix is merged and forgotten.

Layer 4: Bootstrap

Layers 1–3 protect repos where agents are already running. But what about a repo that’s just been onboarded? It has no accumulated failures, no anti-patterns in its role files, no institutional memory. Layer 1 is empty on day one.

The BootstrapPatternsAgent solves this. When a new repository is added to the pipeline, it reads a set of candidate source files — base classes, config schemas, existing conventions — and generates a .github/copilot-instructions.md that seeds the repo with team-wide patterns and role cheatsheets before any agent has written a single line.

class BootstrapPatternsAgent(BaseAgent):
    role_name = "bootstrap_patterns_agent"

    CANDIDATE_FILES = [
        "agents/base_agent.py",
        "repos.yaml",
        "orchestrator.py",
    ]

    def run(self, target_gh: GitHubClient, commit: bool = True) -> str:
        """Read key source files and generate a copilot-instructions.md."""
        context = "\n\n".join(
            f"## {f}\n```python\n{content}\n```"
            for f in self.CANDIDATE_FILES
            if (content := target_gh.get_file_content(f)) is not None
        )
        instructions = self.call(context)
        if commit:
            branch = target_gh.get_default_branch()
            target_gh.commit_file(
                ".github/copilot-instructions.md", instructions,
                "chore: seed copilot instructions from repo patterns",
                branch=branch,
            )
        return instructions

The output is a fully populated Layer 1 cheatsheet derived from the actual codebase — not a generic template. A newly onboarded repo gets the same prevention context that an established repo took months of failures to accumulate. Zero-day protection, not zero-day vulnerability.

What each layer would have caught

Bug Layer 1 Layer 2 Layer 3 Layer 4 self.llm.generate() doesn’t exist Cheatsheet shows self.call() py_compile fails Anti-pattern written to role file Bootstrap seeds cheatsheet from base_agent.py repos.yaml wiped Auto-attach current file — Anti-pattern: read before writing Bootstrap seeds config pattern from repos.yaml Wrong YAML format Cheatsheet shows string stages — Anti-pattern written to role file Bootstrap seeds pipeline format from orchestrator.py Missing registry entries Cheatsheet shows pattern — Anti-pattern written to role file Bootstrap seeds registry pattern from orchestrator.py Fragile relative path — ruff or runtime error on retry Anti-pattern written — Redundant _load_system_prompt Cheatsheet: “do not implement” — Anti-pattern written Bootstrap seeds BaseAgent API from base_agent.py GitHubClient() no args Cheatsheet: constructor signature py_compile-adjacent check Anti-pattern written Bootstrap seeds constructor signature

Layer 1 alone handles 4 bugs. Layer 2 catches 2 more. Layer 3 ensures none of them recur. Layer 4 means a new repo gets Layer 1 populated before any agent has run.

The bigger picture

The root cause of all 7 bugs is the same: the agent was asked to extend a system it couldn’t see. It filled in the blanks with plausible guesses based on patterns it had learned from other codebases. Most of those guesses were reasonable. They just weren’t right for this one.

The four layers address this at different points in the failure mode: - Prevention reduces what the agent has to guess - Detection catches guesses that were wrong before they ship - Learning closes the loop so each failure makes the system slightly harder to fool the same way again - Bootstrap ensures new repos start with prevention context already in place

This isn’t a perfect system. Agents will still make new kinds of mistakes. But it’s a system that degrades gracefully and improves continuously — which is a much better property than “sometimes right, no feedback loop.”

The system shipped across four pull requests, one milestone at a time.

Code: github.com/wanleung/ai-dev-team