Multi-Agent LLM Engineering System: Autonomous Software Development ## 1. Scope and Assumptions - **Models:** Claude (planning, code review, reasoning-heavy tasks) and GPT (implementation, alternative generation, cross-model review). Routing is config-driven, not hardcoded. - **Language:** Python for the orchestrator and tooling. Generated code may target Python or JavaScript/React, but the first release should be scoped to Python. - **Source control:** GitHub (branches, PRs, CI checks). - **Testing:** pytest, ruff/mypy for static checks, coverage. - **Goal:** Given a spec or issue, produce a tested, reviewed PR, with human approval before merge. The most important design principle: **agents propose, deterministic tools verify.** An LLM saying "the tests pass" is not evidence. A pytest exit code is. --- ## 2. System Architecture ``` ┌──────────────────────────┐ │ Human / GitHub Issue │ └─────────────┬────────────┘ ▼ ┌──────────────────────────┐ │ ORCHESTRATOR │ │ (state machine + DAG) │ └──┬───────┬───────┬───────┘ ▼ ▼ ▼ ┌────────────┐ ┌──────────┐ ┌──────────────┐ │ Planner │ │Researcher│ │ Architect │ └────────────┘ └──────────┘ └──────────────┘ ▼ ┌──────────────────────────────────────┐ │ Implementer pool (parallel, per- │ │ file or per-module, isolated │ │ git worktrees) │ └──────────────────┬───────────────────┘ ▼ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ Tester │ │ Debugger │ │ Reviewer │ │ QA │ └────────────┘ └────────────┘ └────────────┘ └────────────┘ ▼ ┌──────────────────────────────────────┐ │ Deterministic Gates (pytest, ruff, │ │ mypy, coverage, secret scan, SAST) │ └──────────────────┬───────────────────┘ ▼ ┌──────────────────────────────────────┐ │ Delivery: branch → PR → CI → human │ └──────────────────────────────────────┘ Shared infrastructure: Artifact Store (specs, diffs, test logs), Message Bus (typed JSON events), Model Gateway (routing, budgets, caching), Tool Sandbox (Docker, no network by default). ``` ### Core components | Component | Responsibility | |---|---| | Orchestrator | Holds the task graph, dispatches agents, enforces budgets and retry limits, owns final state | | Model Gateway | Single interface to Claude and GPT; handles retries, rate limits, token accounting, response caching | | Tool Sandbox | Runs shell, pytest, git inside a container with a read-write worktree and no outbound network | | Artifact Store | Versioned storage for every spec, plan, diff, and log. Agents pass artifact IDs, not raw text | | Gate Runner | Runs deterministic checks and returns structured pass/fail. Not an LLM | | State Store | SQLite or Postgres. Persists task state so runs can resume after crashes | --- ## 3. Agent Role Definitions Each agent has a contract: inputs, outputs, tools, and success criteria. | Agent | Default model | Inputs | Outputs | Tools | Success criterion | |---|---|---|---|---|---| | **Planner** | Claude (Opus/Sonnet tier) | Spec, repo summary | `Plan` JSON: ordered subtasks with file targets, dependencies, acceptance criteria | Read-only repo search | Every acceptance criterion maps to at least one subtask; no cycles in dependencies | | **Researcher** | Claude with web search | Open question from planner | `Finding` JSON: answer, source URLs, confidence | Web search, docs fetch | Each claim cites a source; unsourced claims are marked as assumptions | | **Architect** | Claude | Plan, existing code | Interface contracts (function signatures, data schemas), module boundaries | Read-only repo | Contracts are machine-checkable (type stubs or JSON Schema) | | **Implementer** | GPT (primary), Claude (fallback) | One subtask, its contract, relevant files | Unified diff restricted to declared file targets | Read/write in worktree, lint | Diff applies cleanly; touches only allowed paths; lint passes | | **Tester** | GPT or Claude | Contract, acceptance criteria | Test files | Read/write tests only | Tests fail on the unimplemented stub and are not trivially true (no `assert True`) | | **Debugger** | Claude | Failing test output, diff, stack trace | Minimal patch | Read/write worktree, run tests | Target test passes without breaking others | | **Reviewer** | Cross-model (if Implementer is GPT, Reviewer is Claude, and vice versa) | Diff, contract, plan | `Review` JSON: issues with severity, file, line | Read-only | Every issue is reproducible or cites a line; no style-only blockers | | **QA** | Claude | Full diff, test results, gate output | Go/no-go with rationale | Read-only, gate results | Verifies acceptance criteria are covered by passing tests | | **Security** | Deterministic first (Semgrep, Bandit, gitleaks), then LLM triage | Diff | Findings with triage | SAST tools | No high-severity findings unacknowledged | **Why cross-model review:** models share blind spots when they're the same model reviewing their own output. Alternating GPT and Claude for implementation and review catches more defects than self-review. Measure this rather than assuming it (see Section 9). --- ## 4. Orchestrator Design The orchestrator is a **state machine over a task DAG**, not a free-form chat between agents. This is the single most important reliability decision. Free agent-to-agent conversation drifts, loops, and burns budget. ### Task states ``` PENDING → READY → RUNNING → VERIFYING → DONE │ │ ▼ ▼ FAILED ──→ RETRY (≤ N) ──→ ESCALATED (human) ``` ### Pseudocode ```python def run_pipeline(spec: Spec, budget: Budget) -> RunResult: run = StateStore.create_run(spec, budget) plan = call_agent("planner", spec) # validated against Plan schema validate_plan(plan) # DAG check, path check, criteria coverage graph = TaskGraph.from_plan(plan) while not graph.all_done(): ready = graph.ready_tasks() # dependencies satisfied batch = prioritize(ready, budget)[:MAX_PARALLEL] results = parallel_map(execute_task, batch) # isolated worktrees for task, result in zip(batch, results): graph.mark(task, result.status) if result.status == "failed": if task.retries < MAX_RETRIES and budget.remaining() > 0: task.retries += 1 graph.requeue(task, feedback=result.error) else: escalate(task, result) graph.mark_blocked_dependents(task) budget.check() # hard stop on overspend StateStore.checkpoint(run, graph) integrated = merge_worktrees(graph.completed_diffs()) # conflict = escalate, not auto-resolve gates = run_gates(integrated) # deterministic qa = call_agent("qa", integrated, gates) return deliver(integrated, gates, qa) if qa.go else escalate_run(qa) def execute_task(task): diff = call_agent("implementer", task.contract, task.files) gate = apply_and_test(diff, task) # real pytest run if gate.passed: review = call_agent("reviewer", diff, task.contract) # cross-model if review.blocking_issues: return TaskResult("failed", feedback=review.blocking_issues) return TaskResult("done", diff=diff) debug = call_agent("debugger", diff, gate.log) return TaskResult("failed", feedback=gate.log) if not debug else retest(debug) ``` ### Key orchestrator rules 1. **Plan validation before execution.** Reject plans that reference paths outside the repo, have circular dependencies, or leave acceptance criteria unmapped. 2. **Parallelism only across disjoint file sets.** Two tasks touching the same file run sequentially. Enforce this via a file-lock table, not by trusting the planner. 3. **Retries carry the failure reason.** A retry without the previous error is wasted spend. 4. **Escalate rather than loop.** After `MAX_RETRIES` (suggested: 2 for implementation, 3 for debugging), stop and hand off to a human with the full artifact trail. 5. **Checkpoint after every task.** A crash should resume at the last completed node, not restart the run. --- ## 5. Communication and Data Contracts ### Rules - **Typed JSON only** between agents. Every message is validated against a JSON Schema before it's accepted. Malformed output triggers one repair attempt (re-prompt with the validation error), then fails the task. - **Artifact references, not payloads.** Agents receive `artifact_id` and fetch what they need. This keeps context windows small and makes every decision auditable. - **Context is scoped per agent.** The Implementer sees its subtask, contract, and listed files. It never sees the full plan. This limits drift and cost. - **No agent-to-agent chat.** All communication goes through the orchestrator. This makes the run replayable. ### Example message schemas **Plan (planner → orchestrator)** ```json { "run_id": "r-2026-1028-001", "subtasks": [ { "id": "T1", "goal": "Add rate limiter middleware", "files": ["app/middleware/rate_limit.py"], "depends_on": [], "acceptance": ["Requests over 100/min return HTTP 429"], "contract_ref": "contracts/rate_limit.pyi" } ] } ``` **Diff result (implementer → orchestrator)** ```json { "task_id": "T1", "diff_artifact": "a-7f3c", "files_touched": ["app/middleware/rate_limit.py"], "self_reported_status": "complete", "assumptions": ["Using in-memory store; Redis out of scope"] } ``` **Review (reviewer → orchestrator)** ```json { "task_id": "T1", "blocking_issues": [ {"severity": "high", "file": "app/middleware/rate_limit.py", "line": 42, "issue": "Race condition: counter read and increment are not atomic", "reproducible_by": "tests/test_rate_limit.py::test_concurrent"} ], "non_blocking": [] } ``` Note `assumptions` in the diff schema. Surfacing assumptions explicitly is one of the cheapest ways to catch silent misinterpretation. --- ## 6. Conflict Resolution and Validation | Situation | Resolution | |---|---| | Two agents disagree on an interface | Architect's contract wins. Architect is the only agent that can change contracts; changes require a new plan version | | Implementer and Reviewer disagree | Reproducible failing test beats opinion. If no test exists, the Tester writes one; if the test passes, the review issue is downgraded | | Two diffs modify the same file | Sequential merge with 3-way conflict detection. Conflicts escalate to the Debugger with both versions, never auto-resolved silently | | Model output contradicts gate result | Gate result wins, always | | Agent claims completion without evidence | Orchestrator treats it as incomplete. "Self-reported complete" is not a status | | Researcher finds conflicting sources | Report both with confidence; the Planner decides, and the decision is logged | **Validating assumptions:** every agent lists its assumptions in its output. The orchestrator checks each assumption against the spec. Any assumption not traceable to the spec or repo is flagged for human review. --- ## 7. Workflow: Step by Step 1. **Intake.** Parse the GitHub issue or spec. Reject if acceptance criteria are missing; ask the human for them. Do not invent requirements. 2. **Analysis.** Planner reads the spec and a repo summary (generated by a deterministic script: file tree, dependency graph, existing test coverage). Output: Plan JSON. 3. **Plan validation.** Deterministic checks: schema, DAG, path allowlist, criteria coverage. Failure → re-plan once, then escalate. 4. **Decomposition and contracts.** Architect produces type stubs for new interfaces. Stubs are checked by `mypy`. 5. **Test-first scaffolding.** Tester writes tests against the stubs. Run them; they must fail for the right reason (`NotImplementedError`, not `ImportError`). 6. **Parallel implementation.** Implementers work in isolated git worktrees, one per task. The file-lock table prevents overlapping writes. 7. **Per-task verification.** Run the task's tests, lint, and type-check. Failures go to Debugger (max 3 cycles). 8. **Cross-model review.** Reviewer examines each diff against its contract. Blocking issues return to the Implementer with the review attached. 9. **Integration.** Merge worktrees into an integration branch. Conflicts escalate. 10. **Full gate run.** Entire test suite, coverage threshold, `ruff`, `mypy`, `bandit`, `gitleaks`, and a dependency audit (`pip-audit`). 11. **Iterative refinement.** If the full suite fails, the orchestrator identifies the failing module and reruns steps 6–10 for that module only. 12. **QA.** QA agent checks each acceptance criterion against a specific passing test. Output is a criteria-to-test matrix. 13. **Delivery.** Push branch, open PR with: plan, criteria matrix, gate results, assumptions, cost summary. CI runs independently. A human merges. ### Human checkpoints (non-negotiable) - After plan validation, if the plan touches more than N files or any path matching `auth|payment|migrations|infra`. - Before merge. The system never merges to main. - On any escalation. --- ## 8. Error Handling and Recovery | Failure | Detection | Recovery | |---|---|---| | Malformed JSON from model | Schema validation | One repair re-prompt with error text, then fail task | | Model API timeout or 429 | Gateway | Exponential backoff with jitter; failover to the other provider for the same role | | Tests hang | Per-test timeout (e.g. 60s) | Kill sandbox, mark failed, include timeout in feedback | | Infinite agent loop | Step counter per task | Hard cap on tool calls (e.g. 25) and on tokens per task | | Diff doesn't apply | `git apply --check` | Regenerate once with the patch error included | | Orchestrator crash | Checkpoint store | Resume from last checkpoint; idempotent task execution keyed by `(run_id, task_id, attempt)` | | Hallucinated import or API | Lint / import check / test failure | Fed back as a gate failure; never trusted from model text | | Hallucinated library | `pip install --dry-run` or lockfile check | Reject dependencies not in an allowlist without human approval | **Idempotency matters.** Every side-effecting action (branch creation, PR open, comment post) is keyed so a retry doesn't create duplicates. --- ## 9. Security - **Sandbox every execution.** Generated code runs in a Docker container with no network, a non-root user, CPU/memory/time limits, and only the worktree mounted. - **Secrets never enter prompts.** The gateway redacts environment variables matching key patterns. Implementers get no access to `.env` files; add them to the path denylist. - **Path allowlist per task.** The orchestrator rejects any diff touching files outside `task.files`. - **Dependency allowlist.** New packages require human approval. Prevents typosquatting and supply-chain injection via model suggestions. - **Prompt injection from repo content.** Issues, comments, and READMEs are untrusted input. Wrap them in clearly delimited data blocks, and do not let them change agent roles or tool permissions. The Researcher's web content is treated the same way. - **Scoped GitHub credentials.** A fine-grained token with `contents: write` on feature branches only, and `pull-requests: write`. No admin, no branch protection bypass, no merge rights. - **Audit log.** Every model call, tool call, and diff is stored with timestamps and run IDs. --- ## 10. Performance and Cost Control ### Cost levers, roughly in order of impact 1. **Model tiering.** Use the strongest model only for planning and final QA. Implementation and test writing can use mid-tier models. Debugging escalates tier only after one failure. 2. **Context minimization.** Scoped inputs (Section 5) typically cut tokens per call substantially versus sending the whole repo. Measure this on your own repos; the ratio depends heavily on codebase size. 3. **Prompt caching.** Cache the stable prefix per agent (system prompt, role definition, repo summary). Anthropic supports explicit prompt caching; check your provider's current pricing and cache behavior before budgeting around it. 4. **Response caching.** Key on hash of (model, prompt, temperature=0). Useful for deterministic re-runs and retries of identical prompts; less useful for implementation, where outputs vary. 5. **Early termination.** Stop a task when its gate passes. Don't run the full review pipeline on a diff that fails lint. 6. **Budget ceilings.** Per-run token and dollar caps, enforced by the gateway. The orchestrator halts with a partial result and report when a cap is hit. ### Latency levers - Parallelize implementers across disjoint file sets. - Run cheap deterministic checks (lint, import check, `mypy` on changed files) before any LLM review. - Use incremental test runs (`pytest --lf` or changed-module selection) during the loop; full suite only at integration. --- ## 11. Modularity and Extensibility Structure the codebase so that each agent is a plugin: ``` engine/ orchestrator/ # state machine, DAG, scheduler (no LLM calls) gateway/ # provider adapters: claude.py, openai.py; routing; budgets agents/ base.py # Agent interface: role, schema_in, schema_out, tools, run() planner.py implementer.py ... tools/ sandbox.py # Docker exec git_ops.py # worktrees, apply, merge gates.py # pytest, ruff, mypy, bandit, gitleaks schemas/ # JSON Schemas for every message type store/ # artifacts, state, audit log config/ routing.yaml # role → model, fallback, tier budgets.yaml paths.yaml # allowlists, denylists evals/ # benchmark tasks and scorers (Section 12) ``` **Adding a new agent** means: write a class implementing `Agent`, add its schemas, register its role in `routing.yaml`. No orchestrator changes. **Swapping models** means: change one line in `routing.yaml`. The gateway normalizes provider differences (tool-call formats, system prompt handling). **Adding a language** means: a new gate runner and a new implementer prompt. The orchestrator is language-agnostic. --- ## 12. Failure Points and Mitigations | Failure point | Why it happens | Mitigation | |---|---|---| | Plan misses a requirement | Spec is ambiguous | Criteria-to-subtask coverage check; ask human on gaps | | Tests pass but test nothing | Model writes weak tests | Mutation testing (`mutmut`) on changed code; reject if survivors exceed threshold | | Implementer "fixes" tests instead of code | Easiest path to green | Tests are locked after the Tester phase; Implementer cannot modify test files. Enforce by path rule | | Subtle concurrency bugs | Models underweight races | Reviewer prompt includes a concurrency checklist; add targeted stress tests for shared-state code | | Dependency drift between parallel tasks | Two agents assume different helper signatures | Contracts from Architect are the only shared interface; integration gate catches mismatches | | Cost spiral | Retry loops | Hard caps per task and per run | | Reviewer rubber-stamps | Sycophancy toward code that looks clean | Reviewer must cite line + reproducer for blocking issues; evaluate reviewer on seeded-bug benchmarks | | Stale repo context | Repo changes during run | Pin a base commit at run start; all worktrees branch from it | **Locking tests after the Tester phase** is one of the highest-value rules here. Without it, the most common failure is the system quietly weakening tests to pass. --- ## 13. Evaluation Criteria Measure the system, not just individual agents. Build an eval suite of 30–100 tasks drawn from real issues in past repos, each with a known-good solution and a hidden test set. | Metric | Definition | Target (starting point) | |---|---|---| | **Task success rate** | Hidden tests pass on the delivered PR | Establish baseline first; then improve | | **First-pass gate rate** | Tasks passing all gates without retry | Track as efficiency signal | | **Escalation rate** | Runs requiring human intervention | Lower is better, but a rate near zero suggests the escalation rules are too loose | | **False-complete rate** | Runs marked done that fail hidden tests | Should be near zero; this is the most dangerous failure | | **Reviewer recall** | Fraction of seeded bugs the reviewer catches | Measure per model pairing | | **Test strength** | Mutation score of generated tests | Compare against human-written baseline | | **Cost per successful task** | Total spend ÷ successful deliveries | Track trend, not absolute | | **Wall-clock time** | Intake to PR | Track p50 and p90 | | **Cross-model lift** | Success rate with alternating models vs. same-model review | Run an A/B to justify the design | **Evaluate each agent in isolation too:** planner coverage score, implementer diff-apply rate, reviewer precision and recall on seeded defects, debugger fix rate on a fixed set of failing tests. **Keep the eval suite frozen.** If you tune prompts against it and it changes simultaneously, you lose the ability to tell improvements from overfitting. Keep a held-out split that's never used for tuning. --- ## 14. Practical Recommendations - **Start with a single-task pipeline** (Planner → Implementer → Tester → Reviewer → Gates) on one repo before adding parallelism. Most reliability problems appear at the boundaries; fix them with one agent path first. - **Make the gate runner the source of truth** from day one. Everything else is advisory. - **Log everything from the start.** Retrofitting an audit trail is much harder than building it in. - **Pick one language and one test framework** for v1. Pytest only. - **Don't add agents you can't evaluate.** Each role should have a metric. If you can't measure whether the Security agent helps, it's a deterministic tool, not an LLM agent. - **Expect the Implementer to be the cheapest role and the Planner to be the most consequential.** Spend your prompt-engineering effort on the planner and reviewer. --- ## 15. Actionable Next Steps 1. **Week 1:** Build the Model Gateway with Claude and GPT adapters, schema validation, retry logic, and token accounting. Test with mock responses. 2. **Week 1–2:** Implement the Gate Runner (pytest, ruff, mypy) and the Sandbox. Verify it can run a known-good and known-bad diff correctly without any LLM involved. 3. **Week 2:** Build the single-task pipeline: Planner, Implementer, Tester, Reviewer, with the state store and checkpointing. 4. **Week 3:** Assemble an eval set of 30 tasks from past issues. Establish baseline metrics on the single-task pipeline. 5. **Week 3–4:** Add the cross-model reviewer and measure the lift. Keep it only if it improves false-complete rate or reviewer recall. 6. **Week 4+:** Add parallel execution with the file-lock table, then the Debugger loop, then GitHub PR integration. Add each only after the previous layer meets its metric target. 7. **Ongoing:** Review escalations weekly. Each escalation is either a missing rule, a prompt gap, or a genuinely hard task. Classify them, and fix the cause. --- **Summary:** The architecture is a deterministic orchestrator running a task DAG, with typed-message LLM agents for planning, implementation, testing, debugging, and review. Real tools (pytest, linters, sandboxed execution) act as the verification layer, and humans gate the merge. Build the single-task path first, measure it against a frozen eval set, and add parallelism and cross-model review only where the measurements justify them.