Artifex/docs/graph_runtime_audit.md

# Graph Runtime Audit

Date: 2026-08-15

Baseline inspected: `6a73372665715950aee8416a9d2b3ec79955f816`

Scope: audit current Artifex implementation against the intended graph-runtime architecture. This document does not propose or include a refactor.

## Executive Summary

`GraphRuntime` is currently an interface boundary, not the production execution engine.

`LangGraphRuntime` is a deterministic placeholder adapter. It is not executing project, milestone, feature, task, Coder, Reviewer, Judge, retry, or commit workflows.

Real production task execution is driven by `runtime_loop.autonomous_task_loop.AutonomousTaskLoop`, `graph.scheduler.TaskScheduler`, Django ORM state, direct Python service calls, and persisted Event Bus events.

The current implementation has two separate concepts that are easy to conflate:

- Project DAG: persisted project work decomposition and task dependencies.
- Execution Graph: the operational workflow for executing one task attempt through Coder, tools, tests, Reviewer, Judge, retry, and commit.

The Project DAG exists today in Django models and scheduler logic. The Execution Graph exists only as imperative Python control flow inside `AutonomousTaskLoop`; it is not represented as a graph runtime yet.

Recommendation: keep Django/Postgres as canonical state and migrate execution to graph semantics incrementally. Start by extracting an explicit `TaskExecutionGraph` description around the existing task loop without changing behavior, then add graph telemetry/checkpoint metadata. LangGraph should remain behind `GraphRuntime` and should provide value only when conditional routing, resumable execution, human gates, subgraphs, and agent-team workflows become difficult to manage in plain Python.

## Direct Answers

### 1. Is `GraphRuntime` used in production task execution?

No.

`graph/runtime.py` defines an abstract `GraphRuntime` with `start`, `pause`, `resume`, `cancel`, and `signal`, but no production task execution path depends on it.

Observed production path:

- `runtime_loop/autonomous_task_loop.py` imports `TaskScheduler`, `Coder`, `Reviewer`, `Judge`, `WorktreeTools`, and `DeterministicTestRunner`.
- It does not import or call `GraphRuntime` or `LangGraphRuntime`.

Current role of `GraphRuntime`: architectural boundary and future abstraction.

### 2. Is `LangGraphRuntime` actually executing task workflows?

No.

`graph/langgraph_runtime.py` returns a deterministic string from `start` and no-ops `pause`, `resume`, `cancel`, and `signal`.

It does not import LangGraph, build a state graph, execute graph nodes, route edges, persist graph checkpoints, or call Coder/Reviewer/Judge.

Its only observed test coverage is `tests/test_interfaces.py::test_langgraph_runtime_is_hidden_behind_runtime_interface`, which checks that `start()` returns a run id prefix.

## Real Execution Path

Current task execution path for a single task:

1. Scheduler

`AutonomousTaskLoop.run_once()` calls `TaskScheduler.claim_next_ready_task()`.

`TaskScheduler` queries `Task(status=READY)` ordered by priority and creation time, skips tasks with incomplete dependencies, marks the selected task `RUNNING`, persists it, and publishes `TASK_STARTED`.

2. Investigation/context building

`AutonomousTaskLoop._execute_task()` creates or reuses a worktree and builds context with `WorkerContextBuilder.build_for_task()`.

This context builder reads selected Python files and tests directly from the worktree, excludes migrations and secrets, and attaches the current diff.

Coder also has an inspect-before-edit phase inside `Coder.execute()` for sensitive tasks. It asks the model for read-only inspection operations, then executes `list_directory`, `read_file`, `search_code`, `find_symbol`, `git_status`, or `git_diff` through `WorktreeTools`.

3. Coder

`AutonomousTaskLoop` calls `Coder.execute(context, tools, project, agent_version)`.

`Coder.execute()` may perform inspection, then creates a `CoderToolLoop` and calls `CoderToolLoop.run()`.

4. Mutation

`CoderToolLoop` asks the model for mutation operations and applies them through `WorktreeTools`.

Supported mutation operations include:

- `write_file`
- `apply_patch`
- `delete_file`
- `move_file`
- `create_directory`
- `run_command`

Patch mismatch recovery is internal to `CoderToolLoop`: it records structured failure evidence, re-reads the live target file excerpt, and asks the model once more for a corrected patch. It does not default to full-file replacement.

5. Tests

`AutonomousTaskLoop` calls `DeterministicTestRunner.run()`.

`DeterministicTestRunner` shells out via `WorktreeTools.run()`, persists an `Artifact` with stdout/stderr/returncode, and creates a `TestRun` with `PASS` or `FAIL`.

On failure, `_attach_test_failure_evidence()` stores stdout/stderr excerpts into the attempt's `coder_result.metadata.test_failure_evidence`, so later retry context can include the actual traceback.

6. Reviewer

`AutonomousTaskLoop` stages intent-to-add with `git add -N`, gets `git diff`, then calls `Reviewer.review(task, reviewer_version, diff, test_status)`.

`Reviewer` is currently a deterministic Python policy. It creates a `Review` row with `PASS`, `REWORK_REQUIRED`, or `REJECTED` based on test status, empty diff, and health-route-specific heuristics.

7. Judge

If Reviewer passes, `AutonomousTaskLoop` calls `Judge.judge(project, task, judge_version, diff, test_status)`.

`Judge` is currently deterministic Python. It creates a `Verification` row with task-level acceptance evidence and `PASS` or `FAIL`.

8. Retry

If Coder, Reviewer, or Judge fails, `_retry_or_fail()` updates `TaskAttempt.status`, increments `Task.retry_count`, publishes `TASK_FAILED`, and either loops for another attempt or marks the task `FAILED` and publishes `TASK_RETRY_EXHAUSTED` plus final `TASK_FAILED`.

The effective retry budget is capped at two retries, giving three total attempts.

9. Commit

If tests, Reviewer, and Judge all pass, `WorktreeTools.commit_all()` stages and commits all changes. `CommitRecord` is persisted with coder, reviewer, judge, test run, review, verification, SHA, branch, and message.

The task is marked `COMPLETE`, `COMMIT_CREATED` and `TASK_COMPLETED` are published with telemetry, the worktree is validated clean, and the task worktree is cleaned up.

## Transition Classification

### Explicit graph nodes/edges

Currently only the Project DAG has explicit persisted edges:

- `TaskDependency(task, depends_on)` models task-to-task dependencies.
- `Milestone.dependencies` models milestone-level dependencies but is not currently central in execution scheduling.
- `ProjectPlanBuilder` validates task dependency acyclicity before materializing tasks.
- `TaskScheduler.claim_next_ready_task()` treats incomplete `TaskDependency` rows as blocking edges.

No explicit graph nodes/edges currently model the execution workflow inside a task attempt.

### Direct Python service calls

Most real execution transitions are direct service calls:

- `AutonomousTaskLoop.run_once()` -> `TaskScheduler.claim_next_ready_task()`
- `_execute_task()` -> `WorkerContextBuilder.build_for_task()`
- `_execute_task()` -> `Coder.execute()`
- `Coder.execute()` -> `CoderToolLoop.run()`
- `CoderToolLoop` -> `ModelRouter.complete()`
- `CoderToolLoop` -> `WorktreeTools` mutation/read/test helpers
- `_execute_task()` -> `DeterministicTestRunner.run()`
- `_execute_task()` -> `Reviewer.review()`
- `_execute_task()` -> `Judge.judge()`
- `_execute_task()` -> `WorktreeTools.commit_all()`
- `_retry_or_fail()` direct retry/fail transition

### Django state transitions

Django/Postgres is the canonical durable state machine today:

- `Task.status`: `READY`, `RUNNING`, `FAILED`, `COMPLETE`
- `Task.retry_count`
- `TaskAttempt.status`: `RUNNING`, `REWORK_REQUIRED`, `FAILED`, `COMPLETE`
- `Worktree.status`
- `TestRun.status`
- `Review.status`
- `Verification.result`
- `CommitRecord`
- `ModelRequest.status`
- `ProgenySignal.status`

### Event Bus driven transitions

The Event Bus persists facts but does not currently drive the control flow.

Events published today include:

- `PLAN_APPROVED`
- `MILESTONE_CREATED`
- `TASK_CREATED`
- `TASK_READY`
- `TASK_STARTED`
- `TEST_FAILED`
- `REVIEW_FAILED`
- `TASK_FAILED`
- `TASK_RETRY_EXHAUSTED` as a string event
- `COMMIT_CREATED`
- `TASK_COMPLETED`
- `AGENT_CREATED`
- `AGENT_PROMOTED`
- `PROGENY_SIGNAL_CREATED` as a string event from `ProgenyService`

These are audit/event facts, not asynchronous orchestration triggers in the current implementation.

## Project DAG vs Execution Graph

### Project DAG

The Project DAG answers: what work exists, and what depends on what?

Current implementation:

- `ProjectPlanContract` represents milestones, features, tasks, and task dependency references.
- `validate_project_plan()` validates task dependency references and cycle freedom.
- `ProjectPlanBuilder.apply()` persists `ProjectPlan`, `Milestone`, `Feature`, `Task`, and `TaskDependency` rows.
- `TaskScheduler` consumes `TaskDependency` rows to claim only unblocked `READY` tasks.

This is real and production-relevant today.

### Execution Graph

The Execution Graph answers: how is one task/feature/milestone autonomously executed?

Current implementation:

- This graph is implicit in `AutonomousTaskLoop._execute_task()`.
- Nodes are not persisted as graph nodes.
- Edges are Python `if`/`continue`/`return` branches.
- Looping is a Python `while task.retry_count <= effective_max_retries`.
- Checkpointing is coarse-grained through Django rows such as `TaskAttempt`, `TestRun`, `Review`, `Verification`, and `CommitRecord`.

The execution graph exists conceptually but not as a first-class runtime artifact.

## CoderToolLoop Placement

Recommended placement: graph node containing an internal tool loop.

Do not make `CoderToolLoop` the top-level execution graph yet.

Rationale:

- `CoderToolLoop` is a bounded inner loop for a single agent/tool interaction.
- Its state is local and tactical: inspection results, mutation results, patch mismatch recovery, operation telemetry.
- It should be resumable eventually, but it is not the same graph as the whole task lifecycle.
- Making every tool call a top-level graph node now would increase orchestration complexity before there is enough operational pressure.

Near-term target:

- `TaskExecutionGraph` has a `coder` node.
- The `coder` node invokes `Coder.execute()`.
- `Coder.execute()` owns `CoderToolLoop` internally.
- `CoderToolLoop` emits structured telemetry and failure evidence that the outer graph records.

Later target:

- If long-running coding sessions, human approvals, streamed model/tool execution, or partial resume become necessary, promote internal tool-loop steps to a nested subgraph.

## Runtime Capability Assessment

### Conditional edges

Required.

Current conditional branches include:

- no ready task -> stop
- dependencies incomplete -> skip task
- coder failed -> retry/fail
- tests failed -> still Review, then retry/fail through Review
- reviewer failed -> retry/fail
- judge failed -> retry/fail
- all checks passed -> commit

These map naturally to graph conditional edges.

### Loops

Required.

Current loops include:

- task retry loop capped at three total attempts
- CoderToolLoop patch recovery loop
- future investigation and experiment loops

The retry cap must remain durable and enforced by Django state, not hidden only inside graph runtime memory.

### Parallel branches

Useful, not urgent for current V1/V2 guarantees.

Good candidates later:

- independent task execution across the Project DAG
- parallel tests/static checks/security checks
- multi-agent review lanes
- Progeny benchmark replay across candidates

Parallelism should be introduced only after worktree isolation and resource contention controls are explicit.

### Subgraphs

Required long-term.

Natural subgraphs:

- Task execution
- Coder tool loop
- Reviewer/Judge verification
- Progeny investigation
- Progeny experiment/champion-challenger evaluation
- Project Brain planning/replanning

### Checkpoint/resume

Required, but Django/Postgres should remain canonical.

Current checkpointing already exists at coarse boundaries:

- task claimed
- attempt created
- coder result saved
- test run saved
- review saved
- verification saved
- commit saved

Graph checkpointing should augment these records, not replace them.

### Human approval nodes

Required for safe V2+ autonomy.

Likely approval gates:

- project plan approval
- task split/replan approval
- risky migration approval
- provider credential/resource approval
- promotion of Progeny candidates
- destructive infrastructure changes

Human approval should persist as Django state and be represented as paused graph nodes.

### Dynamic temporary agent teams

Required later, not required for the current loop.

Smart Investigation and Progeny experiments will likely need temporary teams such as:

- incident analyst
- domain expert
- prompt/policy editor
- benchmark designer
- reviewer challenger

The runtime should support this as graph metadata and persisted `AgentRun`/future team records.

### Graph versioning

Required before graph-based production execution.

Every accepted commit and Progeny investigation should know which execution graph version produced it. Without this, regressions in orchestration will be hard to diagnose.

### Graph telemetry

Required.

Existing telemetry should become graph-node telemetry:

- mutation operations
- patch success/mismatch rates
- write-file fallback rate
- model requests per task
- test duration/status
- review findings
- judge evidence
- retry count
- provider errors
- signal counts

This telemetry should feed Progeny.

### Progeny champion/challenger workflow graphs

Required for the future Replay Arena / experiment execution layer, but should not be the next refactor.

The Progeny workflow graph should model:

- investigation selection
- candidate generation
- benchmark construction
- champion/challenger replay
- judge/evaluator comparison
- approval/promotion/rejection

It should reuse the same runtime boundary but have separate graph versions from normal task execution.

## Where LangGraph Adds Material Value

LangGraph is likely valuable when Artifex needs durable, inspectable, conditional, resumable multi-agent workflows with nested loops.

Material-value areas:

- Explicit task execution graph with conditional retry/fail/commit branches.
- Smart Investigation graph: signals -> clusters -> hypotheses -> routing -> candidates.
- Human approval gates with pause/resume semantics.
- Progeny champion/challenger workflows.
- Dynamic temporary agent-team orchestration.
- Future parallel branches for tests/reviews/replays.
- Graph-level observability and visual traceability.

Low-value or risky areas for LangGraph right now:

- Replacing `TaskScheduler` as the canonical Project DAG scheduler.
- Replacing Django/Postgres state transitions.
- Turning every small Python helper call into a graph node.
- Moving worktree mutation safety into graph runtime abstractions.
- Hiding retry counters/checkpoints inside LangGraph-only state.

## Where Django/Postgres Should Remain Canonical

Django/Postgres should remain the source of truth for:

- Projects, plans, milestones, features, tasks, and task dependencies.
- Task statuses and retry counts.
- Worktrees and commit records.
- Attempts, coder results, reviews, judge verifications, test runs, artifacts.
- Events.
- Agent versions and promotion state.
- Progeny signals, investigations, and improvement candidates.
- Human approvals and audit history.

LangGraph, if introduced, should read/write through these canonical records rather than owning hidden state.

## Migration Path

### Phase 0: Preserve current behavior

Do not route production task execution through `LangGraphRuntime` yet.

Current guarantees to preserve:

- M2 autonomous execution
- M3 planning
- M4 Archaeologist
- M5 Agent Registry / Progeny
- M6 self-bootstrap
- CoderToolLoop
- worktree isolation
- inspect-before-edit
- safe mutation tools
- Reviewer/Judge separation
- deterministic tests
- three-attempt retry cap
- Sol/Qwen provider boundaries
- secret isolation

### Phase 1: Document the implicit execution graph

Add a declarative graph spec that mirrors current `AutonomousTaskLoop` behavior without executing it yet.

Suggested nodes:

- `claim_task`
- `prepare_worktree`
- `build_context`
- `coder`
- `run_tests`
- `review`
- `judge`
- `commit`
- `retry_or_fail`
- `cleanup`

Suggested conditional edges:

- `coder.failed -> retry_or_fail`
- `tests.failed -> review`
- `review.failed -> retry_or_fail`
- `judge.failed -> retry_or_fail`
- `judge.passed -> commit`
- `retry.available -> build_context`
- `retry.exhausted -> fail`

### Phase 2: Add graph run telemetry around the existing loop

Add persisted graph-run/node-run records or event payloads while keeping direct Python execution.

This gives observability before changing orchestration.

### Phase 3: Introduce `TaskExecutionRuntime` behind `GraphRuntime`

Implement a runtime adapter that can execute the graph spec using the existing service methods.

At this stage, `AutonomousTaskLoop` can delegate to the runtime but Django state remains canonical.

### Phase 4: Evaluate LangGraph as one adapter

Only after the explicit graph spec and telemetry exist, wire `LangGraphRuntime` as an implementation option.

Acceptance criteria should require deterministic parity with the current loop.

### Phase 5: Add subgraphs and human approval nodes

Promote Progeny investigation and champion/challenger replay workflows into graph subgraphs.

Add explicit approval nodes for candidate promotion and risky changes.

## Current Architecture Risks

1. `GraphRuntime` naming implies production graph execution, but it is currently a placeholder.

Risk: future contributors may assume LangGraph is already in the hot path.

2. Execution graph is implicit in Python.

Risk: retry and failure behavior becomes harder to evolve as Progeny wiring, investigation, and human gates are added.

3. Event Bus is persisted but not orchestration-driving.

Risk: event names may be interpreted as workflow triggers when they are currently audit facts.

4. `CoderToolLoop` has internal iteration but only coarse persistence.

Risk: power loss during model/tool-loop execution resumes only at task-attempt granularity, not tool-step granularity.

5. Reviewer/Judge are deterministic services, not model-driven graph nodes.

Risk: current separation is good, but future multi-agent review designs need explicit graph boundaries.

## Recommendation

Do not refactor execution into LangGraph immediately.

The safest next step is to keep the existing Python/Django loop as the production engine and create a first-class, documented `TaskExecutionGraph` specification plus telemetry records around it. This preserves all working guarantees while making the intended graph architecture concrete.

CoderToolLoop should remain an internal loop inside the Coder graph node for now. Promote it to a subgraph only when resumability or multi-step observability at each tool call becomes necessary.

LangGraph should be introduced as an adapter behind `GraphRuntime` only after deterministic parity tests prove it reproduces the current loop exactly.