Artifex/docs/graph_runtime_audit.md

Graph Runtime Audit

Date: 2026-08-15

Baseline inspected: 6a73372665715950aee8416a9d2b3ec79955f816

Scope: audit current Artifex implementation against the intended graph-runtime architecture. This document does not propose or include a refactor.

Executive Summary

GraphRuntime is currently an interface boundary, not the production execution engine.

LangGraphRuntime is a deterministic placeholder adapter. It is not executing project, milestone, feature, task, Coder, Reviewer, Judge, retry, or commit workflows.

Real production task execution is driven by runtime_loop.autonomous_task_loop.AutonomousTaskLoop, graph.scheduler.TaskScheduler, Django ORM state, direct Python service calls, and persisted Event Bus events.

The current implementation has two separate concepts that are easy to conflate:

• Project DAG: persisted project work decomposition and task dependencies.
• Execution Graph: the operational workflow for executing one task attempt through Coder, tools, tests, Reviewer, Judge, retry, and commit.

The Project DAG exists today in Django models and scheduler logic. The Execution Graph exists only as imperative Python control flow inside AutonomousTaskLoop; it is not represented as a graph runtime yet.

Recommendation: keep Django/Postgres as canonical state and migrate execution to graph semantics incrementally. Start by extracting an explicit TaskExecutionGraph description around the existing task loop without changing behavior, then add graph telemetry/checkpoint metadata. LangGraph should remain behind GraphRuntime and should provide value only when conditional routing, resumable execution, human gates, subgraphs, and agent-team workflows become difficult to manage in plain Python.

Direct Answers

1. Is GraphRuntime used in production task execution?

No.

graph/runtime.py defines an abstract GraphRuntime with start, pause, resume, cancel, and signal, but no production task execution path depends on it.

Observed production path:

• runtime_loop/autonomous_task_loop.py imports TaskScheduler, Coder, Reviewer, Judge, WorktreeTools, and DeterministicTestRunner.
• It does not import or call GraphRuntime or LangGraphRuntime.

Current role of GraphRuntime: architectural boundary and future abstraction.

2. Is LangGraphRuntime actually executing task workflows?

No.

graph/langgraph_runtime.py returns a deterministic string from start and no-ops pause, resume, cancel, and signal.

It does not import LangGraph, build a state graph, execute graph nodes, route edges, persist graph checkpoints, or call Coder/Reviewer/Judge.

Its only observed test coverage is tests/test_interfaces.py::test_langgraph_runtime_is_hidden_behind_runtime_interface, which checks that start() returns a run id prefix.

Real Execution Path

Current task execution path for a single task:

1. Scheduler

AutonomousTaskLoop.run_once() calls TaskScheduler.claim_next_ready_task().

TaskScheduler queries Task(status=READY) ordered by priority and creation time, skips tasks with incomplete dependencies, marks the selected task RUNNING, persists it, and publishes TASK_STARTED.

2. Investigation/context building

AutonomousTaskLoop._execute_task() creates or reuses a worktree and builds context with WorkerContextBuilder.build_for_task().

This context builder reads selected Python files and tests directly from the worktree, excludes migrations and secrets, and attaches the current diff.

Coder also has an inspect-before-edit phase inside Coder.execute() for sensitive tasks. It asks the model for read-only inspection operations, then executes list_directory, read_file, search_code, find_symbol, git_status, or git_diff through WorktreeTools.

3. Coder

AutonomousTaskLoop calls Coder.execute(context, tools, project, agent_version).

Coder.execute() may perform inspection, then creates a CoderToolLoop and calls CoderToolLoop.run().

4. Mutation

CoderToolLoop asks the model for mutation operations and applies them through WorktreeTools.

Supported mutation operations include:

• write_file
• apply_patch
• delete_file
• move_file
• create_directory
• run_command

Patch mismatch recovery is internal to CoderToolLoop: it records structured failure evidence, re-reads the live target file excerpt, and asks the model once more for a corrected patch. It does not default to full-file replacement.

5. Tests

AutonomousTaskLoop calls DeterministicTestRunner.run().

DeterministicTestRunner shells out via WorktreeTools.run(), persists an Artifact with stdout/stderr/returncode, and creates a TestRun with PASS or FAIL.

On failure, _attach_test_failure_evidence() stores stdout/stderr excerpts into the attempt's coder_result.metadata.test_failure_evidence, so later retry context can include the actual traceback.

6. Reviewer

AutonomousTaskLoop stages intent-to-add with git add -N, gets git diff, then calls Reviewer.review(task, reviewer_version, diff, test_status).

Reviewer is currently a deterministic Python policy. It creates a Review row with PASS, REWORK_REQUIRED, or REJECTED based on test status, empty diff, and health-route-specific heuristics.

7. Judge

If Reviewer passes, AutonomousTaskLoop calls Judge.judge(project, task, judge_version, diff, test_status).

Judge is currently deterministic Python. It creates a Verification row with task-level acceptance evidence and PASS or FAIL.

8. Retry

If Coder, Reviewer, or Judge fails, _retry_or_fail() updates TaskAttempt.status, increments Task.retry_count, publishes TASK_FAILED, and either loops for another attempt or marks the task FAILED and publishes TASK_RETRY_EXHAUSTED plus final TASK_FAILED.

The effective retry budget is capped at two retries, giving three total attempts.

9. Commit

If tests, Reviewer, and Judge all pass, WorktreeTools.commit_all() stages and commits all changes. CommitRecord is persisted with coder, reviewer, judge, test run, review, verification, SHA, branch, and message.

The task is marked COMPLETE, COMMIT_CREATED and TASK_COMPLETED are published with telemetry, the worktree is validated clean, and the task worktree is cleaned up.

Transition Classification

Explicit graph nodes/edges

Currently only the Project DAG has explicit persisted edges:

• TaskDependency(task, depends_on) models task-to-task dependencies.
• Milestone.dependencies models milestone-level dependencies but is not currently central in execution scheduling.
• ProjectPlanBuilder validates task dependency acyclicity before materializing tasks.
• TaskScheduler.claim_next_ready_task() treats incomplete TaskDependency rows as blocking edges.

No explicit graph nodes/edges currently model the execution workflow inside a task attempt.

Direct Python service calls

Most real execution transitions are direct service calls:

• AutonomousTaskLoop.run_once() -> TaskScheduler.claim_next_ready_task()
• _execute_task() -> WorkerContextBuilder.build_for_task()
• _execute_task() -> Coder.execute()
• Coder.execute() -> CoderToolLoop.run()
• CoderToolLoop -> ModelRouter.complete()
• CoderToolLoop -> WorktreeTools mutation/read/test helpers
• _execute_task() -> DeterministicTestRunner.run()
• _execute_task() -> Reviewer.review()
• _execute_task() -> Judge.judge()
• _execute_task() -> WorktreeTools.commit_all()
• _retry_or_fail() direct retry/fail transition

Django state transitions

Django/Postgres is the canonical durable state machine today:

• Task.status: READY, RUNNING, FAILED, COMPLETE
• Task.retry_count
• TaskAttempt.status: RUNNING, REWORK_REQUIRED, FAILED, COMPLETE
• Worktree.status
• TestRun.status
• Review.status
• Verification.result
• CommitRecord
• ModelRequest.status
• ProgenySignal.status

Event Bus driven transitions

The Event Bus persists facts but does not currently drive the control flow.

Events published today include:

• PLAN_APPROVED
• MILESTONE_CREATED
• TASK_CREATED
• TASK_READY
• TASK_STARTED
• TEST_FAILED
• REVIEW_FAILED
• TASK_FAILED
• TASK_RETRY_EXHAUSTED as a string event
• COMMIT_CREATED
• TASK_COMPLETED
• AGENT_CREATED
• AGENT_PROMOTED
• PROGENY_SIGNAL_CREATED as a string event from ProgenyService

These are audit/event facts, not asynchronous orchestration triggers in the current implementation.

Project DAG vs Execution Graph

Project DAG

The Project DAG answers: what work exists, and what depends on what?

Current implementation:

• ProjectPlanContract represents milestones, features, tasks, and task dependency references.
• validate_project_plan() validates task dependency references and cycle freedom.
• ProjectPlanBuilder.apply() persists ProjectPlan, Milestone, Feature, Task, and TaskDependency rows.
• TaskScheduler consumes TaskDependency rows to claim only unblocked READY tasks.

This is real and production-relevant today.

Execution Graph

The Execution Graph answers: how is one task/feature/milestone autonomously executed?

Current implementation:

• This graph is implicit in AutonomousTaskLoop._execute_task().
• Nodes are not persisted as graph nodes.
• Edges are Python if/continue/return branches.
• Looping is a Python while task.retry_count <= effective_max_retries.
• Checkpointing is coarse-grained through Django rows such as TaskAttempt, TestRun, Review, Verification, and CommitRecord.

The execution graph exists conceptually but not as a first-class runtime artifact.

CoderToolLoop Placement

Recommended placement: graph node containing an internal tool loop.

Do not make CoderToolLoop the top-level execution graph yet.

Rationale:

• CoderToolLoop is a bounded inner loop for a single agent/tool interaction.
• Its state is local and tactical: inspection results, mutation results, patch mismatch recovery, operation telemetry.
• It should be resumable eventually, but it is not the same graph as the whole task lifecycle.
• Making every tool call a top-level graph node now would increase orchestration complexity before there is enough operational pressure.

Near-term target:

• TaskExecutionGraph has a coder node.
• The coder node invokes Coder.execute().
• Coder.execute() owns CoderToolLoop internally.
• CoderToolLoop emits structured telemetry and failure evidence that the outer graph records.

Later target:

• If long-running coding sessions, human approvals, streamed model/tool execution, or partial resume become necessary, promote internal tool-loop steps to a nested subgraph.

Runtime Capability Assessment

Conditional edges

Required.

Current conditional branches include:

• no ready task -> stop
• dependencies incomplete -> skip task
• coder failed -> retry/fail
• tests failed -> still Review, then retry/fail through Review
• reviewer failed -> retry/fail
• judge failed -> retry/fail
• all checks passed -> commit

These map naturally to graph conditional edges.

Loops

Required.

Current loops include:

• task retry loop capped at three total attempts
• CoderToolLoop patch recovery loop
• future investigation and experiment loops

The retry cap must remain durable and enforced by Django state, not hidden only inside graph runtime memory.

Parallel branches

Useful, not urgent for current V1/V2 guarantees.

Good candidates later:

• independent task execution across the Project DAG
• parallel tests/static checks/security checks
• multi-agent review lanes
• Progeny benchmark replay across candidates

Parallelism should be introduced only after worktree isolation and resource contention controls are explicit.

Subgraphs

Required long-term.

Natural subgraphs:

• Task execution
• Coder tool loop
• Reviewer/Judge verification
• Progeny investigation
• Progeny experiment/champion-challenger evaluation
• Project Brain planning/replanning

Checkpoint/resume

Required, but Django/Postgres should remain canonical.

Current checkpointing already exists at coarse boundaries:

• task claimed
• attempt created
• coder result saved
• test run saved
• review saved
• verification saved
• commit saved

Graph checkpointing should augment these records, not replace them.

Human approval nodes

Required for safe V2+ autonomy.

Likely approval gates:

• project plan approval
• task split/replan approval
• risky migration approval
• provider credential/resource approval
• promotion of Progeny candidates
• destructive infrastructure changes

Human approval should persist as Django state and be represented as paused graph nodes.

Dynamic temporary agent teams

Required later, not required for the current loop.

Smart Investigation and Progeny experiments will likely need temporary teams such as:

• incident analyst
• domain expert
• prompt/policy editor
• benchmark designer
• reviewer challenger

The runtime should support this as graph metadata and persisted AgentRun/future team records.

Graph versioning

Required before graph-based production execution.

Every accepted commit and Progeny investigation should know which execution graph version produced it. Without this, regressions in orchestration will be hard to diagnose.

Graph telemetry

Required.

Existing telemetry should become graph-node telemetry:

• mutation operations
• patch success/mismatch rates
• write-file fallback rate
• model requests per task
• test duration/status
• review findings
• judge evidence
• retry count
• provider errors
• signal counts

This telemetry should feed Progeny.

Progeny champion/challenger workflow graphs

Required for the future Replay Arena / experiment execution layer, but should not be the next refactor.

The Progeny workflow graph should model:

• investigation selection
• candidate generation
• benchmark construction
• champion/challenger replay
• judge/evaluator comparison
• approval/promotion/rejection

It should reuse the same runtime boundary but have separate graph versions from normal task execution.

Where LangGraph Adds Material Value

LangGraph is likely valuable when Artifex needs durable, inspectable, conditional, resumable multi-agent workflows with nested loops.

Material-value areas:

• Explicit task execution graph with conditional retry/fail/commit branches.
• Smart Investigation graph: signals -> clusters -> hypotheses -> routing -> candidates.
• Human approval gates with pause/resume semantics.
• Progeny champion/challenger workflows.
• Dynamic temporary agent-team orchestration.
• Future parallel branches for tests/reviews/replays.
• Graph-level observability and visual traceability.

Low-value or risky areas for LangGraph right now:

• Replacing TaskScheduler as the canonical Project DAG scheduler.
• Replacing Django/Postgres state transitions.
• Turning every small Python helper call into a graph node.
• Moving worktree mutation safety into graph runtime abstractions.
• Hiding retry counters/checkpoints inside LangGraph-only state.

Where Django/Postgres Should Remain Canonical

Django/Postgres should remain the source of truth for:

• Projects, plans, milestones, features, tasks, and task dependencies.
• Task statuses and retry counts.
• Worktrees and commit records.
• Attempts, coder results, reviews, judge verifications, test runs, artifacts.
• Events.
• Agent versions and promotion state.
• Progeny signals, investigations, and improvement candidates.
• Human approvals and audit history.

LangGraph, if introduced, should read/write through these canonical records rather than owning hidden state.

Migration Path

Phase 0: Preserve current behavior

Do not route production task execution through LangGraphRuntime yet.

Current guarantees to preserve:

• M2 autonomous execution
• M3 planning
• M4 Archaeologist
• M5 Agent Registry / Progeny
• M6 self-bootstrap
• CoderToolLoop
• worktree isolation
• inspect-before-edit
• safe mutation tools
• Reviewer/Judge separation
• deterministic tests
• three-attempt retry cap
• Sol/Qwen provider boundaries
• secret isolation

Phase 1: Document the implicit execution graph

Add a declarative graph spec that mirrors current AutonomousTaskLoop behavior without executing it yet.

Suggested nodes:

• claim_task
• prepare_worktree
• build_context
• coder
• run_tests
• review
• judge
• commit
• retry_or_fail
• cleanup

Suggested conditional edges:

• coder.failed -> retry_or_fail
• tests.failed -> review
• review.failed -> retry_or_fail
• judge.failed -> retry_or_fail
• judge.passed -> commit
• retry.available -> build_context
• retry.exhausted -> fail

Phase 2: Add graph run telemetry around the existing loop

Add persisted graph-run/node-run records or event payloads while keeping direct Python execution.

This gives observability before changing orchestration.

Phase 3: Introduce TaskExecutionRuntime behind GraphRuntime

Implement a runtime adapter that can execute the graph spec using the existing service methods.

At this stage, AutonomousTaskLoop can delegate to the runtime but Django state remains canonical.

Phase 4: Evaluate LangGraph as one adapter

Only after the explicit graph spec and telemetry exist, wire LangGraphRuntime as an implementation option.

Acceptance criteria should require deterministic parity with the current loop.

Phase 5: Add subgraphs and human approval nodes

Promote Progeny investigation and champion/challenger replay workflows into graph subgraphs.

Add explicit approval nodes for candidate promotion and risky changes.

Current Architecture Risks

1. GraphRuntime naming implies production graph execution, but it is currently a placeholder.

Risk: future contributors may assume LangGraph is already in the hot path.

2. Execution graph is implicit in Python.

Risk: retry and failure behavior becomes harder to evolve as Progeny wiring, investigation, and human gates are added.

3. Event Bus is persisted but not orchestration-driving.

Risk: event names may be interpreted as workflow triggers when they are currently audit facts.

4. CoderToolLoop has internal iteration but only coarse persistence.

Risk: power loss during model/tool-loop execution resumes only at task-attempt granularity, not tool-step granularity.

5. Reviewer/Judge are deterministic services, not model-driven graph nodes.

Risk: current separation is good, but future multi-agent review designs need explicit graph boundaries.

Recommendation

Do not refactor execution into LangGraph immediately.

The safest next step is to keep the existing Python/Django loop as the production engine and create a first-class, documented TaskExecutionGraph specification plus telemetry records around it. This preserves all working guarantees while making the intended graph architecture concrete.

CoderToolLoop should remain an internal loop inside the Coder graph node for now. Promote it to a subgraph only when resumability or multi-step observability at each tool call becomes necessary.

LangGraph should be introduced as an adapter behind GraphRuntime only after deterministic parity tests prove it reproduces the current loop exactly.