Document execution graph architecture

docs/execution_graph_architecture.md

@@ -0,0 +1,204 @@
+# Execution Graph Architecture
+
+Date: 2026-08-15
+
+## Implemented Architecture
+
+Artifex now has two intentionally separate graph concepts.
+
+Project DAG remains the canonical representation of what work exists:
+
+- `Project`
+- `Milestone`
+- `Feature`
+- `Task`
+- `TaskDependency`
+
+Execution Graph is now the canonical representation of how autonomous work is performed. It is runtime-neutral and persisted in Django/Postgres.
+
+## Graph Domain Models
+
+Implemented graph domain models:
+
+- `ExecutionGraphDefinition`
+- `ExecutionGraphVersion`
+- `GraphRun`
+- `GraphNodeRun`
+- `GraphEdgeTraversal`
+- `GraphApproval`
+
+Execution graph version statuses:
+
+- `DRAFT`
+- `CHALLENGER`
+- `CHAMPION`
+- `RETIRED`
+
+Graph run statuses:
+
+- `PENDING`
+- `RUNNING`
+- `PAUSED`
+- `COMPLETE`
+- `FAILED`
+- `CANCELLED`
+
+Node runs persist status, timing, input/output metadata, failure evidence, telemetry, optional agent version, and optional model request reference.
+
+Edge traversals persist selected conditional transitions.
+
+## Graph Specification
+
+Graph specs are serializable dictionaries with:
+
+- name
+- version
+- graph type
+- entry node
+- nodes
+- edges
+- conditional edge labels
+- terminal nodes
+- metadata
+
+Specs do not execute arbitrary Python. Node execution resolves through `NodeHandlerRegistry`.
+
+## TaskExecutionGraph V1
+
+`TaskExecutionGraph v1` mirrors the prior `AutonomousTaskLoop` behavior.
+
+Nodes:
+
+- `claim_task`
+- `prepare_worktree`
+- `build_context`
+- `coder`
+- `run_tests`
+- `review`
+- `judge`
+- `commit`
+- `retry_or_fail`
+- `cleanup`
+- `complete`
+- `fail`
+
+Important semantics preserved:
+
+- deterministic test failure still reaches Reviewer
+- Reviewer failure goes to retry/fail
+- Judge failure goes to retry/fail
+- maximum semantic task attempts remains three total attempts
+- commit is guarded against duplicate commits on resume
+- CoderToolLoop remains inside the `coder` node
+
+## NativeGraphRuntime
+
+`NativeGraphRuntime` is the reference implementation.
+
+It supports:
+
+- conditional edges
+- loops
+- persisted graph runs
+- persisted node runs
+- edge traversal history
+- checkpoint/resume at major node boundaries
+- terminal success/failure
+- pause state
+- cancellation
+- graph events
+- bounded metadata
+
+Django/Postgres remains canonical for task, attempt, worktree, test, review, judge, commit, event, agent, and Progeny state.
+
+## AutonomousTaskLoop Delegation
+
+`AutonomousTaskLoop` still uses `TaskScheduler` to claim Project DAG work.
+
+After a task is claimed, it now creates a `GraphRun` for the champion `TaskExecutionGraph v1` and delegates task lifecycle execution to `NativeGraphRuntime`.
+
+The scheduler remains Project-DAG-oriented. The execution runtime handles task-attempt workflow.
+
+## LangGraphRuntime
+
+`LangGraphRuntime` is no longer a pure placeholder. It builds a LangGraph `StateGraph` from Artifex graph definitions when the `langgraph` package is installed.
+
+Artifex remains runtime-neutral:
+
+- Artifex owns graph definitions
+- Artifex owns persisted graph state
+- Artifex owns node contracts
+- LangGraph is an execution backend
+
+Current limitation: the local environment used during implementation did not have `langgraph` installed, so deterministic tests validate the adapter boundary and missing-dependency behavior. Full LangGraph execution parity requires installing the declared `langgraph>=0.2,<0.3` dependency in local/Spark environments.
+
+## Checkpoint And Resume
+
+Coarse checkpointing is implemented at graph node boundaries.
+
+Resume behavior avoids repeating a completed interrupted node. Loop re-entry creates a new `GraphNodeRun` visit via `visit_index`.
+
+Irreversible commit behavior is guarded by checking for an existing `CommitRecord` for the task before creating a new commit.
+
+## Human Approval
+
+Graph approval support is implemented with `GraphApproval`.
+
+A node can pause the graph with `AWAITING_APPROVAL`. A signal can approve pending graph approvals and resume execution.
+
+This is intentionally minimal and prepares future gates for risky migrations, deployment, Progeny promotion, destructive infrastructure changes, and project plan approvals.
+
+## Subgraph Support
+
+The graph spec supports runtime-neutral subgraph representation through node metadata.
+
+CoderToolLoop is not moved into a subgraph yet. It remains inside the `coder` node.
+
+## Graph Inspection
+
+`graph_run_inspection()` exposes UI-ready JSON containing:
+
+- graph/version
+- node list
+- edges
+- node statuses
+- current node
+- durations
+- failures
+- selected edge traversals
+
+This is sufficient for a future visual execution graph UI.
+
+## Progeny Integration
+
+`ProgenySignal` can now reference:
+
+- `graph_run`
+- `graph_node_run`
+- `execution_graph_version`
+
+This enables future investigations such as:
+
+- failures by graph version
+- failures by node type
+- patch success rate by workflow version
+- Reviewer rework changes after inserting a verification node
+
+## Migrations
+
+New migrations:
+
+- `graph/0001_initial.py`
+- `graph/0002_graphnoderun_visit_index.py`
+- `graph/0003_graphapproval.py`
+- `projects/0003_commitrecord_graph_run.py`
+- `agents/0005_progenysignal_graph_lineage.py`
+
+## Known Limitations
+
+- Full LangGraph parity execution is implemented but not exercised in this environment because `langgraph` is not installed.
+- Parallel branch execution is represented by the graph model but not executed concurrently by `NativeGraphRuntime`.
+- Subgraph support is represented in the spec but not yet expanded into nested `GraphRun` execution.
+- CoderToolLoop remains attempt-granularity for checkpointing.
+- Graph telemetry is persisted at node level, but model request references are not yet automatically linked to individual node runs.
+- Event Bus records graph facts but is not an orchestration engine.