277 lines
12 KiB
Markdown
277 lines
12 KiB
Markdown
# Artifex Handoff - Current State - 2026-08-17
|
|
|
|
## Current Objective
|
|
|
|
We have been repairing and dogfooding the `CRYPTO / PROTOCOL VENTURE COHORT` pipeline, specifically the V0.3.1 cohort and the V0.3.2B token-judge evidence repair.
|
|
|
|
Hard constraint: **NO NEW IDEAS** for the repair work. Only reprocess the existing six former V0.3.1 survivors.
|
|
|
|
Safety constraints remain active:
|
|
|
|
- Do not issue or sell native tokens.
|
|
- Do not sell NFTs or memberships.
|
|
- Do not launch mainnet.
|
|
- Do not create liquidity or market make.
|
|
- Do not contact investors, users, or customers.
|
|
- Do not fundraise.
|
|
- Do not spend real money.
|
|
|
|
## Environment
|
|
|
|
- Workspace: `S:\PycharmProjects\Artifex`
|
|
- Repo remote: `http://192.168.1.162:3000/daniel/Artifex.git`
|
|
- Database: `sqlite:///db.sqlite3`
|
|
- SearXNG: `http://192.168.1.162:8080`
|
|
- Qwen: `http://192.168.1.162:8002`
|
|
- Current date in environment: `2026-08-17`
|
|
- Known harmless push warning: `git: 'credential-manager-core' is not a git command. See 'git --help'.`
|
|
|
|
Spark remote validation was previously blocked by SSH auth:
|
|
|
|
- `daniel@192.168.1.162: Permission denied (publickey,password).`
|
|
|
|
## Current Cohort
|
|
|
|
- Cohort ID: `CPV031-20260816215626-66d636b2`
|
|
- GraphRun: `10`
|
|
- Existing six former V0.3.1 survivors only:
|
|
- `Agent Passport Clearinghouse`
|
|
- `PatchBond Network`
|
|
- `ProofGrid Compute Attestation Network`
|
|
- `ProofBond`
|
|
- `Compute Clearance Network`
|
|
- `ProofGrid Compute Escrow`
|
|
|
|
## Latest Implemented Work
|
|
|
|
### V0.3.2B Token Judge Evidence Repair
|
|
|
|
Implemented strict Sol Token Utility Judge evidence validation.
|
|
|
|
Required non-empty fields now enforced:
|
|
|
|
- `argument_for_native_token`
|
|
- `argument_against_native_token`
|
|
- `external_collateral_counterfactual`
|
|
- `native_token_removed_breaks`
|
|
- `native_token_removed_explanation`
|
|
- `material_improvements_over_USDC_ETH`
|
|
- `native_asset_costs_and_risks`
|
|
- `final_rationale`
|
|
|
|
Behavior now:
|
|
|
|
- Rejects incomplete Sol judge response.
|
|
- Retries incomplete judge response up to 3 attempts.
|
|
- Raises `ValueError` after exhausted retries.
|
|
- Does **not** fall back to deterministic/heuristic classification when Sol produced an incomplete response.
|
|
- Normalizes Sol response when `external_collateral_counterfactual` appears nested under `counterfactual_analysis`.
|
|
- Uppercases `utility_scores` keys during normalization.
|
|
- Persists `final_rationale` in token metadata as well as `TokenUtilityAssessment.rationale`.
|
|
- `crypto_score_row()` now reuses persisted `token_utility_assessment` instead of calling the token judge again.
|
|
- Final Sol IC prompt was compacted so it returns usable JSON.
|
|
|
|
Rationale invariants now checked:
|
|
|
|
- Every `TOKEN_OPTIONAL` rationale must explain why ETH/USDC/external collateral is preferable despite claimed degradation.
|
|
- Every `TOKEN_ESSENTIAL` / `TOKEN_STRONGLY_JUSTIFIED` rationale must explain what protocol-specific property cannot be reproduced economically enough with ETH/USDC/external collateral.
|
|
- The code does not automatically promote due to material degradation.
|
|
|
|
## Latest Six-Proposal Results
|
|
|
|
Before/after classifications from V0.3.2B:
|
|
|
|
| Company | Before | After | Score | Final IC Decision | Final IC Score | Required Field Gaps |
|
|
|---|---:|---:|---:|---:|---:|---:|
|
|
| Agent Passport Clearinghouse | `TOKEN_OPTIONAL` | `TOKEN_UNNECESSARY` | `18.0` | `REJECT_TOKEN_NOT_NEEDED` | `62.3` | `[]` |
|
|
| PatchBond Network | `TOKEN_OPTIONAL` | `TOKEN_OPTIONAL` | `48.0` | `REJECT_TOKEN_NOT_NEEDED` | `66.4` | `[]` |
|
|
| ProofGrid Compute Attestation Network | `TOKEN_OPTIONAL` | `TOKEN_OPTIONAL` | `42.0` | `REJECT_TOKEN_UTILITY_WEAK` | `63.6` | `[]` |
|
|
| ProofBond | `TOKEN_OPTIONAL` | `TOKEN_OPTIONAL` | `54.0` | `REJECT_TOKEN_UTILITY_WEAK` | `64.3` | `[]` |
|
|
| Compute Clearance Network | `TOKEN_OPTIONAL` | `TOKEN_STRONGLY_JUSTIFIED` | `78.0` | `PROTOCOL_VALIDATE` | `79.7` | `[]` |
|
|
| ProofGrid Compute Escrow | `TOKEN_OPTIONAL` | `TOKEN_OPTIONAL` | `58.0` | `REVISE_TOKEN_MODEL` | `67.9` | `[]` |
|
|
|
|
Current corrected cohort state:
|
|
|
|
- Token unnecessary: `1`
|
|
- Token optional/routed SaaS: `4`
|
|
- Token strongly justified: `1`
|
|
- Token essential: `0`
|
|
- Serious crypto survivors: `0`
|
|
- Crypto survivors: `0`
|
|
- Autonomous crypto survivors: `0`
|
|
- Assisted high-potential: `0`
|
|
- Finalists: `0`
|
|
- V0.3.2B required field gaps: `0`
|
|
- Token red-team failures:
|
|
- `TOKEN_NOT_REQUIRED: 5`
|
|
- `YIELD_DEPENDENCY: 1`
|
|
- `SLASHING_DEPENDS_ON_HUMAN_JUDGMENT: 1`
|
|
|
|
Important nuance: `Compute Clearance Network` became `TOKEN_STRONGLY_JUSTIFIED`, but final cohort finalists remain `0`. It is not automatically promoted to finalist.
|
|
|
|
## Latest Files Changed
|
|
|
|
- `agents/crypto_venture.py`
|
|
- Strict token judge evidence validation/retry/fail-closed behavior.
|
|
- Token judge response normalization.
|
|
- Persisted required fields.
|
|
- Reused persisted token assessment in `crypto_score_row()`.
|
|
- Compact final Sol IC prompt.
|
|
- `tests/test_crypto_venture_cohort.py`
|
|
- Added coverage for complete required judge fields.
|
|
- Added retry-success and retry-exhaustion/fail-closed tests.
|
|
- `control_plane/ventures/management/commands/export_crypto_venture_cohort.py`
|
|
- Added `V0.3.2B Before/After` markdown section.
|
|
- `docs/crypto_venture_cohort_v031_20260816.md`
|
|
- Updated exported V0.3.2B report.
|
|
- `docs/crypto_venture_cohort_v031_20260816.json`
|
|
- Updated exported V0.3.2B report data.
|
|
|
|
## Latest Commits Pushed
|
|
|
|
Recent relevant commits:
|
|
|
|
- `35d3346 Repair crypto token judge evidence export`
|
|
- `6e89ea8 Require complete crypto token judge evidence`
|
|
- `052d1fc Add corrected crypto cohort V0.3.1 export`
|
|
- `94b9ab6 Repair crypto token utility scoring`
|
|
- `54ffe92 Add crypto protocol cohort V0.3.1 export`
|
|
|
|
The most recent push succeeded to `main` with the known harmless credential-manager warning.
|
|
|
|
## Verification
|
|
|
|
Latest verification passed:
|
|
|
|
- `python manage.py check`: passed
|
|
- `python -m pytest -p no:django -q tests/test_crypto_venture_cohort.py`: `14 passed`
|
|
- Full suite: `157 passed, 1 skipped`
|
|
|
|
Warnings are existing Python/Django/LangGraph deprecation warnings and pytest collection warnings for `TestRun`; no failing tests.
|
|
|
|
## Current Git State At Time Of Handoff
|
|
|
|
After the latest commit/push, `git status --short` was clean.
|
|
|
|
This handoff file itself is newly added after that push and should be committed/pushed if the user wants the note persisted remotely.
|
|
|
|
## If Continuing
|
|
|
|
Recommended next steps only if the user asks to continue:
|
|
|
|
1. Commit and push this handoff note if desired.
|
|
2. If further V0.3.2B analysis is requested, inspect `docs/crypto_venture_cohort_v031_20260816.md` first.
|
|
3. Do not regenerate ideas unless explicitly instructed.
|
|
4. Do not rerun broad research unless explicitly requested or needed for a new judge repair.
|
|
5. Keep all token/mainnet/fundraising/user-contact/no-spend stop conditions active.
|
|
|
|
## Model Studio V0.1 Update
|
|
|
|
Commit `d61751a Add Guard model studio foundation` added the initial Model Studio domain.
|
|
|
|
Key files:
|
|
|
|
- `control_plane/model_studio/models.py`
|
|
- Canonical Django state for training projects, immutable dataset versions and recipes, checkpoints, experiments, runs, evaluation suites/runs/results, regression-bank items, failure clusters, promotion policies/decisions, overnight programs, reports, and artifacts.
|
|
- `control_plane/model_studio/services.py`
|
|
- Guard import/archaeology, dataset curation, benchmark gate, experiment contract and fingerprinting, simulated run/promotion logic, deadline reserve checks, and morning report.
|
|
- `control_plane/model_studio/profiles.py`
|
|
- `GuardModelProfile`, based on the discovered ForgeGuard layout.
|
|
- `control_plane/model_studio/backends.py`
|
|
- `FakeTrainingBackend` for tests and `SparkGuardBackend` for profile-generated remote commands only.
|
|
- Commands:
|
|
- `python manage.py modelstudio_import_guard --repository <local ForgeGuard path> --slug guard-3b`
|
|
- `python manage.py modelstudio_dataset_audit --project guard-3b`
|
|
- `python manage.py modelstudio_overnight --project guard-3b --dry-run`
|
|
- Minimal UI:
|
|
- `/model-studio/`
|
|
|
|
Guard was imported locally for archaeology from:
|
|
|
|
- `S:\PycharmProjects\Me3\tools\ForgeGuard`
|
|
|
|
Current Guard project state in Artifex DB:
|
|
|
|
- Training project slug: `guard-3b`
|
|
- Current base-model Champion candidate: `Qwen/Qwen2.5-Coder-3B-Instruct`
|
|
- Selection: explicitly human-mandated starting Champion
|
|
- Evidence status: `PENDING_FRESH_SPARK_BASELINE`
|
|
- Training project status: `NEEDS_REPAIR`
|
|
- Benchmark integrity: `WARNING`, not valid
|
|
- Spark checkout is not registered or verified
|
|
- Fresh baseline is missing
|
|
|
|
No local Guard training/evaluation was run. The local machine is unsuitable and the user explicitly required Spark-only execution.
|
|
|
|
Spark connectivity is confirmed (`ssh spark hostname` returned `Inceptal`), but read-only inspection found no `ForgeGuard` directory under `/workspace`, `/home`, or `/opt`. Do not run training until a Spark ForgeGuard checkout path is provided or explicitly synced, then register it with `--spark-working-directory` and verify it.
|
|
|
|
Dataset curation result:
|
|
|
|
- `79` imported JSON artifacts
|
|
- `49` blocked
|
|
- `30` warning-only
|
|
- `0` approved training datasets
|
|
- Decision: `NO_TRAINING_DATASET_APPROVED`
|
|
|
|
Important curation facts:
|
|
|
|
- Summary JSON files are non-training artifacts.
|
|
- Candidate/provisional corpora are blocked.
|
|
- `full_source_c4_guard_candidate_v12` is explicitly candidate-only and has not been trained.
|
|
- The Qwen 3B verified pilot has 88 records and a claimed EVMBench exclusion, but stays `WARNING` pending record-level provenance/schema/coverage and contamination verification.
|
|
- No Spark baseline/canary/overnight program is permitted yet.
|
|
|
|
Latest verification after Model Studio work:
|
|
|
|
- `python manage.py check`: passed
|
|
- `python -m pytest -p no:django -q tests/test_model_studio_v01.py`: `6 passed`
|
|
- Full suite: `163 passed, 1 skipped`
|
|
|
|
## Guard Dataset Factory Update
|
|
|
|
Commit `e27df77 Add Guard dataset curation pipeline` added Spark-backed dataset curation.
|
|
|
|
The source policy is now:
|
|
|
|
- All existing Guard data, including legacy holdouts, is authorized as a curation source by user mandate.
|
|
- Former holdouts can enter a new training version only after source-hash re-splitting produces new validation and regression manifests.
|
|
- Existing benchmark suites must be replaced/versioned before comparisons if their old holdout sources are used for training.
|
|
- Source data is never overwritten; every output is a new immutable `DatasetVersion`.
|
|
|
|
Spark Guard location:
|
|
|
|
- `/home/daniel/forgeguard`
|
|
- Active data root inspected: `/home/daniel/forgeguard/project/data`
|
|
|
|
The first bad recursive Spark inventory created `227,137` null-record-count DB rows from individual evidence JSON objects. Those malformed local DB rows were purged. The importer now accepts only explicit manifest paths; it does not recurse through evidence stores.
|
|
|
|
Qwen Dataset Curator proposal:
|
|
|
|
- Proposal ID: `0afcb3f5-a097-4461-b890-14d350e9d8f6`
|
|
- Qwen identified heterogeneous schemas, weak/contradictory labels, provenance risks, and exact-holdout handling requirements.
|
|
- Its proposal is persisted as `DatasetCurationProposal` and a Model Studio artifact.
|
|
|
|
Spark materialized versions:
|
|
|
|
1. `guard_curated_v01`
|
|
- `/home/daniel/forgeguard/artifex_dataset_versions/guard_curated_v01/`
|
|
- Train `4,684`; validation `657`; regression `541`.
|
|
|
|
2. `guard_curated_v02` (current recommended corpus)
|
|
- Train manifest: `/home/daniel/forgeguard/artifex_dataset_versions/guard_curated_v02/train.json`
|
|
- Validation manifest: `/home/daniel/forgeguard/artifex_dataset_versions/guard_curated_v02/validation.json`
|
|
- Regression manifest: `/home/daniel/forgeguard/artifex_dataset_versions/guard_curated_v02/regression.json`
|
|
- Artifex DatasetVersion ID: `416fc9dd-c922-4581-8ed5-ca10a488d224`
|
|
- Train: `3,030`
|
|
- Validation: `437`
|
|
- Regression: `332`
|
|
- Train SHA-256: `d4c1a6accee0fac511641a24287cfdb469819fdd6d34af720c7349dc66b4803c`
|
|
- Validation SHA-256: `3e3d5ad0d306bdd4a530fd2cf1d32bea030980af8d964d95c34b28237f6d5f`
|
|
- Regression SHA-256: `a868c4221f291953a91cf5e03f8163b2fa4e134db5f21e369fbf7c75c2387f1c`
|
|
- Source records read: `10,057`
|
|
- Accepted after strict repair: `3,799`
|
|
- Exact duplicates removed: `4,099`
|
|
- Malformed/weak-schema records removed: `2,159`
|
|
- Strict repair requires Solidity pragma, parseable JSON output with `findings`, and excludes `c4_invalid` records.
|
|
|
|
No Guard model training was started. The next required milestone is to deploy/sync Artifex state to Spark, register a new evaluation suite version using the v02 validation/regression manifests, run the fresh base-model baseline on Spark, then create the first actual training canary.
|