diff --git a/docs/superpowers/specs/2026-05-10-foxhunt-specialized-agents-design.md b/docs/superpowers/specs/2026-05-10-foxhunt-specialized-agents-design.md new file mode 100644 index 000000000..f1fafb8d5 --- /dev/null +++ b/docs/superpowers/specs/2026-05-10-foxhunt-specialized-agents-design.md @@ -0,0 +1,272 @@ +# Foxhunt Specialized Agents & Skills — Design + +| | | +|---|---| +| **Date** | 2026-05-10 | +| **Author** | jgrusewski (with claude-opus-4-7) | +| **Scope** | Phase 1 — 12 foxhunt-aware agents/skills (5 agents + 5 workflow skills + 2 maintenance skills) | +| **Storage** | `.claude/agents/foxhunt/`, `.claude/skills/foxhunt/`, `.claude/helpers/foxhunt-audit-router.sh` | +| **Branch** | Separate from SP15 — design-and-tooling work, not part of any SP. | + +## 1. Problem + +The repository has accumulated, under `~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/`, a deep body of project-specific knowledge: + +- **15 `feedback_*` rules** — hard behavioral constraints (no atomicAdd, no stubs, ISV-driven, no nvrtc, push-before-deploy, …). +- **30+ `pearl_*` pattern discoveries** — controllers, bootstrap, distributional Q, GPU mechanics, testing discipline, etc. +- **12+ `project_*` state files** — active hypotheses, regressions, audit findings. +- **30 SP design specs** in `docs/superpowers/specs/` (SP1 → SP15+ trader discipline). + +The `.claude/agents/` and `.claude/skills/` directories contain ~64 generic items inherited from claude-flow and superpowers plugins. None are foxhunt-aware. Consequence: every spec, plan, kernel edit, and review must re-establish constraints already documented in memory. The user has to re-explain "no atomicAdd" or "this should be ISV" *every time*. + +This design adds **foxhunt-namespaced agents and skills** that internalize the accumulated wisdom, with three goals: + +1. **Drift prevention** — auditor agents catch violations on edits/PRs, citing the originating `feedback_*`/`pearl_*` file. +2. **Workflow acceleration** — skills templatize the SP lifecycle (spec → critical review → plan → ISV-slots → kernel-wire → smoke → close-out → pearl-distill). +3. **Anti-staleness** — skills curate `MEMORY.md` and `.claude/worktrees/` so accumulated wisdom does not rot. + +## 2. Approach + +**Pattern-cluster auditors + workflow-skill hybrid.** + +- **Agents** (5) cluster pearls/feedback by *invariant being enforced*, not by file domain. E.g., the GPU-contract-auditor knows that `feedback_no_atomicadd`, `feedback_no_htod_htoh`, `pearl_no_host_branches_in_captured_graph`, and `pearl_cold_path_no_exception_to_gpu_drives` are all manifestations of one underlying invariant ("no CPU compute on the hot path; no host divergence inside graph capture"). That coherence is hard to get from per-pearl skills. +- **Skills** (7) split into workflow (5) and maintenance (2). Workflow skills templatize SP-lifecycle stages. Maintenance skills audit the meta-artifacts (memory + worktrees). + +**Why agents for review, skills for templates.** Agents run in their own context window (via the Task tool), which protects the main session when reviewing 5-20 files. Skills are markdown checklists invoked from the main turn — appropriate when the work is procedural and you want it inline. The split mirrors how the existing claude-flow vs. superpowers tools are used. + +**Hybrid trigger model.** A single `PostToolUse` hook (warn-only, <200 ms) emits a hint when an edit lands on a relevant file path. The hint is a system reminder; whether to spawn the agent stays Claude's call. Agents are also dispatchable explicitly. No PreToolUse blocking, no PreCommit-only — both would either over-friction or miss the editing window where the user gets the most value. + +## 3. Phase-1 catalogue + +### 3.1 Agents (5) — pattern-aligned auditors + +#### `foxhunt-isv-discipline-auditor` + +**Bundles**: `feedback_isv_for_adaptive_bounds`, `feedback_adaptive_not_tuned`, `pearl_controller_anchors_isv_driven`, `pearl_first_observation_bootstrap`, `pearl_wiener_optimal_adaptive_alpha`, `pearl_wiener_alpha_floor_for_nonstationary`, `pearl_blend_formulas_must_have_permanent_floor`, `pearl_zscore_normalization_for_magnitude_asymmetric_signals`, `pearl_per_branch_c51_atom_span`, `pearl_per_branch_iqn_tau_schedule`, `pearl_per_branch_loss_budget`, `pearl_per_branch_noisy_sigma`, `pearl_per_group_adam_hyperparams`, `pearl_kelly_cap_signal_driven_floors`, `pearl_trail_stop_signal_driven`, `pearl_adaptive_moe_lambda`, `pearl_l1_lambda_grad_direction_entropy_deficit`. + +**Triggers on**: `crates/ml-*/**/*.rs`, `**/sp*_isv_slots.rs`. + +**Verifies**: +- Hardcoded multipliers/caps/anchors → flag for ISV-slot conversion. +- EMA initializations use sentinel-bootstrap (`sentinel = 0; first observation replaces directly`). +- Wiener-α controllers in non-stationary loops carry a floor (≥ 0.4 default per `pearl_wiener_alpha_floor_for_nonstationary`). +- Blend formulas use `max(real, floor)`, never `α·real + (1-α)·floor`. +- Controller ratios over magnitude-asymmetric signals z-score-normalize first. +- Per-branch C51/IQN/CQL/Ens budgets derive from per-branch flatness; not shared. + +#### `foxhunt-gpu-contract-auditor` + +**Bundles**: `feedback_cpu_is_read_only`, `feedback_no_atomicadd`, `feedback_no_htod_htoh_only_mapped_pinned`, `feedback_no_cpu_test_fallbacks`, `feedback_no_nvrtc`, `feedback_cudarc_f64_f32_abi`, `pearl_no_host_branches_in_captured_graph`, `pearl_cold_path_no_exception_to_gpu_drives`, `pearl_fused_per_group_statistics_oracle`, `pearl_sp4_histogram_warp_tile_undercount`, `pearl_cublas_lt_vs_classic_sgemm`, `pearl_build_rs_rerun_if_env_changed`, `pearl_canary_input_freshness_launch_order`. + +**Triggers on**: `**/*.cu`, `**/cuda/**/*.rs`, `crates/*/build.rs`. + +**Verifies**: +- No `atomicAdd` (block tree-reduce only). +- No HtoD/HtoH outside mapped-pinned (tests not exempt). +- No `nvrtc` runtime compilation; only pre-compiled cubins. +- f64↔f32 explicit casts on cudarc kernel argument lists. +- No host branches inside CUDA-graph capture region. +- K groups × N stats → ONE fused kernel (per `pearl_fused_per_group_statistics_oracle`). +- Every `std::env::var()` paired with `cargo:rerun-if-env-changed` in build.rs. +- Producer kernels launched before any canary that consumes them. + +#### `foxhunt-reward-controller-auditor` + +**Bundles**: `pearl_audit_unboundedness_for_implicit_asymmetry`, `pearl_event_driven_reward_density_alignment`, `pearl_symmetric_clamp_audit`, `pearl_one_unbounded_signal_per_reward`, `pearl_bounded_modifier_outputs_require_structural_activation`, `pearl_loss_balance_controller`, `pearl_engagement_rate_self_correction`, `pearl_learned_gate_subsumes_handcoded`, `pearl_controller_amplifies_dominant_magnitude_trap`, `pearl_per_bar_vs_segment_pnl_signal_mismatch`, `pearl_thompson_for_distributional_action_selection`, `pearl_trail_fire_pre_vs_action_mag`, `pearl_adam_normalizes_loss_weights`, `pearl_separate_aux_trunk_when_shared_starves`, `pearl_imbalance_bar_ewma_washes_out_configured_threshold`, `pearl_reward_shape_drives_quality_over_quantity` (deprecated — for back-compat detection). + +**Triggers on**: `**/cuda/experience_kernels.cu`, `**/reward*.{rs,cu}`, `**/controller*.rs`, `**/trader_*.rs`. + +**Verifies**: +- Bilateral clamps (`fmaxf(lo, fminf(x, hi))`); one-sided forms flagged. +- Bounded modifier outputs use structural activations (sigmoid/tanh), not runtime clamps. +- Asymmetric bounded clamps where the unboundedness was implicitly providing behavioral asymmetry (loss > win caps for loss aversion). +- Event-driven reward density matches event density — no per-step shaping for trade-close objectives. +- Exactly one unbounded multiplicand per reward term. +- Thompson selector for action selection in train+eval; argmax only for Bellman target. +- Trail filter fires on `pre_mag`, not `action_mag`. +- Per-loss weight changes called out as Adam-normalized no-ops; suggest loss-function changes or per-group LR instead. +- Imbalance-bar samplers using `new_with_ewma()` flagged: data washes out the configured threshold within ~5 bars. + +#### `foxhunt-code-hygiene-auditor` + +**Bundles**: `feedback_no_stubs`, `feedback_no_todo_fixme`, `feedback_no_hiding`, `feedback_no_legacy_aliases`, `feedback_no_feature_flags`, `feedback_no_quickfixes`, `feedback_no_functionality_removal`, `feedback_no_partial_refactor`, `feedback_wire_everything_up`, `feedback_v7_gem_methodology`, `feedback_magnitude_must_be_useful`, `feedback_fix_everything_aggressively`, `pearl_tests_must_prove_not_lock_observations`, `pearl_no_deferrals_for_complementary_fixes`, `feedback_trust_code_not_docs`. + +**Triggers on**: `crates/**/*.rs`, `services/**/*.rs`, `bin/**/*.rs`, `testing/**`. + +**Verifies**: +- No stubs (return-zero, dead params, unused fields). +- No `TODO` / `FIXME` / `XXX` markers. +- No `_var` hiding or `#[allow]` suppression. +- No `*_legacy`, `*_v1`, `enable_*`, `use_*` flags. +- No partial refactors (kernel renamed, ≥1 caller un-migrated). +- New unused functions flagged for measure-or-wire-or-delete (v7-gem 3-step). +- Tests assert invariants (boundedness, monotonicity, fixed-point), not observed values. +- Plans/specs flagged when they sequence two complementary fixes that should ride together (`pearl_no_deferrals_for_complementary_fixes`). + +#### `foxhunt-sp-critical-reviewer` + +**Role**: Performs the "second/third critical review" pass on a freshly written SP spec or plan. + +**Triggers on**: `docs/superpowers/specs/*-design.md` (on Edit), `docs/plans/*.md` (on Edit). Or explicit invocation. + +**Verifies**: Every claim in the spec/plan grounds in a named pearl/feedback file. No-deferrals rule is enforced (asks "is there a sister fix that should ride along?"). Every kernel has a wire-up phase. Smoke gate has explicit kill conditions. ISV slots required are listed before scaffolding. Anti-patterns the spec is avoiding are enumerated. + +**Output**: A numbered issues list in the same shape as the user's existing critical-review iterations (e.g., the `eb5e19d67` "fix all 16 reviewer-flagged issues" pattern). Each issue cites the originating pearl/feedback. Severity-tagged; non-blocking. + +**Composes**: Reads findings from the four auditor agents when applicable (e.g., for ISV claims, asks `isv-discipline-auditor`; for kernel claims, asks `gpu-contract-auditor`). + +### 3.2 Workflow skills (5) + +#### `foxhunt-sp-spec-writer` +**Triggered by**: explicit (`/skill foxhunt-sp-spec-writer` or "write SP spec for X"). +**Produces**: A new `docs/superpowers/specs/YYYY-MM-DD-spXX--design.md` matching the established format: +1. Problem (with metrics or symptoms). +2. Pearl/feedback grounding (forced — every claim links to a memory file). +3. Approach (root-cause framing). +4. Phases A/B/C with bullet steps. +5. ISV slots required (count + index range). +6. Smoke gate with explicit kill conditions. +7. Anti-patterns avoided (enumerated). +8. Pearl-distillation hook for close-out. + +Enforces `pearl_no_deferrals_for_complementary_fixes` by asking the user explicitly: "is there a sister fix?" + +#### `foxhunt-isv-slot-scaffolder` +**Triggered by**: explicit (provide a list of new bounds/anchors/caps). +**Produces**: +- `crates/.../sp{N}_isv_slots.rs` with named constants `[START..END)`. +- `ISV_TOTAL_DIM` bumped at canonical location. +- Slot registration for fold-boundary reset. +- Commit message in the canonical shape: `feat(sp{N}): scaffold sp{N}_isv_slots.rs with K slots [S..E) — ISV_TOTAL_DIM OLD→NEW`. + +#### `foxhunt-pearl-distiller` +**Triggered by**: explicit, post SP close-out. +**Produces**: A new `pearl_.md` in the memory directory with structured body — pattern statement / detection signal / fix / canonical commit:file:line / related pearls — and a `MEMORY.md` index entry placed in the right section. Critical: this skill is one of *only two* tools that write into `memory/`. + +#### `foxhunt-argo-deploy-helper` +**Triggered by**: explicit (e.g., "deploy training X to argo"). +**Produces**: A vetted invocation of `scripts/argo-train.sh` or `scripts/argo-compile-deploy.sh` with: +- Pre-flight: verify `git push` against `origin` is at HEAD (`feedback_push_before_deploy`). +- Default to `--gpu-pool ci-training-l40s` unless explicitly overridden (`feedback_default_to_l40s_pool`). +- Hard-error on missing `--mbp10-data-dir` + `--trades-data-dir` for SP-chain training (`feedback_mbp10_mandatory`). +- Always pass `--per-enabled` (`feedback_always_per`). +- When H100 is chosen, enforce maximum-utilization flags (`feedback_h100_gpu`). + +#### `foxhunt-smoke-pilot` +**Triggered by**: explicit, paired with `argo-deploy-helper`. +**Produces**: A monitoring loop over a running smoke that: +- Tails the workflow log via `kubectl` / `argo logs`. +- Detects anomalies: NaN loss, magnitude collapse, EVAL_DIST anomalies, return-explosions, controller saturation, etc. (Anomaly detector list seeds from `feedback_stop_on_anomaly`, `feedback_kill_runs_on_anomaly_quickly`, and the `project_*` files documenting prior anomalies.) +- Auto-stops the workflow at first useful signal (`feedback_kill_runs_on_anomaly_quickly`). +- Does not duplicate Bash `run_in_background` Monitor (`feedback_no_redundant_monitor`) — uses argo's built-in stream. + +### 3.3 Maintenance skills (2) + +#### `foxhunt-memory-curator` +**Triggered by**: explicit, plus a SessionStart hint when `MEMORY.md` mtime > 30 days. +**Produces**: A curation report. For every memory entry: +- If it references a file/symbol/flag, verify it still exists. +- If it references a commit, verify the commit is still in history. +- Flag stale entries (e.g., `project_*` files where the project is now merged). +- Detect duplicate or fully-superseded pearls. +- Detect entries marked `DEPRECATED` that should be archived. + +The skill **proposes** deletes and merges; it never auto-deletes. User confirms each action. + +#### `foxhunt-stale-worktree-cleaner` +**Triggered by**: explicit (destructive op). +**Produces**: A per-worktree cleanup plan over `.claude/worktrees/*` and `git worktree list`: +- No commits in N days. +- Branch is `[gone]` (tracking ref deleted upstream). +- Branch is fully merged into `main`. +The skill prints the cleanup commands; user confirms each `git worktree remove` interactively. Composes the existing `commit-commands:clean_gone` skill where applicable. + +## 4. Hook integration + +### 4.1 Router contract + +A single shell script — `bash .claude/helpers/foxhunt-audit-router.sh` — is wired to `PostToolUse` for `Edit|Write|MultiEdit`. It reads the tool input from stdin (Claude Code hook protocol), inspects the file path, and emits zero or one hint lines to stdout. It must: + +1. Complete in <200 ms. +2. Never spawn agents itself — only suggest. +3. Never block. Exit code is always 0. +4. Coexist additively with existing hooks (existing `.claude/settings.json` is 619 bytes — not to be replaced; the new hook block is *added*, not overwritten). + +### 4.2 Routing table + +| Path glob | Suggested agent(s) | +|---|---| +| `**/*.cu`, `**/cuda/**/*.rs`, `crates/*/build.rs` | `gpu-contract-auditor` | +| `**/experience_kernels.cu`, `**/reward*.{rs,cu}`, `**/controller*.rs`, `**/trader_*.rs` | `reward-controller-auditor` (+ `gpu-contract-auditor` for `.cu`) | +| `**/sp*_isv_slots.rs`, `crates/ml-*/**/*.rs` (when diff adds numeric literals in suspicious contexts) | `isv-discipline-auditor` | +| `crates/**/*.rs`, `services/**/*.rs`, `bin/**/*.rs` (everything else) | `code-hygiene-auditor` | +| `docs/superpowers/specs/*.md`, `docs/plans/*.md` | `sp-critical-reviewer` | +| `MEMORY.md`, `memory/pearl_*.md`, `memory/feedback_*.md` | `memory-curator` | + +### 4.3 Hint format + +Single-line, parseable, prefixed for grep: + +``` +[foxhunt-audit] crates/ml-dqn/src/cuda/foo.cu — suggest: gpu-contract-auditor, reward-controller-auditor +``` + +### 4.4 Noise control + +State file `.claude/.foxhunt-audit-state` (gitignored). One suggestion per (file path, agent) per session. Cleared at SessionStart. + +### 4.5 Numeric-literal heuristic (ISV) + +The router uses a regex over the diff snippet — `[a-zA-Z_]+\s*[:=]\s*-?\d+\.?\d*` plus name-context heuristics (assignment targets like `*_threshold`, `*_floor`, `*_cap`, `*_alpha`, `*_lambda`). False positives expected; the router only suggests, never blocks. The agent itself decides on the merits. + +## 5. Memory authority + +The four auditor agents and the critical-reviewer **read** memory. They do not write into `memory/`. Only two skills write into `memory/`: + +- `foxhunt-pearl-distiller` (creates new pearls). +- `foxhunt-memory-curator` (deletes/archives stale entries — with explicit user confirmation). + +This invariant prevents memory drift from auditor agents that merely *think* a pattern is established when in fact it isn't yet documented. + +## 6. Acceptance criteria + +A successful Phase-1 rollout means: + +1. Five auditor agents are dispatchable via the Task tool. Each finding cites a specific `feedback_*.md` or `pearl_*.md` file by name (not a generic complaint). +2. The hook router emits hints in <200 ms and never blocks. +3. The seven skills produce artifacts in shapes identical to existing canonical examples: + - `foxhunt-isv-slot-scaffolder` → commit identical in shape to `c146c4fff` (SP15 ISV-slot scaffold). + - `foxhunt-sp-spec-writer` → spec identical in shape to `2026-05-06-trader-discipline-and-recovery-design.md`. + - `foxhunt-pearl-distiller` → pearl identical in shape to recent ones (e.g., `pearl_per_bar_vs_segment_pnl_signal_mismatch.md`). + - `foxhunt-sp-critical-reviewer` → numbered-issues output in the shape of the existing critical-review iterations. + - `foxhunt-argo-deploy-helper` → invocation matching `argo-train.sh` flag conventions and respecting all four `feedback_*` deploy rules. +4. The new `.claude/agents/foxhunt/` and `.claude/skills/foxhunt/` namespaces sit alongside (do not replace) the existing claude-flow / superpowers content. +5. No agent or skill writes into `memory/` other than `foxhunt-pearl-distiller` and `foxhunt-memory-curator`. +6. The new `PostToolUse` hook block is *added* to `.claude/settings.json`, not replacing existing content. + +## 7. Phase-2 backlog (deferred) + +| Item | Why deferred | +|---|---| +| `crate-graph-auditor` (agent) | Workspace-level orphan detection across 40 crates. Lower-frequency than per-file edits; defer until a concrete need surfaces. | +| Splitting `reward-controller-auditor` into `reward-shape-auditor` + `controller-design-auditor` | Only if the combined agent file grows past ~400 lines in practice. | + +`web-dashboard-debugger` is **not** in the Phase-2 backlog (explicitly out-of-scope). + +## 8. Implementation order + +Implementation plan will be developed under `writing-plans` skill, but the dependency order is: + +1. **Foundation** — `.claude/helpers/foxhunt-audit-router.sh`, additive `.claude/settings.json` block, `.claude/.foxhunt-audit-state` gitignore entry. +2. **Auditor agents** (parallel) — `gpu-contract-auditor`, `isv-discipline-auditor`, `reward-controller-auditor`, `code-hygiene-auditor`. +3. **Workflow skills** (parallel) — `sp-spec-writer`, `isv-slot-scaffolder`, `pearl-distiller`. +4. **Deploy + smoke skills** (parallel) — `argo-deploy-helper`, `smoke-pilot`. +5. **Maintenance skills** (parallel) — `memory-curator`, `stale-worktree-cleaner`. +6. **Critical reviewer** (last) — `sp-critical-reviewer` composes the auditor agents from step 2. + +## 9. Concerns flagged + +- **Pearl-bundling overlap**: `pearl_one_unbounded_signal_per_reward` and `pearl_kelly_cap_signal_driven_floors` straddle `isv-discipline-auditor` and `reward-controller-auditor`. Both will be aware; the bundling above decides primary ownership but is not exclusive. +- **Numeric-literal detection is heuristic** — false positives expected. Mitigation: agents are permissive (flag for review, never block). +- **`settings.json` merge risk** — existing 619-byte file must be merged additively. Implementation plan must read it first and append, not replace. +- **Hook latency budget** — 200 ms is tight if the router does any disk I/O beyond reading the state file. Implementation must avoid `find`, `grep -r`, or any large-tree scan.