gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(sp21): plan + revert sp20-aux-h-fixed experiment

Browse Source 
Reverts the forced H=30 diagnostic in aux_horizon_update_kernel.cu
(commit c78c4766c) — the experiment confirmed the data ceiling
hypothesis (aux_dir_acc dropped 0.46→0.50 with pred_tanh collapsing
to ~0, opposite of the bootstrap-failure prediction). Restores the
adaptive Pearl-A + Wiener-α floor logic.

Adds SP21 plan: train/eval coherence + ISV defrost. Catalogs 26
findings across 5 tiers from a pair-audit of MinIO-archived training
logs (xmd6b 30 epochs, d7bj7 2 epochs).

Cross-cutting principle: every threshold or bound in training
control flow must be signal-driven from an ISV slot, not hardcoded
(per feedback_isv_for_adaptive_bounds + feedback_adaptive_not_tuned).

Canonical pattern documented: ISV[CURIOSITY_PRESSURE_INDEX=346]
(SP11 Fix 39) is the reference implementation.

Meta-finding: across SP3-SP20 we've been monitoring training-rollout
metrics (Thompson-noisy) instead of val metrics (deterministic
backtest). val_PF=1.18-1.33 with WR=46-48% across 30 epochs of
xmd6b shows the policy DOES extract asymmetric-payoff alpha — we
just couldn't see it through the meter inflation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-10 20:12:20 +02:00
parent 14bafb5b58
commit 6df44e4c6d
3 changed files with 259 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
crates/ml/src/cuda_pipeline/aux_horizon_update_kernel.cu
View File

@@ -74,22 +74,6 @@ void aux_horizon_update(
* without branching on the host side. */
if (blockIdx.x != 0 || threadIdx.x != 0) return;
/* EXPERIMENT (sp20-aux-h-fixed, 2026-05-10): force H=30 to test the
* "bootstrap failure" hypothesis. Adaptive H gets pulled to ~2 bars
* because only Q-magnitude trades held briefly produce wins, so aux
* predicts at noise horizon → 53.5% accuracy → policy can't improve →
* H stays small. Forcing H=30 lets aux predict at a longer horizon
* (~15 minutes at 30-sec bars), where Kyle's Lambda / VPIN signals
* have demonstrated predictive power per research. If aux_dir_acc
* rises materially at fixed H=30, bootstrap failure confirmed and
* the proper fix is constraint-based (min hold time during
* exploration). If accuracy stays at 53.5%, the data really doesn't
* have signal at this horizon. */
isv[isv_h_idx] = 30.0f;
return;
/* Original adaptive logic preserved below — disabled by the early
* return above. Restore by deleting lines above this comment. */
float h_target = isv[isv_h_target_idx];
float h_current = isv[isv_h_idx];

11
docs/dqn-wire-up-audit.md
View File

@@ -2,9 +2,16 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
## 2026-05-10 — DIAGNOSTIC: aux_horizon_update_kernel forced H=30 (sp20-aux-h-fixed branch)
## 2026-05-10 — REVERT: aux_horizon_update_kernel sp20-aux-h-fixed experiment closed
Throwaway experimental edit. Short-circuits the kernel to always write `ISV[AUX_PRED_HORIZON_BARS_INDEX=450] = 30.0` instead of running adaptive logic. Tests the bootstrap-failure hypothesis: in production, the adaptive H gets pulled to ~2 bars (only brief Q-magnitude trades win → avg winning hold = ~2 bars → H = ~2 bars → aux predicts at noise horizon → 53.5% accuracy → loop closes). If forcing H=30 lifts aux_dir_acc materially, bootstrap failure is real. Diagnostic-only — DO NOT merge to mainline. Original adaptive logic preserved as unreachable code with restoration comment.
Reverts the forced-H=30 diagnostic that tested the bootstrap-failure hypothesis. **Hypothesis verdict: dead.** Across two epochs of `train-d7bj7`:
- Epoch 1: `aux_dir_acc short=0.4586 long=0.4605 pred_tanh=0.3746` — below random, confidently wrong
- Epoch 2: `aux_dir_acc short=0.5021 long=0.5057 pred_tanh=-0.0166` — the regression head **gave up** (pred_tanh collapsed to ~0)
Bootstrap-failure predicted aux would learn signal at H=30; reality showed the head shrinking predictions to escape an unlearnable target. Adaptive H~2 was producing 53.5%; forced H=30 produced ~50% with collapsing confidence. The data really doesn't have aux-predictable signal at H=30 with this feature set. Adaptive logic restored — see `pearl_first_observation_bootstrap`, `pearl_wiener_optimal_adaptive_alpha`, and `feedback_isv_for_adaptive_bounds` for the full reasoning chain.
This revert opens SP21 (train/eval coherence + ISV defrost) — see `docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md`.
## 2026-05-10 — SP20 Phase 5: aux→Q reward gate (close-out)

250
docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md Normal file
View File

@@ -0,0 +1,250 @@
# SP21 — Train/Eval Coherence + ISV Defrost
**Author**: jgrusewski + assistant pair audit
**Date**: 2026-05-10
**Branch**: `feat/sp18-combined` (off `sp19-20-wr-first` worktree)
**Predecessors**: SP19/SP20 (Phase 5 aux→Q gate, 4-quadrant reward, imbalance bars wiring)
## Meta-finding
A pair audit of MinIO-archived training logs (`xmd6b` 2026-05-08, 30 epochs; `d7bj7` 2026-05-10, 2 epochs) revealed three intertwined failure modes that explain why every SP from SP3-SP20 has produced WR pinned at 50% and PF≈1.00 in the surfaced training metrics — **but the policies have actually had asymmetric edge all along**.
### What was hidden
Across 30 epochs of `xmd6b`:
```
Epoch 0: val [sharpe=81.91 win_rate=0.4594 profit_factor=1.328 total_pnl=0.559 trades=139251 active_frac=0.560]
Epoch 5: val [sharpe=59.76 win_rate=0.4645 profit_factor=1.232 total_pnl=0.248 trades=100118 active_frac=0.301]
Epoch 30: val [sharpe=25.54 win_rate=0.4814 profit_factor≈1.18 total_pnl≈0.16 trades≈55000 active_frac≈0.10]
```
The policy is `WR=46-48%, PF=1.18-1.33` — **asymmetric-payoff strategy** (more frequent small losers, fewer larger winners). The previously-tracked `WR=50.0%, PF=1.00` came from training-rollout metrics with Thompson exploration noise + cost curriculum, which average to break-even regardless of what the learned policy is doing.
### What's broken
Three intertwined failure modes:
1. **Wrong-metric early stopping**: patience-based stopping watches `epoch_sharpe` (training-rollout, exploration-noisy) instead of `val_loss` (deterministic backtest). Training ran 30 epochs past peak val (epoch 2: Sharpe=90, total_pnl=0.44), throwing away ~70% of the alpha by training-induced overfitting.
2. **Frozen ISV controllers**: 6+ ISV slots stuck at sentinel/floor values across all epochs (`wr_ema=0`, `hold_pct_ema=0`, `hold_reward_ema=0`, `loss_cap=-1` sentinel, `hold_cost_scale` and `aux_conf_threshold` at floor). Per `feedback_isv_for_adaptive_bounds.md`, these controllers should be driving adaptive behavior — they're not adapting at all.
3. **Fake-trade fabrication**: enrichment pipeline (`extract_eval_trades_from_metrics`) FABRICATES per-trade data from aggregate stats with hardcoded placeholders (`60000.0, 0.0, 0.5`) at the call site. Whatever "self-improving enrichment" feeds back into hyperparams is driven by inputs that don't reflect reality.
### Goal of this superproject
Restore train/val metric coherence and defrost ISV controllers so the system can actually converge on its intended adaptive equilibrium. This is foundational, not optimization — without these fixes, every previous SP's "improvements" were evaluated against blind metrics.
### Canonical signal-driven pattern (template for Tier 3 fixes)
The curiosity-pressure ISV slot (`ISV[CURIOSITY_PRESSURE_INDEX=346]`, SP11 Fix 39) is the **reference implementation** for what every signal-driven slot should look like. When fixing Tier 3 ISVs, mirror this pattern:
1. **Producer kernel** computes from a real upstream signal — not a constant. Curiosity reads val-sharpe trend + magnitude-ratio dispersion.
2. **Pearl-A first-observation bootstrap** — sentinel value (0.0 or designated sentinel) replaced directly on first valid observation, NOT blended (per `pearl_first_observation_bootstrap.md`).
3. **Wiener-α EMA** for steady-state updates (per `pearl_wiener_optimal_adaptive_alpha.md`), or fixed slow blend if no variance EMA available.
4. **Fractional upper bound** scaled to a related magnitude EMA — curiosity uses `0.3 × pnl_reward_magnitude_ema`. Prevents runaway controllers.
5. **Permanent floor** as a fraction of bound — curiosity uses `0.2 × bound` (per `pearl_blend_formulas_must_have_permanent_floor.md`). Prevents deadlock on signal disappearance.
6. **HEALTH_DIAG visibility** — slot value printed every epoch so freezes are observable. Curiosity is in the `sp11_reward [...curiosity_p=...]` line.
**Verification that the curiosity pattern is alive** (xmd6b logs):
```
HEALTH_DIAG[0]: ... curiosity_p=0.000 ... ← Pearl-A sentinel bootstrap
HEALTH_DIAG[1]: ... curiosity_p=0.109 ... ← live, post-bootstrap
HEALTH_DIAG[2]: ... curiosity_p=0.406 ... ← rising as trend signal develops
```
If a Tier 3 fix doesn't show this same epoch-1-bootstrap-then-live trajectory, the fix is incomplete.
### Cross-cutting principle
**Every threshold or bound in training control flow must be signal-driven from an ISV slot, not hardcoded.** Hardcoded constants found during this audit:
| Site | Constant | Should derive from |
|---|---|---|
| `early_stopping.rs:59` `min_delta` | `0.001` | `ISV[VAL_SHARPE_VAR_EMA_INDEX=351]` (T1.1b) |
| `enrichment.rs:17-26` epsilon adapter | `2.0`, `0.5`, `-0.5`, `0.8`, `0.95`, `1.2` | val_sharpe distribution percentiles (T1.3) |
| `enrichment.rs:40` gamma blend | `5.0`, `[0.85, 0.98]` | hold-time distribution (T1.3) |
| `enrichment.rs:57` magnitude filter | `pnl < -0.001` | per-trade pnl variance EMA (T1.3) |
| `enrichment.rs:61` holding-bars filter | `< 3` | observed avg hold time ISV slot (T1.3) |
| `enrichment.rs:67` direction-rate clamp | `[0.5, 2.0]` | per-direction WR variance (T1.3) |
| `enrichment.rs:81` ensemble growth | `× 1.1` | ensemble agreement variance (T1.3) |
| `mod.rs:142` `min_improvement_rate` | `0.1` | val_sharpe variance EMA (T1.4) |
| `training_loop.rs:7425` save threshold | `improvement_rate > 0.01` | val_sharpe variance EMA (T1.4) |
| `training_loop.rs:2187` regime threshold | `adx_threshold = 30.0` | ADX distribution EMA (Tier 5) |
| `training_loop.rs:987` cost-anneal schedule | `1/(1+exp(-((epoch-10)/3)))` — `10`, `3` | training-stability EMA (T1.5) |
| `trade_physics.cuh:714` capital floor | `max_equity × 0.75f` (25% DD) | drawdown distribution EMA (T1.6) |
| `state_layout.cuh:317` `MIN_HOLD_TARGET` | `30.0f` zombie #define (claimed replaced by ISV slot 460 per Class A audit, but still exists) | DELETE — `feedback_no_legacy_aliases` (T2.4) |
| `config.rs:1479-1480` PER schedule | `per_alpha=0.6, per_beta_start=0.6` | priority distribution variance EMA (T2.5) |
| `experience_kernels.cu:450-454` cost curriculum ranges | `PHILOX_UNIFORM(0.5, 2.5)`, etc. | broker-physics priors — fundamental constants, NOT tuning (no fix needed; but document the rationale) |
| `c51_loss_kernel.cu:13,28` | `MAX_PER_SAMPLE_CE=50.0`, `LABEL_SMOOTHING_BASE=0.02` | label-smoothing already has `pearl_loss_balance_controller` infrastructure; CE clamp from gradient norm distribution |
| ~~`gpu_curiosity_trainer.rs:342,401`~~ | ~~`let beta: f32 = 0.0`~~ | **REMOVED 2026-05-10**: false alarm. These are cuBLAS `cublasSgemm_v2` β parameters (`C = α·A·B + β·C` with β=0 = "don't accumulate"), unrelated to curiosity reward weight. Actual curiosity is `ISV[CURIOSITY_PRESSURE_INDEX=346]`, already ISV-driven per SP11 Fix 39, and confirmed firing in xmd6b logs (`curiosity_p: 0.000 → 0.109 → 0.406` across epochs 0-2). |
This principle is the SP21 unifying invariant. Per `feedback_adaptive_not_tuned.md`: *"fixes are signal-driven (EMA/windowed), not tuned constants."* If we accept that principle, ALL the above must be ISV-driven; if we reject it for any specific case, the rejection must cite a concrete reason (e.g., fundamental floor per `feedback_isv_for_adaptive_bounds.md`).
---
## Findings catalog
### Tier 1 — Wrong-source + signal-driven fixes
| # | Finding | Site | Root cause | Fix |
|---|---|---|---|---|
| T1.1a | Patience early-stopping reads `epoch_sharpe` (training rollout) instead of `val_loss` | `crates/ml/src/trainers/dqn/trainer/training_loop.rs:7234` | Copy-paste bug — the comment at `training_loop.rs:1499-1500` explicitly says "Use val_Sharpe (deterministic backtest), NOT epoch_sharpe". Backtracking honors this; patience early-stop doesn't. | Change `-log_output.epoch_sharpe` → `log_output.val_loss`. The latter is already `-val_sharpe` per line 1501; passing directly works for "lower is better" convention. |
| T1.1b | `EarlyStopping::min_delta` is a hardcoded constant `0.001` — violates `feedback_adaptive_not_tuned` | `crates/ml/src/trainers/dqn/early_stopping.rs:31-67` (struct), `crates/ml/src/trainers/dqn/trainer/constructor.rs:684` (construction site) | min_delta as constant means a noisy val_loss with σ=0.5 looks "improving" with min_delta=0.001 and we never trigger; a stable val_loss with σ=0.0001 would trigger spuriously. The threshold must scale with the noise floor. | Refactor `should_stop(val_loss, epoch)` → `should_stop(val_loss, min_delta, epoch)`. Compute `min_delta` per-call from `ISV[VAL_SHARPE_VAR_EMA_INDEX=351]` (variance EMA already produced by `launch_sp11_val_sharpe_delta_compute` per `training_loop.rs:6421`). Formula: `min_delta = k × sqrt(ISV[VAL_SHARPE_VAR_EMA])` where `k≈1.0` (1σ improvement counts as real). Bootstrap: when ISV slot is at sentinel/zero, fall back to a permanent floor per `pearl_blend_formulas_must_have_permanent_floor`. |
| T1.2 | Enrichment caller passes hardcoded `(60000.0, 0.0, 0.5)` placeholders for trade_count, total_pnl, win_rate | `training_loop.rs:1514-1521` | TODO never finished — actual values exist in val `WindowMetrics` but caller used constants | Wire `actual_val_metrics.trade_count, actual_val_metrics.total_pnl, actual_val_metrics.win_rate` through. These are observations, not bounds — no adaptation needed, just stop fabricating them. Verify the val pipeline exposes these on the consumed-loss path (we saw them in HEALTH_DIAG output, so they're being computed and just need to be plumbed to the enrichment call site). |
| T1.3 | Enrichment functions (E1-E8) contain ~10 hardcoded thresholds + magic-number multipliers for epsilon adaptation, gamma blending, magnitude filters, ensemble agreement | `enrichment.rs:17-26, 40, 57, 61, 67, 81, 156, 230` (entire 381-line file) | Per `feedback_adaptive_not_tuned`, every gating threshold should be signal-driven. The enrichment file was written quickly and never adapted. | **DEPENDS on T2.2 outcome** — if we delete `extract_eval_trades_from_metrics`, the enrichment function rewrites must concurrently signal-drive each threshold from the appropriate ISV slot OR a per-call computed quantile/variance. Specifics per threshold listed in cross-cutting principle table above. |
| T1.4 | Backtracking improvement-rate thresholds hardcoded | `training_loop.rs:7425` (`> 0.01`), `trainer/mod.rs:142` (`min_improvement_rate: 0.1`) | Single-fold backtracking save criteria. Not signal-driven. | Both should derive from `ISV[VAL_SHARPE_VAR_EMA_INDEX=351]`. Save threshold = `k1 × sqrt(var_ema)`, route-acceptance threshold = `k2 × sqrt(var_ema)` with k2 > k1 (route acceptance is stricter than save trigger). |
| T1.5 | Cost-anneal schedule hardcoded sigmoid | `training_loop.rs:987` — `1/(1+exp(-((epoch-10)/3)))` | The `10` (transition epoch) and `3` (slope) gate the entire cost curriculum schedule. Choosing them by tuning rather than signal means we don't know if the policy is ready for full cost when full cost arrives. | Drive transition from training-stability EMA — when val_Sharpe variance is below threshold for N consecutive epochs, ramp cost. Concretely: read `ISV[VAL_SHARPE_VAR_EMA]`, compute `cost_anneal = sigmoid((stability_score - 0.5) × k)` where `stability_score = exp(-var_ema/var0)` for some signal-derived `var0`. |
| T1.6 | Capital floor hardcoded at 75% of peak equity | `trade_physics.cuh:714` (`max_equity * 0.75f`) | 25% drawdown trigger is a fundamental risk-management threshold AND a learning-signal gate (capital_floor breach forces episode boundary). Domain-specific but configurable. | This is a judgment call: is 25% DD a *fundamental* floor (regulatory/broker reality) or a *tuning* parameter? Recommendation: keep `0.75f` as fundamental (broker margin call territory) but add a comment explaining it. NO fix unless we find evidence it's incorrect for the actual broker integration. |
**Success criterion**: training run shows `🛑 Early stopping triggered` line near the val_Sharpe peak (within `patience` epochs), `min_delta` value at each call is logged (or asserted) to be derived from ISV slot 351 (not constant 0.001), and enrichment receives non-placeholder values (verifiable by adding an assertion or log line printing what enrichment received).
**Pearl compliance**:
- T1.1a follows `feedback_no_partial_refactor` (the comment-says-one-thing-code-does-another inconsistency must close)
- T1.1b follows `feedback_isv_for_adaptive_bounds` (threshold is an adaptive bound) and `feedback_adaptive_not_tuned` (signal-driven, not constant) and `pearl_blend_formulas_must_have_permanent_floor` (sentinel handling)
- T1.2 follows `feedback_no_todo_fixme` (placeholder constants in production code) and `feedback_no_stubs` (fabricated inputs are stub-shaped)
---
### Tier 2 — Architectural debt requiring design decisions
| # | Finding | Site | Decision needed |
|---|---|---|---|
| T2.1 | `check_early_stopping(avg_q_value)` — separate stop path on Q-value magnitude | `crates/ml/src/trainers/dqn/trainer/metrics.rs:436` | Q-value magnitude is not a learning signal — high Q can mean high alpha OR catastrophic value-explosion. Three options: (a) delete entirely (subsumed by val-loss path); (b) repurpose as Q-explosion guard with **signal-driven** threshold from `ISV[Q_VALUE_VAR_EMA]` per `feedback_isv_for_adaptive_bounds`; (c) keep but rename so its purpose is clear. Recommendation: **(a) delete** — `feedback_no_legacy_aliases` says no deprecated wrappers. If we keep it as a Q-explosion guard, it MUST be signal-driven (no hardcoded threshold). |
| T2.2 | `extract_eval_trades_from_metrics` fabricates per-trade tape from aggregate stats; downstream enrichment functions are themselves constants-soup (T1.3) | `crates/ml/src/trainers/dqn/trainer/enrichment.rs:99-138` (synth) + entire 381-line file | Function is mathematically a fiction — it generates synthetic trades that "average to" the input aggregates. Each downstream enrichment function has multiple hardcoded thresholds. | Two options: (a) emit real per-trade tape from `gpu_evaluator` (significant new code) AND signal-drive each downstream threshold; (b) refactor enrichment functions to consume aggregate stats directly without the fake-trade detour AND signal-drive each downstream threshold (this is the bigger half — consequential refactor since each enrichment fn must read from ISV). Recommendation: **(b)**. Scope: T2.2 + T1.3 are mathematically the same superset of changes — enrichment.rs gets rewritten as a set of ISV-aware functions. |
| T2.3 | `EarlyStopping::should_stop` test signature changes (T1.1b) | `crates/ml/src/trainers/dqn/early_stopping.rs:158, 183, 200, 216, 236, 250` (6 unit tests) | Refactoring `should_stop(val_loss, epoch)` → `should_stop(val_loss, min_delta, epoch)` breaks the test signatures. | Update all 6 tests to pass explicit `min_delta` per call. Per `feedback_no_partial_refactor`, this is part of the atomic T1.1b commit, not a separate task. |
| T2.4 | `MIN_HOLD_TARGET=30.0f` zombie #define after Class A audit | `state_layout.cuh:317` | Comment at lines 177, 296-322 explicitly says this constant was REPLACED by ISV slot 460 (`MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX`). But the #define still exists. Either: (a) zombie code that should be deleted; (b) used as fallback when ISV at sentinel — comment line 320 hints at fallback usage. Audit needed. | Trace usages of `MIN_HOLD_TARGET` macro. If only used via the fallback path, rename to `MIN_HOLD_TARGET_FALLBACK` and document. If unused, DELETE per `feedback_no_legacy_aliases`. |
| T2.5 | PER `per_alpha=0.6` and `per_beta_start=0.6` hardcoded | `config.rs:1479-1480` | PER (Prioritized Experience Replay) parameters control training-data sampling distribution. The original Schaul et al. paper uses these as fixed but other works adapt them. With `feedback_always_per` mandating PER ON, the *parameters* are still fixed. | Two options: (a) keep fixed as paper-canonical (but then the hyperparams should be derived from a research citation, not a magic number); (b) adapt from priority distribution variance EMA. Recommendation: **keep fixed for SP21**, but file the question for a separate SP if PER tuning is later identified as a leverage point. Document choice in `pearl_per_alpha_beta_paper_canonical`. |
| ~~T2.6~~ | ~~Curiosity trainer beta=0.0 hardcoded — silently disabled?~~ | ~~`gpu_curiosity_trainer.rs:342, 401`~~ | **WITHDRAWN 2026-05-10 (false alarm)**: the `beta=0.0` is the cuBLAS `cublasSgemm_v2` β API parameter (`C = α·A·B + β·C`), not a curiosity reward weight. Actual curiosity coefficient is `ISV[CURIOSITY_PRESSURE_INDEX=346]`, produced by SP11 Fix 39's `reward_subsystem_controller_kernel`, already signal-driven per `pearl_blend_formulas_must_have_permanent_floor`. Confirmed firing in xmd6b logs: `curiosity_p: 0.000 → 0.109 → 0.406` across epochs 0-2 (Pearl-A bootstrap from sentinel, then live). | **No fix needed.** System working as designed. |
**Success criterion** (T2.1): file no longer contains `check_early_stopping(avg_q_value, ...)`; tests that exercised it migrated or deleted. (T2.2): `extract_eval_trades_from_metrics` removed; enrichment functions take `&WindowMetrics` directly; behavior preserved for E2/gamma/ensemble code paths.
**Pearl compliance**: T2.1 honors `feedback_no_legacy_aliases`. T2.2 honors `feedback_no_stubs` (fake-trade synthesis is a stub-shaped pattern — produces shaped output without real signal).
---
### Tier 3 — Frozen ISV slots (defrost)
Each ISV slot below stayed constant across all 3 observed epochs of `d7bj7`. The producer kernel for each must be located, the freeze cause identified, and the fix applied per relevant pearl.
| # | ISV slot | Observed | Expected | Producer kernel (to locate) |
|---|---|---|---|---|
| T3.1 | `wr_ema` | 0.0000 (frozen) | EMA of segment-close win rate ∈ [0, 1] | `pearl_per_bar_vs_segment_pnl_signal_mismatch` patch added `is_win_per_env` buffer; verify producer is reading it AND writing to `wr_ema_idx` |
| T3.2 | `hold_pct_ema` | 0.0000 (frozen) | EMA of fraction of bars where policy holds an active position | Likely an EMA producer reading from action_counts or position_active_mask |
| T3.3 | `hold_reward_ema` | 0.0000 (frozen) | EMA of opportunity-cost reward magnitude per bar | Likely populated alongside hold_pct |
| T3.4 | `loss_cap` | -1.0000 (sentinel) | Should be replaced with adaptive value after first observation per Pearl-A | First-observation bootstrap kernel — sentinel-detection guard may have wrong condition |
| T3.5 | `hold_cost_scale` | 0.0100 (at min floor) | Controller-driven within [min, max] bounds | Controller reads `hold_pct_ema` (which is also frozen at 0) — chain failure: T3.2 must fix first |
| T3.6 | `aux_conf_threshold` | 0.0100 (at min floor) | Controller drives based on aux confidence distribution | Producer kernel reads aux confidence histogram; check if histogram producer is firing |
**Strategy**: T3.1 fixed first as canonical example (it has a recent pearl + recent commit `64bbbe418` claiming the fix). Verify the patch actually propagated to the ISV slot, not just to `is_win_per_env` buffer. Once T3.1 is verified working, use the same pattern (find producer → check input availability → check write-to-isv → check sentinel guard) for T3.2-T3.6 in parallel.
**Success criterion**: HEALTH_DIAG `sp20_isv` line shows non-zero, varying values for all 6 slots across consecutive epochs. ISV slot variance > 0 over a 5-epoch window.
**Pearl compliance**:
- `feedback_isv_for_adaptive_bounds` — bounds must adapt
- `pearl_first_observation_bootstrap` — sentinel must be replaced on first valid observation
- `pearl_wiener_optimal_adaptive_alpha` — EMAs use Wiener-optimal α where applicable
- `pearl_blend_formulas_must_have_permanent_floor` — but only AFTER first observation; sentinel is not a floor
---
### Tier 4 — Controller-chain non-adaptation
Five HEALTH_DIAG controller readouts in `xmd6b` showing structural non-adaptation:
| # | Controller | Observed | Expected | Diagnosis |
|---|---|---|---|---|
| T4.1 | `c51_budget_per_branch` | all 0.0000 | per-branch C51 loss budget ∈ [0, 1] adapting on flatness per `pearl_per_branch_loss_budget` | Controller likely never producing any non-zero budget — input signal (Q-flatness) might be zero |
| T4.2 | `cql_budget_per_branch` | all 0.0000 | per-branch CQL loss budget | Same as T4.1 — controller chain dead |
| T4.3 | `q_var_per_branch` | all 0.0000 | per-branch Q-value variance | Either Q-variance producer dead, or all branches producing identical values |
| T4.4 | `iqn_budget_per_branch` | all 1.0000 (max) | adaptive IQN τ-schedule per `pearl_per_branch_iqn_tau_schedule` | Controller stuck at saturation |
| T4.5 | `lb_active_per_branch` | all 1.0000 | per-branch loss-balance active flag — should reflect per-branch loss budget | Likely synthesized from T4.1/T4.2 — fixes there might cascade |
**Strategy**: After T3 defrosts the foundational ISV slots, re-observe T4 readouts. T4 controllers are likely DOWNSTREAM of T3 inputs — fixing the source ISVs may cascade-fix the controllers. If not, audit each controller individually using the same pattern as T3.
**Success criterion**: HEALTH_DIAG controller readouts show non-trivial inter-branch variation across epochs.
**Pearl compliance**: `pearl_per_branch_c51_atom_span`, `pearl_per_branch_iqn_tau_schedule`, `pearl_per_branch_loss_budget`, `pearl_per_branch_noisy_sigma` all apply.
---
### Tier 5 — Design issues (not bugs)
| # | Finding | Implication | Disposition |
|---|---|---|---|
| T5.1 | Train uses Thompson + ε=0.05; val uses pure greedy (ε=0) | Direct metric comparison is apples-to-oranges | Per `pearl_thompson_for_distributional_action_selection.md`, both should use Thompson. **Decision**: align val to Thompson per the pearl, OR document the explicit deviation if greedy-val is required for some other reason. Recommendation: **align** — measuring the deployed policy under its true action distribution. |
| T5.2 | `epoch_sharpe` is per-trade Sharpe × √(trades_per_year), inflating values to [-90, +90] from raw values [-0.05, +0.05] | The metric we've been printing as the headline number is structurally meaningless | Either: (a) print `sharpe_raw` as the headline; (b) reduce trades_per_year annualization to actual trading days; (c) keep both, label clearly. Recommendation: **(c)** — keep both for backward-compat with archived runs, but make `sharpe_raw` the early-stop and best-checkpoint signal. |
| T5.3 | Hardcoded `adx_threshold = 30.0` for regime detection | `training_loop.rs:2187` | Magic number for ADX. ADX is a derived feature; its meaningful threshold depends on the feature's distribution at the current bar resolution. | Drive from `ISV[ADX_DISTRIBUTION_QUANTILE]` or equivalent — should mark "high ADX" as e.g. 75th percentile of observed ADX EMA, not absolute 30. If no producer exists, T5.3 grows to include adding one. |
**Pearl compliance**: T5.1 honors `pearl_thompson_for_distributional_action_selection`.
---
## Sequencing
Per `feedback_no_deferrals_for_complementary_fixes.md`, all findings are scoped into one superproject. But execution is per-tier with clear gates so a regression at any tier doesn't poison subsequent tiers:
1. **Tier 1 first** (smaller-blast-radius fixes):
- T1.1a + T1.1b atomic (early stop swap source AND signal-drive min_delta + update tests)
- T1.2 (wire real values to enrichment caller — observations only)
- T1.4 (signal-drive backtracking thresholds)
- **T1.3 deferred to T2.2** — see below
Verify: training run shows early-stop near val_Sharpe peak; backtracking saves at correct cadence; enrichment caller receives real values (add a one-time log line proving it).
2. **Tier 2 architectural debt** — execute T2.1 (delete avg_q_value path), T2.2+T1.3 jointly (rewrite enrichment.rs as ISV-aware functions). T2.3 is part of T1.1b's atomic commit.
Verify: enrichment outputs differ from previous run (proving it's now signal-driven); early-stop and best-ckpt selection use only val_loss path.
3. **Tier 3 ISV defrost**, T3.1 first as canonical example, then T3.2-T3.6 in parallel using the same template (find producer → check input availability → check write-to-isv → check sentinel guard).
Verify: HEALTH_DIAG `sp20_isv` line shows variance > 0 across consecutive epochs for ALL 6 frozen slots.
4. **Tier 4 controller cascade** — re-check after Tier 3 fix; investigate residuals. Many T4 controllers may unfreeze automatically once their T3 inputs unfreeze (cascade hypothesis).
Verify: HEALTH_DIAG controller readouts show inter-branch variation.
5. **Tier 5 design alignments** — Thompson-in-val and metric clarity (T5.1, T5.2), regime threshold ISV-driving (T5.3). Larger scope; may be split into a separate SP if T5.3 requires building a new ADX EMA producer.
Each tier completion = one commit (or commit set), with smoke test verification before proceeding to next tier. **Tier 1 has roughly 100-200 lines of code change**; **Tier 2+T1.3 has ~400 lines** (enrichment rewrite); **Tier 3 has ~50 lines per slot × 6 slots = ~300 lines**; **Tier 4-5 sized after Tier 3 results.**
---
## Open questions before execution
1. **T1.1 design**: T1.1a (swap source) is a 1-line change. T1.1b (signal-driven min_delta) is a refactor of `EarlyStopping::should_stop` to accept per-call `min_delta`. Two sub-questions:
(a) On the very first call when `ISV[VAL_SHARPE_VAR_EMA]=0` (sentinel), what permanent floor for `min_delta`? Per `pearl_blend_formulas_must_have_permanent_floor`, suggest `max(1e-3, k × sqrt(var_ema))` with `k=1.0`.
(b) Multiplier `k` — 1σ improvement (k=1) vs 0.5σ (k=0.5) vs 2σ (k=2). Higher k = stricter "real improvement" threshold, fewer false-positive resets, more false-negative early-stops. Default k=1.0 as starting point but **k itself could be ISV-driven from val_sharpe trend stability** if we want fully adaptive — that's a sub-pearl decision.
2. **T2.2 scope**: refactoring `extract_eval_trades_from_metrics` away requires touching every enrichment function. How many call sites? Risk of partial refactor per `feedback_no_partial_refactor`.
3. **T3 producer locations**: ISV slot indices are known (`AUX_PRED_HORIZON_BARS_INDEX=450`, etc.) but the producer kernels for each frozen slot need to be located. Some are pearl-named; others may not be wired to anything.
4. **T5.1 implication**: aligning val to Thompson may *change* the val_Sharpe numbers we've been seeing. Pre/post comparison matters — we don't want to "fix" a metric and then look like the policy degraded when it actually didn't.
---
## Success criteria — superproject level
The whole SP21 is complete when, on a fresh training run:
1. `🛑 Early stopping triggered` fires within `patience` epochs of val_Sharpe peak
2. HEALTH_DIAG `sp20_isv` line shows variance > 0 across epochs for ALL 6 frozen slots
3. HEALTH_DIAG controller readouts (`c51_budget`, `cql_budget`, `q_var`, `iqn_budget`, `lb_active`) show inter-branch variation
4. Enrichment log line proves real values are flowing in
5. Train/val metrics are coherent in DIRECTION (both rise OR both fall during a learning epoch; not opposite signs)
6. `evaluate_baseline` (or equivalent eval entry point) reproduces the val_Sharpe / val_PF observed in training (within 5% relative tolerance) when run on the same checkpoint + window
If criterion 6 fails, there's a hidden train/val pipeline divergence we haven't surfaced — open a SP22 to investigate.
---
## Non-goals
- Action-space redesign (Hold/Flat split, magnitude bucketing) — separate SP
- Feature pipeline changes (MBP10, OFI, TLOB, imbalance bar threshold tuning) — separate SP
- Reward shape redesign (asymmetric clamp, opportunity cost, micro-reward weights) — separate SP
- Hyperparameter sweep — moot until coherence is restored
---
## Pearls created or amended by this superproject
Likely candidates (to formalize after execution):
- New pearl: `pearl_validate_metric_inputs_not_just_outputs` — periodically verify that EMA/controller producers are receiving non-trivial inputs, not just that outputs change. Frozen `wr_ema=0` is what `is_win_per_env=0` always looks like — both are silent failures.
- New pearl (or amend `feedback_no_todo_fixme`): hardcoded placeholder constants in production paths must be flagged at compile time, not runtime.
- Possibly amend `pearl_first_observation_bootstrap`: sentinel-detection thresholds must match the producer kernel's actual sentinel values (T3.4 `loss_cap=-1` may have a guard checking for `0` or `NaN` instead of `-1`).