chore(sp13): script bug fix + Layer B implementation notes

While P0b smoke (train-sw4ws on bdc5cb8bb) runs, two prep items:

(A) scripts/argo-train.sh — add `ci-training` to the L40S sm_89 case
    arm. Bare `ci-training` is an L40S pool alias in some clusters;
    previously defaulted to sm_90 (Hopper), causing train-mnpf7 to
    deploy with wrong-arch cubins (terminated + resubmitted manually).
    Now both `*l40s*` and bare `ci-training` resolve correctly.

(C) docs/superpowers/plans/...sp13...md — Layer B section expanded
    with concrete codebase locations discovered during P0a:
    - aux_heads_kernel.cu, aux_heads_loss_ema_kernel.cu locations
    - aux_nb_label_buf populated as column 0 of next_states (log_return)
    - F1/F2 regression history note (don't alias the label buffer)
    - aux_pred_to_isv_tanh_kernel.cu is a P0a placeholder per its own
      header — Layer B should rewrite to read softmax logit-diff
    - dir_acc kernel + oracle tests need softmax-read updates
    - Layer B + P0b combined rationale post-P0a empirics

Saves the next implementer ~30 min of re-investigation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-05 08:27:49 +02:00
parent bdc5cb8bb2
commit 0ad5b6fa42
2 changed files with 33 additions and 2 deletions

View File

@@ -805,6 +805,33 @@ git push
## Layer B — Aux head regression → binary classification (1 atomic commit, only on green Phase 0)
### Implementation notes (discovered during P0a, 2026-05-04)
Concrete codebase locations the Layer B implementer needs (saves re-investigation):
- **Aux head forward**: `crates/ml/src/cuda_pipeline/aux_heads_kernel.cu``aux_heads_fwd.forward_next_bar(...)` produces `aux_nb_pred_buf [B]` (regression scalar). Layer B changes output dim 1 → 2 (softmax logits).
- **Aux head loss**: `crates/ml/src/cuda_pipeline/aux_heads_loss_ema_kernel.cu``next_bar_loss_reduce` computes MSE against `aux_nb_label_buf` and reduces. Layer B changes loss to CrossEntropy.
- **Label generation**: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs:15249-15280``aux_nb_label_buf` is gathered as **column 0 of next_states** (= `log_return` per pipeline.rs, first feature in MARKET group). Continuous signed value. Uses `vsn_logit_gather_kernel` with `col=0`. Layer B replaces with **sign-class** label generation: `class = (price[t+30] - price[t] > 0) ? 1 : 0`. Either modify the gather to apply sign, OR introduce a new `aux_nb_label_classify_kernel` reading next_states or price tensor directly.
- **WARNING from history** (`gpu_dqn_trainer.rs:15252`): "Use the DEDICATED `aux_nb_label_buf` (NOT aliased over a backward partial — the previous WIP aliased `aux_partial_nb_b2` here and triggered the F1/F2 regression)." Layer B must NOT alias the new 2-class label buffer over any backward-partial scratch.
- **Aux pred consumer** (P0a placeholder): `crates/ml/src/cuda_pipeline/aux_pred_to_isv_tanh_kernel.cu` reads scalar `aux_pred [B]`, computes `mean(tanh(aux_pred))` → ISV[375]. Layer B replaces with `mean(softmax[1] - softmax[0])` reduction (logit-diff ∈ [1, +1]). Either modify in place OR create `aux_pred_softmax_to_isv_logit_diff_kernel.cu` and update the launcher to point to the new cubin. Cleanest: rewrite `aux_pred_to_isv_tanh_kernel.cu` since it's a P0a placeholder per its own header comment.
- **dir_acc consumer**: `crates/ml/src/cuda_pipeline/aux_dir_acc_reduce_kernel.cu` reads scalar `aux_pred[i]` and tests `(p > 0.0f) ? 1 : 0`. Layer B changes to read 2-class softmax: `pred_pos = (softmax[i*2+1] > softmax[i*2+0]) ? 1 : 0`. Update the 6 oracle tests in `sp13_phase0_oracle_tests.rs` to construct softmax pairs instead of scalar predictions.
- **Direction Q-head input concat**: `crates/ml/src/cuda_pipeline/dqn_inference.cu` direction Q-head input layer currently consumes `[trunk_features]`. Layer B concatenates `ISV[AUX_DIR_PREDICTION_INDEX]` (single scalar = mean logit-diff) → input dim grows by 1. Update `dqn_param_layout.rs` direction Q-head weight shape `(in_dim, NUM_DIRECTIONS)``(in_dim + 1, NUM_DIRECTIONS)`. Bump layout fingerprint seed to invalidate cached params (greenfield acceptable).
- **No information leakage**: aux uses state s_t at inference; price[t+30] only at training-time (hindsight labels). Same pattern as the existing aux MSE head — no new leakage path introduced.
### Why this is the right next move (rationale post-P0a)
P0a empirics (`train-67gqb`):
- aux_dir_acc climbed 0.149 → 0.483 in 10 epochs at aux_w=0.05 (suppressed)
- The trajectory has decaying deltas (0.107 → 0.069 → 0.054 → ... → 0.005) — signature of bias-term decay toward zero, NOT learning the regression task
- val_win_rate stuck at 0.4638 — the Q-head can't extract directional signal from a regression head that isn't actually learning
Layer B + P0b together address this:
- P0b: aux_w 0.05 → 0.5 gives the head 10× more gradient → it can actually learn the task
- Layer B: 2-class classification with cross-entropy is a STRONGER training signal than MSE on a continuous target (especially when the target sign is what we care about — not its magnitude)
- Layer B: Q-head input now sees the head's confidence directly (logit-diff ∈ [1, +1]) instead of a `tanh(scalar)` of a noisy regression — explicit predictive feature
Combined effect: Q-head should start consuming the aux signal within ~5-10 epochs after Layer B lands, surfacing as WR > 0.50.
### Task B.T1: Aux head softmax + cross-entropy
**Files:** `aux_head_kernel.cu`, `dqn_param_layout.rs`

View File

@@ -111,9 +111,13 @@ fi
# Default pool is ci-training-h100 (sm_90). Override for other architectures:
# ci-training-h100* → sm_90 (Hopper)
# ci-training-l40s → sm_89 (Ada Lovelace)
# ci-training → sm_89 (alias for L40S — pool is named bare in some
# clusters; fixed 2026-05-04 after train-mnpf7 deployed
# with sm_90 cubins on L40S device, requiring terminate
# + resubmit with explicit --gpu-pool ci-training-l40s).
case "${GPU_POOL:-ci-training-h100}" in
*l40s*) CUDA_COMPUTE_CAP="89" ;;
*h100*|*) CUDA_COMPUTE_CAP="90" ;; # default Hopper
*l40s*|ci-training) CUDA_COMPUTE_CAP="89" ;;
*h100*|*) CUDA_COMPUTE_CAP="90" ;; # default Hopper
esac
# ── Route: single-job (existing template) vs multi-seed DAG (new template) ──