chore(sp13): script bug fix + Layer B implementation notes
While P0b smoke (train-sw4ws on bdc5cb8bb) runs, two prep items:
(A) scripts/argo-train.sh — add `ci-training` to the L40S sm_89 case
arm. Bare `ci-training` is an L40S pool alias in some clusters;
previously defaulted to sm_90 (Hopper), causing train-mnpf7 to
deploy with wrong-arch cubins (terminated + resubmitted manually).
Now both `*l40s*` and bare `ci-training` resolve correctly.
(C) docs/superpowers/plans/...sp13...md — Layer B section expanded
with concrete codebase locations discovered during P0a:
- aux_heads_kernel.cu, aux_heads_loss_ema_kernel.cu locations
- aux_nb_label_buf populated as column 0 of next_states (log_return)
- F1/F2 regression history note (don't alias the label buffer)
- aux_pred_to_isv_tanh_kernel.cu is a P0a placeholder per its own
header — Layer B should rewrite to read softmax logit-diff
- dir_acc kernel + oracle tests need softmax-read updates
- Layer B + P0b combined rationale post-P0a empirics
Saves the next implementer ~30 min of re-investigation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -805,6 +805,33 @@ git push
|
||||
|
||||
## Layer B — Aux head regression → binary classification (1 atomic commit, only on green Phase 0)
|
||||
|
||||
### Implementation notes (discovered during P0a, 2026-05-04)
|
||||
|
||||
Concrete codebase locations the Layer B implementer needs (saves re-investigation):
|
||||
|
||||
- **Aux head forward**: `crates/ml/src/cuda_pipeline/aux_heads_kernel.cu` — `aux_heads_fwd.forward_next_bar(...)` produces `aux_nb_pred_buf [B]` (regression scalar). Layer B changes output dim 1 → 2 (softmax logits).
|
||||
- **Aux head loss**: `crates/ml/src/cuda_pipeline/aux_heads_loss_ema_kernel.cu` — `next_bar_loss_reduce` computes MSE against `aux_nb_label_buf` and reduces. Layer B changes loss to CrossEntropy.
|
||||
- **Label generation**: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs:15249-15280` — `aux_nb_label_buf` is gathered as **column 0 of next_states** (= `log_return` per pipeline.rs, first feature in MARKET group). Continuous signed value. Uses `vsn_logit_gather_kernel` with `col=0`. Layer B replaces with **sign-class** label generation: `class = (price[t+30] - price[t] > 0) ? 1 : 0`. Either modify the gather to apply sign, OR introduce a new `aux_nb_label_classify_kernel` reading next_states or price tensor directly.
|
||||
- **WARNING from history** (`gpu_dqn_trainer.rs:15252`): "Use the DEDICATED `aux_nb_label_buf` (NOT aliased over a backward partial — the previous WIP aliased `aux_partial_nb_b2` here and triggered the F1/F2 regression)." Layer B must NOT alias the new 2-class label buffer over any backward-partial scratch.
|
||||
- **Aux pred consumer** (P0a placeholder): `crates/ml/src/cuda_pipeline/aux_pred_to_isv_tanh_kernel.cu` reads scalar `aux_pred [B]`, computes `mean(tanh(aux_pred))` → ISV[375]. Layer B replaces with `mean(softmax[1] - softmax[0])` reduction (logit-diff ∈ [−1, +1]). Either modify in place OR create `aux_pred_softmax_to_isv_logit_diff_kernel.cu` and update the launcher to point to the new cubin. Cleanest: rewrite `aux_pred_to_isv_tanh_kernel.cu` since it's a P0a placeholder per its own header comment.
|
||||
- **dir_acc consumer**: `crates/ml/src/cuda_pipeline/aux_dir_acc_reduce_kernel.cu` reads scalar `aux_pred[i]` and tests `(p > 0.0f) ? 1 : 0`. Layer B changes to read 2-class softmax: `pred_pos = (softmax[i*2+1] > softmax[i*2+0]) ? 1 : 0`. Update the 6 oracle tests in `sp13_phase0_oracle_tests.rs` to construct softmax pairs instead of scalar predictions.
|
||||
- **Direction Q-head input concat**: `crates/ml/src/cuda_pipeline/dqn_inference.cu` direction Q-head input layer currently consumes `[trunk_features]`. Layer B concatenates `ISV[AUX_DIR_PREDICTION_INDEX]` (single scalar = mean logit-diff) → input dim grows by 1. Update `dqn_param_layout.rs` direction Q-head weight shape `(in_dim, NUM_DIRECTIONS)` → `(in_dim + 1, NUM_DIRECTIONS)`. Bump layout fingerprint seed to invalidate cached params (greenfield acceptable).
|
||||
- **No information leakage**: aux uses state s_t at inference; price[t+30] only at training-time (hindsight labels). Same pattern as the existing aux MSE head — no new leakage path introduced.
|
||||
|
||||
### Why this is the right next move (rationale post-P0a)
|
||||
|
||||
P0a empirics (`train-67gqb`):
|
||||
- aux_dir_acc climbed 0.149 → 0.483 in 10 epochs at aux_w=0.05 (suppressed)
|
||||
- The trajectory has decaying deltas (0.107 → 0.069 → 0.054 → ... → 0.005) — signature of bias-term decay toward zero, NOT learning the regression task
|
||||
- val_win_rate stuck at 0.4638 — the Q-head can't extract directional signal from a regression head that isn't actually learning
|
||||
|
||||
Layer B + P0b together address this:
|
||||
- P0b: aux_w 0.05 → 0.5 gives the head 10× more gradient → it can actually learn the task
|
||||
- Layer B: 2-class classification with cross-entropy is a STRONGER training signal than MSE on a continuous target (especially when the target sign is what we care about — not its magnitude)
|
||||
- Layer B: Q-head input now sees the head's confidence directly (logit-diff ∈ [−1, +1]) instead of a `tanh(scalar)` of a noisy regression — explicit predictive feature
|
||||
|
||||
Combined effect: Q-head should start consuming the aux signal within ~5-10 epochs after Layer B lands, surfacing as WR > 0.50.
|
||||
|
||||
### Task B.T1: Aux head softmax + cross-entropy
|
||||
|
||||
**Files:** `aux_head_kernel.cu`, `dqn_param_layout.rs`
|
||||
|
||||
@@ -111,9 +111,13 @@ fi
|
||||
# Default pool is ci-training-h100 (sm_90). Override for other architectures:
|
||||
# ci-training-h100* → sm_90 (Hopper)
|
||||
# ci-training-l40s → sm_89 (Ada Lovelace)
|
||||
# ci-training → sm_89 (alias for L40S — pool is named bare in some
|
||||
# clusters; fixed 2026-05-04 after train-mnpf7 deployed
|
||||
# with sm_90 cubins on L40S device, requiring terminate
|
||||
# + resubmit with explicit --gpu-pool ci-training-l40s).
|
||||
case "${GPU_POOL:-ci-training-h100}" in
|
||||
*l40s*) CUDA_COMPUTE_CAP="89" ;;
|
||||
*h100*|*) CUDA_COMPUTE_CAP="90" ;; # default Hopper
|
||||
*l40s*|ci-training) CUDA_COMPUTE_CAP="89" ;;
|
||||
*h100*|*) CUDA_COMPUTE_CAP="90" ;; # default Hopper
|
||||
esac
|
||||
|
||||
# ── Route: single-job (existing template) vs multi-seed DAG (new template) ──
|
||||
|
||||
Reference in New Issue
Block a user