feat(sp14/sp22): enlarge aux head + trunk capacity (8x head, 2x trunk-H2)

After Path C confirmed aux head's 28% accuracy at H=60 is the H6 Phase 3
bottleneck (not the mechanism itself), enlarge aux capacity:
- AUX_HIDDEN_DIM: 32 -> 256 (8x, matches input dim, removes bottleneck)
- AUX_TRUNK_H2: 128 -> 256 (2x, uniform trunk width)

Architecture changes:
- Aux head: 256 -> Linear -> 256 -> ELU -> Linear -> 2 (was 256->32->2)
- Aux trunk: 256 -> 256 -> 256 -> 256 (was 256 -> 256 -> 128 -> 256)
- +160K params total (mostly aux_nb_w1 + aux_rg_w1: [256, 256] each)

Side effects:
- Checkpoint fingerprint change (intentional)
- Thread utilisation improves: AUX_BLOCK=256 threads x H=256 = 1:1
  (vs 8:1 at H=32 — most threads idle previously)

Phase 3 mechanism stays DORMANT (W=0, beta=0) for this validation
smoke. Verdict criteria: aux_dir_acc improves from 0.28 toward 0.50+
with the larger capacity. If yes, re-activate Phase 3 priors. If no,
the bottleneck is signal/horizon, not capacity.

Cargo build clean (full nvcc rebuild).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-13 19:59:16 +02:00
parent 2c0911981a
commit 787ee7b86c
4 changed files with 59 additions and 9 deletions

View File

@@ -64,7 +64,9 @@
/* AUX_HIDDEN_DIM kept as a compile-time literal so kernel launch configs
* don't need to thread `H` through. Both heads share the same hidden width. */
#ifndef AUX_HIDDEN_DIM
#define AUX_HIDDEN_DIM 32
/* 2026-05-13: enlarged 32 → 256 (matches input h_s2_aux dim). See
* `gpu_aux_heads.rs::AUX_HIDDEN_DIM` for rationale and side effects. */
#define AUX_HIDDEN_DIM 256
#endif
/* Block dim used by reduction-shaped launches (per-tensor backward, label

View File

@@ -49,7 +49,24 @@ use super::gpu_dqn_trainer::{AUX_HEADS_CUBIN, AUX_HEADS_LOSS_EMA_CUBIN};
/// Hidden dim shared by both auxiliary heads (Linear_1 output width).
/// Kept in sync with the `AUX_HIDDEN_DIM` macro at the top of
/// `aux_heads_kernel.cu`.
pub(crate) const AUX_HIDDEN_DIM: usize = 32;
///
/// **Enlarged 32 → 256 (2026-05-13)** as part of the aux-head capacity
/// upgrade after Path C investigation revealed the aux head's 28%
/// accuracy at H=60 bars. The prior 32-unit bottleneck was severely
/// under-capacity for a 256-dim input → 2-way classification task. The
/// new 256 hidden dim matches the input dimension exactly (no
/// dimensionality reduction in Layer 1) and gives the head full
/// capacity to learn an arbitrary mapping. Architecture:
/// 256 (h_s2_aux input) → Linear → 256 (hidden) → ELU → Linear → 2 (logits)
///
/// Side effects:
/// - Param buffer grows: aux_nb_w1/aux_rg_w1 went from [32, 256] to
/// [256, 256] each → +64K params per head → +128K params total.
/// - Fingerprint changes → checkpoint incompatibility (intentional,
/// fresh-start training).
/// - Thread utilisation improves: AUX_BLOCK=256 threads × H=256 hidden
/// = 1 thread per hidden unit (vs 8 threads/unit at H=32).
pub(crate) const AUX_HIDDEN_DIM: usize = 256;
/// Output cardinality of the next-bar direction classification head
/// (SP13 B1.1a, 2026-05-05). Flipped 1 → 2: the head was a K=1 MSE

View File

@@ -29,11 +29,12 @@
//! - `h_aux2 [B, H2]` — Layer-2 post-ELU activation
//! - `h_s2_aux [B, AUX_HIDDEN_DIM]` — final linear output (no activation)
//!
//! Production topology (verified C.2):
//! Production topology (verified C.2; widths uplifted 2026-05-13):
//! - `ENCODER_OUT_DIM = config.shared_h1 = 256`
//! - `H1 = AUX_TRUNK_H1 = 256`
//! - `H2 = AUX_TRUNK_H2 = 128`
//! - `AUX_HIDDEN_DIM = config.shared_h2 = 256` (matches existing aux head input dim)
//! - `H2 = AUX_TRUNK_H2 = 256` (was 128 pre-2026-05-13 — bottleneck removed)
//! - `AUX_HIDDEN_DIM = config.shared_h2 = 256` (matches existing aux head input dim;
//! internal aux head hidden layer ALSO enlarged 32 → 256 in same commit)
//!
//! # Pearls applied
//!
@@ -65,10 +66,16 @@ use super::gpu_dqn_trainer::{
pub(crate) const AUX_TRUNK_H1: usize = 256;
/// Hidden width of the aux trunk's second internal layer (Linear_2 → ELU
/// output). Smaller than H1 for a mild bottleneck — encourages the aux
/// trunk to compress its representation before lifting back up to
/// `AUX_HIDDEN_DIM` in Layer 3.
pub(crate) const AUX_TRUNK_H2: usize = 128;
/// output).
///
/// **Enlarged 128 → 256 (2026-05-13)** alongside `AUX_HIDDEN_DIM 32 → 256`
/// to remove the aux trunk's internal bottleneck. After the Path C
/// investigation found the aux head's 28% accuracy at H=60 bars, the
/// hypothesis is that capacity (not signal) was the bottleneck. The
/// trunk now has uniform 256 width throughout: 256 (encoder) → 256 (L1)
/// → 256 (L2) → 256 (L3 = h_s2_aux). This gives the trunk maximum
/// expressive power before the aux head's classification layer.
pub(crate) const AUX_TRUNK_H2: usize = 256;
/// Block dim used by the per-sample forward kernel. Mirrors
/// `aux_heads_kernel.cu`'s `AUX_BLOCK = 256`.