feat(sp14/sp22): enlarge aux head + trunk capacity (8x head, 2x trunk-H2)

After Path C confirmed aux head's 28% accuracy at H=60 is the H6 Phase 3
bottleneck (not the mechanism itself), enlarge aux capacity:
- AUX_HIDDEN_DIM: 32 -> 256 (8x, matches input dim, removes bottleneck)
- AUX_TRUNK_H2: 128 -> 256 (2x, uniform trunk width)

Architecture changes:
- Aux head: 256 -> Linear -> 256 -> ELU -> Linear -> 2 (was 256->32->2)
- Aux trunk: 256 -> 256 -> 256 -> 256 (was 256 -> 256 -> 128 -> 256)
- +160K params total (mostly aux_nb_w1 + aux_rg_w1: [256, 256] each)

Side effects:
- Checkpoint fingerprint change (intentional)
- Thread utilisation improves: AUX_BLOCK=256 threads x H=256 = 1:1
  (vs 8:1 at H=32 — most threads idle previously)

Phase 3 mechanism stays DORMANT (W=0, beta=0) for this validation
smoke. Verdict criteria: aux_dir_acc improves from 0.28 toward 0.50+
with the larger capacity. If yes, re-activate Phase 3 priors. If no,
the bottleneck is signal/horizon, not capacity.

Cargo build clean (full nvcc rebuild).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-13 19:59:16 +02:00
parent 2c0911981a
commit 787ee7b86c
4 changed files with 59 additions and 9 deletions

View File

@@ -64,7 +64,9 @@
/* AUX_HIDDEN_DIM kept as a compile-time literal so kernel launch configs
* don't need to thread `H` through. Both heads share the same hidden width. */
#ifndef AUX_HIDDEN_DIM
#define AUX_HIDDEN_DIM 32
/* 2026-05-13: enlarged 32 → 256 (matches input h_s2_aux dim). See
* `gpu_aux_heads.rs::AUX_HIDDEN_DIM` for rationale and side effects. */
#define AUX_HIDDEN_DIM 256
#endif
/* Block dim used by reduction-shaped launches (per-tensor backward, label

View File

@@ -49,7 +49,24 @@ use super::gpu_dqn_trainer::{AUX_HEADS_CUBIN, AUX_HEADS_LOSS_EMA_CUBIN};
/// Hidden dim shared by both auxiliary heads (Linear_1 output width).
/// Kept in sync with the `AUX_HIDDEN_DIM` macro at the top of
/// `aux_heads_kernel.cu`.
pub(crate) const AUX_HIDDEN_DIM: usize = 32;
///
/// **Enlarged 32 → 256 (2026-05-13)** as part of the aux-head capacity
/// upgrade after Path C investigation revealed the aux head's 28%
/// accuracy at H=60 bars. The prior 32-unit bottleneck was severely
/// under-capacity for a 256-dim input → 2-way classification task. The
/// new 256 hidden dim matches the input dimension exactly (no
/// dimensionality reduction in Layer 1) and gives the head full
/// capacity to learn an arbitrary mapping. Architecture:
/// 256 (h_s2_aux input) → Linear → 256 (hidden) → ELU → Linear → 2 (logits)
///
/// Side effects:
/// - Param buffer grows: aux_nb_w1/aux_rg_w1 went from [32, 256] to
/// [256, 256] each → +64K params per head → +128K params total.
/// - Fingerprint changes → checkpoint incompatibility (intentional,
/// fresh-start training).
/// - Thread utilisation improves: AUX_BLOCK=256 threads × H=256 hidden
/// = 1 thread per hidden unit (vs 8 threads/unit at H=32).
pub(crate) const AUX_HIDDEN_DIM: usize = 256;
/// Output cardinality of the next-bar direction classification head
/// (SP13 B1.1a, 2026-05-05). Flipped 1 → 2: the head was a K=1 MSE

View File

@@ -29,11 +29,12 @@
//! - `h_aux2 [B, H2]` — Layer-2 post-ELU activation
//! - `h_s2_aux [B, AUX_HIDDEN_DIM]` — final linear output (no activation)
//!
//! Production topology (verified C.2):
//! Production topology (verified C.2; widths uplifted 2026-05-13):
//! - `ENCODER_OUT_DIM = config.shared_h1 = 256`
//! - `H1 = AUX_TRUNK_H1 = 256`
//! - `H2 = AUX_TRUNK_H2 = 128`
//! - `AUX_HIDDEN_DIM = config.shared_h2 = 256` (matches existing aux head input dim)
//! - `H2 = AUX_TRUNK_H2 = 256` (was 128 pre-2026-05-13 — bottleneck removed)
//! - `AUX_HIDDEN_DIM = config.shared_h2 = 256` (matches existing aux head input dim;
//! internal aux head hidden layer ALSO enlarged 32 → 256 in same commit)
//!
//! # Pearls applied
//!
@@ -65,10 +66,16 @@ use super::gpu_dqn_trainer::{
pub(crate) const AUX_TRUNK_H1: usize = 256;
/// Hidden width of the aux trunk's second internal layer (Linear_2 → ELU
/// output). Smaller than H1 for a mild bottleneck — encourages the aux
/// trunk to compress its representation before lifting back up to
/// `AUX_HIDDEN_DIM` in Layer 3.
pub(crate) const AUX_TRUNK_H2: usize = 128;
/// output).
///
/// **Enlarged 128 → 256 (2026-05-13)** alongside `AUX_HIDDEN_DIM 32 → 256`
/// to remove the aux trunk's internal bottleneck. After the Path C
/// investigation found the aux head's 28% accuracy at H=60 bars, the
/// hypothesis is that capacity (not signal) was the bottleneck. The
/// trunk now has uniform 256 width throughout: 256 (encoder) → 256 (L1)
/// → 256 (L2) → 256 (L3 = h_s2_aux). This gives the trunk maximum
/// expressive power before the aux head's classification layer.
pub(crate) const AUX_TRUNK_H2: usize = 256;
/// Block dim used by the per-sample forward kernel. Mirrors
/// `aux_heads_kernel.cu`'s `AUX_BLOCK = 256`.

View File

@@ -17521,3 +17521,27 @@ To re-activate the mechanism, restore W=[-0.5, 0, +0.5, 0] in `aux_w_prior_init_
6. **Use aux as confidence gate only**: keep state[121] = recentered aux p_up, but DON'T use it as direction signal. Use it as a "trust this decision" gate that scales action magnitude. Useful even if direction is non-predictable.
Cargo check clean.
#### Aux head + trunk capacity uplift (2026-05-13)
After Path C confirmed the aux head's 28% accuracy at H=60 bars is the actual H6 Phase 3 bottleneck, this commit enlarges aux capacity 8× on the head + 2× on the trunk's middle layer:
| Component | Before | After |
|-----------|--------|-------|
| `AUX_HIDDEN_DIM` (aux head internal hidden) | 32 | **256** |
| `AUX_TRUNK_H2` (aux trunk middle layer) | 128 | **256** |
**Aux head architecture** (now): `256 (h_s2_aux) → Linear → 256 (hidden) → ELU → Linear → 2 (softmax)`. Removes the 32-unit bottleneck that compressed 256 input dims into 32 features before classification. Param count grows: aux_nb_w1 + aux_rg_w1 each `[32, 256]``[256, 256]` = +128K params total.
**Aux trunk architecture** (now): `256 → 256 → 256 → 256` (uniform width). Removes the H2 bottleneck. Param count for `aux_trunk_w2 [H1, H2]` grows `[256, 128]``[256, 256]` = +32K params.
**Hypothesis under test**: aux head's 28% accuracy was a **capacity bottleneck**, not a fundamental signal/horizon limitation. With 8× hidden dim, the head should learn at least majority-class prediction (≥50% accuracy on a 17/83 split) and ideally better.
**Side effects**:
- Param buffer + tensor layout fingerprint change → checkpoint incompatibility
- Modest compute increase per forward/backward (memory-bandwidth-bound at this scale, negligible wall-time impact on L40S)
- Thread utilisation IMPROVES: AUX_BLOCK=256 threads × H=256 hidden = 1:1 mapping (vs 256:32 ≈ 8:1 idle ratio at H=32)
**Phase 3 mechanism stays DORMANT** for this smoke (W=0, β=0). If aux head accuracy improves to ≥55% with the larger capacity, the next step is to re-activate Phase 3 (W=[-0.5, 0, +0.5, 0], β=0.5) and re-validate WR.
Cargo build clean (1m12s — full nvcc rebuild of aux kernels).