docs(sp22): H6 Phase 3 α spec revision — atom-position shift (not scalar Q bias)

Architectural finding during Phase 3c implementation attempt:

Original spec: α adds `W[a] * state_121[b]` to scalar Q values. Adam-
trained W via dloss/dQ gradient.

Actual codebase: C51 distributional Q-learning. Q is computed from a
probability distribution over fixed atom positions; C51 loss is KL
divergence over distributions, not MSE over scalars. Adding a constant
to logits is softmax-shift-invariant; adding to scalar expected_Q
happens post-loss-path. Either way, the KL loss is invariant under
scalar Q bias → W gradient = 0 → W never trains → α is mathematically
ineffective as originally specified.

Principled fix
──────────────
Per-(b, a) atom-position shift:

    z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b]   for all n

Shifts atom positions state-dependently. C51 Bellman projection
re-projects target onto shifted positions → loss depends on
`W[a] * state_121[b]` smoothly → W gradient flows from projection
arithmetic.

Implementation cost grows from ~25-35 hr to ~40-60 hr because the
atom-shift threads through every kernel that uses atom positions
for the direction branch:

- compute_expected_q (action selection)
- c51_loss_kernel (Bellman projection)
- c51_grad_kernel (backward dW + dstate from projection arithmetic)
- mag_concat_qdir (magnitude-branch conditioning Q_dir compute)
- quantile_q_select / iqn_dual_head_kernel (investigate)

The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become architecturally redundant under the revised
design. The dW + dstate gradient computations integrate into
c51_grad_kernel's existing projection backward. Phase A kernels
stay as compiled cubins (committed in 464bc5f7a); future cleanup
commit may delete them.

Initialization: structural prior `W_init = [-0.5, 0.0, +0.5, 0.0]`
reflects domain belief that aux conviction × position-sign biases
toward aligned trades. Adam refines from there.

  state_121 ∈ [-1, +1] = recentered aux p_up. When +1 (aux up):
    Q_short atoms shift DOWN by 0.5 → action selection avoids Short
    Q_long  atoms shift UP   by 0.5 → action selection biases Long
    Hold / Flat unchanged

Adam-vs-fixed-W: the atom-shift design subsumes the fixed-W case
as a config choice (set Adam LR for W = 0). Per user directive
"no shortcuts", the full version is the canonical Phase 3-final
spec.

β + SP11 controller + A2 eval-side + telemetry unchanged from
prior spec sections — those parts of Phase 3 are architecturally
sound. Only α needed correction.

Files touched
─────────────
- docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md:
    Replaced "α — post-encoder bypass-head" section with
    "α — atom-position shift (architecturally revised 2026-05-13)".
    Documents the C51 incompatibility, the per-(b, a) atom-shift
    fix, the structural-prior initialization, the kernel inventory
    for atom-shift threading, the Adam wireup, and the gradient
    routing decision (option (i) unchanged).

- docs/dqn-wire-up-audit.md:
    "Phase 3c — α architectural finding + spec revision (2026-05-13)"
    entry documenting the math obstacle, the principled fix, the
    Phase A kernel redundancy under the revised design, and the
    scope-revision rationale.

Next session
────────────
Runbook (`docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md`)
α tasks (B9 Steps 4-9, C1) describe the original scalar-bias
implementation and need rewriting for atom-shift threading before
execution can resume.

Refs
────
- pearl_audit_unboundedness_for_implicit_asymmetry (motivates
  per-action W signs for the structural prior)
- C51 distributional Q math: Bellemare, Dabney, Munos 2017
- pearl_no_partial_refactor (atom-shift threading is atomic across
  all consumer kernels)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/dqn-wire-up-audit.md

@@ -16938,3 +16938,44 @@ Phase 3c scope (remaining for full α activation)
   dispatch + verdict.
 
 Estimated remaining: ~20-30 hr engineering + ~37 min smoke wall-clock.
+
+### Phase 3c — α architectural finding + spec revision (2026-05-13)
+
+While implementing Phase 3c (captured-graph integration of α forward), a fundamental architectural mismatch was discovered:
+
+**Original spec**: α adds `W[a] * state_121[b]` to scalar Q values. Adam-trained W via dloss/dQ gradient.
+
+**Actual codebase**: C51 distributional Q-learning. Q is computed from a probability distribution over fixed atom positions: `E[Q_a] = Σ_n p_n * z_n`. The C51 loss is KL divergence over distributions, not MSE over scalars.
+
+**The math obstacle**: Adding a constant to logits is softmax-shift-invariant. Adding to scalar expected_Q happens post-loss-path. Either way, the C51 KL loss is invariant under scalar Q bias → W gradient = 0 → W never trains → α is mathematically ineffective.
+
+Verification: when attempting Phase 3c implementation, traced through the network's forward pass and confirmed that `mag_concat_qdir` (the kernel computing internal Q_dir for magnitude branch conditioning) and `compute_expected_q` (action selection) and `c51_loss_kernel` (training loss) all operate on atom positions and probability distributions. No path for a scalar W bias to influence the KL loss.
+
+**The principled fix** (revised α design in spec `docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md` α section, 2026-05-13):
+
+Per-(b, a) **atom-position shift**:
+
+```
+z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b]    for all n
+```
+
+Shifts atom positions state-dependently. C51 Bellman projection re-projects target onto shifted positions → loss depends on `W[a] * state_121[b]` smoothly → W gradient flows from projection arithmetic.
+
+**Implementation cost**: substantially higher than original spec.
+
+- `compute_expected_q` — atom-shift threading (4 new args)
+- `c51_loss_kernel` — projection uses shifted positions for dir branch
+- `c51_grad_kernel` — backward dW + dstate from projection arithmetic
+- `mag_concat_qdir` — internal Q_dir compute uses shifted positions
+- `quantile_q_select` / `iqn_dual_head_kernel` — investigate dir-branch atom-position usage
+- The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become **architecturally redundant** (dW + dstate gradient integrate into `c51_grad_kernel`'s existing backward). Phase A kernels stay as compiled cubins (committed in `464bc5f7a`); future cleanup commit may delete them.
+
+**Initialization**: structural prior `W_init = [-0.5, 0.0, +0.5, 0.0]` reflects domain belief that aux conviction × position-sign biases toward aligned trades. Adam refines from there.
+
+**Scope adjustment**: estimated full Phase 3-final implementation grows from ~25-35 hr to ~40-60 hr engineering due to the multi-kernel atom-shift threading and backward integration through C51's projection arithmetic.
+
+**Decision rationale (why not Option E "fixed structural prior")**: a fixed-W version (no training) would activate α immediately for action selection but never learn state-dependent corrections. The user explicitly chose "no shortcuts" (2026-05-13 session). The atom-position-shift design is the only mathematically principled fix; the fixed-W version is a special case where Adam updates are skipped. Implementing the full version subsumes the fixed-W version as a config choice (set Adam learning rate for W to zero).
+
+**β + SP11 controller + A2 eval-side + telemetry**: unchanged from previous spec sections — those parts of Phase 3 were architecturally sound. Only the α design needed correction.
+
+Next session: execute the revised α design per the updated spec. The runbook (`docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md`) needs corresponding revision before execution — the runbook's α tasks (B9 Steps 4-9, C1) describe the original scalar-bias implementation and need to be rewritten for atom-shift threading.