Files
foxhunt/docs
jgrusewski 7b74290dd0 docs(dqn): R5 micro-reward — documented intentional disable (Phase 2 Task 2.3)
Per feedback_no_functionality_removal.md: R5 was originally scoped as
DELETE in the Phase 2 plan and the Track 2 triage because
reward_contrib[3] = 0.000 across 60 / 60 smoke epochs and
dqn-smoketest.toml sets micro_reward_scale = 0.0. Re-examination during
Phase 2 Task 2.3 rejected the DELETE path:

- dqn-production.toml already sets micro_reward_scale = 0.1, so R5 is
  load-bearing in production, not dead code. The 0.0 value in smoke is
  deliberate test-isolation (td_propagation / magnitude_distribution /
  reward_component_audit all want the sparse-reward TD path isolated).
- The state-vector OFI block at state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM)
  = [42..62) provides representation features for the encoder (policy
  side). R5 is a per-bar reward gradient on the critic (critic side).
  Different mechanisms — production deploys both together.
- R5 also reads PREV_MID (retrospective hold quality), which is NOT in
  the state vector. That signal exists only in the kernel branch.

Changes — pure documentation, no behavior change:
- experience_kernels.cu: ~30-line comment block at the R5 wiring site
  (~L1915) documenting the parameter-not-flag status, production vs
  smoke values, why state-vector OFI is complementary not redundant,
  and the feedback_no_functionality_removal.md seal.
- experience_kernels.cu: fix stale kernel-signature comment that claimed
  OFI was at state[66..74). Correct range is [42..62) per state_layout.cuh.
- config.rs: extend DQNHyperparameters::micro_reward_scale docstring and
  add a comment at the Default impl pointing back at the kernel site.
- gpu_experience_collector.rs: extend reward_contrib_fractions docstring
  to mark the micro=0.000 slot as a SEMANTIC value when the loaded profile
  has micro_reward_scale=0.0, not a wiring regression.
- track2-triage.md: R5 verdict changed from DELETE to FIX-documented-disable
  with the rationale above; "Proposed Phase 2 changes" section 1 and
  "Next Track 2 steps" updated accordingly.

Smoke tests: 3 / 4 pass (reward_component_audit, controller_activity,
exploration_coverage). magnitude_distribution is failing on baseline
HEAD c0fee5a9b as well — pre-existing H10 magnitude-head collapse
regression (eval dist_mag ef=0.000, eh=0.000), unrelated to R5.
Verified via git-stash round-trip: baseline fails, changes do not
introduce new failures.

Closes Track 2 R5 verdict (originally DELETE, now FIX-documented-disable).
2026-04-22 11:39:27 +02:00
..