21e7dfd63c
Wires the aux supervision path parallel to BCE: - AuxTrunk (64-hidden single-bucket CfC) consumes the same encoder output as the main BCE trunk - AuxHeads (linear regression long/short) maps aux_trunk output to per-(direction, horizon) predicted outcomes - AuxHuberLoss supervises against D-style labels from MultiHorizonLoader Backward path with asymmetric stop-grad at encoder boundary: - aux_trunk gets gradient signal into its OWN params at all times - aux_trunk's encoder-boundary gradient is INITIALLY blocked (stop_grad_aux_to_encoder = true) - Conditional lift per E3 design: if aux_huber_ema < 0.4 AND aux_dir_acc_ema > 0.85 within 200 steps, lift the stop-grad - When lifted, aux_vec_add kernel folds aux's grad_x into the main grad_h_enriched_seq slot (element-wise += per feedback_no_atomicadd) ISV signals added: aux_huber_per_h, aux_dir_acc_per_h (per pearl). Per-trunk scratch + reduced grad buffers (no Adam state sharing per pearl_adam_normalizes_loss_weights — opt_aux is its own Adam group). New helper kernel cuda/aux_vec_add.cu: position-local dst += src for the asymmetric stop-grad lift accumulation. New synthetic test stacked_trainer_aux_supervision_converges_on_constant_signal validates end-to-end: aux_huber_ema_per_h = [0.087, 0.087, 0.087] (converged) aux_dir_acc_ema_per_h = [1.0, 1.0, 1.0] (perfect on constant) stop_grad_aux_to_encoder = false (lift fired) All 5 stacked_trainer tests pass on RTX 3050 (lib still converges, no regression from parallel aux wiring). Not yet consumed by decision policy (B7) — aux output flows through training only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
39 lines
1.6 KiB
Plaintext
39 lines
1.6 KiB
Plaintext
// SDD-3 Layer B5 — aux→encoder gradient accumulator.
|
|
//
|
|
// Single-purpose kernel that performs an element-wise += of a source
|
|
// vector into a destination vector. Used by `PerceptionTrainer` to fold
|
|
// the auxiliary trunk's `grad_x` (computed at each K-step of the
|
|
// reverse-order aux backward) into the main `grad_h_enriched_seq_t_d`
|
|
// slot when the asymmetric stop-grad is LIFTED (E3 design memo).
|
|
//
|
|
// Why a dedicated kernel rather than reusing an existing add primitive:
|
|
// * The other ml-alpha kernels that mutate `grad_h_enriched_seq` write
|
|
// OVERWRITE-style (cfc_step_bwd produces the slot's gradient by
|
|
// itself). Adding aux on top requires an independent += launch.
|
|
// * Per `feedback_no_atomicadd.md` — this is purely position-local
|
|
// (one thread per (b, c)) so no atomics; the destination slot is
|
|
// write-once per launch.
|
|
//
|
|
// Buffer contract:
|
|
// * `dst` — `[n]` destination, accumulated in-place: `dst[i] += src[i]`
|
|
// * `src` — `[n]` source (read-only)
|
|
// * `n` — element count
|
|
//
|
|
// Launch: grid=(ceil(n/256), 1, 1), block=(256, 1, 1), shared=0.
|
|
//
|
|
// Constraints honoured:
|
|
// * `feedback_no_atomicadd.md` — single-thread-per-element write.
|
|
// * `pearl_no_host_branches_in_captured_graph` — the captured graph's
|
|
// scalar `n` is bound at capture time; replay reads the same value.
|
|
// * `feedback_no_nvrtc.md` — pre-compiled cubin via build.rs.
|
|
|
|
extern "C" __global__ void aux_vec_add_inplace(
|
|
float* __restrict__ dst,
|
|
const float* __restrict__ src,
|
|
int n
|
|
) {
|
|
const int i = blockIdx.x * blockDim.x + threadIdx.x;
|
|
if (i >= n) return;
|
|
dst[i] += src[i];
|
|
}
|