gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ml-alpha): multi_horizon_heads kernel (128->5 sigmoid)

Browse Source 
Per-horizon P(up) at h ∈ {30, 100, 300, 1000, 6000} snapshots forward.
Single-block 5-thread kernel; each thread is its own 128-dim dot
product + sigmoid. No atomicAdd.

Tests (5/5 pass on sm_86) assert invariants only:
  - sigmoid output ∈ [0, 1] for all heads
  - zero weights + zero bias → 0.5 exactly
  - bias = +20 → saturates near 1
  - bias = -20 → saturates near 0
  - per-head independence (mixed-bias configuration)

Addendum updated to explicitly state no-CPU-mirror discipline per
feedback_no_cpu_test_fallbacks.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-16 21:51:48 +02:00
parent f927469ed3
commit d4e46aba94
5 changed files with 238 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
docs/superpowers/plans/2026-05-16-ml-alpha-phase-a-api-addendum.md
View File

@@ -3,8 +3,31 @@
> Companion to `2026-05-16-ml-alpha-phase-a.md`. When the plan's binding-code
> blocks reference `dev.htod_copy`, `dev.alloc_zeros`, `dev.load_cubin_function`,
> `func.launch_async`, or `dev.default_stream`, **use the patterns in this
> addendum instead.** The plan's kernel `.cu` source, CPU oracles, tests, and
> assertion thresholds are unchanged.
> addendum instead.** The plan's kernel `.cu` source and assertion thresholds
> are unchanged.
## Validation discipline override: no CPU mirrors
Plan 1 incorrectly listed a "CPU oracle + bit-equiv test" step for every kernel
task. Per `feedback_no_cpu_test_fallbacks.md` — kernels must be validated via
**invariants on the GPU output**, not CPU mirrors. The user re-confirmed this
mid-execution.
**Canonical validation pattern for each kernel:**
1. Build analytically-known synthetic inputs (zeros, identities, magic constants
with known kernel outputs).
2. Run the GPU kernel.
3. Assert algebraic invariants on the output: ranges (e.g. sigmoid outputs
∈ [0, 1]), fixed-points (dt=0 → identity), monotonicity (input ↑ → output ↑),
conservation (sum of softmax probs ≈ 1), bounded growth (output ≤ deterministic
bound from input).
4. Where forward + backward are paired (BCE, PPO loss), validate the gradient
via **finite-difference on the GPU forward** at a small set of perturbed
inputs — NOT against a CPU autograd.
Whenever Plan 1's task body says "CPU oracle" or "bit-equiv test", read it as
"GPU-output invariant test."
This addendum exists because Plan 1 was written against an older cudarc
device-centric API. The vendored cudarc 0.19 (`vendor/cudarc/`) moved