spec: f32 backtest pipeline — eliminate bf16 financial arithmetic
bf16 has $32 resolution at $5K equity. Transaction costs ($12.50), tick PnL ($12.50), and cumulative returns are all below resolution. All previous hyperopt evaluations used garbage metrics. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
# f32 Backtest Pipeline — Eliminate bf16 Financial Arithmetic
|
||||
|
||||
## Goal
|
||||
|
||||
Promote the entire GPU backtest pipeline from bf16 to f32. bf16 has $32 resolution at $5,000 equity — transaction costs ($12.50), tick-level PnL ($12.50-$50), and cumulative returns are all below bf16 resolution. This produces garbage Sharpe scores (quantized to ~0.004) and incorrect drawdown (100% from rounding errors). All previous hyperopt evaluations were computed with broken bf16 metrics.
|
||||
|
||||
## Scope
|
||||
|
||||
Three CUDA kernels + one Rust host file. Zero bf16 in the backtest pipeline after this change.
|
||||
|
||||
## Changes
|
||||
|
||||
### 1. `backtest_env_kernel.cu` — both `backtest_env_step` and `backtest_env_step_batch`
|
||||
|
||||
| Buffer | Before | After |
|
||||
|---|---|---|
|
||||
| `prices` | `__nv_bfloat16*` | `float*` |
|
||||
| `portfolio_state` | `__nv_bfloat16*` | `float*` |
|
||||
| `step_rewards` | `__nv_bfloat16*` | `float*` |
|
||||
| `step_returns` | `__nv_bfloat16*` | `float*` |
|
||||
|
||||
- Remove ALL `bf16()`, `__bfloat162float()`, `bf16_zero()`, `bf16_one()`, `bf16_fabs()`, `bf16_fmax()` calls
|
||||
- Pure `float` arithmetic throughout — no type conversions
|
||||
- `PORTFOLIO_STATE_SIZE` stays 8, just `float` instead of `__nv_bfloat16`
|
||||
- Shared memory in `backtest_env_step`: `shmem_pf[256 * PORTFOLIO_STATE_SIZE]` → `float` (doubles shmem but still <48KB)
|
||||
- `done_flags` stays `int*`, `actions_history` stays `int*`
|
||||
|
||||
### 2. `backtest_metrics_kernel.cu` — `compute_backtest_metrics`
|
||||
|
||||
| Buffer | Before | After |
|
||||
|---|---|---|
|
||||
| `step_returns` | `__nv_bfloat16*` | `float*` |
|
||||
| `portfolio_state` | `__nv_bfloat16*` | `float*` |
|
||||
| `metrics_out` | `__nv_bfloat16*` | `float*` |
|
||||
| `annualization_factor` | `__nv_bfloat16` | `float` |
|
||||
|
||||
- All shared memory arrays: `__nv_bfloat16` → `float`
|
||||
- All accumulators: already pattern `local_sum += r` but `r` was bf16 — now `float`
|
||||
- All `bf16_*()` helper calls → native float ops (`fabsf`, `fmaxf`, `sqrtf`)
|
||||
- Shared memory doubles in size (2B → 4B per element) — verify within 48KB limit
|
||||
|
||||
### 3. `gpu_backtest_evaluator.rs` — Rust host
|
||||
|
||||
| Field | Before | After |
|
||||
|---|---|---|
|
||||
| `prices_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
|
||||
| `portfolio_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
|
||||
| `step_rewards_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
|
||||
| `step_returns_buf` | `CudaSlice<half::bf16>` | `CudaSlice<f32>` |
|
||||
|
||||
- `init_portfolio_state()` → remove `clone_htod_f32_to_bf16`, use `stream.clone_htod::<f32>` directly
|
||||
- `reset_evaluation_state()` → same
|
||||
- Price upload in `new()` → remove bf16 conversion, upload f32 directly
|
||||
- `annualization_factor` parameter: `half::bf16::from_f32(...)` → direct `f32`
|
||||
- `metrics_out` parsing: already reads as f32 from `metrics_host: Vec<f32>` — just change allocation
|
||||
- Remove `clone_htod_f32_to_bf16` calls for prices, portfolio, step_returns
|
||||
- `launch_gather` kernel: reads from `features_buf` (bf16 market features) → keep bf16 for features (not financial data), OR promote to f32 too
|
||||
|
||||
### 4. States/features buffer (decision needed)
|
||||
|
||||
`features_buf` and `states_buf` contain market features (z-scored indicators). These are used as neural network INPUT — bf16 is fine for forward inference. Keep bf16 for features.
|
||||
|
||||
`prices_buf` is SEPARATE from features — it contains raw [open, high, low, close] for PnL computation. This goes to f32.
|
||||
|
||||
## What stays bf16
|
||||
|
||||
- `features_buf` / `states_buf` — market features for cuBLAS forward (tensor core input)
|
||||
- Training pipeline (forward/backward GEMMs) — intentional bf16 for performance
|
||||
- Experience collector rewards — individual scalars, not accumulated
|
||||
|
||||
## Testing
|
||||
|
||||
- Smoke tests: verify Sharpe is no longer quantized to bf16 grid
|
||||
- Compare: old bf16 Sharpe vs new f32 Sharpe on same data
|
||||
- Verify: transaction costs are now non-zero in the backtest
|
||||
- H100: deploy and check val_loss varies meaningfully across epochs
|
||||
|
||||
## Impact
|
||||
|
||||
- All previous hyperopt results are INVALID (bf16 Sharpe was noise)
|
||||
- Hyperopt must be restarted from scratch after this fix
|
||||
- Expected: val_loss will show meaningful variation, TPE can actually optimize
|
||||
- Expected: max_dd will be realistic (not 100% from rounding errors)
|
||||
Reference in New Issue
Block a user