7e9a8f6ef1
Per Class A audit-fix Batch 4-A (deferred from P1-wiring/P1-producer due to
audit-doc errors). Fixes 2 of 4 deferred items; Batch 4-B handles plan_threshold
floor + MIN_HOLD_TEMPERATURE in a separate commit.
Item 1: DD saturation floor (the upper end of the DD ramp at trade_physics.cuh:154
in apply_margin_cap, NOT line 548 as the audit doc claimed — that line is a
magnitude action constant; the actual saturation floor lives in apply_margin_cap)
- NEW slot DD_SATURATION_FLOOR_ADAPTIVE_INDEX=458
- Producer dd_saturation_floor_update_kernel.cu — p75(per-env DD_MAX) × 1.5
via Welford `mean + Z_75 × sigma` estimator with `max(p75, mean)` robustness
guard, mirrors P0-A REWARD_POS_CAP producer pattern (Pearl-A bootstrap +
Welford α=0.01)
- Cold-start fallback: 0.25f (DD_SATURATION_FLOOR_DEFAULT in state_layout.cuh)
- Bounds: [0.10, 0.50] (Category-1 dimensional safety)
- Distinct from SP15_DD_THRESHOLD_INDEX=421 (the SP15 quadratic DD-penalty
*trigger* threshold, a *lower* bound; this slot is the *upper* end of the
linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac/floor))
- Threaded `isv_signals_ptr` into `apply_margin_cap` with NULL-tolerant
cold-start fallback to DD_SATURATION_FLOOR_DEFAULT
- 4 oracle tests (Pearl-A bootstrap, no-DD guard, bounds clamp, Welford EMA)
Item 2: Legacy compute_drawdown_penalty path → Case A (DELETED)
- Decision rationale: SP15's quadratic asymmetric DD penalty
(compute_sp15_final_reward_kernel.cu:154 via sp15_dd_penalty helper) runs
unconditionally as a post-modifier on the SP11-composed reward with
ISV-driven λ_dd (slot 420) and DD threshold (slot 421). Layering the legacy
linear-ramp penalty inside the SP11 composer on top of the SP15 quadratic
creates double-counting of DD shaping — exactly the code-smell the Class A
audit was designed to eliminate. Per `feedback_no_legacy_aliases.md` and
`feedback_no_partial_refactor.md`.
- Atomic deletion across:
- `compute_drawdown_penalty` device function (trade_physics.cuh)
- Single call site at experience_kernels.cu:3822
- `dd_threshold` and `w_dd` kernel arguments
- `w_dd` Rust config field (gpu_experience_collector.rs +
trainers/dqn/config.rs DQNHyperparameters)
- `w_dd` profile section + dispatch (training_profile.rs RewardSection,
OptimizableParameterRanges, FixedRewardParameters, ParamLookup
dispatch, profile→hyperparam mapping, test assertion)
- `w_dd *= rki` risk-intensity multiplier (config.rs)
- `w_dd` TOML keys (dqn-hyperopt.toml × 2, dqn-localdev.toml,
dqn-production.toml, dqn-smoketest.toml)
- Stale doc comments on hyperopt/adapters/dqn.rs + config.rs
risk_intensity field
- `config.dd_threshold` SURVIVES (still consumed by `launch_sp15_dd_state`
as the dd_budget for DD_PCT scaling). Documented in field comment.
ISV_TOTAL_DIM: 458 → 459 (Item 1 adds 1 slot; Item 2 is pure deletion)
Cumulative WR-plateau fix series (this is commit 7):
- Class C bug 1 + P0-B (8f218cab2)
- P0-C (316db416b)
- P0-A (394de7d43)
- P1 wiring (c4b6d6ef2) — 1 of 4 wireable
- P0-A downstream (657972a4b)
- P1 producer (87d597d5d)
- audit-fix 4-A (this commit)
Verification: 16 sp14_oracle_tests pass (incl. 4 new), 36 sp15_phase1_oracle_tests
pass, 12 sp14_isv_slots layout tests pass, 4 state_reset_registry tests pass
(every-FoldReset-arm-has-dispatch contract holds), workspace cargo check clean.
Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor +
feedback_no_legacy_aliases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
168 lines
4.7 KiB
TOML
168 lines
4.7 KiB
TOML
# DQN Hyperopt Profile — PSO search space definition
|
|
# All bounds are [min, max] ranges. The adapter reads these at runtime.
|
|
# To constrain the search space, narrow the ranges here — no code changes needed.
|
|
#
|
|
# Mean-reduced gradients (2026-04-09): LR range shifted down because gradient
|
|
# clipping no longer acts as a hidden LR reducer. Old range [1e-5, 3e-4] had
|
|
# effective LR of [~5e-10, ~1.5e-8] due to SUM-reduced clipping at batch=16384.
|
|
|
|
[search_space]
|
|
# Base parameters
|
|
learning_rate = [1e-7, 1e-4] # log scale in adapter — mean-reduced gradients
|
|
batch_size = [4096, 16384]
|
|
gamma = [0.90, 0.99] # wider range — v_range computed dynamically from gamma
|
|
buffer_size = [50000, 100000] # log scale in adapter
|
|
max_leverage = [2.0, 10.0] # leverage ratio; position computed from capital/price
|
|
huber_delta = [10.0, 40.0] # log scale in adapter
|
|
entropy_coefficient = [0.05, 0.5]
|
|
transaction_cost_multiplier = [0.5, 2.0]
|
|
per_alpha = [0.4, 0.8]
|
|
per_beta_start = [0.2, 0.6]
|
|
|
|
# Rainbow DQN extensions
|
|
v_range = [1.0, 1.0] # IGNORED: v_range now computed dynamically from gamma
|
|
noisy_sigma_init = [0.1, 1.0] # log scale in adapter
|
|
dueling_hidden_dim = [128, 512]
|
|
n_steps = [3, 5]
|
|
num_atoms = [52, 100] # 51=C51 paper minimum, 101=higher resolution (2x slower but better signal)
|
|
|
|
# Weight decay
|
|
weight_decay = [0.0001, 0.01] # log scale in adapter
|
|
|
|
# Kelly risk parameters
|
|
kelly_fractional = [0.25, 0.75]
|
|
kelly_max_fraction = [0.1, 0.5]
|
|
|
|
# Volatility
|
|
volatility_window = [10, 30]
|
|
|
|
# Soft update
|
|
tau = [0.005, 0.01] # log scale in adapter
|
|
|
|
# Network sizing
|
|
hidden_dim_base = [128, 256] # capped at production default: 512 is 4x slower for marginal benefit
|
|
|
|
# CQL regularization
|
|
cql_alpha = [0.5, 5.0]
|
|
|
|
# Training dynamics
|
|
lr_decay_type = [0, 2] # discrete: 0=constant, 1=linear, 2=cosine
|
|
minimum_profit_factor = [1.1, 2.0]
|
|
|
|
# Exploration
|
|
count_bonus_coefficient = [0.0, 0.3]
|
|
|
|
# Risk-adjusted returns
|
|
sharpe_weight = [0.0, 0.5]
|
|
|
|
# Branching DQN
|
|
branch_hidden_dim = [64, 256]
|
|
|
|
# Gradient accumulation
|
|
gradient_accumulation_steps = [1, 1] # fixed at 1 for now
|
|
|
|
# IQN dual-head
|
|
iqn_lambda = [0.0, 2.0]
|
|
|
|
# Spectral normalization
|
|
spectral_norm_sigma_max = [1.0, 10.0]
|
|
|
|
# C51 warmup
|
|
c51_warmup_epochs = [0, 10]
|
|
c51_alpha_max = [0.3, 0.9]
|
|
|
|
# HER ratio
|
|
her_ratio = [0.0, 0.8]
|
|
|
|
# Composite reward weights
|
|
# `w_dd` removed (Class A audit-fix Batch 4-A, 2026-05-08) atomically with the
|
|
# legacy compute_drawdown_penalty path. SP15's λ_dd (ISV slot 420) is the
|
|
# producer-driven replacement and is not exposed as a hyperopt search dim.
|
|
w_pnl = [0.0, 1.0]
|
|
w_idle = [0.0, 0.1]
|
|
dd_threshold = [0.005, 0.03] # HFT: tight drawdown tolerance (0.5%-3%)
|
|
loss_aversion = [1.0, 1.0]
|
|
time_decay_rate = [0.0001, 0.005]
|
|
|
|
# Trade conviction filter
|
|
q_gap_threshold = [0.0, 0.5] # 0.0=trade every bar, 0.5=high conviction only
|
|
|
|
# Reward v7 — Counterfactual Branch Attribution
|
|
cea_weight = [0.1, 1.0]
|
|
# exposure_aux_weight removed (4-branch refactor)
|
|
b3_size = 3
|
|
|
|
# v8 search ranges
|
|
micro_reward_scale = [0.0, 0.05]
|
|
td_lambda = [0.5, 0.99]
|
|
hindsight_fraction = [0.0, 0.3]
|
|
hindsight_lookahead = [5, 20]
|
|
epsilon_start = [0.1, 0.5]
|
|
epsilon_end = [0.01, 0.05]
|
|
|
|
[experience]
|
|
initial_capital = 100000.0
|
|
|
|
[experience.fill_simulation]
|
|
ioc_fill_prob = 0.85
|
|
limit_fill_min = 0.30
|
|
limit_fill_max = 0.80
|
|
tx_cost_multiplier = 0.18 # IBKR ES: 0.18 bps = $4.50/contract RT
|
|
spread_cost_frac = 0.50
|
|
spread_capture_frac = 0.50
|
|
|
|
[risk]
|
|
q_clip_min = -200.0
|
|
q_clip_max = 200.0
|
|
|
|
[reward]
|
|
# v8 comprehensive training overhaul
|
|
micro_reward_scale = 0.01
|
|
td_lambda = 0.9
|
|
max_trace_length = 7
|
|
hindsight_fraction = 0.1
|
|
hindsight_lookahead = 10
|
|
w_pnl = 0.3
|
|
# `w_dd` removed (Class A audit-fix Batch 4-A, 2026-05-08) atomically with legacy compute_drawdown_penalty.
|
|
w_idle = 0.01
|
|
dd_threshold = 0.01
|
|
loss_aversion = 1.0
|
|
time_decay_rate = 0.0005
|
|
q_gap_threshold = 0.1
|
|
cea_weight = 0.3
|
|
b3_size = 3
|
|
|
|
[fixed]
|
|
|
|
[pso]
|
|
swarm_size = 20
|
|
max_iterations = 50
|
|
inertia = 0.7
|
|
cognitive = 1.5
|
|
social = 1.5
|
|
|
|
# Two-phase hyperopt: Phase 1 fixes architecture to small network,
|
|
# searches only learning dynamics (~15D). Phase 2 fixes best dynamics
|
|
# from Phase 1 JSON, searches architecture (~5D).
|
|
# See: docs/superpowers/specs/2026-03-22-two-phase-hyperopt-design.md
|
|
[phase_fast]
|
|
hidden_dim_base = 128
|
|
num_atoms = 101
|
|
branch_hidden_dim = 64
|
|
dueling_hidden_dim = 128
|
|
v_range = 1.0
|
|
# v8 phase_fast overrides
|
|
micro_reward_scale = 0.01
|
|
td_lambda = 0.9
|
|
hindsight_fraction = 0.1
|
|
hindsight_lookahead = 10
|
|
w_pnl = 0.3
|
|
# `w_dd` removed (Class A audit-fix Batch 4-A, 2026-05-08) atomically with legacy compute_drawdown_penalty.
|
|
w_idle = 0.01
|
|
dd_threshold = 0.01
|
|
loss_aversion = 1.0
|
|
time_decay_rate = 0.0005
|
|
q_gap_threshold = 0.1
|
|
cea_weight = 0.3
|
|
b3_size = 3
|