gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(generalization): #31 bottleneck weight foundation + crown jewels plan

Browse Source 
Weight system expansion: param layout [20] → [22] tensors with
NUM_WEIGHT_TENSORS constant. Tensors 20-21 (w_bn, b_bn) hold temporal
causal bottleneck weights. Size 0 when bottleneck_dim=0 (backward
compatible — existing models load without changes).

Config: bottleneck_dim field added to DQNHyperparameters, GpuDqnTrainConfig,
TOML [generalization] section, and training profile. Default: 0 (disabled).
Set to 2 for maximum information compression.

Crown Jewels plan (Tasks 31-34):
- Gem (#31): 2D Temporal Causal Bottleneck (architecture defense)
- Pearl (#32): Gradient Vaccine (optimization defense)
- King (#33): Adversarial Self-Play with Past Self (strategic defense)
- Emperor (#34): Causal Intervention Training (epistemic defense)

Four layers of defense making memorization impossible at every level.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-30 09:29:46 +02:00
parent 18069b24aa
commit af61813e3b
11 changed files with 256 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
config/training/dqn-localdev.toml
View File

@@ -82,6 +82,7 @@ anti_lr_sharpe_threshold = 0.3
feature_mask_fraction = 0.3
feature_noise_scale = 0.1
enable_vol_normalization = true
bottleneck_dim = 0
stochastic_depth_prob = 0.2
asymmetric_dd_weight = 0.5
time_reversal_mod = 5

1
config/training/dqn-production.toml
View File

@@ -87,6 +87,7 @@ anti_lr_sharpe_threshold = 0.3
feature_mask_fraction = 0.3
feature_noise_scale = 0.1
enable_vol_normalization = true
bottleneck_dim = 0
stochastic_depth_prob = 0.2
asymmetric_dd_weight = 0.5
time_reversal_mod = 5

2
crates/ml/src/cuda_pipeline/batched_backward.rs
View File

@@ -449,7 +449,7 @@ impl CublasBackward {
save_h_v: u64, // [B, VH]
save_h_b: &[u64; 3], // [B, AH] each
// Online weight pointers (read-only for W^T computation)
w_ptrs: &[u64; 20],
w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
// Flat gradient accumulator (must be zeroed by caller)
grad_buf_base: u64,
// Scratch buffers for inter-layer gradients (f32)

23
crates/ml/src/cuda_pipeline/batched_forward.rs
View File

@@ -234,7 +234,7 @@ impl CublasForward {
&self,
stream: &Arc<CudaStream>,
states_ptr: u64,
w_ptrs: &[u64; 20],
w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
h_s1_ptr: u64, h_s2_ptr: u64, h_v_ptr: u64,
h_b0_ptr: u64, h_b1_ptr: u64, h_b2_ptr: u64,
v_logits_ptr: u64, b_logits_ptr: u64,
@@ -285,7 +285,7 @@ impl CublasForward {
&self,
stream: &Arc<CudaStream>,
states_buf: &CudaSlice<half::bf16>,
w_ptrs: &[u64; 20],
w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
save_h_s1: &CudaSlice<half::bf16>,
save_h_s2: &CudaSlice<half::bf16>,
save_h_v: &CudaSlice<half::bf16>,
@@ -367,7 +367,7 @@ impl CublasForward {
&self,
stream: &Arc<CudaStream>,
states_ptr: u64,
tg_w_ptrs: &[u64; 20],
tg_w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
h_s1_ptr: u64, h_s2_ptr: u64, h_v_ptr: u64, h_b_ptr: u64,
v_logits_ptr: u64, b_logits_ptr: u64,
) -> Result<(), MLError> {
@@ -426,7 +426,7 @@ impl CublasForward {
&self,
stream: &Arc<CudaStream>,
next_states_buf: &CudaSlice<half::bf16>,
tg_w_ptrs: &[u64; 20],
tg_w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
tg_h_s1_scratch: &CudaSlice<half::bf16>,
tg_h_s2_buf: &CudaSlice<half::bf16>,
tg_h_v_scratch: &CudaSlice<half::bf16>,
@@ -789,9 +789,9 @@ fn compile_bias_kernels(
/// Returns 20 raw u64 device pointers (base + byte_offset for each tensor).
pub fn bf16_weight_ptrs(
params_buf: &CudaSlice<half::bf16>,
param_sizes: &[usize; 20],
param_sizes: &[usize; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
stream: &Arc<CudaStream>,
) -> [u64; 20] {
) -> [u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS] {
let base = {
let (ptr, guard) = params_buf.device_ptr(stream);
let _no_drop = ManuallyDrop::new(guard);
@@ -800,15 +800,16 @@ pub fn bf16_weight_ptrs(
bf16_weight_ptrs_from_base(base, param_sizes)
}
/// Compute 20 raw BF16 device pointers from a pre-resolved base pointer.
/// Compute raw BF16 device pointers from a pre-resolved base pointer.
/// Graph-safe: no device_ptr calls, no event recording.
pub fn bf16_weight_ptrs_from_base(
base: u64,
param_sizes: &[usize; 20],
) -> [u64; 20] {
let mut ptrs = [0_u64; 20];
param_sizes: &[usize; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
) -> [u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS] {
let n = super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS;
let mut ptrs = [0_u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS];
let mut byte_offset: u64 = 0;
for i in 0..20 {
for i in 0..n {
ptrs[i] = base + byte_offset;
byte_offset += (param_sizes[i] * std::mem::size_of::<half::bf16>()) as u64;
}

7
crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs
View File

@@ -930,7 +930,7 @@ impl GpuBacktestEvaluator {
/// a 64-bar (~1 hour) staleness window has negligible impact on action quality.
fn submit_dqn_step_loop_cublas(
&self,
w_ptrs: &[u64; 20],
w_ptrs: &[u64; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
dqn_cfg: &DqnBacktestConfig,
) -> Result<(), MLError> {
// ── Unwrap per-step cuBLAS state (used only for fallback / CUDA graph) ──
@@ -1778,7 +1778,7 @@ impl GpuBacktestEvaluator {
/// w_b0fc, b_b0fc, w_b0out, b_b0out,
/// w_b1fc, b_b1fc, w_b1out, b_b1out,
/// w_b2fc, b_b2fc, w_b2out, b_b2out]`
fn compute_backtest_param_sizes(cfg: &DqnBacktestConfig, state_dim: usize) -> [usize; 20] {
fn compute_backtest_param_sizes(cfg: &DqnBacktestConfig, state_dim: usize) -> [usize; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS] {
[
cfg.shared_h1 * state_dim, // w_s1
cfg.shared_h1, // b_s1
@@ -1800,6 +1800,9 @@ fn compute_backtest_param_sizes(cfg: &DqnBacktestConfig, state_dim: usize) -> [u
cfg.adv_h, // b_b2fc
cfg.branch_2_size * cfg.num_atoms * cfg.adv_h, // w_b2out
cfg.branch_2_size * cfg.num_atoms, // b_b2out
// #31 bottleneck: backtest always uses bottleneck_dim=0 (no bottleneck at inference)
0, // w_bn (unused)
0, // b_bn (unused)
]
}

75
crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
View File

@@ -189,6 +189,12 @@ pub struct GpuDqnTrainConfig {
pub ensemble_disagreement_penalty: f32,
/// #21 Stochastic depth drop probability. 0.0 = disabled, 0.2 = 20% drop per layer.
pub stochastic_depth_prob: f32,
/// #31 Temporal causal bottleneck dimension. 0 = disabled (no bottleneck).
/// 2 = maximum compression (14 market features → 2 abstract dimensions).
pub bottleneck_dim: usize,
/// Market feature dimension (e.g. 42). Used by bottleneck to separate
/// market features from portfolio features in the state vector.
pub market_dim: usize,
}
impl Default for GpuDqnTrainConfig {
@@ -228,6 +234,8 @@ impl Default for GpuDqnTrainConfig {
asymmetric_dd_weight: 0.0,
ensemble_disagreement_penalty: 0.0,
stochastic_depth_prob: 0.0,
bottleneck_dim: 0,
market_dim: 42,
}
}
}
@@ -291,29 +299,52 @@ pub struct FusedTrainScalars {
// w_b0fc, b_b0fc, w_b0out, b_b0out, w_b1fc, b_b1fc, w_b1out, b_b1out,
// w_b2fc, b_b2fc, w_b2out, b_b2out.
/// Compute the size (element count) of each of the 20 weight tensors.
pub(crate) fn compute_param_sizes(cfg: &GpuDqnTrainConfig) -> [usize; 20] {
/// Number of weight tensors in the flat parameter buffer.
/// 20 original (DQN network) + 2 bottleneck (w_bn, b_bn) = 22.
pub(crate) const NUM_WEIGHT_TENSORS: usize = 22;
/// Compute the size (element count) of each weight tensor.
///
/// Tensors 0-19: standard DQN network (unchanged layout for backward compat).
/// Tensors 20-21: temporal causal bottleneck (0 elements when bottleneck_dim=0).
///
/// When bottleneck is active, w_s1 input dimension changes from state_dim to
/// (bottleneck_dim + portfolio_dim) where portfolio_dim = state_dim - market_dim.
pub(crate) fn compute_param_sizes(cfg: &GpuDqnTrainConfig) -> [usize; NUM_WEIGHT_TENSORS] {
// When bottleneck is active, h_s1 input is [bottleneck_dim + portfolio_features]
// instead of [state_dim]. portfolio_features = state_dim - market_dim (typically 6-8).
let s1_input_dim = if cfg.bottleneck_dim > 0 {
cfg.bottleneck_dim + (cfg.state_dim - cfg.market_dim)
} else {
cfg.state_dim
};
let bn_dim = cfg.bottleneck_dim;
let market_dim = cfg.market_dim;
[
cfg.shared_h1 * cfg.state_dim, // w_s1
cfg.shared_h1, // b_s1
cfg.shared_h2 * cfg.shared_h1, // w_s2
cfg.shared_h2, // b_s2
cfg.value_h * cfg.shared_h2, // w_v1
cfg.value_h, // b_v1
cfg.num_atoms * cfg.value_h, // w_v2
cfg.num_atoms, // b_v2
cfg.adv_h * cfg.shared_h2, // w_b0fc
cfg.adv_h, // b_b0fc
cfg.branch_0_size * cfg.num_atoms * cfg.adv_h, // w_b0out
cfg.branch_0_size * cfg.num_atoms, // b_b0out
cfg.adv_h * cfg.shared_h2, // w_b1fc
cfg.adv_h, // b_b1fc
cfg.branch_1_size * cfg.num_atoms * cfg.adv_h, // w_b1out
cfg.branch_1_size * cfg.num_atoms, // b_b1out
cfg.adv_h * cfg.shared_h2, // w_b2fc
cfg.adv_h, // b_b2fc
cfg.branch_2_size * cfg.num_atoms * cfg.adv_h, // w_b2out
cfg.branch_2_size * cfg.num_atoms, // b_b2out
cfg.shared_h1 * s1_input_dim, // [0] w_s1 (input dim changes with bottleneck)
cfg.shared_h1, // [1] b_s1
cfg.shared_h2 * cfg.shared_h1, // [2] w_s2
cfg.shared_h2, // [3] b_s2
cfg.value_h * cfg.shared_h2, // [4] w_v1
cfg.value_h, // [5] b_v1
cfg.num_atoms * cfg.value_h, // [6] w_v2
cfg.num_atoms, // [7] b_v2
cfg.adv_h * cfg.shared_h2, // [8] w_b0fc
cfg.adv_h, // [9] b_b0fc
cfg.branch_0_size * cfg.num_atoms * cfg.adv_h, // [10] w_b0out
cfg.branch_0_size * cfg.num_atoms, // [11] b_b0out
cfg.adv_h * cfg.shared_h2, // [12] w_b1fc
cfg.adv_h, // [13] b_b1fc
cfg.branch_1_size * cfg.num_atoms * cfg.adv_h, // [14] w_b1out
cfg.branch_1_size * cfg.num_atoms, // [15] b_b1out
cfg.adv_h * cfg.shared_h2, // [16] w_b2fc
cfg.adv_h, // [17] b_b2fc
cfg.branch_2_size * cfg.num_atoms * cfg.adv_h, // [18] w_b2out
cfg.branch_2_size * cfg.num_atoms, // [19] b_b2out
// #31 Temporal Causal Bottleneck (0 when disabled)
bn_dim * market_dim, // [20] w_bn [bottleneck_dim, market_dim]
bn_dim, // [21] b_bn [bottleneck_dim]
]
}

2
crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
View File

@@ -444,7 +444,7 @@ pub struct GpuExperienceCollector {
/// Total number of F32 parameters in the flat buffer.
total_params: usize,
/// Per-tensor sizes for computing weight pointers.
param_sizes: [usize; 20],
param_sizes: [usize; super::gpu_dqn_trainer::NUM_WEIGHT_TENSORS],
// ── CVaR position scaling (IQN dual-head) ──────────────────────
/// Device pointer to CVaR scales [N]. 0 = NULL = no scaling.

6
crates/ml/src/trainers/dqn/config.rs
View File

@@ -1004,6 +1004,11 @@ pub struct DQNHyperparameters {
/// 0.0 = disabled.
pub trade_clustering_penalty: f64,
/// #31 Temporal causal bottleneck dimension. 0 = disabled, 2 = maximum compression.
/// All market features are compressed through [market_dim → bottleneck_dim] linear + tanh
/// before reaching the Q-network. Makes memorization structurally impossible.
pub bottleneck_dim: usize,
/// #21 Stochastic depth: drop probability per hidden layer during training.
/// Each layer is independently dropped with this probability. Surviving layers
/// are scaled by 1/(1-p) for expected-value correction. 0.0 = disabled.
@@ -1524,6 +1529,7 @@ impl DQNHyperparameters {
enable_mirror_universe: true, // #10: alternate mirrored/normal epochs
position_entropy_weight: 0.01, // #19: reward += 0.01 * H(position_histogram)
trade_clustering_penalty: 0.05, // #25: penalize temporally clustered trades
bottleneck_dim: 0, // #31: disabled by default (0 = no bottleneck, 2 = max compression)
stochastic_depth_prob: 0.2, // #21: 20% drop probability per hidden layer
asymmetric_dd_weight: 0.5, // #18: extra loss on Q-overestimation in drawdown

2
crates/ml/src/trainers/dqn/fused_training.rs
View File

@@ -209,6 +209,8 @@ impl FusedTrainingCtx {
asymmetric_dd_weight: hyperparams.asymmetric_dd_weight as f32,
ensemble_disagreement_penalty: hyperparams.ensemble_disagreement_penalty as f32,
stochastic_depth_prob: hyperparams.stochastic_depth_prob as f32,
bottleneck_dim: hyperparams.bottleneck_dim,
market_dim: 42, // MARKET_DIM constant — 42 preprocessed features
};
// Extract weight sets from VarMaps (online + target)

2
crates/ml/src/training_profile.rs
View File

@@ -161,6 +161,7 @@ pub struct GeneralizationSection {
pub feature_mask_fraction: Option<f64>,
pub feature_noise_scale: Option<f64>,
pub enable_vol_normalization: Option<bool>,
pub bottleneck_dim: Option<usize>,
pub stochastic_depth_prob: Option<f64>,
pub asymmetric_dd_weight: Option<f64>,
pub time_reversal_mod: Option<usize>,
@@ -811,6 +812,7 @@ impl DqnTrainingProfile {
if let Some(v) = g.feature_mask_fraction { hp.feature_mask_fraction = v; }
if let Some(v) = g.feature_noise_scale { hp.feature_noise_scale = v; }
if let Some(v) = g.enable_vol_normalization { hp.enable_vol_normalization = v; }
if let Some(v) = g.bottleneck_dim { hp.bottleneck_dim = v; }
if let Some(v) = g.stochastic_depth_prob { hp.stochastic_depth_prob = v; }
if let Some(v) = g.asymmetric_dd_weight { hp.asymmetric_dd_weight = v; }
if let Some(v) = g.time_reversal_mod { hp.time_reversal_mod = v; }

173
docs/superpowers/plans/2026-03-30-gems-and-pearls-generalization.md
View File

@@ -1,4 +1,4 @@
# Gems & Pearls — 28 Generalization Techniques for DQN Trading Agent
# Gems & Pearls — 32+ Generalization Techniques for DQN Trading Agent
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
@@ -934,3 +934,174 @@ The model literally cannot learn anything that doesn't generalize.
```
- [ ] **Step 6: Compile + test + commit**
---
## THE CROWN JEWELS — Four Layers of Defense
Together these form a complete anti-memorization fortress:
| Layer | Technique | Attacks | Level |
|-------|-----------|---------|-------|
| **Gem** | Temporal Causal Bottleneck (#31) | Can only remember 2 numbers | Architecture |
| **Pearl** | Gradient Vaccine (#32) | Those 2 numbers must work on unseen data | Optimization |
| **King** | Adversarial Self-Play (#33) | Past self exploits any memorized pattern | Strategic |
| **Emperor** | Causal Intervention (#34) | Only features that CAUSE outcomes, not correlate | Epistemic |
---
### Task 33: Adversarial Self-Play with Future Self (THE KING) [PLAN]
**Phase:** 3 (Architecture) | **Difficulty:** Hard | **Impact:** TRANSFORMATIVE
**Files:**
- Create: `crates/ml/src/trainers/dqn/adversarial_self_play.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (saboteur controls microstructure)
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (checkpoint management + self-play cycle)
- Modify: `crates/ml/src/trainers/dqn/config.rs`
**The Core Idea:**
Train TWO copies of the DQN against each other. The "trader" picks entries and exits.
The "saboteur" controls market microstructure (spread, slippage, fill probability,
latency) and tries to MAXIMIZE the trader's losses. The twist: the saboteur is a
PAST version of the trader (checkpoint from 50 epochs ago). It knows exactly what the
current trader USED to do — every pattern it exploited. It specifically attacks those.
The trader must evolve AWAY from its own past strategies to survive. It can't settle
into any fixed policy because its past self knows that policy and will destroy it.
This creates an evolutionary arms race where the only stable equilibrium is a policy
that works against ALL possible adversaries — including itself.
**Why this is the king:**
1. **No technique in RL trading literature does this.** Game theory meets self-play
meets temporal evolution. AlphaGo's self-play but for market microstructure.
2. **Past-self adversary is the perfect opponent.** It has full knowledge of the
trader's old strategies — correlation patterns, entry signals, regime preferences.
It generates EXACTLY the market conditions that would have exploited those patterns.
3. **Evolutionary pressure toward robustness.** The only stable policy is one that
works regardless of market microstructure — which IS generalization.
**Implementation:**
- [ ] **Step 1: Checkpoint manager** — save model weights every 50 epochs
- [ ] **Step 2: Saboteur network** — small MLP [state_dim → 32 → 3] outputting
(spread_mult, fill_prob, slippage_mult). Initialized from past checkpoint.
- [ ] **Step 3: Self-play cycle** — alternating phases:
- Epochs 1-50: normal training (build initial policy)
- Epochs 51-55: saboteur trains against current trader (maximize losses)
- Epochs 56-100: trader trains against frozen saboteur
- Epoch 100: save new checkpoint, load as next saboteur
- [ ] **Step 4: Saboteur output → ExperienceCollectorConfig** — saboteur's output
replaces fixed spread/fill/slippage values per epoch
- [ ] **Step 5: Compile + test + commit**
---
### Task 34: Causal Intervention Training (THE EMPEROR) [PLAN]
**Phase:** 3 (Architecture) | **Difficulty:** Hard | **Impact:** TRANSFORMATIVE
**Files:**
- Create: `crates/ml/src/cuda_pipeline/causal_intervention_kernel.cu`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (intervention forward passes)
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (causal loss term)
- Modify: `crates/ml/src/trainers/dqn/config.rs`
**The Core Idea:**
Instead of learning correlations (feature X predicts profit), learn CAUSATION
(changing feature X CAUSES different outcomes). At each training step:
1. Take the current state batch [B, state_dim]
2. For each feature k, create an INTERVENED copy: set feature k to a counterfactual
value (e.g. +1, 0, -1) while keeping everything else fixed
3. Run forward passes on the intervened states
4. Compare Q-values: if changing feature k doesn't change Q, it's noise
Add a causal regularization loss:
```
L_causal = -sum_k variance(Q(do(X_k = x)) for x in {-1, 0, +1})
```
This MAXIMIZES sensitivity to features that matter and MINIMIZES sensitivity
to noise. Pearl's do-calculus applied to RL.
**Why this is the emperor:**
1. **Correlations break OOS. Causation doesn't.** The only features that survive
the causal intervention test are features that genuinely CAUSE different trading
outcomes across all regimes. Spurious correlations (which cause overfitting)
are mathematically eliminated.
2. **Nobody has done this in RL trading.** Causal inference in ML is growing,
but applying do-calculus to Q-function feature sensitivity is novel.
3. **Synergy with bottleneck (#31).** The bottleneck compresses 14 features into 2
dimensions. The causal intervention ensures those 2 dimensions capture features
with genuine causal influence, not spurious correlations.
**Implementation:**
- [ ] **Step 1: Intervention kernel** — for each feature k (0..market_dim), create
3 copies of the state batch with feature k set to {-1, 0, +1}. Total:
3 * market_dim forward passes. With market_dim=14 (active features), that's
42 extra forward passes per training step — ~10% overhead with cuBLAS.
- [ ] **Step 2: Compute per-feature causal sensitivity** — for each feature k,
var(Q(do(k=-1)), Q(do(k=0)), Q(do(k=+1))). High variance = causal feature.
- [ ] **Step 3: Causal regularization loss** — add to total loss:
`L_causal = -causal_weight * sum(sensitivities)`
The negative sign MAXIMIZES sensitivity (model is rewarded for using causal features).
- [ ] **Step 4: Causal feature importance logging** — log per-feature sensitivity
at epoch boundary. Provides interpretable feature ranking.
- [ ] **Step 5: Config**
```rust
pub enable_causal_intervention: bool, // default: false (expensive)
pub causal_weight: f64, // default: 0.1
pub causal_intervention_values: Vec<f64>, // default: [-1.0, 0.0, 1.0]
```
- [ ] **Step 6: Compile + test + commit**
---
## Complete Defense Architecture
```
┌──────────────────────────────┐
│ EMPEROR: Causal │
│ Only causal features survive │
│ (epistemic defense) │
└──────────────┬───────────────┘
│
┌──────────────▼───────────────┐
│ KING: Self-Play │
│ Past self exploits memorized │
│ patterns (strategic defense) │
└──────────────┬───────────────┘
│
┌──────────────▼───────────────┐
│ PEARL: Vaccine │
│ Only generalizing gradients │
│ survive (optimization defense)│
└──────────────┬───────────────┘
│
┌──────────────▼───────────────┐
│ GEM: Bottleneck │
│ 2D information compression │
│ (architecture defense) │
└──────────────┬───────────────┘
│
┌──────────────▼───────────────┐
│ DQN Q-Network │
│ Can only learn generalizable │
│ trading strategies │
└──────────────────────────────┘
```