foxhunt/WAVE_30_RISK_ACTION_MASKING_TDD.md

Agent 30: TDD - Risk-Based Action Masking Tests

Date: 2025-11-13 Status: ✅ COMPLETE - Test Suite Created (Ready for Implementation) Test File: /home/jgrusewski/Work/foxhunt/ml/tests/risk_action_masking_test.rs Lines of Code: ~1,050 (comprehensive test coverage)

---

Executive Summary

Agent 30 created a complete Test-Driven Development (TDD) test suite for risk-based action masking in the DQN system. The tests are designed to FAIL initially (as expected in TDD), waiting for Agent 31 to implement the actual risk masking logic.

The test suite validates:

• Position limit enforcement with constraints
• Drawdown-based masking (restrict aggressive actions when equity draws down)
• VaR (Value-at-Risk) limits preventing excessive loss potential
• Risk-reducing actions always allowed (selling when long, buying when short)
• Q-value computation respecting masking
• Exploration & exploitation respecting masks
• Dynamic mask updates per training step
• Fallback handling when all actions masked
• Performance validation (<1ms per mask calculation)
• Diversity metrics excluding masked actions
• Multi-constraint interaction (most restrictive wins)

---

Test Suite Structure

1. Core Infrastructure

MockTradingState - Simulates trading state with:

• Current position (-2.0 to +2.0)
• Equity value and starting capital
• Value-at-Risk (1-day loss at 95% confidence)
• Drawdown percentage
• Mid price for calculations

RiskActionMasking - Action masking engine (API to implement):

fn get_valid_actions(
    state: &MockTradingState,
    max_position: f64,           // Position limit (typically ±2.0)
    max_drawdown_pct: f64,        // Drawdown threshold (typically 12%)
    max_var: Option<f64>,         // VaR limit in dollars
) -> Vec<usize>                   // Valid action indices (0-44)

fn get_action_mask(...) -> Vec<bool>  // Boolean mask

fn count_masked(mask: &[bool]) -> usize  // Count masked actions

---

Test Coverage (12 Tests)

Test 1: Position Limit Enforcement ✅

File: test_mask_actions_exceeding_position_limit() Validates:

• Long100 (target=+1.0) masked when max_position < 1.0
• Short100 (target=-1.0) masked when max_position < 1.0
• Position constraints work correctly

Scenarios:

• max_position=2.0: Allows all exposures
• max_position=0.8: Masks ±100% exposures
• Expected: Exposure-based filtering

---

Test 2: Drawdown-Based Masking ✅

File: test_mask_actions_exceeding_drawdown() Validates:

• Aggressive actions masked when drawdown > threshold
• Patient actions allowed (especially risk-reducing)
• Drawdown=15%, threshold=12% → Aggressive masked

Risk-Reducing Detection:

• Long position + sell action → Allowed
• Short position + buy action → Allowed
• Hold always allowed during drawdown

---

Test 3: VaR-Based Masking ✅

File: test_mask_actions_violating_var_limit() Validates:

• Large exposures mask when projected VaR would exceed limit
• VaR=$5k, limit=$6k, Large position → Masked
• Small exposures (±50%, Flat) → Allowed

Implementation Detail: VaR recalculated as:

new_var = current_var × (1 + |new_position| × 0.1)

---

Test 4: Risk-Reducing Actions Always Allowed ✅

File: test_allow_actions_reducing_risk() Validates:

• Long position (pos=+1.0), Selling allowed
• Short position (pos=-1.0), Buying allowed
• HOLD always allowed (risk-neutral)

Logic:

if position > 0.0 && action.is_sell() { allow }
if position < 0.0 && action.is_buy() { allow }
if action.is_hold() { allow }

---

Test 5: Mask Count Logging ✅

File: test_mask_count_logged() Validates:

• Correct count of masked actions
• Log format: "X/45 actions masked"
• Permissive (no mask) vs. Restrictive (some mask)

Expected Log Output:

Mask statistics: 9/45 actions masked (20% restriction)

---

Test 6: Q-Value Computation ✅

File: test_q_values_only_for_valid_actions() Validates:

• Q-network forward pass includes batch dimension
• Only valid actions receive Q-values
• Masked action Q-values set to -infinity or excluded

Implementation Detail:

• Valid indices must be non-empty
• Q-values for valid actions must be finite
• Masked actions excluded from selection

---

Test 7: Epsilon Exploration Respects Mask ✅

File: test_epsilon_exploration_respects_mask() Validates:

• Epsilon-greedy samples only from valid actions
• No invalid actions can be selected
• Uniform random from valid set

Example:

epsilon=0.1: With 10% probability, sample from valid actions
epsilon=0.9: With 90% probability, sample from valid actions

---

Test 8: Greedy Selection Respects Mask ✅

File: test_greedy_selection_respects_mask() Validates:

• Argmax operates only over valid actions
• Highest Q-value among valid actions selected
• Invalid actions never selected

---

Test 9: Mask Updates Each Step ✅

File: test_mask_updates_each_step() Validates:

• Mask recalculated at every training step
• State changes (position, drawdown, VaR) update mask
• No stale masks from previous steps

Timeline:

• Step 1: pos=0.0, drawdown=0% → Mask A
• Step 2: pos=+1.0, drawdown=10% → Mask B
• Step 3: pos=+1.0, drawdown=15% → Mask C (different from B)

---

Test 10: All-Masked Fallback ✅

File: test_all_actions_masked_fallback() Validates:

• If all actions masked, HOLD action available as fallback
• Never returns empty valid action set
• Graceful degradation under extreme constraints

Edge Case:

• max_position=0.001, max_drawdown=0.001, max_var=$1
• Expected: HOLD actions (indices 18-26) remain valid

---

Test 11: Performance Validation ✅

File: test_masking_performance() Validates:

• Mask calculation completes in <1ms
• O(45) masking is instant
• No performance regression from multi-constraint logic

Requirement: Used every training step, must be efficient

---

Test 12: Masked Actions Not in Diversity ✅

File: test_masked_actions_not_in_diversity_count() Validates:

• Diversity metric only counts valid actions
• Masked actions don't affect diversity statistics
• 100% diversity = all valid actions used

Formula:

diversity = (unique_valid_actions_used) / (total_valid_actions)

---

Integration Tests (3 Additional)

Test 13: Realistic ES Futures Scenario ✅

Validates: Real-world trading constraints

• Capital: $100k
• Position: 0.5 contracts
• Drawdown: 5% (well below 12% threshold)
• VaR: $5k
• Expected: Many valid actions (>10)

---

Test 14: Extreme Constraints Stress Test ✅

Validates: Graceful degradation

• Position limit: ±0.1
• Drawdown threshold: 5%
• VaR limit: $500
• Severe drawdown: -20%
• Expected: Always >= 1 valid action (fallback)

---

Test 15: Multiple Constraints Interaction ✅

Validates: Constraints work together (most restrictive wins)

• Position-only mask: 10+ valid actions
• Drawdown-only mask: 10+ valid actions
• VaR-only mask: 10+ valid actions
• Combined (all 3): <= min(position, drawdown, var)

Logic: Most restrictive constraint dominates

---

Key Test Assertions

Position Constraint

// Long100 (target=+1.0) is masked when max_position < 1.0
assert!(!mask[36..45].iter().all(|&v| v));

Drawdown Constraint

// Aggressive actions masked when drawdown > threshold
assert!(!aggressive_actions.iter().all(|&idx| mask[idx]));

Risk-Reducing Always Allowed

// Selling when long is always allowed
if state.position > 0.0 && action.is_sell() {
    assert!(mask[idx]);
}

Fallback Guarantee

// If all masked, HOLD actions available
if valid_count == 0 {
    assert!(!flat_indices.is_empty());
}

---

Test Statistics

Metric	Value
Total Tests	15
Core Tests	12
Integration Tests	3
Lines of Code	~1,050
Scenarios Covered	25+
Assertion Count	~60
Expected Status	ALL FAIL (TDD)

---

Implementation Roadmap (Agent 31)

Phase 1: Basic Structure (30-45 min)

1. Implement get_valid_actions() function
2. Add position limit filtering
3. Add basic masking logic

Phase 2: Advanced Constraints (45-60 min)

1. Add drawdown-based filtering
2. Identify risk-reducing actions (positive positions + sell, negative + buy)
3. Always allow HOLD actions

Phase 3: VaR Integration (30-45 min)

1. Add VaR limit checking
2. Implement VaR projection calculation
3. Integrate with position/drawdown logic

Phase 4: Testing & Optimization (30-45 min)

1. Run all 15 tests
2. Fix implementation to pass each test
3. Ensure <1ms performance

Phase 5: Integration (15-30 min)

1. Integrate with DQN training loop
2. Add logging ("X/45 actions masked")
3. Validate in actual training

---

Expected Test Results (Before Implementation)

All tests should FAIL initially with:

• Assertion failures (actual != expected)
• Index out of bounds errors
• Mock API not implemented

test_mask_actions_exceeding_position_limit ... FAILED
test_mask_actions_exceeding_drawdown ... FAILED
test_mask_actions_violating_var_limit ... FAILED
... (12 more failures)

failures: 15

test result: FAILED. 0 passed; 15 failed; 0 ignored

---

Success Criteria (Post-Implementation)

test_mask_actions_exceeding_position_limit ... ok
test_mask_actions_exceeding_drawdown ... ok
test_mask_actions_violating_var_limit ... ok
... (12 more passing)

test result: ok. 15 passed; 0 failed; 0 ignored

---

Files Created

1. /home/jgrusewski/Work/foxhunt/ml/tests/risk_action_masking_test.rs

   • 1,050 lines of comprehensive test code
   • 15 complete test functions
   • Full API specification in comments

2. /home/jgrusewski/Work/foxhunt/WAVE_30_RISK_ACTION_MASKING_TDD.md

   • This documentation file
   • Complete implementation guide for Agent 31

---

Test Execution

Run All Tests

cargo test --test risk_action_masking_test -p ml

Run Specific Test

cargo test --test risk_action_masking_test test_mask_actions_exceeding_position_limit -p ml -- --nocapture

Run with Output

cargo test --test risk_action_masking_test -p ml -- --nocapture --test-threads=1

---

Next Steps (Agent 31)

1. Review this test file (risk_action_masking_test.rs)
2. Understand the API: get_valid_actions(), get_action_mask(), count_masked()
3. Implement the logic:
   • Position limit filtering (check if exposure target > max_position)
   • Drawdown-based filtering (mask Aggressive urgency if drawdown > threshold)
   • VaR filtering (project new position VaR, mask if > limit)
   • Risk-reduction check (allow if reduces risk)
   • Fallback (always allow HOLD)
4. Run tests: Watch them turn from FAILED to ok
5. Validate integration: Integrate with DQN training loop

---

Technical Notes

FactoredAction to RiskActionMasking Mapping

Index Range | Exposure  | Meaning
0-8         | Short100  | -100% exposure
9-17        | Short50   | -50% exposure
18-26       | Flat      | 0% (neutral)
27-35       | Long50    | +50% exposure
36-44       | Long100   | +100% exposure

Each exposure level has 9 variants (3 order types × 3 urgency levels).

Drawdown Sensitivity

Drawdown < threshold:    All actions allowed
Drawdown == threshold:   Boundary case (implementation decides)
Drawdown > threshold:    Aggressive actions masked, Patient allowed

VaR Projection

new_var = current_var × (1.0 + |new_position| × 0.1)

Example:
current_var = $5,000
new_position = 1.0
new_var = $5,000 × (1.0 + 1.0 × 0.1) = $5,500

Risk-Reducing Actions

Long position (pos > 0):
  - Sell (Short50, Short100) → Reduces risk → ALLOW
  - Flat (maintain position) → Neutral → ALLOW
  - Buy (Long50, Long100) → Increases risk → May mask

Short position (pos < 0):
  - Buy (Long50, Long100) → Reduces risk → ALLOW
  - Flat (maintain position) → Neutral → ALLOW
  - Sell (Short50, Short100) → Increases risk → May mask

---

Documentation References

• Action Space: /ml/src/dqn/action_space.rs (FactoredAction definitions)
• Existing Masking: dqn_action_masking_integration_test.rs (position-only masking)
• DQN Trainer: /ml/src/trainers/dqn.rs (where masking integrates)

---

Success Metrics

Metric	Target	Expected
Test Pass Rate	100%	15/15 passing
Implementation Time	2-3 hours	TBD (Agent 31)
Performance	<1ms/mask	Measured in Test 11
Code Quality	0 errors	Zero Clippy warnings
Test Coverage	25+ scenarios	15 tests × 2-3 scenarios each

---

Status: ✅ COMPLETE - Waiting for Agent 31 to implement the risk masking logic.