3853988af7
- Fixed PSO budget calculation bug in ml/src/hyperopt/optimizer.rs - Root cause: Division by n_particles in sequential execution - Now correctly calculates max_iters = remaining_trials (no division) - Result: 50 trials complete instead of 23 (100% vs 46%) - Added comprehensive DQN hyperopt results analysis - 39/50 trials analyzed across 2 RunPod deployments - Best hyperparameters identified: LR 4.89e-5 (ultra-low) - Created DQN_HYPEROPT_RESULTS_SUMMARY.md with expert validation - GitLab CI/CD pipeline operational (48 lines fixed) - Fixed YAML syntax errors (unquoted colons) - All 7 jobs validated and working - Warning cleanup complete (136 → 0 warnings) - Removed 143 lines dead code - Fixed visibility, unused imports, Debug traits - Archived Wave D reports to docs/archive/ - 8 early stopping reports moved - Root directory cleaned up 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
100 lines
1.8 KiB
Markdown
100 lines
1.8 KiB
Markdown
# S3 Monitor Quick Reference
|
|
|
|
## Prerequisites
|
|
|
|
```bash
|
|
export AWS_ACCESS_KEY_ID="your_runpod_access_key"
|
|
export AWS_SECRET_ACCESS_KEY="your_runpod_secret_key"
|
|
```
|
|
|
|
## Common Commands
|
|
|
|
### List Logs
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --list
|
|
```
|
|
|
|
### Quick Check (last 20 lines)
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --tail 20
|
|
```
|
|
|
|
### Follow in Real-Time
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --follow
|
|
```
|
|
|
|
### Follow with Context
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --follow --tail 10
|
|
```
|
|
|
|
### Filter for Metrics
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --follow --filter "epoch|loss|accuracy"
|
|
```
|
|
|
|
### Show Errors Only
|
|
```bash
|
|
foxhunt-deploy monitor <pod_id> --follow --filter "ERROR|WARN|CRITICAL"
|
|
```
|
|
|
|
## Config
|
|
|
|
Location: `~/.foxhunt/config.toml`
|
|
|
|
```toml
|
|
[s3]
|
|
endpoint = "https://s3api-eur-is-1.runpod.io"
|
|
bucket = "se3zdnb5o4"
|
|
region = "us-east-1"
|
|
poll_interval_secs = 5 # Decrease for faster updates
|
|
```
|
|
|
|
## Colorization
|
|
|
|
- 🟢 INFO (green)
|
|
- 🟡 WARN (yellow)
|
|
- 🔴 ERROR (red)
|
|
- 🔵 DEBUG (cyan)
|
|
- ⚫ TRACE (dimmed)
|
|
|
|
## Troubleshooting
|
|
|
|
### No logs found
|
|
```bash
|
|
# Check credentials
|
|
echo $AWS_ACCESS_KEY_ID
|
|
|
|
# List logs
|
|
foxhunt-deploy monitor <pod_id> --list
|
|
|
|
# Verify pod ID in RunPod dashboard
|
|
```
|
|
|
|
### Permission denied
|
|
```bash
|
|
# Check S3 config
|
|
cat ~/.foxhunt/config.toml | grep -A 4 "\[s3\]"
|
|
|
|
# Verify credentials match RunPod account
|
|
```
|
|
|
|
### Slow updates
|
|
```bash
|
|
# Edit config: reduce poll_interval_secs
|
|
nano ~/.foxhunt/config.toml
|
|
# Change: poll_interval_secs = 2
|
|
|
|
# Or check network
|
|
ping s3api-eur-is-1.runpod.io
|
|
```
|
|
|
|
## Tips
|
|
|
|
1. **Ctrl+C** to exit follow mode
|
|
2. **Completion auto-detected**: Exits when training finishes
|
|
3. **Regex patterns**: Use standard regex syntax
|
|
4. **Multiple flags**: Combine `--follow --tail 10 --filter "epoch"`
|
|
5. **Log priority**: training.log > stdout > stderr (auto-selected)
|