fix(ml-alpha): drop synchronize() in per-horizon pool + head bindings
After adaf275af removed the synchronizes in PerHorizonTrainState's
forward_with_blend / backward_through_blend, smoke alpha-perception-9l6hw
still failed with CUDA_ERROR_STREAM_CAPTURE_UNSUPPORTED. Root cause:
PerHorizonAttentionPool::{forward,backward} and PerHorizonResidualHead::
{forward,backward} each end with their own self.stream.synchronize(),
which is illegal during CUDA Graph capture.
Same fix: drop the four synchronizes. Same-stream issue order ensures
the next kernel sees the previous one's output. Capture invariant
preserved.
Verified locally: per_horizon_full_pipeline_smoke (2/2), attention pool
numgrad (1/1), residual head numgrad (1/1) all pass on RTX 3050.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -82,7 +82,8 @@ impl PerHorizonAttentionPool {
|
||||
.launch(cfg)
|
||||
.context("per_horizon_attention_pool_fwd")?;
|
||||
}
|
||||
self.stream.synchronize()?;
|
||||
// No synchronize: same-stream issue order is sufficient and
|
||||
// synchronize is illegal during CUDA Graph capture.
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -122,7 +123,8 @@ impl PerHorizonAttentionPool {
|
||||
.launch(cfg)
|
||||
.context("per_horizon_attention_pool_bwd")?;
|
||||
}
|
||||
self.stream.synchronize()?;
|
||||
// No synchronize: same-stream issue order is sufficient and
|
||||
// synchronize is illegal during CUDA Graph capture.
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -71,7 +71,8 @@ impl PerHorizonResidualHead {
|
||||
.launch(cfg)
|
||||
.context("per_horizon_residual_head_fwd")?;
|
||||
}
|
||||
self.stream.synchronize()?;
|
||||
// No synchronize: same-stream issue order is sufficient and
|
||||
// synchronize is illegal during CUDA Graph capture.
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -107,7 +108,8 @@ impl PerHorizonResidualHead {
|
||||
.launch(cfg)
|
||||
.context("per_horizon_residual_head_bwd")?;
|
||||
}
|
||||
self.stream.synchronize()?;
|
||||
// No synchronize: same-stream issue order is sufficient and
|
||||
// synchronize is illegal during CUDA Graph capture.
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user