Platform
a2a3sim (Ascend 910B/C simulation)
Runtime Variant
All / Unknown
Description
Valid PyPTO programs using explicit FFTS cross-core synchronization compile
for A2/A3 hardware but fail during A2/A3 simulator kernel compilation. PTOAS
emits the device FFTS message helpers and synchronization builtins, while the
CPU simulator compilation environment provides no compatible definitions or
event semantics.
This currently blocks the DeepSeek V4 DSpark CSA, HCA, and SWA simulator
tests. Their generated AIC/AIV kernels use the same explicit sync_set and
sync_wait operations. Failure occurs before numerical execution.
Steps to Reproduce
Use the following matching dependency chain:
- pypto-lib:
3daaee2c0b3f7ff274cc9980408287ad0bf47cec
- PyPTO:
a9b9e0be9730f6b5fe63fb151b6d4331f4d770c8
- simpler:
39ce891dbb3f665e72e99b4a1387b47012fb18bd
- PTOAS:
v0.60
- PTO ISA:
5a4f74cbf627d4aac2e0ce10d5e0d8b118343265
Run a DSpark attention entry on CPU simulation:
python models/deepseek_v4_flash_dspark/decode_csa.py -p a2a3sim
The same compile failure is reproduced by a minimal mixed kernel that:
- Calls
pl.system.set_ffts(workspace) on AIC and both AIV lanes.
- Stores an AIC matmul result to GM.
- Calls
pl.system.sync_set(0, pipe=pl.PipeType.FIX, ffts_mode=2, core_type=pl.KernelType.AIC).
- Calls
pl.system.sync_wait(0, pipe=pl.PipeType.MTE2, core_type=pl.KernelType.AIV) before each AIV lane reads its half.
- Compiles with
platform="a2a3sim".
The failing downstream CI job is
https://github.com/hw-native-sys/pypto-lib/actions/runs/34444660251/job/102766782704.
Expected Behavior
The A2/A3 simulator should compile and execute explicit FFTS mode-2 events
with device-compatible synchronization semantics:
- An AIC notification is visible to both AIV lanes.
- An AIC wait completes only after both AIV lanes notify it.
- Notifications are consuming credits, so queued and repeated events do not
collapse into one boolean flag.
- Event state is isolated by simulated device and cluster and reset between
runs.
Actual Behavior
Generated simulator C++ fails to compile with errors including:
error: '__builtin_cce_wait_flag_dev' was not declared in this scope
error: 'FFTS_MODE_VAL' was not declared in this scope
error: 'getFFTSMsg' was not declared in this scope
error: '__builtin_cce_ffts_cross_core_sync' was not declared in this scope
CSA, HCA, and SWA all fail in this compilation phase. Native A2/A3 execution
passes, confirming that this is specific to the simulator compatibility path.
Git Commit ID
39ce891dbb3f665e72e99b4a1387b47012fb18bd
CANN Version
9.0.0
Driver Version
Not applicable to CPU simulation.
Host Platform
Linux (aarch64)
Additional Context
Proposed fix: #2186.
The fix supplies the emitted FFTS API on a2a3sim, uses the existing
runtime-owned per-device/per-cluster shared registry for cross-library event
credits, and forwards the existing simulator hooks into each independently
loaded kernel library. Unsupported modes/counts fail explicitly. Native and A5
compiler paths are unchanged.
Validation of the proposed fix includes the minimal two-AIV transfer golden,
cross-library event protocol tests, all five simulator scripts selected by the
downstream PR, and the runtime unit suite.
Platform
a2a3sim (Ascend 910B/C simulation)
Runtime Variant
All / Unknown
Description
Valid PyPTO programs using explicit FFTS cross-core synchronization compile
for A2/A3 hardware but fail during A2/A3 simulator kernel compilation. PTOAS
emits the device FFTS message helpers and synchronization builtins, while the
CPU simulator compilation environment provides no compatible definitions or
event semantics.
This currently blocks the DeepSeek V4 DSpark CSA, HCA, and SWA simulator
tests. Their generated AIC/AIV kernels use the same explicit
sync_setandsync_waitoperations. Failure occurs before numerical execution.Steps to Reproduce
Use the following matching dependency chain:
3daaee2c0b3f7ff274cc9980408287ad0bf47ceca9b9e0be9730f6b5fe63fb151b6d4331f4d770c839ce891dbb3f665e72e99b4a1387b47012fb18bdv0.605a4f74cbf627d4aac2e0ce10d5e0d8b118343265Run a DSpark attention entry on CPU simulation:
The same compile failure is reproduced by a minimal mixed kernel that:
pl.system.set_ffts(workspace)on AIC and both AIV lanes.pl.system.sync_set(0, pipe=pl.PipeType.FIX, ffts_mode=2, core_type=pl.KernelType.AIC).pl.system.sync_wait(0, pipe=pl.PipeType.MTE2, core_type=pl.KernelType.AIV)before each AIV lane reads its half.platform="a2a3sim".The failing downstream CI job is
https://github.com/hw-native-sys/pypto-lib/actions/runs/34444660251/job/102766782704.
Expected Behavior
The A2/A3 simulator should compile and execute explicit FFTS mode-2 events
with device-compatible synchronization semantics:
collapse into one boolean flag.
runs.
Actual Behavior
Generated simulator C++ fails to compile with errors including:
CSA, HCA, and SWA all fail in this compilation phase. Native A2/A3 execution
passes, confirming that this is specific to the simulator compatibility path.
Git Commit ID
39ce891dbb3f665e72e99b4a1387b47012fb18bdCANN Version
9.0.0
Driver Version
Not applicable to CPU simulation.
Host Platform
Linux (aarch64)
Additional Context
Proposed fix: #2186.
The fix supplies the emitted FFTS API on
a2a3sim, uses the existingruntime-owned per-device/per-cluster shared registry for cross-library event
credits, and forwards the existing simulator hooks into each independently
loaded kernel library. Unsupported modes/counts fail explicitly. Native and A5
compiler paths are unchanged.
Validation of the proposed fix includes the minimal two-AIV transfer golden,
cross-library event protocol tests, all five simulator scripts selected by the
downstream PR, and the runtime unit suite.