SME2/SVE/NEON heuristic - ACL#1294
Closed
damdoo01-arm wants to merge 1 commit into
Closed
Conversation
damdoo01-arm
changed the title
Expose runtime controls in CPUInfo so clients can mask SME/SME2 and SVE capabilities when selecting CPU kernels. This lets higher-level frameworks steer ACL away from ISA paths that should not be used for a graph while preserving default hardware-based selection when no override is supplied. Full context in the ArmNN PR: ARM-software/armnn#820 Signed-off-by: Damien Dooley <damien.dooley@arm.com> Change-Id: I602cebdd58942930d248948788bfac9e2be56474
damdoo01-arm
force-pushed
the
damdoo01/sme2_neon_heuristic
branch
from
June 16, 2026 12:10
992d402 to
6d46ae5
Compare
gunes-arm
requested changes
Jun 16, 2026
gunes-arm
left a comment
gunes-arm
left a comment
Contributor
There was a problem hiding this comment.
Can we change the commit message as:
feat: Add experimental SME/SVE runtime selection controls
And, also make the PR title the same?
| */ | ||
| bool has_sme_b16f32() const; | ||
| /** Sets whether SVE and SVE2 implementations are allowed to be selected at runtime. | ||
| * |
Contributor
There was a problem hiding this comment.
Can you add the following into all?
@experimental This API is under development and may change or be removed without notice in future releases.
Author
|
Superseded by #1295. |
CianMcGriskinARM
pushed a commit
to ARM-software/armnn
that referenced
this pull request
Jul 2, 2026
Title: Add CpuAcc SME/SVE shape policy for Geekbench AI workloads Description: This PR adds a graph-level CpuAcc policy that controls whether SME/SME2 and SVE/SVE2 implementations are exposed to ACL for a given optimized ArmNN graph. - Note associated PR at ARM-software/ComputeLibrary#1294 Problem statement: On SME2-capable client devices, some Geekbench AI workloads showed inferior scores when SME2 kernels were selected under high thread count. The main regression was seen in quantized INT8 models with awkward GEMM shapes, where the cost of SME2 packing and contention under 8-thread execution outweighed the matmul benefit. This is due a hardware/resource pressure issue that is particularly acute in situations where a single cme unit is included: SME2 can improve some shapes, but high thread count plus unfriendly Conv2D/GEMM decompositions can regress overall benchmark score. Previous scores, note in particular the regression in Qunatized Score due to quantization overhead on the SME2 core. Device SP HP Q non-SME2 2655 2658 4305 SME2 2750 3991 3690 High-level approach: The policy scans the optimized ArmNN graph and records datatype and GEMM-like shape features from Convolution2d, FullyConnected, BatchMatMul, and DepthwiseConvolution2d. It then applies a conservative heuristic: FP16 graphs: hide SME/SME2. Quantized graphs: hide SME/SME2 for the known regressing shape classes, while keeping SVE/SVE2 available. FP32 graphs: hide SME/SME2 only for detected regression-risk spatial/dense graph patterns. Quantized graphs that keep SME enabled may have CpuAcc thread count capped for specific shape classes. The heuristic does not rewrite the graph or force a specific kernel. It emits CpuAcc ModelOptions: SmeEnabled = true/false SveEnabled = true/false NumberOfThreads = optional override Those options are later consumed by the Neon backend model context and passed into ACL CPU feature masking. Latest representative results: Datatype-isolated latest run Mode SP HP Q NEON 3032 5324 7117 SME2 3458 5193 7072 The latest run shows the quantized path recovering from the previous S26 CME result of 3690 to roughly 7072, bringing it close to the non-SME/QMX-class quantized results while preserving strong SP/HP performance. --------- Signed-off-by: Damien Dooley <damien.dooley@arm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title:
Allow runtime masking of SVE/SVE2 CPU feature exposure
Description:
This PR adds runtime control over whether ACL exposes SVE/SVE2 capabilities through arm_compute::CPUInfo. For full context, please refer to the associated ArmNN PR at: ARM-software/armnn#820
Problem statement:
ArmNN needs a way to steer ACL away from SME/SME2 and, in some cases, SVE/SVE2 kernel families when graph-level shape heuristics indicate that those paths regress performance. The regression is most visible on SME2-capable hardware under high thread count, where the hardware/resource pressure around SME2 packing can dominate the expected matmul acceleration for some Geekbench AI shapes.
High-level approach:
ACL already had runtime masking for SME/SME2 via set_sme_allowed(). This PR adds equivalent SVE/SVE2 masking:
void CPUInfo::set_sve_allowed(bool is_allowed);
When disabled, ACL reports SVE/SVE2 and related features as unavailable through:
has_sve()
has_sve2()
has_svebf16()
has_svei8mm()
has_svef32mm()
get_isa()
This lets ArmNN apply its graph-level policy while keeping ACL’s existing kernel selection mechanisms intact.
Relationship to ArmNN PR:
The ArmNN PR emits CpuAcc options such as:
SmeEnabled=false
SveEnabled=true
or:
SmeEnabled=false
SveEnabled=false
The ACL PR provides the underlying mechanism that makes those options affect runtime kernel selection.