SME2/SVE/NEON heuristic - ArmNN

[damdoo01-arm]

Title:
Add CpuAcc SME/SVE shape policy for Geekbench AI workloads

Description:

This PR adds a graph-level CpuAcc policy that controls whether SME/SME2 and SVE/SVE2 implementations are exposed to ACL for a given optimized ArmNN graph. - Note associated PR at ARM-software/ComputeLibrary#1294

Problem statement:

On SME2-capable client devices, some Geekbench AI workloads showed inferior scores when SME2 kernels were selected under high thread count. The main regression was seen in quantized INT8 models with awkward GEMM shapes, where the cost of SME2 packing and contention under 8-thread execution outweighed the matmul benefit. This is due a hardware/resource pressure issue that is particularly acute in situations where a single cme unit is included: SME2 can improve some shapes, but high thread count plus unfriendly Conv2D/GEMM decompositions can regress overall benchmark score.

Previous scores, note in particular the regression in Qunatized Score due to quantization overhead on the SME2 core.
Device SP HP Q
non-SME2 2655 2658 4305
SME2 2750 3991 3690

High-level approach:

The policy scans the optimized ArmNN graph and records datatype and GEMM-like shape features from Convolution2d, FullyConnected, BatchMatMul, and DepthwiseConvolution2d.

It then applies a conservative heuristic:

FP16 graphs: hide SME/SME2.
Quantized graphs: hide SME/SME2 for the known regressing shape classes, while keeping SVE/SVE2 available.
FP32 graphs: hide SME/SME2 only for detected regression-risk spatial/dense graph patterns.
Quantized graphs that keep SME enabled may have CpuAcc thread count capped for specific shape classes.
The heuristic does not rewrite the graph or force a specific kernel. It emits CpuAcc ModelOptions:

SmeEnabled = true/false
SveEnabled = true/false
NumberOfThreads = optional override
Those options are later consumed by the Neon backend model context and passed into ACL CPU feature masking.

Latest representative results:

Datatype-isolated latest run

Mode SP HP Q
NEON 3032 5324 7117
SME2 3458 5193 7072

The latest run shows the quantized path recovering from the previous S26 CME result of 3690 to roughly 7072, bringing it close to the non-SME/QMX-class quantized results while preserving strong SP/HP performance.

[gunes-arm]

Description should have sign-off

[Copilot]

Pull request overview

Adds a graph-level heuristic (“SME2/SVE/NEON shape policy”) that inspects the optimized ArmNN graph and injects CpuAcc ModelOptions to influence Arm Compute Library CPU ISA selection (SME/SME2 vs SVE/SVE2) and, in some cases, cap the CpuAcc thread count—targeting Geekbench AI regressions on SME2-capable client devices.

Changes:

• Introduces Sme2ShapePolicy to profile GEMM-like shapes/datatypes and emit CpuAcc ModelOptions (SmeEnabled, SveEnabled, optional NumberOfThreads override).
• Applies the policy during Network::Optimize() when CpuAcc is in backend preferences, and wires ISA masking into the Neon backend model context / workload factory.
• Reworks half “floating point” type-trait handling and adjusts TfLite static link library handling.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/backends/neon/NeonWorkloadFactory.cpp	Applies ACL ISA policy from the Neon model context during factory setup.
src/backends/neon/NeonBackendModelContext.hpp	Adds CpuAcc model options for SveEnabled / SmeEnabled and exposes ApplyAclIsaPolicy().
src/backends/neon/NeonBackendModelContext.cpp	Parses new options and applies ACL CPU feature masking via arm_compute::CPUInfo.
src/backends/neon/NeonBackend.cpp	Ensures GetLayerSupport(modelOptions) returns a context-specific instance (no static caching).
src/armnnUtils/Half.hpp	Replaces std::is_floating_point specialization approach with armnn::IsFloatingPoint.
src/armnn/Sme2ShapePolicy.hpp	Declares the new graph-level SME2/SVE policy entry point.
src/armnn/Sme2ShapePolicy.cpp	Implements graph profiling + heuristic decisions for SME/SVE enablement and thread capping.
src/armnn/Network.cpp	Invokes the policy for CpuAcc graphs and forwards the modified optimizedOptions.
include/armnnUtils/QuantizeHelper.hpp	Switches iterator float detection to armnn::IsFloatingPoint.
delegate/cmake/Modules/FindTfLite.cmake	Makes abseil_kernel_timeout optional for static TfLite linkage.
CMakeLists.txt	Adds the new policy files to the build.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.