Choosing between fixed-width NEON and length-agnostic SVE for portable AArch64 SIMD

Direct Answer

Use -march=armv8-a without +sve to restrict code generation to NEON/FP in both GCC and Clang; no portable per-function attribute forces NEON across both compilers. For SVE with NEON fallback, isolate SVE kernels in translation units compiled with -march=armv8-a+sve or +sve2, guard entry with getauxval(AT_HWCAP) & HWCAP_SVE, and select implementations with GCC ifunc resolvers (Clang support is limited). Choose a single fat binary when distribution simplicity matters and resolver behavior is verified; choose separate NEON/SVE packages when SVE code is large, per-microarchitecture -mtune is needed, or toolchain ifunc support is uneven.

Confirmed Facts

• NEON is fixed 128-bit and effectively mandatory for AArch64; GCC and Clang enable it by default for -march=armv8-a.
• SVE is optional and length-agnostic (VL 128–2048 bits in 128-bit steps). Enable with -march=armv8-a+sve or +sve2 and write length-agnostic loops via <arm_sve.h> intrinsics or auto-vectorization.
• No standardized pragma or attribute forces a specific SIMD backend in both GCC and Clang. OpenMP simd clauses, -mtune, optimization level, and alignment influence but do not guarantee NEON or SVE selection.
• Runtime SVE detection uses getauxval(AT_HWCAP) for HWCAP_SVE/HWCAP_SVE2. On heterogeneous big.LITTLE systems, this may reflect only the boot CPU; check /sys/devices/system/cpu/cpu*/features before pinning SVE work.
• GCC ifunc resolvers can select SVE or NEON implementations at load time based on HWCAP_SVE. Clang ifunc support on AArch64 is more limited; verify linker behavior in your toolchain.
• Compiling SVE instructions into a binary is safe only if that code path is never entered on a CPU lacking SVE; an ifunc resolver or explicit HWCAP guard is required to avoid SIGILL.

Likely Explanation (Not Fully Verified)

• Auto-vectorizer backend choice is cost-model driven; -march=armv8-a does not guarantee every vectorizable loop becomes NEON, and -march=...+sve does not guarantee SVE for every loop. Inspect assembly with objdump -d to confirm.
• For most projects, a single fat binary with ifunc resolvers reduces distribution complexity, but resolver testing on both SVE-capable and SVE-less hardware (or QEMU -cpu without +sve) is essential.
• Separate packages make sense when SVE-optimized kernels are large (e.g., ML inference, HPC loops) and you want per-microarchitecture -mtune without bloating the NEON binary.

Steps for This Case

1. Create a NEON-only baseline: compile core libraries with -march=armv8-a -O2 -ftree-vectorize. Verify NEON-only instructions via objdump -d and confirm no SVE mnemonics such as whilelt or predicate-register loads.
2. Isolate SVE-optimized kernels in separate source files. Compile them with -march=armv8-a+sve -O2 (or +sve2 for SVE2). Use <arm_sve.h> intrinsics for length-agnostic loops; avoid hard-coded VL assumptions.
3. Implement a GCC ifunc resolver for each SVE entry point:

   extern void kernel_neon(void);
   extern void kernel_sve(void);
   
   void *resolve_kernel(void) {
       unsigned long hwcap = getauxval(AT_HWCAP);
       return (hwcap & HWCAP_SVE) ? (void*)kernel_sve : (void*)kernel_neon;
   }
   
   void kernel(void) __attribute__((ifunc("resolve_kernel")));

4. Test the resolver on an SVE-capable system and on a system or QEMU without SVE. Confirm no SIGILL and correct fallback.
5. On big.LITTLE targets, before launching SVE threads, read /sys/devices/system/cpu/cpu*/features for the sve flag and set thread affinity accordingly.
6. Decide packaging:
   • Single fat binary: if resolver tests pass, binary size increase is acceptable, and you want one artifact.
   • Separate packages: if SVE code adds substantial size, you need per-CPU -mtune, or Clang ifunc is required but unsupported.

One Missing Diagnostic Detail

Which compiler (GCC version, Clang version) and linker (ld.bfd, ld.gold, lld) are you using for the final link? Clang ifunc support on AArch64 varies by version and linker; this determines whether a single fat binary is viable or separate packages are required.