Martin Storsjö 8e4c904c8e checkasm: ac3dsp: Increase the float tolerance for sum_square_butterfly_float ...

Accept up to 13 ULP difference.

This fixes running "checkasm --test=ac3dsp 3044836819" on ARM.

Depending on how the SIMD implementations aggregate numbers,
larger/smaller values might not end up accumulated in exactly
the same way; the current NEON implementation for ARM aggregates
into vectors of 2 elements. If it would aggregate into vectors
of 4 elements instead, like the AArch64 version does, this particular
case would end up with a smaller difference.

2025-08-10 02:27:44 +00:00

5.8 KiB

Raw Blame History

View Raw