lib/crc: x86/crc32c: question about the "Future work" note (crc32q + VPCLMULQDQ on Zen)

lib/crc: x86/crc32c: question about the "Future work" note (crc32q + VPCLMULQDQ on Zen)

[parker]

Hi Eric,

There's a comment in crc32c_arch() in lib/crc/x86/crc32.h that says that
the optimal strategy on Zen 3--5 is to use crc32q and VPCLMULQDQ in
parallel, but that different numbers of streams and vector lengths are
optimal on each microarchitecture.  I've been experimenting with this on
Zen 5, and am curious about what you're looking for here.

I've tuned an impl for Zen 5 only: two zmm registers folding with
VPCLMULQDQ while four crc32q streams run alongside, merged at the end
using multipliers from a 1 KiB table.  Compared with
crc32_lsb_vpclmul_avx512() at len=4096:

    AMD Ryzen 7 9800X3D (Zen 5)            +53%   (crc_kunit)
    AMD EPYC 9V45 (Zen 5)                  +57%   (userspace)
    AMD EPYC 9V74 (Zen 4)                  +15%, but slower at len=1024
    Intel Xeon 6975P-C (Granite Rapids)    -43%


1. What would you want a solution to look like?  A variant per
   microarchitecture, a single variant for whichever one is worth it, or
   nothing until there's an approach that covers several of them?

2. Would a Zen 5-only variant be acceptable for now?


Thanks,
Parker

Re: lib/crc: x86/crc32c: question about the "Future work" note (crc32q + VPCLMULQDQ on Zen)

[Eric Biggers]

Hi Parker,

On Fri, Nov 13, 2026 at 10:27:09PM -0500, parker@cronstable.dev wrote:
> Hi Eric,
> 
> There's a comment in crc32c_arch() in lib/crc/x86/crc32.h that says that
> the optimal strategy on Zen 3--5 is to use crc32q and VPCLMULQDQ in
> parallel, but that different numbers of streams and vector lengths are
> optimal on each microarchitecture.  I've been experimenting with this on
> Zen 5, and am curious about what you're looking for here.
> 
> I've tuned an impl for Zen 5 only: two zmm registers folding with
> VPCLMULQDQ while four crc32q streams run alongside, merged at the end
> using multipliers from a 1 KiB table.  Compared with
> crc32_lsb_vpclmul_avx512() at len=4096:
> 
>     AMD Ryzen 7 9800X3D (Zen 5)            +53%   (crc_kunit)
>     AMD EPYC 9V45 (Zen 5)                  +57%   (userspace)
>     AMD EPYC 9V74 (Zen 4)                  +15%, but slower at len=1024
>     Intel Xeon 6975P-C (Granite Rapids)    -43%
> 
> 
> 1. What would you want a solution to look like?  A variant per
>    microarchitecture, a single variant for whichever one is worth it, or
>    nothing until there's an approach that covers several of them?
> 
> 2. Would a Zen 5-only variant be acceptable for now?

I don't think a separate CRC-32C implementation for each
microarchitecture is worth it currently.  CRC-32C is already incredibly
fast.  In particular, crc32_lsb_vpclmul_avx512() (which we have anyway,
to support the IEEE CRC-32) already reaches up to 80 GB/s on CPUs with
good AVX-512 support, so we already have something better than the
traditional 3-way scalar CRC on them.

And "a separate variant for each microarchitecture" is indeed what each
of Zen 3, Zen 4, and Zen 5 would need, which is annoying.

Also, virtually every kernel has CONFIG_CRC32=y enabled, so it would be
nice not to bloat it too much.

If we could generate them all from a template, that could make it more
worthwhile.  But it's hard to handle different numbers of streams in
that way, and Zen 3 is extra special since it doesn't support AVX-512.

Generally, when we add CRC or crypto assembly files, each one is useful
on multiple CPU microarchitectures from both Intel and AMD.

I'm kind of waiting for Zen 6 and Nova Lake at this point and hoping
that a single CRC-32C implementation (maybe 2 ZMM + 4 crc32q, as you
implemented) will be close to optimum on all of those and be
meaningfully better than crc32_lsb_vpclmul_avx512() on each.  In that
case, it would be more worthwhile to add.

So yes, I think your implementation may be what we want eventually.  I'd
just like to be a bit more sure it won't be a one-off for Zen 5.

(And FWIW, Zen 6 having the same 3 scalar CRC units as Zen 5 is likely,
but also not guaranteed.  Tech news is reporting that one of the
multiplication units is going away since it was deemed not worthwhile.)

- Eric