Re: xor_gen() md-raid6 performance improvement

[Miroslav Cimerman <mc@raidlab.org>]

Hi Anton,

thanks for sharing the numbers. I recently submitted a raid6 patch that
makes read-modify-writes faster, at least on the Intel machines I have.
If you have time, could you please test it on your Zen 5 box?

  https://lore.kernel.org/all/20261109030805.25709-1-mc@raidlab.org/

It changes xor_syndrome(), which md uses for small read-modify-writes.
Your 2 MiB sequential test does not use it, as full-stripe writes
compute P and Q with gen_syndrome(). Random 4 KiB writes do use it,
i.e. --rw=randwrite --bs=4k:

  echo 2 > /sys/block/md111/md/rmw_level
  fio --name=test --rw=randwrite --bs=4k --filename=/dev/md111 \
      --direct=1 --numjobs=1 --iodepth=64 --exitall --group_reporting \
      --ioengine=io_uring --size=100g --runtime=60 --time_based=1

rmw_level=2 is needed on a 4+2 array: a read-modify-write and
a reconstruct-write both need 3 reads there, and by default md picks
reconstruct-write on a tie (1 is the default).

One job at QD64 should be enough. md's single raid6 thread should be
the limit. If the md111_raid6 thread is not near 100% in "top -H", try
--numjobs=4 --iodepth=32.

If md111 has the new lockless bitmap and was created without
--assume-clean, md uses reconstruct-writes in regions not written since
the array was created. Adding --size=100g to fio keeps the writes in
the start of the array, which your sequential tests have probably
already written.

Also please make sure that no resync is in progress.

My numbers were measured without a write-intent bitmap, so the
difference may be smaller with one.

Please make sure that both runs, with and without the patch, use
avx512x4:

  dmesg | grep 'raid6: using algorithm' | tail -1

The kernel picks the implementation by gen_syndrome() speed when
raid6_pq is loaded, and the patch only changes the x4 versions (x1 and
x2 already use normal stores). If another one is picked, reloading
raid6_pq with md111 stopped runs the benchmark again, or I can send a
two-line test-only change that forces avx512x4.

Either way, sharing your numbers would help, and a reply with Tested-by
to the patch if it works for you would be very appreciated.

Thanks,
Miroslav