public inbox for linux-btrfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Eric Biggers <ebiggers@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Richard Henderson <richard.henderson@linaro.org>,
	Matt Turner <mattst88@gmail.com>,
	Magnus Lindholm <linmag7@gmail.com>,
	Russell King <linux@armlinux.org.uk>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>,
	Huacai Chen <chenhuacai@kernel.org>,
	WANG Xuerui <kernel@xen0n.name>,
	Madhavan Srinivasan <maddy@linux.ibm.com>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Nicholas Piggin <npiggin@gmail.com>,
	"Christophe Leroy (CS GROUP)" <chleroy@kernel.org>,
	Paul Walmsley <pjw@kernel.org>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Alexandre Ghiti <alex@ghiti.fr>,
	Heiko Carstens <hca@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Alexander Gordeev <agordeev@linux.ibm.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	Sven Schnelle <svens@linux.ibm.com>,
	"David S. Miller" <davem@davemloft.net>,
	Andreas Larsson <andreas@gaisler.com>,
	Richard Weinberger <richard@nod.at>,
	Anton Ivanov <anton.ivanov@cambridgegreys.com>,
	Johannes Berg <johannes@sipsolutions.net>,
	Thomas Gleixner <tglx@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Dan Williams <dan.j.williams@intel.com>, Chris Mason <clm@fb.com>,
	David Sterba <dsterba@suse.com>, Arnd Bergmann <arnd@arndb.de>,
	Song Liu <song@kernel.org>, Yu Kuai <yukuai@fnnas.com>,
	Li Nan <linan122@huawei.com>, Theodore Ts'o <tytso@mit.edu>,
	"Jason A. Donenfeld" <Jason@zx2c4.com>,
	linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev,
	linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org,
	linux-s390@vger.kernel.org, sparclinux@vger.kernel.org,
	linux-um@lists.infradead.org, linux-crypto@vger.kernel.org,
	linux-btrfs@vger.kernel.org, linux-arch@vger.kernel.org,
	linux-raid@vger.kernel.org
Subject: Re: [PATCH 27/27] xor: add a kunit test case
Date: Wed, 11 Mar 2026 17:54:44 -0700	[thread overview]
Message-ID: <20260312005444.GA31906@quark> (raw)
In-Reply-To: <20260311070416.972667-28-hch@lst.de>

On Wed, Mar 11, 2026 at 08:03:59AM +0100, Christoph Hellwig wrote:
> diff --git a/lib/raid/Kconfig b/lib/raid/Kconfig
> index 4359971ebd04..97c123806466 100644
> --- a/lib/raid/Kconfig
> +++ b/lib/raid/Kconfig
> @@ -6,3 +6,14 @@ config XOR_BLOCKS
>  # selected by architectures that provide an optimized XOR implementation
>  config XOR_BLOCKS_ARCH
>  	bool
> +
> +config XOR_KUNIT_TEST
> +	tristate "KUnit tests for xor_gen" if !KUNIT_ALL_TESTS
> +	depends on KUNIT
> +	default KUNIT_ALL_TESTS
> +	select XOR_BLOCKS
> +	help
> +	  Unit tests for the XOR library functions.
> +
> +	  This is intended to help people writing architecture-specific
> +	  optimized versions.  If unsure, say N.

The convention for KUnit tests is actually to depend on the code they
test, not select it, so that it's easy to enable only the tests that are
relevant to a particular kernel build.  So instead of
"select XOR_BLOCKS", this should use "depends on KUNIT && XOR_BLOCKS".

(Yes, I got this wrong in the crypto and CRC tests.  I recently fixed it
in the crypto tests, and I have pending patches that fix the CRC test.)

There should also be a lib/raid/.kunitconfig file containing something
like:

    CONFIG_KUNIT=y
    CONFIG_BTRFS_FS=y
    CONFIG_XOR_KUNIT_TEST=y

(CONFIG_BTRFS_FS is there because it's one of the visible symbols that
select the hidden symbol XOR_BLOCKS.)

> +static u32 rand32(void)
> +{
> +	return prandom_u32_state(&rng);
> +}
> +
> +static u32 rand32_below(u32 ceil)
> +{
> +	return __limit_random_u32_below(ceil, prandom_u32_state(&rng));
> +}
> +
[...]
> +
> +/* Generate a random length that is a multiple of 512. */
> +static unsigned int generate_random_length(unsigned int max_length)
> +{
> +	return (rand32_below(max_length / 512) + 1) * 512;
> +}
> +
> +/* Generate a random alignment that is a multiple of 32. */
> +static unsigned int generate_random_alignment(unsigned int max_alignment)
> +{
> +	return (rand32_below((max_alignment + 1) / 32)) * 32;
> +}

As per my comment on patch 26, these should just use a simple mod
operations so that the new random.c helper function (which conflates
cryptographic and non-cryptographic random numbers) isn't needed.

Maybe:

        return (rand32() % (max_length + 1)) & ~511;

and

        return (rand32() % (max_alignment + 1)) & ~63;

> +/* Test that xor_gen gives the same result as a reference implementation. */
> +static void xor_test(struct kunit *test)
> +{
> +	void *aligned_buffers[XOR_KUNIT_MAX_BUFFERS];
> +	size_t i;
> +
> +	for (i = 0; i < XOR_KUNIT_NUM_TEST_ITERS; i++) {
> +		unsigned int nr_buffers =
> +			rand32_below(XOR_KUNIT_MAX_BUFFERS) + 1;
> +		unsigned int len = generate_random_length(XOR_KUNIT_MAX_BYTES);
> +		unsigned int max_alignment, align = 0;
> +		void *buffers;
> +
> +		if (rand32() % 8 == 0)
> +			/* Refresh the data occasionally. */
> +			xor_generate_random_data();
> +
> +		/*
> +		 * If we're not using the entire buffer size, inject randomize
> +		 * alignment into the buffer.
> +		 */
> +		max_alignment = XOR_KUNIT_MAX_BYTES - len;
> +		if (max_alignment) {
> +			int j;
> +
> +			align = generate_random_alignment(max_alignment);
> +			for (j = 0; j < nr_buffers; j++)
> +				aligned_buffers[j] = test_buffers[j] +
> +					generate_random_alignment(max_alignment);
> +			buffers = aligned_buffers;
> +		} else {
> +			buffers = test_buffers;
> +		}

This isn't taking advantage of the guard pages properly, since it rarely
selects buffers that go all the way up to the guard page.

If the guard page testing is going to be included (which is a good idea;
the crypto and CRC tests have it and they already caught a bug using
it), then the data should be placed at the very end of the buffers more
often, like what the CRC test does.

> +		/*
> +		 * Compute the XOR, and verify that it equals the XOR computed
> +		 * by a simple byte-at-a-time reference implementation.
> +		 */
> +		xor_ref(test_ref + align, buffers, nr_buffers, len);
> +		xor_gen(test_dest + align, buffers, nr_buffers, len);
> +		KUNIT_EXPECT_MEMEQ_MSG(test, test_ref, test_dest, len,
> +				"Wrong result with buffers=%u, len=%u, align=%s",
> +				nr_buffers, len, str_yes_no(max_alignment));

When align != 0, this does the comparison at the wrong offset.

The message also shows "align=no" if fully aligned buffers were used and
"align=yes" if they were not, which is a bit confusing.  Maybe replace
align=%s with randalign=%s.

> +MODULE_DESCRIPTION("Unit tests and benchmarks for the XOR library functions");

There's no benchmark included (yet), so that should be left out of the
description.

Also, I tried running this test on different architectures, and in
qemu-system-sparc64 it crashes with an alignment fault in xor_vis_5().

It goes away if the minimum tested alignment is increased from 32 bytes
to 64 bytes.  lib/raid/xor/sparc/xor-sparc64.S has a comment that
documents a requirement of "!(((long)dest | (long)sourceN) & (64 - 1))",
i.e. 64-byte alignment.

So, it seems the assumption that 32 bytes is the maximum required
alignment over all architectures is not correct.  The tested alignment
will need to be increased to 64 bytes, and the kerneldoc for xor_gen()
will need to be updated as well.

- Eric

  reply	other threads:[~2026-03-12  0:54 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-11  7:03 cleanup the RAID5 XOR library v2 Christoph Hellwig
2026-03-11  7:03 ` [PATCH 01/27] xor: assert that xor_blocks is not from preemptible user context Christoph Hellwig
2026-03-11  7:03 ` [PATCH 02/27] arm/xor: remove in_interrupt() handling Christoph Hellwig
2026-03-11  7:03 ` [PATCH 03/27] um/xor: cleanup xor.h Christoph Hellwig
2026-03-11  8:45   ` Richard Weinberger
2026-03-11  7:03 ` [PATCH 04/27] xor: move to lib/raid/ Christoph Hellwig
2026-03-11  7:03 ` [PATCH 05/27] xor: small cleanups Christoph Hellwig
2026-03-11  7:03 ` [PATCH 06/27] xor: cleanup registration and probing Christoph Hellwig
2026-03-11  7:03 ` [PATCH 07/27] xor: split xor.h Christoph Hellwig
2026-03-11  7:03 ` [PATCH 08/27] xor: remove macro abuse for XOR implementation registrations Christoph Hellwig
2026-03-11  7:03 ` [PATCH 09/27] xor: move generic implementations out of asm-generic/xor.h Christoph Hellwig
2026-03-11  7:03 ` [PATCH 10/27] alpha: move the XOR code to lib/raid/ Christoph Hellwig
2026-03-16 22:12   ` Magnus Lindholm
2026-03-11  7:03 ` [PATCH 11/27] arm: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 12/27] arm64: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 13/27] loongarch: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 14/27] powerpc: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 15/27] riscv: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 16/27] sparc: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 17/27] s390: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 18/27] x86: " Christoph Hellwig
2026-03-11  7:03 ` [PATCH 19/27] xor: avoid indirect calls for arm64-optimized ops Christoph Hellwig
2026-03-11  7:03 ` [PATCH 20/27] xor: make xor.ko self-contained in lib/raid/ Christoph Hellwig
2026-03-11  7:03 ` [PATCH 21/27] xor: add a better public API Christoph Hellwig
2026-03-11  7:03 ` [PATCH 22/27] async_xor: use xor_gen Christoph Hellwig
2026-03-11  7:03 ` [PATCH 23/27] btrfs: " Christoph Hellwig
2026-03-12  6:14   ` David Sterba
2026-03-11  7:03 ` [PATCH 24/27] xor: pass the entire operation to the low-level ops Christoph Hellwig
2026-03-11  7:03 ` [PATCH 25/27] xor: use static_call for xor_gen Christoph Hellwig
2026-03-11  7:03 ` [PATCH 26/27] random: factor out a __limit_random_u32_below helper Christoph Hellwig
2026-03-11 22:29   ` Eric Biggers
2026-03-12  8:38     ` David Laight
2026-03-12 13:46   ` Jason A. Donenfeld
2026-03-11  7:03 ` [PATCH 27/27] xor: add a kunit test case Christoph Hellwig
2026-03-12  0:54   ` Eric Biggers [this message]
2026-03-11 18:57 ` cleanup the RAID5 XOR library v2 Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260312005444.GA31906@quark \
    --to=ebiggers@kernel.org \
    --cc=Jason@zx2c4.com \
    --cc=agordeev@linux.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=alex@ghiti.fr \
    --cc=andreas@gaisler.com \
    --cc=anton.ivanov@cambridgegreys.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=arnd@arndb.de \
    --cc=borntraeger@linux.ibm.com \
    --cc=bp@alien8.de \
    --cc=catalin.marinas@arm.com \
    --cc=chenhuacai@kernel.org \
    --cc=chleroy@kernel.org \
    --cc=clm@fb.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=davem@davemloft.net \
    --cc=dsterba@suse.com \
    --cc=gor@linux.ibm.com \
    --cc=hca@linux.ibm.com \
    --cc=hch@lst.de \
    --cc=herbert@gondor.apana.org.au \
    --cc=hpa@zytor.com \
    --cc=johannes@sipsolutions.net \
    --cc=kernel@xen0n.name \
    --cc=linan122@huawei.com \
    --cc=linmag7@gmail.com \
    --cc=linux-alpha@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=linux-um@lists.infradead.org \
    --cc=linux@armlinux.org.uk \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=loongarch@lists.linux.dev \
    --cc=maddy@linux.ibm.com \
    --cc=mattst88@gmail.com \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=npiggin@gmail.com \
    --cc=palmer@dabbelt.com \
    --cc=pjw@kernel.org \
    --cc=richard.henderson@linaro.org \
    --cc=richard@nod.at \
    --cc=song@kernel.org \
    --cc=sparclinux@vger.kernel.org \
    --cc=svens@linux.ibm.com \
    --cc=tglx@kernel.org \
    --cc=tytso@mit.edu \
    --cc=will@kernel.org \
    --cc=x86@kernel.org \
    --cc=yukuai@fnnas.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox