Linux cryptographic layer development
 help / color / mirror / Atom feed
From: Eric Biggers <ebiggers@kernel.org>
To: Shreesh Adiga <16567adigashreesh@gmail.com>
Cc: Ard Biesheuvel <ardb@kernel.org>,
	Demian Shulhan <demyansh@gmail.com>,
	linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] lib/crc: arm64: Add NEON PMULL+EOR3 crc64_nvme implementation
Date: Tue, 17 Nov 2026 00:01:19 +0000	[thread overview]
Message-ID: <20261117000119.GH229817@google.com> (raw)
In-Reply-To: <20261114172955.1598929-1-16567adigashreesh@gmail.com>

On Sat, Nov 14, 2026 at 10:59:51PM +0530, Shreesh Adiga wrote:
> Add a 4x-folding implementation for CPUs with FEAT_SHA3 which benefits
> newer ARMv9 CPUs like Cortex-X925.
> Usage of EOR3 from SHA3 extension improves performance by 20% compared
> to the regular PMULL without EOR3 on Cortex-X925 measure in userspace
> application via libbenchmark for 512 byte input size.
> Measured about 3x speedup for 512 byte input compared with the existing
> NEON single fold implementation on Cortex-X925 in userspace.

Was crc_kunit run?  It already has a benchmark in it.

The commit message should also clarify that this benefits ARMv8.2 and
later processors with the sha3 extension, not just ARMv9.  (E.g., Apple
M1 is an example of an ARMv8.4 processor that benefits.)

> +#ifdef CONFIG_ARM64
> +#define SHA3_TARGET __attribute__((target("arch=armv8.2-a+crypto+sha3")))

Doesn't this break the build with gcc 8.1?  The kernel supports gcc 8.1
and later, but support for the sha3 target and intrinsics were added in
gcc 9.1.  Either this code will need to check the gcc version, or inline
assembly will need to be used for eor3 like lib/raid/xor/ does.

> +	v0 = fold_single_round(v0, v2, fold_consts_2x);
> +	v1 = fold_single_round(v1, v3, fold_consts_2x);
> +	v0 = fold_single_round(v0, v1, fold_consts);
> +
> +	while (len >= 16) {
> +		uint64x2_t t0 = vreinterpretq_u64_u8(vld1q_u8(p));
> +
> +		v0 = fold_single_round(v0, t0, fold_consts);
> +		p += 16;
> +		len -= 16;
> +	}
> +
> +	/* Multiply the 128-bit value by x^64 and reduce it back to 128 bits. */
> +	v0 = vextq_u64(v0, zero, 1) ^ pmull64_hi_lo(fold_consts, v0);
> +
> +	/* Final Barrett reduction */
> +	uint64x2_t bconsts = vld1q_u64(bconsts_val);
> +	uint64x2_t final = pmull64(bconsts, v0);
> +
> +	v0 = veor3q_u64(v0, vextq_u64(zero, final, 1),
> +			pmull64_hi_lo(bconsts, final));
> +
> +	return vgetq_lane_u64(v0, 1);

Note that this is missing several optimizations compared to the x86
version (lib/crc/x86/crc-pclmul-template.S), including using
fold_consts_2x for a tail of 'len & 32', efficiently handling lengths
that aren't a multiple of 16 bytes, and reducing to the final CRC in two
carryless multiplication instructions rather than three.

This is also just one of several CRC variants.  This would have to be
done all over again for each variant if they are not going to be
generated by a template like x86 and riscv do.

So we can do this for now, but it isn't a very scalable solution.

- Eric

  reply	other threads:[~2026-11-17  0:01 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-14 17:29 [PATCH v2] lib/crc: arm64: Add NEON PMULL+EOR3 crc64_nvme implementation Shreesh Adiga
2026-11-17  0:01 ` Eric Biggers [this message]
2026-11-17  0:45 ` Demian Shulhan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261117000119.GH229817@google.com \
    --to=ebiggers@kernel.org \
    --cc=16567adigashreesh@gmail.com \
    --cc=ardb@kernel.org \
    --cc=demyansh@gmail.com \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox