* Re: [PATCH 01/19] dt-bindings: crypto: add Rambus CryptoManager Hub
From: Krzysztof Kozlowski @ 2026-06-26 10:55 UTC (permalink / raw)
To: Saravanakrishnan Krishnamoorthy
Cc: Albert Ou, Alex Ousherovitch, Conor Dooley, David S. Miller,
Herbert Xu, Jonathan Corbet, Krzysztof Kozlowski, Palmer Dabbelt,
Paul Walmsley, Rob Herring, Shuah Khan, Alexandre Ghiti,
devicetree, Joel Wittenauer, linux-api, linux-crypto, linux-doc,
linux-kernel, linux-kselftest, linux-riscv, Shuah Khan,
sipsupport, Thi Nguyen
In-Reply-To: <20260625173328.1140487-2-skrishnamoorthy@rambus.com>
On Thu, Jun 25, 2026 at 10:33:09AM -0700, Saravanakrishnan Krishnamoorthy wrote:
> From: Alex Ousherovitch <aousherovitch@rambus.com>
>
> Add device tree binding schema for the CRI CryptoManager Hub (CMH)
> hardware crypto accelerator. The binding covers the parent SoC-level
> node with register region, interrupt, DMA properties, and per-core
> child nodes identified by compatible string and unit address.
...
>
> ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
OK, we are done. I am removing your posting from Patchwork.
Best regards,
Krzysztof
^ permalink raw reply
* Re: [PATCH 2/8] um: Check for missing AVX and AVX-512 xstate bits
From: David Laight @ 2026-06-26 10:49 UTC (permalink / raw)
To: Anton Ivanov
Cc: Eric Biggers, x86, linux-um, linux-raid, linux-crypto,
linux-kernel, Christoph Hellwig, Andrew Morton
In-Reply-To: <6a20b442-b97f-4cae-9168-30201d5ef82c@cambridgegreys.com>
On Fri, 26 Jun 2026 09:21:49 +0100
Anton Ivanov <anton.ivanov@cambridgegreys.com> wrote:
> On 26/06/2026 08:41, David Laight wrote:
> > On Thu, 25 Jun 2026 21:37:25 -0700
> > Eric Biggers <ebiggers@kernel.org> wrote:
> >
> >> If the CPU declares AVX or AVX-512 support, verify that all the
> >> corresponding xstate bits are also set. If any are missing, warn and
> >> don't set the corresponding X86_FEATURE_* flags.
> >>
> >> This eliminates the perceived need for UML-supporting AVX and AVX-512
> >> optimized code in the kernel (that is, lib/raid/ currently) to start
> >> checking the xstate bits in addition to X86_FEATURE_AVX*.
> >>
> > ...
> >> static void __init parse_host_cpu_flags(char *line)
> >> {
> >> + u64 xcr0 = read_xcr0();
> >> int i;
> >> +
> >> for (i = 0; i < 32*NCAPINTS; i++) {
> >> if ((x86_cap_flags[i] != NULL) && strstr(line, x86_cap_flags[i]))
> >
> > 'line' comes from /proc/cpuinfo
> > Surely something would be terribly wrong if that included something the kernel
> > had disabled (or didn't support).
> >
> > David
> >
> >
> >> - set_cpu_cap(&boot_cpu_data, i);
> >> + validate_and_set_cpu_cap(i, xcr0);
> >> }
> >> }
> >>
> >> static void __init parse_cache_line(char *line)
> >> {
> >
> >
> >
> >
> Lots of other stuff will go wrong before that. Glibc, things compiled with LLVM, python, perl, etc.
>
> Half of the userland will go belly up, because AVX is used in string operations and hashing if it is available.
And glibc will check xcr0.
>
> UML is just another userland application from this perspective, so there is no reason for it to behave any different from the rest of the userland.
^ permalink raw reply
* Re: [PATCH] hwrng: omap: disable runtime PM on resume failure
From: Myeonghun Pak @ 2026-06-26 9:07 UTC (permalink / raw)
To: Deepak Saxena
Cc: Olivia Mackall, Herbert Xu, linux-crypto, linux-kernel, Ijae Kim
In-Reply-To: <20260626090302.42134-1-mhun512@gmail.com>
Please ignore this patch.
I noticed after sending it that this patch was not properly checked
against the existing omap-rng probe error-path cleanup discussion, and
it should not be reviewed as-is.
Sorry for the noise.
Thanks,
Myeonghun
2026년 6월 26일 (금) 오후 6:03, Myeonghun Pak <mhun512@gmail.com>님이 작성:
>
> omap_rng_probe() enables runtime PM before calling
> pm_runtime_resume_and_get(). If the resume call fails, the error path
> currently jumps to err_ioremap and returns without disabling runtime PM
> again.
>
> Add a runtime-PM-only error label and route the resume failure through it.
> The label is placed after the runtime PM usage-count unwind, so later
> probe failures keep using the existing pm_runtime_put_sync() path while
> this early failure only disables the runtime PM state that was already
> enabled.
>
> Fixes: 61dc0a446e5d ("hwrng: omap - Fix assumption that runtime_get_sync will always succeed")
> Co-developed-by: Ijae Kim <ae878000@gmail.com>
> Signed-off-by: Ijae Kim <ae878000@gmail.com>
> Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
> ---
> drivers/char/hw_random/omap-rng.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/char/hw_random/omap-rng.c b/drivers/char/hw_random/omap-rng.c
> index 5e8b50f15dc3..44b8bb4a24a8 100644
> --- a/drivers/char/hw_random/omap-rng.c
> +++ b/drivers/char/hw_random/omap-rng.c
> @@ -455,7 +455,7 @@ static int omap_rng_probe(struct platform_device *pdev)
> ret = pm_runtime_resume_and_get(&pdev->dev);
> if (ret < 0) {
> dev_err(&pdev->dev, "Failed to runtime_get device: %d\n", ret);
> - goto err_ioremap;
> + goto err_pm_disable;
> }
>
> priv->clk = devm_clk_get(&pdev->dev, NULL);
> @@ -499,6 +499,7 @@ err_register:
> err_register:
> priv->base = NULL;
> pm_runtime_put_sync(&pdev->dev);
> +err_pm_disable:
> pm_runtime_disable(&pdev->dev);
>
> clk_disable_unprepare(priv->clk_reg);
> --
> 2.47.1
^ permalink raw reply
* [PATCH] hwrng: omap: disable runtime PM on resume failure
From: Myeonghun Pak @ 2026-06-26 9:02 UTC (permalink / raw)
To: Deepak Saxena
Cc: Olivia Mackall, Herbert Xu, linux-crypto, linux-kernel,
Myeonghun Pak, Ijae Kim
omap_rng_probe() enables runtime PM before calling
pm_runtime_resume_and_get(). If the resume call fails, the error path
currently jumps to err_ioremap and returns without disabling runtime PM
again.
Add a runtime-PM-only error label and route the resume failure through it.
The label is placed after the runtime PM usage-count unwind, so later
probe failures keep using the existing pm_runtime_put_sync() path while
this early failure only disables the runtime PM state that was already
enabled.
Fixes: 61dc0a446e5d ("hwrng: omap - Fix assumption that runtime_get_sync will always succeed")
Co-developed-by: Ijae Kim <ae878000@gmail.com>
Signed-off-by: Ijae Kim <ae878000@gmail.com>
Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
---
drivers/char/hw_random/omap-rng.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/char/hw_random/omap-rng.c b/drivers/char/hw_random/omap-rng.c
index 5e8b50f15dc3..44b8bb4a24a8 100644
--- a/drivers/char/hw_random/omap-rng.c
+++ b/drivers/char/hw_random/omap-rng.c
@@ -455,7 +455,7 @@ static int omap_rng_probe(struct platform_device *pdev)
ret = pm_runtime_resume_and_get(&pdev->dev);
if (ret < 0) {
dev_err(&pdev->dev, "Failed to runtime_get device: %d\n", ret);
- goto err_ioremap;
+ goto err_pm_disable;
}
priv->clk = devm_clk_get(&pdev->dev, NULL);
@@ -499,6 +499,7 @@ err_register:
err_register:
priv->base = NULL;
pm_runtime_put_sync(&pdev->dev);
+err_pm_disable:
pm_runtime_disable(&pdev->dev);
clk_disable_unprepare(priv->clk_reg);
--
2.47.1
^ permalink raw reply related
* Re: [PATCH 2/8] um: Check for missing AVX and AVX-512 xstate bits
From: Anton Ivanov @ 2026-06-26 8:21 UTC (permalink / raw)
To: David Laight, Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton
In-Reply-To: <20260626084113.42eae31c@pumpkin>
On 26/06/2026 08:41, David Laight wrote:
> On Thu, 25 Jun 2026 21:37:25 -0700
> Eric Biggers <ebiggers@kernel.org> wrote:
>
>> If the CPU declares AVX or AVX-512 support, verify that all the
>> corresponding xstate bits are also set. If any are missing, warn and
>> don't set the corresponding X86_FEATURE_* flags.
>>
>> This eliminates the perceived need for UML-supporting AVX and AVX-512
>> optimized code in the kernel (that is, lib/raid/ currently) to start
>> checking the xstate bits in addition to X86_FEATURE_AVX*.
>>
> ...
>> static void __init parse_host_cpu_flags(char *line)
>> {
>> + u64 xcr0 = read_xcr0();
>> int i;
>> +
>> for (i = 0; i < 32*NCAPINTS; i++) {
>> if ((x86_cap_flags[i] != NULL) && strstr(line, x86_cap_flags[i]))
>
> 'line' comes from /proc/cpuinfo
> Surely something would be terribly wrong if that included something the kernel
> had disabled (or didn't support).
>
> David
>
>
>> - set_cpu_cap(&boot_cpu_data, i);
>> + validate_and_set_cpu_cap(i, xcr0);
>> }
>> }
>>
>> static void __init parse_cache_line(char *line)
>> {
>
>
>
>
Lots of other stuff will go wrong before that. Glibc, things compiled with LLVM, python, perl, etc.
Half of the userland will go belly up, because AVX is used in string operations and hashing if it is available.
UML is just another userland application from this perspective, so there is no reason for it to behave any different from the rest of the userland.
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: patchwork-bot+linux-riscv @ 2026-06-26 8:21 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-riscv, akpm, catalin.marinas, will, ardb, chenhuacai,
kernel, maddy, mpe, npiggin, chleroy, pjw, palmer, aou, alex, hca,
gor, agordeev, borntraeger, svens, tglx, mingo, bp, dave.hansen,
x86, hpa, herbert, dan.j.williams, clm, dsterba, arnd, song,
yukuai, linan122, linux-kernel, linux-arm-kernel, loongarch,
linuxppc-dev, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-2-hch@lst.de>
Hello:
This series was applied to riscv/linux.git (fixes)
by Andrew Morton <akpm@linux-foundation.org>:
On Tue, 12 May 2026 07:20:41 +0200 you wrote:
> While the RAID6 algorithm could in theory support 3 devices by just
> copying the data disk to the two parity disks, this version is not only
> useless because it is a suboptimal version of 3-way mirroring, but also
> broken with various crashes and incorrect parity generation in various
> architecture-optimized implementations. Disallow it similar to mdraid
> which requires at least 4 devices for RAID 6.
>
> [...]
Here is the summary with links:
- [01/19] btrfs: require at least 4 devices for RAID 6
(no matching commit)
- [02/19] raid6: turn the userspace test harness into a kunit test
(no matching commit)
- [03/19] raid6: remove __KERNEL__ ifdefs
https://git.kernel.org/riscv/c/3d6beb659ddf
- [04/19] raid6: move to lib/raid/
(no matching commit)
- [05/19] raid6: remove unused defines in pq.h
https://git.kernel.org/riscv/c/06d2a66fb7c0
- [06/19] raid6: remove raid6_get_zero_page
https://git.kernel.org/riscv/c/885d31423183
- [07/19] raid6: use named initializers for struct raid6_calls
https://git.kernel.org/riscv/c/7e91f76a9668
- [08/19] raid6: improve the public interface
(no matching commit)
- [09/19] raid6: hide internals
(no matching commit)
- [10/19] raid6: rework the init helpers
(no matching commit)
- [11/19] raid6: use static_call for gen_syndrom and xor_syndrom
https://git.kernel.org/riscv/c/10f4b8e2a164
- [12/19] raid6: use static_call for raid6_recov_2data and raid6_recov_datap
(no matching commit)
- [13/19] raid6: update top of file comments
https://git.kernel.org/riscv/c/30bf04bd13a5
- [14/19] raid6_kunit: use KUNIT_CASE_PARAM
https://git.kernel.org/riscv/c/2175395f76c3
- [15/19] raid6_kunit: dynamically allocate data buffers using vmalloc
https://git.kernel.org/riscv/c/d67c25712fe3
- [16/19] raid6_kunit: cleanup dataptr handling
https://git.kernel.org/riscv/c/562bcbfcb99b
- [17/19] raid6_kunit: randomize parameters and increase limits
(no matching commit)
- [18/19] raid6_kunit: randomize parameters and increase limits
(no matching commit)
- [19/19] raid6_kunit: randomize buffer alignment
https://git.kernel.org/riscv/c/8cf0a6c4bb9e
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* Re: [PATCH 01/18] raid6: turn the userspace test harness into a kunit test
From: patchwork-bot+linux-riscv @ 2026-06-26 8:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-riscv, akpm, catalin.marinas, will, ardb, chenhuacai,
kernel, maddy, mpe, npiggin, chleroy, pjw, palmer, aou, alex, hca,
gor, agordeev, borntraeger, svens, tglx, mingo, bp, dave.hansen,
x86, hpa, herbert, dan.j.williams, clm, dsterba, arnd, song,
yukuai, linan122, linux-kernel, linux-arm-kernel, loongarch,
linuxppc-dev, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260518051804.462141-2-hch@lst.de>
Hello:
This series was applied to riscv/linux.git (fixes)
by Andrew Morton <akpm@linux-foundation.org>:
On Mon, 18 May 2026 07:17:44 +0200 you wrote:
> Currently the raid6 code can be compiled as userspace code to run the
> test suite. Convert that to be a kunit case with minimal changes to
> avoid mutating global state so that we can drop this requirement.
>
> Note that this is not a good kunit test case yet and will need a lot more
> work, but that is deferred until the raid6 code is moved to it's new
> place, which is easier if the userspace makefile doesn't need adjustments
> for the new location first.
>
> [...]
Here is the summary with links:
- [01/18] raid6: turn the userspace test harness into a kunit test
https://git.kernel.org/riscv/c/c4697486fc23
- [02/18] raid6: remove __KERNEL__ ifdefs
https://git.kernel.org/riscv/c/3d6beb659ddf
- [03/18] raid6: move to lib/raid/
https://git.kernel.org/riscv/c/3626738bc714
- [04/18] raid6: remove unused defines in pq.h
https://git.kernel.org/riscv/c/06d2a66fb7c0
- [05/18] raid6: remove raid6_get_zero_page
https://git.kernel.org/riscv/c/885d31423183
- [06/18] raid6: use named initializers for struct raid6_calls
https://git.kernel.org/riscv/c/7e91f76a9668
- [07/18] raid6: improve the public interface
https://git.kernel.org/riscv/c/35472bc6f31b
- [08/18] raid6: warn when using less than four devices
https://git.kernel.org/riscv/c/2790045a62eb
- [09/18] raid6: hide internals
(no matching commit)
- [10/18] raid6: rework registration of optimized algorithms
(no matching commit)
- [11/18] raid6: use static_call for gen_syndrom and xor_syndrom
https://git.kernel.org/riscv/c/10f4b8e2a164
- [12/18] raid6: use static_call for raid6_recov_2data and raid6_recov_datap
https://git.kernel.org/riscv/c/dd83de0341da
- [13/18] raid6: update top of file comments
https://git.kernel.org/riscv/c/30bf04bd13a5
- [14/18] raid6_kunit: use KUNIT_CASE_PARAM
https://git.kernel.org/riscv/c/2175395f76c3
- [15/18] raid6_kunit: dynamically allocate data buffers using vmalloc
https://git.kernel.org/riscv/c/d67c25712fe3
- [16/18] raid6_kunit: cleanup dataptr handling
https://git.kernel.org/riscv/c/562bcbfcb99b
- [17/18] raid6_kunit: randomize parameters and increase limits
https://git.kernel.org/riscv/c/fa0c812c0aa5
- [18/18] raid6_kunit: randomize buffer alignment
https://git.kernel.org/riscv/c/8cf0a6c4bb9e
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* Re: [PATCH 2/8] um: Check for missing AVX and AVX-512 xstate bits
From: David Laight @ 2026-06-26 7:41 UTC (permalink / raw)
To: Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton
In-Reply-To: <20260626043731.319287-3-ebiggers@kernel.org>
On Thu, 25 Jun 2026 21:37:25 -0700
Eric Biggers <ebiggers@kernel.org> wrote:
> If the CPU declares AVX or AVX-512 support, verify that all the
> corresponding xstate bits are also set. If any are missing, warn and
> don't set the corresponding X86_FEATURE_* flags.
>
> This eliminates the perceived need for UML-supporting AVX and AVX-512
> optimized code in the kernel (that is, lib/raid/ currently) to start
> checking the xstate bits in addition to X86_FEATURE_AVX*.
>
...
> static void __init parse_host_cpu_flags(char *line)
> {
> + u64 xcr0 = read_xcr0();
> int i;
> +
> for (i = 0; i < 32*NCAPINTS; i++) {
> if ((x86_cap_flags[i] != NULL) && strstr(line, x86_cap_flags[i]))
'line' comes from /proc/cpuinfo
Surely something would be terribly wrong if that included something the kernel
had disabled (or didn't support).
David
> - set_cpu_cap(&boot_cpu_data, i);
> + validate_and_set_cpu_cap(i, xcr0);
> }
> }
>
> static void __init parse_cache_line(char *line)
> {
^ permalink raw reply
* Re: [PATCH 8/8] lib/raid/xor: x86: Add AVX-512 optimized xor_gen()
From: Christoph Hellwig @ 2026-06-26 5:47 UTC (permalink / raw)
To: Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Andrew Morton, David Laight
In-Reply-To: <20260626054731.GC9629@lst.de>
On Fri, Jun 26, 2026 at 07:47:31AM +0200, Christoph Hellwig wrote:
> On Thu, Jun 25, 2026 at 09:37:31PM -0700, Eric Biggers wrote:
> > + if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_AVX512F) &&
> > + !boot_cpu_has(X86_FEATURE_PREFER_YMM)) {
> > + /* AVX-512 will be the best; no need to try others. */
> > + /* !PREFER_YMM excludes CPUs with overly-eager downclocking. */
>
> Can you turn this into a single block comment using full sentences?
> Right now the two separate comments almost feel contradictory even
> if I get what you mean. While you're at it also through in a blurb
> why we dont bother with AVX-512 (number of register, no one in the right
> mind would bother running high performance code on modern cpus in 32-bit
> mode).
Otherwise looks good, btw:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply
* Re: [PATCH 8/8] lib/raid/xor: x86: Add AVX-512 optimized xor_gen()
From: Christoph Hellwig @ 2026-06-26 5:47 UTC (permalink / raw)
To: Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, David Laight
In-Reply-To: <20260626043731.319287-9-ebiggers@kernel.org>
On Thu, Jun 25, 2026 at 09:37:31PM -0700, Eric Biggers wrote:
> + if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_AVX512F) &&
> + !boot_cpu_has(X86_FEATURE_PREFER_YMM)) {
> + /* AVX-512 will be the best; no need to try others. */
> + /* !PREFER_YMM excludes CPUs with overly-eager downclocking. */
Can you turn this into a single block comment using full sentences?
Right now the two separate comments almost feel contradictory even
if I get what you mean. While you're at it also through in a blurb
why we dont bother with AVX-512 (number of register, no one in the right
mind would bother running high performance code on modern cpus in 32-bit
mode).
^ permalink raw reply
* Re: [PATCH 7/8] lib/raid/xor: x86: Remove redundant X86_FEATURE_OSXSAVE check
From: Christoph Hellwig @ 2026-06-26 5:40 UTC (permalink / raw)
To: Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton
In-Reply-To: <20260626043731.319287-8-ebiggers@kernel.org>
Looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply
* Re: [PATCH 1/8] x86/fpu: Check for missing AVX and AVX-512 xstate bits
From: Christoph Hellwig @ 2026-06-26 5:39 UTC (permalink / raw)
To: Eric Biggers
Cc: x86, linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton
In-Reply-To: <20260626043731.319287-2-ebiggers@kernel.org>
On Thu, Jun 25, 2026 at 09:37:24PM -0700, Eric Biggers wrote:
> If the CPU declares AVX or AVX-512 support, verify that the
> corresponding xstate bits are also set. If not, warn and clear them.
>
> This eliminates the perceived need for AVX and AVX-512 optimized code in
> the kernel to call cpu_has_xfeatures(). That has never been universally
> done, which strongly suggests that it has never really been needed in
> practice, but this should remove any remaining doubt.
I'll leave it to the x86-experts if the low-level details are right,
but the model behind this makes life so much easier, thanks a lot!
^ permalink raw reply
* [PATCH 8/8] lib/raid/xor: x86: Add AVX-512 optimized xor_gen()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers, David Laight
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
Add an implementation of xor_gen() using AVX-512.
It uses 512-bit vectors, i.e. ZMM registers. It also uses the
vpternlogq instruction to do three-input XORs when applicable.
It's enabled on x86_64 CPUs that have AVX512F && !PREFER_YMM. In
practice that means:
- AMD Zen 4 and later (client and server)
- Intel Sapphire Rapids and later (server)
- Intel Rocket Lake (client)
- Intel Nova Lake and later (client)
The !PREFER_YMM condition excludes the older AVX-512 implementations in
Intel Skylake Server and Intel Ice Lake. They could run this code, but
they're known to have overly-eager downclocking when ZMM registers are
used. This is the same policy that the crypto and CRC code uses.
Benchmark on AMD Ryzen 9 9950X (Zen 5):
src_cnt avx avx512 Improvement
======= ========== ========== ===========
1 56353 MB/s 75388 MB/s 33%
2 54274 MB/s 68409 MB/s 26%
3 44649 MB/s 64042 MB/s 43%
4 41315 MB/s 55002 MB/s 33%
Reviewed-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
lib/raid/xor/Makefile | 2 +-
lib/raid/xor/x86/xor-avx512.c | 121 ++++++++++++++++++++++++++++++++++
lib/raid/xor/x86/xor_arch.h | 23 ++++---
3 files changed, 135 insertions(+), 11 deletions(-)
create mode 100644 lib/raid/xor/x86/xor-avx512.c
diff --git a/lib/raid/xor/Makefile b/lib/raid/xor/Makefile
index e8ecec3c09f9..4a0e5c6d8298 100644
--- a/lib/raid/xor/Makefile
+++ b/lib/raid/xor/Makefile
@@ -27,11 +27,11 @@ xor-$(CONFIG_ALTIVEC) += powerpc/xor_vmx.o powerpc/xor_vmx_glue.o
xor-$(CONFIG_RISCV_ISA_V) += riscv/xor.o riscv/xor-glue.o
xor-$(CONFIG_SPARC32) += sparc/xor-sparc32.o
xor-$(CONFIG_SPARC64) += sparc/xor-sparc64.o sparc/xor-sparc64-glue.o
xor-$(CONFIG_S390) += s390/xor.o
xor-$(CONFIG_X86_32) += x86/xor-avx.o x86/xor-sse.o x86/xor-mmx.o
-xor-$(CONFIG_X86_64) += x86/xor-avx.o x86/xor-sse.o
+xor-$(CONFIG_X86_64) += x86/xor-avx.o x86/xor-sse.o x86/xor-avx512.o
obj-y += tests/
CFLAGS_xor-neon.o += $(CC_FLAGS_FPU) -I$(src)/$(SRCARCH)
CFLAGS_REMOVE_xor-neon.o += $(CC_FLAGS_NO_FPU)
diff --git a/lib/raid/xor/x86/xor-avx512.c b/lib/raid/xor/x86/xor-avx512.c
new file mode 100644
index 000000000000..17f57900d827
--- /dev/null
+++ b/lib/raid/xor/x86/xor-avx512.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AVX-512 optimized implementation of xor_gen()
+ *
+ * Copyright 2026 Google LLC
+ */
+
+#include <linux/types.h>
+#include <asm/fpu/api.h>
+#include "xor_impl.h"
+#include "xor_arch.h"
+
+/*
+ * Implementation notes:
+ *
+ * Unrolling by the number of buffers (2-5) is very important.
+ *
+ * Unrolling by length is less important, especially when using register-indexed
+ * addressing with negative indices from the end of the buffers. That approach
+ * results in just two loop control instructions being needed per iteration,
+ * regardless of the number of buffers.
+ *
+ * In fact, benchmarks showed that the 2 and 3 buffer cases require only 2x
+ * unrolling by length, while the 4 and 5 buffer cases don't require any
+ * unrolling by length. Benchmarks also showed that the register-indexed
+ * addressing isn't a bottleneck either; i.e., we can't do any better by
+ * incrementing the pointers as we go along, even with more unrolling.
+ */
+
+static void xor_avx512_2(long bytes, u8 *p1, const u8 *p2)
+{
+ long i = -bytes;
+
+ asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n"
+ "vmovdqa64 64(%1,%0), %%zmm1\n"
+ "vpxorq (%2,%0), %%zmm0, %%zmm0\n"
+ "vpxorq 64(%2,%0), %%zmm1, %%zmm1\n"
+ "vmovdqa64 %%zmm0, (%1,%0)\n"
+ "vmovdqa64 %%zmm1, 64(%1,%0)\n"
+ "add $128, %0\n"
+ "jnz 1b\n"
+ : "+&r"(i)
+ : "r"(p1 + bytes), "r"(p2 + bytes)
+ : "memory", "cc");
+}
+
+static void xor_avx512_3(long bytes, u8 *p1, const u8 *p2, const u8 *p3)
+{
+ long i = -bytes;
+
+ asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n"
+ "vmovdqa64 64(%1,%0), %%zmm1\n"
+ "vmovdqa64 (%2,%0), %%zmm2\n"
+ "vmovdqa64 64(%2,%0), %%zmm3\n"
+ "vpternlogq $0x96, (%3,%0), %%zmm2, %%zmm0\n"
+ "vpternlogq $0x96, 64(%3,%0), %%zmm3, %%zmm1\n"
+ "vmovdqa64 %%zmm0, (%1,%0)\n"
+ "vmovdqa64 %%zmm1, 64(%1,%0)\n"
+ "add $128, %0\n"
+ "jnz 1b\n"
+ : "+&r"(i)
+ : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes)
+ : "memory", "cc");
+}
+
+static void xor_avx512_4(long bytes, u8 *p1, const u8 *p2, const u8 *p3,
+ const u8 *p4)
+{
+ long i = -bytes;
+
+ asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n"
+ "vmovdqa64 (%2,%0), %%zmm1\n"
+ "vpxorq (%3,%0), %%zmm0, %%zmm0\n"
+ "vpternlogq $0x96, (%4,%0), %%zmm1, %%zmm0\n"
+ "vmovdqa64 %%zmm0, (%1,%0)\n"
+ "add $64, %0\n"
+ "jnz 1b\n"
+ : "+&r"(i)
+ : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes),
+ "r"(p4 + bytes)
+ : "memory", "cc");
+}
+
+static void xor_avx512_5(long bytes, u8 *p1, const u8 *p2, const u8 *p3,
+ const u8 *p4, const u8 *p5)
+{
+ long i = -bytes;
+
+ asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n"
+ "vmovdqa64 (%2,%0), %%zmm1\n"
+ "vpternlogq $0x96, (%3,%0), %%zmm1, %%zmm0\n"
+ "vmovdqa64 (%4,%0), %%zmm1\n"
+ "vpternlogq $0x96, (%5,%0), %%zmm1, %%zmm0\n"
+ "vmovdqa64 %%zmm0, (%1,%0)\n"
+ "add $64, %0\n"
+ "jnz 1b\n"
+ : "+&r"(i)
+ : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes),
+ "r"(p4 + bytes), "r"(p5 + bytes)
+ : "memory", "cc");
+}
+
+DO_XOR_BLOCKS(avx512_inner, xor_avx512_2, xor_avx512_3, xor_avx512_4,
+ xor_avx512_5);
+
+/*
+ * Preconditions: bytes is a nonzero multiple of 512, and all buffers are
+ * 64-byte aligned.
+ */
+static void xor_gen_avx512(void *dest, void **srcs, unsigned int src_cnt,
+ unsigned int bytes)
+{
+ kernel_fpu_begin();
+ xor_gen_avx512_inner(dest, srcs, src_cnt, bytes);
+ kernel_fpu_end();
+}
+
+struct xor_block_template xor_block_avx512 = {
+ .name = "avx512",
+ .xor_gen = xor_gen_avx512,
+};
diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h
index 991abe3f4bbd..d5e192b8793f 100644
--- a/lib/raid/xor/x86/xor_arch.h
+++ b/lib/raid/xor/x86/xor_arch.h
@@ -4,25 +4,28 @@
extern struct xor_block_template xor_block_pII_mmx;
extern struct xor_block_template xor_block_p5_mmx;
extern struct xor_block_template xor_block_sse;
extern struct xor_block_template xor_block_sse_pf64;
extern struct xor_block_template xor_block_avx;
+extern struct xor_block_template xor_block_avx512;
-/*
- * When SSE is available, use it as it can write around L2. We may also be able
- * to load into the L1 only depending on how the cpu deals with a load to a line
- * that is being prefetched.
- *
- * When AVX2 is available, force using it as it is better by all measures.
- *
- * 32-bit without MMX can fall back to the generic routines.
- */
static __always_inline void __init arch_xor_init(void)
{
- if (boot_cpu_has(X86_FEATURE_AVX)) {
+ if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_AVX512F) &&
+ !boot_cpu_has(X86_FEATURE_PREFER_YMM)) {
+ /* AVX-512 will be the best; no need to try others. */
+ /* !PREFER_YMM excludes CPUs with overly-eager downclocking. */
+ xor_force(&xor_block_avx512);
+ } else if (boot_cpu_has(X86_FEATURE_AVX)) {
+ /* AVX will be the best; no need to try others. */
xor_force(&xor_block_avx);
} else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) {
+ /*
+ * When SSE is available, use it as it can write around L2. We
+ * may also be able to load into the L1 only depending on how
+ * the cpu deals with a load to a line that is being prefetched.
+ */
xor_register(&xor_block_sse);
xor_register(&xor_block_sse_pf64);
} else if (boot_cpu_has(X86_FEATURE_MMX)) {
xor_register(&xor_block_pII_mmx);
xor_register(&xor_block_p5_mmx);
--
2.54.0
^ permalink raw reply related
* [PATCH 7/8] lib/raid/xor: x86: Remove redundant X86_FEATURE_OSXSAVE check
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
X86_FEATURE_AVX implies X86_FEATURE_OSXSAVE already.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
lib/raid/xor/x86/xor_arch.h | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h
index 99fe85a213c6..991abe3f4bbd 100644
--- a/lib/raid/xor/x86/xor_arch.h
+++ b/lib/raid/xor/x86/xor_arch.h
@@ -16,12 +16,11 @@ extern struct xor_block_template xor_block_avx;
*
* 32-bit without MMX can fall back to the generic routines.
*/
static __always_inline void __init arch_xor_init(void)
{
- if (boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ if (boot_cpu_has(X86_FEATURE_AVX)) {
xor_force(&xor_block_avx);
} else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) {
xor_register(&xor_block_sse);
xor_register(&xor_block_sse_pf64);
} else if (boot_cpu_has(X86_FEATURE_MMX)) {
--
2.54.0
^ permalink raw reply related
* [PATCH 6/8] x86/fpu: Remove cpu_has_xfeatures()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
The only remaining caller of cpu_has_xfeatures() is
print_xstate_features(), which uses it only to check and get the name of
a single feature.
Remove it and just inline the needed code into print_xstate_features().
This also makes the "unknown xstate feature" entry at index XFEATURE_MAX
of xfeature_names[] unnecessary, so remove that too.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/x86/include/asm/fpu/api.h | 9 -------
arch/x86/kernel/fpu/xstate.c | 44 +++-------------------------------
2 files changed, 3 insertions(+), 50 deletions(-)
diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h
index 90c63fe19c0f..cfed8b24d64f 100644
--- a/arch/x86/include/asm/fpu/api.h
+++ b/arch/x86/include/asm/fpu/api.h
@@ -97,19 +97,10 @@ static inline void fpregs_assert_state_consistent(void) { }
/*
* Load the task FPU state before returning to userspace.
*/
extern void switch_fpu_return(void);
-/*
- * Query the presence of one or more xfeatures. Works on any legacy CPU as well.
- *
- * If 'feature_name' is set then put a human-readable description of
- * the feature there as well - this can be used to print error (or success)
- * messages.
- */
-extern int cpu_has_xfeatures(u64 xfeatures_mask, const char **feature_name);
-
/* Trap handling */
extern int fpu__exception_code(struct fpu *fpu, int trap_nr);
extern void fpu_sync_fpstate(struct fpu *fpu);
extern void fpu_reset_from_exception_fixup(void);
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index 7f7e62e4ebc5..c6f0264f957c 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -64,12 +64,12 @@ static const char *xfeature_names[] =
"unknown xstate feature",
"unknown xstate feature",
"AMX Tile config",
"AMX Tile data",
"APX registers",
- "unknown xstate feature",
};
+static_assert(ARRAY_SIZE(xfeature_names) == XFEATURE_MAX);
static unsigned short xsave_cpuid_features[] __initdata = {
[XFEATURE_FP] = X86_FEATURE_FPU,
[XFEATURE_SSE] = X86_FEATURE_XMM,
[XFEATURE_YMM] = X86_FEATURE_AVX,
@@ -120,48 +120,10 @@ static inline unsigned int next_xfeature_order(unsigned int i, u64 mask)
i++)
#define XSTATE_FLAG_SUPERVISOR BIT(0)
#define XSTATE_FLAG_ALIGNED64 BIT(1)
-/*
- * Return whether the system supports a given xfeature.
- *
- * Also return the name of the (most advanced) feature that the caller requested:
- */
-int cpu_has_xfeatures(u64 xfeatures_needed, const char **feature_name)
-{
- u64 xfeatures_missing = xfeatures_needed & ~fpu_kernel_cfg.max_features;
-
- if (unlikely(feature_name)) {
- long xfeature_idx, max_idx;
- u64 xfeatures_print;
- /*
- * So we use FLS here to be able to print the most advanced
- * feature that was requested but is missing. So if a driver
- * asks about "XFEATURE_MASK_SSE | XFEATURE_MASK_YMM" we'll print the
- * missing AVX feature - this is the most informative message
- * to users:
- */
- if (xfeatures_missing)
- xfeatures_print = xfeatures_missing;
- else
- xfeatures_print = xfeatures_needed;
-
- xfeature_idx = fls64(xfeatures_print)-1;
- max_idx = ARRAY_SIZE(xfeature_names)-1;
- xfeature_idx = min(xfeature_idx, max_idx);
-
- *feature_name = xfeature_names[xfeature_idx];
- }
-
- if (xfeatures_missing)
- return 0;
-
- return 1;
-}
-EXPORT_SYMBOL_GPL(cpu_has_xfeatures);
-
static bool xfeature_is_aligned64(int xfeature_nr)
{
return xstate_flags[xfeature_nr] & XSTATE_FLAG_ALIGNED64;
}
@@ -300,13 +262,13 @@ static void __init print_xstate_features(void)
{
int i;
for (i = 0; i < XFEATURE_MAX; i++) {
u64 mask = BIT_ULL(i);
- const char *name;
+ const char *name = xfeature_names[i];
- if (cpu_has_xfeatures(mask, &name))
+ if (fpu_kernel_cfg.max_features & mask)
pr_info("x86/fpu: Supporting XSAVE feature 0x%03Lx: '%s'\n", mask, name);
}
}
/*
--
2.54.0
^ permalink raw reply related
* [PATCH 5/8] lib/crc: x86: Stop using cpu_has_xfeatures()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
Checking both boot_cpu_has() and cpu_has_xfeatures() has never really
been needed in practice, and it's never been universally done (e.g.,
lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML
now explicitly clear the AVX and AVX-512 flags if their xfeatures are
missing, which should remove any remaining doubts.
Thus, remove all the calls to cpu_has_xfeatures().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
lib/crc/x86/crc-pclmul-template.h | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/lib/crc/x86/crc-pclmul-template.h b/lib/crc/x86/crc-pclmul-template.h
index 02744831c6fa..893119bb7c07 100644
--- a/lib/crc/x86/crc-pclmul-template.h
+++ b/lib/crc/x86/crc-pclmul-template.h
@@ -25,20 +25,18 @@ crc_t prefix##_vpclmul_avx512(crc_t crc, const u8 *p, size_t len, \
DEFINE_STATIC_CALL(prefix##_pclmul, prefix##_pclmul_sse)
static inline bool have_vpclmul(void)
{
return boot_cpu_has(X86_FEATURE_VPCLMULQDQ) &&
- boot_cpu_has(X86_FEATURE_AVX2) &&
- cpu_has_xfeatures(XFEATURE_MASK_YMM, NULL);
+ boot_cpu_has(X86_FEATURE_AVX2);
}
static inline bool have_avx512(void)
{
return boot_cpu_has(X86_FEATURE_AVX512BW) &&
boot_cpu_has(X86_FEATURE_AVX512VL) &&
- !boot_cpu_has(X86_FEATURE_PREFER_YMM) &&
- cpu_has_xfeatures(XFEATURE_MASK_AVX512, NULL);
+ !boot_cpu_has(X86_FEATURE_PREFER_YMM);
}
/*
* Call a [V]PCLMULQDQ optimized CRC function if the data length is at least 16
* bytes, the CPU has PCLMULQDQ support, and the current context may use SIMD.
--
2.54.0
^ permalink raw reply related
* [PATCH 4/8] lib/crypto: x86: Stop using cpu_has_xfeatures()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
Checking both boot_cpu_has() and cpu_has_xfeatures() has never really
been needed in practice, and it's never been universally done (e.g.,
lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML
now explicitly clear the AVX and AVX-512 flags if their xfeatures are
missing, which should remove any remaining doubts.
Thus, remove all the calls to cpu_has_xfeatures().
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
lib/crypto/x86/blake2s.h | 4 +---
lib/crypto/x86/chacha.h | 3 +--
lib/crypto/x86/nh.h | 4 +---
lib/crypto/x86/poly1305.h | 7 ++-----
lib/crypto/x86/sha1.h | 4 +---
lib/crypto/x86/sha256.h | 4 +---
lib/crypto/x86/sha512.h | 3 +--
lib/crypto/x86/sm3.h | 3 +--
8 files changed, 9 insertions(+), 23 deletions(-)
diff --git a/lib/crypto/x86/blake2s.h b/lib/crypto/x86/blake2s.h
index f8eed6cb042e..0f7c51f055c8 100644
--- a/lib/crypto/x86/blake2s.h
+++ b/lib/crypto/x86/blake2s.h
@@ -53,10 +53,8 @@ static void blake2s_mod_init_arch(void)
static_branch_enable(&blake2s_use_ssse3);
if (boot_cpu_has(X86_FEATURE_AVX) &&
boot_cpu_has(X86_FEATURE_AVX2) &&
boot_cpu_has(X86_FEATURE_AVX512F) &&
- boot_cpu_has(X86_FEATURE_AVX512VL) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
- XFEATURE_MASK_AVX512, NULL))
+ boot_cpu_has(X86_FEATURE_AVX512VL))
static_branch_enable(&blake2s_use_avx512);
}
diff --git a/lib/crypto/x86/chacha.h b/lib/crypto/x86/chacha.h
index 10cf8f1c569d..c79562aac56b 100644
--- a/lib/crypto/x86/chacha.h
+++ b/lib/crypto/x86/chacha.h
@@ -163,12 +163,11 @@ static void chacha_mod_init_arch(void)
return;
static_branch_enable(&chacha_use_simd);
if (boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_AVX2) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) {
+ boot_cpu_has(X86_FEATURE_AVX2)) {
static_branch_enable(&chacha_use_avx2);
if (boot_cpu_has(X86_FEATURE_AVX512VL) &&
boot_cpu_has(X86_FEATURE_AVX512BW)) /* kmovq */
static_branch_enable(&chacha_use_avx512vl);
diff --git a/lib/crypto/x86/nh.h b/lib/crypto/x86/nh.h
index 83361c2e9783..342636dcb750 100644
--- a/lib/crypto/x86/nh.h
+++ b/lib/crypto/x86/nh.h
@@ -35,11 +35,9 @@ static bool nh_arch(const u32 *key, const u8 *message, size_t message_len,
#define nh_mod_init_arch nh_mod_init_arch
static void nh_mod_init_arch(void)
{
if (boot_cpu_has(X86_FEATURE_XMM2)) {
static_branch_enable(&have_sse2);
- if (boot_cpu_has(X86_FEATURE_AVX2) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- NULL))
+ if (boot_cpu_has(X86_FEATURE_AVX2))
static_branch_enable(&have_avx2);
}
}
diff --git a/lib/crypto/x86/poly1305.h b/lib/crypto/x86/poly1305.h
index ee92e3740a78..b061b9926fa5 100644
--- a/lib/crypto/x86/poly1305.h
+++ b/lib/crypto/x86/poly1305.h
@@ -141,18 +141,15 @@ static void poly1305_emit(const struct poly1305_state *ctx,
}
#define poly1305_mod_init_arch poly1305_mod_init_arch
static void poly1305_mod_init_arch(void)
{
- if (boot_cpu_has(X86_FEATURE_AVX) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
+ if (boot_cpu_has(X86_FEATURE_AVX))
static_branch_enable(&poly1305_use_avx);
- if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
+ if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2))
static_branch_enable(&poly1305_use_avx2);
if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) &&
boot_cpu_has(X86_FEATURE_AVX512F) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | XFEATURE_MASK_AVX512, NULL) &&
/* Skylake downclocks unacceptably much when using zmm, but later generations are fast. */
boot_cpu_data.x86_vfm != INTEL_SKYLAKE_X)
static_branch_enable(&poly1305_use_avx512);
}
diff --git a/lib/crypto/x86/sha1.h b/lib/crypto/x86/sha1.h
index c48a0131fd12..6aff433466e7 100644
--- a/lib/crypto/x86/sha1.h
+++ b/lib/crypto/x86/sha1.h
@@ -57,13 +57,11 @@ static void sha1_blocks(struct sha1_block_state *state,
#define sha1_mod_init_arch sha1_mod_init_arch
static void sha1_mod_init_arch(void)
{
if (boot_cpu_has(X86_FEATURE_SHA_NI)) {
static_call_update(sha1_blocks_x86, sha1_blocks_ni);
- } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- NULL) &&
- boot_cpu_has(X86_FEATURE_AVX)) {
+ } else if (boot_cpu_has(X86_FEATURE_AVX)) {
if (boot_cpu_has(X86_FEATURE_AVX2) &&
boot_cpu_has(X86_FEATURE_BMI1) &&
boot_cpu_has(X86_FEATURE_BMI2))
static_call_update(sha1_blocks_x86, sha1_blocks_avx2);
else
diff --git a/lib/crypto/x86/sha256.h b/lib/crypto/x86/sha256.h
index 0ee69d8e39fe..e98ffdaf4b14 100644
--- a/lib/crypto/x86/sha256.h
+++ b/lib/crypto/x86/sha256.h
@@ -102,13 +102,11 @@ static void sha256_mod_init_arch(void)
static_branch_enable(&have_sha_ni);
} else if (IS_ENABLED(CONFIG_CPU_SUP_ZHAOXIN) &&
boot_cpu_has(X86_FEATURE_PHE_EN) &&
boot_cpu_data.x86 >= 0x07) {
static_call_update(sha256_blocks_x86, sha256_blocks_phe);
- } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- NULL) &&
- boot_cpu_has(X86_FEATURE_AVX)) {
+ } else if (boot_cpu_has(X86_FEATURE_AVX)) {
if (boot_cpu_has(X86_FEATURE_AVX2) &&
boot_cpu_has(X86_FEATURE_BMI2))
static_call_update(sha256_blocks_x86,
sha256_blocks_avx2);
else
diff --git a/lib/crypto/x86/sha512.h b/lib/crypto/x86/sha512.h
index 0213c70cedd0..4e177b4606bd 100644
--- a/lib/crypto/x86/sha512.h
+++ b/lib/crypto/x86/sha512.h
@@ -35,12 +35,11 @@ static void sha512_blocks(struct sha512_block_state *state,
}
#define sha512_mod_init_arch sha512_mod_init_arch
static void sha512_mod_init_arch(void)
{
- if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL) &&
- boot_cpu_has(X86_FEATURE_AVX)) {
+ if (boot_cpu_has(X86_FEATURE_AVX)) {
if (boot_cpu_has(X86_FEATURE_AVX2) &&
boot_cpu_has(X86_FEATURE_BMI2))
static_call_update(sha512_blocks_x86,
sha512_blocks_avx2);
else
diff --git a/lib/crypto/x86/sm3.h b/lib/crypto/x86/sm3.h
index 3834780f2f6a..e06d4a22e4fa 100644
--- a/lib/crypto/x86/sm3.h
+++ b/lib/crypto/x86/sm3.h
@@ -31,9 +31,8 @@ static void sm3_blocks(struct sm3_block_state *state,
}
#define sm3_mod_init_arch sm3_mod_init_arch
static void sm3_mod_init_arch(void)
{
- if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2) &&
- cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
+ if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2))
static_call_update(sm3_blocks_x86, sm3_blocks_avx);
}
--
2.54.0
^ permalink raw reply related
* [PATCH 3/8] crypto: x86 - Stop using cpu_has_xfeatures()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
Checking both boot_cpu_has(X86_FEATURE_AVX*) and cpu_has_xfeatures() has
never really been needed in practice, and it's never been universally
done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both
x86 and UML now explicitly clear the AVX and AVX-512 flags if their
xfeatures are missing, which should remove any remaining doubts.
Thus, remove all the calls to cpu_has_xfeatures(), as well as the
related checks of boot_cpu_has(X86_FEATURE_OSXSAVE).
In a few cases there was no corresponding boot_cpu_has(X86_FEATURE_AVX*)
check, so add the missing ones.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/x86/crypto/aegis128-aesni-glue.c | 3 +--
arch/x86/crypto/aesni-intel_glue.c | 7 ++-----
arch/x86/crypto/aria_aesni_avx2_glue.c | 11 +----------
arch/x86/crypto/aria_aesni_avx_glue.c | 11 +----------
arch/x86/crypto/aria_gfni_avx512_glue.c | 11 +----------
arch/x86/crypto/camellia_aesni_avx2_glue.c | 11 +----------
arch/x86/crypto/camellia_aesni_avx_glue.c | 11 +----------
arch/x86/crypto/cast5_avx_glue.c | 7 ++-----
arch/x86/crypto/cast6_avx_glue.c | 7 ++-----
arch/x86/crypto/serpent_avx2_glue.c | 9 +--------
arch/x86/crypto/serpent_avx_glue.c | 7 ++-----
arch/x86/crypto/sm4_aesni_avx2_glue.c | 11 +----------
arch/x86/crypto/sm4_aesni_avx_glue.c | 11 +----------
arch/x86/crypto/twofish_avx_glue.c | 6 ++----
14 files changed, 19 insertions(+), 104 deletions(-)
diff --git a/arch/x86/crypto/aegis128-aesni-glue.c b/arch/x86/crypto/aegis128-aesni-glue.c
index f1adfba1a76e..09fc0b15b0e9 100644
--- a/arch/x86/crypto/aegis128-aesni-glue.c
+++ b/arch/x86/crypto/aegis128-aesni-glue.c
@@ -263,12 +263,11 @@ static struct aead_alg crypto_aegis128_aesni_alg = {
};
static int __init crypto_aegis128_aesni_module_init(void)
{
if (!boot_cpu_has(X86_FEATURE_XMM4_1) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL))
+ !boot_cpu_has(X86_FEATURE_AES))
return -ENODEV;
return crypto_register_aead(&crypto_aegis128_aesni_alg);
}
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index f522fff9231e..f6f899db7482 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -1546,12 +1546,11 @@ static int __init register_avx_algs(void)
* For simplicity, just always check for VAES and VPCLMULQDQ together.
*/
if (!boot_cpu_has(X86_FEATURE_AVX2) ||
!boot_cpu_has(X86_FEATURE_VAES) ||
!boot_cpu_has(X86_FEATURE_VPCLMULQDQ) ||
- !boot_cpu_has(X86_FEATURE_PCLMULQDQ) ||
- !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
+ !boot_cpu_has(X86_FEATURE_PCLMULQDQ))
return 0;
err = crypto_register_skciphers(skcipher_algs_vaes_avx2,
ARRAY_SIZE(skcipher_algs_vaes_avx2));
if (err)
return err;
@@ -1560,13 +1559,11 @@ static int __init register_avx_algs(void)
if (err)
return err;
if (!boot_cpu_has(X86_FEATURE_AVX512BW) ||
!boot_cpu_has(X86_FEATURE_AVX512VL) ||
- !boot_cpu_has(X86_FEATURE_BMI2) ||
- !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
- XFEATURE_MASK_AVX512, NULL))
+ !boot_cpu_has(X86_FEATURE_BMI2))
return 0;
if (boot_cpu_has(X86_FEATURE_PREFER_YMM)) {
int i;
diff --git a/arch/x86/crypto/aria_aesni_avx2_glue.c b/arch/x86/crypto/aria_aesni_avx2_glue.c
index 1487a49bfbac..371be2fb6469 100644
--- a/arch/x86/crypto/aria_aesni_avx2_glue.c
+++ b/arch/x86/crypto/aria_aesni_avx2_glue.c
@@ -193,26 +193,17 @@ static struct skcipher_alg aria_algs[] = {
}
};
static int __init aria_avx2_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
!boot_cpu_has(X86_FEATURE_AVX2) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX2 or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
if (boot_cpu_has(X86_FEATURE_GFNI)) {
aria_ops.aria_encrypt_16way = aria_aesni_avx_gfni_encrypt_16way;
aria_ops.aria_decrypt_16way = aria_aesni_avx_gfni_decrypt_16way;
aria_ops.aria_ctr_crypt_16way = aria_aesni_avx_gfni_ctr_crypt_16way;
aria_ops.aria_encrypt_32way = aria_aesni_avx2_gfni_encrypt_32way;
diff --git a/arch/x86/crypto/aria_aesni_avx_glue.c b/arch/x86/crypto/aria_aesni_avx_glue.c
index e4e3d78915a5..d23fc91c0ebd 100644
--- a/arch/x86/crypto/aria_aesni_avx_glue.c
+++ b/arch/x86/crypto/aria_aesni_avx_glue.c
@@ -180,25 +180,16 @@ static struct skcipher_alg aria_algs[] = {
}
};
static int __init aria_avx_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
if (boot_cpu_has(X86_FEATURE_GFNI)) {
aria_ops.aria_encrypt_16way = aria_aesni_avx_gfni_encrypt_16way;
aria_ops.aria_decrypt_16way = aria_aesni_avx_gfni_decrypt_16way;
aria_ops.aria_ctr_crypt_16way = aria_aesni_avx_gfni_ctr_crypt_16way;
} else {
diff --git a/arch/x86/crypto/aria_gfni_avx512_glue.c b/arch/x86/crypto/aria_gfni_avx512_glue.c
index 363cbf4399cc..e05bbeb22d4a 100644
--- a/arch/x86/crypto/aria_gfni_avx512_glue.c
+++ b/arch/x86/crypto/aria_gfni_avx512_glue.c
@@ -194,28 +194,19 @@ static struct skcipher_alg aria_algs[] = {
}
};
static int __init aria_avx512_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
!boot_cpu_has(X86_FEATURE_AVX2) ||
!boot_cpu_has(X86_FEATURE_AVX512F) ||
!boot_cpu_has(X86_FEATURE_AVX512VL) ||
- !boot_cpu_has(X86_FEATURE_GFNI) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_GFNI)) {
pr_info("AVX512/GFNI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
- XFEATURE_MASK_AVX512, &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
aria_ops.aria_encrypt_16way = aria_aesni_avx_gfni_encrypt_16way;
aria_ops.aria_decrypt_16way = aria_aesni_avx_gfni_decrypt_16way;
aria_ops.aria_ctr_crypt_16way = aria_aesni_avx_gfni_ctr_crypt_16way;
aria_ops.aria_encrypt_32way = aria_aesni_avx2_gfni_encrypt_32way;
aria_ops.aria_decrypt_32way = aria_aesni_avx2_gfni_decrypt_32way;
diff --git a/arch/x86/crypto/camellia_aesni_avx2_glue.c b/arch/x86/crypto/camellia_aesni_avx2_glue.c
index 2d2f4e16537c..073fa3bb8388 100644
--- a/arch/x86/crypto/camellia_aesni_avx2_glue.c
+++ b/arch/x86/crypto/camellia_aesni_avx2_glue.c
@@ -95,26 +95,17 @@ static struct skcipher_alg camellia_algs[] = {
},
};
static int __init camellia_aesni_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
!boot_cpu_has(X86_FEATURE_AVX2) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX2 or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
return crypto_register_skciphers(camellia_algs,
ARRAY_SIZE(camellia_algs));
}
static void __exit camellia_aesni_fini(void)
diff --git a/arch/x86/crypto/camellia_aesni_avx_glue.c b/arch/x86/crypto/camellia_aesni_avx_glue.c
index 5c321f255eb7..872e5e07220f 100644
--- a/arch/x86/crypto/camellia_aesni_avx_glue.c
+++ b/arch/x86/crypto/camellia_aesni_avx_glue.c
@@ -96,25 +96,16 @@ static struct skcipher_alg camellia_algs[] = {
}
};
static int __init camellia_aesni_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
return crypto_register_skciphers(camellia_algs,
ARRAY_SIZE(camellia_algs));
}
static void __exit camellia_aesni_fini(void)
diff --git a/arch/x86/crypto/cast5_avx_glue.c b/arch/x86/crypto/cast5_avx_glue.c
index 3aca04d43b34..5de35e863370 100644
--- a/arch/x86/crypto/cast5_avx_glue.c
+++ b/arch/x86/crypto/cast5_avx_glue.c
@@ -90,15 +90,12 @@ static struct skcipher_alg cast5_algs[] = {
}
};
static int __init cast5_init(void)
{
- const char *feature_name;
-
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
+ if (!boot_cpu_has(X86_FEATURE_AVX)) {
+ pr_info("AVX instructions are not detected.\n");
return -ENODEV;
}
return crypto_register_skciphers(cast5_algs,
ARRAY_SIZE(cast5_algs));
diff --git a/arch/x86/crypto/cast6_avx_glue.c b/arch/x86/crypto/cast6_avx_glue.c
index c4dd28c30303..3d7ea48007bc 100644
--- a/arch/x86/crypto/cast6_avx_glue.c
+++ b/arch/x86/crypto/cast6_avx_glue.c
@@ -90,15 +90,12 @@ static struct skcipher_alg cast6_algs[] = {
},
};
static int __init cast6_init(void)
{
- const char *feature_name;
-
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
+ if (!boot_cpu_has(X86_FEATURE_AVX)) {
+ pr_info("AVX instructions are not detected.\n");
return -ENODEV;
}
return crypto_register_skciphers(cast6_algs, ARRAY_SIZE(cast6_algs));
}
diff --git a/arch/x86/crypto/serpent_avx2_glue.c b/arch/x86/crypto/serpent_avx2_glue.c
index f5f2121b7956..72a9e2b306d6 100644
--- a/arch/x86/crypto/serpent_avx2_glue.c
+++ b/arch/x86/crypto/serpent_avx2_glue.c
@@ -91,21 +91,14 @@ static struct skcipher_alg serpent_algs[] = {
},
};
static int __init serpent_avx2_init(void)
{
- const char *feature_name;
-
- if (!boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ if (!boot_cpu_has(X86_FEATURE_AVX2)) {
pr_info("AVX2 instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
return crypto_register_skciphers(serpent_algs,
ARRAY_SIZE(serpent_algs));
}
diff --git a/arch/x86/crypto/serpent_avx_glue.c b/arch/x86/crypto/serpent_avx_glue.c
index 9c8b3a335d5c..42c4e1569674 100644
--- a/arch/x86/crypto/serpent_avx_glue.c
+++ b/arch/x86/crypto/serpent_avx_glue.c
@@ -98,15 +98,12 @@ static struct skcipher_alg serpent_algs[] = {
},
};
static int __init serpent_init(void)
{
- const char *feature_name;
-
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
+ if (!boot_cpu_has(X86_FEATURE_AVX)) {
+ pr_info("AVX instructions are not detected.\n");
return -ENODEV;
}
return crypto_register_skciphers(serpent_algs,
ARRAY_SIZE(serpent_algs));
diff --git a/arch/x86/crypto/sm4_aesni_avx2_glue.c b/arch/x86/crypto/sm4_aesni_avx2_glue.c
index fec0ab7a63dd..eef73894e777 100644
--- a/arch/x86/crypto/sm4_aesni_avx2_glue.c
+++ b/arch/x86/crypto/sm4_aesni_avx2_glue.c
@@ -96,26 +96,17 @@ static struct skcipher_alg sm4_aesni_avx2_skciphers[] = {
}
};
static int __init sm4_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
!boot_cpu_has(X86_FEATURE_AVX2) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX2 or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
return crypto_register_skciphers(sm4_aesni_avx2_skciphers,
ARRAY_SIZE(sm4_aesni_avx2_skciphers));
}
static void __exit sm4_exit(void)
diff --git a/arch/x86/crypto/sm4_aesni_avx_glue.c b/arch/x86/crypto/sm4_aesni_avx_glue.c
index 88caf418a06f..ed383da5ff46 100644
--- a/arch/x86/crypto/sm4_aesni_avx_glue.c
+++ b/arch/x86/crypto/sm4_aesni_avx_glue.c
@@ -312,25 +312,16 @@ static struct skcipher_alg sm4_aesni_avx_skciphers[] = {
}
};
static int __init sm4_init(void)
{
- const char *feature_name;
-
if (!boot_cpu_has(X86_FEATURE_AVX) ||
- !boot_cpu_has(X86_FEATURE_AES) ||
- !boot_cpu_has(X86_FEATURE_OSXSAVE)) {
+ !boot_cpu_has(X86_FEATURE_AES)) {
pr_info("AVX or AES-NI instructions are not detected.\n");
return -ENODEV;
}
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
return crypto_register_skciphers(sm4_aesni_avx_skciphers,
ARRAY_SIZE(sm4_aesni_avx_skciphers));
}
static void __exit sm4_exit(void)
diff --git a/arch/x86/crypto/twofish_avx_glue.c b/arch/x86/crypto/twofish_avx_glue.c
index 9e20db013750..985bc54a2340 100644
--- a/arch/x86/crypto/twofish_avx_glue.c
+++ b/arch/x86/crypto/twofish_avx_glue.c
@@ -100,14 +100,12 @@ static struct skcipher_alg twofish_algs[] = {
},
};
static int __init twofish_init(void)
{
- const char *feature_name;
-
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
+ if (!boot_cpu_has(X86_FEATURE_AVX)) {
+ pr_info("AVX instructions are not detected.\n");
return -ENODEV;
}
return crypto_register_skciphers(twofish_algs,
ARRAY_SIZE(twofish_algs));
--
2.54.0
^ permalink raw reply related
* [PATCH 1/8] x86/fpu: Check for missing AVX and AVX-512 xstate bits
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
If the CPU declares AVX or AVX-512 support, verify that the
corresponding xstate bits are also set. If not, warn and clear them.
This eliminates the perceived need for AVX and AVX-512 optimized code in
the kernel to call cpu_has_xfeatures(). That has never been universally
done, which strongly suggests that it has never really been needed in
practice, but this should remove any remaining doubt.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/x86/kernel/fpu/xstate.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index a7b6524a9dea..7f7e62e4ebc5 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -797,10 +797,27 @@ static u64 __init guest_default_mask(void)
* for KVM guests.
*/
return ~(u64)XFEATURE_MASK_USER_DYNAMIC;
}
+/* Clear any X86_FEATURE_* used by the kernel whose xfeatures are missing. */
+static void __init clear_cpu_caps_with_missing_xfeatures(u64 xfeatures)
+{
+ u64 mask;
+
+ mask = XFEATURE_MASK_FPSSE | XFEATURE_MASK_YMM;
+ if (boot_cpu_has(X86_FEATURE_AVX) && (xfeatures & mask) != mask) {
+ pr_err("x86/fpu: Disabling AVX support due to missing xstate features\n");
+ setup_clear_cpu_cap(X86_FEATURE_AVX);
+ }
+ mask = XFEATURE_MASK_FPSSE | XFEATURE_MASK_YMM | XFEATURE_MASK_AVX512;
+ if (boot_cpu_has(X86_FEATURE_AVX512F) && (xfeatures & mask) != mask) {
+ pr_err("x86/fpu: Disabling AVX-512 support due to missing xstate features\n");
+ setup_clear_cpu_cap(X86_FEATURE_AVX512F);
+ }
+}
+
/*
* Enable and initialize the xsave feature.
* Called once per system bootup.
*/
void __init fpu__init_system_xstate(unsigned int legacy_size)
@@ -853,10 +870,12 @@ void __init fpu__init_system_xstate(unsigned int legacy_size)
pr_err("x86/fpu: Both APX/MPX present in the CPU's xstate features: 0x%llx.\n",
fpu_kernel_cfg.max_features);
goto out_disable;
}
+ clear_cpu_caps_with_missing_xfeatures(fpu_kernel_cfg.max_features);
+
fpu_kernel_cfg.independent_features = fpu_kernel_cfg.max_features &
XFEATURE_MASK_INDEPENDENT;
/*
* Clear XSAVE features that are disabled in the normal CPUID.
--
2.54.0
^ permalink raw reply related
* [PATCH 2/8] um: Check for missing AVX and AVX-512 xstate bits
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
In-Reply-To: <20260626043731.319287-1-ebiggers@kernel.org>
If the CPU declares AVX or AVX-512 support, verify that all the
corresponding xstate bits are also set. If any are missing, warn and
don't set the corresponding X86_FEATURE_* flags.
This eliminates the perceived need for UML-supporting AVX and AVX-512
optimized code in the kernel (that is, lib/raid/ currently) to start
checking the xstate bits in addition to X86_FEATURE_AVX*.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/um/kernel/um_arch.c | 78 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 77 insertions(+), 1 deletion(-)
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 2141f5f1f5a2..aafbaef2ae82 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -262,16 +262,92 @@ EXPORT_SYMBOL(task_size);
unsigned long brk_start;
#define MIN_VMALLOC (32 * 1024 * 1024)
+static u64 __init read_xcr0(void)
+{
+ u32 a, b, c, d;
+
+ asm volatile("cpuid"
+ : "=a"(a), "=b"(b), "=c"(c), "=d"(d)
+ : "a"(0), "c"(0));
+ if (a >= 1) { /* max_leaf >= 1 */
+ asm volatile("cpuid"
+ : "=a"(a), "=b"(b), "=c"(c), "=d"(d)
+ : "a"(1), "c"(0));
+ if (c & (1 << 27)) { /* XSAVE enabled by OS */
+ asm volatile("xgetbv" : "=d"(d), "=a"(a) : "c"(0));
+ return ((u64)d << 32) | a;
+ }
+ }
+ return 0;
+}
+
+static void __init validate_and_set_cpu_cap(int cap, u64 xcr0)
+{
+ /*
+ * Check for missing xstate features right away, so that there's no
+ * perceived need for all optimized code in the kernel to do so.
+ */
+ switch (cap) {
+ case X86_FEATURE_AVX:
+ case X86_FEATURE_AVX2:
+ case X86_FEATURE_AVX_VNNI:
+ case X86_FEATURE_FMA:
+ case X86_FEATURE_VAES:
+ case X86_FEATURE_VPCLMULQDQ:
+ if ((xcr0 & 0x7) != 0x7) {
+ static bool warned;
+
+ if (!warned) {
+ os_warn("Disabling AVX support due to missing xstate features\n");
+ warned = true;
+ }
+ return;
+ }
+ break;
+ case X86_FEATURE_AVX512F:
+ case X86_FEATURE_AVX512BW:
+ case X86_FEATURE_AVX512CD:
+ case X86_FEATURE_AVX512DQ:
+ case X86_FEATURE_AVX512ER:
+ case X86_FEATURE_AVX512IFMA:
+ case X86_FEATURE_AVX512PF:
+ case X86_FEATURE_AVX512VBMI:
+ case X86_FEATURE_AVX512VL:
+ case X86_FEATURE_AVX512_4FMAPS:
+ case X86_FEATURE_AVX512_4VNNIW:
+ case X86_FEATURE_AVX512_BF16:
+ case X86_FEATURE_AVX512_BITALG:
+ case X86_FEATURE_AVX512_FP16:
+ case X86_FEATURE_AVX512_VBMI2:
+ case X86_FEATURE_AVX512_VNNI:
+ case X86_FEATURE_AVX512_VP2INTERSECT:
+ case X86_FEATURE_AVX512_VPOPCNTDQ:
+ if ((xcr0 & 0xe7) != 0xe7) {
+ static bool warned;
+
+ if (!warned) {
+ os_warn("Disabling AVX-512 support due to missing xstate features\n");
+ warned = true;
+ }
+ return;
+ }
+ break;
+ }
+ set_cpu_cap(&boot_cpu_data, cap);
+}
+
static void __init parse_host_cpu_flags(char *line)
{
+ u64 xcr0 = read_xcr0();
int i;
+
for (i = 0; i < 32*NCAPINTS; i++) {
if ((x86_cap_flags[i] != NULL) && strstr(line, x86_cap_flags[i]))
- set_cpu_cap(&boot_cpu_data, i);
+ validate_and_set_cpu_cap(i, xcr0);
}
}
static void __init parse_cache_line(char *line)
{
--
2.54.0
^ permalink raw reply related
* [PATCH 0/8] x86: Remove cpu_has_xfeatures() and add AVX-512 xor_gen()
From: Eric Biggers @ 2026-06-26 4:37 UTC (permalink / raw)
To: x86
Cc: linux-um, linux-raid, linux-crypto, linux-kernel,
Christoph Hellwig, Andrew Morton, Eric Biggers
My patch "lib/raid/xor: x86: Add AVX-512 optimized xor_gen()"
(https://lore.kernel.org/r/20260615190338.26581-1-ebiggers@kernel.org/)
still seems to be blocked on a Sashiko comment about cpu_has_xfeatures()
not being called.
However, the x86-optimized RAID library code supports UML, and currently
UML doesn't implement cpu_has_xfeatures(). That's perhaps why the
existing AVX-512 optimized RAID6 code doesn't check it either.
In fact, it seems to have been getting by fine without it, which
suggests that it's not truly needed.
But to eliminate any doubts, I've had a go at fully resolving the
situation by making both native x86 and UML explicitly clear any
X86_FEATURE_* flags at boot time whose xfeatures are missing.
Then, cpu_has_xfeatures() is entirely removed from the kernel.
The last patch adds the AVX-512 optimized xor_gen(). I do still think
it would be fine to proceed with it without the rest. But if there are
any doubts, we can take this more comprehensive cleanup route.
Eric Biggers (8):
x86/fpu: Check for missing AVX and AVX-512 xstate bits
um: Check for missing AVX and AVX-512 xstate bits
crypto: x86 - Stop using cpu_has_xfeatures()
lib/crypto: x86: Stop using cpu_has_xfeatures()
lib/crc: x86: Stop using cpu_has_xfeatures()
x86/fpu: Remove cpu_has_xfeatures()
lib/raid/xor: x86: Remove redundant X86_FEATURE_OSXSAVE check
lib/raid/xor: x86: Add AVX-512 optimized xor_gen()
arch/um/kernel/um_arch.c | 78 ++++++++++++-
arch/x86/crypto/aegis128-aesni-glue.c | 3 +-
arch/x86/crypto/aesni-intel_glue.c | 7 +-
arch/x86/crypto/aria_aesni_avx2_glue.c | 11 +-
arch/x86/crypto/aria_aesni_avx_glue.c | 11 +-
arch/x86/crypto/aria_gfni_avx512_glue.c | 11 +-
arch/x86/crypto/camellia_aesni_avx2_glue.c | 11 +-
arch/x86/crypto/camellia_aesni_avx_glue.c | 11 +-
arch/x86/crypto/cast5_avx_glue.c | 7 +-
arch/x86/crypto/cast6_avx_glue.c | 7 +-
arch/x86/crypto/serpent_avx2_glue.c | 9 +-
arch/x86/crypto/serpent_avx_glue.c | 7 +-
arch/x86/crypto/sm4_aesni_avx2_glue.c | 11 +-
arch/x86/crypto/sm4_aesni_avx_glue.c | 11 +-
arch/x86/crypto/twofish_avx_glue.c | 6 +-
arch/x86/include/asm/fpu/api.h | 9 --
arch/x86/kernel/fpu/xstate.c | 63 ++++-------
lib/crc/x86/crc-pclmul-template.h | 6 +-
lib/crypto/x86/blake2s.h | 4 +-
lib/crypto/x86/chacha.h | 3 +-
lib/crypto/x86/nh.h | 4 +-
lib/crypto/x86/poly1305.h | 7 +-
lib/crypto/x86/sha1.h | 4 +-
lib/crypto/x86/sha256.h | 4 +-
lib/crypto/x86/sha512.h | 3 +-
lib/crypto/x86/sm3.h | 3 +-
lib/raid/xor/Makefile | 2 +-
lib/raid/xor/x86/xor-avx512.c | 121 +++++++++++++++++++++
lib/raid/xor/x86/xor_arch.h | 24 ++--
29 files changed, 264 insertions(+), 194 deletions(-)
create mode 100644 lib/raid/xor/x86/xor-avx512.c
base-commit: 4edcdefd4083ae04b1a5656f4be6cd83ae919ef4
--
2.54.0
^ permalink raw reply
* Re: [PATCH v9 3/6] x86/sev: Disable CPU hotplug while SNP is active
From: K Prateek Nayak @ 2026-06-26 4:01 UTC (permalink / raw)
To: Kalra, Ashish, Borislav Petkov
Cc: tglx, mingo, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb, pbonzini, aik,
Michael.Roth, Tycho.Andersen, Nathan.Fontenot, ackerleytng,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <b9777de5-a6fa-418c-92d2-89c095e91837@amd.com>
Hello Ashish,
On 6/26/2026 8:08 AM, Kalra, Ashish wrote:
>> Looking at snp_prepare(), we have an early-bailout for
>>
>> rdmsrq(MSR_AMD64_SYSCFG, val);
>> if (val & MSR_AMD64_SYSCFG_SNP_EN)
>> return;
>>
>> Does executing SHUTDOWN command lead to the firmware clearing SNP_EN in
>> SYSCFG on all CPUS?
>
> Yes, in case of X86_SNP_SHUTDOWN (available if firmware supports X86SnpShutdown feature)
> SNP is disabled on all cores by clearing SYSCFG[SNPEn] bit.
>
> If X86_SNP_SHUTDOWN is set to 1, the firmware clears the SYSCFG[SNPEn] bit in each core.
>
> But, in case of legacy SNP shutdown, SNP_EN bit is not cleared and so SNP remains enabled.
Ah! That was the bit I was missing. Thanks a ton for clarifying.
>
>>
>> If SNP_EN remains set (and Linux can't clear it since it is
>> "Write-1-only" bit), then a subsequent snp_prepare() will skip setting
>> SYSCFG if it sees SNP_EN on local CPU.
>>
>> It can so happen that we enable hotlpug at shutdown, CPUs come online
>> without setting SNP_EN in SYSCFG, subsequent snp_prepare() runs on a CPU
>> where SNP_EN is still set and skips configuring it for the CPUs that
>> don't have it set, and we'll be in a pickle still.
>>
>> The comment above that bailout saying "this can happen in case of kexec
>> boot" makes me believe that SNP_EN remains set until a full system
>> reset.
>>
>> The only safe way to do this is to ensure all possible CPUs are online
>> during snp_prepare() and do snp_enable() regardless of whether local CPU
>> has SNP_EN or not.
>>
>> Am I missing something?
>>
>
> The piece that makes the early bailout safe is the disable this patch adds:
> hotplug is disabled while SNP is active, so the online set can't change under an
> active SNP. snp_prepare() already requires online == present, so at a successful
> init every present CPU gets SNP_EN,
How is this enforced? AFAICT, on_each_cpu(snp_enable) will only covers
the online CPUs and there could be CPUs that have been offlined before
that right?
> and because hotplug is then disabled none
> can leave or rejoin without it. So whenever the bailout is hit with SNP active,
> every online CPU already has SNP_EN:
>
> - kexec: SNP_EN is already set on all CPUs by the previous kernel.
There is a catch here: you can have offline CPUs during the previous boot
(say you have maxcpus=8 in your cmdline), and then you kexec with a different
kernel / cmdline that brings online a bunch more CPUs.
SNP_EN will only be set for a subset of then with the legacy SNP_INIT and
if snp_prepare() runs on those legacy CPUs, you still skip setting it for
the ones that don't have SNP_EN set.
Is that case covered somehow or is it a non-issue?
> - re-init while SNP is still active (e.g. after a legacy SNP_SHUTDOWN that
> leaves SNP_EN set): hotplug was disabled the whole time, so the online set is
> unchanged and all of them still have SNP_EN.
>
> The only way a CPU can be online without SNP_EN is when SNP is not active --
> i.e. after an SNP_INIT failure, where this patch re-enables hotplug. That is
> deliberately the same as the behavior before this support existed (hotplug was
> never disabled then), and it is benign: SNP_EN only gates RMP checks, the RMP
> itself is initialized by SNP_INIT, so on a failed init the RMP is all-zeroes --
> every entry is in the default HV-owned state, no page is assigned, no check ever blocks
> and snp_initialized stays false, so no SNP guest can be created.
> Nothing is enforced and nothing is protected.
>
> So I've kept snp_prepare()'s existing bailout / snp_enable() behavior unchanged;
> what this patch adds is disabling hotplug while SNP is active, which is what
> actually closes the window (a CPU coming online without SNP_EN while SNP is
> live). That window -- and the SNP_EN-stays-set-on-failure situation -- already
> exist in today's code, this patch constrains the dangerous (active) case and
> otherwise matches current behavior.
Ack! Just that one small bit up above bothers me but other than that,
doing it in snp_prepare() should be good.
This is all new to me so thanks a ton for answering my queries.
>
> (On the v9 placement specifically: I'm moving the disable into snp_prepare()
> ahead of SNP_EN in the next version; in v9 it sits after SNP_INIT, which leaves
> the window you originally pointed out.)
--
Thanks and Regards,
Prateek
^ permalink raw reply
* Re: [PATCH v9 3/6] x86/sev: Disable CPU hotplug while SNP is active
From: Kalra, Ashish @ 2026-06-26 2:38 UTC (permalink / raw)
To: K Prateek Nayak, Borislav Petkov
Cc: tglx, mingo, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb, pbonzini, aik,
Michael.Roth, Tycho.Andersen, Nathan.Fontenot, ackerleytng,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <898e378a-cf7c-4310-b439-e28ec0a71338@amd.com>
On 6/25/2026 5:16 PM, K Prateek Nayak wrote:
> Hello Ashish,
>
> On 6/26/2026 1:12 AM, Kalra, Ashish wrote:
>> Hello Boris,
>>
>> On 6/25/2026 10:02 AM, Borislav Petkov wrote:
>>> On Wed, Jun 24, 2026 at 09:56:49PM +0000, Ashish Kalra wrote:
>>>> +/* Set while SNP has CPU hotplug disabled (kernel-lifetime; survives ccp reload). */
>>>> +static bool snp_cpu_hotplug_disabled;
>>>
>>> Do you really need this?
>>>
>>
>> Yes.
>>
>> cpu_hotplug_disable()/cpu_hotplug_enable() are refcounted (cpu_hotplug_disabled++/--,
>> with a WARN on underflow), so they have to be balanced. This flag collapses them to
>> exactly one outstanding disable per SNP-active window, because the disable and enable
>> sites are not reached a symmetric number of times:
>>> - On firmware without SNP_X86_SHUTDOWN_SUPPORTED, __sev_snp_shutdown_locked() does not
>> call snp_shutdown() (it's gated on data.x86_snp_shutdown), so SNP stays enabled in
>> hardware — SNP_EN stays set and hotplug stays disabled — while sev->snp_initialized is
>> cleared. Re-init after that is routine, the SNP ioctls self-bracket init and shutdown
>> (e.g. SNP_COMMIT, SNP_SET_CONFIG, SNP_VLEK_LOAD):
>>
>> if (!sev->snp_initialized)
>> snp_move_to_init_state(...); /* -> __sev_snp_init_locked -> snp_prepare() */
>> ... SNP_CMD ...
>> if (shutdown_required)
>> __sev_snp_shutdown_locked(...);
>> - So whenever SNP isn't already initialized (psp_init_on_probe off, or after a prior
>> legacy shutdown), every such ioctl does init -> command -> legacy shutdown. Each init
>> reaches snp_prepare() with SNP_EN already set, and the disable now sits at the top of
>> snp_prepare(), so it fires on every cycle. Without this flag that keeps bumping
>> cpu_hotplug_disabled while the legacy shutdown never re-enables — hotplug ends up stuck
>> disabled. This flag makes all but the first disable a no-op.
>>
>> - Also, importantly, kvm-amd module reload on legacy firmware is the same pattern:
>> unload leaves SNP_EN set, reload re-inits.)
>
> Looking at snp_prepare(), we have an early-bailout for
>
> rdmsrq(MSR_AMD64_SYSCFG, val);
> if (val & MSR_AMD64_SYSCFG_SNP_EN)
> return;
>
> Does executing SHUTDOWN command lead to the firmware clearing SNP_EN in
> SYSCFG on all CPUS?
Yes, in case of X86_SNP_SHUTDOWN (available if firmware supports X86SnpShutdown feature)
SNP is disabled on all cores by clearing SYSCFG[SNPEn] bit.
If X86_SNP_SHUTDOWN is set to 1, the firmware clears the SYSCFG[SNPEn] bit in each core.
But, in case of legacy SNP shutdown, SNP_EN bit is not cleared and so SNP remains enabled.
>
> If SNP_EN remains set (and Linux can't clear it since it is
> "Write-1-only" bit), then a subsequent snp_prepare() will skip setting
> SYSCFG if it sees SNP_EN on local CPU.
>
> It can so happen that we enable hotlpug at shutdown, CPUs come online
> without setting SNP_EN in SYSCFG, subsequent snp_prepare() runs on a CPU
> where SNP_EN is still set and skips configuring it for the CPUs that
> don't have it set, and we'll be in a pickle still.
>
> The comment above that bailout saying "this can happen in case of kexec
> boot" makes me believe that SNP_EN remains set until a full system
> reset.
>
> The only safe way to do this is to ensure all possible CPUs are online
> during snp_prepare() and do snp_enable() regardless of whether local CPU
> has SNP_EN or not.
>
> Am I missing something?
>
The piece that makes the early bailout safe is the disable this patch adds:
hotplug is disabled while SNP is active, so the online set can't change under an
active SNP. snp_prepare() already requires online == present, so at a successful
init every present CPU gets SNP_EN, and because hotplug is then disabled none
can leave or rejoin without it. So whenever the bailout is hit with SNP active,
every online CPU already has SNP_EN:
- kexec: SNP_EN is already set on all CPUs by the previous kernel.
- re-init while SNP is still active (e.g. after a legacy SNP_SHUTDOWN that
leaves SNP_EN set): hotplug was disabled the whole time, so the online set is
unchanged and all of them still have SNP_EN.
The only way a CPU can be online without SNP_EN is when SNP is not active --
i.e. after an SNP_INIT failure, where this patch re-enables hotplug. That is
deliberately the same as the behavior before this support existed (hotplug was
never disabled then), and it is benign: SNP_EN only gates RMP checks, the RMP
itself is initialized by SNP_INIT, so on a failed init the RMP is all-zeroes --
every entry is in the default HV-owned state, no page is assigned, no check ever blocks
and snp_initialized stays false, so no SNP guest can be created.
Nothing is enforced and nothing is protected.
So I've kept snp_prepare()'s existing bailout / snp_enable() behavior unchanged;
what this patch adds is disabling hotplug while SNP is active, which is what
actually closes the window (a CPU coming online without SNP_EN while SNP is
live). That window -- and the SNP_EN-stays-set-on-failure situation -- already
exist in today's code, this patch constrains the dangerous (active) case and
otherwise matches current behavior.
(On the v9 placement specifically: I'm moving the disable into snp_prepare()
ahead of SNP_EN in the next version; in v9 it sits after SNP_INIT, which leaves
the window you originally pointed out.)
Thanks,
Ashish
>>
>> - On the enable side it avoids an unbalanced cpu_hotplug_enable() when the teardown/failure
>> paths run without an outstanding disable (e.g. shutdown of a never-fully-initialized SNP).
>>
>> So it's not redundant with cpu_hotplug_disabled — it tracks whether the outstanding disable
>> belongs to this SNP-active window in this kernel, which keeps the single disable/enable
>> balanced across the asymmetric legacy-vs-full SNP teardown paths and re-init.
^ permalink raw reply
* Re: [PATCH v9 3/6] x86/sev: Disable CPU hotplug while SNP is active
From: K Prateek Nayak @ 2026-06-25 22:16 UTC (permalink / raw)
To: Kalra, Ashish, Borislav Petkov
Cc: tglx, mingo, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb, pbonzini, aik,
Michael.Roth, Tycho.Andersen, Nathan.Fontenot, ackerleytng,
jackyli, pgonda, rientjes, jacobhxu, xin, pawan.kumar.gupta,
babu.moger, dyoung, nikunj, john.allen, darwi, linux-kernel,
linux-crypto, kvm, linux-coco
In-Reply-To: <7c64d96f-f932-4db9-8119-b9e40d5b7fd9@amd.com>
Hello Ashish,
On 6/26/2026 1:12 AM, Kalra, Ashish wrote:
> Hello Boris,
>
> On 6/25/2026 10:02 AM, Borislav Petkov wrote:
>> On Wed, Jun 24, 2026 at 09:56:49PM +0000, Ashish Kalra wrote:
>>> +/* Set while SNP has CPU hotplug disabled (kernel-lifetime; survives ccp reload). */
>>> +static bool snp_cpu_hotplug_disabled;
>>
>> Do you really need this?
>>
>
> Yes.
>
> cpu_hotplug_disable()/cpu_hotplug_enable() are refcounted (cpu_hotplug_disabled++/--,
> with a WARN on underflow), so they have to be balanced. This flag collapses them to
> exactly one outstanding disable per SNP-active window, because the disable and enable
> sites are not reached a symmetric number of times:
> > - On firmware without SNP_X86_SHUTDOWN_SUPPORTED, __sev_snp_shutdown_locked() does not
> call snp_shutdown() (it's gated on data.x86_snp_shutdown), so SNP stays enabled in
> hardware — SNP_EN stays set and hotplug stays disabled — while sev->snp_initialized is
> cleared. Re-init after that is routine, the SNP ioctls self-bracket init and shutdown
> (e.g. SNP_COMMIT, SNP_SET_CONFIG, SNP_VLEK_LOAD):
>
> if (!sev->snp_initialized)
> snp_move_to_init_state(...); /* -> __sev_snp_init_locked -> snp_prepare() */
> ... SNP_CMD ...
> if (shutdown_required)
> __sev_snp_shutdown_locked(...);
> - So whenever SNP isn't already initialized (psp_init_on_probe off, or after a prior
> legacy shutdown), every such ioctl does init -> command -> legacy shutdown. Each init
> reaches snp_prepare() with SNP_EN already set, and the disable now sits at the top of
> snp_prepare(), so it fires on every cycle. Without this flag that keeps bumping
> cpu_hotplug_disabled while the legacy shutdown never re-enables — hotplug ends up stuck
> disabled. This flag makes all but the first disable a no-op.
>
> - Also, importantly, kvm-amd module reload on legacy firmware is the same pattern:
> unload leaves SNP_EN set, reload re-inits.)
Looking at snp_prepare(), we have an early-bailout for
rdmsrq(MSR_AMD64_SYSCFG, val);
if (val & MSR_AMD64_SYSCFG_SNP_EN)
return;
Does executing SHUTDOWN command lead to the firmware clearing SNP_EN in
SYSCFG on all CPUS?
If SNP_EN remains set (and Linux can't clear it since it is
"Write-1-only" bit), then a subsequent snp_prepare() will skip setting
SYSCFG if it sees SNP_EN on local CPU.
It can so happen that we enable hotlpug at shutdown, CPUs come online
without setting SNP_EN in SYSCFG, subsequent snp_prepare() runs on a CPU
where SNP_EN is still set and skips configuring it for the CPUs that
don't have it set, and we'll be in a pickle still.
The comment above that bailout saying "this can happen in case of kexec
boot" makes me believe that SNP_EN remains set until a full system
reset.
The only safe way to do this is to ensure all possible CPUs are online
during snp_prepare() and do snp_enable() regardless of whether local CPU
has SNP_EN or not.
Am I missing something?
>
> - On the enable side it avoids an unbalanced cpu_hotplug_enable() when the teardown/failure
> paths run without an outstanding disable (e.g. shutdown of a never-fully-initialized SNP).
>
> So it's not redundant with cpu_hotplug_disabled — it tracks whether the outstanding disable
> belongs to this SNP-active window in this kernel, which keeps the single disable/enable
> balanced across the asymmetric legacy-vs-full SNP teardown paths and re-init.
--
Thanks and Regards,
Prateek
^ permalink raw reply
* Re: [PATCH 17/19] Documentation: ioctl: add CMH ioctl documentation and register 'J'
From: Krishnamoorthy, Saravanakrishnan @ 2026-06-25 21:21 UTC (permalink / raw)
To: Randy Dunlap, Albert Ou, Ousherovitch, Alex, Conor Dooley,
David S. Miller, Herbert Xu, Jonathan Corbet, Krzysztof Kozlowski,
Palmer Dabbelt, Paul Walmsley, Rob Herring, Shuah Khan
Cc: Alexandre Ghiti, devicetree@vger.kernel.org, Wittenauer, Joel,
linux-api@vger.kernel.org, linux-crypto@vger.kernel.org,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org,
Shuah Khan, SIPSupport, Nguyen, Thi
In-Reply-To: <60a7850d-1527-4517-aae8-cd5cbebcb9b5@infradead.org>
Thanks for catching this, Randy. You're right. cmh_mgmt.rst isn't pulled into any toctree. We will fix it in v2 by adding cmh_mgmt to the toctree in Documentation/userspace-api/ioctl/index.rst (alongside cdrom/hdio).
Krishnan
________________________________________
From: Randy Dunlap <rdunlap@infradead.org>
Sent: Thursday, June 25, 2026 11:29 AM
To: Krishnamoorthy, Saravanakrishnan; Albert Ou; Ousherovitch, Alex; Conor Dooley; David S. Miller; Herbert Xu; Jonathan Corbet; Krzysztof Kozlowski; Palmer Dabbelt; Paul Walmsley; Rob Herring; Shuah Khan
Cc: Alexandre Ghiti; devicetree@vger.kernel.org; Wittenauer, Joel; linux-api@vger.kernel.org; linux-crypto@vger.kernel.org; linux-doc@vger.kernel.org; linux-kernel@vger.kernel.org; linux-kselftest@vger.kernel.org; linux-riscv@lists.infradead.org; Shuah Khan; SIPSupport; Nguyen, Thi
Subject: Re: [PATCH 17/19] Documentation: ioctl: add CMH ioctl documentation and register 'J'
[Some people who received this message don't often get email from rdunlap@infradead.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Caution: < External Email >
On 6/25/26 10:33 AM, Saravanakrishnan Krishnamoorthy wrote:
> From: Alex Ousherovitch <aousherovitch@rambus.com>
>
> Add Documentation/userspace-api/ioctl/cmh_mgmt.rst documenting the
> ioctl commands on the /dev/cmh_mgmt misc device for the CRI
> CryptoManager Hub (CMH) hardware crypto accelerator driver. Covers
> key management, KIC key derivation, PKE (RSA, ECDSA, ECDH, EdDSA),
> PQC (ML-KEM, ML-DSA, SLH-DSA), SM2, EAC, and DRBG.
>
> Register ioctl magic number 'J' (0x4A) in ioctl-number.rst. The
> driver uses ioctls 0x01-0x40.
>
> Co-developed-by: Saravanakrishnan Krishnamoorthy <skrishnamoorthy@rambus.com>
> Signed-off-by: Saravanakrishnan Krishnamoorthy <skrishnamoorthy@rambus.com>
> Signed-off-by: Alex Ousherovitch <aousherovitch@rambus.com>
> Reviewed-by: Joel Wittenauer <Joel.Wittenauer@cryptography.com>
> Reviewed-by: Thi Nguyen <thin@rambus.com>
> ---
> .../userspace-api/ioctl/cmh_mgmt.rst | 941 ++++++++++++++++++
> .../userspace-api/ioctl/ioctl-number.rst | 1 +
> 2 files changed, 942 insertions(+)
> create mode 100644 Documentation/userspace-api/ioctl/cmh_mgmt.rst
>
One docs build warning:
Documentation/userspace-api/ioctl/cmh_mgmt.rst: WARNING: document isn't included in any toctree [toc.not_included]
--
~Randy
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox