* [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
@ 2025-06-07 20:04 Eric Biggers
2025-06-07 20:04 ` [PATCH v2 01/12] lib/crc: move files into lib/crc/ Eric Biggers
` (17 more replies)
0 siblings, 18 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
This series is also available at:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
This series improves how lib/crc supports arch-optimized code. First,
instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
functions (e.g. crc32c_base()) will now be part of a single module for
each CRC type, allowing better inlining and dead code elimination. The
second change is made possible by the first.
As an example, consider CONFIG_CRC32=m on x86. We'll now have just
crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
were already coupled together and always both got loaded together via
direct symbol dependency, so the separation provided no benefit.
Note: later I'd like to apply the same design to lib/crypto/ too, where
often the API functions are out-of-line so this will work even better.
In those cases, for each algorithm we currently have 3 modules all
coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
sha256-x86.ko. We should have just one, inline things properly, and
rely on the compiler's dead code elimination to decide the inclusion of
the generic code instead of manually setting it via kconfig.
Having arch-specific code outside arch/ was somewhat controversial when
Zinc proposed it back in 2018. But I don't think the concerns are
warranted. It's better from a technical perspective, as it enables the
improvements mentioned above. This model is already successfully used
in other places in the kernel such as lib/raid6/. The community of each
architecture still remains free to work on the code, even if it's not in
arch/. At the time there was also a desire to put the library code in
the same files as the old-school crypto API, but that was a mistake; now
that the library is separate, that's no longer a constraint either.
Changed in v2:
- Fixed build warning on architectures without any optimized CRC code
- Fixed build warning in sparc/crc32.h by removing pr_fmt
- Moved fallback definitions of crc32*_arch back into arch files
- Remove ARCH_HAS_CRC* symbols at end of series instead of beginning,
so that they're not removed until they're no longer being selected
- Slightly improved some commit messages
- Rebased onto other pending lib/crc changes
Eric Biggers (12):
lib/crc: move files into lib/crc/
lib/crc: prepare for arch-optimized code in subdirs of lib/crc/
lib/crc/arm: migrate arm-optimized CRC code into lib/crc/
lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/
lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/
lib/crc/mips: migrate mips-optimized CRC code into lib/crc/
lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/
lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/
lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/
lib/crc/x86: migrate x86-optimized CRC code into lib/crc/
lib/crc: remove ARCH_HAS_* kconfig symbols
Documentation/core-api/kernel-api.rst | 14 +--
MAINTAINERS | 4 +-
arch/arm/Kconfig | 2 -
arch/arm/lib/Makefile | 6 -
arch/arm64/Kconfig | 2 -
arch/arm64/lib/Makefile | 6 -
arch/loongarch/Kconfig | 1 -
arch/loongarch/lib/Makefile | 2 -
arch/mips/Kconfig | 1 -
arch/mips/lib/Makefile | 2 -
arch/powerpc/Kconfig | 2 -
arch/powerpc/lib/Makefile | 6 -
arch/riscv/Kconfig | 3 -
arch/riscv/lib/Makefile | 6 -
arch/s390/Kconfig | 1 -
arch/s390/lib/Makefile | 3 -
arch/sparc/Kconfig | 1 -
arch/sparc/lib/Makefile | 2 -
arch/x86/Kconfig | 3 -
arch/x86/lib/Makefile | 10 --
include/linux/crc-t10dif.h | 10 +-
include/linux/crc32.h | 30 +----
include/linux/crc64.h | 22 +---
lib/Kconfig | 87 +------------
lib/Kconfig.debug | 21 ----
lib/Makefile | 32 +----
lib/crc/.gitignore | 5 +
lib/crc/Kconfig | 118 ++++++++++++++++++
lib/crc/Makefile | 63 ++++++++++
.../arm/lib => lib/crc/arm}/crc-t10dif-core.S | 0
.../crc-t10dif.c => lib/crc/arm/crc-t10dif.h | 23 +---
{arch/arm/lib => lib/crc/arm}/crc32-core.S | 0
arch/arm/lib/crc32.c => lib/crc/arm/crc32.h | 38 ++----
.../lib => lib/crc/arm64}/crc-t10dif-core.S | 0
.../crc/arm64/crc-t10dif.h | 22 +---
.../arm64/lib => lib/crc/arm64}/crc32-core.S | 0
.../lib/crc32.c => lib/crc/arm64/crc32.h | 19 +--
lib/{ => crc}/crc-ccitt.c | 3 -
lib/{ => crc}/crc-itu-t.c | 0
lib/{crc-t10dif.c => crc/crc-t10dif-main.c} | 37 ++++--
lib/{ => crc}/crc16.c | 0
lib/{crc32.c => crc/crc32-main.c} | 69 ++++++++--
lib/{ => crc}/crc4.c | 0
lib/{crc64.c => crc/crc64-main.c} | 47 +++++--
lib/{ => crc}/crc7.c | 0
lib/{ => crc}/crc8.c | 0
lib/{ => crc}/gen_crc32table.c | 4 +-
lib/{ => crc}/gen_crc64table.c | 11 +-
.../crc/loongarch/crc32.h | 32 +----
.../lib/crc32-mips.c => lib/crc/mips/crc32.h | 33 +----
.../crc/powerpc/crc-t10dif.h | 20 +--
.../crc/powerpc}/crc-vpmsum-template.S | 0
.../lib/crc32.c => lib/crc/powerpc/crc32.h | 38 ++----
.../crc/powerpc}/crc32c-vpmsum_asm.S | 0
.../crc/powerpc}/crct10dif-vpmsum_asm.S | 0
.../lib => lib/crc/riscv}/crc-clmul-consts.h | 0
.../crc/riscv}/crc-clmul-template.h | 0
{arch/riscv/lib => lib/crc/riscv}/crc-clmul.h | 0
.../crc/riscv/crc-t10dif.h | 8 +-
{arch/riscv/lib => lib/crc/riscv}/crc16_msb.c | 0
.../lib/crc32.c => lib/crc/riscv/crc32.h | 17 +--
{arch/riscv/lib => lib/crc/riscv}/crc32_lsb.c | 0
{arch/riscv/lib => lib/crc/riscv}/crc32_msb.c | 0
.../lib/crc64.c => lib/crc/riscv/crc64.h | 11 +-
{arch/riscv/lib => lib/crc/riscv}/crc64_lsb.c | 0
{arch/riscv/lib => lib/crc/riscv}/crc64_msb.c | 0
{arch/s390/lib => lib/crc/s390}/crc32-vx.h | 0
arch/s390/lib/crc32.c => lib/crc/s390/crc32.h | 16 +--
{arch/s390/lib => lib/crc/s390}/crc32be-vx.c | 0
{arch/s390/lib => lib/crc/s390}/crc32le-vx.c | 0
.../lib/crc32.c => lib/crc/sparc/crc32.h | 42 ++-----
.../sparc/lib => lib/crc/sparc}/crc32c_asm.S | 0
lib/crc/tests/Makefile | 2 +
lib/{ => crc}/tests/crc_kunit.c | 0
.../lib => lib/crc/x86}/crc-pclmul-consts.h | 0
.../lib => lib/crc/x86}/crc-pclmul-template.S | 0
.../lib => lib/crc/x86}/crc-pclmul-template.h | 0
.../crc-t10dif.c => lib/crc/x86/crc-t10dif.h | 18 +--
.../lib => lib/crc/x86}/crc16-msb-pclmul.S | 0
{arch/x86/lib => lib/crc/x86}/crc32-pclmul.S | 0
arch/x86/lib/crc32.c => lib/crc/x86/crc32.h | 30 +----
{arch/x86/lib => lib/crc/x86}/crc32c-3way.S | 0
{arch/x86/lib => lib/crc/x86}/crc64-pclmul.S | 0
arch/x86/lib/crc64.c => lib/crc/x86/crc64.h | 21 +---
lib/tests/Makefile | 1 -
85 files changed, 406 insertions(+), 621 deletions(-)
create mode 100644 lib/crc/.gitignore
create mode 100644 lib/crc/Kconfig
create mode 100644 lib/crc/Makefile
rename {arch/arm/lib => lib/crc/arm}/crc-t10dif-core.S (100%)
rename arch/arm/lib/crc-t10dif.c => lib/crc/arm/crc-t10dif.h (70%)
rename {arch/arm/lib => lib/crc/arm}/crc32-core.S (100%)
rename arch/arm/lib/crc32.c => lib/crc/arm/crc32.h (69%)
rename {arch/arm64/lib => lib/crc/arm64}/crc-t10dif-core.S (100%)
rename arch/arm64/lib/crc-t10dif.c => lib/crc/arm64/crc-t10dif.h (70%)
rename {arch/arm64/lib => lib/crc/arm64}/crc32-core.S (100%)
rename arch/arm64/lib/crc32.c => lib/crc/arm64/crc32.h (81%)
rename lib/{ => crc}/crc-ccitt.c (98%)
rename lib/{ => crc}/crc-itu-t.c (100%)
rename lib/{crc-t10dif.c => crc/crc-t10dif-main.c} (78%)
rename lib/{ => crc}/crc16.c (100%)
rename lib/{crc32.c => crc/crc32-main.c} (58%)
rename lib/{ => crc}/crc4.c (100%)
rename lib/{crc64.c => crc/crc64-main.c} (66%)
rename lib/{ => crc}/crc7.c (100%)
rename lib/{ => crc}/crc8.c (100%)
rename lib/{ => crc}/gen_crc32table.c (95%)
rename lib/{ => crc}/gen_crc64table.c (81%)
rename arch/loongarch/lib/crc32-loongarch.c => lib/crc/loongarch/crc32.h (71%)
rename arch/mips/lib/crc32-mips.c => lib/crc/mips/crc32.h (82%)
rename arch/powerpc/lib/crc-t10dif.c => lib/crc/powerpc/crc-t10dif.h (75%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crc-vpmsum-template.S (100%)
rename arch/powerpc/lib/crc32.c => lib/crc/powerpc/crc32.h (64%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crc32c-vpmsum_asm.S (100%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crct10dif-vpmsum_asm.S (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul-consts.h (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul-template.h (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul.h (100%)
rename arch/riscv/lib/crc-t10dif.c => lib/crc/riscv/crc-t10dif.h (62%)
rename {arch/riscv/lib => lib/crc/riscv}/crc16_msb.c (100%)
rename arch/riscv/lib/crc32.c => lib/crc/riscv/crc32.h (66%)
rename {arch/riscv/lib => lib/crc/riscv}/crc32_lsb.c (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc32_msb.c (100%)
rename arch/riscv/lib/crc64.c => lib/crc/riscv/crc64.h (65%)
rename {arch/riscv/lib => lib/crc/riscv}/crc64_lsb.c (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc64_msb.c (100%)
rename {arch/s390/lib => lib/crc/s390}/crc32-vx.h (100%)
rename arch/s390/lib/crc32.c => lib/crc/s390/crc32.h (81%)
rename {arch/s390/lib => lib/crc/s390}/crc32be-vx.c (100%)
rename {arch/s390/lib => lib/crc/s390}/crc32le-vx.c (100%)
rename arch/sparc/lib/crc32.c => lib/crc/sparc/crc32.h (60%)
rename {arch/sparc/lib => lib/crc/sparc}/crc32c_asm.S (100%)
create mode 100644 lib/crc/tests/Makefile
rename lib/{ => crc}/tests/crc_kunit.c (100%)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-consts.h (100%)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.h (100%)
rename arch/x86/lib/crc-t10dif.c => lib/crc/x86/crc-t10dif.h (56%)
rename {arch/x86/lib => lib/crc/x86}/crc16-msb-pclmul.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc32-pclmul.S (100%)
rename arch/x86/lib/crc32.c => lib/crc/x86/crc32.h (76%)
rename {arch/x86/lib => lib/crc/x86}/crc32c-3way.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc64-pclmul.S (100%)
rename arch/x86/lib/crc64.c => lib/crc/x86/crc64.h (61%)
base-commit: 464e77d8417003dfa3b0f556eba0297169483249
--
2.49.0
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v2 01/12] lib/crc: move files into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 02/12] lib/crc: prepare for arch-optimized code in subdirs of lib/crc/ Eric Biggers
` (16 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move all CRC files in lib/ into a subdirectory lib/crc/ to keep them
from cluttering up the main lib/ directory.
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
Documentation/core-api/kernel-api.rst | 14 ++--
MAINTAINERS | 3 +-
include/linux/crc64.h | 3 -
lib/Kconfig | 87 +-------------------
lib/Kconfig.debug | 21 -----
lib/Makefile | 32 +-------
lib/crc/.gitignore | 5 ++
lib/crc/Kconfig | 111 ++++++++++++++++++++++++++
lib/crc/Makefile | 32 ++++++++
lib/{ => crc}/crc-ccitt.c | 3 -
lib/{ => crc}/crc-itu-t.c | 0
lib/{ => crc}/crc-t10dif.c | 0
lib/{ => crc}/crc16.c | 0
lib/{ => crc}/crc32.c | 0
lib/{ => crc}/crc4.c | 0
lib/{ => crc}/crc64.c | 0
lib/{ => crc}/crc7.c | 0
lib/{ => crc}/crc8.c | 0
lib/{ => crc}/gen_crc32table.c | 4 +-
lib/{ => crc}/gen_crc64table.c | 11 +--
lib/crc/tests/Makefile | 2 +
lib/{ => crc}/tests/crc_kunit.c | 0
lib/tests/Makefile | 1 -
23 files changed, 164 insertions(+), 165 deletions(-)
create mode 100644 lib/crc/.gitignore
create mode 100644 lib/crc/Kconfig
create mode 100644 lib/crc/Makefile
rename lib/{ => crc}/crc-ccitt.c (98%)
rename lib/{ => crc}/crc-itu-t.c (100%)
rename lib/{ => crc}/crc-t10dif.c (100%)
rename lib/{ => crc}/crc16.c (100%)
rename lib/{ => crc}/crc32.c (100%)
rename lib/{ => crc}/crc4.c (100%)
rename lib/{ => crc}/crc64.c (100%)
rename lib/{ => crc}/crc7.c (100%)
rename lib/{ => crc}/crc8.c (100%)
rename lib/{ => crc}/gen_crc32table.c (95%)
rename lib/{ => crc}/gen_crc64table.c (81%)
create mode 100644 lib/crc/tests/Makefile
rename lib/{ => crc}/tests/crc_kunit.c (100%)
diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api/kernel-api.rst
index ae92a2571388a..c4642d9f13a9c 100644
--- a/Documentation/core-api/kernel-api.rst
+++ b/Documentation/core-api/kernel-api.rst
@@ -134,28 +134,28 @@ Arithmetic Overflow Checking
:internal:
CRC Functions
-------------
-.. kernel-doc:: lib/crc4.c
+.. kernel-doc:: lib/crc/crc4.c
:export:
-.. kernel-doc:: lib/crc7.c
+.. kernel-doc:: lib/crc/crc7.c
:export:
-.. kernel-doc:: lib/crc8.c
+.. kernel-doc:: lib/crc/crc8.c
:export:
-.. kernel-doc:: lib/crc16.c
+.. kernel-doc:: lib/crc/crc16.c
:export:
-.. kernel-doc:: lib/crc32.c
+.. kernel-doc:: lib/crc/crc32.c
-.. kernel-doc:: lib/crc-ccitt.c
+.. kernel-doc:: lib/crc/crc-ccitt.c
:export:
-.. kernel-doc:: lib/crc-itu-t.c
+.. kernel-doc:: lib/crc/crc-itu-t.c
:export:
Base 2 log and power Functions
------------------------------
diff --git a/MAINTAINERS b/MAINTAINERS
index 44c70071acb08..05ff204b016c5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6358,12 +6358,11 @@ L: linux-crypto@vger.kernel.org
S: Maintained
T: git https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git crc-next
F: Documentation/staging/crc*
F: arch/*/lib/crc*
F: include/linux/crc*
-F: lib/crc*
-F: lib/tests/crc_kunit.c
+F: lib/crc/
F: scripts/gen-crc-consts.py
CREATIVE SB0540
M: Bastien Nocera <hadess@hadess.net>
L: linux-input@vger.kernel.org
diff --git a/include/linux/crc64.h b/include/linux/crc64.h
index 41de30b907dff..b6aa290a79312 100644
--- a/include/linux/crc64.h
+++ b/include/linux/crc64.h
@@ -1,9 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * See lib/crc64.c for the related specification and polynomial arithmetic.
- */
#ifndef _LINUX_CRC64_H
#define _LINUX_CRC64_H
#include <linux/types.h>
diff --git a/lib/Kconfig b/lib/Kconfig
index 6c1b8f1842678..3db88ab0d2b4c 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -134,98 +134,13 @@ config TRACE_MMIO_ACCESS
depends on TRACING && ARCH_HAVE_TRACE_MMIO_ACCESS
help
Create tracepoints for MMIO read/write operations. These trace events
can be used for logging all MMIO read/write operations.
+source "lib/crc/Kconfig"
source "lib/crypto/Kconfig"
-config CRC_CCITT
- tristate
- help
- The CRC-CCITT library functions. Select this if your module uses any
- of the functions from <linux/crc-ccitt.h>.
-
-config CRC16
- tristate
- help
- The CRC16 library functions. Select this if your module uses any of
- the functions from <linux/crc16.h>.
-
-config CRC_T10DIF
- tristate
- help
- The CRC-T10DIF library functions. Select this if your module uses
- any of the functions from <linux/crc-t10dif.h>.
-
-config ARCH_HAS_CRC_T10DIF
- bool
-
-config CRC_T10DIF_ARCH
- tristate
- default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS
-
-config CRC_ITU_T
- tristate
- help
- The CRC-ITU-T library functions. Select this if your module uses
- any of the functions from <linux/crc-itu-t.h>.
-
-config CRC32
- tristate
- select BITREVERSE
- help
- The CRC32 library functions. Select this if your module uses any of
- the functions from <linux/crc32.h> or <linux/crc32c.h>.
-
-config ARCH_HAS_CRC32
- bool
-
-config CRC32_ARCH
- tristate
- default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS
-
-config CRC64
- tristate
- help
- The CRC64 library functions. Select this if your module uses any of
- the functions from <linux/crc64.h>.
-
-config ARCH_HAS_CRC64
- bool
-
-config CRC64_ARCH
- tristate
- default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS
-
-config CRC4
- tristate
- help
- The CRC4 library functions. Select this if your module uses any of
- the functions from <linux/crc4.h>.
-
-config CRC7
- tristate
- help
- The CRC7 library functions. Select this if your module uses any of
- the functions from <linux/crc7.h>.
-
-config CRC8
- tristate
- help
- The CRC8 library functions. Select this if your module uses any of
- the functions from <linux/crc8.h>.
-
-config CRC_OPTIMIZATIONS
- bool "Enable optimized CRC implementations" if EXPERT
- default y
- help
- Disabling this option reduces code size slightly by disabling the
- architecture-optimized implementations of any CRC variants that are
- enabled. CRC checksumming performance may get much slower.
-
- Keep this enabled unless you're really trying to minimize the size of
- the kernel.
config XXHASH
tristate
config AUDIT_GENERIC
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index ebe33181b6e6e..3fda96761adbc 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2899,31 +2899,10 @@ config HW_BREAKPOINT_KUNIT_TEST
help
Tests for hw_breakpoint constraints accounting.
If unsure, say N.
-config CRC_KUNIT_TEST
- tristate "KUnit tests for CRC functions" if !KUNIT_ALL_TESTS
- depends on KUNIT
- default KUNIT_ALL_TESTS
- select CRC7
- select CRC16
- select CRC_T10DIF
- select CRC32
- select CRC64
- help
- Unit tests for the CRC library functions.
-
- This is intended to help people writing architecture-specific
- optimized versions. If unsure, say N.
-
-config CRC_BENCHMARK
- bool "Benchmark for the CRC functions"
- depends on CRC_KUNIT_TEST
- help
- Include benchmarks in the KUnit test suite for the CRC functions.
-
config SIPHASH_KUNIT_TEST
tristate "Perform selftest on siphash functions" if !KUNIT_ALL_TESTS
depends on KUNIT
default KUNIT_ALL_TESTS
help
diff --git a/lib/Makefile b/lib/Makefile
index c38582f187dd8..14a5928bb57f5 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -120,11 +120,11 @@ CFLAGS_kobject_uevent.o += -DDEBUG
endif
obj-$(CONFIG_DEBUG_INFO_REDUCED) += debug_info.o
CFLAGS_debug_info.o += $(call cc-option, -femit-struct-debug-detailed=any)
-obj-y += math/ crypto/ tests/ vdso/
+obj-y += math/ crc/ crypto/ tests/ vdso/
obj-$(CONFIG_GENERIC_IOMAP) += iomap.o
obj-$(CONFIG_HAS_IOMEM) += iomap_copy.o devres.o
obj-$(CONFIG_CHECK_SIGNATURE) += check_signature.o
obj-$(CONFIG_DEBUG_LOCKING_API_SELFTESTS) += locking-selftest.o
@@ -146,19 +146,10 @@ obj-$(CONFIG_DEBUG_OBJECTS) += debugobjects.o
obj-$(CONFIG_BITREVERSE) += bitrev.o
obj-$(CONFIG_LINEAR_RANGES) += linear_ranges.o
obj-$(CONFIG_PACKING) += packing.o
obj-$(CONFIG_PACKING_KUNIT_TEST) += packing_test.o
-obj-$(CONFIG_CRC_CCITT) += crc-ccitt.o
-obj-$(CONFIG_CRC16) += crc16.o
-obj-$(CONFIG_CRC_T10DIF)+= crc-t10dif.o
-obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o
-obj-$(CONFIG_CRC32) += crc32.o
-obj-$(CONFIG_CRC64) += crc64.o
-obj-$(CONFIG_CRC4) += crc4.o
-obj-$(CONFIG_CRC7) += crc7.o
-obj-$(CONFIG_CRC8) += crc8.o
obj-$(CONFIG_XXHASH) += xxhash.o
obj-$(CONFIG_GENERIC_ALLOCATOR) += genalloc.o
obj-$(CONFIG_842_COMPRESS) += 842/
obj-$(CONFIG_842_DECOMPRESS) += 842/
@@ -292,31 +283,10 @@ obj-$(CONFIG_PERCPU_TEST) += percpu_test.o
obj-$(CONFIG_ASN1) += asn1_decoder.o
obj-$(CONFIG_ASN1_ENCODER) += asn1_encoder.o
obj-$(CONFIG_FONT_SUPPORT) += fonts/
-hostprogs := gen_crc32table
-hostprogs += gen_crc64table
-clean-files := crc32table.h
-clean-files += crc64table.h
-
-$(obj)/crc32.o: $(obj)/crc32table.h
-
-quiet_cmd_crc32 = GEN $@
- cmd_crc32 = $< > $@
-
-$(obj)/crc32table.h: $(obj)/gen_crc32table
- $(call cmd,crc32)
-
-$(obj)/crc64.o: $(obj)/crc64table.h
-
-quiet_cmd_crc64 = GEN $@
- cmd_crc64 = $< > $@
-
-$(obj)/crc64table.h: $(obj)/gen_crc64table
- $(call cmd,crc64)
-
#
# Build a fast OID lookip registry from include/linux/oid_registry.h
#
obj-$(CONFIG_OID_REGISTRY) += oid_registry.o
diff --git a/lib/crc/.gitignore b/lib/crc/.gitignore
new file mode 100644
index 0000000000000..a9e48103c9fb1
--- /dev/null
+++ b/lib/crc/.gitignore
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0-only
+/crc32table.h
+/crc64table.h
+/gen_crc32table
+/gen_crc64table
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
new file mode 100644
index 0000000000000..e0e7168b74c75
--- /dev/null
+++ b/lib/crc/Kconfig
@@ -0,0 +1,111 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+# Kconfig for the kernel's cyclic redundancy check (CRC) library code
+
+config CRC4
+ tristate
+ help
+ The CRC4 library functions. Select this if your module uses any of
+ the functions from <linux/crc4.h>.
+
+config CRC7
+ tristate
+ help
+ The CRC7 library functions. Select this if your module uses any of
+ the functions from <linux/crc7.h>.
+
+config CRC8
+ tristate
+ help
+ The CRC8 library functions. Select this if your module uses any of
+ the functions from <linux/crc8.h>.
+
+config CRC16
+ tristate
+ help
+ The CRC16 library functions. Select this if your module uses any of
+ the functions from <linux/crc16.h>.
+
+config CRC_CCITT
+ tristate
+ help
+ The CRC-CCITT library functions. Select this if your module uses any
+ of the functions from <linux/crc-ccitt.h>.
+
+config CRC_ITU_T
+ tristate
+ help
+ The CRC-ITU-T library functions. Select this if your module uses
+ any of the functions from <linux/crc-itu-t.h>.
+
+config CRC_T10DIF
+ tristate
+ help
+ The CRC-T10DIF library functions. Select this if your module uses
+ any of the functions from <linux/crc-t10dif.h>.
+
+config ARCH_HAS_CRC_T10DIF
+ bool
+
+config CRC_T10DIF_ARCH
+ tristate
+ default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS
+
+config CRC32
+ tristate
+ select BITREVERSE
+ help
+ The CRC32 library functions. Select this if your module uses any of
+ the functions from <linux/crc32.h> or <linux/crc32c.h>.
+
+config ARCH_HAS_CRC32
+ bool
+
+config CRC32_ARCH
+ tristate
+ default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS
+
+config CRC64
+ tristate
+ help
+ The CRC64 library functions. Select this if your module uses any of
+ the functions from <linux/crc64.h>.
+
+config ARCH_HAS_CRC64
+ bool
+
+config CRC64_ARCH
+ tristate
+ default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS
+
+config CRC_OPTIMIZATIONS
+ bool "Enable optimized CRC implementations" if EXPERT
+ default y
+ help
+ Disabling this option reduces code size slightly by disabling the
+ architecture-optimized implementations of any CRC variants that are
+ enabled. CRC checksumming performance may get much slower.
+
+ Keep this enabled unless you're really trying to minimize the size of
+ the kernel.
+
+config CRC_KUNIT_TEST
+ tristate "KUnit tests for CRC functions" if !KUNIT_ALL_TESTS
+ depends on KUNIT
+ default KUNIT_ALL_TESTS
+ select CRC7
+ select CRC16
+ select CRC_T10DIF
+ select CRC32
+ select CRC64
+ help
+ Unit tests for the CRC library functions.
+
+ This is intended to help people writing architecture-specific
+ optimized versions. If unsure, say N.
+
+config CRC_BENCHMARK
+ bool "Benchmark for the CRC functions"
+ depends on CRC_KUNIT_TEST
+ help
+ Include benchmarks in the KUnit test suite for the CRC functions.
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
new file mode 100644
index 0000000000000..ff4c30dda4528
--- /dev/null
+++ b/lib/crc/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+# Makefile for the kernel's cyclic redundancy check (CRC) library code
+
+obj-$(CONFIG_CRC4) += crc4.o
+obj-$(CONFIG_CRC7) += crc7.o
+obj-$(CONFIG_CRC8) += crc8.o
+obj-$(CONFIG_CRC16) += crc16.o
+obj-$(CONFIG_CRC_CCITT) += crc-ccitt.o
+obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o
+obj-$(CONFIG_CRC_T10DIF) += crc-t10dif.o
+obj-$(CONFIG_CRC32) += crc32.o
+obj-$(CONFIG_CRC64) += crc64.o
+obj-y += tests/
+
+hostprogs := gen_crc32table gen_crc64table
+clean-files := crc32table.h crc64table.h
+
+$(obj)/crc32.o: $(obj)/crc32table.h
+$(obj)/crc64.o: $(obj)/crc64table.h
+
+quiet_cmd_crc32 = GEN $@
+ cmd_crc32 = $< > $@
+
+quiet_cmd_crc64 = GEN $@
+ cmd_crc64 = $< > $@
+
+$(obj)/crc32table.h: $(obj)/gen_crc32table
+ $(call cmd,crc32)
+
+$(obj)/crc64table.h: $(obj)/gen_crc64table
+ $(call cmd,crc64)
diff --git a/lib/crc-ccitt.c b/lib/crc/crc-ccitt.c
similarity index 98%
rename from lib/crc-ccitt.c
rename to lib/crc/crc-ccitt.c
index 9cddf35d3b66e..8d2bc419230b3 100644
--- a/lib/crc-ccitt.c
+++ b/lib/crc/crc-ccitt.c
@@ -1,9 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/*
- * linux/lib/crc-ccitt.c
- */
#include <linux/types.h>
#include <linux/module.h>
#include <linux/crc-ccitt.h>
diff --git a/lib/crc-itu-t.c b/lib/crc/crc-itu-t.c
similarity index 100%
rename from lib/crc-itu-t.c
rename to lib/crc/crc-itu-t.c
diff --git a/lib/crc-t10dif.c b/lib/crc/crc-t10dif.c
similarity index 100%
rename from lib/crc-t10dif.c
rename to lib/crc/crc-t10dif.c
diff --git a/lib/crc16.c b/lib/crc/crc16.c
similarity index 100%
rename from lib/crc16.c
rename to lib/crc/crc16.c
diff --git a/lib/crc32.c b/lib/crc/crc32.c
similarity index 100%
rename from lib/crc32.c
rename to lib/crc/crc32.c
diff --git a/lib/crc4.c b/lib/crc/crc4.c
similarity index 100%
rename from lib/crc4.c
rename to lib/crc/crc4.c
diff --git a/lib/crc64.c b/lib/crc/crc64.c
similarity index 100%
rename from lib/crc64.c
rename to lib/crc/crc64.c
diff --git a/lib/crc7.c b/lib/crc/crc7.c
similarity index 100%
rename from lib/crc7.c
rename to lib/crc/crc7.c
diff --git a/lib/crc8.c b/lib/crc/crc8.c
similarity index 100%
rename from lib/crc8.c
rename to lib/crc/crc8.c
diff --git a/lib/gen_crc32table.c b/lib/crc/gen_crc32table.c
similarity index 95%
rename from lib/gen_crc32table.c
rename to lib/crc/gen_crc32table.c
index 6d03425b849e4..9a7f31658e355 100644
--- a/lib/gen_crc32table.c
+++ b/lib/crc/gen_crc32table.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
#include <stdio.h>
-#include "../include/linux/crc32poly.h"
-#include "../include/generated/autoconf.h"
+#include "../../include/linux/crc32poly.h"
+#include "../../include/generated/autoconf.h"
#include <inttypes.h>
static uint32_t crc32table_le[256];
static uint32_t crc32table_be[256];
static uint32_t crc32ctable_le[256];
diff --git a/lib/gen_crc64table.c b/lib/crc/gen_crc64table.c
similarity index 81%
rename from lib/gen_crc64table.c
rename to lib/crc/gen_crc64table.c
index e05a4230a0a08..f2be9f62bab71 100644
--- a/lib/gen_crc64table.c
+++ b/lib/crc/gen_crc64table.c
@@ -1,16 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
/*
- * Generate lookup table for the table-driven CRC64 calculation.
- *
- * gen_crc64table is executed in kernel build time and generates
- * lib/crc64table.h. This header is included by lib/crc64.c for
- * the table-driven CRC64 calculation.
- *
- * See lib/crc64.c for more information about which specification
- * and polynomial arithmetic that gen_crc64table.c follows to
- * generate the lookup table.
+ * This host program runs at kernel build time and generates the lookup tables
+ * used by the generic CRC64 code.
*
* Copyright 2018 SUSE Linux.
* Author: Coly Li <colyli@suse.de>
*/
#include <inttypes.h>
diff --git a/lib/crc/tests/Makefile b/lib/crc/tests/Makefile
new file mode 100644
index 0000000000000..65f63c318ef5e
--- /dev/null
+++ b/lib/crc/tests/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_CRC_KUNIT_TEST) += crc_kunit.o
diff --git a/lib/tests/crc_kunit.c b/lib/crc/tests/crc_kunit.c
similarity index 100%
rename from lib/tests/crc_kunit.c
rename to lib/crc/tests/crc_kunit.c
diff --git a/lib/tests/Makefile b/lib/tests/Makefile
index 56d6450144828..741d3ac2cba25 100644
--- a/lib/tests/Makefile
+++ b/lib/tests/Makefile
@@ -8,11 +8,10 @@ obj-$(CONFIG_BITFIELD_KUNIT) += bitfield_kunit.o
obj-$(CONFIG_BITS_TEST) += test_bits.o
obj-$(CONFIG_BLACKHOLE_DEV_KUNIT_TEST) += blackhole_dev_kunit.o
obj-$(CONFIG_CHECKSUM_KUNIT) += checksum_kunit.o
obj-$(CONFIG_CMDLINE_KUNIT_TEST) += cmdline_kunit.o
obj-$(CONFIG_CPUMASK_KUNIT_TEST) += cpumask_kunit.o
-obj-$(CONFIG_CRC_KUNIT_TEST) += crc_kunit.o
CFLAGS_fortify_kunit.o += $(call cc-disable-warning, unsequenced)
CFLAGS_fortify_kunit.o += $(call cc-disable-warning, stringop-overread)
CFLAGS_fortify_kunit.o += $(call cc-disable-warning, stringop-truncation)
CFLAGS_fortify_kunit.o += $(DISABLE_STRUCTLEAK_PLUGIN)
obj-$(CONFIG_FORTIFY_KUNIT_TEST) += fortify_kunit.o
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 02/12] lib/crc: prepare for arch-optimized code in subdirs of lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
2025-06-07 20:04 ` [PATCH v2 01/12] lib/crc: move files into lib/crc/ Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 03/12] lib/crc/arm: migrate arm-optimized CRC code into lib/crc/ Eric Biggers
` (15 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Rework how lib/crc/ supports arch-optimized code. First, instead of the
arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it will now be in
lib/crc/$(SRCARCH)/. Second, the API functions (e.g. crc32c()),
arch-optimized functions (e.g. crc32c_arch()), and generic functions
(e.g. crc32c_base()) will now be part of a single module for each CRC
type, allowing better inlining and dead code elimination. The second
change is made possible by the first.
As an example, consider CONFIG_CRC32=m on x86. We'll now have just
crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
were already coupled together and always both got loaded together via
direct symbol dependency, so the separation provided no benefit.
Note: later I'd like to apply the same design to lib/crypto/ too, where
often the API functions are out-of-line so this will work even better.
In those cases, for each algorithm we currently have 3 modules all
coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
sha256-x86.ko. We should have just one, inline things properly, and
rely on the compiler's dead code elimination to decide the inclusion of
the generic code instead of manually setting it via kconfig.
Having arch-specific code outside arch/ was somewhat controversial when
Zinc proposed it back in 2018. But I don't think the concerns are
warranted. It's better from a technical perspective, as it enables the
improvements mentioned above. This model is already successfully used
in other places in the kernel such as lib/raid6/. The community of each
architecture still remains free to work on the code, even if it's not in
arch/. At the time there was also a desire to put the library code in
the same files as the old-school crypto API, but that was a mistake; now
that the library is separate, that's no longer a constraint either.
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
Documentation/core-api/kernel-api.rst | 2 +-
MAINTAINERS | 1 -
include/linux/crc-t10dif.h | 10 +--
include/linux/crc32.h | 30 +--------
include/linux/crc64.h | 19 +-----
lib/crc/Kconfig | 12 ++--
lib/crc/Makefile | 20 +++++-
lib/crc/{crc-t10dif.c => crc-t10dif-main.c} | 37 ++++++++---
lib/crc/{crc32.c => crc32-main.c} | 69 +++++++++++++++++----
lib/crc/{crc64.c => crc64-main.c} | 47 +++++++++++---
10 files changed, 158 insertions(+), 89 deletions(-)
rename lib/crc/{crc-t10dif.c => crc-t10dif-main.c} (78%)
rename lib/crc/{crc32.c => crc32-main.c} (58%)
rename lib/crc/{crc64.c => crc64-main.c} (66%)
diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api/kernel-api.rst
index c4642d9f13a9c..9c8370891a39b 100644
--- a/Documentation/core-api/kernel-api.rst
+++ b/Documentation/core-api/kernel-api.rst
@@ -146,11 +146,11 @@ CRC Functions
:export:
.. kernel-doc:: lib/crc/crc16.c
:export:
-.. kernel-doc:: lib/crc/crc32.c
+.. kernel-doc:: lib/crc/crc32-main.c
.. kernel-doc:: lib/crc/crc-ccitt.c
:export:
.. kernel-doc:: lib/crc/crc-itu-t.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 05ff204b016c5..a729d80594557 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6356,11 +6356,10 @@ M: Eric Biggers <ebiggers@kernel.org>
R: Ard Biesheuvel <ardb@kernel.org>
L: linux-crypto@vger.kernel.org
S: Maintained
T: git https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git crc-next
F: Documentation/staging/crc*
-F: arch/*/lib/crc*
F: include/linux/crc*
F: lib/crc/
F: scripts/gen-crc-consts.py
CREATIVE SB0540
diff --git a/include/linux/crc-t10dif.h b/include/linux/crc-t10dif.h
index a559fdff3f7e2..ecc8bc2dd7f4c 100644
--- a/include/linux/crc-t10dif.h
+++ b/include/linux/crc-t10dif.h
@@ -2,19 +2,11 @@
#ifndef _LINUX_CRC_T10DIF_H
#define _LINUX_CRC_T10DIF_H
#include <linux/types.h>
-u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len);
-u16 crc_t10dif_generic(u16 crc, const u8 *p, size_t len);
-
-static inline u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC_T10DIF_ARCH))
- return crc_t10dif_arch(crc, p, len);
- return crc_t10dif_generic(crc, p, len);
-}
+u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len);
static inline u16 crc_t10dif(const u8 *p, size_t len)
{
return crc_t10dif_update(0, p, len);
}
diff --git a/include/linux/crc32.h b/include/linux/crc32.h
index 36bbc0405aa04..22dbe7144eb44 100644
--- a/include/linux/crc32.h
+++ b/include/linux/crc32.h
@@ -3,37 +3,13 @@
#define _LINUX_CRC32_H
#include <linux/types.h>
#include <linux/bitrev.h>
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len);
-u32 crc32_le_base(u32 crc, const u8 *p, size_t len);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len);
-u32 crc32_be_base(u32 crc, const u8 *p, size_t len);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len);
-u32 crc32c_base(u32 crc, const u8 *p, size_t len);
-
-static inline u32 crc32_le(u32 crc, const void *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC32_ARCH))
- return crc32_le_arch(crc, p, len);
- return crc32_le_base(crc, p, len);
-}
-
-static inline u32 crc32_be(u32 crc, const void *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC32_ARCH))
- return crc32_be_arch(crc, p, len);
- return crc32_be_base(crc, p, len);
-}
-
-static inline u32 crc32c(u32 crc, const void *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC32_ARCH))
- return crc32c_arch(crc, p, len);
- return crc32c_base(crc, p, len);
-}
+u32 crc32_le(u32 crc, const void *p, size_t len);
+u32 crc32_be(u32 crc, const void *p, size_t len);
+u32 crc32c(u32 crc, const void *p, size_t len);
/*
* crc32_optimizations() returns flags that indicate which CRC32 library
* functions are using architecture-specific optimizations. Unlike
* IS_ENABLED(CONFIG_CRC32_ARCH) it takes into account the different CRC32
diff --git a/include/linux/crc64.h b/include/linux/crc64.h
index b6aa290a79312..fc0c06ab1993c 100644
--- a/include/linux/crc64.h
+++ b/include/linux/crc64.h
@@ -2,28 +2,18 @@
#ifndef _LINUX_CRC64_H
#define _LINUX_CRC64_H
#include <linux/types.h>
-u64 crc64_be_arch(u64 crc, const u8 *p, size_t len);
-u64 crc64_be_generic(u64 crc, const u8 *p, size_t len);
-u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len);
-u64 crc64_nvme_generic(u64 crc, const u8 *p, size_t len);
-
/**
* crc64_be - Calculate bitwise big-endian ECMA-182 CRC64
* @crc: seed value for computation. 0 or (u64)~0 for a new CRC calculation,
* or the previous crc64 value if computing incrementally.
* @p: pointer to buffer over which CRC64 is run
* @len: length of buffer @p
*/
-static inline u64 crc64_be(u64 crc, const void *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC64_ARCH))
- return crc64_be_arch(crc, p, len);
- return crc64_be_generic(crc, p, len);
-}
+u64 crc64_be(u64 crc, const void *p, size_t len);
/**
* crc64_nvme - Calculate CRC64-NVME
* @crc: seed value for computation. 0 for a new CRC calculation, or the
* previous crc64 value if computing incrementally.
@@ -31,13 +21,8 @@ static inline u64 crc64_be(u64 crc, const void *p, size_t len)
* @len: length of buffer @p
*
* This computes the CRC64 defined in the NVME NVM Command Set Specification,
* *including the bitwise inversion at the beginning and end*.
*/
-static inline u64 crc64_nvme(u64 crc, const void *p, size_t len)
-{
- if (IS_ENABLED(CONFIG_CRC64_ARCH))
- return ~crc64_nvme_arch(~crc, p, len);
- return ~crc64_nvme_generic(~crc, p, len);
-}
+u64 crc64_nvme(u64 crc, const void *p, size_t len);
#endif /* _LINUX_CRC64_H */
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index e0e7168b74c75..e049f01a4d2c8 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -46,12 +46,12 @@ config CRC_T10DIF
config ARCH_HAS_CRC_T10DIF
bool
config CRC_T10DIF_ARCH
- tristate
- default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS
+ bool
+ depends on CRC_T10DIF && CRC_OPTIMIZATIONS
config CRC32
tristate
select BITREVERSE
help
@@ -60,12 +60,12 @@ config CRC32
config ARCH_HAS_CRC32
bool
config CRC32_ARCH
- tristate
- default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS
+ bool
+ depends on CRC32 && CRC_OPTIMIZATIONS
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
@@ -73,12 +73,12 @@ config CRC64
config ARCH_HAS_CRC64
bool
config CRC64_ARCH
- tristate
- default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS
+ bool
+ depends on CRC64 && CRC_OPTIMIZATIONS
config CRC_OPTIMIZATIONS
bool "Enable optimized CRC implementations" if EXPERT
default y
help
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index ff4c30dda4528..926edc3b035f6 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -6,20 +6,36 @@ obj-$(CONFIG_CRC4) += crc4.o
obj-$(CONFIG_CRC7) += crc7.o
obj-$(CONFIG_CRC8) += crc8.o
obj-$(CONFIG_CRC16) += crc16.o
obj-$(CONFIG_CRC_CCITT) += crc-ccitt.o
obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o
+
obj-$(CONFIG_CRC_T10DIF) += crc-t10dif.o
+crc-t10dif-y := crc-t10dif-main.o
+ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
+CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
+endif
+
obj-$(CONFIG_CRC32) += crc32.o
+crc32-y := crc32-main.o
+ifeq ($(CONFIG_CRC32_ARCH),y)
+CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
+endif
+
obj-$(CONFIG_CRC64) += crc64.o
+crc64-y := crc64-main.o
+ifeq ($(CONFIG_CRC64_ARCH),y)
+CFLAGS_crc64-main.o += -I$(src)/$(SRCARCH)
+endif
+
obj-y += tests/
hostprogs := gen_crc32table gen_crc64table
clean-files := crc32table.h crc64table.h
-$(obj)/crc32.o: $(obj)/crc32table.h
-$(obj)/crc64.o: $(obj)/crc64table.h
+$(obj)/crc32-main.o: $(obj)/crc32table.h
+$(obj)/crc64-main.o: $(obj)/crc64table.h
quiet_cmd_crc32 = GEN $@
cmd_crc32 = $< > $@
quiet_cmd_crc64 = GEN $@
diff --git a/lib/crc/crc-t10dif.c b/lib/crc/crc-t10dif-main.c
similarity index 78%
rename from lib/crc/crc-t10dif.c
rename to lib/crc/crc-t10dif-main.c
index 311c2ab829f15..08f1f1042a712 100644
--- a/lib/crc/crc-t10dif.c
+++ b/lib/crc/crc-t10dif-main.c
@@ -48,18 +48,41 @@ static const u16 t10_dif_crc_table[256] = {
0xA415, 0x2FA2, 0x38CC, 0xB37B, 0x1610, 0x9DA7, 0x8AC9, 0x017E,
0x1F65, 0x94D2, 0x83BC, 0x080B, 0xAD60, 0x26D7, 0x31B9, 0xBA0E,
0xF0D8, 0x7B6F, 0x6C01, 0xE7B6, 0x42DD, 0xC96A, 0xDE04, 0x55B3
};
-u16 crc_t10dif_generic(u16 crc, const u8 *p, size_t len)
+static inline __maybe_unused u16
+crc_t10dif_generic(u16 crc, const u8 *p, size_t len)
{
- size_t i;
+ while (len--)
+ crc = (crc << 8) ^ t10_dif_crc_table[(crc >> 8) ^ *p++];
+ return crc;
+}
- for (i = 0; i < len; i++)
- crc = (crc << 8) ^ t10_dif_crc_table[(crc >> 8) ^ p[i]];
+#ifdef CONFIG_CRC_T10DIF_ARCH
+#include "crc-t10dif.h" /* $(SRCARCH)/crc-t10dif.h */
+#else
+#define crc_t10dif_arch crc_t10dif_generic
+#endif
- return crc;
+u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len)
+{
+ return crc_t10dif_arch(crc, p, len);
+}
+EXPORT_SYMBOL(crc_t10dif_update);
+
+#ifdef crc_t10dif_mod_init_arch
+static int __init crc_t10dif_mod_init(void)
+{
+ crc_t10dif_mod_init_arch();
+ return 0;
+}
+subsys_initcall(crc_t10dif_mod_init);
+
+static void __exit crc_t10dif_mod_exit(void)
+{
}
-EXPORT_SYMBOL(crc_t10dif_generic);
+module_exit(crc_t10dif_mod_exit);
+#endif
-MODULE_DESCRIPTION("T10 DIF CRC calculation");
+MODULE_DESCRIPTION("CRC-T10DIF library functions");
MODULE_LICENSE("GPL");
diff --git a/lib/crc/crc32.c b/lib/crc/crc32-main.c
similarity index 58%
rename from lib/crc/crc32.c
rename to lib/crc/crc32-main.c
index 6811b37df2aad..b86ee66075d0e 100644
--- a/lib/crc/crc32.c
+++ b/lib/crc/crc32-main.c
@@ -28,32 +28,77 @@
#include <linux/module.h>
#include <linux/types.h>
#include "crc32table.h"
-MODULE_AUTHOR("Matt Domsch <Matt_Domsch@dell.com>");
-MODULE_DESCRIPTION("Various CRC32 calculations");
-MODULE_LICENSE("GPL");
-
-u32 crc32_le_base(u32 crc, const u8 *p, size_t len)
+static inline __maybe_unused u32
+crc32_le_base(u32 crc, const u8 *p, size_t len)
{
while (len--)
crc = (crc >> 8) ^ crc32table_le[(crc & 255) ^ *p++];
return crc;
}
-EXPORT_SYMBOL(crc32_le_base);
-u32 crc32c_base(u32 crc, const u8 *p, size_t len)
+static inline __maybe_unused u32
+crc32_be_base(u32 crc, const u8 *p, size_t len)
{
while (len--)
- crc = (crc >> 8) ^ crc32ctable_le[(crc & 255) ^ *p++];
+ crc = (crc << 8) ^ crc32table_be[(crc >> 24) ^ *p++];
return crc;
}
-EXPORT_SYMBOL(crc32c_base);
-u32 crc32_be_base(u32 crc, const u8 *p, size_t len)
+static inline __maybe_unused u32
+crc32c_base(u32 crc, const u8 *p, size_t len)
{
while (len--)
- crc = (crc << 8) ^ crc32table_be[(crc >> 24) ^ *p++];
+ crc = (crc >> 8) ^ crc32ctable_le[(crc & 255) ^ *p++];
return crc;
}
-EXPORT_SYMBOL(crc32_be_base);
+
+#ifdef CONFIG_CRC32_ARCH
+#include "crc32.h" /* $(SRCARCH)/crc32.h */
+
+u32 crc32_optimizations(void)
+{
+ return crc32_optimizations_arch();
+}
+EXPORT_SYMBOL(crc32_optimizations);
+#else
+#define crc32_le_arch crc32_le_base
+#define crc32_be_arch crc32_be_base
+#define crc32c_arch crc32c_base
+#endif
+
+u32 crc32_le(u32 crc, const void *p, size_t len)
+{
+ return crc32_le_arch(crc, p, len);
+}
+EXPORT_SYMBOL(crc32_le);
+
+u32 crc32_be(u32 crc, const void *p, size_t len)
+{
+ return crc32_be_arch(crc, p, len);
+}
+EXPORT_SYMBOL(crc32_be);
+
+u32 crc32c(u32 crc, const void *p, size_t len)
+{
+ return crc32c_arch(crc, p, len);
+}
+EXPORT_SYMBOL(crc32c);
+
+#ifdef crc32_mod_init_arch
+static int __init crc32_mod_init(void)
+{
+ crc32_mod_init_arch();
+ return 0;
+}
+subsys_initcall(crc32_mod_init);
+
+static void __exit crc32_mod_exit(void)
+{
+}
+module_exit(crc32_mod_exit);
+#endif
+
+MODULE_DESCRIPTION("CRC32 library functions");
+MODULE_LICENSE("GPL");
diff --git a/lib/crc/crc64.c b/lib/crc/crc64-main.c
similarity index 66%
rename from lib/crc/crc64.c
rename to lib/crc/crc64-main.c
index 5b1b17057f0ae..e4a6d879e84c3 100644
--- a/lib/crc/crc64.c
+++ b/lib/crc/crc64-main.c
@@ -36,23 +36,56 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/crc64.h>
#include "crc64table.h"
-MODULE_DESCRIPTION("CRC64 calculations");
-MODULE_LICENSE("GPL v2");
-
-u64 crc64_be_generic(u64 crc, const u8 *p, size_t len)
+static inline __maybe_unused u64
+crc64_be_generic(u64 crc, const u8 *p, size_t len)
{
while (len--)
crc = (crc << 8) ^ crc64table[(crc >> 56) ^ *p++];
return crc;
}
-EXPORT_SYMBOL_GPL(crc64_be_generic);
-u64 crc64_nvme_generic(u64 crc, const u8 *p, size_t len)
+static inline __maybe_unused u64
+crc64_nvme_generic(u64 crc, const u8 *p, size_t len)
{
while (len--)
crc = (crc >> 8) ^ crc64nvmetable[(crc & 0xff) ^ *p++];
return crc;
}
-EXPORT_SYMBOL_GPL(crc64_nvme_generic);
+
+#ifdef CONFIG_CRC64_ARCH
+#include "crc64.h" /* $(SRCARCH)/crc64.h */
+#else
+#define crc64_be_arch crc64_be_generic
+#define crc64_nvme_arch crc64_nvme_generic
+#endif
+
+u64 crc64_be(u64 crc, const void *p, size_t len)
+{
+ return crc64_be_arch(crc, p, len);
+}
+EXPORT_SYMBOL_GPL(crc64_be);
+
+u64 crc64_nvme(u64 crc, const void *p, size_t len)
+{
+ return ~crc64_nvme_arch(~crc, p, len);
+}
+EXPORT_SYMBOL_GPL(crc64_nvme);
+
+#ifdef crc64_mod_init_arch
+static int __init crc64_mod_init(void)
+{
+ crc64_mod_init_arch();
+ return 0;
+}
+subsys_initcall(crc64_mod_init);
+
+static void __exit crc64_mod_exit(void)
+{
+}
+module_exit(crc64_mod_exit);
+#endif
+
+MODULE_DESCRIPTION("CRC64 library functions");
+MODULE_LICENSE("GPL");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 03/12] lib/crc/arm: migrate arm-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
2025-06-07 20:04 ` [PATCH v2 01/12] lib/crc: move files into lib/crc/ Eric Biggers
2025-06-07 20:04 ` [PATCH v2 02/12] lib/crc: prepare for arch-optimized code in subdirs of lib/crc/ Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 04/12] lib/crc/arm64: migrate arm64-optimized " Eric Biggers
` (14 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the arm-optimized CRC code from arch/arm/lib/crc* into its new
location in lib/crc/arm/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/arm/Kconfig | 2 -
arch/arm/lib/Makefile | 6 ---
lib/crc/Kconfig | 2 +
lib/crc/Makefile | 2 +
.../arm/lib => lib/crc/arm}/crc-t10dif-core.S | 0
.../crc-t10dif.c => lib/crc/arm/crc-t10dif.h | 23 ++---------
{arch/arm/lib => lib/crc/arm}/crc32-core.S | 0
arch/arm/lib/crc32.c => lib/crc/arm/crc32.h | 38 ++++---------------
8 files changed, 15 insertions(+), 58 deletions(-)
rename {arch/arm/lib => lib/crc/arm}/crc-t10dif-core.S (100%)
rename arch/arm/lib/crc-t10dif.c => lib/crc/arm/crc-t10dif.h (70%)
rename {arch/arm/lib => lib/crc/arm}/crc32-core.S (100%)
rename arch/arm/lib/crc32.c => lib/crc/arm/crc32.h (69%)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 3072731fe09c5..c531b49aa98ea 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -6,12 +6,10 @@ config ARM
select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE if HAVE_KRETPROBES && FRAME_POINTER && !ARM_UNWIND
select ARCH_HAS_BINFMT_FLAT
select ARCH_HAS_CACHE_LINE_SIZE if OF
select ARCH_HAS_CPU_CACHE_ALIASING
select ARCH_HAS_CPU_FINALIZE_INIT if MMU
- select ARCH_HAS_CRC32 if KERNEL_MODE_NEON
- select ARCH_HAS_CRC_T10DIF if KERNEL_MODE_NEON
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL if MMU
select ARCH_HAS_DMA_ALLOC if MMU
select ARCH_HAS_DMA_OPS
select ARCH_HAS_DMA_WRITE_COMBINE if !ARM_DMA_MEM_BUFFERABLE
diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile
index 91ea0e29107af..d1b9ea2023648 100644
--- a/arch/arm/lib/Makefile
+++ b/arch/arm/lib/Makefile
@@ -45,11 +45,5 @@ ifeq ($(CONFIG_KERNEL_MODE_NEON),y)
CFLAGS_xor-neon.o += $(CC_FLAGS_FPU)
obj-$(CONFIG_XOR_BLOCKS) += xor-neon.o
endif
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
-
-obj-$(CONFIG_CRC32_ARCH) += crc32-arm.o
-crc32-arm-y := crc32.o crc32-core.o
-
-obj-$(CONFIG_CRC_T10DIF_ARCH) += crc-t10dif-arm.o
-crc-t10dif-arm-y := crc-t10dif.o crc-t10dif-core.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index e049f01a4d2c8..edd1b99098003 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -48,10 +48,11 @@ config ARCH_HAS_CRC_T10DIF
bool
config CRC_T10DIF_ARCH
bool
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
+ default y if ARM && KERNEL_MODE_NEON
config CRC32
tristate
select BITREVERSE
help
@@ -62,10 +63,11 @@ config ARCH_HAS_CRC32
bool
config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
+ default y if ARM && KERNEL_MODE_NEON
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index 926edc3b035f6..c72d351be6cb8 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -11,16 +11,18 @@ obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o
obj-$(CONFIG_CRC_T10DIF) += crc-t10dif.o
crc-t10dif-y := crc-t10dif-main.o
ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
+crc-t10dif-$(CONFIG_ARM) += arm/crc-t10dif-core.o
endif
obj-$(CONFIG_CRC32) += crc32.o
crc32-y := crc32-main.o
ifeq ($(CONFIG_CRC32_ARCH),y)
CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
+crc32-$(CONFIG_ARM) += arm/crc32-core.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
diff --git a/arch/arm/lib/crc-t10dif-core.S b/lib/crc/arm/crc-t10dif-core.S
similarity index 100%
rename from arch/arm/lib/crc-t10dif-core.S
rename to lib/crc/arm/crc-t10dif-core.S
diff --git a/arch/arm/lib/crc-t10dif.c b/lib/crc/arm/crc-t10dif.h
similarity index 70%
rename from arch/arm/lib/crc-t10dif.c
rename to lib/crc/arm/crc-t10dif.h
index 1093f8ec13b0b..2edf7e9681d05 100644
--- a/arch/arm/lib/crc-t10dif.c
+++ b/lib/crc/arm/crc-t10dif.h
@@ -3,16 +3,10 @@
* Accelerated CRC-T10DIF using ARM NEON and Crypto Extensions instructions
*
* Copyright (C) 2016 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
-#include <linux/crc-t10dif.h>
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
-
#include <crypto/internal/simd.h>
#include <asm/neon.h>
#include <asm/simd.h>
@@ -23,11 +17,11 @@ static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pmull);
asmlinkage u16 crc_t10dif_pmull64(u16 init_crc, const u8 *buf, size_t len);
asmlinkage void crc_t10dif_pmull8(u16 init_crc, const u8 *buf, size_t len,
u8 out[16]);
-u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
+static inline u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
{
if (length >= CRC_T10DIF_PMULL_CHUNK_SIZE) {
if (static_branch_likely(&have_pmull)) {
if (crypto_simd_usable()) {
kernel_neon_begin();
@@ -47,26 +41,15 @@ u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
return crc_t10dif_generic(0, buf, sizeof(buf));
}
}
return crc_t10dif_generic(crc, data, length);
}
-EXPORT_SYMBOL(crc_t10dif_arch);
-static int __init crc_t10dif_arm_init(void)
+#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch
+static inline void crc_t10dif_mod_init_arch(void)
{
if (elf_hwcap & HWCAP_NEON) {
static_branch_enable(&have_neon);
if (elf_hwcap2 & HWCAP2_PMULL)
static_branch_enable(&have_pmull);
}
- return 0;
}
-subsys_initcall(crc_t10dif_arm_init);
-
-static void __exit crc_t10dif_arm_exit(void)
-{
-}
-module_exit(crc_t10dif_arm_exit);
-
-MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
-MODULE_DESCRIPTION("Accelerated CRC-T10DIF using ARM NEON and Crypto Extensions");
-MODULE_LICENSE("GPL v2");
diff --git a/arch/arm/lib/crc32-core.S b/lib/crc/arm/crc32-core.S
similarity index 100%
rename from arch/arm/lib/crc32-core.S
rename to lib/crc/arm/crc32-core.S
diff --git a/arch/arm/lib/crc32.c b/lib/crc/arm/crc32.h
similarity index 69%
rename from arch/arm/lib/crc32.c
rename to lib/crc/arm/crc32.h
index f2bef8849c7c3..018007e162a2b 100644
--- a/arch/arm/lib/crc32.c
+++ b/lib/crc/arm/crc32.h
@@ -4,15 +4,10 @@
*
* Copyright (C) 2016 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
#include <linux/cpufeature.h>
-#include <linux/crc32.h>
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
#include <crypto/internal/simd.h>
#include <asm/hwcap.h>
#include <asm/neon.h>
@@ -27,18 +22,18 @@ asmlinkage u32 crc32_pmull_le(const u8 buf[], u32 len, u32 init_crc);
asmlinkage u32 crc32_armv8_le(u32 init_crc, const u8 buf[], u32 len);
asmlinkage u32 crc32c_pmull_le(const u8 buf[], u32 len, u32 init_crc);
asmlinkage u32 crc32c_armv8_le(u32 init_crc, const u8 buf[], u32 len);
-static u32 crc32_le_scalar(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_scalar(u32 crc, const u8 *p, size_t len)
{
if (static_branch_likely(&have_crc32))
return crc32_armv8_le(crc, p, len);
return crc32_le_base(crc, p, len);
}
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
if (len >= PMULL_MIN_LEN + 15 &&
static_branch_likely(&have_pmull) && crypto_simd_usable()) {
size_t n = -(uintptr_t)p & 15;
@@ -55,20 +50,19 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
p += n;
len -= n;
}
return crc32_le_scalar(crc, p, len);
}
-EXPORT_SYMBOL(crc32_le_arch);
-static u32 crc32c_scalar(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_scalar(u32 crc, const u8 *p, size_t len)
{
if (static_branch_likely(&have_crc32))
return crc32c_armv8_le(crc, p, len);
return crc32c_base(crc, p, len);
}
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
if (len >= PMULL_MIN_LEN + 15 &&
static_branch_likely(&have_pmull) && crypto_simd_usable()) {
size_t n = -(uintptr_t)p & 15;
@@ -85,39 +79,23 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
p += n;
len -= n;
}
return crc32c_scalar(crc, p, len);
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_be_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
-static int __init crc32_arm_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
if (elf_hwcap2 & HWCAP2_CRC32)
static_branch_enable(&have_crc32);
if (elf_hwcap2 & HWCAP2_PMULL)
static_branch_enable(&have_pmull);
- return 0;
}
-subsys_initcall(crc32_arm_init);
-static void __exit crc32_arm_exit(void)
-{
-}
-module_exit(crc32_arm_exit);
-
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (elf_hwcap2 & (HWCAP2_CRC32 | HWCAP2_PMULL))
return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
-MODULE_DESCRIPTION("Accelerated CRC32(C) using ARM CRC, NEON and Crypto Extensions");
-MODULE_LICENSE("GPL v2");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 04/12] lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (2 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 03/12] lib/crc/arm: migrate arm-optimized CRC code into lib/crc/ Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 05/12] lib/crc/loongarch: migrate loongarch-optimized " Eric Biggers
` (13 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the arm64-optimized CRC code from arch/arm64/lib/crc* into its new
location in lib/crc/arm64/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/arm64/Kconfig | 2 --
arch/arm64/lib/Makefile | 6 -----
lib/crc/Kconfig | 2 ++
lib/crc/Makefile | 2 ++
.../lib => lib/crc/arm64}/crc-t10dif-core.S | 0
.../crc/arm64/crc-t10dif.h | 22 +++----------------
.../arm64/lib => lib/crc/arm64}/crc32-core.S | 0
.../lib/crc32.c => lib/crc/arm64/crc32.h | 19 ++++------------
8 files changed, 11 insertions(+), 42 deletions(-)
rename {arch/arm64/lib => lib/crc/arm64}/crc-t10dif-core.S (100%)
rename arch/arm64/lib/crc-t10dif.c => lib/crc/arm64/crc-t10dif.h (70%)
rename {arch/arm64/lib => lib/crc/arm64}/crc32-core.S (100%)
rename arch/arm64/lib/crc32.c => lib/crc/arm64/crc32.h (81%)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 55fc331af3371..cbeac225903f7 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -19,12 +19,10 @@ config ARM64
select ARCH_ENABLE_MEMORY_HOTREMOVE
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select ARCH_HAS_CACHE_LINE_SIZE
select ARCH_HAS_CC_PLATFORM
- select ARCH_HAS_CRC32
- select ARCH_HAS_CRC_T10DIF if KERNEL_MODE_NEON
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DMA_OPS if XEN
select ARCH_HAS_DMA_PREP_COHERENT
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index 027bfa9689c6a..9b255d9332479 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -14,16 +14,10 @@ CFLAGS_xor-neon.o += $(CC_FLAGS_FPU)
CFLAGS_REMOVE_xor-neon.o += $(CC_FLAGS_NO_FPU)
endif
lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o
-obj-$(CONFIG_CRC32_ARCH) += crc32-arm64.o
-crc32-arm64-y := crc32.o crc32-core.o
-
-obj-$(CONFIG_CRC_T10DIF_ARCH) += crc-t10dif-arm64.o
-crc-t10dif-arm64-y := crc-t10dif.o crc-t10dif-core.o
-
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
obj-$(CONFIG_ARM64_MTE) += mte.o
obj-$(CONFIG_KASAN_SW_TAGS) += kasan_sw_tags.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index edd1b99098003..63edb487daff8 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -49,10 +49,11 @@ config ARCH_HAS_CRC_T10DIF
config CRC_T10DIF_ARCH
bool
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
+ default y if ARM64 && KERNEL_MODE_NEON
config CRC32
tristate
select BITREVERSE
help
@@ -64,10 +65,11 @@ config ARCH_HAS_CRC32
config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
+ default y if ARM64
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index c72d351be6cb8..8adff4ae1ba63 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -12,17 +12,19 @@ obj-$(CONFIG_CRC_ITU_T) += crc-itu-t.o
obj-$(CONFIG_CRC_T10DIF) += crc-t10dif.o
crc-t10dif-y := crc-t10dif-main.o
ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
crc-t10dif-$(CONFIG_ARM) += arm/crc-t10dif-core.o
+crc-t10dif-$(CONFIG_ARM64) += arm64/crc-t10dif-core.o
endif
obj-$(CONFIG_CRC32) += crc32.o
crc32-y := crc32-main.o
ifeq ($(CONFIG_CRC32_ARCH),y)
CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
crc32-$(CONFIG_ARM) += arm/crc32-core.o
+crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
diff --git a/arch/arm64/lib/crc-t10dif-core.S b/lib/crc/arm64/crc-t10dif-core.S
similarity index 100%
rename from arch/arm64/lib/crc-t10dif-core.S
rename to lib/crc/arm64/crc-t10dif-core.S
diff --git a/arch/arm64/lib/crc-t10dif.c b/lib/crc/arm64/crc-t10dif.h
similarity index 70%
rename from arch/arm64/lib/crc-t10dif.c
rename to lib/crc/arm64/crc-t10dif.h
index c2ffe4fdb59d1..c4521a7f1ee9b 100644
--- a/arch/arm64/lib/crc-t10dif.c
+++ b/lib/crc/arm64/crc-t10dif.h
@@ -4,15 +4,10 @@
*
* Copyright (C) 2016 - 2017 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
#include <linux/cpufeature.h>
-#include <linux/crc-t10dif.h>
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
#include <crypto/internal/simd.h>
#include <asm/neon.h>
#include <asm/simd.h>
@@ -24,11 +19,11 @@ static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pmull);
asmlinkage void crc_t10dif_pmull_p8(u16 init_crc, const u8 *buf, size_t len,
u8 out[16]);
asmlinkage u16 crc_t10dif_pmull_p64(u16 init_crc, const u8 *buf, size_t len);
-u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
+static inline u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
{
if (length >= CRC_T10DIF_PMULL_CHUNK_SIZE) {
if (static_branch_likely(&have_pmull)) {
if (crypto_simd_usable()) {
kernel_neon_begin();
@@ -48,26 +43,15 @@ u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length)
return crc_t10dif_generic(0, buf, sizeof(buf));
}
}
return crc_t10dif_generic(crc, data, length);
}
-EXPORT_SYMBOL(crc_t10dif_arch);
-static int __init crc_t10dif_arm64_init(void)
+#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch
+static inline void crc_t10dif_mod_init_arch(void)
{
if (cpu_have_named_feature(ASIMD)) {
static_branch_enable(&have_asimd);
if (cpu_have_named_feature(PMULL))
static_branch_enable(&have_pmull);
}
- return 0;
}
-subsys_initcall(crc_t10dif_arm64_init);
-
-static void __exit crc_t10dif_arm64_exit(void)
-{
-}
-module_exit(crc_t10dif_arm64_exit);
-
-MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
-MODULE_DESCRIPTION("CRC-T10DIF using arm64 NEON and Crypto Extensions");
-MODULE_LICENSE("GPL v2");
diff --git a/arch/arm64/lib/crc32-core.S b/lib/crc/arm64/crc32-core.S
similarity index 100%
rename from arch/arm64/lib/crc32-core.S
rename to lib/crc/arm64/crc32-core.S
diff --git a/arch/arm64/lib/crc32.c b/lib/crc/arm64/crc32.h
similarity index 81%
rename from arch/arm64/lib/crc32.c
rename to lib/crc/arm64/crc32.h
index ed3acd71178f8..6e5dec45f05d2 100644
--- a/arch/arm64/lib/crc32.c
+++ b/lib/crc/arm64/crc32.h
@@ -1,11 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
-#include <linux/crc32.h>
-#include <linux/linkage.h>
-#include <linux/module.h>
-
#include <asm/alternative.h>
#include <asm/cpufeature.h>
#include <asm/neon.h>
#include <asm/simd.h>
@@ -20,11 +16,11 @@ asmlinkage u32 crc32_be_arm64(u32 crc, unsigned char const *p, size_t len);
asmlinkage u32 crc32_le_arm64_4way(u32 crc, unsigned char const *p, size_t len);
asmlinkage u32 crc32c_le_arm64_4way(u32 crc, unsigned char const *p, size_t len);
asmlinkage u32 crc32_be_arm64_4way(u32 crc, unsigned char const *p, size_t len);
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
if (!alternative_has_cap_likely(ARM64_HAS_CRC32))
return crc32_le_base(crc, p, len);
if (len >= min_len && cpu_have_named_feature(PMULL) && crypto_simd_usable()) {
@@ -39,13 +35,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
return crc;
}
return crc32_le_arm64(crc, p, len);
}
-EXPORT_SYMBOL(crc32_le_arch);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
if (!alternative_has_cap_likely(ARM64_HAS_CRC32))
return crc32c_base(crc, p, len);
if (len >= min_len && cpu_have_named_feature(PMULL) && crypto_simd_usable()) {
@@ -60,13 +55,12 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
return crc;
}
return crc32c_le_arm64(crc, p, len);
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
{
if (!alternative_has_cap_likely(ARM64_HAS_CRC32))
return crc32_be_base(crc, p, len);
if (len >= min_len && cpu_have_named_feature(PMULL) && crypto_simd_usable()) {
@@ -81,19 +75,14 @@ u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
return crc;
}
return crc32_be_arm64(crc, p, len);
}
-EXPORT_SYMBOL(crc32_be_arch);
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (alternative_has_cap_likely(ARM64_HAS_CRC32))
return CRC32_LE_OPTIMIZATION |
CRC32_BE_OPTIMIZATION |
CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("arm64-optimized CRC32 functions");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 05/12] lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (3 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 04/12] lib/crc/arm64: migrate arm64-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 06/12] lib/crc/mips: migrate mips-optimized " Eric Biggers
` (12 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the loongarch-optimized CRC code from arch/loongarch/lib/crc* into
its new location in lib/crc/loongarch/, and wire it up in the new way.
This new way of organizing the CRC code eliminates the need to
artificially split the code for each CRC variant into separate arch and
generic modules, enabling better inlining and dead code elimination.
For more details, see "lib/crc: prepare for arch-optimized code in
subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/loongarch/Kconfig | 1 -
arch/loongarch/lib/Makefile | 2 --
lib/crc/Kconfig | 1 +
.../crc/loongarch/crc32.h | 32 ++++---------------
4 files changed, 7 insertions(+), 29 deletions(-)
rename arch/loongarch/lib/crc32-loongarch.c => lib/crc/loongarch/crc32.h (71%)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 1a2cf012b8f2f..1b19893a6bdf0 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -13,11 +13,10 @@ config LOONGARCH
select ARCH_ENABLE_MEMORY_HOTPLUG
select ARCH_ENABLE_MEMORY_HOTREMOVE
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_CPU_FINALIZE_INIT
- select ARCH_HAS_CRC32
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_FAST_MULTIPLIER
select ARCH_HAS_FORTIFY_SOURCE
select ARCH_HAS_KCOV
diff --git a/arch/loongarch/lib/Makefile b/arch/loongarch/lib/Makefile
index fae77809048b8..ccea3bbd43531 100644
--- a/arch/loongarch/lib/Makefile
+++ b/arch/loongarch/lib/Makefile
@@ -9,7 +9,5 @@ lib-y += delay.o memset.o memcpy.o memmove.o \
obj-$(CONFIG_ARCH_SUPPORTS_INT128) += tishift.o
obj-$(CONFIG_CPU_HAS_LSX) += xor_simd.o xor_simd_glue.o
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
-
-obj-$(CONFIG_CRC32_ARCH) += crc32-loongarch.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index 63edb487daff8..c1629f07768f9 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -66,10 +66,11 @@ config ARCH_HAS_CRC32
config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
+ default y if LOONGARCH
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/arch/loongarch/lib/crc32-loongarch.c b/lib/crc/loongarch/crc32.h
similarity index 71%
rename from arch/loongarch/lib/crc32-loongarch.c
rename to lib/crc/loongarch/crc32.h
index b37cd8537b459..6de5c96594afc 100644
--- a/arch/loongarch/lib/crc32-loongarch.c
+++ b/lib/crc/loongarch/crc32.h
@@ -8,12 +8,10 @@
* Copyright (C) 2018 MIPS Tech, LLC
* Copyright (C) 2020-2023 Loongson Technology Corporation Limited
*/
#include <asm/cpu-features.h>
-#include <linux/crc32.h>
-#include <linux/module.h>
#include <linux/unaligned.h>
#define _CRC32(crc, value, size, type) \
do { \
__asm__ __volatile__( \
@@ -26,11 +24,11 @@ do { \
#define CRC32(crc, value, size) _CRC32(crc, value, size, crc)
#define CRC32C(crc, value, size) _CRC32(crc, value, size, crcc)
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32);
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
if (!static_branch_likely(&have_crc32))
return crc32_le_base(crc, p, len);
while (len >= sizeof(u64)) {
@@ -61,13 +59,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
CRC32(crc, value, b);
}
return crc;
}
-EXPORT_SYMBOL(crc32_le_arch);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
if (!static_branch_likely(&have_crc32))
return crc32c_base(crc, p, len);
while (len >= sizeof(u64)) {
@@ -98,38 +95,21 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
CRC32C(crc, value, b);
}
return crc;
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_be_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
-static int __init crc32_loongarch_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
if (cpu_has_crc32)
static_branch_enable(&have_crc32);
- return 0;
}
-subsys_initcall(crc32_loongarch_init);
-static void __exit crc32_loongarch_exit(void)
-{
-}
-module_exit(crc32_loongarch_exit);
-
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (static_key_enabled(&have_crc32))
return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_AUTHOR("Min Zhou <zhoumin@loongson.cn>");
-MODULE_AUTHOR("Huacai Chen <chenhuacai@loongson.cn>");
-MODULE_DESCRIPTION("CRC32 and CRC32C using LoongArch crc* instructions");
-MODULE_LICENSE("GPL v2");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 06/12] lib/crc/mips: migrate mips-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (4 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 05/12] lib/crc/loongarch: migrate loongarch-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 07/12] lib/crc/powerpc: migrate powerpc-optimized " Eric Biggers
` (11 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the mips-optimized CRC code from arch/mips/lib/crc* into its new
location in lib/crc/mips/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/mips/Kconfig | 1 -
arch/mips/lib/Makefile | 2 --
lib/crc/Kconfig | 1 +
.../lib/crc32-mips.c => lib/crc/mips/crc32.h | 33 ++++---------------
4 files changed, 7 insertions(+), 30 deletions(-)
rename arch/mips/lib/crc32-mips.c => lib/crc/mips/crc32.h (82%)
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 1e48184ecf1ec..934eb961bd0dd 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2022,11 +2022,10 @@ config CPU_MIPSR5
select MIPS_SPRAM
config CPU_MIPSR6
bool
default y if CPU_MIPS32_R6 || CPU_MIPS64_R6
- select ARCH_HAS_CRC32
select CPU_HAS_RIXI
select CPU_HAS_DIEI if !CPU_DIEI_BROKEN
select HAVE_ARCH_BITREVERSE
select MIPS_ASID_BITS_VARIABLE
select MIPS_SPRAM
diff --git a/arch/mips/lib/Makefile b/arch/mips/lib/Makefile
index 9d75845ef78e1..8c40ffb09c420 100644
--- a/arch/mips/lib/Makefile
+++ b/arch/mips/lib/Makefile
@@ -14,9 +14,7 @@ obj-$(CONFIG_PCI) += iomap-pci.o
lib-$(CONFIG_GENERIC_CSUM) := $(filter-out csum_partial.o, $(lib-y))
obj-$(CONFIG_CPU_GENERIC_DUMP_TLB) += dump_tlb.o
obj-$(CONFIG_CPU_R3000) += r3k_dump_tlb.o
-obj-$(CONFIG_CRC32_ARCH) += crc32-mips.o
-
# libgcc-style stuff needed in the kernel
obj-y += bswapsi.o bswapdi.o multi3.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index c1629f07768f9..3f534fbfba951 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -67,10 +67,11 @@ config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
default y if LOONGARCH
+ default y if MIPS && CPU_MIPSR6
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/arch/mips/lib/crc32-mips.c b/lib/crc/mips/crc32.h
similarity index 82%
rename from arch/mips/lib/crc32-mips.c
rename to lib/crc/mips/crc32.h
index 45e4d2c9fbf54..11cb272c63a69 100644
--- a/arch/mips/lib/crc32-mips.c
+++ b/lib/crc/mips/crc32.h
@@ -7,14 +7,10 @@
* Copyright (C) 2014 Linaro Ltd <yazen.ghannam@linaro.org>
* Copyright (C) 2018 MIPS Tech, LLC
*/
#include <linux/cpufeature.h>
-#include <linux/crc32.h>
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
#include <asm/mipsregs.h>
#include <linux/unaligned.h>
#ifndef TOOLCHAIN_SUPPORTS_CRC
#define _ASM_SET_CRC(OP, SZ, TYPE) \
@@ -62,11 +58,11 @@ do { \
#define CRC32C(crc, value, size) \
_CRC32(crc, value, size, crc32c)
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32);
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
if (!static_branch_likely(&have_crc32))
return crc32_le_base(crc, p, len);
if (IS_ENABLED(CONFIG_64BIT)) {
@@ -104,13 +100,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
CRC32(crc, value, b);
}
return crc;
}
-EXPORT_SYMBOL(crc32_le_arch);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
if (!static_branch_likely(&have_crc32))
return crc32c_base(crc, p, len);
if (IS_ENABLED(CONFIG_64BIT)) {
@@ -147,37 +142,21 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
CRC32C(crc, value, b);
}
return crc;
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_be_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
-static int __init crc32_mips_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
if (cpu_have_feature(cpu_feature(MIPS_CRC32)))
static_branch_enable(&have_crc32);
- return 0;
}
-subsys_initcall(crc32_mips_init);
-static void __exit crc32_mips_exit(void)
-{
-}
-module_exit(crc32_mips_exit);
-
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (static_key_enabled(&have_crc32))
return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_AUTHOR("Marcin Nowakowski <marcin.nowakowski@mips.com");
-MODULE_DESCRIPTION("CRC32 and CRC32C using optional MIPS instructions");
-MODULE_LICENSE("GPL v2");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 07/12] lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (5 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 06/12] lib/crc/mips: migrate mips-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 08/12] lib/crc/riscv: migrate riscv-optimized " Eric Biggers
` (10 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the powerpc-optimized CRC code from arch/powerpc/lib/crc* into its
new location in lib/crc/powerpc/, and wire it up in the new way. This
new way of organizing the CRC code eliminates the need to artificially
split the code for each CRC variant into separate arch and generic
modules, enabling better inlining and dead code elimination. For more
details, see "lib/crc: prepare for arch-optimized code in subdirs of
lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/powerpc/Kconfig | 2 -
arch/powerpc/lib/Makefile | 6 ---
lib/crc/Kconfig | 2 +
lib/crc/Makefile | 2 +
.../crc/powerpc/crc-t10dif.h | 20 ++--------
.../crc/powerpc}/crc-vpmsum-template.S | 0
.../lib/crc32.c => lib/crc/powerpc/crc32.h | 38 ++++---------------
.../crc/powerpc}/crc32c-vpmsum_asm.S | 0
.../crc/powerpc}/crct10dif-vpmsum_asm.S | 0
9 files changed, 14 insertions(+), 56 deletions(-)
rename arch/powerpc/lib/crc-t10dif.c => lib/crc/powerpc/crc-t10dif.h (75%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crc-vpmsum-template.S (100%)
rename arch/powerpc/lib/crc32.c => lib/crc/powerpc/crc32.h (64%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crc32c-vpmsum_asm.S (100%)
rename {arch/powerpc/lib => lib/crc/powerpc}/crct10dif-vpmsum_asm.S (100%)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c3e0cc83f1205..45b4fa7b9b02f 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -125,12 +125,10 @@ config PPC
select ARCH_DISABLE_KASAN_INLINE if PPC_RADIX_MMU
select ARCH_DMA_DEFAULT_COHERENT if !NOT_COHERENT_CACHE
select ARCH_ENABLE_MEMORY_HOTPLUG
select ARCH_ENABLE_MEMORY_HOTREMOVE
select ARCH_HAS_COPY_MC if PPC64
- select ARCH_HAS_CRC32 if PPC64 && ALTIVEC
- select ARCH_HAS_CRC_T10DIF if PPC64 && ALTIVEC
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEBUG_WX if STRICT_KERNEL_RWX
select ARCH_HAS_DEVMEM_IS_ALLOWED
diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile
index 481f968e42c7b..59de2e2232df6 100644
--- a/arch/powerpc/lib/Makefile
+++ b/arch/powerpc/lib/Makefile
@@ -78,12 +78,6 @@ obj-$(CONFIG_FTR_FIXUP_SELFTEST) += feature-fixups-test.o
obj-$(CONFIG_ALTIVEC) += xor_vmx.o xor_vmx_glue.o
CFLAGS_xor_vmx.o += -mhard-float -maltivec $(call cc-option,-mabi=altivec)
# Enable <altivec.h>
CFLAGS_xor_vmx.o += -isystem $(shell $(CC) -print-file-name=include)
-obj-$(CONFIG_CRC32_ARCH) += crc32-powerpc.o
-crc32-powerpc-y := crc32.o crc32c-vpmsum_asm.o
-
-obj-$(CONFIG_CRC_T10DIF_ARCH) += crc-t10dif-powerpc.o
-crc-t10dif-powerpc-y := crc-t10dif.o crct10dif-vpmsum_asm.o
-
obj-$(CONFIG_PPC64) += $(obj64-y)
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index 3f534fbfba951..c3f9091ee512e 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -50,10 +50,11 @@ config ARCH_HAS_CRC_T10DIF
config CRC_T10DIF_ARCH
bool
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64 && KERNEL_MODE_NEON
+ default y if PPC64 && ALTIVEC
config CRC32
tristate
select BITREVERSE
help
@@ -68,10 +69,11 @@ config CRC32_ARCH
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
default y if LOONGARCH
default y if MIPS && CPU_MIPSR6
+ default y if PPC64 && ALTIVEC
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index 8adff4ae1ba63..555fd3fb6d197 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -13,18 +13,20 @@ obj-$(CONFIG_CRC_T10DIF) += crc-t10dif.o
crc-t10dif-y := crc-t10dif-main.o
ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
crc-t10dif-$(CONFIG_ARM) += arm/crc-t10dif-core.o
crc-t10dif-$(CONFIG_ARM64) += arm64/crc-t10dif-core.o
+crc-t10dif-$(CONFIG_PPC) += powerpc/crct10dif-vpmsum_asm.o
endif
obj-$(CONFIG_CRC32) += crc32.o
crc32-y := crc32-main.o
ifeq ($(CONFIG_CRC32_ARCH),y)
CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
crc32-$(CONFIG_ARM) += arm/crc32-core.o
crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
+crc32-$(CONFIG_PPC) += powerpc/crc32c-vpmsum_asm.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
diff --git a/arch/powerpc/lib/crc-t10dif.c b/lib/crc/powerpc/crc-t10dif.h
similarity index 75%
rename from arch/powerpc/lib/crc-t10dif.c
rename to lib/crc/powerpc/crc-t10dif.h
index be23ded3a9df6..59e16804a6eae 100644
--- a/arch/powerpc/lib/crc-t10dif.c
+++ b/lib/crc/powerpc/crc-t10dif.h
@@ -7,14 +7,11 @@
*/
#include <asm/switch_to.h>
#include <crypto/internal/simd.h>
#include <linux/cpufeature.h>
-#include <linux/crc-t10dif.h>
#include <linux/jump_label.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
#include <linux/preempt.h>
#include <linux/uaccess.h>
#define VMX_ALIGN 16
#define VMX_ALIGN_MASK (VMX_ALIGN-1)
@@ -23,11 +20,11 @@
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_vec_crypto);
u32 __crct10dif_vpmsum(u32 crc, unsigned char const *p, size_t len);
-u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len)
+static inline u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len)
{
unsigned int prealign;
unsigned int tail;
u32 crc = crci;
@@ -60,24 +57,13 @@ u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len)
crc = crc_t10dif_generic(crc, p, tail);
}
return crc & 0xffff;
}
-EXPORT_SYMBOL(crc_t10dif_arch);
-static int __init crc_t10dif_powerpc_init(void)
+#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch
+static inline void crc_t10dif_mod_init_arch(void)
{
if (cpu_has_feature(CPU_FTR_ARCH_207S) &&
(cur_cpu_spec->cpu_user_features2 & PPC_FEATURE2_VEC_CRYPTO))
static_branch_enable(&have_vec_crypto);
- return 0;
}
-subsys_initcall(crc_t10dif_powerpc_init);
-
-static void __exit crc_t10dif_powerpc_exit(void)
-{
-}
-module_exit(crc_t10dif_powerpc_exit);
-
-MODULE_AUTHOR("Daniel Axtens <dja@axtens.net>");
-MODULE_DESCRIPTION("CRCT10DIF using vector polynomial multiply-sum instructions");
-MODULE_LICENSE("GPL");
diff --git a/arch/powerpc/lib/crc-vpmsum-template.S b/lib/crc/powerpc/crc-vpmsum-template.S
similarity index 100%
rename from arch/powerpc/lib/crc-vpmsum-template.S
rename to lib/crc/powerpc/crc-vpmsum-template.S
diff --git a/arch/powerpc/lib/crc32.c b/lib/crc/powerpc/crc32.h
similarity index 64%
rename from arch/powerpc/lib/crc32.c
rename to lib/crc/powerpc/crc32.h
index 0d9befb6e7b83..811cc2e6ed24d 100644
--- a/arch/powerpc/lib/crc32.c
+++ b/lib/crc/powerpc/crc32.h
@@ -1,32 +1,26 @@
// SPDX-License-Identifier: GPL-2.0-only
#include <asm/switch_to.h>
#include <crypto/internal/simd.h>
#include <linux/cpufeature.h>
-#include <linux/crc32.h>
#include <linux/jump_label.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
#include <linux/preempt.h>
#include <linux/uaccess.h>
#define VMX_ALIGN 16
#define VMX_ALIGN_MASK (VMX_ALIGN-1)
#define VECTOR_BREAKPOINT 512
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_vec_crypto);
-u32 __crc32c_vpmsum(u32 crc, const u8 *p, size_t len);
+#define crc32_le_arch crc32_le_base /* not implemented on this arch */
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_le_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_le_arch);
+u32 __crc32c_vpmsum(u32 crc, const u8 *p, size_t len);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
unsigned int prealign;
unsigned int tail;
if (len < (VECTOR_BREAKPOINT + VMX_ALIGN) ||
@@ -56,38 +50,20 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
crc = crc32c_base(crc, p, tail);
}
return crc;
}
-EXPORT_SYMBOL(crc32c_arch);
-
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_be_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
-static int __init crc32_powerpc_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
if (cpu_has_feature(CPU_FTR_ARCH_207S) &&
(cur_cpu_spec->cpu_user_features2 & PPC_FEATURE2_VEC_CRYPTO))
static_branch_enable(&have_vec_crypto);
- return 0;
-}
-subsys_initcall(crc32_powerpc_init);
-
-static void __exit crc32_powerpc_exit(void)
-{
}
-module_exit(crc32_powerpc_exit);
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (static_key_enabled(&have_vec_crypto))
return CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_AUTHOR("Anton Blanchard <anton@samba.org>");
-MODULE_DESCRIPTION("CRC32C using vector polynomial multiply-sum instructions");
-MODULE_LICENSE("GPL");
diff --git a/arch/powerpc/lib/crc32c-vpmsum_asm.S b/lib/crc/powerpc/crc32c-vpmsum_asm.S
similarity index 100%
rename from arch/powerpc/lib/crc32c-vpmsum_asm.S
rename to lib/crc/powerpc/crc32c-vpmsum_asm.S
diff --git a/arch/powerpc/lib/crct10dif-vpmsum_asm.S b/lib/crc/powerpc/crct10dif-vpmsum_asm.S
similarity index 100%
rename from arch/powerpc/lib/crct10dif-vpmsum_asm.S
rename to lib/crc/powerpc/crct10dif-vpmsum_asm.S
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 08/12] lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (6 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 07/12] lib/crc/powerpc: migrate powerpc-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized " Eric Biggers
` (9 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the riscv-optimized CRC code from arch/riscv/lib/crc* into its new
location in lib/crc/riscv/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/riscv/Kconfig | 3 ---
arch/riscv/lib/Makefile | 6 ------
lib/crc/Kconfig | 3 +++
lib/crc/Makefile | 3 +++
.../lib => lib/crc/riscv}/crc-clmul-consts.h | 0
.../lib => lib/crc/riscv}/crc-clmul-template.h | 0
{arch/riscv/lib => lib/crc/riscv}/crc-clmul.h | 0
.../crc-t10dif.c => lib/crc/riscv/crc-t10dif.h | 8 +-------
{arch/riscv/lib => lib/crc/riscv}/crc16_msb.c | 0
arch/riscv/lib/crc32.c => lib/crc/riscv/crc32.h | 17 ++++-------------
{arch/riscv/lib => lib/crc/riscv}/crc32_lsb.c | 0
{arch/riscv/lib => lib/crc/riscv}/crc32_msb.c | 0
arch/riscv/lib/crc64.c => lib/crc/riscv/crc64.h | 11 ++---------
{arch/riscv/lib => lib/crc/riscv}/crc64_lsb.c | 0
{arch/riscv/lib => lib/crc/riscv}/crc64_msb.c | 0
15 files changed, 13 insertions(+), 38 deletions(-)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul-consts.h (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul-template.h (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc-clmul.h (100%)
rename arch/riscv/lib/crc-t10dif.c => lib/crc/riscv/crc-t10dif.h (62%)
rename {arch/riscv/lib => lib/crc/riscv}/crc16_msb.c (100%)
rename arch/riscv/lib/crc32.c => lib/crc/riscv/crc32.h (66%)
rename {arch/riscv/lib => lib/crc/riscv}/crc32_lsb.c (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc32_msb.c (100%)
rename arch/riscv/lib/crc64.c => lib/crc/riscv/crc64.h (65%)
rename {arch/riscv/lib => lib/crc/riscv}/crc64_lsb.c (100%)
rename {arch/riscv/lib => lib/crc/riscv}/crc64_msb.c (100%)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 36061f4732b74..d963d8faf2aeb 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -22,13 +22,10 @@ config RISCV
select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM_VMEMMAP
select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select ARCH_HAS_BINFMT_FLAT
- select ARCH_HAS_CRC32 if RISCV_ISA_ZBC
- select ARCH_HAS_CRC64 if 64BIT && RISCV_ISA_ZBC
- select ARCH_HAS_CRC_T10DIF if RISCV_ISA_ZBC
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL if MMU
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEBUG_WX
select ARCH_HAS_FAST_MULTIPLIER
diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile
index 0baec92d2f55b..a4f4b48ed3a47 100644
--- a/arch/riscv/lib/Makefile
+++ b/arch/riscv/lib/Makefile
@@ -14,14 +14,8 @@ ifeq ($(CONFIG_MMU), y)
lib-$(CONFIG_RISCV_ISA_V) += uaccess_vector.o
endif
lib-$(CONFIG_MMU) += uaccess.o
lib-$(CONFIG_64BIT) += tishift.o
lib-$(CONFIG_RISCV_ISA_ZICBOZ) += clear_page.o
-obj-$(CONFIG_CRC32_ARCH) += crc32-riscv.o
-crc32-riscv-y := crc32.o crc32_msb.o crc32_lsb.o
-obj-$(CONFIG_CRC64_ARCH) += crc64-riscv.o
-crc64-riscv-y := crc64.o crc64_msb.o crc64_lsb.o
-obj-$(CONFIG_CRC_T10DIF_ARCH) += crc-t10dif-riscv.o
-crc-t10dif-riscv-y := crc-t10dif.o crc16_msb.o
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_RISCV_ISA_V) += xor.o
lib-$(CONFIG_RISCV_ISA_V) += riscv_v_helpers.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index c3f9091ee512e..d0c293b1a4182 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -51,10 +51,11 @@ config CRC_T10DIF_ARCH
bool
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64 && KERNEL_MODE_NEON
default y if PPC64 && ALTIVEC
+ default y if RISCV && RISCV_ISA_ZBC
config CRC32
tristate
select BITREVERSE
help
@@ -70,10 +71,11 @@ config CRC32_ARCH
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
default y if LOONGARCH
default y if MIPS && CPU_MIPSR6
default y if PPC64 && ALTIVEC
+ default y if RISCV && RISCV_ISA_ZBC
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
@@ -83,10 +85,11 @@ config ARCH_HAS_CRC64
bool
config CRC64_ARCH
bool
depends on CRC64 && CRC_OPTIMIZATIONS
+ default y if RISCV && RISCV_ISA_ZBC && 64BIT
config CRC_OPTIMIZATIONS
bool "Enable optimized CRC implementations" if EXPERT
default y
help
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index 555fd3fb6d197..190f889adf556 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -14,25 +14,28 @@ crc-t10dif-y := crc-t10dif-main.o
ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
crc-t10dif-$(CONFIG_ARM) += arm/crc-t10dif-core.o
crc-t10dif-$(CONFIG_ARM64) += arm64/crc-t10dif-core.o
crc-t10dif-$(CONFIG_PPC) += powerpc/crct10dif-vpmsum_asm.o
+crc-t10dif-$(CONFIG_RISCV) += riscv/crc16_msb.o
endif
obj-$(CONFIG_CRC32) += crc32.o
crc32-y := crc32-main.o
ifeq ($(CONFIG_CRC32_ARCH),y)
CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
crc32-$(CONFIG_ARM) += arm/crc32-core.o
crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
crc32-$(CONFIG_PPC) += powerpc/crc32c-vpmsum_asm.o
+crc32-$(CONFIG_RISCV) += riscv/crc32_lsb.o riscv/crc32_msb.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
CFLAGS_crc64-main.o += -I$(src)/$(SRCARCH)
+crc64-$(CONFIG_RISCV) += riscv/crc64_lsb.o riscv/crc64_msb.o
endif
obj-y += tests/
hostprogs := gen_crc32table gen_crc64table
diff --git a/arch/riscv/lib/crc-clmul-consts.h b/lib/crc/riscv/crc-clmul-consts.h
similarity index 100%
rename from arch/riscv/lib/crc-clmul-consts.h
rename to lib/crc/riscv/crc-clmul-consts.h
diff --git a/arch/riscv/lib/crc-clmul-template.h b/lib/crc/riscv/crc-clmul-template.h
similarity index 100%
rename from arch/riscv/lib/crc-clmul-template.h
rename to lib/crc/riscv/crc-clmul-template.h
diff --git a/arch/riscv/lib/crc-clmul.h b/lib/crc/riscv/crc-clmul.h
similarity index 100%
rename from arch/riscv/lib/crc-clmul.h
rename to lib/crc/riscv/crc-clmul.h
diff --git a/arch/riscv/lib/crc-t10dif.c b/lib/crc/riscv/crc-t10dif.h
similarity index 62%
rename from arch/riscv/lib/crc-t10dif.c
rename to lib/crc/riscv/crc-t10dif.h
index e6b0051ccd86c..cd6136cbfda1c 100644
--- a/arch/riscv/lib/crc-t10dif.c
+++ b/lib/crc/riscv/crc-t10dif.h
@@ -5,20 +5,14 @@
* Copyright 2025 Google LLC
*/
#include <asm/hwcap.h>
#include <asm/alternative-macros.h>
-#include <linux/crc-t10dif.h>
-#include <linux/module.h>
#include "crc-clmul.h"
-u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len)
+static inline u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc16_msb_clmul(crc, p, len, &crc16_msb_0x8bb7_consts);
return crc_t10dif_generic(crc, p, len);
}
-EXPORT_SYMBOL(crc_t10dif_arch);
-
-MODULE_DESCRIPTION("RISC-V optimized CRC-T10DIF function");
-MODULE_LICENSE("GPL");
diff --git a/arch/riscv/lib/crc16_msb.c b/lib/crc/riscv/crc16_msb.c
similarity index 100%
rename from arch/riscv/lib/crc16_msb.c
rename to lib/crc/riscv/crc16_msb.c
diff --git a/arch/riscv/lib/crc32.c b/lib/crc/riscv/crc32.h
similarity index 66%
rename from arch/riscv/lib/crc32.c
rename to lib/crc/riscv/crc32.h
index a3188b7d9c403..3ec6eee98afa8 100644
--- a/arch/riscv/lib/crc32.c
+++ b/lib/crc/riscv/crc32.h
@@ -5,49 +5,40 @@
* Copyright 2025 Google LLC
*/
#include <asm/hwcap.h>
#include <asm/alternative-macros.h>
-#include <linux/crc32.h>
-#include <linux/module.h>
#include "crc-clmul.h"
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc32_lsb_clmul(crc, p, len,
&crc32_lsb_0xedb88320_consts);
return crc32_le_base(crc, p, len);
}
-EXPORT_SYMBOL(crc32_le_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc32_msb_clmul(crc, p, len,
&crc32_msb_0x04c11db7_consts);
return crc32_be_base(crc, p, len);
}
-EXPORT_SYMBOL(crc32_be_arch);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc32_lsb_clmul(crc, p, len,
&crc32_lsb_0x82f63b78_consts);
return crc32c_base(crc, p, len);
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return CRC32_LE_OPTIMIZATION |
CRC32_BE_OPTIMIZATION |
CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_DESCRIPTION("RISC-V optimized CRC32 functions");
-MODULE_LICENSE("GPL");
diff --git a/arch/riscv/lib/crc32_lsb.c b/lib/crc/riscv/crc32_lsb.c
similarity index 100%
rename from arch/riscv/lib/crc32_lsb.c
rename to lib/crc/riscv/crc32_lsb.c
diff --git a/arch/riscv/lib/crc32_msb.c b/lib/crc/riscv/crc32_msb.c
similarity index 100%
rename from arch/riscv/lib/crc32_msb.c
rename to lib/crc/riscv/crc32_msb.c
diff --git a/arch/riscv/lib/crc64.c b/lib/crc/riscv/crc64.h
similarity index 65%
rename from arch/riscv/lib/crc64.c
rename to lib/crc/riscv/crc64.h
index f0015a27836a4..a1b7873fde579 100644
--- a/arch/riscv/lib/crc64.c
+++ b/lib/crc/riscv/crc64.h
@@ -5,30 +5,23 @@
* Copyright 2025 Google LLC
*/
#include <asm/hwcap.h>
#include <asm/alternative-macros.h>
-#include <linux/crc64.h>
-#include <linux/module.h>
#include "crc-clmul.h"
-u64 crc64_be_arch(u64 crc, const u8 *p, size_t len)
+static inline u64 crc64_be_arch(u64 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc64_msb_clmul(crc, p, len,
&crc64_msb_0x42f0e1eba9ea3693_consts);
return crc64_be_generic(crc, p, len);
}
-EXPORT_SYMBOL(crc64_be_arch);
-u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len)
+static inline u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len)
{
if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC))
return crc64_lsb_clmul(crc, p, len,
&crc64_lsb_0x9a6c9329ac4bc9b5_consts);
return crc64_nvme_generic(crc, p, len);
}
-EXPORT_SYMBOL(crc64_nvme_arch);
-
-MODULE_DESCRIPTION("RISC-V optimized CRC64 functions");
-MODULE_LICENSE("GPL");
diff --git a/arch/riscv/lib/crc64_lsb.c b/lib/crc/riscv/crc64_lsb.c
similarity index 100%
rename from arch/riscv/lib/crc64_lsb.c
rename to lib/crc/riscv/crc64_lsb.c
diff --git a/arch/riscv/lib/crc64_msb.c b/lib/crc/riscv/crc64_msb.c
similarity index 100%
rename from arch/riscv/lib/crc64_msb.c
rename to lib/crc/riscv/crc64_msb.c
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (7 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 08/12] lib/crc/riscv: migrate riscv-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-13 16:01 ` Alexander Gordeev
2025-06-07 20:04 ` [PATCH v2 10/12] lib/crc/sparc: migrate sparc-optimized " Eric Biggers
` (8 subsequent siblings)
17 siblings, 1 reply; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the s390-optimized CRC code from arch/s390/lib/crc* into its new
location in lib/crc/s390/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/s390/Kconfig | 1 -
arch/s390/lib/Makefile | 3 ---
lib/crc/Kconfig | 1 +
lib/crc/Makefile | 1 +
{arch/s390/lib => lib/crc/s390}/crc32-vx.h | 0
arch/s390/lib/crc32.c => lib/crc/s390/crc32.h | 16 +++-------------
{arch/s390/lib => lib/crc/s390}/crc32be-vx.c | 0
{arch/s390/lib => lib/crc/s390}/crc32le-vx.c | 0
8 files changed, 5 insertions(+), 17 deletions(-)
rename {arch/s390/lib => lib/crc/s390}/crc32-vx.h (100%)
rename arch/s390/lib/crc32.c => lib/crc/s390/crc32.h (81%)
rename {arch/s390/lib => lib/crc/s390}/crc32be-vx.c (100%)
rename {arch/s390/lib => lib/crc/s390}/crc32le-vx.c (100%)
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 0c16dc443e2f6..22b90f6aa1a09 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -73,11 +73,10 @@ config S390
select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM
select ARCH_ENABLE_MEMORY_HOTREMOVE
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_HAS_CPU_FINALIZE_INIT
- select ARCH_HAS_CRC32
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEBUG_WX
select ARCH_HAS_DEVMEM_IS_ALLOWED
diff --git a/arch/s390/lib/Makefile b/arch/s390/lib/Makefile
index cd35cdbfa8713..7c8583d46eca1 100644
--- a/arch/s390/lib/Makefile
+++ b/arch/s390/lib/Makefile
@@ -23,8 +23,5 @@ obj-$(CONFIG_S390_MODULES_SANITY_TEST) += test_modules.o
obj-$(CONFIG_S390_MODULES_SANITY_TEST_HELPERS) += test_modules_helpers.o
lib-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
obj-$(CONFIG_EXPOLINE_EXTERN) += expoline.o
-
-obj-$(CONFIG_CRC32_ARCH) += crc32-s390.o
-crc32-s390-y := crc32.o crc32le-vx.o crc32be-vx.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index d0c293b1a4182..1b69a8bef4a85 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -72,10 +72,11 @@ config CRC32_ARCH
default y if ARM64
default y if LOONGARCH
default y if MIPS && CPU_MIPSR6
default y if PPC64 && ALTIVEC
default y if RISCV && RISCV_ISA_ZBC
+ default y if S390
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index 190f889adf556..bec58266251f8 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -25,10 +25,11 @@ ifeq ($(CONFIG_CRC32_ARCH),y)
CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
crc32-$(CONFIG_ARM) += arm/crc32-core.o
crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
crc32-$(CONFIG_PPC) += powerpc/crc32c-vpmsum_asm.o
crc32-$(CONFIG_RISCV) += riscv/crc32_lsb.o riscv/crc32_msb.o
+crc32-$(CONFIG_S390) += s390/crc32le-vx.o s390/crc32be-vx.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
diff --git a/arch/s390/lib/crc32-vx.h b/lib/crc/s390/crc32-vx.h
similarity index 100%
rename from arch/s390/lib/crc32-vx.h
rename to lib/crc/s390/crc32-vx.h
diff --git a/arch/s390/lib/crc32.c b/lib/crc/s390/crc32.h
similarity index 81%
rename from arch/s390/lib/crc32.c
rename to lib/crc/s390/crc32.h
index 3c4b344417c11..59c8983d428be 100644
--- a/arch/s390/lib/crc32.c
+++ b/lib/crc/s390/crc32.h
@@ -3,16 +3,12 @@
* CRC-32 implemented with the z/Architecture Vector Extension Facility.
*
* Copyright IBM Corp. 2015
* Author(s): Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
*/
-#define KMSG_COMPONENT "crc32-vx"
-#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
-#include <linux/module.h>
#include <linux/cpufeature.h>
-#include <linux/crc32.h>
#include <asm/fpu.h>
#include "crc32-vx.h"
#define VX_MIN_LEN 64
#define VX_ALIGNMENT 16L
@@ -25,11 +21,11 @@
* on the message buffer, the hardware-accelerated or software implementation
* is used. Note that the message buffer is aligned to improve fetch
* operations of VECTOR LOAD MULTIPLE instructions.
*/
#define DEFINE_CRC32_VX(___fname, ___crc32_vx, ___crc32_sw) \
- u32 ___fname(u32 crc, const u8 *data, size_t datalen) \
+ static inline u32 ___fname(u32 crc, const u8 *data, size_t datalen) \
{ \
unsigned long prealign, aligned, remaining; \
DECLARE_KERNEL_FPU_ONSTACK16(vxstate); \
\
if (datalen < VX_MIN_LEN + VX_ALIGN_MASK || !cpu_has_vx()) \
@@ -52,26 +48,20 @@
\
if (remaining) \
crc = ___crc32_sw(crc, data + aligned, remaining); \
\
return crc; \
- } \
- EXPORT_SYMBOL(___fname);
+ }
DEFINE_CRC32_VX(crc32_le_arch, crc32_le_vgfm_16, crc32_le_base)
DEFINE_CRC32_VX(crc32_be_arch, crc32_be_vgfm_16, crc32_be_base)
DEFINE_CRC32_VX(crc32c_arch, crc32c_le_vgfm_16, crc32c_base)
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (cpu_has_vx()) {
return CRC32_LE_OPTIMIZATION |
CRC32_BE_OPTIMIZATION |
CRC32C_OPTIMIZATION;
}
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_AUTHOR("Hendrik Brueckner <brueckner@linux.vnet.ibm.com>");
-MODULE_DESCRIPTION("CRC-32 algorithms using z/Architecture Vector Extension Facility");
-MODULE_LICENSE("GPL");
diff --git a/arch/s390/lib/crc32be-vx.c b/lib/crc/s390/crc32be-vx.c
similarity index 100%
rename from arch/s390/lib/crc32be-vx.c
rename to lib/crc/s390/crc32be-vx.c
diff --git a/arch/s390/lib/crc32le-vx.c b/lib/crc/s390/crc32le-vx.c
similarity index 100%
rename from arch/s390/lib/crc32le-vx.c
rename to lib/crc/s390/crc32le-vx.c
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 10/12] lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (8 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 11/12] lib/crc/x86: migrate x86-optimized " Eric Biggers
` (7 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the sparc-optimized CRC code from arch/sparc/lib/crc* into its new
location in lib/crc/sparc/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/sparc/Kconfig | 1 -
arch/sparc/lib/Makefile | 2 -
lib/crc/Kconfig | 1 +
lib/crc/Makefile | 1 +
.../lib/crc32.c => lib/crc/sparc/crc32.h | 42 ++++---------------
.../sparc/lib => lib/crc/sparc}/crc32c_asm.S | 0
6 files changed, 10 insertions(+), 37 deletions(-)
rename arch/sparc/lib/crc32.c => lib/crc/sparc/crc32.h (60%)
rename {arch/sparc/lib => lib/crc/sparc}/crc32c_asm.S (100%)
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 0f88123925a4f..dcfdb7f1dae97 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -108,11 +108,10 @@ config SPARC64
select ARCH_HAS_GIGANTIC_PAGE
select HAVE_SOFTIRQ_ON_OWN_STACK
select HAVE_SETUP_PER_CPU_AREA
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
- select ARCH_HAS_CRC32
config ARCH_PROC_KCORE_TEXT
def_bool y
config CPU_BIG_ENDIAN
diff --git a/arch/sparc/lib/Makefile b/arch/sparc/lib/Makefile
index 5cf9781d68b40..2d6c3c5352734 100644
--- a/arch/sparc/lib/Makefile
+++ b/arch/sparc/lib/Makefile
@@ -52,7 +52,5 @@ lib-$(CONFIG_SPARC64) += copy_in_user.o memmove.o
lib-$(CONFIG_SPARC64) += mcount.o ipcsum.o xor.o hweight.o ffs.o
obj-$(CONFIG_SPARC64) += iomap.o
obj-$(CONFIG_SPARC32) += atomic32.o
obj-$(CONFIG_SPARC64) += PeeCeeI.o
-obj-$(CONFIG_CRC32_ARCH) += crc32-sparc.o
-crc32-sparc-y := crc32.o crc32c_asm.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index 1b69a8bef4a85..af4fa857a81f8 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -73,10 +73,11 @@ config CRC32_ARCH
default y if LOONGARCH
default y if MIPS && CPU_MIPSR6
default y if PPC64 && ALTIVEC
default y if RISCV && RISCV_ISA_ZBC
default y if S390
+ default y if SPARC64
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index bec58266251f8..81e176db0947c 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -26,10 +26,11 @@ CFLAGS_crc32-main.o += -I$(src)/$(SRCARCH)
crc32-$(CONFIG_ARM) += arm/crc32-core.o
crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
crc32-$(CONFIG_PPC) += powerpc/crc32c-vpmsum_asm.o
crc32-$(CONFIG_RISCV) += riscv/crc32_lsb.o riscv/crc32_msb.o
crc32-$(CONFIG_S390) += s390/crc32le-vx.o s390/crc32be-vx.o
+crc32-$(CONFIG_SPARC) += sparc/crc32c_asm.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
diff --git a/arch/sparc/lib/crc32.c b/lib/crc/sparc/crc32.h
similarity index 60%
rename from arch/sparc/lib/crc32.c
rename to lib/crc/sparc/crc32.h
index 40d4720a42a1b..60f2765ac0157 100644
--- a/arch/sparc/lib/crc32.c
+++ b/lib/crc/sparc/crc32.h
@@ -6,30 +6,21 @@
* Copyright (C) 2008 Intel Corporation
* Authors: Austin Zhang <austin_zhang@linux.intel.com>
* Kent Liu <kent.liu@intel.com>
*/
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/crc32.h>
#include <asm/pstate.h>
#include <asm/elf.h>
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32c_opcode);
-u32 crc32_le_arch(u32 crc, const u8 *data, size_t len)
-{
- return crc32_le_base(crc, data, len);
-}
-EXPORT_SYMBOL(crc32_le_arch);
+#define crc32_le_arch crc32_le_base /* not implemented on this arch */
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
void crc32c_sparc64(u32 *crcp, const u64 *data, size_t len);
-u32 crc32c_arch(u32 crc, const u8 *data, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *data, size_t len)
{
size_t n = -(uintptr_t)data & 7;
if (!static_branch_likely(&have_crc32c_opcode))
return crc32c_base(crc, data, len);
@@ -49,45 +40,28 @@ u32 crc32c_arch(u32 crc, const u8 *data, size_t len)
}
if (len)
crc = crc32c_base(crc, data, len);
return crc;
}
-EXPORT_SYMBOL(crc32c_arch);
-
-u32 crc32_be_arch(u32 crc, const u8 *data, size_t len)
-{
- return crc32_be_base(crc, data, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
-static int __init crc32_sparc_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
unsigned long cfr;
if (!(sparc64_elf_hwcap & HWCAP_SPARC_CRYPTO))
- return 0;
+ return;
__asm__ __volatile__("rd %%asr26, %0" : "=r" (cfr));
if (!(cfr & CFR_CRC32C))
- return 0;
+ return;
static_branch_enable(&have_crc32c_opcode);
pr_info("Using sparc64 crc32c opcode optimized CRC32C implementation\n");
- return 0;
}
-subsys_initcall(crc32_sparc_init);
-static void __exit crc32_sparc_exit(void)
-{
-}
-module_exit(crc32_sparc_exit);
-
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
if (static_key_enabled(&have_crc32c_opcode))
return CRC32C_OPTIMIZATION;
return 0;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("CRC32c (Castagnoli), sparc64 crc32c opcode accelerated");
diff --git a/arch/sparc/lib/crc32c_asm.S b/lib/crc/sparc/crc32c_asm.S
similarity index 100%
rename from arch/sparc/lib/crc32c_asm.S
rename to lib/crc/sparc/crc32c_asm.S
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 11/12] lib/crc/x86: migrate x86-optimized CRC code into lib/crc/
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (9 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 10/12] lib/crc/sparc: migrate sparc-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 20:04 ` [PATCH v2 12/12] lib/crc: remove ARCH_HAS_* kconfig symbols Eric Biggers
` (6 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
Move the x86-optimized CRC code from arch/x86/lib/crc* into its new
location in lib/crc/x86/, and wire it up in the new way. This new way
of organizing the CRC code eliminates the need to artificially split the
code for each CRC variant into separate arch and generic modules,
enabling better inlining and dead code elimination. For more details,
see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
arch/x86/Kconfig | 3 --
arch/x86/lib/Makefile | 10 -------
lib/crc/Kconfig | 3 ++
lib/crc/Makefile | 4 +++
.../lib => lib/crc/x86}/crc-pclmul-consts.h | 0
.../lib => lib/crc/x86}/crc-pclmul-template.S | 0
.../lib => lib/crc/x86}/crc-pclmul-template.h | 0
.../crc-t10dif.c => lib/crc/x86/crc-t10dif.h | 18 ++---------
.../lib => lib/crc/x86}/crc16-msb-pclmul.S | 0
{arch/x86/lib => lib/crc/x86}/crc32-pclmul.S | 0
arch/x86/lib/crc32.c => lib/crc/x86/crc32.h | 30 ++++---------------
{arch/x86/lib => lib/crc/x86}/crc32c-3way.S | 0
{arch/x86/lib => lib/crc/x86}/crc64-pclmul.S | 0
arch/x86/lib/crc64.c => lib/crc/x86/crc64.h | 21 +++----------
14 files changed, 20 insertions(+), 69 deletions(-)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-consts.h (100%)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.h (100%)
rename arch/x86/lib/crc-t10dif.c => lib/crc/x86/crc-t10dif.h (56%)
rename {arch/x86/lib => lib/crc/x86}/crc16-msb-pclmul.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc32-pclmul.S (100%)
rename arch/x86/lib/crc32.c => lib/crc/x86/crc32.h (76%)
rename {arch/x86/lib => lib/crc/x86}/crc32c-3way.S (100%)
rename {arch/x86/lib => lib/crc/x86}/crc64-pclmul.S (100%)
rename arch/x86/lib/crc64.c => lib/crc/x86/crc64.h (61%)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 340e5468980e0..6181f414f87ec 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -77,13 +77,10 @@ config X86
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_CACHE_LINE_SIZE
select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
select ARCH_HAS_CPU_FINALIZE_INIT
select ARCH_HAS_CPU_PASID if IOMMU_SVA
- select ARCH_HAS_CRC32
- select ARCH_HAS_CRC64 if X86_64
- select ARCH_HAS_CRC_T10DIF
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE if !X86_PAE
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_DMA_OPS if GART_IOMMU || XEN
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index 4fa5c4e1ba8a0..dc5ee2a6938c4 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -38,20 +38,10 @@ lib-$(CONFIG_ARCH_HAS_COPY_MC) += copy_mc.o copy_mc_64.o
lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o insn-eval.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
lib-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_MITIGATION_RETPOLINE) += retpoline.o
-obj-$(CONFIG_CRC32_ARCH) += crc32-x86.o
-crc32-x86-y := crc32.o crc32-pclmul.o
-crc32-x86-$(CONFIG_64BIT) += crc32c-3way.o
-
-obj-$(CONFIG_CRC64_ARCH) += crc64-x86.o
-crc64-x86-y := crc64.o crc64-pclmul.o
-
-obj-$(CONFIG_CRC_T10DIF_ARCH) += crc-t10dif-x86.o
-crc-t10dif-x86-y := crc-t10dif.o crc16-msb-pclmul.o
-
obj-y += msr.o msr-reg.o msr-reg-export.o hweight.o
obj-y += iomem.o
ifeq ($(CONFIG_X86_32),y)
obj-y += atomic64_32.o
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index af4fa857a81f8..de4b2182ae7ff 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -52,10 +52,11 @@ config CRC_T10DIF_ARCH
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64 && KERNEL_MODE_NEON
default y if PPC64 && ALTIVEC
default y if RISCV && RISCV_ISA_ZBC
+ default y if X86
config CRC32
tristate
select BITREVERSE
help
@@ -74,10 +75,11 @@ config CRC32_ARCH
default y if MIPS && CPU_MIPSR6
default y if PPC64 && ALTIVEC
default y if RISCV && RISCV_ISA_ZBC
default y if S390
default y if SPARC64
+ default y if X86
config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
@@ -88,10 +90,11 @@ config ARCH_HAS_CRC64
config CRC64_ARCH
bool
depends on CRC64 && CRC_OPTIMIZATIONS
default y if RISCV && RISCV_ISA_ZBC && 64BIT
+ default y if X86_64
config CRC_OPTIMIZATIONS
bool "Enable optimized CRC implementations" if EXPERT
default y
help
diff --git a/lib/crc/Makefile b/lib/crc/Makefile
index 81e176db0947c..7543ad295ab6f 100644
--- a/lib/crc/Makefile
+++ b/lib/crc/Makefile
@@ -15,10 +15,11 @@ ifeq ($(CONFIG_CRC_T10DIF_ARCH),y)
CFLAGS_crc-t10dif-main.o += -I$(src)/$(SRCARCH)
crc-t10dif-$(CONFIG_ARM) += arm/crc-t10dif-core.o
crc-t10dif-$(CONFIG_ARM64) += arm64/crc-t10dif-core.o
crc-t10dif-$(CONFIG_PPC) += powerpc/crct10dif-vpmsum_asm.o
crc-t10dif-$(CONFIG_RISCV) += riscv/crc16_msb.o
+crc-t10dif-$(CONFIG_X86) += x86/crc16-msb-pclmul.o
endif
obj-$(CONFIG_CRC32) += crc32.o
crc32-y := crc32-main.o
ifeq ($(CONFIG_CRC32_ARCH),y)
@@ -27,17 +28,20 @@ crc32-$(CONFIG_ARM) += arm/crc32-core.o
crc32-$(CONFIG_ARM64) += arm64/crc32-core.o
crc32-$(CONFIG_PPC) += powerpc/crc32c-vpmsum_asm.o
crc32-$(CONFIG_RISCV) += riscv/crc32_lsb.o riscv/crc32_msb.o
crc32-$(CONFIG_S390) += s390/crc32le-vx.o s390/crc32be-vx.o
crc32-$(CONFIG_SPARC) += sparc/crc32c_asm.o
+crc32-$(CONFIG_X86) += x86/crc32-pclmul.o
+crc32-$(CONFIG_X86_64) += x86/crc32c-3way.o
endif
obj-$(CONFIG_CRC64) += crc64.o
crc64-y := crc64-main.o
ifeq ($(CONFIG_CRC64_ARCH),y)
CFLAGS_crc64-main.o += -I$(src)/$(SRCARCH)
crc64-$(CONFIG_RISCV) += riscv/crc64_lsb.o riscv/crc64_msb.o
+crc64-$(CONFIG_X86) += x86/crc64-pclmul.o
endif
obj-y += tests/
hostprogs := gen_crc32table gen_crc64table
diff --git a/arch/x86/lib/crc-pclmul-consts.h b/lib/crc/x86/crc-pclmul-consts.h
similarity index 100%
rename from arch/x86/lib/crc-pclmul-consts.h
rename to lib/crc/x86/crc-pclmul-consts.h
diff --git a/arch/x86/lib/crc-pclmul-template.S b/lib/crc/x86/crc-pclmul-template.S
similarity index 100%
rename from arch/x86/lib/crc-pclmul-template.S
rename to lib/crc/x86/crc-pclmul-template.S
diff --git a/arch/x86/lib/crc-pclmul-template.h b/lib/crc/x86/crc-pclmul-template.h
similarity index 100%
rename from arch/x86/lib/crc-pclmul-template.h
rename to lib/crc/x86/crc-pclmul-template.h
diff --git a/arch/x86/lib/crc-t10dif.c b/lib/crc/x86/crc-t10dif.h
similarity index 56%
rename from arch/x86/lib/crc-t10dif.c
rename to lib/crc/x86/crc-t10dif.h
index db7ce59c31ace..eb1f23db4daa1 100644
--- a/arch/x86/lib/crc-t10dif.c
+++ b/lib/crc/x86/crc-t10dif.h
@@ -3,38 +3,26 @@
* CRC-T10DIF using [V]PCLMULQDQ instructions
*
* Copyright 2024 Google LLC
*/
-#include <linux/crc-t10dif.h>
-#include <linux/module.h>
#include "crc-pclmul-template.h"
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq);
DECLARE_CRC_PCLMUL_FUNCS(crc16_msb, u16);
-u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len)
+static inline u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len)
{
CRC_PCLMUL(crc, p, len, crc16_msb, crc16_msb_0x8bb7_consts,
have_pclmulqdq);
return crc_t10dif_generic(crc, p, len);
}
-EXPORT_SYMBOL(crc_t10dif_arch);
-static int __init crc_t10dif_x86_init(void)
+#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch
+static inline void crc_t10dif_mod_init_arch(void)
{
if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) {
static_branch_enable(&have_pclmulqdq);
INIT_CRC_PCLMUL(crc16_msb);
}
- return 0;
}
-subsys_initcall(crc_t10dif_x86_init);
-
-static void __exit crc_t10dif_x86_exit(void)
-{
-}
-module_exit(crc_t10dif_x86_exit);
-
-MODULE_DESCRIPTION("CRC-T10DIF using [V]PCLMULQDQ instructions");
-MODULE_LICENSE("GPL");
diff --git a/arch/x86/lib/crc16-msb-pclmul.S b/lib/crc/x86/crc16-msb-pclmul.S
similarity index 100%
rename from arch/x86/lib/crc16-msb-pclmul.S
rename to lib/crc/x86/crc16-msb-pclmul.S
diff --git a/arch/x86/lib/crc32-pclmul.S b/lib/crc/x86/crc32-pclmul.S
similarity index 100%
rename from arch/x86/lib/crc32-pclmul.S
rename to lib/crc/x86/crc32-pclmul.S
diff --git a/arch/x86/lib/crc32.c b/lib/crc/x86/crc32.h
similarity index 76%
rename from arch/x86/lib/crc32.c
rename to lib/crc/x86/crc32.h
index d09343e2cea93..28451d5769c3a 100644
--- a/arch/x86/lib/crc32.c
+++ b/lib/crc/x86/crc32.h
@@ -5,26 +5,23 @@
* Copyright (C) 2008 Intel Corporation
* Copyright 2012 Xyratex Technology Limited
* Copyright 2024 Google LLC
*/
-#include <linux/crc32.h>
-#include <linux/module.h>
#include "crc-pclmul-template.h"
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32);
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq);
DECLARE_CRC_PCLMUL_FUNCS(crc32_lsb, u32);
-u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len)
{
CRC_PCLMUL(crc, p, len, crc32_lsb, crc32_lsb_0xedb88320_consts,
have_pclmulqdq);
return crc32_le_base(crc, p, len);
}
-EXPORT_SYMBOL(crc32_le_arch);
#ifdef CONFIG_X86_64
#define CRC32_INST "crc32q %1, %q0"
#else
#define CRC32_INST "crc32l %1, %0"
@@ -36,11 +33,11 @@ EXPORT_SYMBOL(crc32_le_arch);
*/
#define CRC32C_PCLMUL_BREAKEVEN 512
asmlinkage u32 crc32c_x86_3way(u32 crc, const u8 *buffer, size_t len);
-u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
+static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
{
size_t num_longs;
if (!static_branch_likely(&have_crc32))
return crc32c_base(crc, p, len);
@@ -68,44 +65,29 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
if (len & 1)
asm("crc32b %1, %0" : "+r" (crc) : ASM_INPUT_RM (*p));
return crc;
}
-EXPORT_SYMBOL(crc32c_arch);
-u32 crc32_be_arch(u32 crc, const u8 *p, size_t len)
-{
- return crc32_be_base(crc, p, len);
-}
-EXPORT_SYMBOL(crc32_be_arch);
+#define crc32_be_arch crc32_be_base /* not implemented on this arch */
-static int __init crc32_x86_init(void)
+#define crc32_mod_init_arch crc32_mod_init_arch
+static inline void crc32_mod_init_arch(void)
{
if (boot_cpu_has(X86_FEATURE_XMM4_2))
static_branch_enable(&have_crc32);
if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) {
static_branch_enable(&have_pclmulqdq);
INIT_CRC_PCLMUL(crc32_lsb);
}
- return 0;
-}
-subsys_initcall(crc32_x86_init);
-
-static void __exit crc32_x86_exit(void)
-{
}
-module_exit(crc32_x86_exit);
-u32 crc32_optimizations(void)
+static inline u32 crc32_optimizations_arch(void)
{
u32 optimizations = 0;
if (static_key_enabled(&have_crc32))
optimizations |= CRC32C_OPTIMIZATION;
if (static_key_enabled(&have_pclmulqdq))
optimizations |= CRC32_LE_OPTIMIZATION;
return optimizations;
}
-EXPORT_SYMBOL(crc32_optimizations);
-
-MODULE_DESCRIPTION("x86-optimized CRC32 functions");
-MODULE_LICENSE("GPL");
diff --git a/arch/x86/lib/crc32c-3way.S b/lib/crc/x86/crc32c-3way.S
similarity index 100%
rename from arch/x86/lib/crc32c-3way.S
rename to lib/crc/x86/crc32c-3way.S
diff --git a/arch/x86/lib/crc64-pclmul.S b/lib/crc/x86/crc64-pclmul.S
similarity index 100%
rename from arch/x86/lib/crc64-pclmul.S
rename to lib/crc/x86/crc64-pclmul.S
diff --git a/arch/x86/lib/crc64.c b/lib/crc/x86/crc64.h
similarity index 61%
rename from arch/x86/lib/crc64.c
rename to lib/crc/x86/crc64.h
index 351a09f5813e2..54aca3a9475c9 100644
--- a/arch/x86/lib/crc64.c
+++ b/lib/crc/x86/crc64.h
@@ -3,48 +3,35 @@
* CRC64 using [V]PCLMULQDQ instructions
*
* Copyright 2025 Google LLC
*/
-#include <linux/crc64.h>
-#include <linux/module.h>
#include "crc-pclmul-template.h"
static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq);
DECLARE_CRC_PCLMUL_FUNCS(crc64_msb, u64);
DECLARE_CRC_PCLMUL_FUNCS(crc64_lsb, u64);
-u64 crc64_be_arch(u64 crc, const u8 *p, size_t len)
+static inline u64 crc64_be_arch(u64 crc, const u8 *p, size_t len)
{
CRC_PCLMUL(crc, p, len, crc64_msb, crc64_msb_0x42f0e1eba9ea3693_consts,
have_pclmulqdq);
return crc64_be_generic(crc, p, len);
}
-EXPORT_SYMBOL_GPL(crc64_be_arch);
-u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len)
+static inline u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len)
{
CRC_PCLMUL(crc, p, len, crc64_lsb, crc64_lsb_0x9a6c9329ac4bc9b5_consts,
have_pclmulqdq);
return crc64_nvme_generic(crc, p, len);
}
-EXPORT_SYMBOL_GPL(crc64_nvme_arch);
-static int __init crc64_x86_init(void)
+#define crc64_mod_init_arch crc64_mod_init_arch
+static inline void crc64_mod_init_arch(void)
{
if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) {
static_branch_enable(&have_pclmulqdq);
INIT_CRC_PCLMUL(crc64_msb);
INIT_CRC_PCLMUL(crc64_lsb);
}
- return 0;
}
-subsys_initcall(crc64_x86_init);
-
-static void __exit crc64_x86_exit(void)
-{
-}
-module_exit(crc64_x86_exit);
-
-MODULE_DESCRIPTION("CRC64 using [V]PCLMULQDQ instructions");
-MODULE_LICENSE("GPL");
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 12/12] lib/crc: remove ARCH_HAS_* kconfig symbols
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (10 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 11/12] lib/crc/x86: migrate x86-optimized " Eric Biggers
@ 2025-06-07 20:04 ` Eric Biggers
2025-06-07 23:47 ` [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Jason A. Donenfeld
` (5 subsequent siblings)
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-07 20:04 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
From: Eric Biggers <ebiggers@google.com>
These symbols are no longer used, so remove them.
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
lib/crc/Kconfig | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig
index de4b2182ae7ff..5858b3acc6630 100644
--- a/lib/crc/Kconfig
+++ b/lib/crc/Kconfig
@@ -42,13 +42,10 @@ config CRC_T10DIF
tristate
help
The CRC-T10DIF library functions. Select this if your module uses
any of the functions from <linux/crc-t10dif.h>.
-config ARCH_HAS_CRC_T10DIF
- bool
-
config CRC_T10DIF_ARCH
bool
depends on CRC_T10DIF && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64 && KERNEL_MODE_NEON
@@ -61,13 +58,10 @@ config CRC32
select BITREVERSE
help
The CRC32 library functions. Select this if your module uses any of
the functions from <linux/crc32.h> or <linux/crc32c.h>.
-config ARCH_HAS_CRC32
- bool
-
config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
@@ -83,13 +77,10 @@ config CRC64
tristate
help
The CRC64 library functions. Select this if your module uses any of
the functions from <linux/crc64.h>.
-config ARCH_HAS_CRC64
- bool
-
config CRC64_ARCH
bool
depends on CRC64 && CRC_OPTIMIZATIONS
default y if RISCV && RISCV_ISA_ZBC && 64BIT
default y if X86_64
--
2.49.0
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (11 preceding siblings ...)
2025-06-07 20:04 ` [PATCH v2 12/12] lib/crc: remove ARCH_HAS_* kconfig symbols Eric Biggers
@ 2025-06-07 23:47 ` Jason A. Donenfeld
2025-06-08 23:48 ` Eric Biggers
2025-06-09 7:40 ` Ingo Molnar
` (4 subsequent siblings)
17 siblings, 1 reply; 29+ messages in thread
From: Jason A. Donenfeld @ 2025-06-07 23:47 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Linus Torvalds
On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
> Having arch-specific code outside arch/ was somewhat controversial when
> Zinc proposed it back in 2018. But I don't think the concerns are
> warranted. It's better from a technical perspective, as it enables the
> improvements mentioned above. This model is already successfully used
> in other places in the kernel such as lib/raid6/. The community of each
> architecture still remains free to work on the code, even if it's not in
> arch/. At the time there was also a desire to put the library code in
> the same files as the old-school crypto API, but that was a mistake; now
> that the library is separate, that's no longer a constraint either.
I can't express how happy I am to see this revived. It's clearly the
right way forward and makes it a lot simpler for us to dispatch to
various arch implementations and also is organizationally simpler.
Jason
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 23:47 ` [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Jason A. Donenfeld
@ 2025-06-08 23:48 ` Eric Biggers
2025-06-10 17:39 ` Jason A. Donenfeld
0 siblings, 1 reply; 29+ messages in thread
From: Eric Biggers @ 2025-06-08 23:48 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Linus Torvalds
On Sat, Jun 07, 2025 at 05:47:02PM -0600, Jason A. Donenfeld wrote:
> On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
> > Having arch-specific code outside arch/ was somewhat controversial when
> > Zinc proposed it back in 2018. But I don't think the concerns are
> > warranted. It's better from a technical perspective, as it enables the
> > improvements mentioned above. This model is already successfully used
> > in other places in the kernel such as lib/raid6/. The community of each
> > architecture still remains free to work on the code, even if it's not in
> > arch/. At the time there was also a desire to put the library code in
> > the same files as the old-school crypto API, but that was a mistake; now
> > that the library is separate, that's no longer a constraint either.
>
> I can't express how happy I am to see this revived. It's clearly the
> right way forward and makes it a lot simpler for us to dispatch to
> various arch implementations and also is organizationally simpler.
>
> Jason
Thanks! Can I turn that into an Acked-by?
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (12 preceding siblings ...)
2025-06-07 23:47 ` [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Jason A. Donenfeld
@ 2025-06-09 7:40 ` Ingo Molnar
2025-06-09 18:54 ` Eric Biggers
2025-06-09 8:15 ` Julian Calaby
` (3 subsequent siblings)
17 siblings, 1 reply; 29+ messages in thread
From: Ingo Molnar @ 2025-06-09 7:40 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
* Eric Biggers <ebiggers@kernel.org> wrote:
> This series is also available at:
>
> git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
>
> This series improves how lib/crc supports arch-optimized code. First,
> instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> functions (e.g. crc32c_base()) will now be part of a single module for
> each CRC type, allowing better inlining and dead code elimination. The
> second change is made possible by the first.
>
> As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> were already coupled together and always both got loaded together via
> direct symbol dependency, so the separation provided no benefit.
>
> Note: later I'd like to apply the same design to lib/crypto/ too, where
> often the API functions are out-of-line so this will work even better.
> In those cases, for each algorithm we currently have 3 modules all
> coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> sha256-x86.ko. We should have just one, inline things properly, and
> rely on the compiler's dead code elimination to decide the inclusion of
> the generic code instead of manually setting it via kconfig.
>
> Having arch-specific code outside arch/ was somewhat controversial when
> Zinc proposed it back in 2018. But I don't think the concerns are
> warranted. It's better from a technical perspective, as it enables the
> improvements mentioned above. This model is already successfully used
> in other places in the kernel such as lib/raid6/. The community of each
> architecture still remains free to work on the code, even if it's not in
> arch/. At the time there was also a desire to put the library code in
> the same files as the old-school crypto API, but that was a mistake; now
> that the library is separate, that's no longer a constraint either.
>
> Changed in v2:
> - Fixed build warning on architectures without any optimized CRC code
> - Fixed build warning in sparc/crc32.h by removing pr_fmt
> - Moved fallback definitions of crc32*_arch back into arch files
> - Remove ARCH_HAS_CRC* symbols at end of series instead of beginning,
> so that they're not removed until they're no longer being selected
> - Slightly improved some commit messages
> - Rebased onto other pending lib/crc changes
>
> Eric Biggers (12):
> lib/crc: move files into lib/crc/
> lib/crc: prepare for arch-optimized code in subdirs of lib/crc/
> lib/crc/arm: migrate arm-optimized CRC code into lib/crc/
> lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/
> lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/
> lib/crc/mips: migrate mips-optimized CRC code into lib/crc/
> lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/
> lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/
> lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
> lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/
> lib/crc/x86: migrate x86-optimized CRC code into lib/crc/
> lib/crc: remove ARCH_HAS_* kconfig symbols
For the movement of the x86 bits:
Acked-by: Ingo Molnar <mingo@kernel.org>
> rename {arch/s390/lib => lib/crc/s390}/crc32be-vx.c (100%)
> rename {arch/s390/lib => lib/crc/s390}/crc32le-vx.c (100%)
> rename arch/sparc/lib/crc32.c => lib/crc/sparc/crc32.h (60%)
> rename {arch/sparc/lib => lib/crc/sparc}/crc32c_asm.S (100%)
> create mode 100644 lib/crc/tests/Makefile
> rename lib/{ => crc}/tests/crc_kunit.c (100%)
> rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-consts.h (100%)
> rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.S (100%)
> rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.h (100%)
> rename arch/x86/lib/crc-t10dif.c => lib/crc/x86/crc-t10dif.h (56%)
> rename {arch/x86/lib => lib/crc/x86}/crc16-msb-pclmul.S (100%)
> rename {arch/x86/lib => lib/crc/x86}/crc32-pclmul.S (100%)
One small namespace suggestion: wouldn't it be better to move the arch
support code to lib/crc/arch/, instead of lib/crc/? That way any
generic code will stand out better and architecture directories don't
crowd out what is supposed to be generic code.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (13 preceding siblings ...)
2025-06-09 7:40 ` Ingo Molnar
@ 2025-06-09 8:15 ` Julian Calaby
2025-06-09 19:48 ` Eric Biggers
2025-06-09 19:16 ` Martin K. Petersen
` (2 subsequent siblings)
17 siblings, 1 reply; 29+ messages in thread
From: Julian Calaby @ 2025-06-09 8:15 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
Hi Eric,
On Sun, Jun 8, 2025 at 6:07 AM Eric Biggers <ebiggers@kernel.org> wrote:
>
> This series is also available at:
>
> git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
>
> This series improves how lib/crc supports arch-optimized code. First,
> instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> functions (e.g. crc32c_base()) will now be part of a single module for
> each CRC type, allowing better inlining and dead code elimination. The
> second change is made possible by the first.
>
> As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> were already coupled together and always both got loaded together via
> direct symbol dependency, so the separation provided no benefit.
>
> Note: later I'd like to apply the same design to lib/crypto/ too, where
> often the API functions are out-of-line so this will work even better.
> In those cases, for each algorithm we currently have 3 modules all
> coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> sha256-x86.ko. We should have just one, inline things properly, and
> rely on the compiler's dead code elimination to decide the inclusion of
> the generic code instead of manually setting it via kconfig.
>
> Having arch-specific code outside arch/ was somewhat controversial when
> Zinc proposed it back in 2018. But I don't think the concerns are
> warranted. It's better from a technical perspective, as it enables the
> improvements mentioned above. This model is already successfully used
> in other places in the kernel such as lib/raid6/. The community of each
> architecture still remains free to work on the code, even if it's not in
> arch/. At the time there was also a desire to put the library code in
> the same files as the old-school crypto API, but that was a mistake; now
> that the library is separate, that's no longer a constraint either.
Quick question, and apologies if this has been covered elsewhere.
Why not just use choice blocks in Kconfig to choose the compiled-in
crc32 variant instead of this somewhat indirect scheme?
This would keep the dependencies grouped by arch and provide a single
place to choose whether the generic or arch-specific method is used.
It would also allow for alternatives if that ever becomes a thing and
compile testing of the arch-specific variants if that even offers any
actual value.
Thanks,
--
Julian Calaby
Email: julian.calaby@gmail.com
Profile: http://www.google.com/profiles/julian.calaby/
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-09 7:40 ` Ingo Molnar
@ 2025-06-09 18:54 ` Eric Biggers
0 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-09 18:54 UTC (permalink / raw)
To: Ingo Molnar
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Mon, Jun 09, 2025 at 09:40:40AM +0200, Ingo Molnar wrote:
>
> * Eric Biggers <ebiggers@kernel.org> wrote:
>
> > This series is also available at:
> >
> > git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
> >
> > This series improves how lib/crc supports arch-optimized code. First,
> > instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> > will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> > crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> > functions (e.g. crc32c_base()) will now be part of a single module for
> > each CRC type, allowing better inlining and dead code elimination. The
> > second change is made possible by the first.
> >
> > As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> > crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> > were already coupled together and always both got loaded together via
> > direct symbol dependency, so the separation provided no benefit.
> >
> > Note: later I'd like to apply the same design to lib/crypto/ too, where
> > often the API functions are out-of-line so this will work even better.
> > In those cases, for each algorithm we currently have 3 modules all
> > coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> > sha256-x86.ko. We should have just one, inline things properly, and
> > rely on the compiler's dead code elimination to decide the inclusion of
> > the generic code instead of manually setting it via kconfig.
> >
> > Having arch-specific code outside arch/ was somewhat controversial when
> > Zinc proposed it back in 2018. But I don't think the concerns are
> > warranted. It's better from a technical perspective, as it enables the
> > improvements mentioned above. This model is already successfully used
> > in other places in the kernel such as lib/raid6/. The community of each
> > architecture still remains free to work on the code, even if it's not in
> > arch/. At the time there was also a desire to put the library code in
> > the same files as the old-school crypto API, but that was a mistake; now
> > that the library is separate, that's no longer a constraint either.
> >
> > Changed in v2:
> > - Fixed build warning on architectures without any optimized CRC code
> > - Fixed build warning in sparc/crc32.h by removing pr_fmt
> > - Moved fallback definitions of crc32*_arch back into arch files
> > - Remove ARCH_HAS_CRC* symbols at end of series instead of beginning,
> > so that they're not removed until they're no longer being selected
> > - Slightly improved some commit messages
> > - Rebased onto other pending lib/crc changes
> >
> > Eric Biggers (12):
> > lib/crc: move files into lib/crc/
> > lib/crc: prepare for arch-optimized code in subdirs of lib/crc/
> > lib/crc/arm: migrate arm-optimized CRC code into lib/crc/
> > lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/
> > lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/
> > lib/crc/mips: migrate mips-optimized CRC code into lib/crc/
> > lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/
> > lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/
> > lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
> > lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/
> > lib/crc/x86: migrate x86-optimized CRC code into lib/crc/
> > lib/crc: remove ARCH_HAS_* kconfig symbols
>
> For the movement of the x86 bits:
>
> Acked-by: Ingo Molnar <mingo@kernel.org>
>
> > rename {arch/s390/lib => lib/crc/s390}/crc32be-vx.c (100%)
> > rename {arch/s390/lib => lib/crc/s390}/crc32le-vx.c (100%)
> > rename arch/sparc/lib/crc32.c => lib/crc/sparc/crc32.h (60%)
> > rename {arch/sparc/lib => lib/crc/sparc}/crc32c_asm.S (100%)
> > create mode 100644 lib/crc/tests/Makefile
> > rename lib/{ => crc}/tests/crc_kunit.c (100%)
> > rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-consts.h (100%)
> > rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.S (100%)
> > rename {arch/x86/lib => lib/crc/x86}/crc-pclmul-template.h (100%)
> > rename arch/x86/lib/crc-t10dif.c => lib/crc/x86/crc-t10dif.h (56%)
> > rename {arch/x86/lib => lib/crc/x86}/crc16-msb-pclmul.S (100%)
> > rename {arch/x86/lib => lib/crc/x86}/crc32-pclmul.S (100%)
>
> One small namespace suggestion: wouldn't it be better to move the arch
> support code to lib/crc/arch/, instead of lib/crc/? That way any
> generic code will stand out better and architecture directories don't
> crowd out what is supposed to be generic code.
I don't think that yet another level of directories would provide much value
here. The only non-arch subdirectory of lib/crc/ is "tests", so it's not like
there are a lot of subdirectories that could be confused with arch names.
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (14 preceding siblings ...)
2025-06-09 8:15 ` Julian Calaby
@ 2025-06-09 19:16 ` Martin K. Petersen
2025-06-10 18:47 ` Eric Biggers
2025-06-14 12:59 ` Alexander Gordeev
17 siblings, 0 replies; 29+ messages in thread
From: Martin K. Petersen @ 2025-06-09 19:16 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
Eric,
> This series improves how lib/crc supports arch-optimized code. First,
> instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/,
> it will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> functions (e.g. crc32c_base()) will now be part of a single module for
> each CRC type, allowing better inlining and dead code elimination. The
> second change is made possible by the first.
Looks good to me!
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
--
Martin K. Petersen
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-09 8:15 ` Julian Calaby
@ 2025-06-09 19:48 ` Eric Biggers
2025-06-09 22:36 ` Julian Calaby
0 siblings, 1 reply; 29+ messages in thread
From: Eric Biggers @ 2025-06-09 19:48 UTC (permalink / raw)
To: Julian Calaby
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Mon, Jun 09, 2025 at 06:15:24PM +1000, Julian Calaby wrote:
> Hi Eric,
>
> On Sun, Jun 8, 2025 at 6:07 AM Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > This series is also available at:
> >
> > git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
> >
> > This series improves how lib/crc supports arch-optimized code. First,
> > instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> > will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> > crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> > functions (e.g. crc32c_base()) will now be part of a single module for
> > each CRC type, allowing better inlining and dead code elimination. The
> > second change is made possible by the first.
> >
> > As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> > crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> > were already coupled together and always both got loaded together via
> > direct symbol dependency, so the separation provided no benefit.
> >
> > Note: later I'd like to apply the same design to lib/crypto/ too, where
> > often the API functions are out-of-line so this will work even better.
> > In those cases, for each algorithm we currently have 3 modules all
> > coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> > sha256-x86.ko. We should have just one, inline things properly, and
> > rely on the compiler's dead code elimination to decide the inclusion of
> > the generic code instead of manually setting it via kconfig.
> >
> > Having arch-specific code outside arch/ was somewhat controversial when
> > Zinc proposed it back in 2018. But I don't think the concerns are
> > warranted. It's better from a technical perspective, as it enables the
> > improvements mentioned above. This model is already successfully used
> > in other places in the kernel such as lib/raid6/. The community of each
> > architecture still remains free to work on the code, even if it's not in
> > arch/. At the time there was also a desire to put the library code in
> > the same files as the old-school crypto API, but that was a mistake; now
> > that the library is separate, that's no longer a constraint either.
>
> Quick question, and apologies if this has been covered elsewhere.
>
> Why not just use choice blocks in Kconfig to choose the compiled-in
> crc32 variant instead of this somewhat indirect scheme?
>
> This would keep the dependencies grouped by arch and provide a single place to
> choose whether the generic or arch-specific method is used.
It's not clear exactly what you're suggesting, but it sounds like you're
complaining about this:
config CRC32_ARCH
bool
depends on CRC32 && CRC_OPTIMIZATIONS
default y if ARM && KERNEL_MODE_NEON
default y if ARM64
default y if LOONGARCH
default y if MIPS && CPU_MIPSR6
default y if PPC64 && ALTIVEC
default y if RISCV && RISCV_ISA_ZBC
default y if S390
default y if SPARC64
default y if X86
We could instead make each arch be responsible for selecting this from
lib/crc/$(SRCARCH)/Kconfig, which lib/crc/Kconfig would then have to include.
But I don't think the small bit of additional per-arch separation would be worth
the extra complexity here. Something similar applies to lib/crc/Makefile too.
This patchset strikes a balance where the vast majority of the arch-specific CRC
code is isolated in lib/crc/$(SRCARCH), and the exceptions are just
lib/crc/Makefile and lib/crc/Kconfig. I think these exceptions make sense,
given that we're building a single module per CRC variant. We'd have to go
through some hoops to isolate the arch-specific Kconfig and Makefile snippets
into per-arch files, which don't seem worth it here IMO.
> It would also allow for alternatives if that ever becomes a thing and
If you mean one arch with multiple alternative implementations of a particular
CRC variant, that already exists for many of the architectures. They just build
in as many as can be, and the best one is chosen at boot or module load time.
But that's existing behavior, unchanged by this patchset.
> compile testing of the arch-specific variants if that even offers any
> actual value.
They all use instructions specific to the corresponding arch, so I don't think
any of them would be compatible with COMPILE_TEST.
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-09 19:48 ` Eric Biggers
@ 2025-06-09 22:36 ` Julian Calaby
2025-06-09 22:59 ` Eric Biggers
0 siblings, 1 reply; 29+ messages in thread
From: Julian Calaby @ 2025-06-09 22:36 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
Hi Eric,
On Tue, Jun 10, 2025 at 5:49 AM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Mon, Jun 09, 2025 at 06:15:24PM +1000, Julian Calaby wrote:
> > Hi Eric,
> >
> > On Sun, Jun 8, 2025 at 6:07 AM Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > This series is also available at:
> > >
> > > git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
> > >
> > > This series improves how lib/crc supports arch-optimized code. First,
> > > instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> > > will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> > > crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> > > functions (e.g. crc32c_base()) will now be part of a single module for
> > > each CRC type, allowing better inlining and dead code elimination. The
> > > second change is made possible by the first.
> > >
> > > As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> > > crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> > > were already coupled together and always both got loaded together via
> > > direct symbol dependency, so the separation provided no benefit.
> > >
> > > Note: later I'd like to apply the same design to lib/crypto/ too, where
> > > often the API functions are out-of-line so this will work even better.
> > > In those cases, for each algorithm we currently have 3 modules all
> > > coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> > > sha256-x86.ko. We should have just one, inline things properly, and
> > > rely on the compiler's dead code elimination to decide the inclusion of
> > > the generic code instead of manually setting it via kconfig.
> > >
> > > Having arch-specific code outside arch/ was somewhat controversial when
> > > Zinc proposed it back in 2018. But I don't think the concerns are
> > > warranted. It's better from a technical perspective, as it enables the
> > > improvements mentioned above. This model is already successfully used
> > > in other places in the kernel such as lib/raid6/. The community of each
> > > architecture still remains free to work on the code, even if it's not in
> > > arch/. At the time there was also a desire to put the library code in
> > > the same files as the old-school crypto API, but that was a mistake; now
> > > that the library is separate, that's no longer a constraint either.
> >
> > Quick question, and apologies if this has been covered elsewhere.
> >
> > Why not just use choice blocks in Kconfig to choose the compiled-in
> > crc32 variant instead of this somewhat indirect scheme?
> >
> > This would keep the dependencies grouped by arch and provide a single place to
> > choose whether the generic or arch-specific method is used.
>
> It's not clear exactly what you're suggesting, but it sounds like you're
> complaining about this:
>
> config CRC32_ARCH
> bool
> depends on CRC32 && CRC_OPTIMIZATIONS
> default y if ARM && KERNEL_MODE_NEON
> default y if ARM64
> default y if LOONGARCH
> default y if MIPS && CPU_MIPSR6
> default y if PPC64 && ALTIVEC
> default y if RISCV && RISCV_ISA_ZBC
> default y if S390
> default y if SPARC64
> default y if X86
I was suggesting something roughly like:
choice
prompt "CRC32 Variant"
depends on CRC32 && CRC_OPTIMIZATIONS
config CRC32_ARCH_ARM_NEON
bool "ARM NEON"
default y
depends ARM && KERNEL_MODE_NEON
...
config CRC32_GENERIC
bool "Generic"
endchoice
> This patchset strikes a balance where the vast majority of the arch-specific CRC
> code is isolated in lib/crc/$(SRCARCH), and the exceptions are just
> lib/crc/Makefile and lib/crc/Kconfig. I think these exceptions make sense,
> given that we're building a single module per CRC variant. We'd have to go
> through some hoops to isolate the arch-specific Kconfig and Makefile snippets
> into per-arch files, which don't seem worth it here IMO.
I was only really concerned with the Kconfig structure, I was
expecting Kbuild to look roughly like this: (filenames are wrong)
crc32-y += crc32-base.o
crc32-$(CRC32_ARCH_ARM_NEON) += arch/arm/crc32-neon.o
...
crc32-$(CRC32_GENERIC) += crc32-generic.o
but yeah, your proposal here has grown on me now that I think about it
and the only real "benefit" mine has is that architectures can display
choices for variants that have Kconfig-visible requirements, which
probably isn't that many so it wouldn't be useful in practice.
Thanks for answering my question,
--
Julian Calaby
Email: julian.calaby@gmail.com
Profile: http://www.google.com/profiles/julian.calaby/
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-09 22:36 ` Julian Calaby
@ 2025-06-09 22:59 ` Eric Biggers
0 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-09 22:59 UTC (permalink / raw)
To: Julian Calaby
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Tue, Jun 10, 2025 at 08:36:39AM +1000, Julian Calaby wrote:
> Hi Eric,
>
> On Tue, Jun 10, 2025 at 5:49 AM Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Mon, Jun 09, 2025 at 06:15:24PM +1000, Julian Calaby wrote:
> > > Hi Eric,
> > >
> > > On Sun, Jun 8, 2025 at 6:07 AM Eric Biggers <ebiggers@kernel.org> wrote:
> > > >
> > > > This series is also available at:
> > > >
> > > > git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
> > > >
> > > > This series improves how lib/crc supports arch-optimized code. First,
> > > > instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> > > > will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> > > > crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> > > > functions (e.g. crc32c_base()) will now be part of a single module for
> > > > each CRC type, allowing better inlining and dead code elimination. The
> > > > second change is made possible by the first.
> > > >
> > > > As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> > > > crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> > > > were already coupled together and always both got loaded together via
> > > > direct symbol dependency, so the separation provided no benefit.
> > > >
> > > > Note: later I'd like to apply the same design to lib/crypto/ too, where
> > > > often the API functions are out-of-line so this will work even better.
> > > > In those cases, for each algorithm we currently have 3 modules all
> > > > coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> > > > sha256-x86.ko. We should have just one, inline things properly, and
> > > > rely on the compiler's dead code elimination to decide the inclusion of
> > > > the generic code instead of manually setting it via kconfig.
> > > >
> > > > Having arch-specific code outside arch/ was somewhat controversial when
> > > > Zinc proposed it back in 2018. But I don't think the concerns are
> > > > warranted. It's better from a technical perspective, as it enables the
> > > > improvements mentioned above. This model is already successfully used
> > > > in other places in the kernel such as lib/raid6/. The community of each
> > > > architecture still remains free to work on the code, even if it's not in
> > > > arch/. At the time there was also a desire to put the library code in
> > > > the same files as the old-school crypto API, but that was a mistake; now
> > > > that the library is separate, that's no longer a constraint either.
> > >
> > > Quick question, and apologies if this has been covered elsewhere.
> > >
> > > Why not just use choice blocks in Kconfig to choose the compiled-in
> > > crc32 variant instead of this somewhat indirect scheme?
> > >
> > > This would keep the dependencies grouped by arch and provide a single place to
> > > choose whether the generic or arch-specific method is used.
> >
> > It's not clear exactly what you're suggesting, but it sounds like you're
> > complaining about this:
> >
> > config CRC32_ARCH
> > bool
> > depends on CRC32 && CRC_OPTIMIZATIONS
> > default y if ARM && KERNEL_MODE_NEON
> > default y if ARM64
> > default y if LOONGARCH
> > default y if MIPS && CPU_MIPSR6
> > default y if PPC64 && ALTIVEC
> > default y if RISCV && RISCV_ISA_ZBC
> > default y if S390
> > default y if SPARC64
> > default y if X86
>
> I was suggesting something roughly like:
>
> choice
> prompt "CRC32 Variant"
> depends on CRC32 && CRC_OPTIMIZATIONS
>
> config CRC32_ARCH_ARM_NEON
> bool "ARM NEON"
> default y
> depends ARM && KERNEL_MODE_NEON
>
> ...
>
> config CRC32_GENERIC
> bool "Generic"
>
> endchoice
>
> > This patchset strikes a balance where the vast majority of the arch-specific CRC
> > code is isolated in lib/crc/$(SRCARCH), and the exceptions are just
> > lib/crc/Makefile and lib/crc/Kconfig. I think these exceptions make sense,
> > given that we're building a single module per CRC variant. We'd have to go
> > through some hoops to isolate the arch-specific Kconfig and Makefile snippets
> > into per-arch files, which don't seem worth it here IMO.
>
> I was only really concerned with the Kconfig structure, I was
> expecting Kbuild to look roughly like this: (filenames are wrong)
>
> crc32-y += crc32-base.o
> crc32-$(CRC32_ARCH_ARM_NEON) += arch/arm/crc32-neon.o
> ...
> crc32-$(CRC32_GENERIC) += crc32-generic.o
>
> but yeah, your proposal here has grown on me now that I think about it
> and the only real "benefit" mine has is that architectures can display
> choices for variants that have Kconfig-visible requirements, which
> probably isn't that many so it wouldn't be useful in practice.
>
> Thanks for answering my question,
The CRC32 implementation did used to be user-selectable, but that was already
removed in v6.14 (except for the coarse-grained knob CONFIG_CRC_OPTIMIZATIONS
that remains and can be disabled only when CONFIG_EXPERT=y) since the vast
majority of users simply want the optimized CRC32 code enabled. The fact that
it wasn't just enabled by default was a longstanding bug.
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-08 23:48 ` Eric Biggers
@ 2025-06-10 17:39 ` Jason A. Donenfeld
2025-06-10 19:12 ` Eric Biggers
0 siblings, 1 reply; 29+ messages in thread
From: Jason A. Donenfeld @ 2025-06-10 17:39 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Linus Torvalds
On Sun, Jun 08, 2025 at 04:48:17PM -0700, Eric Biggers wrote:
> On Sat, Jun 07, 2025 at 05:47:02PM -0600, Jason A. Donenfeld wrote:
> > On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
> > > Having arch-specific code outside arch/ was somewhat controversial when
> > > Zinc proposed it back in 2018. But I don't think the concerns are
> > > warranted. It's better from a technical perspective, as it enables the
> > > improvements mentioned above. This model is already successfully used
> > > in other places in the kernel such as lib/raid6/. The community of each
> > > architecture still remains free to work on the code, even if it's not in
> > > arch/. At the time there was also a desire to put the library code in
> > > the same files as the old-school crypto API, but that was a mistake; now
> > > that the library is separate, that's no longer a constraint either.
> >
> > I can't express how happy I am to see this revived. It's clearly the
> > right way forward and makes it a lot simpler for us to dispatch to
> > various arch implementations and also is organizationally simpler.
> >
> > Jason
>
> Thanks! Can I turn that into an Acked-by?
Took me a little while longer to fully review it. Sure,
Acked-by: Jason A. Donenfeld <Jason@zx2c4.com>
Side note: I wonder about eventually turning some of the static branches
into static calls.
Jason
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (15 preceding siblings ...)
2025-06-09 19:16 ` Martin K. Petersen
@ 2025-06-10 18:47 ` Eric Biggers
2025-06-14 12:59 ` Alexander Gordeev
17 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-10 18:47 UTC (permalink / raw)
To: linux-kernel
Cc: linux-crypto, linux-arm-kernel, loongarch, linux-mips,
linuxppc-dev, linux-riscv, linux-s390, sparclinux, x86,
linux-arch, Ard Biesheuvel, Jason A . Donenfeld, Linus Torvalds
On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
> This series is also available at:
>
> git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
>
> This series improves how lib/crc supports arch-optimized code. First,
> instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it
> will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g.
> crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic
> functions (e.g. crc32c_base()) will now be part of a single module for
> each CRC type, allowing better inlining and dead code elimination. The
> second change is made possible by the first.
>
> As an example, consider CONFIG_CRC32=m on x86. We'll now have just
> crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules
> were already coupled together and always both got loaded together via
> direct symbol dependency, so the separation provided no benefit.
>
> Note: later I'd like to apply the same design to lib/crypto/ too, where
> often the API functions are out-of-line so this will work even better.
> In those cases, for each algorithm we currently have 3 modules all
> coupled together, e.g. libsha256.ko, libsha256-generic.ko, and
> sha256-x86.ko. We should have just one, inline things properly, and
> rely on the compiler's dead code elimination to decide the inclusion of
> the generic code instead of manually setting it via kconfig.
>
> Having arch-specific code outside arch/ was somewhat controversial when
> Zinc proposed it back in 2018. But I don't think the concerns are
> warranted. It's better from a technical perspective, as it enables the
> improvements mentioned above. This model is already successfully used
> in other places in the kernel such as lib/raid6/. The community of each
> architecture still remains free to work on the code, even if it's not in
> arch/. At the time there was also a desire to put the library code in
> the same files as the old-school crypto API, but that was a mistake; now
> that the library is separate, that's no longer a constraint either.
>
> Changed in v2:
> - Fixed build warning on architectures without any optimized CRC code
> - Fixed build warning in sparc/crc32.h by removing pr_fmt
> - Moved fallback definitions of crc32*_arch back into arch files
> - Remove ARCH_HAS_CRC* symbols at end of series instead of beginning,
> so that they're not removed until they're no longer being selected
> - Slightly improved some commit messages
> - Rebased onto other pending lib/crc changes
>
> Eric Biggers (12):
> lib/crc: move files into lib/crc/
> lib/crc: prepare for arch-optimized code in subdirs of lib/crc/
> lib/crc/arm: migrate arm-optimized CRC code into lib/crc/
> lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/
> lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/
> lib/crc/mips: migrate mips-optimized CRC code into lib/crc/
> lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/
> lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/
> lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
> lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/
> lib/crc/x86: migrate x86-optimized CRC code into lib/crc/
> lib/crc: remove ARCH_HAS_* kconfig symbols
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=crc-next
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-10 17:39 ` Jason A. Donenfeld
@ 2025-06-10 19:12 ` Eric Biggers
0 siblings, 0 replies; 29+ messages in thread
From: Eric Biggers @ 2025-06-10 19:12 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Linus Torvalds
On Tue, Jun 10, 2025 at 11:39:22AM -0600, Jason A. Donenfeld wrote:
> On Sun, Jun 08, 2025 at 04:48:17PM -0700, Eric Biggers wrote:
> > On Sat, Jun 07, 2025 at 05:47:02PM -0600, Jason A. Donenfeld wrote:
> > > On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
> > > > Having arch-specific code outside arch/ was somewhat controversial when
> > > > Zinc proposed it back in 2018. But I don't think the concerns are
> > > > warranted. It's better from a technical perspective, as it enables the
> > > > improvements mentioned above. This model is already successfully used
> > > > in other places in the kernel such as lib/raid6/. The community of each
> > > > architecture still remains free to work on the code, even if it's not in
> > > > arch/. At the time there was also a desire to put the library code in
> > > > the same files as the old-school crypto API, but that was a mistake; now
> > > > that the library is separate, that's no longer a constraint either.
> > >
> > > I can't express how happy I am to see this revived. It's clearly the
> > > right way forward and makes it a lot simpler for us to dispatch to
> > > various arch implementations and also is organizationally simpler.
> > >
> > > Jason
> >
> > Thanks! Can I turn that into an Acked-by?
>
> Took me a little while longer to fully review it. Sure,
>
> Acked-by: Jason A. Donenfeld <Jason@zx2c4.com>
>
> Side note: I wonder about eventually turning some of the static branches
> into static calls.
Yes, Linus was wondering the same thing earlier. It does run into a couple
issues. First, only x86 and powerpc implement static_call properly; everywhere
else it's just an indirect call. Second, there's often some code shared above
the level at which we'd like to do the dispatch. For example, consider crc32_le
on x86. If we expand the CRC_PCLMUL macro and inline crc32_le_arch and
crc32_le_base as the compiler does, crc32_le ends up as:
u32 crc32_le(u32 crc, const u8 *p, size_t len)
{
if (len >= 16 && static_branch_likely(&have_pclmulqdq) &&
crypto_simd_usable()) {
const void *consts_ptr;
consts_ptr = crc32_lsb_0xedb88320_consts.fold_across_128_bits_consts;
kernel_fpu_begin();
crc = static_call(crc32_lsb_pclmul)(crc, p, len, consts_ptr);
kernel_fpu_end();
return crc;
}
while (len--)
crc = (crc >> 8) ^ crc32table_le[(crc & 255) ^ *p++];
return crc;
}
The existing static_call selects between 3 different assembly functions, all of
which require a kernel-mode FPU section and only support len >= 16.
We could instead unconditionally do a static_call upon entry to the function,
with 4 possible targets. But then we'd have to duplicate the kernel FPU
begin/end sequence in 3 different functions. Also, it would add an extra
function call for the case where 'len < 16', which is a common case and is
exactly the case where per-call overhead matters the most.
However, if we could actually inline the static call into the *callers* of
crc32_le(), that would make it more worthwhile. I'm not sure that's possible,
though, especially considering that this code is tristate.
Anyway, this is tangential to this patchset. Though the new way the code is
organized does make it more feasible to have e.g. a centralized static_call in
the future if we choose to go in that direction.
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
2025-06-07 20:04 ` [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized " Eric Biggers
@ 2025-06-13 16:01 ` Alexander Gordeev
2025-06-13 17:11 ` Eric Biggers
0 siblings, 1 reply; 29+ messages in thread
From: Alexander Gordeev @ 2025-06-13 16:01 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Sat, Jun 07, 2025 at 01:04:51PM -0700, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
>
> Move the s390-optimized CRC code from arch/s390/lib/crc* into its new
> location in lib/crc/s390/, and wire it up in the new way. This new way
> of organizing the CRC code eliminates the need to artificially split the
> code for each CRC variant into separate arch and generic modules,
> enabling better inlining and dead code elimination. For more details,
> see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
...
Hi Eric,
With this series I am getting on s390:
alg: hash: skipping comparison tests for crc32c-s390 because crc32c-generic is unavailable
Thanks!
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
2025-06-13 16:01 ` Alexander Gordeev
@ 2025-06-13 17:11 ` Eric Biggers
2025-06-14 13:24 ` Alexander Gordeev
0 siblings, 1 reply; 29+ messages in thread
From: Eric Biggers @ 2025-06-13 17:11 UTC (permalink / raw)
To: Alexander Gordeev
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Fri, Jun 13, 2025 at 06:01:41PM +0200, Alexander Gordeev wrote:
> On Sat, Jun 07, 2025 at 01:04:51PM -0700, Eric Biggers wrote:
> > From: Eric Biggers <ebiggers@google.com>
> >
> > Move the s390-optimized CRC code from arch/s390/lib/crc* into its new
> > location in lib/crc/s390/, and wire it up in the new way. This new way
> > of organizing the CRC code eliminates the need to artificially split the
> > code for each CRC variant into separate arch and generic modules,
> > enabling better inlining and dead code elimination. For more details,
> > see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/".
> >
> > Signed-off-by: Eric Biggers <ebiggers@google.com>
> ...
>
> Hi Eric,
>
> With this series I am getting on s390:
>
> alg: hash: skipping comparison tests for crc32c-s390 because crc32c-generic is unavailable
>
> Thanks!
I think that's actually from "crypto/crc32c: register only one shash_alg"
(https://lore.kernel.org/linux-crypto/20250601224441.778374-3-ebiggers@kernel.org/),
not the patch you replied to.
Those self-test warnings are expected. But I guess they are going to confuse
people, so we should do something to make them go away.
I think we should do what I've proposed for SHA-512: stop worrying about setting
the cra_driver_name to something meaningful (which has never really worked
anyway), instead just use *-lib, and update crypto/testmgr.c accordingly.
I'll send out patches that do that.
- Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
` (16 preceding siblings ...)
2025-06-10 18:47 ` Eric Biggers
@ 2025-06-14 12:59 ` Alexander Gordeev
17 siblings, 0 replies; 29+ messages in thread
From: Alexander Gordeev @ 2025-06-14 12:59 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Sat, Jun 07, 2025 at 01:04:42PM -0700, Eric Biggers wrote:
Hi Eric,
> This series is also available at:
>
> git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git lib-crc-arch-v2
...
I tried git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git crc-next
With s390 debug_defconfig It causes this:
alg: hash: skipping comparison tests for crc32c-s390 because crc32c-generic is unavailable
No other alg complains. Is it expected?
Thanks!
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized CRC code into lib/crc/
2025-06-13 17:11 ` Eric Biggers
@ 2025-06-14 13:24 ` Alexander Gordeev
0 siblings, 0 replies; 29+ messages in thread
From: Alexander Gordeev @ 2025-06-14 13:24 UTC (permalink / raw)
To: Eric Biggers
Cc: linux-kernel, linux-crypto, linux-arm-kernel, loongarch,
linux-mips, linuxppc-dev, linux-riscv, linux-s390, sparclinux,
x86, linux-arch, Ard Biesheuvel, Jason A . Donenfeld,
Linus Torvalds
On Fri, Jun 13, 2025 at 10:11:43AM -0700, Eric Biggers wrote:
> > Hi Eric,
> >
> > With this series I am getting on s390:
> >
> > alg: hash: skipping comparison tests for crc32c-s390 because crc32c-generic is unavailable
> >
> > Thanks!
>
> I think that's actually from "crypto/crc32c: register only one shash_alg"
> (https://lore.kernel.org/linux-crypto/20250601224441.778374-3-ebiggers@kernel.org/),
> not the patch you replied to.
>
> Those self-test warnings are expected. But I guess they are going to confuse
> people, so we should do something to make them go away.
>
> I think we should do what I've proposed for SHA-512: stop worrying about setting
> the cra_driver_name to something meaningful (which has never really worked
> anyway), instead just use *-lib, and update crypto/testmgr.c accordingly.
>
> I'll send out patches that do that.
Thanks, Eric!
Please, ignore my other email - I though I did not send this one.
> - Eric
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2025-06-14 13:24 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-07 20:04 [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Eric Biggers
2025-06-07 20:04 ` [PATCH v2 01/12] lib/crc: move files into lib/crc/ Eric Biggers
2025-06-07 20:04 ` [PATCH v2 02/12] lib/crc: prepare for arch-optimized code in subdirs of lib/crc/ Eric Biggers
2025-06-07 20:04 ` [PATCH v2 03/12] lib/crc/arm: migrate arm-optimized CRC code into lib/crc/ Eric Biggers
2025-06-07 20:04 ` [PATCH v2 04/12] lib/crc/arm64: migrate arm64-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 05/12] lib/crc/loongarch: migrate loongarch-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 06/12] lib/crc/mips: migrate mips-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 07/12] lib/crc/powerpc: migrate powerpc-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 08/12] lib/crc/riscv: migrate riscv-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized " Eric Biggers
2025-06-13 16:01 ` Alexander Gordeev
2025-06-13 17:11 ` Eric Biggers
2025-06-14 13:24 ` Alexander Gordeev
2025-06-07 20:04 ` [PATCH v2 10/12] lib/crc/sparc: migrate sparc-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 11/12] lib/crc/x86: migrate x86-optimized " Eric Biggers
2025-06-07 20:04 ` [PATCH v2 12/12] lib/crc: remove ARCH_HAS_* kconfig symbols Eric Biggers
2025-06-07 23:47 ` [PATCH v2 00/12] lib/crc: improve how arch-optimized code is integrated Jason A. Donenfeld
2025-06-08 23:48 ` Eric Biggers
2025-06-10 17:39 ` Jason A. Donenfeld
2025-06-10 19:12 ` Eric Biggers
2025-06-09 7:40 ` Ingo Molnar
2025-06-09 18:54 ` Eric Biggers
2025-06-09 8:15 ` Julian Calaby
2025-06-09 19:48 ` Eric Biggers
2025-06-09 22:36 ` Julian Calaby
2025-06-09 22:59 ` Eric Biggers
2025-06-09 19:16 ` Martin K. Petersen
2025-06-10 18:47 ` Eric Biggers
2025-06-14 12:59 ` Alexander Gordeev
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).