Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration

[Petr Pavlu]

+Cc: kbuild, modules

On 5/25/26 9:03 AM, Wentao Guan wrote:
> Use hardware 'crc32b' to partial_crc32_one() when support,
> it shows 2x speed up than crctab32 way.
> 
> I think it will be scaled to more architecture.

Could you please provide measurements showing how this improves the
runtime of genksyms and affects the overall kernel build time? Are
CRC calculations actually a hot path in genksyms? I don't think it is
worth complicating the genksyms code with architecture-specific
optimizations unless there is a clear benefit.

Additionally, many distributions already switched to gendwarfksyms last
year. Unless someone provides a good reason to keep genksyms around,
I expect it will be deprecated soon and eventually removed. It would be
more valuable to show whether the same optimization is worthwhile for
gendwarfksyms.

-- 
Thanks,
Petr

> 
> Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
> ---
> changelog:
> 1. remove change in partial_crc32(),
> for partial_crc32_one() already use crc32b.
> 
> ---
> ---
>  scripts/genksyms/genksyms.c | 34 ++++++++++++++++++++++++++++++++++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
> index 83e48670c2fcf..80b7797c842aa 100644
> --- a/scripts/genksyms/genksyms.c
> +++ b/scripts/genksyms/genksyms.c
> @@ -116,8 +116,40 @@ static const uint32_t crctab32[] = {
>  	0x2d02ef8dU
>  };
>  
> +/*
> + * Architecture-specific CRC32 hardware acceleration.
> + */
> +static int crc32_hw_available;
> +
> +#ifdef __aarch64__
> +#include <sys/auxv.h>
> +#include <asm/hwcap.h>
> +
> +static void crc32_check_hw(void)
> +{
> +	crc32_hw_available = (getauxval(AT_HWCAP) & HWCAP_CRC32) != 0;
> +}
> +
> +static inline uint32_t crc32_hw_byte(uint8_t c, uint32_t crc)
> +{
> +	asm volatile(".arch_extension crc\n\t"
> +				"crc32b %w0, %w0, %w1" : "+r"(crc) : "r"(c));
> +	return crc;
> +}
> +
> +#else
> +static void crc32_check_hw(void)
> +{
> +	crc32_hw_available = 0;
> +}
> +#endif
> +
>  static uint32_t partial_crc32_one(uint8_t c, uint32_t crc)
>  {
> +#if defined(__aarch64__)
> +	if (__builtin_expect(crc32_hw_available, 0))
> +		return crc32_hw_byte(c, crc);
> +#endif
>  	return crctab32[(crc ^ c) & 0xff] ^ (crc >> 8);
>  }
>  
> @@ -740,6 +772,8 @@ int main(int argc, char **argv)
>  	FILE *dumpfile = NULL, *ref_file = NULL;
>  	int o;
>  
> +	crc32_check_hw();
> +
>  	struct option long_opts[] = {
>  		{"debug", 0, 0, 'd'},
>  		{"warnings", 0, 0, 'w'},

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Wentao Guan]

Hello Petr,

Thanks for your reply,

> Could you please provide measurements showing how this improves the
> runtime of genksyms and affects the overall kernel build time? Are
From my build kernel workload, genksyms perf report (record -g -F 99) show:
without patch:
-  7.83% 0.00% genksyms genksyms [.] _start
   _start
 - __libc_start_main
   + 3.19% yylex
     1.81% find_symbol
     0.92% is_reserved_word
     0.62% free_list
   + 0.56% export_symbol

with patch:(find_symbol() and export_symbol() benifit from this):
-  7.12% 0.00% genksyms genksyms [.] _start
   _start
 - __libc_start_main
   - 3.01% yylex
       0.51% malloc
     1.65% find_symbol
     0.82% is_reserved_word
     0.50% free_list

> CRC calculations actually a hot path in genksyms? I don't think it is
> worth complicating the genksyms code with architecture-specific
> optimizations unless there is a clear benefit.
Right.

> Additionally, many distributions already switched to gendwarfksyms last
> year. Unless someone provides a good reason to keep genksyms around,
> I expect it will be deprecated soon and eventually removed. It would be
> more valuable to show whether the same optimization is worthwhile for
> gendwarfksyms.
I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.


BRs
Wentao Guan

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Petr Pavlu]

On 5/25/26 10:02 AM, Wentao Guan wrote:
>> Additionally, many distributions already switched to gendwarfksyms last
>> year. Unless someone provides a good reason to keep genksyms around,
>> I expect it will be deprecated soon and eventually removed. It would be
>> more valuable to show whether the same optimization is worthwhile for
>> gendwarfksyms.
> I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.

Ah, I forgot about this, so gendwarfksyms should already be well
optimized. I think that is the most important thing.

-- 
Thanks,
Petr

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Wentao Guan]

>On 5/25/26 10:02 AM, Wentao Guan wrote:
>>> Additionally, many distributions already switched to gendwarfksyms last
>>> year. Unless someone provides a good reason to keep genksyms around,
>>> I expect it will be deprecated soon and eventually removed. It would be
>>> more valuable to show whether the same optimization is worthwhile for
>>> gendwarfksyms.
>> I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.
>
> Ah, I forgot about this, so gendwarfksyms should already be well
> optimized. I think that is the most important thing.

Which way do you prefer? There are two ways now:
1. this work,
2. import zlib like gendwarfksyms and remove the huge crctab,
but need make sure any arch do not cause performance regression?
(Assert the user libs precompile with +crc32 is right?)
I do some early tests show that:
1) zlib in x86 fast than genksym software
2) zlib in arm64 debian precompile slow than genksym software(crc32 not in armv8)
3) zlib in arm64 recompile with +crc32 same speed with genksyms hardware

BRs
Wentao Guan

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Petr Pavlu]

On 5/25/26 3:41 PM, Wentao Guan wrote:
>> On 5/25/26 10:02 AM, Wentao Guan wrote:
>>>> Additionally, many distributions already switched to gendwarfksyms last
>>>> year. Unless someone provides a good reason to keep genksyms around,
>>>> I expect it will be deprecated soon and eventually removed. It would be
>>>> more valuable to show whether the same optimization is worthwhile for
>>>> gendwarfksyms.
>>> I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.
>>
>> Ah, I forgot about this, so gendwarfksyms should already be well
>> optimized. I think that is the most important thing.
> 
> Which way do you prefer? There are two ways now:
> 1. this work,
> 2. import zlib like gendwarfksyms and remove the huge crctab,
> but need make sure any arch do not cause performance regression?
> (Assert the user libs precompile with +crc32 is right?)
> I do some early tests show that:
> 1) zlib in x86 fast than genksym software
> 2) zlib in arm64 debian precompile slow than genksym software(crc32 not in armv8)
> 3) zlib in arm64 recompile with +crc32 same speed with genksyms hardware

I think it would be useful to first have more data showing that the CRC
calculation in genksyms is a hot path and contributes significantly to
the overall kernel build time.

-- 
Thanks,
Petr

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Wentao Guan]

Hello Petr,

> On 5/25/26 3:41 PM, Wentao Guan wrote:
>>> On 5/25/26 10:02 AM, Wentao Guan wrote:
>>>>> Additionally, many distributions already switched to gendwarfksyms last
>>>>> year. Unless someone provides a good reason to keep genksyms around,
>>>>> I expect it will be deprecated soon and eventually removed. It would be
>>>>> more valuable to show whether the same optimization is worthwhile for
>>>>> gendwarfksyms.
>>>> I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.
>>>
>>> Ah, I forgot about this, so gendwarfksyms should already be well
>>> optimized. I think that is the most important thing.
>>
>> Which way do you prefer? There are two ways now:
>> 1. this work,
>> 2. import zlib like gendwarfksyms and remove the huge crctab,
>> but need make sure any arch do not cause performance regression?
>> (Assert the user libs precompile with +crc32 is right?)
>> I do some early tests show that:
>> 1) zlib in x86 fast than genksym software
>> 2) zlib in arm64 debian precompile slow than genksym software(crc32 not in armv8)
>> 3) zlib in arm64 recompile with +crc32 same speed with genksyms hardware
> 
> I think it would be useful to first have more data showing that the CRC
> calculation in genksyms is a hot path and contributes significantly to
> the overall kernel build time.
You are right.
1. genksyms contributes significantly to the overall kernel build time.
Test with arm64 defconfig and enable/disable CONFIG_GENKSYMS (11m50s/9m48s)
2. crc32 shows ~10% affect in perf record w/o patch.
3. but this patch w/o doesn`t affect the overall kernel build time 11m50s(+-1s).
4. so try to use zlib and delete the hard code crctab32 table?

BRs
Wentao Guan

RFC PATCH:
From f8f1c2301ddb70fb42a56fd7228db3dbd248e2b4 Mon Sep 17 00:00:00 2001
From: Wentao Guan <guanwentao@uniontech.com>
Date: Mon, 25 May 2026 20:02:03 +0800
Subject: [PATCH] genksyms: use crc32 from zlib

Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
---
 scripts/genksyms/Makefile   |  1 +
 scripts/genksyms/genksyms.c | 70 +++++--------------------------------
 2 files changed, 9 insertions(+), 62 deletions(-)

diff --git a/scripts/genksyms/Makefile b/scripts/genksyms/Makefile
index 4350311fb7b39..068e79626ffe7 100644
--- a/scripts/genksyms/Makefile
+++ b/scripts/genksyms/Makefile
@@ -7,6 +7,7 @@ genksyms-objs   := genksyms.o parse.tab.o lex.lex.o
 # -I needed for generated C source to include headers in source tree
 HOSTCFLAGS_parse.tab.o := -I $(src)
 HOSTCFLAGS_lex.lex.o := -I $(src)
+HOSTLDLIBS_genksyms :=  -lz
 
 # dependencies on generated files need to be listed explicitly
 $(obj)/lex.lex.o: $(obj)/parse.tab.h
diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
index 83e48670c2fcf..40d25dd898874 100644
--- a/scripts/genksyms/genksyms.c
+++ b/scripts/genksyms/genksyms.c
@@ -20,6 +20,7 @@
 #include <getopt.h>
 
 #include <hashtable.h>
+#include <zlib.h>
 
 #include "genksyms.h"
 /*----------------------------------------------------------------------*/
@@ -61,78 +62,23 @@ static void print_type_name(enum symbol_type type, const char *name);
 
 /*----------------------------------------------------------------------*/
 
-static const uint32_t crctab32[] = {
-       0x00000000U, 0x77073096U, 0xee0e612cU, 0x990951baU, 0x076dc419U,
-       0x706af48fU, 0xe963a535U, 0x9e6495a3U, 0x0edb8832U, 0x79dcb8a4U,
-       0xe0d5e91eU, 0x97d2d988U, 0x09b64c2bU, 0x7eb17cbdU, 0xe7b82d07U,
-       0x90bf1d91U, 0x1db71064U, 0x6ab020f2U, 0xf3b97148U, 0x84be41deU,
-       0x1adad47dU, 0x6ddde4ebU, 0xf4d4b551U, 0x83d385c7U, 0x136c9856U,
-       0x646ba8c0U, 0xfd62f97aU, 0x8a65c9ecU, 0x14015c4fU, 0x63066cd9U,
-       0xfa0f3d63U, 0x8d080df5U, 0x3b6e20c8U, 0x4c69105eU, 0xd56041e4U,
-       0xa2677172U, 0x3c03e4d1U, 0x4b04d447U, 0xd20d85fdU, 0xa50ab56bU,
-       0x35b5a8faU, 0x42b2986cU, 0xdbbbc9d6U, 0xacbcf940U, 0x32d86ce3U,
-       0x45df5c75U, 0xdcd60dcfU, 0xabd13d59U, 0x26d930acU, 0x51de003aU,
-       0xc8d75180U, 0xbfd06116U, 0x21b4f4b5U, 0x56b3c423U, 0xcfba9599U,
-       0xb8bda50fU, 0x2802b89eU, 0x5f058808U, 0xc60cd9b2U, 0xb10be924U,
-       0x2f6f7c87U, 0x58684c11U, 0xc1611dabU, 0xb6662d3dU, 0x76dc4190U,
-       0x01db7106U, 0x98d220bcU, 0xefd5102aU, 0x71b18589U, 0x06b6b51fU,
-       0x9fbfe4a5U, 0xe8b8d433U, 0x7807c9a2U, 0x0f00f934U, 0x9609a88eU,
-       0xe10e9818U, 0x7f6a0dbbU, 0x086d3d2dU, 0x91646c97U, 0xe6635c01U,
-       0x6b6b51f4U, 0x1c6c6162U, 0x856530d8U, 0xf262004eU, 0x6c0695edU,
-       0x1b01a57bU, 0x8208f4c1U, 0xf50fc457U, 0x65b0d9c6U, 0x12b7e950U,
-       0x8bbeb8eaU, 0xfcb9887cU, 0x62dd1ddfU, 0x15da2d49U, 0x8cd37cf3U,
-       0xfbd44c65U, 0x4db26158U, 0x3ab551ceU, 0xa3bc0074U, 0xd4bb30e2U,
-       0x4adfa541U, 0x3dd895d7U, 0xa4d1c46dU, 0xd3d6f4fbU, 0x4369e96aU,
-       0x346ed9fcU, 0xad678846U, 0xda60b8d0U, 0x44042d73U, 0x33031de5U,
-       0xaa0a4c5fU, 0xdd0d7cc9U, 0x5005713cU, 0x270241aaU, 0xbe0b1010U,
-       0xc90c2086U, 0x5768b525U, 0x206f85b3U, 0xb966d409U, 0xce61e49fU,
-       0x5edef90eU, 0x29d9c998U, 0xb0d09822U, 0xc7d7a8b4U, 0x59b33d17U,
-       0x2eb40d81U, 0xb7bd5c3bU, 0xc0ba6cadU, 0xedb88320U, 0x9abfb3b6U,
-       0x03b6e20cU, 0x74b1d29aU, 0xead54739U, 0x9dd277afU, 0x04db2615U,
-       0x73dc1683U, 0xe3630b12U, 0x94643b84U, 0x0d6d6a3eU, 0x7a6a5aa8U,
-       0xe40ecf0bU, 0x9309ff9dU, 0x0a00ae27U, 0x7d079eb1U, 0xf00f9344U,
-       0x8708a3d2U, 0x1e01f268U, 0x6906c2feU, 0xf762575dU, 0x806567cbU,
-       0x196c3671U, 0x6e6b06e7U, 0xfed41b76U, 0x89d32be0U, 0x10da7a5aU,
-       0x67dd4accU, 0xf9b9df6fU, 0x8ebeeff9U, 0x17b7be43U, 0x60b08ed5U,
-       0xd6d6a3e8U, 0xa1d1937eU, 0x38d8c2c4U, 0x4fdff252U, 0xd1bb67f1U,
-       0xa6bc5767U, 0x3fb506ddU, 0x48b2364bU, 0xd80d2bdaU, 0xaf0a1b4cU,
-       0x36034af6U, 0x41047a60U, 0xdf60efc3U, 0xa867df55U, 0x316e8eefU,
-       0x4669be79U, 0xcb61b38cU, 0xbc66831aU, 0x256fd2a0U, 0x5268e236U,
-       0xcc0c7795U, 0xbb0b4703U, 0x220216b9U, 0x5505262fU, 0xc5ba3bbeU,
-       0xb2bd0b28U, 0x2bb45a92U, 0x5cb36a04U, 0xc2d7ffa7U, 0xb5d0cf31U,
-       0x2cd99e8bU, 0x5bdeae1dU, 0x9b64c2b0U, 0xec63f226U, 0x756aa39cU,
-       0x026d930aU, 0x9c0906a9U, 0xeb0e363fU, 0x72076785U, 0x05005713U,
-       0x95bf4a82U, 0xe2b87a14U, 0x7bb12baeU, 0x0cb61b38U, 0x92d28e9bU,
-       0xe5d5be0dU, 0x7cdcefb7U, 0x0bdbdf21U, 0x86d3d2d4U, 0xf1d4e242U,
-       0x68ddb3f8U, 0x1fda836eU, 0x81be16cdU, 0xf6b9265bU, 0x6fb077e1U,
-       0x18b74777U, 0x88085ae6U, 0xff0f6a70U, 0x66063bcaU, 0x11010b5cU,
-       0x8f659effU, 0xf862ae69U, 0x616bffd3U, 0x166ccf45U, 0xa00ae278U,
-       0xd70dd2eeU, 0x4e048354U, 0x3903b3c2U, 0xa7672661U, 0xd06016f7U,
-       0x4969474dU, 0x3e6e77dbU, 0xaed16a4aU, 0xd9d65adcU, 0x40df0b66U,
-       0x37d83bf0U, 0xa9bcae53U, 0xdebb9ec5U, 0x47b2cf7fU, 0x30b5ffe9U,
-       0xbdbdf21cU, 0xcabac28aU, 0x53b39330U, 0x24b4a3a6U, 0xbad03605U,
-       0xcdd70693U, 0x54de5729U, 0x23d967bfU, 0xb3667a2eU, 0xc4614ab8U,
-       0x5d681b02U, 0x2a6f2b94U, 0xb40bbe37U, 0xc30c8ea1U, 0x5a05df1bU,
-       0x2d02ef8dU
-};
 
 static uint32_t partial_crc32_one(uint8_t c, uint32_t crc)
 {
-       return crctab32[(crc ^ c) & 0xff] ^ (crc >> 8);
+       return crc32(crc ^ 0xffffffff, (const unsigned char *)&c, 1) ^ 0xffffffff;
 }
 
 static uint32_t partial_crc32(const char *s, uint32_t crc)
 {
-       while (*s)
-               crc = partial_crc32_one(*s++, crc);
-       return crc;
+       return crc32(crc ^ 0xffffffff, (const unsigned char *)s, strlen(s)) ^ 0xffffffff;
 }
 
-static uint32_t crc32(const char *s)
+static uint32_t crc32_name(const char *s)
 {
-       return partial_crc32(s, 0xffffffff) ^ 0xffffffff;
+       return crc32(0, (const unsigned char *)s, strlen(s));
 }
 
+
 /*----------------------------------------------------------------------*/
 
 static enum symbol_type map_to_ns(enum symbol_type t)
@@ -154,7 +100,7 @@ struct symbol *find_symbol(const char *name, enum symbol_type ns, int exact)
 {
        struct symbol *sym;
 
-       hash_for_each_possible(symbol_hashtable, sym, hnode, crc32(name)) {
+       hash_for_each_possible(symbol_hashtable, sym, hnode, crc32_name(name)) {
                if (map_to_ns(sym->type) == map_to_ns(ns) &&
                    strcmp(name, sym->name) == 0 &&
                    sym->is_declared)
@@ -238,7 +184,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type,
                        return NULL;
        }
 
-       h = crc32(name);
+       h = crc32_name(name);
        hash_for_each_possible(symbol_hashtable, sym, hnode, h) {
                if (map_to_ns(sym->type) != map_to_ns(type) ||
                    strcmp(name, sym->name))
-- 
2.30.2

Re: [PATCH v2] genksyms: Support arm64 CRC32 hardware acceleration1~

[Wentao Guan]

Hello Petr,

> On 5/25/26 3:41 PM, Wentao Guan wrote:
> >> On 5/25/26 10:02 AM, Wentao Guan wrote:
> >>>> Additionally, many distributions already switched to gendwarfksyms last
> >>>> year. Unless someone provides a good reason to keep genksyms around,
> >>>> I expect it will be deprecated soon and eventually removed. It would be
> >>>> more valuable to show whether the same optimization is worthwhile for
> >>>> gendwarfksyms.
> >>> I see gendwarfksyms use crc32 from <zlib.h>, which from zlib1g-dev and zlib1g.
> >>
> >> Ah, I forgot about this, so gendwarfksyms should already be well
> >> optimized. I think that is the most important thing.
> >
> > Which way do you prefer? There are two ways now:
> > 1. this work,
> > 2. import zlib like gendwarfksyms and remove the huge crctab,
> > but need make sure any arch do not cause performance regression?
> > (Assert the user libs precompile with +crc32 is right?)
> > I do some early tests show that:
> > 1) zlib in x86 fast than genksym software
> > 2) zlib in arm64 debian precompile slow than genksym software(crc32 not in armv8)
> > 3) zlib in arm64 recompile with +crc32 same speed with genksyms hardware
> 
> I think it would be useful to first have more data showing that the CRC
> calculation in genksyms is a hot path and contributes significantly to
> the overall kernel build time.
I found that 'nm' is slow and a really crazy but significantly improve my build speed,
from 11m50s down to 11m18s:
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 0c838c467..d5b9db6d3 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -231,7 +231,7 @@ ifdef CONFIG_MODVERSIONS
 #   be compiled and linked to the kernel and/or modules.
 
 gen_symversions =                                                              \
-       if $(NM) $@ 2>/dev/null | grep -q ' __export_symbol_'; then             \
+       if strings $@ 2>/dev/null | grep -q '__export_symbol_'; then            \
                $(cmd_gensymtypes_$1) >> $(dot-target).cmd;                     \
        fi

BRs
Wentao Guan