[PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Guan-Chun Wu]

This series introduces a generic Base64 encoder/decoder to the kernel
library, eliminating duplicated implementations and delivering significant
performance improvements.

The Base64 API has been extended to support multiple variants (Standard,
URL-safe, and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes
a variant parameter and an option to control padding. As part of this
series, users are migrated to the new interface while preserving their
specific formats: fscrypt now uses BASE64_URLSAFE, Ceph uses BASE64_IMAP,
and NVMe is updated to BASE64_STD.

On the encoder side, the implementation processes input in 3-byte blocks,
mapping 24 bits directly to 4 output symbols. This avoids bit-by-bit
streaming and reduces loop overhead, achieving about a 2.7x speedup compared
to previous implementations.

On the decoder side, replace strchr() lookups with per-variant reverse tables
and process input in 4-character groups. Each group is mapped to numeric values
and combined into 3 bytes. Padded and unpadded forms are validated explicitly,
rejecting invalid '=' usage and enforcing tail rules. This improves throughput
by ~43-52x.

Thanks,
Guan-Chun Wu

Link: https://lore.kernel.org/lkml/20250926065235.13623-1-409411716@gms.tku.edu.tw/

---

v3 -> v4:
  - lib/base64: Implemented padding support in the first commit to address the
    previously mentioned issue.
  - lib/base64: Replace the manually written reverse lookup table initialization
    with the BASE64_REV_INIT() macro for cleaner and more maintainable code while
    keeping the same behavior.
  - lib/base64: Simplify branching and tail handling while preserving behavior,
    reducing overhead and improving performance.

---

Guan-Chun Wu (4):
  lib/base64: rework encode/decode for speed and stricter validation
  lib: add KUnit tests for base64 encoding/decoding
  fscrypt: replace local base64url helpers with lib/base64
  ceph: replace local base64 helpers with lib/base64

Kuan-Wei Chiu (2):
  lib/base64: Add support for multiple variants
  lib/base64: Optimize base64_decode() with reverse lookup tables

 drivers/nvme/common/auth.c |   4 +-
 fs/ceph/crypto.c           |  60 +-------
 fs/ceph/crypto.h           |   6 +-
 fs/ceph/dir.c              |   5 +-
 fs/ceph/inode.c            |   2 +-
 fs/crypto/fname.c          |  89 +----------
 include/linux/base64.h     |  10 +-
 lib/Kconfig.debug          |  19 ++-
 lib/base64.c               | 161 +++++++++++++-------
 lib/tests/Makefile         |   1 +
 lib/tests/base64_kunit.c   | 294 +++++++++++++++++++++++++++++++++++++
 11 files changed, 445 insertions(+), 206 deletions(-)
 create mode 100644 lib/tests/base64_kunit.c

-- 
2.34.1

[PATCH v4 1/6] lib/base64: Add support for multiple variants

[Guan-Chun Wu]

From: Kuan-Wei Chiu <visitorckw@gmail.com>

Extend the base64 API to support multiple variants (standard, URL-safe,
and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes a
variant parameter and an option to control padding. Update NVMe auth
code to use the new interface with BASE64_STD.

Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Co-developed-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 drivers/nvme/common/auth.c |  4 +--
 include/linux/base64.h     | 10 ++++--
 lib/base64.c               | 62 ++++++++++++++++++++++----------------
 3 files changed, 46 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/common/auth.c b/drivers/nvme/common/auth.c
index 91e273b89..5fecb53cb 100644
--- a/drivers/nvme/common/auth.c
+++ b/drivers/nvme/common/auth.c
@@ -178,7 +178,7 @@ struct nvme_dhchap_key *nvme_auth_extract_key(unsigned char *secret,
 	if (!key)
 		return ERR_PTR(-ENOMEM);
 
-	key_len = base64_decode(secret, allocated_len, key->key);
+	key_len = base64_decode(secret, allocated_len, key->key, true, BASE64_STD);
 	if (key_len < 0) {
 		pr_debug("base64 key decoding error %d\n",
 			 key_len);
@@ -663,7 +663,7 @@ int nvme_auth_generate_digest(u8 hmac_id, u8 *psk, size_t psk_len,
 	if (ret)
 		goto out_free_digest;
 
-	ret = base64_encode(digest, digest_len, enc);
+	ret = base64_encode(digest, digest_len, enc, true, BASE64_STD);
 	if (ret < hmac_len) {
 		ret = -ENOKEY;
 		goto out_free_digest;
diff --git a/include/linux/base64.h b/include/linux/base64.h
index 660d4cb1e..a2c6c9222 100644
--- a/include/linux/base64.h
+++ b/include/linux/base64.h
@@ -8,9 +8,15 @@
 
 #include <linux/types.h>
 
+enum base64_variant {
+	BASE64_STD,       /* RFC 4648 (standard) */
+	BASE64_URLSAFE,   /* RFC 4648 (base64url) */
+	BASE64_IMAP,      /* RFC 3501 */
+};
+
 #define BASE64_CHARS(nbytes)   DIV_ROUND_UP((nbytes) * 4, 3)
 
-int base64_encode(const u8 *src, int len, char *dst);
-int base64_decode(const char *src, int len, u8 *dst);
+int base64_encode(const u8 *src, int len, char *dst, bool padding, enum base64_variant variant);
+int base64_decode(const char *src, int len, u8 *dst, bool padding, enum base64_variant variant);
 
 #endif /* _LINUX_BASE64_H */
diff --git a/lib/base64.c b/lib/base64.c
index b736a7a43..a7c20a8e8 100644
--- a/lib/base64.c
+++ b/lib/base64.c
@@ -1,12 +1,12 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * base64.c - RFC4648-compliant base64 encoding
+ * base64.c - Base64 with support for multiple variants
  *
  * Copyright (c) 2020 Hannes Reinecke, SUSE
  *
  * Based on the base64url routines from fs/crypto/fname.c
- * (which are using the URL-safe base64 encoding),
- * modified to use the standard coding table from RFC4648 section 4.
+ * (which are using the URL-safe Base64 encoding),
+ * modified to support multiple Base64 variants.
  */
 
 #include <linux/kernel.h>
@@ -15,26 +15,31 @@
 #include <linux/string.h>
 #include <linux/base64.h>
 
-static const char base64_table[65] =
-	"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+static const char base64_tables[][65] = {
+	[BASE64_STD] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/",
+	[BASE64_URLSAFE] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_",
+	[BASE64_IMAP] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+,",
+};
 
 /**
- * base64_encode() - base64-encode some binary data
+ * base64_encode() - Base64-encode some binary data
  * @src: the binary data to encode
  * @srclen: the length of @src in bytes
- * @dst: (output) the base64-encoded string.  Not NUL-terminated.
+ * @dst: (output) the Base64-encoded string.  Not NUL-terminated.
+ * @padding: whether to append '=' padding characters
+ * @variant: which base64 variant to use
  *
- * Encodes data using base64 encoding, i.e. the "Base 64 Encoding" specified
- * by RFC 4648, including the  '='-padding.
+ * Encodes data using the selected Base64 variant.
  *
- * Return: the length of the resulting base64-encoded string in bytes.
+ * Return: the length of the resulting Base64-encoded string in bytes.
  */
-int base64_encode(const u8 *src, int srclen, char *dst)
+int base64_encode(const u8 *src, int srclen, char *dst, bool padding, enum base64_variant variant)
 {
 	u32 ac = 0;
 	int bits = 0;
 	int i;
 	char *cp = dst;
+	const char *base64_table = base64_tables[variant];
 
 	for (i = 0; i < srclen; i++) {
 		ac = (ac << 8) | src[i];
@@ -48,44 +53,49 @@ int base64_encode(const u8 *src, int srclen, char *dst)
 		*cp++ = base64_table[(ac << (6 - bits)) & 0x3f];
 		bits -= 6;
 	}
-	while (bits < 0) {
-		*cp++ = '=';
-		bits += 2;
+	if (padding) {
+		while (bits < 0) {
+			*cp++ = '=';
+			bits += 2;
+		}
 	}
 	return cp - dst;
 }
 EXPORT_SYMBOL_GPL(base64_encode);
 
 /**
- * base64_decode() - base64-decode a string
+ * base64_decode() - Base64-decode a string
  * @src: the string to decode.  Doesn't need to be NUL-terminated.
  * @srclen: the length of @src in bytes
  * @dst: (output) the decoded binary data
+ * @padding: whether to append '=' padding characters
+ * @variant: which base64 variant to use
  *
- * Decodes a string using base64 encoding, i.e. the "Base 64 Encoding"
- * specified by RFC 4648, including the  '='-padding.
+ * Decodes a string using the selected Base64 variant.
  *
  * This implementation hasn't been optimized for performance.
  *
  * Return: the length of the resulting decoded binary data in bytes,
- *	   or -1 if the string isn't a valid base64 string.
+ *	   or -1 if the string isn't a valid Base64 string.
  */
-int base64_decode(const char *src, int srclen, u8 *dst)
+int base64_decode(const char *src, int srclen, u8 *dst, bool padding, enum base64_variant variant)
 {
 	u32 ac = 0;
 	int bits = 0;
 	int i;
 	u8 *bp = dst;
+	const char *base64_table = base64_tables[variant];
 
 	for (i = 0; i < srclen; i++) {
 		const char *p = strchr(base64_table, src[i]);
-
-		if (src[i] == '=') {
-			ac = (ac << 6);
-			bits += 6;
-			if (bits >= 8)
-				bits -= 8;
-			continue;
+		if (padding) {
+			if (src[i] == '=') {
+				ac = (ac << 6);
+				bits += 6;
+				if (bits >= 8)
+					bits -= 8;
+				continue;
+			}
 		}
 		if (p == NULL || src[i] == 0)
 			return -1;
-- 
2.34.1

[PATCH v4 2/6] lib/base64: Optimize base64_decode() with reverse lookup tables

[Guan-Chun Wu]

From: Kuan-Wei Chiu <visitorckw@gmail.com>

Replace the use of strchr() in base64_decode() with precomputed reverse
lookup tables for each variant. This avoids repeated string scans and
improves performance. Use -1 in the tables to mark invalid characters.

Decode:
  64B   ~1530ns  ->  ~80ns    (~19.1x)
  1KB  ~27726ns  -> ~1239ns   (~22.4x)

Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Co-developed-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 lib/base64.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/lib/base64.c b/lib/base64.c
index a7c20a8e8..8a0d28908 100644
--- a/lib/base64.c
+++ b/lib/base64.c
@@ -21,6 +21,21 @@ static const char base64_tables[][65] = {
 	[BASE64_IMAP] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+,",
 };
 
+#define BASE64_REV_INIT(ch_62, ch_63) { \
+	[0 ... 255] = -1, \
+	['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
+		13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, \
+	['a'] = 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, \
+		39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, \
+	['0'] = 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
+	[ch_62] = 62, [ch_63] = 63, \
+}
+
+static const s8 base64_rev_maps[][256] = {
+	[BASE64_STD] = BASE64_REV_INIT('+', '/'),
+	[BASE64_URLSAFE] = BASE64_REV_INIT('-', '_'),
+	[BASE64_IMAP] = BASE64_REV_INIT('+', ',')
+};
 /**
  * base64_encode() - Base64-encode some binary data
  * @src: the binary data to encode
@@ -84,10 +99,9 @@ int base64_decode(const char *src, int srclen, u8 *dst, bool padding, enum base6
 	int bits = 0;
 	int i;
 	u8 *bp = dst;
-	const char *base64_table = base64_tables[variant];
+	s8 ch;
 
 	for (i = 0; i < srclen; i++) {
-		const char *p = strchr(base64_table, src[i]);
 		if (padding) {
 			if (src[i] == '=') {
 				ac = (ac << 6);
@@ -97,9 +111,10 @@ int base64_decode(const char *src, int srclen, u8 *dst, bool padding, enum base6
 				continue;
 			}
 		}
-		if (p == NULL || src[i] == 0)
+		ch = base64_rev_maps[variant][(u8)src[i]];
+		if (ch == -1)
 			return -1;
-		ac = (ac << 6) | (p - base64_table);
+		ac = (ac << 6) | ch;
 		bits += 6;
 		if (bits >= 8) {
 			bits -= 8;
-- 
2.34.1

[PATCH v4 3/6] lib/base64: rework encode/decode for speed and stricter validation

[Guan-Chun Wu]

The old base64 implementation relied on a bit-accumulator loop, which was
slow for larger inputs and too permissive in validation. It would accept
extra '=', missing '=', or even '=' appearing in the middle of the input,
allowing malformed strings to pass. This patch reworks the internals to
improve performance and enforce stricter validation.

Changes:
 - Encoder:
   * Process input in 3-byte blocks, mapping 24 bits into four 6-bit
     symbols, avoiding bit-by-bit shifting and reducing loop iterations.
   * Handle the final 1-2 leftover bytes explicitly and emit '=' only when
     requested.
 - Decoder:
   * Based on the reverse lookup tables from the previous patch, decode
     input in 4-character groups.
   * Each group is looked up directly, converted into numeric values, and
     combined into 3 output bytes.
   * Explicitly handle padded and unpadded forms:
      - With padding: input length must be a multiple of 4, and '=' is
        allowed only in the last two positions. Reject stray or early '='.
      - Without padding: validate tail lengths (2 or 3 chars) and require
        unused low bits to be zero.
   * Removed the bit-accumulator style loop to reduce loop iterations.

Performance (x86_64, Intel Core i7-10700 @ 2.90GHz, avg over 1000 runs,
KUnit):

Encode:
  64B   ~90ns   -> ~32ns   (~2.8x)
  1KB  ~1332ns  -> ~510ns  (~2.6x)

Decode:
  64B  ~1530ns  -> ~35ns   (~43.7x)
  1KB ~27726ns  -> ~530ns  (~52.3x)

Co-developed-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Co-developed-by: Yu-Sheng Huang <home7438072@gmail.com>
Signed-off-by: Yu-Sheng Huang <home7438072@gmail.com>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 lib/base64.c | 110 ++++++++++++++++++++++++++++++++-------------------
 1 file changed, 69 insertions(+), 41 deletions(-)

diff --git a/lib/base64.c b/lib/base64.c
index 8a0d28908..bcdbd411d 100644
--- a/lib/base64.c
+++ b/lib/base64.c
@@ -51,28 +51,38 @@ static const s8 base64_rev_maps[][256] = {
 int base64_encode(const u8 *src, int srclen, char *dst, bool padding, enum base64_variant variant)
 {
 	u32 ac = 0;
-	int bits = 0;
-	int i;
 	char *cp = dst;
 	const char *base64_table = base64_tables[variant];
 
-	for (i = 0; i < srclen; i++) {
-		ac = (ac << 8) | src[i];
-		bits += 8;
-		do {
-			bits -= 6;
-			*cp++ = base64_table[(ac >> bits) & 0x3f];
-		} while (bits >= 6);
-	}
-	if (bits) {
-		*cp++ = base64_table[(ac << (6 - bits)) & 0x3f];
-		bits -= 6;
+	while (srclen >= 3) {
+		ac = (u32)src[0] << 16 | (u32)src[1] << 8 | (u32)src[2];
+		*cp++ = base64_table[ac >> 18];
+		*cp++ = base64_table[(ac >> 12) & 0x3f];
+		*cp++ = base64_table[(ac >> 6) & 0x3f];
+		*cp++ = base64_table[ac & 0x3f];
+
+		src += 3;
+		srclen -= 3;
 	}
-	if (padding) {
-		while (bits < 0) {
+
+	switch (srclen) {
+	case 2:
+		ac = (u32)src[0] << 16 | (u32)src[1] << 8;
+		*cp++ = base64_table[ac >> 18];
+		*cp++ = base64_table[(ac >> 12) & 0x3f];
+		*cp++ = base64_table[(ac >> 6) & 0x3f];
+		if (padding)
+			*cp++ = '=';
+		break;
+	case 1:
+		ac = (u32)src[0] << 16;
+		*cp++ = base64_table[ac >> 18];
+		*cp++ = base64_table[(ac >> 12) & 0x3f];
+		if (padding) {
+			*cp++ = '=';
 			*cp++ = '=';
-			bits += 2;
 		}
+		break;
 	}
 	return cp - dst;
 }
@@ -88,41 +98,59 @@ EXPORT_SYMBOL_GPL(base64_encode);
  *
  * Decodes a string using the selected Base64 variant.
  *
- * This implementation hasn't been optimized for performance.
- *
  * Return: the length of the resulting decoded binary data in bytes,
  *	   or -1 if the string isn't a valid Base64 string.
  */
 int base64_decode(const char *src, int srclen, u8 *dst, bool padding, enum base64_variant variant)
 {
-	u32 ac = 0;
-	int bits = 0;
-	int i;
 	u8 *bp = dst;
-	s8 ch;
+	s8 input[4];
+	s32 val;
+	const u8 *s = (const u8 *)src;
+	const s8 *base64_rev_tables = base64_rev_maps[variant];
 
-	for (i = 0; i < srclen; i++) {
-		if (padding) {
-			if (src[i] == '=') {
-				ac = (ac << 6);
-				bits += 6;
-				if (bits >= 8)
-					bits -= 8;
-				continue;
-			}
-		}
-		ch = base64_rev_maps[variant][(u8)src[i]];
-		if (ch == -1)
-			return -1;
-		ac = (ac << 6) | ch;
-		bits += 6;
-		if (bits >= 8) {
-			bits -= 8;
-			*bp++ = (u8)(ac >> bits);
+	while (srclen >= 4) {
+		input[0] = base64_rev_tables[s[0]];
+		input[1] = base64_rev_tables[s[1]];
+		input[2] = base64_rev_tables[s[2]];
+		input[3] = base64_rev_tables[s[3]];
+
+		val = input[0] << 18 | input[1] << 12 | input[2] << 6 | input[3];
+
+		if (unlikely(val < 0)) {
+			if (!padding || srclen != 4 || s[3] != '=')
+				return -1;
+			padding = 0;
+			srclen = s[2] == '=' ? 2 : 3;
+			break;
 		}
+
+		*bp++ = val >> 16;
+		*bp++ = val >> 8;
+		*bp++ = val;
+
+		s += 4;
+		srclen -= 4;
 	}
-	if (ac & ((1 << bits) - 1))
+
+	if (likely(!srclen))
+		return bp - dst;
+	if (padding || srclen == 1)
 		return -1;
+
+	val = (base64_rev_tables[s[0]] << 12) | (base64_rev_tables[s[1]] << 6);
+	*bp++ = val >> 10;
+
+	if (srclen == 2) {
+		if (val & 0x800003ff)
+			return -1;
+	} else {
+		val |= base64_rev_tables[s[2]];
+		if (val & 0x80000003)
+			return -1;
+		*bp++ = val >> 2;
+	}
 	return bp - dst;
 }
 EXPORT_SYMBOL_GPL(base64_decode);
+
-- 
2.34.1

[PATCH v4 4/6] lib: add KUnit tests for base64 encoding/decoding

[Guan-Chun Wu]

Add a KUnit test suite to validate the base64 helpers. The tests cover
both encoding and decoding, including padded and unpadded forms as defined
by RFC 4648 (standard base64), and add negative cases for malformed inputs
and padding errors.

The test suite also validates other variants (URLSAFE, IMAP) to ensure
their correctness.

In addition to functional checks, the suite includes simple microbenchmarks
which report average encode/decode latency for small (64B) and larger
(1KB) inputs. These numbers are informational only and do not gate the
tests.

Kconfig (BASE64_KUNIT) and lib/tests/Makefile are updated accordingly.

Sample KUnit output:

    KTAP version 1
    # Subtest: base64
    # module: base64_kunit
    1..4
    # base64_performance_tests: [64B] encode run : 32ns
    # base64_performance_tests: [64B] decode run : 35ns
    # base64_performance_tests: [1KB] encode run : 510ns
    # base64_performance_tests: [1KB] decode run : 530ns
    ok 1 base64_performance_tests
    ok 2 base64_std_encode_tests
    ok 3 base64_std_decode_tests
    ok 4 base64_variant_tests
    # base64: pass:4 fail:0 skip:0 total:4
    # Totals: pass:4 fail:0 skip:0 total:4

Reviewed-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 lib/Kconfig.debug        |  19 ++-
 lib/tests/Makefile       |   1 +
 lib/tests/base64_kunit.c | 294 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 313 insertions(+), 1 deletion(-)
 create mode 100644 lib/tests/base64_kunit.c

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index dc0e0c6ed..1cfb12d02 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2794,8 +2794,25 @@ config CMDLINE_KUNIT_TEST
 
 	  If unsure, say N.
 
+config BASE64_KUNIT
+	tristate "KUnit test for base64 decoding and encoding" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	default KUNIT_ALL_TESTS
+	help
+	  This builds the base64 unit tests.
+
+	  The tests cover the encoding and decoding logic of Base64 functions
+	  in the kernel.
+	  In addition to correctness checks, simple performance benchmarks
+	  for both encoding and decoding are also included.
+
+	  For more information on KUnit and unit tests in general please refer
+	  to the KUnit documentation in Documentation/dev-tools/kunit/.
+
+	  If unsure, say N.
+
 config BITS_TEST
-	tristate "KUnit test for bits.h" if !KUNIT_ALL_TESTS
+	tristate "KUnit test for bit functions and macros" if !KUNIT_ALL_TESTS
 	depends on KUNIT
 	default KUNIT_ALL_TESTS
 	help
diff --git a/lib/tests/Makefile b/lib/tests/Makefile
index fa6d728a8..6593a2873 100644
--- a/lib/tests/Makefile
+++ b/lib/tests/Makefile
@@ -4,6 +4,7 @@
 
 # KUnit tests
 CFLAGS_bitfield_kunit.o := $(DISABLE_STRUCTLEAK_PLUGIN)
+obj-$(CONFIG_BASE64_KUNIT) += base64_kunit.o
 obj-$(CONFIG_BITFIELD_KUNIT) += bitfield_kunit.o
 obj-$(CONFIG_BITS_TEST) += test_bits.o
 obj-$(CONFIG_BLACKHOLE_DEV_KUNIT_TEST) += blackhole_dev_kunit.o
diff --git a/lib/tests/base64_kunit.c b/lib/tests/base64_kunit.c
new file mode 100644
index 000000000..f7252070c
--- /dev/null
+++ b/lib/tests/base64_kunit.c
@@ -0,0 +1,294 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * base64_kunit_test.c - KUnit tests for base64 encoding and decoding functions
+ *
+ * Copyright (c) 2025, Guan-Chun Wu <409411716@gms.tku.edu.tw>
+ */
+
+#include <kunit/test.h>
+#include <linux/base64.h>
+
+/* ---------- Benchmark helpers ---------- */
+static u64 bench_encode_ns(const u8 *data, int len, char *dst, int reps,
+			   enum base64_variant variant)
+{
+	u64 t0, t1;
+
+	t0 = ktime_get_ns();
+	for (int i = 0; i < reps; i++)
+		base64_encode(data, len, dst, true, variant);
+	t1 = ktime_get_ns();
+
+	return div64_u64(t1 - t0, (u64)reps);
+}
+
+static u64 bench_decode_ns(const char *data, int len, u8 *dst, int reps,
+			   enum base64_variant variant)
+{
+	u64 t0, t1;
+
+	t0 = ktime_get_ns();
+	for (int i = 0; i < reps; i++)
+		base64_decode(data, len, dst, true, variant);
+	t1 = ktime_get_ns();
+
+	return div64_u64(t1 - t0, (u64)reps);
+}
+
+static void run_perf_and_check(struct kunit *test, const char *label, int size,
+			       enum base64_variant variant)
+{
+	const int reps = 1000;
+	size_t outlen = DIV_ROUND_UP(size, 3) * 4;
+	u8 *in = kmalloc(size, GFP_KERNEL);
+	char *enc = kmalloc(outlen, GFP_KERNEL);
+	u8 *decoded = kmalloc(size, GFP_KERNEL);
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, in);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, enc);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, decoded);
+
+	get_random_bytes(in, size);
+	int enc_len = base64_encode(in, size, enc, true, variant);
+	int dec_len = base64_decode(enc, enc_len, decoded, true, variant);
+
+	/* correctness sanity check */
+	KUNIT_EXPECT_EQ(test, dec_len, size);
+	KUNIT_EXPECT_MEMEQ(test, decoded, in, size);
+
+	/* benchmark encode */
+
+	u64 t1 = bench_encode_ns(in, size, enc, reps, variant);
+
+	kunit_info(test, "[%s] encode run : %lluns", label, t1);
+
+	u64 t2 = bench_decode_ns(enc, enc_len, decoded, reps, variant);
+
+	kunit_info(test, "[%s] decode run : %lluns", label, t2);
+
+	kfree(in);
+	kfree(enc);
+	kfree(decoded);
+}
+
+static void base64_performance_tests(struct kunit *test)
+{
+	/* run on STD variant only */
+	run_perf_and_check(test, "64B", 64, BASE64_STD);
+	run_perf_and_check(test, "1KB", 1024, BASE64_STD);
+}
+
+/* ---------- Helpers for encode ---------- */
+static void expect_encode_ok(struct kunit *test, const u8 *src, int srclen,
+			     const char *expected, bool padding,
+			     enum base64_variant variant)
+{
+	char buf[128];
+	int encoded_len = base64_encode(src, srclen, buf, padding, variant);
+
+	buf[encoded_len] = '\0';
+
+	KUNIT_EXPECT_EQ(test, encoded_len, strlen(expected));
+	KUNIT_EXPECT_STREQ(test, buf, expected);
+}
+
+/* ---------- Helpers for decode ---------- */
+static void expect_decode_ok(struct kunit *test, const char *src,
+			     const u8 *expected, int expected_len, bool padding,
+			     enum base64_variant variant)
+{
+	u8 buf[128];
+	int decoded_len = base64_decode(src, strlen(src), buf, padding, variant);
+
+	KUNIT_EXPECT_EQ(test, decoded_len, expected_len);
+	KUNIT_EXPECT_MEMEQ(test, buf, expected, expected_len);
+}
+
+static void expect_decode_err(struct kunit *test, const char *src,
+			      int srclen, bool padding,
+			      enum base64_variant variant)
+{
+	u8 buf[64];
+	int decoded_len = base64_decode(src, srclen, buf, padding, variant);
+
+	KUNIT_EXPECT_EQ(test, decoded_len, -1);
+}
+
+/* ---------- Encode Tests ---------- */
+static void base64_std_encode_tests(struct kunit *test)
+{
+	/* With padding */
+	expect_encode_ok(test, (const u8 *)"", 0, "", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"f", 1, "Zg==", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"fo", 2, "Zm8=", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foo", 3, "Zm9v", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foob", 4, "Zm9vYg==", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"fooba", 5, "Zm9vYmE=", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foobar", 6, "Zm9vYmFy", true, BASE64_STD);
+
+	/* Extra cases with padding */
+	expect_encode_ok(test, (const u8 *)"Hello, world!", 13, "SGVsbG8sIHdvcmxkIQ==",
+			 true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26,
+			 "QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"abcdefghijklmnopqrstuvwxyz", 26,
+			 "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo=", true, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"0123456789+/", 12, "MDEyMzQ1Njc4OSsv",
+			 true, BASE64_STD);
+
+	/* Without padding */
+	expect_encode_ok(test, (const u8 *)"", 0, "", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"f", 1, "Zg", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"fo", 2, "Zm8", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foo", 3, "Zm9v", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foob", 4, "Zm9vYg", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"fooba", 5, "Zm9vYmE", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"foobar", 6, "Zm9vYmFy", false, BASE64_STD);
+
+	/* Extra cases without padding */
+	expect_encode_ok(test, (const u8 *)"Hello, world!", 13, "SGVsbG8sIHdvcmxkIQ",
+			 false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26,
+			 "QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"abcdefghijklmnopqrstuvwxyz", 26,
+			 "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo", false, BASE64_STD);
+	expect_encode_ok(test, (const u8 *)"0123456789+/", 12, "MDEyMzQ1Njc4OSsv",
+			 false, BASE64_STD);
+}
+
+/* ---------- Decode Tests ---------- */
+static void base64_std_decode_tests(struct kunit *test)
+{
+	/* -------- With padding --------*/
+	expect_decode_ok(test, "", (const u8 *)"", 0, true, BASE64_STD);
+	expect_decode_ok(test, "Zg==", (const u8 *)"f", 1, true, BASE64_STD);
+	expect_decode_ok(test, "Zm8=", (const u8 *)"fo", 2, true, BASE64_STD);
+	expect_decode_ok(test, "Zm9v", (const u8 *)"foo", 3, true, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYg==", (const u8 *)"foob", 4, true, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYmE=", (const u8 *)"fooba", 5, true, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYmFy", (const u8 *)"foobar", 6, true, BASE64_STD);
+	expect_decode_ok(test, "SGVsbG8sIHdvcmxkIQ==", (const u8 *)"Hello, world!", 13,
+			 true, BASE64_STD);
+	expect_decode_ok(test, "QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=",
+			 (const u8 *)"ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26, true, BASE64_STD);
+	expect_decode_ok(test, "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo=",
+			 (const u8 *)"abcdefghijklmnopqrstuvwxyz", 26, true, BASE64_STD);
+
+	/* Error cases */
+	expect_decode_err(test, "Zg=!", 4, true, BASE64_STD);
+	expect_decode_err(test, "Zm$=", 4, true, BASE64_STD);
+	expect_decode_err(test, "Z===", 4, true, BASE64_STD);
+	expect_decode_err(test, "Zg", 2, true, BASE64_STD);
+	expect_decode_err(test, "Zm9v====", 8, true, BASE64_STD);
+	expect_decode_err(test, "Zm==A", 5, true, BASE64_STD);
+
+	{
+		char with_nul[4] = { 'Z', 'g', '\0', '=' };
+
+		expect_decode_err(test, with_nul, 4, true, BASE64_STD);
+	}
+
+	/* -------- Without padding --------*/
+	expect_decode_ok(test, "", (const u8 *)"", 0, false, BASE64_STD);
+	expect_decode_ok(test, "Zg", (const u8 *)"f", 1, false, BASE64_STD);
+	expect_decode_ok(test, "Zm8", (const u8 *)"fo", 2, false, BASE64_STD);
+	expect_decode_ok(test, "Zm9v", (const u8 *)"foo", 3, false, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYg", (const u8 *)"foob", 4, false, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYmE", (const u8 *)"fooba", 5, false, BASE64_STD);
+	expect_decode_ok(test, "Zm9vYmFy", (const u8 *)"foobar", 6, false, BASE64_STD);
+	expect_decode_ok(test, "TWFu", (const u8 *)"Man", 3, false, BASE64_STD);
+	expect_decode_ok(test, "SGVsbG8sIHdvcmxkIQ", (const u8 *)"Hello, world!", 13,
+			 false, BASE64_STD);
+	expect_decode_ok(test, "QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo",
+			 (const u8 *)"ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26, false, BASE64_STD);
+	expect_decode_ok(test, "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo",
+			 (const u8 *)"abcdefghijklmnopqrstuvwxyz", 26, false, BASE64_STD);
+	expect_decode_ok(test, "MDEyMzQ1Njc4OSsv", (const u8 *)"0123456789+/", 12,
+			 false, BASE64_STD);
+
+	/* Error cases */
+	expect_decode_err(test, "Zg=!", 4, false, BASE64_STD);
+	expect_decode_err(test, "Zm$=", 4, false, BASE64_STD);
+	expect_decode_err(test, "Z===", 4, false, BASE64_STD);
+	expect_decode_err(test, "Zg=", 3, false, BASE64_STD);
+	expect_decode_err(test, "Zm9v====", 8, false, BASE64_STD);
+	expect_decode_err(test, "Zm==v", 4, false, BASE64_STD);
+
+	{
+		char with_nul[4] = { 'Z', 'g', '\0', '=' };
+
+		expect_decode_err(test, with_nul, 4, false, BASE64_STD);
+	}
+}
+
+/* ---------- Variant tests (URLSAFE / IMAP) ---------- */
+static void base64_variant_tests(struct kunit *test)
+{
+	const u8 sample1[] = { 0x00, 0xfb, 0xff, 0x7f, 0x80 };
+	char std_buf[128], url_buf[128], imap_buf[128];
+	u8 back[128];
+	int n_std, n_url, n_imap, m;
+	int i;
+
+	n_std = base64_encode(sample1, sizeof(sample1), std_buf, false, BASE64_STD);
+	n_url = base64_encode(sample1, sizeof(sample1), url_buf, false, BASE64_URLSAFE);
+	std_buf[n_std] = '\0';
+	url_buf[n_url] = '\0';
+
+	for (i = 0; i < n_std; i++) {
+		if (std_buf[i] == '+')
+			std_buf[i] = '-';
+		else if (std_buf[i] == '/')
+			std_buf[i] = '_';
+	}
+	KUNIT_EXPECT_STREQ(test, std_buf, url_buf);
+
+	m = base64_decode(url_buf, n_url, back, false, BASE64_URLSAFE);
+	KUNIT_EXPECT_EQ(test, m, (int)sizeof(sample1));
+	KUNIT_EXPECT_MEMEQ(test, back, sample1, sizeof(sample1));
+
+	n_std  = base64_encode(sample1, sizeof(sample1), std_buf, false, BASE64_STD);
+	n_imap = base64_encode(sample1, sizeof(sample1), imap_buf, false, BASE64_IMAP);
+	std_buf[n_std]   = '\0';
+	imap_buf[n_imap] = '\0';
+
+	for (i = 0; i < n_std; i++)
+		if (std_buf[i] == '/')
+			std_buf[i] = ',';
+	KUNIT_EXPECT_STREQ(test, std_buf, imap_buf);
+
+	m = base64_decode(imap_buf, n_imap, back, false, BASE64_IMAP);
+	KUNIT_EXPECT_EQ(test, m, (int)sizeof(sample1));
+	KUNIT_EXPECT_MEMEQ(test, back, sample1, sizeof(sample1));
+
+	{
+		const char *bad = "Zg==";
+		u8 tmp[8];
+
+		m = base64_decode(bad, strlen(bad), tmp, false, BASE64_URLSAFE);
+		KUNIT_EXPECT_EQ(test, m, -1);
+
+		m = base64_decode(bad, strlen(bad), tmp, false, BASE64_IMAP);
+		KUNIT_EXPECT_EQ(test, m, -1);
+	}
+}
+
+/* ---------- Test registration ---------- */
+static struct kunit_case base64_test_cases[] = {
+	KUNIT_CASE(base64_performance_tests),
+	KUNIT_CASE(base64_std_encode_tests),
+	KUNIT_CASE(base64_std_decode_tests),
+	KUNIT_CASE(base64_variant_tests),
+	{}
+};
+
+static struct kunit_suite base64_test_suite = {
+	.name = "base64",
+	.test_cases = base64_test_cases,
+};
+
+kunit_test_suite(base64_test_suite);
+
+MODULE_AUTHOR("Guan-Chun Wu <409411716@gms.tku.edu.tw>");
+MODULE_DESCRIPTION("KUnit tests for Base64 encoding/decoding, including performance checks");
+MODULE_LICENSE("GPL");
-- 
2.34.1

[PATCH v4 5/6] fscrypt: replace local base64url helpers with lib/base64

[Guan-Chun Wu]

Replace the base64url encoding and decoding functions in fscrypt with the
generic base64_encode() and base64_decode() helpers from lib/base64.

This removes the custom implementation in fscrypt, reduces code
duplication, and relies on the shared Base64 implementation in lib.
The helpers preserve RFC 4648-compliant URL-safe Base64 encoding without
padding, so there are no functional changes.

This change also improves performance: encoding is about 2.7x faster and
decoding achieves 43-52x speedups compared to the previous implementation.

Reviewed-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 fs/crypto/fname.c | 89 ++++-------------------------------------------
 1 file changed, 6 insertions(+), 83 deletions(-)

diff --git a/fs/crypto/fname.c b/fs/crypto/fname.c
index f9f6713e1..dcf7cff70 100644
--- a/fs/crypto/fname.c
+++ b/fs/crypto/fname.c
@@ -17,6 +17,7 @@
 #include <linux/export.h>
 #include <linux/namei.h>
 #include <linux/scatterlist.h>
+#include <linux/base64.h>
 
 #include "fscrypt_private.h"
 
@@ -72,7 +73,7 @@ struct fscrypt_nokey_name {
 
 /* Encoded size of max-size no-key name */
 #define FSCRYPT_NOKEY_NAME_MAX_ENCODED \
-		FSCRYPT_BASE64URL_CHARS(FSCRYPT_NOKEY_NAME_MAX)
+		BASE64_CHARS(FSCRYPT_NOKEY_NAME_MAX)
 
 static inline bool fscrypt_is_dot_dotdot(const struct qstr *str)
 {
@@ -163,84 +164,6 @@ static int fname_decrypt(const struct inode *inode,
 	return 0;
 }
 
-static const char base64url_table[65] =
-	"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
-
-#define FSCRYPT_BASE64URL_CHARS(nbytes)	DIV_ROUND_UP((nbytes) * 4, 3)
-
-/**
- * fscrypt_base64url_encode() - base64url-encode some binary data
- * @src: the binary data to encode
- * @srclen: the length of @src in bytes
- * @dst: (output) the base64url-encoded string.  Not NUL-terminated.
- *
- * Encodes data using base64url encoding, i.e. the "Base 64 Encoding with URL
- * and Filename Safe Alphabet" specified by RFC 4648.  '='-padding isn't used,
- * as it's unneeded and not required by the RFC.  base64url is used instead of
- * base64 to avoid the '/' character, which isn't allowed in filenames.
- *
- * Return: the length of the resulting base64url-encoded string in bytes.
- *	   This will be equal to FSCRYPT_BASE64URL_CHARS(srclen).
- */
-static int fscrypt_base64url_encode(const u8 *src, int srclen, char *dst)
-{
-	u32 ac = 0;
-	int bits = 0;
-	int i;
-	char *cp = dst;
-
-	for (i = 0; i < srclen; i++) {
-		ac = (ac << 8) | src[i];
-		bits += 8;
-		do {
-			bits -= 6;
-			*cp++ = base64url_table[(ac >> bits) & 0x3f];
-		} while (bits >= 6);
-	}
-	if (bits)
-		*cp++ = base64url_table[(ac << (6 - bits)) & 0x3f];
-	return cp - dst;
-}
-
-/**
- * fscrypt_base64url_decode() - base64url-decode a string
- * @src: the string to decode.  Doesn't need to be NUL-terminated.
- * @srclen: the length of @src in bytes
- * @dst: (output) the decoded binary data
- *
- * Decodes a string using base64url encoding, i.e. the "Base 64 Encoding with
- * URL and Filename Safe Alphabet" specified by RFC 4648.  '='-padding isn't
- * accepted, nor are non-encoding characters such as whitespace.
- *
- * This implementation hasn't been optimized for performance.
- *
- * Return: the length of the resulting decoded binary data in bytes,
- *	   or -1 if the string isn't a valid base64url string.
- */
-static int fscrypt_base64url_decode(const char *src, int srclen, u8 *dst)
-{
-	u32 ac = 0;
-	int bits = 0;
-	int i;
-	u8 *bp = dst;
-
-	for (i = 0; i < srclen; i++) {
-		const char *p = strchr(base64url_table, src[i]);
-
-		if (p == NULL || src[i] == 0)
-			return -1;
-		ac = (ac << 6) | (p - base64url_table);
-		bits += 6;
-		if (bits >= 8) {
-			bits -= 8;
-			*bp++ = (u8)(ac >> bits);
-		}
-	}
-	if (ac & ((1 << bits) - 1))
-		return -1;
-	return bp - dst;
-}
-
 bool __fscrypt_fname_encrypted_size(const union fscrypt_policy *policy,
 				    u32 orig_len, u32 max_len,
 				    u32 *encrypted_len_ret)
@@ -387,8 +310,8 @@ int fscrypt_fname_disk_to_usr(const struct inode *inode,
 		       nokey_name.sha256);
 		size = FSCRYPT_NOKEY_NAME_MAX;
 	}
-	oname->len = fscrypt_base64url_encode((const u8 *)&nokey_name, size,
-					      oname->name);
+	oname->len = base64_encode((const u8 *)&nokey_name, size,
+				   oname->name, false, BASE64_URLSAFE);
 	return 0;
 }
 EXPORT_SYMBOL(fscrypt_fname_disk_to_usr);
@@ -467,8 +390,8 @@ int fscrypt_setup_filename(struct inode *dir, const struct qstr *iname,
 	if (fname->crypto_buf.name == NULL)
 		return -ENOMEM;
 
-	ret = fscrypt_base64url_decode(iname->name, iname->len,
-				       fname->crypto_buf.name);
+	ret = base64_decode(iname->name, iname->len,
+			    fname->crypto_buf.name, false, BASE64_URLSAFE);
 	if (ret < (int)offsetof(struct fscrypt_nokey_name, bytes[1]) ||
 	    (ret > offsetof(struct fscrypt_nokey_name, sha256) &&
 	     ret != FSCRYPT_NOKEY_NAME_MAX)) {
-- 
2.34.1

[PATCH v4 6/6] ceph: replace local base64 helpers with lib/base64

[Guan-Chun Wu]

Remove the ceph_base64_encode() and ceph_base64_decode() functions and
replace their usage with the generic base64_encode() and base64_decode()
helpers from lib/base64.

This eliminates the custom implementation in Ceph, reduces code
duplication, and relies on the shared Base64 code in lib.
The helpers preserve RFC 3501-compliant Base64 encoding without padding,
so there are no functional changes.

This change also improves performance: encoding is about 2.7x faster and
decoding achieves 43-52x speedups compared to the previous local
implementation.

Reviewed-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Guan-Chun Wu <409411716@gms.tku.edu.tw>
---
 fs/ceph/crypto.c | 60 ++++--------------------------------------------
 fs/ceph/crypto.h |  6 +----
 fs/ceph/dir.c    |  5 ++--
 fs/ceph/inode.c  |  2 +-
 4 files changed, 9 insertions(+), 64 deletions(-)

diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
index cab722619..9bb0f320b 100644
--- a/fs/ceph/crypto.c
+++ b/fs/ceph/crypto.c
@@ -15,59 +15,6 @@
 #include "mds_client.h"
 #include "crypto.h"
 
-/*
- * The base64url encoding used by fscrypt includes the '_' character, which may
- * cause problems in snapshot names (which can not start with '_').  Thus, we
- * used the base64 encoding defined for IMAP mailbox names (RFC 3501) instead,
- * which replaces '-' and '_' by '+' and ','.
- */
-static const char base64_table[65] =
-	"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+,";
-
-int ceph_base64_encode(const u8 *src, int srclen, char *dst)
-{
-	u32 ac = 0;
-	int bits = 0;
-	int i;
-	char *cp = dst;
-
-	for (i = 0; i < srclen; i++) {
-		ac = (ac << 8) | src[i];
-		bits += 8;
-		do {
-			bits -= 6;
-			*cp++ = base64_table[(ac >> bits) & 0x3f];
-		} while (bits >= 6);
-	}
-	if (bits)
-		*cp++ = base64_table[(ac << (6 - bits)) & 0x3f];
-	return cp - dst;
-}
-
-int ceph_base64_decode(const char *src, int srclen, u8 *dst)
-{
-	u32 ac = 0;
-	int bits = 0;
-	int i;
-	u8 *bp = dst;
-
-	for (i = 0; i < srclen; i++) {
-		const char *p = strchr(base64_table, src[i]);
-
-		if (p == NULL || src[i] == 0)
-			return -1;
-		ac = (ac << 6) | (p - base64_table);
-		bits += 6;
-		if (bits >= 8) {
-			bits -= 8;
-			*bp++ = (u8)(ac >> bits);
-		}
-	}
-	if (ac & ((1 << bits) - 1))
-		return -1;
-	return bp - dst;
-}
-
 static int ceph_crypt_get_context(struct inode *inode, void *ctx, size_t len)
 {
 	struct ceph_inode_info *ci = ceph_inode(inode);
@@ -316,7 +263,7 @@ int ceph_encode_encrypted_dname(struct inode *parent, char *buf, int elen)
 	}
 
 	/* base64 encode the encrypted name */
-	elen = ceph_base64_encode(cryptbuf, len, p);
+	elen = base64_encode(cryptbuf, len, p, false, BASE64_IMAP);
 	doutc(cl, "base64-encoded ciphertext name = %.*s\n", elen, p);
 
 	/* To understand the 240 limit, see CEPH_NOHASH_NAME_MAX comments */
@@ -410,7 +357,8 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
 			tname = &_tname;
 		}
 
-		declen = ceph_base64_decode(name, name_len, tname->name);
+		declen = base64_decode(name, name_len,
+				       tname->name, false, BASE64_IMAP);
 		if (declen <= 0) {
 			ret = -EIO;
 			goto out;
@@ -424,7 +372,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
 
 	ret = fscrypt_fname_disk_to_usr(dir, 0, 0, &iname, oname);
 	if (!ret && (dir != fname->dir)) {
-		char tmp_buf[CEPH_BASE64_CHARS(NAME_MAX)];
+		char tmp_buf[BASE64_CHARS(NAME_MAX)];
 
 		name_len = snprintf(tmp_buf, sizeof(tmp_buf), "_%.*s_%ld",
 				    oname->len, oname->name, dir->i_ino);
diff --git a/fs/ceph/crypto.h b/fs/ceph/crypto.h
index 23612b2e9..b748e2060 100644
--- a/fs/ceph/crypto.h
+++ b/fs/ceph/crypto.h
@@ -8,6 +8,7 @@
 
 #include <crypto/sha2.h>
 #include <linux/fscrypt.h>
+#include <linux/base64.h>
 
 #define CEPH_FSCRYPT_BLOCK_SHIFT   12
 #define CEPH_FSCRYPT_BLOCK_SIZE    (_AC(1, UL) << CEPH_FSCRYPT_BLOCK_SHIFT)
@@ -89,11 +90,6 @@ static inline u32 ceph_fscrypt_auth_len(struct ceph_fscrypt_auth *fa)
  */
 #define CEPH_NOHASH_NAME_MAX (180 - SHA256_DIGEST_SIZE)
 
-#define CEPH_BASE64_CHARS(nbytes) DIV_ROUND_UP((nbytes) * 4, 3)
-
-int ceph_base64_encode(const u8 *src, int srclen, char *dst);
-int ceph_base64_decode(const char *src, int srclen, u8 *dst);
-
 void ceph_fscrypt_set_ops(struct super_block *sb);
 
 void ceph_fscrypt_free_dummy_policy(struct ceph_fs_client *fsc);
diff --git a/fs/ceph/dir.c b/fs/ceph/dir.c
index 8478e7e75..25045d817 100644
--- a/fs/ceph/dir.c
+++ b/fs/ceph/dir.c
@@ -998,13 +998,14 @@ static int prep_encrypted_symlink_target(struct ceph_mds_request *req,
 	if (err)
 		goto out;
 
-	req->r_path2 = kmalloc(CEPH_BASE64_CHARS(osd_link.len) + 1, GFP_KERNEL);
+	req->r_path2 = kmalloc(BASE64_CHARS(osd_link.len) + 1, GFP_KERNEL);
 	if (!req->r_path2) {
 		err = -ENOMEM;
 		goto out;
 	}
 
-	len = ceph_base64_encode(osd_link.name, osd_link.len, req->r_path2);
+	len = base64_encode(osd_link.name, osd_link.len,
+			    req->r_path2, false, BASE64_IMAP);
 	req->r_path2[len] = '\0';
 out:
 	fscrypt_fname_free_buffer(&osd_link);
diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
index fc543075b..d06fb76fc 100644
--- a/fs/ceph/inode.c
+++ b/fs/ceph/inode.c
@@ -911,7 +911,7 @@ static int decode_encrypted_symlink(struct ceph_mds_client *mdsc,
 	if (!sym)
 		return -ENOMEM;
 
-	declen = ceph_base64_decode(encsym, enclen, sym);
+	declen = base64_decode(encsym, enclen, sym, false, BASE64_IMAP);
 	if (declen < 0) {
 		pr_err_client(cl,
 			"can't decode symlink (%d). Content: %.*s\n",
-- 
2.34.1

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andrew Morton]

On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:

> This series introduces a generic Base64 encoder/decoder to the kernel
> library, eliminating duplicated implementations and delivering significant
> performance improvements.
> 
> The Base64 API has been extended to support multiple variants (Standard,
> URL-safe, and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes
> a variant parameter and an option to control padding. As part of this
> series, users are migrated to the new interface while preserving their
> specific formats: fscrypt now uses BASE64_URLSAFE, Ceph uses BASE64_IMAP,
> and NVMe is updated to BASE64_STD.
> 
> On the encoder side, the implementation processes input in 3-byte blocks,
> mapping 24 bits directly to 4 output symbols. This avoids bit-by-bit
> streaming and reduces loop overhead, achieving about a 2.7x speedup compared
> to previous implementations.
> 
> On the decoder side, replace strchr() lookups with per-variant reverse tables
> and process input in 4-character groups. Each group is mapped to numeric values
> and combined into 3 bytes. Padded and unpadded forms are validated explicitly,
> rejecting invalid '=' usage and enforcing tail rules.

Looks like wonderful work, thanks.  And it's good to gain a selftest
for this code.

> This improves throughput by ~43-52x.

Well that isn't a thing we see every day.

: Decode:
:   64B   ~1530ns  ->  ~80ns    (~19.1x)
:   1KB  ~27726ns  -> ~1239ns   (~22.4x)


: Encode:
:   64B   ~90ns   -> ~32ns   (~2.8x)
:   1KB  ~1332ns  -> ~510ns  (~2.6x)
: 
: Decode:
:   64B  ~1530ns  -> ~35ns   (~43.7x)
:   1KB ~27726ns  -> ~530ns  (~52.3x)


: This change also improves performance: encoding is about 2.7x faster and
: decoding achieves 43-52x speedups compared to the previous implementation.

: This change also improves performance: encoding is about 2.7x faster and
: decoding achieves 43-52x speedups compared to the previous local
: implementation.


Do any of these callers spend a sufficient amount of time in this
encoder/decoder for the above improvements to be observable/useful?


I'll add the series to mm.git's mm-nonmm-unstable branch to give it
linux-next exposure.  I ask the NVMe, ceph and fscrypt teams to check
the code and give it a test in the next few weeks, thanks.  

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:
> On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:
> 
> > This series introduces a generic Base64 encoder/decoder to the kernel
> > library, eliminating duplicated implementations and delivering significant
> > performance improvements.
> > 
> > The Base64 API has been extended to support multiple variants (Standard,
> > URL-safe, and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes
> > a variant parameter and an option to control padding. As part of this
> > series, users are migrated to the new interface while preserving their
> > specific formats: fscrypt now uses BASE64_URLSAFE, Ceph uses BASE64_IMAP,
> > and NVMe is updated to BASE64_STD.
> > 
> > On the encoder side, the implementation processes input in 3-byte blocks,
> > mapping 24 bits directly to 4 output symbols. This avoids bit-by-bit
> > streaming and reduces loop overhead, achieving about a 2.7x speedup compared
> > to previous implementations.
> > 
> > On the decoder side, replace strchr() lookups with per-variant reverse tables
> > and process input in 4-character groups. Each group is mapped to numeric values
> > and combined into 3 bytes. Padded and unpadded forms are validated explicitly,
> > rejecting invalid '=' usage and enforcing tail rules.
> 
> Looks like wonderful work, thanks.  And it's good to gain a selftest
> for this code.
> 
> > This improves throughput by ~43-52x.
> 
> Well that isn't a thing we see every day.

I agree with the judgement, the problem is that this broke drastically a build:

lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
   35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
   26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
      |                  ^
lib/base64.c:35:17: note: previous initialization is here
   35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
   25 |         [0 ... 255] = -1, \
      |                       ^~
...
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.


> : Decode:
> :   64B   ~1530ns  ->  ~80ns    (~19.1x)
> :   1KB  ~27726ns  -> ~1239ns   (~22.4x)
> 
> 
> : Encode:
> :   64B   ~90ns   -> ~32ns   (~2.8x)
> :   1KB  ~1332ns  -> ~510ns  (~2.6x)
> : 
> : Decode:
> :   64B  ~1530ns  -> ~35ns   (~43.7x)
> :   1KB ~27726ns  -> ~530ns  (~52.3x)
> 
> 
> : This change also improves performance: encoding is about 2.7x faster and
> : decoding achieves 43-52x speedups compared to the previous implementation.
> 
> : This change also improves performance: encoding is about 2.7x faster and
> : decoding achieves 43-52x speedups compared to the previous local
> : implementation.
> 
> 
> Do any of these callers spend a sufficient amount of time in this
> encoder/decoder for the above improvements to be observable/useful?
> 
> 
> I'll add the series to mm.git's mm-nonmm-unstable branch to give it
> linux-next exposure.  I ask the NVMe, ceph and fscrypt teams to check
> the code and give it a test in the next few weeks, thanks.  
> 

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Kuan-Wei Chiu]

+Cc David

Hi Guan-Chun,

If we need to respin this series, please Cc David when sending the next
version.

On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:
> On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:
> > On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:
> > 
> > > This series introduces a generic Base64 encoder/decoder to the kernel
> > > library, eliminating duplicated implementations and delivering significant
> > > performance improvements.
> > > 
> > > The Base64 API has been extended to support multiple variants (Standard,
> > > URL-safe, and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes
> > > a variant parameter and an option to control padding. As part of this
> > > series, users are migrated to the new interface while preserving their
> > > specific formats: fscrypt now uses BASE64_URLSAFE, Ceph uses BASE64_IMAP,
> > > and NVMe is updated to BASE64_STD.
> > > 
> > > On the encoder side, the implementation processes input in 3-byte blocks,
> > > mapping 24 bits directly to 4 output symbols. This avoids bit-by-bit
> > > streaming and reduces loop overhead, achieving about a 2.7x speedup compared
> > > to previous implementations.
> > > 
> > > On the decoder side, replace strchr() lookups with per-variant reverse tables
> > > and process input in 4-character groups. Each group is mapped to numeric values
> > > and combined into 3 bytes. Padded and unpadded forms are validated explicitly,
> > > rejecting invalid '=' usage and enforcing tail rules.
> > 
> > Looks like wonderful work, thanks.  And it's good to gain a selftest
> > for this code.
> > 
> > > This improves throughput by ~43-52x.
> > 
> > Well that isn't a thing we see every day.
> 
> I agree with the judgement, the problem is that this broke drastically a build:
> 
> lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
>    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
>       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
>    26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
>       |                  ^
> lib/base64.c:35:17: note: previous initialization is here
>    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
>       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
>    25 |         [0 ... 255] = -1, \
>       |                       ^~
> ...
> fatal error: too many errors emitted, stopping now [-ferror-limit=]
> 20 errors generated.
> 
Since I didn't notice this build failure, I guess this happens during a
W=1 build? Sorry for that. Maybe I should add W=1 compilation testing
to my checklist before sending patches in the future. I also got an
email from the kernel test robot with a duplicate initialization
warning from the sparse tool [1], pointing to the same code.

This implementation was based on David's previous suggestion [2] to
first default all entries to -1 and then set the values for the 64
character entries. This was to avoid expanding the large 256 * 3 table
and improve code readability.

Hi David,

Since I believe many people test and care about W=1 builds, I think we
need to find another way to avoid this warning? Perhaps we could
consider what you suggested:

#define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
	[ 0 ... '+'-1 ] = -1, \
	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
	[ '9'+1 ... 'A'-1 ] = -1, \
	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
	[ 'Z'+1 ... '_'-1 ] = -1, \
	[ '_' ] = val_under, \
	[ '_'+1 ... 'a'-1 ] = -1, \
	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
	[ 'z'+1 ... 255 ] = -1 \
}

Or should we just expand the 256 * 3 table as it was before?

[1]: https://lore.kernel.org/oe-kbuild-all/202511021343.107utehN-lkp@intel.com/
[2]: https://lore.kernel.org/lkml/20250928195736.71bec9ae@pumpkin/

Regards,
Kuan-Wei

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Mon, 3 Nov 2025 19:07:24 +0800
Kuan-Wei Chiu <visitorckw@gmail.com> wrote:

> +Cc David
> 
> Hi Guan-Chun,
> 
> If we need to respin this series, please Cc David when sending the next
> version.
> 
> On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:
> > On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:  
> > > On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:
> > >   
> > > > This series introduces a generic Base64 encoder/decoder to the kernel
> > > > library, eliminating duplicated implementations and delivering significant
> > > > performance improvements.
> > > > 
> > > > The Base64 API has been extended to support multiple variants (Standard,
> > > > URL-safe, and IMAP) as defined in RFC 4648 and RFC 3501. The API now takes
> > > > a variant parameter and an option to control padding. As part of this
> > > > series, users are migrated to the new interface while preserving their
> > > > specific formats: fscrypt now uses BASE64_URLSAFE, Ceph uses BASE64_IMAP,
> > > > and NVMe is updated to BASE64_STD.
> > > > 
> > > > On the encoder side, the implementation processes input in 3-byte blocks,
> > > > mapping 24 bits directly to 4 output symbols. This avoids bit-by-bit
> > > > streaming and reduces loop overhead, achieving about a 2.7x speedup compared
> > > > to previous implementations.
> > > > 
> > > > On the decoder side, replace strchr() lookups with per-variant reverse tables
> > > > and process input in 4-character groups. Each group is mapped to numeric values
> > > > and combined into 3 bytes. Padded and unpadded forms are validated explicitly,
> > > > rejecting invalid '=' usage and enforcing tail rules.  
> > > 
> > > Looks like wonderful work, thanks.  And it's good to gain a selftest
> > > for this code.
> > >   
> > > > This improves throughput by ~43-52x.  
> > > 
> > > Well that isn't a thing we see every day.  
> > 
> > I agree with the judgement, the problem is that this broke drastically a build:
> > 
> > lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
> >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
> >    26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
> >       |                  ^
> > lib/base64.c:35:17: note: previous initialization is here
> >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
> >    25 |         [0 ... 255] = -1, \
> >       |                       ^~
> > ...
> > fatal error: too many errors emitted, stopping now [-ferror-limit=]
> > 20 errors generated.
> >   
> Since I didn't notice this build failure, I guess this happens during a
> W=1 build? Sorry for that. Maybe I should add W=1 compilation testing
> to my checklist before sending patches in the future. I also got an
> email from the kernel test robot with a duplicate initialization
> warning from the sparse tool [1], pointing to the same code.
> 
> This implementation was based on David's previous suggestion [2] to
> first default all entries to -1 and then set the values for the 64
> character entries. This was to avoid expanding the large 256 * 3 table
> and improve code readability.
> 
> Hi David,
> 
> Since I believe many people test and care about W=1 builds,

Last time I tried a W=1 build it failed horribly because of 'type-limits'.
The kernel does that all the time - usually for its own error tests inside
#define and inline functions.
Certainly some of the changes I've seen to stop W=1 warnings are really
a bad idea - but that is a bit of a digression.

Warnings can be temporarily disabled using #pragma.
That might be the best thing to do here with this over-zealous warning.

This compiles on gcc and clang (even though the warnings have different names):
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Woverride-init"
int x[16] = { [0 ... 15] = -1, [5] = 5};
#pragma GCC diagnostic pop

> I think we need to find another way to avoid this warning?
> Perhaps we could consider what you suggested:
> 
> #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> 	[ 0 ... '+'-1 ] = -1, \
> 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> 	[ '9'+1 ... 'A'-1 ] = -1, \
> 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> 	[ 'Z'+1 ... '_'-1 ] = -1, \
> 	[ '_' ] = val_under, \
> 	[ '_'+1 ... 'a'-1 ] = -1, \
> 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> 	[ 'z'+1 ... 255 ] = -1 \
> }

I just checked, neither gcc nor clang allow empty ranges (eg [ 6 ... 5 ] = -1).
Which means the coder has to know which characters are adjacent as well
as getting the order right.
Basically avoiding the warning sucks.

> Or should we just expand the 256 * 3 table as it was before?

That has much the same issue - IIRC it relies on three big sequential lists.

The #pragma may be best - but doesn't solve sparse (unless it processes
them as well).

	David

> 
> [1]: https://lore.kernel.org/oe-kbuild-all/202511021343.107utehN-lkp@intel.com/
> [2]: https://lore.kernel.org/lkml/20250928195736.71bec9ae@pumpkin/
> 
> Regards,
> Kuan-Wei

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:
> On Mon, 3 Nov 2025 19:07:24 +0800
> Kuan-Wei Chiu <visitorckw@gmail.com> wrote:
> > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:
> > > On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:  
> > > > On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:

...

> > > > Looks like wonderful work, thanks.  And it's good to gain a selftest
> > > > for this code.
> > > >   
> > > > > This improves throughput by ~43-52x.  
> > > > 
> > > > Well that isn't a thing we see every day.  
> > > 
> > > I agree with the judgement, the problem is that this broke drastically a build:
> > > 
> > > lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
> > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
> > >    26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
> > >       |                  ^
> > > lib/base64.c:35:17: note: previous initialization is here
> > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
> > >    25 |         [0 ... 255] = -1, \
> > >       |                       ^~
> > > ...
> > > fatal error: too many errors emitted, stopping now [-ferror-limit=]
> > > 20 errors generated.
> > >   
> > Since I didn't notice this build failure, I guess this happens during a
> > W=1 build? Sorry for that. Maybe I should add W=1 compilation testing
> > to my checklist before sending patches in the future. I also got an
> > email from the kernel test robot with a duplicate initialization
> > warning from the sparse tool [1], pointing to the same code.
> > 
> > This implementation was based on David's previous suggestion [2] to
> > first default all entries to -1 and then set the values for the 64
> > character entries. This was to avoid expanding the large 256 * 3 table
> > and improve code readability.
> > 
> > Since I believe many people test and care about W=1 builds,
> 
> Last time I tried a W=1 build it failed horribly because of 'type-limits'.
> The kernel does that all the time - usually for its own error tests inside
> #define and inline functions.
> Certainly some of the changes I've seen to stop W=1 warnings are really
> a bad idea - but that is a bit of a digression.
> 
> Warnings can be temporarily disabled using #pragma.
> That might be the best thing to do here with this over-zealous warning.
> 
> This compiles on gcc and clang (even though the warnings have different names):
> #pragma GCC diagnostic push
> #pragma GCC diagnostic ignored "-Woverride-init"
> int x[16] = { [0 ... 15] = -1, [5] = 5};
> #pragma GCC diagnostic pop
> 
> > I think we need to find another way to avoid this warning?
> > Perhaps we could consider what you suggested:
> > 
> > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > 	[ 0 ... '+'-1 ] = -1, \
> > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > 	[ '_' ] = val_under, \
> > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > 	[ 'z'+1 ... 255 ] = -1 \
> > }
> 
> I just checked, neither gcc nor clang allow empty ranges (eg [ 6 ... 5 ] = -1).
> Which means the coder has to know which characters are adjacent as well
> as getting the order right.
> Basically avoiding the warning sucks.
> 
> > Or should we just expand the 256 * 3 table as it was before?
> 
> That has much the same issue - IIRC it relies on three big sequential lists.
> 
> The #pragma may be best - but doesn't solve sparse (unless it processes
> them as well).

Pragma will be hated. I believe there is a better way to do what you want. Let
me cook a PoC.

> > [1]: https://lore.kernel.org/oe-kbuild-all/202511021343.107utehN-lkp@intel.com/
> > [2]: https://lore.kernel.org/lkml/20250928195736.71bec9ae@pumpkin/

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Mon, Nov 03, 2025 at 04:41:41PM +0200, Andy Shevchenko wrote:
> On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:
> > On Mon, 3 Nov 2025 19:07:24 +0800
> > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:
> > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:
> > > > On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:  
> > > > > On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:

...

> > > > > Looks like wonderful work, thanks.  And it's good to gain a selftest
> > > > > for this code.
> > > > >   
> > > > > > This improves throughput by ~43-52x.  
> > > > > 
> > > > > Well that isn't a thing we see every day.  
> > > > 
> > > > I agree with the judgement, the problem is that this broke drastically a build:
> > > > 
> > > > lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
> > > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > > lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
> > > >    26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
> > > >       |                  ^
> > > > lib/base64.c:35:17: note: previous initialization is here
> > > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > > lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
> > > >    25 |         [0 ... 255] = -1, \
> > > >       |                       ^~
> > > > ...
> > > > fatal error: too many errors emitted, stopping now [-ferror-limit=]
> > > > 20 errors generated.
> > > >   
> > > Since I didn't notice this build failure, I guess this happens during a
> > > W=1 build? Sorry for that. Maybe I should add W=1 compilation testing
> > > to my checklist before sending patches in the future. I also got an
> > > email from the kernel test robot with a duplicate initialization
> > > warning from the sparse tool [1], pointing to the same code.
> > > 
> > > This implementation was based on David's previous suggestion [2] to
> > > first default all entries to -1 and then set the values for the 64
> > > character entries. This was to avoid expanding the large 256 * 3 table
> > > and improve code readability.
> > > 
> > > Since I believe many people test and care about W=1 builds,
> > 
> > Last time I tried a W=1 build it failed horribly because of 'type-limits'.
> > The kernel does that all the time - usually for its own error tests inside
> > #define and inline functions.
> > Certainly some of the changes I've seen to stop W=1 warnings are really
> > a bad idea - but that is a bit of a digression.
> > 
> > Warnings can be temporarily disabled using #pragma.
> > That might be the best thing to do here with this over-zealous warning.
> > 
> > This compiles on gcc and clang (even though the warnings have different names):
> > #pragma GCC diagnostic push
> > #pragma GCC diagnostic ignored "-Woverride-init"
> > int x[16] = { [0 ... 15] = -1, [5] = 5};
> > #pragma GCC diagnostic pop
> > 
> > > I think we need to find another way to avoid this warning?
> > > Perhaps we could consider what you suggested:
> > > 
> > > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > > 	[ 0 ... '+'-1 ] = -1, \
> > > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > > 	[ '_' ] = val_under, \
> > > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > > 	[ 'z'+1 ... 255 ] = -1 \
> > > }
> > 
> > I just checked, neither gcc nor clang allow empty ranges (eg [ 6 ... 5 ] = -1).
> > Which means the coder has to know which characters are adjacent as well
> > as getting the order right.
> > Basically avoiding the warning sucks.
> > 
> > > Or should we just expand the 256 * 3 table as it was before?
> > 
> > That has much the same issue - IIRC it relies on three big sequential lists.
> > 
> > The #pragma may be best - but doesn't solve sparse (unless it processes
> > them as well).
> 
> Pragma will be hated. I believe there is a better way to do what you want. Let
> me cook a PoC.

I tried locally several approaches and the best I can come up with is the pre-generated
(via Python script) pieces of C code that we can copy'n'paste instead of that shortened
form. So basically having a full 256 tables in the code is my suggestion to fix the build
issue. Alternatively we can generate that at run-time (on the first run) in
the similar way how prime_numbers.c does. The downside of such an approach is loosing
the const specifier, which I consider kinda important.

Btw, in the future here might be also the side-channel attack concerns appear, which would
require to reconsider the whole algo to get it constant-time execution.

> > > [1]: https://lore.kernel.org/oe-kbuild-all/202511021343.107utehN-lkp@intel.com/
> > > [2]: https://lore.kernel.org/lkml/20250928195736.71bec9ae@pumpkin/

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Mon, 3 Nov 2025 20:16:46 +0200
Andy Shevchenko <andriy.shevchenko@intel.com> wrote:

> On Mon, Nov 03, 2025 at 04:41:41PM +0200, Andy Shevchenko wrote:
> > On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:  
> > > On Mon, 3 Nov 2025 19:07:24 +0800
> > > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:  
> > > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:  
> > > > > On Fri, Oct 31, 2025 at 09:09:47PM -0700, Andrew Morton wrote:    
> > > > > > On Wed, 29 Oct 2025 18:17:25 +0800 Guan-Chun Wu <409411716@gms.tku.edu.tw> wrote:  
> 
> ...
> 
> > > > > > Looks like wonderful work, thanks.  And it's good to gain a selftest
> > > > > > for this code.
> > > > > >     
> > > > > > > This improves throughput by ~43-52x.    
> > > > > > 
> > > > > > Well that isn't a thing we see every day.    
> > > > > 
> > > > > I agree with the judgement, the problem is that this broke drastically a build:
> > > > > 
> > > > > lib/base64.c:35:17: error: initializer overrides prior initialization of this subobject [-Werror,-Winitializer-overrides]
> > > > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > > > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > lib/base64.c:26:11: note: expanded from macro 'BASE64_REV_INIT'
> > > > >    26 |         ['A'] =  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, \
> > > > >       |                  ^
> > > > > lib/base64.c:35:17: note: previous initialization is here
> > > > >    35 |         [BASE64_STD] = BASE64_REV_INIT('+', '/'),
> > > > >       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > lib/base64.c:25:16: note: expanded from macro 'BASE64_REV_INIT'
> > > > >    25 |         [0 ... 255] = -1, \
> > > > >       |                       ^~
> > > > > ...
> > > > > fatal error: too many errors emitted, stopping now [-ferror-limit=]
> > > > > 20 errors generated.
> > > > >     
> > > > Since I didn't notice this build failure, I guess this happens during a
> > > > W=1 build? Sorry for that. Maybe I should add W=1 compilation testing
> > > > to my checklist before sending patches in the future. I also got an
> > > > email from the kernel test robot with a duplicate initialization
> > > > warning from the sparse tool [1], pointing to the same code.
> > > > 
> > > > This implementation was based on David's previous suggestion [2] to
> > > > first default all entries to -1 and then set the values for the 64
> > > > character entries. This was to avoid expanding the large 256 * 3 table
> > > > and improve code readability.
> > > > 
> > > > Since I believe many people test and care about W=1 builds,  
> > > 
> > > Last time I tried a W=1 build it failed horribly because of 'type-limits'.
> > > The kernel does that all the time - usually for its own error tests inside
> > > #define and inline functions.
> > > Certainly some of the changes I've seen to stop W=1 warnings are really
> > > a bad idea - but that is a bit of a digression.
> > > 
> > > Warnings can be temporarily disabled using #pragma.
> > > That might be the best thing to do here with this over-zealous warning.
> > > 
> > > This compiles on gcc and clang (even though the warnings have different names):
> > > #pragma GCC diagnostic push
> > > #pragma GCC diagnostic ignored "-Woverride-init"
> > > int x[16] = { [0 ... 15] = -1, [5] = 5};
> > > #pragma GCC diagnostic pop
> > >   
> > > > I think we need to find another way to avoid this warning?
> > > > Perhaps we could consider what you suggested:
> > > > 
> > > > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > > > 	[ 0 ... '+'-1 ] = -1, \
> > > > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > > > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > > > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > > > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > > > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > > > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > > > 	[ '_' ] = val_under, \
> > > > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > > > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > > > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > > > 	[ 'z'+1 ... 255 ] = -1 \
> > > > }  
> > > 
> > > I just checked, neither gcc nor clang allow empty ranges (eg [ 6 ... 5 ] = -1).
> > > Which means the coder has to know which characters are adjacent as well
> > > as getting the order right.
> > > Basically avoiding the warning sucks.
> > >   
> > > > Or should we just expand the 256 * 3 table as it was before?  
> > > 
> > > That has much the same issue - IIRC it relies on three big sequential lists.
> > > 
> > > The #pragma may be best - but doesn't solve sparse (unless it processes
> > > them as well).  
> > 
> > Pragma will be hated.

They have been used in a few other places.
and to disable more 'useful' warnings.

> > I believe there is a better way to do what you want. Let me cook a PoC.  
> 
> I tried locally several approaches and the best I can come up with is the pre-generated
> (via Python script) pieces of C code that we can copy'n'paste instead of that shortened
> form. So basically having a full 256 tables in the code is my suggestion to fix the build
> issue. Alternatively we can generate that at run-time (on the first run) in
> the similar way how prime_numbers.c does. The downside of such an approach is loosing
> the const specifier, which I consider kinda important.
> 
> Btw, in the future here might be also the side-channel attack concerns appear, which would
> require to reconsider the whole algo to get it constant-time execution.

The array lookup version is 'reasonably' time constant.
One option is to offset all the array entries by 1 and subtract 1 after reading the entry.
That means that the 'error' characters have zero in the array (not -1).
At least the compiler won't error that!
The extra 'subtract 1' is probably just measurable.

But I'd consider raising a bug on gcc :-)
One of the uses of ranged designated initialisers for arrays is to change the
default value - as been done here.
It shouldn't cause a warning.

	David

> 
> > > > [1]: https://lore.kernel.org/oe-kbuild-all/202511021343.107utehN-lkp@intel.com/
> > > > [2]: https://lore.kernel.org/lkml/20250928195736.71bec9ae@pumpkin/  
> 

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Mon, Nov 03, 2025 at 07:29:08PM +0000, David Laight wrote:
> On Mon, 3 Nov 2025 20:16:46 +0200
> Andy Shevchenko <andriy.shevchenko@intel.com> wrote:
> > On Mon, Nov 03, 2025 at 04:41:41PM +0200, Andy Shevchenko wrote:
> > > On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:  

...

> > > Pragma will be hated.
> 
> They have been used in a few other places.
> and to disable more 'useful' warnings.

You can go with pragma, but even though it just hides the potential issues.
Not my choice.

> > > I believe there is a better way to do what you want. Let me cook a PoC.  
> > 
> > I tried locally several approaches and the best I can come up with is the pre-generated
> > (via Python script) pieces of C code that we can copy'n'paste instead of that shortened
> > form. So basically having a full 256 tables in the code is my suggestion to fix the build
> > issue. Alternatively we can generate that at run-time (on the first run) in
> > the similar way how prime_numbers.c does. The downside of such an approach is loosing
> > the const specifier, which I consider kinda important.
> > 
> > Btw, in the future here might be also the side-channel attack concerns appear, which would
> > require to reconsider the whole algo to get it constant-time execution.
> 
> The array lookup version is 'reasonably' time constant.

The array doesn't fit the cacheline.

> One option is to offset all the array entries by 1 and subtract 1 after reading the entry.

Yes, I was thinking of it, but found a bit weird.

> That means that the 'error' characters have zero in the array (not -1).
> At least the compiler won't error that!
> The extra 'subtract 1' is probably just measurable.

> But I'd consider raising a bug on gcc :-)

And clang? :-)

> One of the uses of ranged designated initialisers for arrays is to change the
> default value - as been done here.
> It shouldn't cause a warning.

This is prone to mistakes when it's not the default rewrite. I fixed already
twice such an issue in drivers/hid in the past few months.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Mon, 3 Nov 2025 21:37:17 +0200
Andy Shevchenko <andriy.shevchenko@intel.com> wrote:

> On Mon, Nov 03, 2025 at 07:29:08PM +0000, David Laight wrote:
> > On Mon, 3 Nov 2025 20:16:46 +0200
> > Andy Shevchenko <andriy.shevchenko@intel.com> wrote:  
> > > On Mon, Nov 03, 2025 at 04:41:41PM +0200, Andy Shevchenko wrote:  
> > > > On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:    
> 
> ...
> 
> > > > Pragma will be hated.  
> > 
> > They have been used in a few other places.
> > and to disable more 'useful' warnings.  
> 
> You can go with pragma, but even though it just hides the potential issues.
> Not my choice.

In this case you really want the version that has '[ 0 .. 255 ] = -1,',
everything else is unreadable and difficult to easily verify.

> 
> > > > I believe there is a better way to do what you want. Let me cook a PoC.    
> > > 
> > > I tried locally several approaches and the best I can come up with is the pre-generated
> > > (via Python script) pieces of C code that we can copy'n'paste instead of that shortened
> > > form. So basically having a full 256 tables in the code is my suggestion to fix the build
> > > issue. Alternatively we can generate that at run-time (on the first run) in
> > > the similar way how prime_numbers.c does. The downside of such an approach is loosing
> > > the const specifier, which I consider kinda important.
> > > 
> > > Btw, in the future here might be also the side-channel attack concerns appear, which would
> > > require to reconsider the whole algo to get it constant-time execution.  
> > 
> > The array lookup version is 'reasonably' time constant.  
> 
> The array doesn't fit the cacheline.

Ignoring all the error characters it is 2 (64 byte) cache lines (if aligned
on a 32 byte boundary).
They'll both be resident for any sane input, I doubt an attacker can determine
when the second one is loaded.
In any case you can load both at the start just to make sure.

> 
> > One option is to offset all the array entries by 1 and subtract 1 after reading the entry.  
> 
> Yes, I was thinking of it, but found a bit weird.
> 
> > That means that the 'error' characters have zero in the array (not -1).
> > At least the compiler won't error that!
> > The extra 'subtract 1' is probably just measurable.  
> 
> > But I'd consider raising a bug on gcc :-)  
> 
> And clang? :-)

clang is probably easier to get fixed.
The warning can be disabled for 'old' compilers - only one build 'tool'
needs to detect errors.

One solution is to disable the warnings in the compilers, but get sparse
(which I think is easier to change?) to do a sane check that allows
the entire array to default to non-zero while still checking for
other errors.

> > One of the uses of ranged designated initialisers for arrays is to change the
> > default value - as been done here.
> > It shouldn't cause a warning.  
> 
> This is prone to mistakes when it's not the default rewrite. I fixed already
> twice such an issue in drivers/hid in the past few months.

I was thinking that if the first initialiser is [ low ... high ] = value
then it should be valid to change any value.
I'm not sure what you fixed, clearly [ 4 ] = 5, [ 4 ] = 6, is an error,
but it might be sane to allow any update of a 'range' initialiser.

	David

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andrew Morton]

On Mon, 3 Nov 2025 19:07:24 +0800 Kuan-Wei Chiu <visitorckw@gmail.com> wrote:

> Since I believe many people test and care about W=1 builds, I think we
> need to find another way to avoid this warning? Perhaps we could
> consider what you suggested:

I'll drop the v4 series from mm.git, thanks.

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Mon, Nov 03, 2025 at 10:32:55PM +0000, David Laight wrote:
> On Mon, 3 Nov 2025 21:37:17 +0200
> Andy Shevchenko <andriy.shevchenko@intel.com> wrote:
> > On Mon, Nov 03, 2025 at 07:29:08PM +0000, David Laight wrote:
> > > On Mon, 3 Nov 2025 20:16:46 +0200
> > > Andy Shevchenko <andriy.shevchenko@intel.com> wrote:  
> > > > On Mon, Nov 03, 2025 at 04:41:41PM +0200, Andy Shevchenko wrote:  
> > > > > On Mon, Nov 03, 2025 at 01:22:13PM +0000, David Laight wrote:    

...

> > > > > Pragma will be hated.  
> > > 
> > > They have been used in a few other places.
> > > and to disable more 'useful' warnings.  
> > 
> > You can go with pragma, but even though it just hides the potential issues.
> > Not my choice.
> 
> In this case you really want the version that has '[ 0 .. 255 ] = -1,',
> everything else is unreadable and difficult to easily verify.

No, if it's a generated via a helper script.

> > > > > I believe there is a better way to do what you want. Let me cook a PoC.    
> > > > 
> > > > I tried locally several approaches and the best I can come up with is the pre-generated
> > > > (via Python script) pieces of C code that we can copy'n'paste instead of that shortened
> > > > form. So basically having a full 256 tables in the code is my suggestion to fix the build
> > > > issue. Alternatively we can generate that at run-time (on the first run) in
> > > > the similar way how prime_numbers.c does. The downside of such an approach is loosing
> > > > the const specifier, which I consider kinda important.
> > > > 
> > > > Btw, in the future here might be also the side-channel attack concerns appear, which would
> > > > require to reconsider the whole algo to get it constant-time execution.  
> > > 
> > > The array lookup version is 'reasonably' time constant.  
> > 
> > The array doesn't fit the cacheline.
> 
> Ignoring all the error characters it is 2 (64 byte) cache lines (if aligned
> on a 32 byte boundary).
> They'll both be resident for any sane input, I doubt an attacker can determine
> when the second one is loaded.
> In any case you can load both at the start just to make sure.

> > > One option is to offset all the array entries by 1 and subtract 1 after reading the entry.  
> > 
> > Yes, I was thinking of it, but found a bit weird.
> > 
> > > That means that the 'error' characters have zero in the array (not -1).
> > > At least the compiler won't error that!
> > > The extra 'subtract 1' is probably just measurable.  
> > 
> > > But I'd consider raising a bug on gcc :-)  
> > 
> > And clang? :-)
> 
> clang is probably easier to get fixed.
> The warning can be disabled for 'old' compilers - only one build 'tool'
> needs to detect errors.
> 
> One solution is to disable the warnings in the compilers, but get sparse
> (which I think is easier to change?) to do a sane check that allows
> the entire array to default to non-zero while still checking for
> other errors.
> 
> > > One of the uses of ranged designated initialisers for arrays is to change the
> > > default value - as been done here.
> > > It shouldn't cause a warning.  
> > 
> > This is prone to mistakes when it's not the default rewrite. I fixed already
> > twice such an issue in drivers/hid in the past few months.
> 
> I was thinking that if the first initialiser is [ low ... high ] = value
> then it should be valid to change any value.
> I'm not sure what you fixed, clearly [ 4 ] = 5, [ 4 ] = 6, is an error,
> but it might be sane to allow any update of a 'range' initialiser.

You can check a Git history for that.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Mon, Nov 03, 2025 at 05:27:39PM -0800, Andrew Morton wrote:
> On Mon, 3 Nov 2025 19:07:24 +0800 Kuan-Wei Chiu <visitorckw@gmail.com> wrote:
> 
> > Since I believe many people test and care about W=1 builds, I think we
> > need to find another way to avoid this warning? Perhaps we could
> > consider what you suggested:
> 
> I'll drop the v4 series from mm.git, thanks.

Thanks! I haven't got yet a newest Linux Next, but I think tomorrow it will be
okay for sure.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Mon, 3 Nov 2025 19:07:24 +0800
Kuan-Wei Chiu <visitorckw@gmail.com> wrote:

> +Cc David
> 
> Hi Guan-Chun,
> 
> If we need to respin this series, please Cc David when sending the next
> version.
> 
> On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:
...
> Hi David,
> 
> Since I believe many people test and care about W=1 builds, I think we
> need to find another way to avoid this warning? Perhaps we could
> consider what you suggested:
> 
> #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> 	[ 0 ... '+'-1 ] = -1, \
> 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> 	[ '9'+1 ... 'A'-1 ] = -1, \
> 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> 	[ 'Z'+1 ... '_'-1 ] = -1, \
> 	[ '_' ] = val_under, \
> 	[ '_'+1 ... 'a'-1 ] = -1, \
> 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> 	[ 'z'+1 ... 255 ] = -1 \
> }

I've a slightly better version:

#define INIT_62_63(ch, ch_62, ch_63) \
	[ ch ] = ch == ch_62 ? 62 : ch == ch_63 ? 63 : -1

#define BASE64_REV_INIT(ch_62, ch_63) { \
	[ 0 ... '0' - 6 ] = -1, \
	INIT_62_63('+', ch_62, ch_63), \
	INIT_62_63(',', ch_62, ch_63), \
	INIT_62_63('-', ch_62, ch_63), \
	INIT_62_63('.', ch_62, ch_63), \
	INIT_62_63('/', ch_62, ch_63), \
	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
	[ '9' + 1 ... 'A' - 1 ] = -1, \
	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
	[ 'Z' + 1 ... '_' - 1 ] = -1, \
	INIT_62_63('_', ch_62, ch_63), \
	[ '_' + 1 ... 'a' - 1 ] = -1, \
	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
	[ 'z' + 1 ... 255 ] = -1 \
}

that only requires that INIT_62_63() be used for all the characters
that are used for 62 and 63 - it can be used for extra ones (eg '.').
If some code wants to use different characters; the -1 need replacing
with INIT_62_63() but nothing else has to be changed.

I used '0' - 6 (rather than '+' - 1 - or any other expression for 0x2a)
to (possibly) make the table obviously correct without referring to the
ascii code table.

	David

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Tue, Nov 04, 2025 at 09:03:26AM +0000, David Laight wrote:
> On Mon, 3 Nov 2025 19:07:24 +0800
> Kuan-Wei Chiu <visitorckw@gmail.com> wrote:
> > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:

...

> > Since I believe many people test and care about W=1 builds, I think we
> > need to find another way to avoid this warning? Perhaps we could
> > consider what you suggested:
> > 
> > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > 	[ 0 ... '+'-1 ] = -1, \
> > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > 	[ '_' ] = val_under, \
> > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > 	[ 'z'+1 ... 255 ] = -1 \
> > }
> 
> I've a slightly better version:
> 
> #define INIT_62_63(ch, ch_62, ch_63) \
> 	[ ch ] = ch == ch_62 ? 62 : ch == ch_63 ? 63 : -1
> 
> #define BASE64_REV_INIT(ch_62, ch_63) { \
> 	[ 0 ... '0' - 6 ] = -1, \
> 	INIT_62_63('+', ch_62, ch_63), \
> 	INIT_62_63(',', ch_62, ch_63), \
> 	INIT_62_63('-', ch_62, ch_63), \
> 	INIT_62_63('.', ch_62, ch_63), \
> 	INIT_62_63('/', ch_62, ch_63), \
> 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> 	[ '9' + 1 ... 'A' - 1 ] = -1, \
> 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> 	[ 'Z' + 1 ... '_' - 1 ] = -1, \
> 	INIT_62_63('_', ch_62, ch_63), \
> 	[ '_' + 1 ... 'a' - 1 ] = -1, \
> 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> 	[ 'z' + 1 ... 255 ] = -1 \
> }
> 
> that only requires that INIT_62_63() be used for all the characters
> that are used for 62 and 63 - it can be used for extra ones (eg '.').
> If some code wants to use different characters; the -1 need replacing
> with INIT_62_63() but nothing else has to be changed.
> 
> I used '0' - 6 (rather than '+' - 1 - or any other expression for 0x2a)
> to (possibly) make the table obviously correct without referring to the
> ascii code table.

Still it's heavily depends on the values of '+,-./_' as an index that
makes it not so flexible.

Moreover this table is basically a dup of the strings in the first array.
Which already makes an unnecessary duplication. That's why I prefer to
see a script (one source of data) to generate the header or something like
this to have the tables and strings robust against typos.

The above is simply an unreadable mess.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Tue, 4 Nov 2025 11:48:57 +0200
Andy Shevchenko <andriy.shevchenko@intel.com> wrote:

> On Tue, Nov 04, 2025 at 09:03:26AM +0000, David Laight wrote:
> > On Mon, 3 Nov 2025 19:07:24 +0800
> > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:  
> > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:  
> 
> ...
> 
> > > Since I believe many people test and care about W=1 builds, I think we
> > > need to find another way to avoid this warning? Perhaps we could
> > > consider what you suggested:
> > > 
> > > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > > 	[ 0 ... '+'-1 ] = -1, \
> > > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > > 	[ '_' ] = val_under, \
> > > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > > 	[ 'z'+1 ... 255 ] = -1 \
> > > }  
> > 
> > I've a slightly better version:
> > 
> > #define INIT_62_63(ch, ch_62, ch_63) \
> > 	[ ch ] = ch == ch_62 ? 62 : ch == ch_63 ? 63 : -1
> > 
> > #define BASE64_REV_INIT(ch_62, ch_63) { \
> > 	[ 0 ... '0' - 6 ] = -1, \
> > 	INIT_62_63('+', ch_62, ch_63), \
> > 	INIT_62_63(',', ch_62, ch_63), \
> > 	INIT_62_63('-', ch_62, ch_63), \
> > 	INIT_62_63('.', ch_62, ch_63), \
> > 	INIT_62_63('/', ch_62, ch_63), \
> > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > 	[ '9' + 1 ... 'A' - 1 ] = -1, \
> > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > 	[ 'Z' + 1 ... '_' - 1 ] = -1, \
> > 	INIT_62_63('_', ch_62, ch_63), \
> > 	[ '_' + 1 ... 'a' - 1 ] = -1, \
> > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > 	[ 'z' + 1 ... 255 ] = -1 \
> > }
> > 
> > that only requires that INIT_62_63() be used for all the characters
> > that are used for 62 and 63 - it can be used for extra ones (eg '.').
> > If some code wants to use different characters; the -1 need replacing
> > with INIT_62_63() but nothing else has to be changed.
> > 
> > I used '0' - 6 (rather than '+' - 1 - or any other expression for 0x2a)
> > to (possibly) make the table obviously correct without referring to the
> > ascii code table.  
> 
> Still it's heavily depends on the values of '+,-./_' as an index that
> makes it not so flexible.

How about this one?
#define INIT_1(v, ch_lo, ch_hi, off, ch_62, ch_63) \
	[ v ] = ((v) >= ch_lo && (v) <= ch_hi) ? (v) - ch_lo + off \
		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
#define INIT_2(v, ...) INIT_1(v, __VA_ARGS__), INIT_1((v) + 1, __VA_ARGS__)
#define INIT_4(v, ...) INIT_2(v, __VA_ARGS__), INIT_2((v) + 2, __VA_ARGS__)
#define INIT_8(v, ...) INIT_4(v, __VA_ARGS__), INIT_4((v) + 4, __VA_ARGS__)
#define INIT_16(v, ...) INIT_8(v, __VA_ARGS__), INIT_8((v) + 8, __VA_ARGS__)
#define INIT_32(v, ...) INIT_16(v, __VA_ARGS__), INIT_16((v) + 16, __VA_ARGS__)

#define BASE64_REV_INIT(ch_62, ch_63) { \
	[ 0 ... 0x1f ] = -1, \
	INIT_32(0x20, '0', '9', 0, ch_62, ch_63), \
	INIT_32(0x40, 'A', 'Z', 10, ch_62, ch_63), \
	INIT_32(0x60, 'a', 'z', 26, ch_62, ch_63), \
	[ 0x80 ... 0xff ] = -1 }

which gets the pre-processor to do all the work.
ch_62 and ch_63 can be any printable characters.

Note that the #define names are all in a .c file - so don't need any
kind of namespace protection.
They can also all be #undef after the initialiser.

> Moreover this table is basically a dup of the strings in the first array.
> Which already makes an unnecessary duplication.

That is what the self tests are for.

> That's why I prefer to
> see a script (one source of data) to generate the header or something like
> this to have the tables and strings robust against typos.

We have to differ on that one.
Especially in cases (like this) where generating that data is reasonably trivial.

	David

> 
> The above is simply an unreadable mess.
> 

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Andy Shevchenko]

On Wed, Nov 05, 2025 at 09:48:27AM +0000, David Laight wrote:
> On Tue, 4 Nov 2025 11:48:57 +0200
> Andy Shevchenko <andriy.shevchenko@intel.com> wrote:
> > On Tue, Nov 04, 2025 at 09:03:26AM +0000, David Laight wrote:
> > > On Mon, 3 Nov 2025 19:07:24 +0800
> > > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:  
> > > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:  

...

> > > > Since I believe many people test and care about W=1 builds, I think we
> > > > need to find another way to avoid this warning? Perhaps we could
> > > > consider what you suggested:
> > > > 
> > > > #define BASE64_REV_INIT(val_plus, val_comma, val_minus, val_slash, val_under) { \
> > > > 	[ 0 ... '+'-1 ] = -1, \
> > > > 	[ '+' ] = val_plus, val_comma, val_minus, -1, val_slash, \
> > > > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > > > 	[ '9'+1 ... 'A'-1 ] = -1, \
> > > > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > > > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > > > 	[ 'Z'+1 ... '_'-1 ] = -1, \
> > > > 	[ '_' ] = val_under, \
> > > > 	[ '_'+1 ... 'a'-1 ] = -1, \
> > > > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > > > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > > > 	[ 'z'+1 ... 255 ] = -1 \
> > > > }  
> > > 
> > > I've a slightly better version:
> > > 
> > > #define INIT_62_63(ch, ch_62, ch_63) \
> > > 	[ ch ] = ch == ch_62 ? 62 : ch == ch_63 ? 63 : -1
> > > 
> > > #define BASE64_REV_INIT(ch_62, ch_63) { \
> > > 	[ 0 ... '0' - 6 ] = -1, \
> > > 	INIT_62_63('+', ch_62, ch_63), \
> > > 	INIT_62_63(',', ch_62, ch_63), \
> > > 	INIT_62_63('-', ch_62, ch_63), \
> > > 	INIT_62_63('.', ch_62, ch_63), \
> > > 	INIT_62_63('/', ch_62, ch_63), \
> > > 	[ '0' ] = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, \
> > > 	[ '9' + 1 ... 'A' - 1 ] = -1, \
> > > 	[ 'A' ] = 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, \
> > > 		  23, 24, 25, 26, 27, 28, 28, 30, 31, 32, 33, 34, 35, \
> > > 	[ 'Z' + 1 ... '_' - 1 ] = -1, \
> > > 	INIT_62_63('_', ch_62, ch_63), \
> > > 	[ '_' + 1 ... 'a' - 1 ] = -1, \
> > > 	[ 'a' ] = 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, \
> > > 		  49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, \
> > > 	[ 'z' + 1 ... 255 ] = -1 \
> > > }
> > > 
> > > that only requires that INIT_62_63() be used for all the characters
> > > that are used for 62 and 63 - it can be used for extra ones (eg '.').
> > > If some code wants to use different characters; the -1 need replacing
> > > with INIT_62_63() but nothing else has to be changed.
> > > 
> > > I used '0' - 6 (rather than '+' - 1 - or any other expression for 0x2a)
> > > to (possibly) make the table obviously correct without referring to the
> > > ascii code table.  
> > 
> > Still it's heavily depends on the values of '+,-./_' as an index that
> > makes it not so flexible.
> 
> How about this one?

Better than previous one(s) but quite cryptic to understand. Will need a
comment explaining the logic behind, if we go this way.

> #define INIT_1(v, ch_lo, ch_hi, off, ch_62, ch_63) \
> 	[ v ] = ((v) >= ch_lo && (v) <= ch_hi) ? (v) - ch_lo + off \
> 		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
> #define INIT_2(v, ...) INIT_1(v, __VA_ARGS__), INIT_1((v) + 1, __VA_ARGS__)
> #define INIT_4(v, ...) INIT_2(v, __VA_ARGS__), INIT_2((v) + 2, __VA_ARGS__)
> #define INIT_8(v, ...) INIT_4(v, __VA_ARGS__), INIT_4((v) + 4, __VA_ARGS__)
> #define INIT_16(v, ...) INIT_8(v, __VA_ARGS__), INIT_8((v) + 8, __VA_ARGS__)
> #define INIT_32(v, ...) INIT_16(v, __VA_ARGS__), INIT_16((v) + 16, __VA_ARGS__)
> 
> #define BASE64_REV_INIT(ch_62, ch_63) { \
> 	[ 0 ... 0x1f ] = -1, \
> 	INIT_32(0x20, '0', '9', 0, ch_62, ch_63), \
> 	INIT_32(0x40, 'A', 'Z', 10, ch_62, ch_63), \
> 	INIT_32(0x60, 'a', 'z', 26, ch_62, ch_63), \
> 	[ 0x80 ... 0xff ] = -1 }
> 
> which gets the pre-processor to do all the work.
> ch_62 and ch_63 can be any printable characters.
> 
> Note that the #define names are all in a .c file - so don't need any
> kind of namespace protection.

> They can also all be #undef after the initialiser.

Yes, that's too.

> > Moreover this table is basically a dup of the strings in the first array.
> > Which already makes an unnecessary duplication.
> 
> That is what the self tests are for.
> 
> > That's why I prefer to
> > see a script (one source of data) to generate the header or something like
> > this to have the tables and strings robust against typos.
> 
> We have to differ on that one.
> Especially in cases (like this) where generating that data is reasonably trivial.
> 
> > The above is simply an unreadable mess.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[David Laight]

On Wed, 5 Nov 2025 16:13:45 +0200
Andy Shevchenko <andriy.shevchenko@intel.com> wrote:

> On Wed, Nov 05, 2025 at 09:48:27AM +0000, David Laight wrote:
> > On Tue, 4 Nov 2025 11:48:57 +0200
> > Andy Shevchenko <andriy.shevchenko@intel.com> wrote:  
> > > On Tue, Nov 04, 2025 at 09:03:26AM +0000, David Laight wrote:  
> > > > On Mon, 3 Nov 2025 19:07:24 +0800
> > > > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:    
> > > > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:    
> 
...
> > How about this one?  
> 
> Better than previous one(s) but quite cryptic to understand. Will need a
> comment explaining the logic behind, if we go this way.

My first version (of this version) had all three character ranges in the define:
so:
#define INIT_1(v, ch_62, ch_63) \
	[ v ] = (v) >= '0' && (v) <= '9' ? (v) - '0' \
		: (v) >= 'A' && (v) <= 'Z' ? (v) - 'A' + 10 \
		: (v) >= 'a' && (v) <= 'z' ? (v) - 'a' + 36 \
		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
Perhaps less cryptic - even if the .i line will be rather longer.
It could be replicated for all 256 bytes, but I think the range
initialisers are reasonable for the non-printable ranges.

I did wonder if the encode and decode lookup tables count be interleaved
and both initialisers generated from the same #define.
But I can't think of a way of generating 'x' and "X" from a #define parameter.
(I don't think "X"[0] is constant enough...)

	David

> 
> > #define INIT_1(v, ch_lo, ch_hi, off, ch_62, ch_63) \
> > 	[ v ] = ((v) >= ch_lo && (v) <= ch_hi) ? (v) - ch_lo + off \
> > 		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
> > #define INIT_2(v, ...) INIT_1(v, __VA_ARGS__), INIT_1((v) + 1, __VA_ARGS__)
> > #define INIT_4(v, ...) INIT_2(v, __VA_ARGS__), INIT_2((v) + 2, __VA_ARGS__)
> > #define INIT_8(v, ...) INIT_4(v, __VA_ARGS__), INIT_4((v) + 4, __VA_ARGS__)
> > #define INIT_16(v, ...) INIT_8(v, __VA_ARGS__), INIT_8((v) + 8, __VA_ARGS__)
> > #define INIT_32(v, ...) INIT_16(v, __VA_ARGS__), INIT_16((v) + 16, __VA_ARGS__)
> > 
> > #define BASE64_REV_INIT(ch_62, ch_63) { \
> > 	[ 0 ... 0x1f ] = -1, \
> > 	INIT_32(0x20, '0', '9', 0, ch_62, ch_63), \
> > 	INIT_32(0x40, 'A', 'Z', 10, ch_62, ch_63), \
> > 	INIT_32(0x60, 'a', 'z', 26, ch_62, ch_63), \
> > 	[ 0x80 ... 0xff ] = -1 }
> > 
> > which gets the pre-processor to do all the work.
> > ch_62 and ch_63 can be any printable characters.
> > 
> > Note that the #define names are all in a .c file - so don't need any
> > kind of namespace protection.  
> 
> > They can also all be #undef after the initialiser.  
> 
> Yes, that's too.
> 
> > > Moreover this table is basically a dup of the strings in the first array.
> > > Which already makes an unnecessary duplication.  
> > 
> > That is what the self tests are for.
> >   
> > > That's why I prefer to
> > > see a script (one source of data) to generate the header or something like
> > > this to have the tables and strings robust against typos.  
> > 
> > We have to differ on that one.
> > Especially in cases (like this) where generating that data is reasonably trivial.
> >   
> > > The above is simply an unreadable mess.  
> 

Re: [PATCH v4 0/6] lib/base64: add generic encoder/decoder, migrate users

[Guan-Chun Wu]

On Wed, Nov 05, 2025 at 02:38:20PM +0000, David Laight wrote:
> On Wed, 5 Nov 2025 16:13:45 +0200
> Andy Shevchenko <andriy.shevchenko@intel.com> wrote:
> 
> > On Wed, Nov 05, 2025 at 09:48:27AM +0000, David Laight wrote:
> > > On Tue, 4 Nov 2025 11:48:57 +0200
> > > Andy Shevchenko <andriy.shevchenko@intel.com> wrote:  
> > > > On Tue, Nov 04, 2025 at 09:03:26AM +0000, David Laight wrote:  
> > > > > On Mon, 3 Nov 2025 19:07:24 +0800
> > > > > Kuan-Wei Chiu <visitorckw@gmail.com> wrote:    
> > > > > > On Mon, Nov 03, 2025 at 11:24:35AM +0100, Andy Shevchenko wrote:    
> > 
> ...
> > > How about this one?  
> > 
> > Better than previous one(s) but quite cryptic to understand. Will need a
> > comment explaining the logic behind, if we go this way.
> 
> My first version (of this version) had all three character ranges in the define:
> so:
> #define INIT_1(v, ch_62, ch_63) \
> 	[ v ] = (v) >= '0' && (v) <= '9' ? (v) - '0' \
> 		: (v) >= 'A' && (v) <= 'Z' ? (v) - 'A' + 10 \
> 		: (v) >= 'a' && (v) <= 'z' ? (v) - 'a' + 36 \
> 		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
> Perhaps less cryptic - even if the .i line will be rather longer.
> It could be replicated for all 256 bytes, but I think the range
> initialisers are reasonable for the non-printable ranges.
> 
> I did wonder if the encode and decode lookup tables count be interleaved
> and both initialisers generated from the same #define.
> But I can't think of a way of generating 'x' and "X" from a #define parameter.
> (I don't think "X"[0] is constant enough...)
> 
> 	David
>

Thanks for your reply!
We’ll adopt the approach you suggested in the next version.

Best regards,
Guan-Chun

> > 
> > > #define INIT_1(v, ch_lo, ch_hi, off, ch_62, ch_63) \
> > > 	[ v ] = ((v) >= ch_lo && (v) <= ch_hi) ? (v) - ch_lo + off \
> > > 		: (v) == ch_62 ? 62 : (v) == ch_63 ? 63 : -1
> > > #define INIT_2(v, ...) INIT_1(v, __VA_ARGS__), INIT_1((v) + 1, __VA_ARGS__)
> > > #define INIT_4(v, ...) INIT_2(v, __VA_ARGS__), INIT_2((v) + 2, __VA_ARGS__)
> > > #define INIT_8(v, ...) INIT_4(v, __VA_ARGS__), INIT_4((v) + 4, __VA_ARGS__)
> > > #define INIT_16(v, ...) INIT_8(v, __VA_ARGS__), INIT_8((v) + 8, __VA_ARGS__)
> > > #define INIT_32(v, ...) INIT_16(v, __VA_ARGS__), INIT_16((v) + 16, __VA_ARGS__)
> > > 
> > > #define BASE64_REV_INIT(ch_62, ch_63) { \
> > > 	[ 0 ... 0x1f ] = -1, \
> > > 	INIT_32(0x20, '0', '9', 0, ch_62, ch_63), \
> > > 	INIT_32(0x40, 'A', 'Z', 10, ch_62, ch_63), \
> > > 	INIT_32(0x60, 'a', 'z', 26, ch_62, ch_63), \
> > > 	[ 0x80 ... 0xff ] = -1 }
> > > 
> > > which gets the pre-processor to do all the work.
> > > ch_62 and ch_63 can be any printable characters.
> > > 
> > > Note that the #define names are all in a .c file - so don't need any
> > > kind of namespace protection.  
> > 
> > > They can also all be #undef after the initialiser.  
> > 
> > Yes, that's too.
> > 
> > > > Moreover this table is basically a dup of the strings in the first array.
> > > > Which already makes an unnecessary duplication.  
> > > 
> > > That is what the self tests are for.
> > >   
> > > > That's why I prefer to
> > > > see a script (one source of data) to generate the header or something like
> > > > this to have the tables and strings robust against typos.  
> > > 
> > > We have to differ on that one.
> > > Especially in cases (like this) where generating that data is reasonably trivial.
> > >   
> > > > The above is simply an unreadable mess.  
> > 
>