[PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Ian Rogers]

Commit a339671db64b ("vdso: Switch get/put_unaligned() from packed struct
to memcpy()") switched __get_unaligned_t() and __put_unaligned_t() from
using a __packed struct to __builtin_memcpy() to avoid strict aliasing
undefined behavior.

On ARMv5 with GCC, __builtin_memcpy() may not be inlined and instead emit
a much slower out-of-line memcpy() call. Add CONFIG_VDSO_UNALIGNED_PACKED,
enabled on ARMv5 (CPU_32v5), to use the __packed struct implementation on
ARMv5 while keeping __builtin_memcpy() elsewhere.

Fixes: a339671db64b ("vdso: Switch get/put_unaligned() from packed struct to memcpy()")
Reported-by: Stefan Kerkmann <s.kerkmann@pengutronix.de>
Closes: https://lore.kernel.org/lkml/47f8c30b-e326-4ff0-a2bc-5683209a9e2d@pengutronix.de/
Signed-off-by: Ian Rogers <irogers@google.com>
---
 include/vdso/unaligned.h | 16 ++++++++++++++++
 lib/vdso/Kconfig         |  8 ++++++++
 2 files changed, 24 insertions(+)

diff --git a/include/vdso/unaligned.h b/include/vdso/unaligned.h
index 9076483c9fbb..726fd15efc82 100644
--- a/include/vdso/unaligned.h
+++ b/include/vdso/unaligned.h
@@ -4,6 +4,20 @@
 
 #include <linux/compiler_types.h>
 
+#ifdef CONFIG_VDSO_UNALIGNED_PACKED
+
+#define __get_unaligned_t(type, ptr) ({							\
+	const struct { type x; } __packed * __get_pptr = (typeof(__get_pptr))(ptr);	\
+	__get_pptr->x;									\
+})
+
+#define __put_unaligned_t(type, val, ptr) do {						\
+	struct { type x; } __packed * __put_pptr = (typeof(__put_pptr))(ptr);		\
+	__put_pptr->x = (val);								\
+} while (0)
+
+#else /* !CONFIG_VDSO_UNALIGNED_PACKED */
+
 /**
  * __get_unaligned_t - read an unaligned value from memory.
  * @type:	the type to load from the pointer.
@@ -41,4 +55,6 @@
 			 sizeof(__put_unaligned_val));			\
 } while (0)
 
+#endif /* CONFIG_VDSO_UNALIGNED_PACKED */
+
 #endif /* __VDSO_UNALIGNED_H */
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
index 597f5f0f9681..709eb193999f 100644
--- a/lib/vdso/Kconfig
+++ b/lib/vdso/Kconfig
@@ -23,3 +23,11 @@ config VDSO_GETRANDOM
 	select VDSO_DATASTORE
 	help
 	  Selected by architectures that support vDSO getrandom().
+
+config VDSO_UNALIGNED_PACKED
+	bool
+	default CPU_32v5
+	help
+	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
+	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
+	  out-of-line memcpy() calls instead of inlining byte accesses.
-- 
2.56.0.212.g129ede20d9-goog

[PATCH v1 2/2] tools headers: Update the vdso/unaligned.h copy with the kernel sources

[Ian Rogers]

To pick up the changes in:

  vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/include/vdso/unaligned.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/include/vdso/unaligned.h b/tools/include/vdso/unaligned.h
index 9076483c9fbb..726fd15efc82 100644
--- a/tools/include/vdso/unaligned.h
+++ b/tools/include/vdso/unaligned.h
@@ -4,6 +4,20 @@
 
 #include <linux/compiler_types.h>
 
+#ifdef CONFIG_VDSO_UNALIGNED_PACKED
+
+#define __get_unaligned_t(type, ptr) ({							\
+	const struct { type x; } __packed * __get_pptr = (typeof(__get_pptr))(ptr);	\
+	__get_pptr->x;									\
+})
+
+#define __put_unaligned_t(type, val, ptr) do {						\
+	struct { type x; } __packed * __put_pptr = (typeof(__put_pptr))(ptr);		\
+	__put_pptr->x = (val);								\
+} while (0)
+
+#else /* !CONFIG_VDSO_UNALIGNED_PACKED */
+
 /**
  * __get_unaligned_t - read an unaligned value from memory.
  * @type:	the type to load from the pointer.
@@ -41,4 +55,6 @@
 			 sizeof(__put_unaligned_val));			\
 } while (0)
 
+#endif /* CONFIG_VDSO_UNALIGNED_PACKED */
+
 #endif /* __VDSO_UNALIGNED_H */
-- 
2.56.0.212.g129ede20d9-goog

Re: [PATCH v1 2/2] tools headers: Update the vdso/unaligned.h copy with the kernel sources

[sashiko-bot]

> To pick up the changes in:
> 
>   vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5
> 
> Signed-off-by: Ian Rogers <irogers@google.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261019201257.3805356-1-irogers@google.com?part=2

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] vdso: VDSO_UNALIGNED_PACKED default omits non-ARMv5 strict-alignment targets
--

commit 1a3f0256f7d442ba6867246bd156ba9784b5b9a5
Author: Ian Rogers <irogers@google.com>

vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

This commit adds CONFIG_VDSO_UNALIGNED_PACKED, defaulted to CPU_32v5, to
restore the packed struct implementation of __get_unaligned_t() and
__put_unaligned_t() on ARMv5 where GCC may emit out-of-line memcpy() calls
for __builtin_memcpy().

> diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
> index 597f5f0f96819..709eb193999fd 100644
> --- a/lib/vdso/Kconfig
> +++ b/lib/vdso/Kconfig
> @@ -23,3 +23,11 @@ config VDSO_GETRANDOM
>  	select VDSO_DATASTORE
>  	help
>  	  Selected by architectures that support vDSO getrandom().
> +
> +config VDSO_UNALIGNED_PACKED
> +	bool
> +	default CPU_32v5
> +	help
> +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> +	  out-of-line memcpy() calls instead of inlining byte accesses.

[Severity: Medium]
Does defaulting VDSO_UNALIGNED_PACKED only to CPU_32v5 leave out other
strict-alignment targets that experience the same GCC __builtin_memcpy()
regression from commit a339671db64b ("vdso: Switch get/put_unaligned() from
packed struct to memcpy()")?

include/linux/unaligned.h includes include/vdso/unaligned.h to define
get_unaligned() and put_unaligned() across the entire kernel and boot
decompressors. On strict-alignment targets, GCC cannot fold
__builtin_memcpy() into a scalar access when the pointer alignment is
smaller than the type alignment, resulting in stack spills or out-of-line
memcpy() calls instead of the register shifts generated for a __packed
struct access.

In arch/arm/Makefile, ARMv3, ARMv4, and ARMv4T targets use separate Kconfig
symbols without selecting CPU_32v5, and !CONFIG_MMU builds pass
-mno-unaligned-access:

    ifeq ($(CONFIG_MMU),)
    CFLAGS_ABI	+=-mno-unaligned-access
    endif
    ...
    arch-$(CONFIG_CPU_32v5)		:=-march=armv5te
    arch-$(CONFIG_CPU_32v4T)	:=-march=armv4t
    arch-$(CONFIG_CPU_32v4)		:=-march=armv4
    arch-$(CONFIG_CPU_32v3)		:=-march=armv3m

In arch/arm/Kconfig, hardware unaligned access support is distinguished via
HAVE_EFFICIENT_UNALIGNED_ACCESS:

    select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU

Should VDSO_UNALIGNED_PACKED default to !HAVE_EFFICIENT_UNALIGNED_ACCESS (or
could include/vdso/unaligned.h use a single __packed __may_alias definition)
so that ARMv3, ARMv4, ARMv4T, ARM !MMU, and other strict-alignment
architectures are covered as well?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261019201257.3805356-1-irogers@google.com?part=1

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> +
> +config VDSO_UNALIGNED_PACKED
> +	bool
> +	default CPU_32v5
> +	help
> +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> +	  out-of-line memcpy() calls instead of inlining byte accesses.

As explained, I think this misrepresents the problem as Arm specific and
isn't the correct method for -munaligned-access targets on Arm, which
include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.

I still hope we can use a version that just produces the intended code
on both -munaligned-access and -mno-unaligned-access targets like the
old version did.

     Arnd

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On 20.10.2026 00:12:57, Arnd Bergmann wrote:
> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> > +
> > +config VDSO_UNALIGNED_PACKED
> > +	bool
> > +	default CPU_32v5
> > +	help
> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
>
> As explained, I think this misrepresents the problem as Arm specific and
> isn't the correct method for -munaligned-access targets on Arm, which
> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
>
> I still hope we can use a version that just produces the intended code
> on both -munaligned-access and -mno-unaligned-access targets like the
> old version did.

FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
make use of it?

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde          |
Embedded Linux                   | https://www.pengutronix.de |
Vertretung Nürnberg              | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-9   |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
> On 20.10.2026 00:12:57, Arnd Bergmann wrote:
>> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
>> > +
>> > +config VDSO_UNALIGNED_PACKED
>> > +	bool
>> > +	default CPU_32v5
>> > +	help
>> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
>> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
>> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
>>
>> As explained, I think this misrepresents the problem as Arm specific and
>> isn't the correct method for -munaligned-access targets on Arm, which
>> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
>>
>> I still hope we can use a version that just produces the intended code
>> on both -munaligned-access and -mno-unaligned-access targets like the
>> old version did.
>
> FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
> make use of it?

That would be better, but I still hope we can just get back to the
portable code across all architectures for the in-kernel version.

I have one more request for you: can you check /proc/cpu/alignment
on the system that shows the performance regression with and without
the patch? In your original report, you have shown that Ian's patch
caused the drop in performance and that it caused the added function
calls, but I don't see any sufficiently hot code path that would
actually explain the scale of the regression, unless there is now
also a case where get_unaligned() ends up doing the wrong case and
running into a fixup handler. If this happens, you should see a
nontrivial number of fixups listed in /proc/cpu/alignment. On
a working system, the numbers should all be low but likely nonzero
because there tend to be callers that still lack get_unaligned()
wrappers and just access misaligned pointers directly.

      Arnd

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 2701 bytes --]

On 20.10.2026 10:24:51, Arnd Bergmann wrote:
> On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
> > On 20.10.2026 00:12:57, Arnd Bergmann wrote:
> >> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> >> > +
> >> > +config VDSO_UNALIGNED_PACKED
> >> > +	bool
> >> > +	default CPU_32v5
> >> > +	help
> >> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> >> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> >> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
> >>
> >> As explained, I think this misrepresents the problem as Arm specific and
> >> isn't the correct method for -munaligned-access targets on Arm, which
> >> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
> >>
> >> I still hope we can use a version that just produces the intended code
> >> on both -munaligned-access and -mno-unaligned-access targets like the
> >> old version did.
> >
> > FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
> > make use of it?
>
> That would be better, but I still hope we can just get back to the
> portable code across all architectures for the in-kernel version.
>
> I have one more request for you: can you check /proc/cpu/alignment
> on the system that shows the performance regression with and without
> the patch?

I asked Stefan Kerkmann (who wrote the original regression report) the
same question, but we haven't had time for it yet. We'll get around to
it.

> In your original report, you have shown that Ian's patch
> caused the drop in performance and that it caused the added function
> calls, but I don't see any sufficiently hot code path that would
> actually explain the scale of the regression,

Don't underestimate the overhead of function calls on those old CPUs
with tiny I-caches.

Little side note: IIRC, even on ARMv7 you can measure the performance
overhead of the built-in, but disabled, function tracer.

> unless there is now
> also a case where get_unaligned() ends up doing the wrong case and
> running into a fixup handler. If this happens, you should see a
> nontrivial number of fixups listed in /proc/cpu/alignment. On
> a working system, the numbers should all be low but likely nonzero
> because there tend to be callers that still lack get_unaligned()
> wrappers and just access misaligned pointers directly.

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde          |
Embedded Linux                   | https://www.pengutronix.de |
Vertretung Nürnberg              | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-9   |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Tue, Oct 20, 2026, at 10:41, Marc Kleine-Budde wrote:
> On 20.10.2026 10:24:51, Arnd Bergmann wrote:
>> On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
>>
>> That would be better, but I still hope we can just get back to the
>> portable code across all architectures for the in-kernel version.
>>
>> I have one more request for you: can you check /proc/cpu/alignment
>> on the system that shows the performance regression with and without
>> the patch?
>
> I asked Stefan Kerkmann (who wrote the original regression report) the
> same question, but we haven't had time for it yet. We'll get around to
> it.

Ok, thanks!

>> In your original report, you have shown that Ian's patch
>> caused the drop in performance and that it caused the added function
>> calls, but I don't see any sufficiently hot code path that would
>> actually explain the scale of the regression,
>
> Don't underestimate the overhead of function calls on those old CPUs
> with tiny I-caches.

What I meant is that I tried to look for calls to get_unaligned()
in the code path you'd hit with UBIFS on nand flash, and I only
found a possible call once per page written
(ebu_nand_write_page_hwecc), which should have a much lower
overhead than what you observe.

While ubifs does use crc32, it uses the lib/crc/ version rather
than the crypto/crc32c.c that you listed, and there is no
get_unaligned() in the lib/ one. If you have NOR flash, the
caller is likely do_write_buffer() calling map_word_load().
That one could explain the problem and could also be
optimized further if it turns out to be the bottleneck.

     Arnd