[OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

When 64K page size is enabled, for 32bit multilib, we'll need to build
applications with 64K alignment, otherwise, we'll see errors like below at runtime:

  root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
    Segmentation fault

marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
And the issue was first discovered there.

However, there's a kernel config fragment in yocto-kernel-cache which enableds
64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So it's easy for other
arm64 BSPs to enable this. And when they do so, they'll have problem with
multilib. This patch adds a config file to help people a little bit in such
situation.

The max-page-size option is used to fix this issue. This option is added
to both CFLAGS and LDFLAGS for the purpose of covering as many recipes as
possible.

The systemd-boot (bootloader) is an exception. It uses max-page-size=4096
in src/boot/efi/meson.build and it has a hardcoded value in elf2efi.py.
See https://github.com/systemd/systemd/blob/main/tools/elf2efi.py#L233
and https://github.com/systemd/systemd/blob/main/src/boot/meson.build#L192.

As an example, below are the lines I used in local.conf to enable 64K page size
for qemuarm64 and use mutlilib on that.

  KERNEL_FEATURES += "arch/arm/arm64-64kb-pages.scc"
  require conf/multilib.conf
  MULTILIBS ?= "multilib:lib32"
  DEFAULTTUNE:virtclass-multilib-lib32 ?= "armv7athf-neon"
  require conf/distro/include/lib32-64k-pagesize.inc
  IMAGE_INSTALL:append = " lib32-sysstat"

Note that people will need to enable CONFIG_EXPERT=y to make kernel
recognize 32bit binaries. As qemuarm64 does not enable that by default,
we need to enable that too:

  EXTRA_KERNEL_FILES =. "/PATH/TO/build-poky/kernel-cfgs:"
  EXTRA_KERNEL_SRC_URI += "file://enable-expert.cfg"
  SRC_URI:append:pn-linux-yocto = " ${EXTRA_KERNEL_SRC_URI}"
  FILESEXTRAPATHS:prepend:pn-linux-yocto := "${EXTRA_KERNEL_FILES}:"

With the above lines, lib32-sysstat can work correctly on core-image-full-cmdline.

Extra testing include:
1. qemuarm64 enables 64K page size + lib32-core-image-full-cmdline can boot up correctly
2. oe-core world build succeeds

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/distro/include/lib32-64k-pagesize.inc | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 meta/conf/distro/include/lib32-64k-pagesize.inc

diff --git a/meta/conf/distro/include/lib32-64k-pagesize.inc b/meta/conf/distro/include/lib32-64k-pagesize.inc
new file mode 100644
index 0000000000..851a3a65a5
--- /dev/null
+++ b/meta/conf/distro/include/lib32-64k-pagesize.inc
@@ -0,0 +1,4 @@
+MAX_PAGE_SIZE_FLAGS ?= " -Wl,-z,max-page-size=0x10000"
+MAX_PAGE_SIZE_FLAGS:pn-lib32-systemd-boot = ""
+CFLAGS:append:virtclass-multilib-lib32 = "${MAX_PAGE_SIZE_FLAGS}"
+LDFLAGS:append:virtclass-multilib-lib32 = "${MAX_PAGE_SIZE_FLAGS}"
-- 
2.25.1

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Ross Burton]

On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> When 64K page size is enabled, for 32bit multilib, we'll need to build
> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
> 
>  root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
>    Segmentation fault
> 
> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
> And the issue was first discovered there.

Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?

> However, there's a kernel config fragment in yocto-kernel-cache which enableds
> 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So it's easy for other
> arm64 BSPs to enable this. And when they do so, they'll have problem with
> multilib. This patch adds a config file to help people a little bit in such
> situation.
> 
> The max-page-size option is used to fix this issue. This option is added
> to both CFLAGS and LDFLAGS for the purpose of covering as many recipes as
> possible.

First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:

https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435

That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?

Ross

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Ross Burton]

> On 8 Jan 2025, at 17:02, Ross Burton via lists.openembedded.org <ross.burton=arm.com@lists.openembedded.org> wrote:
> First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:
> 
> https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435

Which was subsequently effectively reverted:

https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1a26a53a0dee39106ba58fcb15496c5f13074652

Fun times.

Ross

RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Chen, Qi]

Hi Ross,

The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.

This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal. 

The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
So 32bit arm was not supported at first, but at some point, it was.

Regards,
Qi

-----Original Message-----
From: Ross Burton <Ross.Burton@arm.com> 
Sent: Thursday, January 9, 2025 1:02 AM
To: Chen, Qi <Qi.Chen@windriver.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> When 64K page size is enabled, for 32bit multilib, we'll need to build 
> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
> 
>  root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
>    Segmentation fault
> 
> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
> And the issue was first discovered there.

Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?

> However, there's a kernel config fragment in yocto-kernel-cache which 
> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So 
> it's easy for other
> arm64 BSPs to enable this. And when they do so, they'll have problem 
> with multilib. This patch adds a config file to help people a little 
> bit in such situation.
> 
> The max-page-size option is used to fix this issue. This option is 
> added to both CFLAGS and LDFLAGS for the purpose of covering as many 
> recipes as possible.

First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:

https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435

That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?

Ross

RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Chen, Qi]

Ping

Ross & Richard, is there anything else I need to do for this patch?

Regards,
Qi

-----Original Message-----
From: Chen, Qi 
Sent: Thursday, January 9, 2025 9:54 AM
To: Ross Burton <Ross.Burton@arm.com>
Cc: openembedded-core@lists.openembedded.org
Subject: RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

Hi Ross,

The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.

This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal. 

The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
So 32bit arm was not supported at first, but at some point, it was.

Regards,
Qi

-----Original Message-----
From: Ross Burton <Ross.Burton@arm.com>
Sent: Thursday, January 9, 2025 1:02 AM
To: Chen, Qi <Qi.Chen@windriver.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> When 64K page size is enabled, for 32bit multilib, we'll need to build 
> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
> 
>  root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
>    Segmentation fault
> 
> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
> And the issue was first discovered there.

Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?

> However, there's a kernel config fragment in yocto-kernel-cache which 
> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So 
> it's easy for other
> arm64 BSPs to enable this. And when they do so, they'll have problem 
> with multilib. This patch adds a config file to help people a little 
> bit in such situation.
> 
> The max-page-size option is used to fix this issue. This option is 
> added to both CFLAGS and LDFLAGS for the purpose of covering as many 
> recipes as possible.

First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:

https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435

That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?

Ross

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[ChenQi]

ping

Is there something I need to do for this patch? Or is this patch not 
suitable for oe-core?

Regards,
Qi

On 1/16/25 12:34, Chen Qi via lists.openembedded.org wrote:
> Ping
>
> Ross & Richard, is there anything else I need to do for this patch?
>
> Regards,
> Qi
>
> -----Original Message-----
> From: Chen, Qi
> Sent: Thursday, January 9, 2025 9:54 AM
> To: Ross Burton <Ross.Burton@arm.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
>
> Hi Ross,
>
> The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
> In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.
>
> This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal.
>
> The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
> So 32bit arm was not supported at first, but at some point, it was.
>
> Regards,
> Qi
>
> -----Original Message-----
> From: Ross Burton <Ross.Burton@arm.com>
> Sent: Thursday, January 9, 2025 1:02 AM
> To: Chen, Qi <Qi.Chen@windriver.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
>
> On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>> When 64K page size is enabled, for 32bit multilib, we'll need to build
>> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
>>
>>   root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
>>     Segmentation fault
>>
>> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
>> And the issue was first discovered there.
> Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?
>
>> However, there's a kernel config fragment in yocto-kernel-cache which
>> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So
>> it's easy for other
>> arm64 BSPs to enable this. And when they do so, they'll have problem
>> with multilib. This patch adds a config file to help people a little
>> bit in such situation.
>>
>> The max-page-size option is used to fix this issue. This option is
>> added to both CFLAGS and LDFLAGS for the purpose of covering as many
>> recipes as possible.
> First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:
>
> https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435
>
> That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?
>
> Ross
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#209933): https://lists.openembedded.org/g/openembedded-core/message/209933
> Mute This Topic: https://lists.openembedded.org/mt/110280373/3618072
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Khem Raj]

On Tue, Jan 21, 2025 at 7:03 PM Chen Qi via lists.openembedded.org
<Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>
> ping
>
> Is there something I need to do for this patch? Or is this patch not
> suitable for oe-core?
>

if we are enabling 64K pages then 32bit aarch32 apps should also be
compiled using 64k pages. Usually this could mean different
performance characteristics and 32bit apps are usually only rebuilt if
not used as prebuilts, so how will it function in a multilib
environment?

> Regards,
> Qi
>
> On 1/16/25 12:34, Chen Qi via lists.openembedded.org wrote:
> > Ping
> >
> > Ross & Richard, is there anything else I need to do for this patch?
> >
> > Regards,
> > Qi
> >
> > -----Original Message-----
> > From: Chen, Qi
> > Sent: Thursday, January 9, 2025 9:54 AM
> > To: Ross Burton <Ross.Burton@arm.com>
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
> >
> > Hi Ross,
> >
> > The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
> > In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.
> >
> > This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal.
> >
> > The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
> > So 32bit arm was not supported at first, but at some point, it was.
> >
> > Regards,
> > Qi
> >
> > -----Original Message-----
> > From: Ross Burton <Ross.Burton@arm.com>
> > Sent: Thursday, January 9, 2025 1:02 AM
> > To: Chen, Qi <Qi.Chen@windriver.com>
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
> >
> > On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> >> When 64K page size is enabled, for 32bit multilib, we'll need to build
> >> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
> >>
> >>   root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
> >>     Segmentation fault
> >>
> >> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
> >> And the issue was first discovered there.
> > Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?
> >
> >> However, there's a kernel config fragment in yocto-kernel-cache which
> >> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So
> >> it's easy for other
> >> arm64 BSPs to enable this. And when they do so, they'll have problem
> >> with multilib. This patch adds a config file to help people a little
> >> bit in such situation.
> >>
> >> The max-page-size option is used to fix this issue. This option is
> >> added to both CFLAGS and LDFLAGS for the purpose of covering as many
> >> recipes as possible.
> > First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:
> >
> > https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435
> >
> > That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?
> >
> > Ross
> >
> >
> >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210113): https://lists.openembedded.org/g/openembedded-core/message/210113
> Mute This Topic: https://lists.openembedded.org/mt/110280373/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[ChenQi]

On 1/22/25 12:18, Khem Raj wrote:
> On Tue, Jan 21, 2025 at 7:03 PM Chen Qi via lists.openembedded.org
> <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>> ping
>>
>> Is there something I need to do for this patch? Or is this patch not
>> suitable for oe-core?
>>
> if we are enabling 64K pages then 32bit aarch32 apps should also be
> compiled using 64k pages. Usually this could mean different
> performance characteristics and 32bit apps are usually only rebuilt if
> not used as prebuilts, so how will it function in a multilib
> environment?

Hi Khem,

Sorry I don't quite understand what you mean. Could you please elaborate 
it a little bit?

This patch's purpose is that if users enable 64K page for arm64 and they 
want multilib support, they can include this file in some conf file, 
e.g. some-machine.conf or local.conf, to make lib32 apps work.

Do you mean that when users use multilib, they usually have some 
prebuilt app or library with 4K alignment? If so, then this patch is not 
suitable. In such case, they need to turn off the 64K page in kernel config.

Regards,

Qi

>
>> Regards,
>> Qi
>>
>> On 1/16/25 12:34, Chen Qi via lists.openembedded.org wrote:
>>> Ping
>>>
>>> Ross & Richard, is there anything else I need to do for this patch?
>>>
>>> Regards,
>>> Qi
>>>
>>> -----Original Message-----
>>> From: Chen, Qi
>>> Sent: Thursday, January 9, 2025 9:54 AM
>>> To: Ross Burton <Ross.Burton@arm.com>
>>> Cc: openembedded-core@lists.openembedded.org
>>> Subject: RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
>>>
>>> Hi Ross,
>>>
>>> The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
>>> In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.
>>>
>>> This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal.
>>>
>>> The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
>>> So 32bit arm was not supported at first, but at some point, it was.
>>>
>>> Regards,
>>> Qi
>>>
>>> -----Original Message-----
>>> From: Ross Burton <Ross.Burton@arm.com>
>>> Sent: Thursday, January 9, 2025 1:02 AM
>>> To: Chen, Qi <Qi.Chen@windriver.com>
>>> Cc: openembedded-core@lists.openembedded.org
>>> Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
>>>
>>> On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>>>> When 64K page size is enabled, for 32bit multilib, we'll need to build
>>>> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
>>>>
>>>>    root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
>>>>      Segmentation fault
>>>>
>>>> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
>>>> And the issue was first discovered there.
>>> Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?
>>>
>>>> However, there's a kernel config fragment in yocto-kernel-cache which
>>>> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So
>>>> it's easy for other
>>>> arm64 BSPs to enable this. And when they do so, they'll have problem
>>>> with multilib. This patch adds a config file to help people a little
>>>> bit in such situation.
>>>>
>>>> The max-page-size option is used to fix this issue. This option is
>>>> added to both CFLAGS and LDFLAGS for the purpose of covering as many
>>>> recipes as possible.
>>> First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:
>>>
>>> https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435
>>>
>>> That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?
>>>
>>> Ross
>>>
>>>
>>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#210113): https://lists.openembedded.org/g/openembedded-core/message/210113
>> Mute This Topic: https://lists.openembedded.org/mt/110280373/1997914
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment

[Khem Raj]

On Tue, Jan 21, 2025 at 8:52 PM ChenQi <Qi.Chen@windriver.com> wrote:
>
> On 1/22/25 12:18, Khem Raj wrote:
> > On Tue, Jan 21, 2025 at 7:03 PM Chen Qi via lists.openembedded.org
> > <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> >> ping
> >>
> >> Is there something I need to do for this patch? Or is this patch not
> >> suitable for oe-core?
> >>
> > if we are enabling 64K pages then 32bit aarch32 apps should also be
> > compiled using 64k pages. Usually this could mean different
> > performance characteristics and 32bit apps are usually only rebuilt if
> > not used as prebuilts, so how will it function in a multilib
> > environment?
>
> Hi Khem,
>
> Sorry I don't quite understand what you mean. Could you please elaborate
> it a little bit?
>
> This patch's purpose is that if users enable 64K page for arm64 and they
> want multilib support, they can include this file in some conf file,
> e.g. some-machine.conf or local.conf, to make lib32 apps work.
>
> Do you mean that when users use multilib, they usually have some
> prebuilt app or library with 4K alignment? If so, then this patch is not
> suitable. In such case, they need to turn off the 64K page in kernel config.
>

Right, so I am ok with it being a separate machine config with
appropriate guardrails
but not default, as such someone could enable it as it might be
appealing for performance
etc.

> Regards,
>
> Qi
>
> >
> >> Regards,
> >> Qi
> >>
> >> On 1/16/25 12:34, Chen Qi via lists.openembedded.org wrote:
> >>> Ping
> >>>
> >>> Ross & Richard, is there anything else I need to do for this patch?
> >>>
> >>> Regards,
> >>> Qi
> >>>
> >>> -----Original Message-----
> >>> From: Chen, Qi
> >>> Sent: Thursday, January 9, 2025 9:54 AM
> >>> To: Ross Burton <Ross.Burton@arm.com>
> >>> Cc: openembedded-core@lists.openembedded.org
> >>> Subject: RE: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
> >>>
> >>> Hi Ross,
> >>>
> >>> The "./bsp/marvell-cn10xxx" exists in yocto-kernel-cache's origin/yocto-6.6 branch.
> >>> In its master branch, the marvell-cn96xx.cfg also enables CONFIG_ARM64_64K_PAGES=y.
> >>>
> >>> This patch was tested on pristine poky + qemuarm64. The test steps are in the commit message. I didn't test all softwares in oe-core, but as lib32-core-image-full-cmdline started well with no failed systemd service, I assumed that most softwares should also behave well. I also tested lib32-sysstat with core-image-minimal.
> >>>
> >>> The help text for ARM64_64K_PAGES changed with this commit: https://github.com/torvalds/linux/commit/db488be354bc85724d7b9523e94435fdaa761a35
> >>> So 32bit arm was not supported at first, but at some point, it was.
> >>>
> >>> Regards,
> >>> Qi
> >>>
> >>> -----Original Message-----
> >>> From: Ross Burton <Ross.Burton@arm.com>
> >>> Sent: Thursday, January 9, 2025 1:02 AM
> >>> To: Chen, Qi <Qi.Chen@windriver.com>
> >>> Cc: openembedded-core@lists.openembedded.org
> >>> Subject: Re: [OE-core][PATCH V2] lib32-64k-pagesize.inc: add conf for building 32bit binary with 64K alignment
> >>>
> >>> On 25 Dec 2024, at 03:04, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> >>>> When 64K page size is enabled, for 32bit multilib, we'll need to build
> >>>> applications with 64K alignment, otherwise, we'll see errors like below at runtime:
> >>>>
> >>>>    root@marvell-cn10xxx:~# /lib/ld-linux-armhf.so.3
> >>>>      Segmentation fault
> >>>>
> >>>> marvell-cn10xxx in yocto-kernel-cache enables CONFIG_ARM64_4K_PAGES=y by default.
> >>>> And the issue was first discovered there.
> >>> Do you mean _64K_PAGES?  Where is this marvell-cn100xxx?  In yocto-kernel-cache there are no users of arm64-64kb-pages and nothing called cn10xxx.  Is this a Wind River-specific fork of yocto-kernel-cache?
> >>>
> >>>> However, there's a kernel config fragment in yocto-kernel-cache which
> >>>> enableds 64K page size for arm64: arch/arm/arm64-64kb-pages.scc. So
> >>>> it's easy for other
> >>>> arm64 BSPs to enable this. And when they do so, they'll have problem
> >>>> with multilib. This patch adds a config file to help people a little
> >>>> bit in such situation.
> >>>>
> >>>> The max-page-size option is used to fix this issue. This option is
> >>>> added to both CFLAGS and LDFLAGS for the purpose of covering as many
> >>>> recipes as possible.
> >>> First, I should point out that changing the page size is an ABI break, and the 32-bit ABI was formalised long before 64kb pages were a thing, so things _will_ break.  That said the loader should work, as a decade ago binutils 2.25 changed the alignment to 64kb pages by default:
> >>>
> >>> https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=ld/emulparams/armelf_linux.sh;h=280db842208e06e3231ec3a8322369193630a07d;hp=35891f1fc91bddd03677aaefd7146ae5333e6cf0;hb=7572ca8989ead4c3425a1500bc241eaaeffa2c89;hpb=8335d6aa34b88ce31b62e1b578d54ab4aa364435
> >>>
> >>> That won’t help specific pieces of software that eg assume the page size is a built-time constant (and use the wrong constant), but helloworld should be good.  Can you replicate with pristine poky master and a minimal configuration in a qemu?
> >>>
> >>> Ross
> >>>
> >>>
> >>>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#210113): https://lists.openembedded.org/g/openembedded-core/message/210113
> >> Mute This Topic: https://lists.openembedded.org/mt/110280373/1997914
> >> Group Owner: openembedded-core+owner@lists.openembedded.org
> >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
>