[PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Andrea Bolognani]

Currently the script won't generate a configuration file that
sets up qemu-user-riscv32 on riscv64, likely under the
assumption that 64-bit RISC-V machines can natively run 32-bit
RISC-V code.

However this functionality, while theoretically possible, in
practice is missing from most commonly available RISC-V hardware
and not enabled at the distro level. So qemu-user-riscv32 really
is the only option to run riscv32 binaries on riscv64.

Make riscv32 and riscv64 each its own family, so that the
configuration file we need to make 32-on-64 userspace emulation
work gets generated.

Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
Thanks: David Abdurachmanov <davidlt@rivosinc.com>
Thanks: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 scripts/qemu-binfmt-conf.sh | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
index 6ef9f118d9..e38b767c24 100755
--- a/scripts/qemu-binfmt-conf.sh
+++ b/scripts/qemu-binfmt-conf.sh
@@ -110,11 +110,11 @@ hppa_family=hppa
 
 riscv32_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
 riscv32_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
-riscv32_family=riscv
+riscv32_family=riscv32
 
 riscv64_magic='\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
 riscv64_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
-riscv64_family=riscv
+riscv64_family=riscv64
 
 xtensa_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x5e\x00'
 xtensa_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
@@ -168,9 +168,6 @@ qemu_get_family() {
     sparc*)
         echo "sparc"
         ;;
-    riscv*)
-        echo "riscv"
-        ;;
     loongarch*)
         echo "loongarch"
         ;;
-- 
2.47.0

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Philippe Mathieu-Daudé]

Hi Andrea,

On 3/12/24 10:47, Andrea Bolognani wrote:
> Currently the script won't generate a configuration file that
> sets up qemu-user-riscv32 on riscv64, likely under the
> assumption that 64-bit RISC-V machines can natively run 32-bit

I'm confused by the "machines" description used for user emulation.

> RISC-V code.
> 
> However this functionality, while theoretically possible, in
> practice is missing from most commonly available RISC-V hardware
> and not enabled at the distro level. So qemu-user-riscv32 really
> is the only option to run riscv32 binaries on riscv64.

We have definitions such ELF_ARCH/ELF_PLATFORM/ELF_MACHINE to
parse ELF header and select the best CPU / flags. Maybe RISC-V
lacks them?

BTW we should expose that for linux-user as target_arch_elf.h,
like bsd-user does, that would reduce all these #ifdef'ry in
linux-user/elfload.c...

> 
> Make riscv32 and riscv64 each its own family, so that the
> configuration file we need to make 32-on-64 userspace emulation
> work gets generated.

Does this patch aim for 9.2? Otherwise FYI  I'm working on unifying
32/64-bit targets, maybe for 10.0...

> 
> Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> Thanks: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>   scripts/qemu-binfmt-conf.sh | 7 ++-----
>   1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
> index 6ef9f118d9..e38b767c24 100755
> --- a/scripts/qemu-binfmt-conf.sh
> +++ b/scripts/qemu-binfmt-conf.sh
> @@ -110,11 +110,11 @@ hppa_family=hppa
>   
>   riscv32_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
>   riscv32_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> -riscv32_family=riscv
> +riscv32_family=riscv32
>   
>   riscv64_magic='\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
>   riscv64_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> -riscv64_family=riscv
> +riscv64_family=riscv64
>   
>   xtensa_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x5e\x00'
>   xtensa_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> @@ -168,9 +168,6 @@ qemu_get_family() {
>       sparc*)
>           echo "sparc"
>           ;;
> -    riscv*)
> -        echo "riscv"
> -        ;;
>       loongarch*)
>           echo "loongarch"
>           ;;

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Andrea Bolognani]

On Tue, Dec 03, 2024 at 10:59:24AM +0100, Philippe Mathieu-Daudé wrote:
> On 3/12/24 10:47, Andrea Bolognani wrote:
> > Currently the script won't generate a configuration file that
> > sets up qemu-user-riscv32 on riscv64, likely under the
> > assumption that 64-bit RISC-V machines can natively run 32-bit
>
> I'm confused by the "machines" description used for user emulation.

I meant it in the sense of physical machines. I can use the word
"hosts" instead if you think that's less ambiguous.

> > However this functionality, while theoretically possible, in
> > practice is missing from most commonly available RISC-V hardware
> > and not enabled at the distro level. So qemu-user-riscv32 really
> > is the only option to run riscv32 binaries on riscv64.
>
> We have definitions such ELF_ARCH/ELF_PLATFORM/ELF_MACHINE to
> parse ELF header and select the best CPU / flags. Maybe RISC-V
> lacks them?
>
> BTW we should expose that for linux-user as target_arch_elf.h,
> like bsd-user does, that would reduce all these #ifdef'ry in
> linux-user/elfload.c...

All of this is flying way over my head, sorry :)

qemu-user-riscv32 already works great on riscv64 as far as I can
tell. I tested it by chrooting into a riscv32 Gentoo rootfs from a
riscv64 Fedora installation. We just need the configuration file to
be generated.

> > Make riscv32 and riscv64 each its own family, so that the
> > configuration file we need to make 32-on-64 userspace emulation
> > work gets generated.
>
> Does this patch aim for 9.2? Otherwise FYI  I'm working on unifying
> 32/64-bit targets, maybe for 10.0...

Having this in 9.2 would be great.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Daniel P. Berrangé]

On Tue, Dec 03, 2024 at 10:59:24AM +0100, Philippe Mathieu-Daudé wrote:
> Hi Andrea,
> 
> On 3/12/24 10:47, Andrea Bolognani wrote:
> > Currently the script won't generate a configuration file that
> > sets up qemu-user-riscv32 on riscv64, likely under the
> > assumption that 64-bit RISC-V machines can natively run 32-bit
> 
> I'm confused by the "machines" description used for user emulation.

It is referring to the host machines, being able (or not) to
run 32-bit usermode code on 64-bit host kernel.

> 
> > RISC-V code.
> > 
> > However this functionality, while theoretically possible, in
> > practice is missing from most commonly available RISC-V hardware
> > and not enabled at the distro level. So qemu-user-riscv32 really
> > is the only option to run riscv32 binaries on riscv64.
> 
> We have definitions such ELF_ARCH/ELF_PLATFORM/ELF_MACHINE to
> parse ELF header and select the best CPU / flags. Maybe RISC-V
> lacks them?

Is that relevant, as we're not runing QEMU code at all in
the problematic scenario ?

Currently the script below will skip generating a binfmt
rule for riscv32, when on a riscv64 host. Thus qemu-riscv32
will never get called, and the kernel will try & fail to
run riscv32 binaries natively.

This change would make us generate riscv32 binfmt rules
and thus use qemu-riscv32 on riscv64 hosts for linux-user.


Separatley this from patch, we should also consider whether
it is time to do the same for aarch64/arm7.

If I look at this page:

  https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html

and sort by 'announced' to see msot recent CPUs first, then
almost all of them have "NO" in the "aarch32 support" column.

IOW, on modern aarch64 CPUs, qemu-arm is the only viable way
to run 32-bit usermode binaries AFAICT, and suggests we ought
to be creating a binfmt rule for that on aarch64 hosts.

> BTW we should expose that for linux-user as target_arch_elf.h,
> like bsd-user does, that would reduce all these #ifdef'ry in
> linux-user/elfload.c...
> 
> > 
> > Make riscv32 and riscv64 each its own family, so that the
> > configuration file we need to make 32-on-64 userspace emulation
> > work gets generated.
> 
> Does this patch aim for 9.2? Otherwise FYI  I'm working on unifying
> 32/64-bit targets, maybe for 10.0...

Well in Fedora we'll backport it to 9.2 at least, and from that
POV I'd consider it stable-9.2 material if accepted here.

> > Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> > Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> > Thanks: Daniel P. Berrangé <berrange@redhat.com>
> > Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> > ---
> >   scripts/qemu-binfmt-conf.sh | 7 ++-----
> >   1 file changed, 2 insertions(+), 5 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

> > 
> > diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
> > index 6ef9f118d9..e38b767c24 100755
> > --- a/scripts/qemu-binfmt-conf.sh
> > +++ b/scripts/qemu-binfmt-conf.sh
> > @@ -110,11 +110,11 @@ hppa_family=hppa
> >   riscv32_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
> >   riscv32_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> > -riscv32_family=riscv
> > +riscv32_family=riscv32
> >   riscv64_magic='\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
> >   riscv64_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> > -riscv64_family=riscv
> > +riscv64_family=riscv64
> >   xtensa_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x5e\x00'
> >   xtensa_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> > @@ -168,9 +168,6 @@ qemu_get_family() {
> >       sparc*)
> >           echo "sparc"
> >           ;;
> > -    riscv*)
> > -        echo "riscv"
> > -        ;;
> >       loongarch*)
> >           echo "loongarch"
> >           ;;
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Peter Maydell]

On Tue, 3 Dec 2024 at 10:19, Daniel P. Berrangé <berrange@redhat.com> wrote:
> Separatley this from patch, we should also consider whether
> it is time to do the same for aarch64/arm7.
>
> If I look at this page:
>
>   https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html
>
> and sort by 'announced' to see msot recent CPUs first, then
> almost all of them have "NO" in the "aarch32 support" column.
>
> IOW, on modern aarch64 CPUs, qemu-arm is the only viable way
> to run 32-bit usermode binaries AFAICT, and suggests we ought
> to be creating a binfmt rule for that on aarch64 hosts.

What happens if you have a host CPU that *does* support 32-bit
natively and you also register the binfmt rule? Does the
host kernel prefer to execute natively or does it invoke
QEMU? I don't think we want to roll out something that
silently downgrades native execution to emulation...

thanks
-- PMM

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Richard Henderson]

On 12/3/24 04:35, Peter Maydell wrote:
> On Tue, 3 Dec 2024 at 10:19, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> Separatley this from patch, we should also consider whether
>> it is time to do the same for aarch64/arm7.
>>
>> If I look at this page:
>>
>>    https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html
>>
>> and sort by 'announced' to see msot recent CPUs first, then
>> almost all of them have "NO" in the "aarch32 support" column.
>>
>> IOW, on modern aarch64 CPUs, qemu-arm is the only viable way
>> to run 32-bit usermode binaries AFAICT, and suggests we ought
>> to be creating a binfmt rule for that on aarch64 hosts.
> 
> What happens if you have a host CPU that *does* support 32-bit
> natively and you also register the binfmt rule? Does the
> host kernel prefer to execute natively or does it invoke
> QEMU? I don't think we want to roll out something that
> silently downgrades native execution to emulation...

The registered rule applies and the kernel invokes qemu.


r~

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Daniel P. Berrangé]

On Tue, Dec 03, 2024 at 07:57:14AM -0600, Richard Henderson wrote:
> On 12/3/24 04:35, Peter Maydell wrote:
> > On Tue, 3 Dec 2024 at 10:19, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > Separatley this from patch, we should also consider whether
> > > it is time to do the same for aarch64/arm7.
> > > 
> > > If I look at this page:
> > > 
> > >    https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html
> > > 
> > > and sort by 'announced' to see msot recent CPUs first, then
> > > almost all of them have "NO" in the "aarch32 support" column.
> > > 
> > > IOW, on modern aarch64 CPUs, qemu-arm is the only viable way
> > > to run 32-bit usermode binaries AFAICT, and suggests we ought
> > > to be creating a binfmt rule for that on aarch64 hosts.
> > 
> > What happens if you have a host CPU that *does* support 32-bit
> > natively and you also register the binfmt rule? Does the
> > host kernel prefer to execute natively or does it invoke
> > QEMU? I don't think we want to roll out something that
> > silently downgrades native execution to emulation...
> 
> The registered rule applies and the kernel invokes qemu.

This is all quiet difficult from a distro POV, but not QEMU's fault.

We want to install the binfmt rules in a way that we "do the right thing"
regardless of hardware out of the box.

The systemd logic for loading binfmt rules is unconditional, loading
everything from /usr/lib/binfmt.d, but we need a way to make things
conditional on the lack of support for aarch32 on the currently running
platform.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Laurent Vivier]

Le 04/12/2024 à 11:17, Daniel P. Berrangé a écrit :
> On Tue, Dec 03, 2024 at 07:57:14AM -0600, Richard Henderson wrote:
>> On 12/3/24 04:35, Peter Maydell wrote:
>>> On Tue, 3 Dec 2024 at 10:19, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>>> Separatley this from patch, we should also consider whether
>>>> it is time to do the same for aarch64/arm7.
>>>>
>>>> If I look at this page:
>>>>
>>>>     https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html
>>>>
>>>> and sort by 'announced' to see msot recent CPUs first, then
>>>> almost all of them have "NO" in the "aarch32 support" column.
>>>>
>>>> IOW, on modern aarch64 CPUs, qemu-arm is the only viable way
>>>> to run 32-bit usermode binaries AFAICT, and suggests we ought
>>>> to be creating a binfmt rule for that on aarch64 hosts.
>>>
>>> What happens if you have a host CPU that *does* support 32-bit
>>> natively and you also register the binfmt rule? Does the
>>> host kernel prefer to execute natively or does it invoke
>>> QEMU? I don't think we want to roll out something that
>>> silently downgrades native execution to emulation...
>>
>> The registered rule applies and the kernel invokes qemu.
> 
> This is all quiet difficult from a distro POV, but not QEMU's fault.
> 
> We want to install the binfmt rules in a way that we "do the right thing"
> regardless of hardware out of the box.
> 
> The systemd logic for loading binfmt rules is unconditional, loading
> everything from /usr/lib/binfmt.d, but we need a way to make things
> conditional on the lack of support for aarch32 on the currently running
> platform.

Now, there is another alternative: binfmt_misc is now part of user namespace, so you can define a 
binfmt rule for a given namespace:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=21ca59b365c091d583f36ac753eaa8baf947be6f

I have added a new parameter to unshare to help to run a new namespace with binfmt_misc:

https://github.com/util-linux/util-linux/commit/9d55de0d0d5c6298b38b58c3f4dc876c56213f85

Thanks,
Laurent

Re: [PATCH] binfmt: Don't consider riscv{32,64} part of the same family

[Laurent Vivier]

Le 03/12/2024 à 10:47, Andrea Bolognani a écrit :
> Currently the script won't generate a configuration file that
> sets up qemu-user-riscv32 on riscv64, likely under the
> assumption that 64-bit RISC-V machines can natively run 32-bit
> RISC-V code.
> 
> However this functionality, while theoretically possible, in
> practice is missing from most commonly available RISC-V hardware
> and not enabled at the distro level. So qemu-user-riscv32 really
> is the only option to run riscv32 binaries on riscv64.
> 
> Make riscv32 and riscv64 each its own family, so that the
> configuration file we need to make 32-on-64 userspace emulation
> work gets generated.
> 
> Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> Thanks: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>   scripts/qemu-binfmt-conf.sh | 7 ++-----
>   1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
> index 6ef9f118d9..e38b767c24 100755
> --- a/scripts/qemu-binfmt-conf.sh
> +++ b/scripts/qemu-binfmt-conf.sh
> @@ -110,11 +110,11 @@ hppa_family=hppa
>   
>   riscv32_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
>   riscv32_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> -riscv32_family=riscv
> +riscv32_family=riscv32
>   
>   riscv64_magic='\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00'
>   riscv64_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> -riscv64_family=riscv
> +riscv64_family=riscv64
>   
>   xtensa_magic='\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x5e\x00'
>   xtensa_mask='\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'
> @@ -168,9 +168,6 @@ qemu_get_family() {
>       sparc*)
>           echo "sparc"
>           ;;
> -    riscv*)
> -        echo "riscv"
> -        ;;
>       loongarch*)
>           echo "loongarch"
>           ;;

Reviewed-by: Laurent Vivier <laurent@vivier.eu>

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Andrea Bolognani]

On Tue, Dec 03, 2024 at 10:47:02AM +0100, Andrea Bolognani wrote:
> Currently the script won't generate a configuration file that
> sets up qemu-user-riscv32 on riscv64, likely under the
> assumption that 64-bit RISC-V machines can natively run 32-bit
> RISC-V code.
>
> However this functionality, while theoretically possible, in
> practice is missing from most commonly available RISC-V hardware
> and not enabled at the distro level. So qemu-user-riscv32 really
> is the only option to run riscv32 binaries on riscv64.
>
> Make riscv32 and riscv64 each its own family, so that the
> configuration file we need to make 32-on-64 userspace emulation
> work gets generated.
>
> Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> Thanks: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  scripts/qemu-binfmt-conf.sh | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)

ping

There are already two ACKs so I think we just need a maintainer to
pick this up.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Alistair Francis]

On Fri, Jan 3, 2025 at 2:04 AM Andrea Bolognani <abologna@redhat.com> wrote:
>
> On Tue, Dec 03, 2024 at 10:47:02AM +0100, Andrea Bolognani wrote:
> > Currently the script won't generate a configuration file that
> > sets up qemu-user-riscv32 on riscv64, likely under the
> > assumption that 64-bit RISC-V machines can natively run 32-bit
> > RISC-V code.
> >
> > However this functionality, while theoretically possible, in
> > practice is missing from most commonly available RISC-V hardware
> > and not enabled at the distro level. So qemu-user-riscv32 really
> > is the only option to run riscv32 binaries on riscv64.
> >
> > Make riscv32 and riscv64 each its own family, so that the
> > configuration file we need to make 32-on-64 userspace emulation
> > work gets generated.
> >
> > Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> > Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> > Thanks: Daniel P. Berrangé <berrange@redhat.com>
> > Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> > ---
> >  scripts/qemu-binfmt-conf.sh | 7 ++-----
> >  1 file changed, 2 insertions(+), 5 deletions(-)
>
> ping
>
> There are already two ACKs so I think we just need a maintainer to
> pick this up.

We didn't get an answer to the issue of a CPU supporting RV32 and yet
the kernel still calls QEMU.

I understand this allows things to work out of the box, but seems like
a disservice to any hardware that does support RV32

Alistair

>
> --
> Andrea Bolognani / Red Hat / Virtualization
>
>

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Peter Maydell]

On Mon, 6 Jan 2025 at 01:29, Alistair Francis <alistair23@gmail.com> wrote:
>
> On Fri, Jan 3, 2025 at 2:04 AM Andrea Bolognani <abologna@redhat.com> wrote:
> >
> > On Tue, Dec 03, 2024 at 10:47:02AM +0100, Andrea Bolognani wrote:
> > > Currently the script won't generate a configuration file that
> > > sets up qemu-user-riscv32 on riscv64, likely under the
> > > assumption that 64-bit RISC-V machines can natively run 32-bit
> > > RISC-V code.
> > >
> > > However this functionality, while theoretically possible, in
> > > practice is missing from most commonly available RISC-V hardware
> > > and not enabled at the distro level. So qemu-user-riscv32 really
> > > is the only option to run riscv32 binaries on riscv64.
> > >
> > > Make riscv32 and riscv64 each its own family, so that the
> > > configuration file we need to make 32-on-64 userspace emulation
> > > work gets generated.
> > >
> > > Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> > > Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> > > Thanks: Daniel P. Berrangé <berrange@redhat.com>
> > > Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> > > ---
> > >  scripts/qemu-binfmt-conf.sh | 7 ++-----
> > >  1 file changed, 2 insertions(+), 5 deletions(-)
> >
> > ping
> >
> > There are already two ACKs so I think we just need a maintainer to
> > pick this up.
>
> We didn't get an answer to the issue of a CPU supporting RV32 and yet
> the kernel still calls QEMU.
>
> I understand this allows things to work out of the box, but seems like
> a disservice to any hardware that does support RV32

There's the same thing on Arm too -- we don't set up qemu-user
aarch32 binfmt-misc on an aarch64 system because the host might
be able to natively execute the aarch32 binary. This is becoming
less true, but we still don't want to silently downgrade
native execution to emulation on the hosts where native execution
used to work.

I'm not sure the best approach here -- ideally we would want to
be able to register a binfmt-misc to the host kernel with "only use
this if you could not already natively execute it", but AFAIK that's
not possible.

-- PMM

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Daniel P. Berrangé]

On Mon, Jan 06, 2025 at 11:47:00AM +0000, Peter Maydell wrote:
> On Mon, 6 Jan 2025 at 01:29, Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Fri, Jan 3, 2025 at 2:04 AM Andrea Bolognani <abologna@redhat.com> wrote:
> > >
> > > On Tue, Dec 03, 2024 at 10:47:02AM +0100, Andrea Bolognani wrote:
> > > > Currently the script won't generate a configuration file that
> > > > sets up qemu-user-riscv32 on riscv64, likely under the
> > > > assumption that 64-bit RISC-V machines can natively run 32-bit
> > > > RISC-V code.
> > > >
> > > > However this functionality, while theoretically possible, in
> > > > practice is missing from most commonly available RISC-V hardware
> > > > and not enabled at the distro level. So qemu-user-riscv32 really
> > > > is the only option to run riscv32 binaries on riscv64.
> > > >
> > > > Make riscv32 and riscv64 each its own family, so that the
> > > > configuration file we need to make 32-on-64 userspace emulation
> > > > work gets generated.
> > > >
> > > > Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> > > > Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> > > > Thanks: Daniel P. Berrangé <berrange@redhat.com>
> > > > Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> > > > ---
> > > >  scripts/qemu-binfmt-conf.sh | 7 ++-----
> > > >  1 file changed, 2 insertions(+), 5 deletions(-)
> > >
> > > ping
> > >
> > > There are already two ACKs so I think we just need a maintainer to
> > > pick this up.
> >
> > We didn't get an answer to the issue of a CPU supporting RV32 and yet
> > the kernel still calls QEMU.
> >
> > I understand this allows things to work out of the box, but seems like
> > a disservice to any hardware that does support RV32
> 
> There's the same thing on Arm too -- we don't set up qemu-user
> aarch32 binfmt-misc on an aarch64 system because the host might
> be able to natively execute the aarch32 binary. This is becoming
> less true, but we still don't want to silently downgrade
> native execution to emulation on the hosts where native execution
> used to work.

Arm is a bigger problem as historically there genuinely was a
non-trivial set of CPUs with 32-on-64 support in HW.

IIUC, the riscv situation is much less likely to be a real problem

> I'm not sure the best approach here -- ideally we would want to
> be able to register a binfmt-misc to the host kernel with "only use
> this if you could not already natively execute it", but AFAIK that's
> not possible.

The other thing is that qemu-binfmt-conf.sh is not really the right
place to decide this, as we can't assume it is being run on the machine
that QEMU will be deployed on. eg in distro case, qemu-binfmt-conf.sh
may be run in a build farm to statically generate files.

Any conditional loading of binfmt rules would required extra magic to be
implemented by systemd, or would have to be done by the user selectively
installing different packages to omit the binfmt rules they don't want.

As a immediate bandaid, I'd suggest that qemu-binfmt-conf.sh could keep
its current logic as the default, and have a switch "--32-on-64" [1] to
tell it to generate the binfmt for 32-bit arch, even if 64-bit arch
could have 32-bit support.

Distros/users could then choose whether to pass --32-on-64 when statically
generating the binfmt files.

With regards,
Daniel

[1] better names welcome
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Peter Maydell]

On Mon, 6 Jan 2025 at 11:58, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Mon, Jan 06, 2025 at 11:47:00AM +0000, Peter Maydell wrote:
> > On Mon, 6 Jan 2025 at 01:29, Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > On Fri, Jan 3, 2025 at 2:04 AM Andrea Bolognani <abologna@redhat.com> wrote:
> > > >
> > > > On Tue, Dec 03, 2024 at 10:47:02AM +0100, Andrea Bolognani wrote:
> > > > > Currently the script won't generate a configuration file that
> > > > > sets up qemu-user-riscv32 on riscv64, likely under the
> > > > > assumption that 64-bit RISC-V machines can natively run 32-bit
> > > > > RISC-V code.
> > > > >
> > > > > However this functionality, while theoretically possible, in
> > > > > practice is missing from most commonly available RISC-V hardware
> > > > > and not enabled at the distro level. So qemu-user-riscv32 really
> > > > > is the only option to run riscv32 binaries on riscv64.
> > > > >
> > > > > Make riscv32 and riscv64 each its own family, so that the
> > > > > configuration file we need to make 32-on-64 userspace emulation
> > > > > work gets generated.
> > > > >
> > > > > Link: https://src.fedoraproject.org/rpms/qemu/pull-request/72
> > > > > Thanks: David Abdurachmanov <davidlt@rivosinc.com>
> > > > > Thanks: Daniel P. Berrangé <berrange@redhat.com>
> > > > > Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> > > > > ---
> > > > >  scripts/qemu-binfmt-conf.sh | 7 ++-----
> > > > >  1 file changed, 2 insertions(+), 5 deletions(-)
> > > >
> > > > ping
> > > >
> > > > There are already two ACKs so I think we just need a maintainer to
> > > > pick this up.
> > >
> > > We didn't get an answer to the issue of a CPU supporting RV32 and yet
> > > the kernel still calls QEMU.
> > >
> > > I understand this allows things to work out of the box, but seems like
> > > a disservice to any hardware that does support RV32
> >
> > There's the same thing on Arm too -- we don't set up qemu-user
> > aarch32 binfmt-misc on an aarch64 system because the host might
> > be able to natively execute the aarch32 binary. This is becoming
> > less true, but we still don't want to silently downgrade
> > native execution to emulation on the hosts where native execution
> > used to work.
>
> Arm is a bigger problem as historically there genuinely was a
> non-trivial set of CPUs with 32-on-64 support in HW.
>
> IIUC, the riscv situation is much less likely to be a real problem
>
> > I'm not sure the best approach here -- ideally we would want to
> > be able to register a binfmt-misc to the host kernel with "only use
> > this if you could not already natively execute it", but AFAIK that's
> > not possible.
>
> The other thing is that qemu-binfmt-conf.sh is not really the right
> place to decide this, as we can't assume it is being run on the machine
> that QEMU will be deployed on. eg in distro case, qemu-binfmt-conf.sh
> may be run in a build farm to statically generate files.
>
> Any conditional loading of binfmt rules would required extra magic to be
> implemented by systemd, or would have to be done by the user selectively
> installing different packages to omit the binfmt rules they don't want.

If the kernel supported this via a binfmt flag, it wouldn't need
systemd-specific magic, user intervention, or for qemu-binfmt-conf.sh
to be running on the target machine. (But of course there's the "does
anybody care enough to implement that" problem plus the long delay of
actually deploying kernels that know about the flag...)

thanks
-- PMM

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Andrea Bolognani]

On Mon, Jan 06, 2025 at 11:57:58AM +0000, Daniel P. Berrangé wrote:
> On Mon, Jan 06, 2025 at 11:47:00AM +0000, Peter Maydell wrote:
> > On Mon, 6 Jan 2025 at 01:29, Alistair Francis <alistair23@gmail.com> wrote:
> > > We didn't get an answer to the issue of a CPU supporting RV32 and yet
> > > the kernel still calls QEMU.
> > >
> > > I understand this allows things to work out of the box, but seems like
> > > a disservice to any hardware that does support RV32
> >
> > There's the same thing on Arm too -- we don't set up qemu-user
> > aarch32 binfmt-misc on an aarch64 system because the host might
> > be able to natively execute the aarch32 binary. This is becoming
> > less true, but we still don't want to silently downgrade
> > native execution to emulation on the hosts where native execution
> > used to work.
>
> Arm is a bigger problem as historically there genuinely was a
> non-trivial set of CPUs with 32-on-64 support in HW.
>
> IIUC, the riscv situation is much less likely to be a real problem

Exactly.

My understanding is that, while 64-bit RISC-V CPUs that can natively
run 32-bit applications are theoretically possible, no such CPU
actually exists right now.

Even if it did exist, distros would have to set up things to support
this scenario, which they don't.

So in the current situation we're effectively making it impossible to
run riscv32 binaries on riscv64 for the benefit of a hypotetical
scenario.

> As a immediate bandaid, I'd suggest that qemu-binfmt-conf.sh could keep
> its current logic as the default, and have a switch "--32-on-64" [1] to
> tell it to generate the binfmt for 32-bit arch, even if 64-bit arch
> could have 32-bit support.
>
> Distros/users could then choose whether to pass --32-on-64 when statically
> generating the binfmt files.

While I'm still convinced that this patch could be safely applied
as-is, I'd be happy to go with your proposed approach if doing so
would help move things forward.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] binfmt: Don't consider riscv{32, 64} part of the same family

[Alistair Francis]

On Tue, Jan 7, 2025 at 3:54 AM Andrea Bolognani <abologna@redhat.com> wrote:
>
> On Mon, Jan 06, 2025 at 11:57:58AM +0000, Daniel P. Berrangé wrote:
> > On Mon, Jan 06, 2025 at 11:47:00AM +0000, Peter Maydell wrote:
> > > On Mon, 6 Jan 2025 at 01:29, Alistair Francis <alistair23@gmail.com> wrote:
> > > > We didn't get an answer to the issue of a CPU supporting RV32 and yet
> > > > the kernel still calls QEMU.
> > > >
> > > > I understand this allows things to work out of the box, but seems like
> > > > a disservice to any hardware that does support RV32
> > >
> > > There's the same thing on Arm too -- we don't set up qemu-user
> > > aarch32 binfmt-misc on an aarch64 system because the host might
> > > be able to natively execute the aarch32 binary. This is becoming
> > > less true, but we still don't want to silently downgrade
> > > native execution to emulation on the hosts where native execution
> > > used to work.
> >
> > Arm is a bigger problem as historically there genuinely was a
> > non-trivial set of CPUs with 32-on-64 support in HW.
> >
> > IIUC, the riscv situation is much less likely to be a real problem
>
> Exactly.
>
> My understanding is that, while 64-bit RISC-V CPUs that can natively
> run 32-bit applications are theoretically possible, no such CPU
> actually exists right now.

I do think T-HEAD are working on CPUs that do that though

>
> Even if it did exist, distros would have to set up things to support
> this scenario, which they don't.

Fair point

>
> So in the current situation we're effectively making it impossible to
> run riscv32 binaries on riscv64 for the benefit of a hypotetical
> scenario.

My worry is that in the future there is hardware that can do this and
we are stuck with this decision.

It does seem unlikely that lots of hardware will start supporting RV32

>
> > As a immediate bandaid, I'd suggest that qemu-binfmt-conf.sh could keep
> > its current logic as the default, and have a switch "--32-on-64" [1] to
> > tell it to generate the binfmt for 32-bit arch, even if 64-bit arch
> > could have 32-bit support.
> >
> > Distros/users could then choose whether to pass --32-on-64 when statically
> > generating the binfmt files.
>
> While I'm still convinced that this patch could be safely applied
> as-is, I'd be happy to go with your proposed approach if doing so
> would help move things forward.

That might be the best step, that way we allow distros to decide

Alistair

>
> --
> Andrea Bolognani / Red Hat / Virtualization
>