Detecting AArch32 support from a AArch64 process in user space

Detecting AArch32 support from a AArch64 process in user space

[Stefan Agner]

[resend this time with the correct mailing list address]

Hello,

I am trying to detect whether an ARMv8 system running in AArch64 state
supports AArch32 state from a user space process. The arm64_features[]
in
arch/arm64/kernel/cpufeature.c lists a CPU feature "32-bit EL0 Support".
However, afaik this CPU feature is not directly exposed to user-space.
The features do get printed in the kernel log, but that requires
privileges and only works directly after boot. There is
system_supports_32bit_el0() which is used in various places in the arm64
architecture code. One of the instances where I can make sense of from
user space is through the personality system call. One idea is to call
personality(PER_LINUX32). It would then return error code 22 in case
32-bit is not supported in user space. However, if successful this
changes the personality of the current process which might have side
effects which I do not want...?

I started to ask myself what PER_LINUX32 actually changes. From what I
can tell it only changes the behavior of /proc/cpuinfo? The personality
seems not to be applied automatically to 32-bit processes, so this is a
opt-in backward compatibility feature?

To be on the safe side, I was thinking about executing the system call
in a separate process. However, at that point I could also just execute
a statically linked AArch32 binary and see whether I get a "exec format
error". I guess this could then be either due to missing AArch32 CPU
support or the kernel not being compiled with 32-bit compatibility.

At last I was considering reading directly from the CPU. But from what I
understand the register used in the kernel to determine 32-bit
compatibility (ID_AA64PFR0_EL1) is not accessible by user space (due to
the suffix _EL1).

Any advice/thoughts on this topic?

--
Stefan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Marc Zyngier]

Hi Stefan,

On 08/08/2019 08:36, Stefan Agner wrote:
> [resend this time with the correct mailing list address]
> 
> Hello,
> 
> I am trying to detect whether an ARMv8 system running in AArch64 state
> supports AArch32 state from a user space process. The arm64_features[]
> in
> arch/arm64/kernel/cpufeature.c lists a CPU feature "32-bit EL0 Support".
> However, afaik this CPU feature is not directly exposed to user-space.
> The features do get printed in the kernel log, but that requires
> privileges and only works directly after boot. There is
> system_supports_32bit_el0() which is used in various places in the arm64
> architecture code. One of the instances where I can make sense of from
> user space is through the personality system call. One idea is to call
> personality(PER_LINUX32). It would then return error code 22 in case
> 32-bit is not supported in user space. However, if successful this
> changes the personality of the current process which might have side
> effects which I do not want...?

You should be able to revert the effects of PER_LINUX_32 by feeding back
the return value of the initial call to personality() to a second
personality() call.

> I started to ask myself what PER_LINUX32 actually changes. From what I
> can tell it only changes the behavior of /proc/cpuinfo? The personality
> seems not to be applied automatically to 32-bit processes, so this is a
> opt-in backward compatibility feature?

It's all about giving the illusion that the process runs in a "real"
32bit environment, with all its warts. It doesn't change the CPU mode
you're running in (that'd be a bit harsh). It's only once you exec
something that requires AArch32 that we engage the COMPAT mode.

Provided that your kernel contains 00377277166b or a backport of it (or
that it predates 4378a7d4be30), the following program should do the
right thing:

#include <sys/personality.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
        int old, new;

        old = personality(PER_LINUX32);
        if (old < 0) {
                perror("No 32bit for you");
                return 1;
        }

        new = personality(0xffffffff);
        printf("Running with personality %d\n", new);

        personality(old);
        new = personality(0xffffffff);

        printf("Running with personality %d\n", new);

        return 0;
}

> To be on the safe side, I was thinking about executing the system call
> in a separate process. However, at that point I could also just execute
> a statically linked AArch32 binary and see whether I get a "exec format
> error". I guess this could then be either due to missing AArch32 CPU
> support or the kernel not being compiled with 32-bit compatibility.

Overkill ;-). The above should be enough.

> At last I was considering reading directly from the CPU. But from what I
> understand the register used in the kernel to determine 32-bit
> compatibility (ID_AA64PFR0_EL1) is not accessible by user space (due to
> the suffix _EL1).

Hey, you could create a VM, a vcpu and dump the ID registers by issuing
a set of KVM_GET_ONE_REG ioctls. Not necessarily recommended... ;-)

Cheers,

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Will Deacon]

On Thu, Aug 08, 2019 at 10:04:27AM +0100, Marc Zyngier wrote:
> On 08/08/2019 08:36, Stefan Agner wrote:
> > I started to ask myself what PER_LINUX32 actually changes. From what I
> > can tell it only changes the behavior of /proc/cpuinfo? The personality
> > seems not to be applied automatically to 32-bit processes, so this is a
> > opt-in backward compatibility feature?
> 
> It's all about giving the illusion that the process runs in a "real"
> 32bit environment, with all its warts. It doesn't change the CPU mode
> you're running in (that'd be a bit harsh). It's only once you exec
> something that requires AArch32 that we engage the COMPAT mode.
> 
> Provided that your kernel contains 00377277166b or a backport of it (or
> that it predates 4378a7d4be30), the following program should do the
> right thing:
> 
> #include <sys/personality.h>
> #include <stdio.h>
> 
> int main(int argc, char *argv[])
> {
>         int old, new;
> 
>         old = personality(PER_LINUX32);
>         if (old < 0) {
>                 perror("No 32bit for you");
>                 return 1;
>         }
> 
>         new = personality(0xffffffff);
>         printf("Running with personality %d\n", new);
> 
>         personality(old);
>         new = personality(0xffffffff);
> 
>         printf("Running with personality %d\n", new);
> 
>         return 0;
> }

Or you can use the setarch/linux32 utility.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Dave Martin]

On Thu, Aug 08, 2019 at 09:36:42AM +0200, Stefan Agner wrote:
> [resend this time with the correct mailing list address]
> 
> Hello,
> 
> I am trying to detect whether an ARMv8 system running in AArch64 state
> supports AArch32 state from a user space process. The arm64_features[]
> in

Why?  Is this just for diagnostic purposes, or some programmatic reason?

In the latter case, just try to do what ever it is you want to do that
depends on AArch32: if it fails, you don't have AArch32.

> arch/arm64/kernel/cpufeature.c lists a CPU feature "32-bit EL0 Support".
> However, afaik this CPU feature is not directly exposed to user-space.
> The features do get printed in the kernel log, but that requires
> privileges and only works directly after boot. There is

Please don't scrape dmesg :)

However, detecting AArch32 support is a bit annoying due to the fact
that there's no hwcap or similar.

> system_supports_32bit_el0() which is used in various places in the arm64
> architecture code. One of the instances where I can make sense of from
> user space is through the personality system call. One idea is to call
> personality(PER_LINUX32). It would then return error code 22 in case
> 32-bit is not supported in user space. However, if successful this
> changes the personality of the current process which might have side
> effects which I do not want...?
> 
> I started to ask myself what PER_LINUX32 actually changes. From what I
> can tell it only changes the behavior of /proc/cpuinfo? The personality
> seems not to be applied automatically to 32-bit processes, so this is a
> opt-in backward compatibility feature?

Basically yes.  Nonetheless, this is probably a reasonable way to test
for AArch32 userspace support.

> To be on the safe side, I was thinking about executing the system call
> in a separate process. However, at that point I could also just execute
> a statically linked AArch32 binary and see whether I get a "exec format
> error". I guess this could then be either due to missing AArch32 CPU
> support or the kernel not being compiled with 32-bit compatibility.

personality() returns the old personality, so you providing you don't
have multiple threads you can probably try to set it to PER_LINUX32
and then restore it.

Otherwise, you would need to fork and try personality() from the child.

Or as you suggest, try to exec a 32-bit binary.
		
> At last I was considering reading directly from the CPU. But from what I
> understand the register used in the kernel to determine 32-bit
> compatibility (ID_AA64PFR0_EL1) is not accessible by user space (due to
> the suffix _EL1).
> 
> Any advice/thoughts on this topic?

This register is emulated for userspace, so you can read it.  However,
the relevant field gets masked out, so this is probably not much use to
you.

We could expose the field, but a test that relies on it wouldn't be
backwards compatible.

If you just want to do this test from a script for diagnostic purposes
and the filesystem has util-linux, then something like

	linux32 /bin/true

might also work (this is effectively a scripted version of the
personality(PER_LINUX32) test).

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Stefan Agner]

On 2019-08-08 11:04, Marc Zyngier wrote:
> Hi Stefan,
> 
> On 08/08/2019 08:36, Stefan Agner wrote:
>> [resend this time with the correct mailing list address]
>>
>> Hello,
>>
>> I am trying to detect whether an ARMv8 system running in AArch64 state
>> supports AArch32 state from a user space process. The arm64_features[]
>> in
>> arch/arm64/kernel/cpufeature.c lists a CPU feature "32-bit EL0 Support".
>> However, afaik this CPU feature is not directly exposed to user-space.
>> The features do get printed in the kernel log, but that requires
>> privileges and only works directly after boot. There is
>> system_supports_32bit_el0() which is used in various places in the arm64
>> architecture code. One of the instances where I can make sense of from
>> user space is through the personality system call. One idea is to call
>> personality(PER_LINUX32). It would then return error code 22 in case
>> 32-bit is not supported in user space. However, if successful this
>> changes the personality of the current process which might have side
>> effects which I do not want...?
> 
> You should be able to revert the effects of PER_LINUX_32 by feeding back
> the return value of the initial call to personality() to a second
> personality() call.
> 

Oh, of course, that makes sense.

>> I started to ask myself what PER_LINUX32 actually changes. From what I
>> can tell it only changes the behavior of /proc/cpuinfo? The personality
>> seems not to be applied automatically to 32-bit processes, so this is a
>> opt-in backward compatibility feature?
> 
> It's all about giving the illusion that the process runs in a "real"
> 32bit environment, with all its warts. It doesn't change the CPU mode
> you're running in (that'd be a bit harsh). It's only once you exec
> something that requires AArch32 that we engage the COMPAT mode.
> 
> Provided that your kernel contains 00377277166b or a backport of it (or
> that it predates 4378a7d4be30), the following program should do the
> right thing:
> 
> #include <sys/personality.h>
> #include <stdio.h>
> 
> int main(int argc, char *argv[])
> {
>         int old, new;
> 
>         old = personality(PER_LINUX32);
>         if (old < 0) {
>                 perror("No 32bit for you");
>                 return 1;
>         }
> 
>         new = personality(0xffffffff);
>         printf("Running with personality %d\n", new);
> 
>         personality(old);
>         new = personality(0xffffffff);
> 
>         printf("Running with personality %d\n", new);
> 
>         return 0;
> }
> 

Thanks for the example. I had something similar already, just not with
the revert part.

>> To be on the safe side, I was thinking about executing the system call
>> in a separate process. However, at that point I could also just execute
>> a statically linked AArch32 binary and see whether I get a "exec format
>> error". I guess this could then be either due to missing AArch32 CPU
>> support or the kernel not being compiled with 32-bit compatibility.
> 
> Overkill ;-). The above should be enough.
> 
>> At last I was considering reading directly from the CPU. But from what I
>> understand the register used in the kernel to determine 32-bit
>> compatibility (ID_AA64PFR0_EL1) is not accessible by user space (due to
>> the suffix _EL1).
> 
> Hey, you could create a VM, a vcpu and dump the ID registers by issuing
> a set of KVM_GET_ONE_REG ioctls. Not necessarily recommended... ;-)

I see, no no, I think I leave that exercise for somebody else to try :)

--
Stefan


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Stefan Agner]

On 2019-08-08 11:15, Will Deacon wrote:
> On Thu, Aug 08, 2019 at 10:04:27AM +0100, Marc Zyngier wrote:
>> On 08/08/2019 08:36, Stefan Agner wrote:
>> > I started to ask myself what PER_LINUX32 actually changes. From what I
>> > can tell it only changes the behavior of /proc/cpuinfo? The personality
>> > seems not to be applied automatically to 32-bit processes, so this is a
>> > opt-in backward compatibility feature?
>>
>> It's all about giving the illusion that the process runs in a "real"
>> 32bit environment, with all its warts. It doesn't change the CPU mode
>> you're running in (that'd be a bit harsh). It's only once you exec
>> something that requires AArch32 that we engage the COMPAT mode.
>>
>> Provided that your kernel contains 00377277166b or a backport of it (or
>> that it predates 4378a7d4be30), the following program should do the
>> right thing:
>>
>> #include <sys/personality.h>
>> #include <stdio.h>
>>
>> int main(int argc, char *argv[])
>> {
>>         int old, new;
>>
>>         old = personality(PER_LINUX32);
>>         if (old < 0) {
>>                 perror("No 32bit for you");
>>                 return 1;
>>         }
>>
>>         new = personality(0xffffffff);
>>         printf("Running with personality %d\n", new);
>>
>>         personality(old);
>>         new = personality(0xffffffff);
>>
>>         printf("Running with personality %d\n", new);
>>
>>         return 0;
>> }
> 
> Or you can use the setarch/linux32 utility.

Wasn't aware of this utility. I guess something like this should work:

$ setarch linux32 true

Thanks for the pointer.

--
Stefan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Stefan Agner]

On 2019-08-08 11:35, Dave Martin wrote:
> On Thu, Aug 08, 2019 at 09:36:42AM +0200, Stefan Agner wrote:
>> [resend this time with the correct mailing list address]
>>
>> Hello,
>>
>> I am trying to detect whether an ARMv8 system running in AArch64 state
>> supports AArch32 state from a user space process. The arm64_features[]
>> in
> 
> Why?  Is this just for diagnostic purposes, or some programmatic reason?

The use case I currently have in mind is to decide whether to show
32-bit ARM Docker images in a UI (or arm32v7 images how it is nowadays
called in Docker land).

> 
> In the latter case, just try to do what ever it is you want to do that
> depends on AArch32: if it fails, you don't have AArch32.


Yeah one option I considered was just fetching a minimalistic arm32v7
container, but still seems a bit excessive.

>
>> arch/arm64/kernel/cpufeature.c lists a CPU feature "32-bit EL0 Support".
>> However, afaik this CPU feature is not directly exposed to user-space.
>> The features do get printed in the kernel log, but that requires
>> privileges and only works directly after boot. There is
> 
> Please don't scrape dmesg :)
> 
> However, detecting AArch32 support is a bit annoying due to the fact
> that there's no hwcap or similar.
> 
>> system_supports_32bit_el0() which is used in various places in the arm64
>> architecture code. One of the instances where I can make sense of from
>> user space is through the personality system call. One idea is to call
>> personality(PER_LINUX32). It would then return error code 22 in case
>> 32-bit is not supported in user space. However, if successful this
>> changes the personality of the current process which might have side
>> effects which I do not want...?
>>
>> I started to ask myself what PER_LINUX32 actually changes. From what I
>> can tell it only changes the behavior of /proc/cpuinfo? The personality
>> seems not to be applied automatically to 32-bit processes, so this is a
>> opt-in backward compatibility feature?
> 
> Basically yes.  Nonetheless, this is probably a reasonable way to test
> for AArch32 userspace support.
> 
>> To be on the safe side, I was thinking about executing the system call
>> in a separate process. However, at that point I could also just execute
>> a statically linked AArch32 binary and see whether I get a "exec format
>> error". I guess this could then be either due to missing AArch32 CPU
>> support or the kernel not being compiled with 32-bit compatibility.
> 
> personality() returns the old personality, so you providing you don't
> have multiple threads you can probably try to set it to PER_LINUX32
> and then restore it.

Yeah that is what Marc also suggested. Probably will go down this road.

> 
> Otherwise, you would need to fork and try personality() from the child.
> 
> Or as you suggest, try to exec a 32-bit binary.
> 
>> At last I was considering reading directly from the CPU. But from what I
>> understand the register used in the kernel to determine 32-bit
>> compatibility (ID_AA64PFR0_EL1) is not accessible by user space (due to
>> the suffix _EL1).
>>
>> Any advice/thoughts on this topic?
> 
> This register is emulated for userspace, so you can read it.  However,
> the relevant field gets masked out, so this is probably not much use to
> you.

Argh, I see.

> 
> We could expose the field, but a test that relies on it wouldn't be
> backwards compatible.
> 
> If you just want to do this test from a script for diagnostic purposes
> and the filesystem has util-linux, then something like
> 
> 	linux32 /bin/true
> 
> might also work (this is effectively a scripted version of the
> personality(PER_LINUX32) test).

Wasn't aware of that one. Thanks!

Thanks for all the answers. Google really did not offer a lot about this
topic, and I am happy we have some options now publicly documented :-)

--
Stefan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Stefan Agner]

On 2019-08-08 12:22, Stefan Agner wrote:
> On 2019-08-08 11:15, Will Deacon wrote:
>> On Thu, Aug 08, 2019 at 10:04:27AM +0100, Marc Zyngier wrote:
>>> On 08/08/2019 08:36, Stefan Agner wrote:
>>> > I started to ask myself what PER_LINUX32 actually changes. From what I
>>> > can tell it only changes the behavior of /proc/cpuinfo? The personality
>>> > seems not to be applied automatically to 32-bit processes, so this is a
>>> > opt-in backward compatibility feature?
>>>
>>> It's all about giving the illusion that the process runs in a "real"
>>> 32bit environment, with all its warts. It doesn't change the CPU mode
>>> you're running in (that'd be a bit harsh). It's only once you exec
>>> something that requires AArch32 that we engage the COMPAT mode.
>>>
>>> Provided that your kernel contains 00377277166b or a backport of it (or
>>> that it predates 4378a7d4be30), the following program should do the
>>> right thing:
>>>
>>> #include <sys/personality.h>
>>> #include <stdio.h>
>>>
>>> int main(int argc, char *argv[])
>>> {
>>>         int old, new;
>>>
>>>         old = personality(PER_LINUX32);
>>>         if (old < 0) {
>>>                 perror("No 32bit for you");
>>>                 return 1;
>>>         }
>>>
>>>         new = personality(0xffffffff);
>>>         printf("Running with personality %d\n", new);
>>>
>>>         personality(old);
>>>         new = personality(0xffffffff);
>>>
>>>         printf("Running with personality %d\n", new);
>>>
>>>         return 0;
>>> }
>>
>> Or you can use the setarch/linux32 utility.
> 
> Wasn't aware of this utility. I guess something like this should work:
> 
> $ setarch linux32 true
> 
> Thanks for the pointer.

FWIW, tested on a Cavium ThunderX on Packet:

test@arm64test:~$ setarch linux32 true
setarch: failed to set personality to linux32: Invalid argument
test@arm64test:~$ echo $?
1

And works on our i.MX 8QuadMax based machine:

apalis-imx8-06446438:~$ setarch linux32 true
apalis-imx8-06446438:~$ echo $?
0

--
Stefan

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Dave Martin]

On Thu, Aug 08, 2019 at 12:30:03PM +0200, Stefan Agner wrote:
> On 2019-08-08 11:35, Dave Martin wrote:
> > On Thu, Aug 08, 2019 at 09:36:42AM +0200, Stefan Agner wrote:
> >> [resend this time with the correct mailing list address]
> >>
> >> Hello,
> >>
> >> I am trying to detect whether an ARMv8 system running in AArch64 state
> >> supports AArch32 state from a user space process. The arm64_features[]
> >> in
> > 
> > Why?  Is this just for diagnostic purposes, or some programmatic reason?
> 
> The use case I currently have in mind is to decide whether to show
> 32-bit ARM Docker images in a UI (or arm32v7 images how it is nowadays
> called in Docker land).
> 
> > 
> > In the latter case, just try to do what ever it is you want to do that
> > depends on AArch32: if it fails, you don't have AArch32.
> 
> 
> Yeah one option I considered was just fetching a minimalistic arm32v7
> container, but still seems a bit excessive.

Ah, right.

So, I guess trying to set the personality or trying to exec a trivial
32-bit binary would answer that.

You could ship a trivial static binary with docker for that: trying
to run a whole container is probably overkill.

If trying to set the personality is more convenient though, that seems
an equally good approach in practice.

[...]

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Detecting AArch32 support from a AArch64 process in user space

[Dave Martin]

On Thu, Aug 08, 2019 at 12:17:26PM +0200, Stefan Agner wrote:
> On 2019-08-08 11:04, Marc Zyngier wrote:

[...]

> > Hey, you could create a VM, a vcpu and dump the ID registers by issuing
> > a set of KVM_GET_ONE_REG ioctls. Not necessarily recommended... ;-)
> 
> I see, no no, I think I leave that exercise for somebody else to try :)

I think KVM guest AArch32 support can exist even when compat is
configured out though.  So just because it works in a guest doesn't mean
that the host kernel supports it for userspace, IIUC.

Otherwise, this would be a fun hack ;)

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel