Aarch64 kernel with 32bit userspace question

Aarch64 kernel with 32bit userspace question

[Marek Vasut]

Hi,

I'm trying multilib userland on aarch64, but I'm running into a problem.
I have a simple test code:

-->8--
#include <signal.h>

int main(void) {
    return 0;
}
--8<--

If I compile that with aarch64 gcc , it compiles just fine.

If I compile the same thing with 32bit armv7ahf multilib gcc, the
build fails on "unknown type name '__uint128_t'". This comes from
arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
struct fpsimd_context {} . The signal.h includes that (through a few
glibc headers) and that's what triggers the failure. __uint128_t is
defined on aarch64 , but it is not on armv7a (32bit).

Grepping through the kernel sources , __uint128_t is used in some three
places on aarch64 (and in bluetooth/ecc.c , where it's defined as a
struct of two u64 internally in that file, so irrelevant). I wonder
whether the usage of __uint128_t is really correct in sigcontext.h and
whether it shouldn't be replaced by some u64 array or such ?

Or is there any other way to use aarch64 kernel (and kernel headers)
with 32bit userspace ?

Thanks!

-- 
Best regards,
Marek Vasut

Aarch64 kernel with 32bit userspace question

[Jisheng Zhang]

On Thu, 9 Feb 2017 11:14:19 +0100
Marek Vasut <marex@denx.de> wrote:

> Hi,
> 
> I'm trying multilib userland on aarch64, but I'm running into a problem.
> I have a simple test code:
> 
> -->8--  
> #include <signal.h>
> 
> int main(void) {
>     return 0;
> }
> --8<--
> 
> If I compile that with aarch64 gcc , it compiles just fine.
> 
> If I compile the same thing with 32bit armv7ahf multilib gcc, the
> build fails on "unknown type name '__uint128_t'". This comes from
> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
> struct fpsimd_context {} . The signal.h includes that (through a few
> glibc headers) and that's what triggers the failure. __uint128_t is
> defined on aarch64 , but it is not on armv7a (32bit).
> 

which 32bit toolchain are you using?

I didn't see any problems with 32bit armhf linaro toolchain, or the
toolchain in debian armhf or ubuntu

Aarch64 kernel with 32bit userspace question

[Marek Vasut]

On 02/09/2017 11:29 AM, Jisheng Zhang wrote:
> On Thu, 9 Feb 2017 11:14:19 +0100
> Marek Vasut <marex@denx.de> wrote:
> 
>> Hi,
>>
>> I'm trying multilib userland on aarch64, but I'm running into a problem.
>> I have a simple test code:
>>
>> -->8--  
>> #include <signal.h>
>>
>> int main(void) {
>>     return 0;
>> }
>> --8<--
>>
>> If I compile that with aarch64 gcc , it compiles just fine.
>>
>> If I compile the same thing with 32bit armv7ahf multilib gcc, the
>> build fails on "unknown type name '__uint128_t'". This comes from
>> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
>> struct fpsimd_context {} . The signal.h includes that (through a few
>> glibc headers) and that's what triggers the failure. __uint128_t is
>> defined on aarch64 , but it is not on armv7a (32bit).
>>
> 
> which 32bit toolchain are you using?

Multilib toolchain from poky 2.2 (gcc 6.2 , glibc 2.24).

> I didn't see any problems with 32bit armhf linaro toolchain, or the
> toolchain in debian armhf or ubuntu

Can you be more specific about the version ?

-- 
Best regards,
Marek Vasut

Aarch64 kernel with 32bit userspace question

[Jisheng Zhang]

On Thu, 9 Feb 2017 11:43:03 +0100 Marek Vasut wrote:

> On 02/09/2017 11:29 AM, Jisheng Zhang wrote:
> > On Thu, 9 Feb 2017 11:14:19 +0100
> > Marek Vasut <marex@denx.de> wrote:
> >   
> >> Hi,
> >>
> >> I'm trying multilib userland on aarch64, but I'm running into a problem.
> >> I have a simple test code:
> >>  
> >> -->8--    
> >> #include <signal.h>
> >>
> >> int main(void) {
> >>     return 0;
> >> }
> >> --8<--
> >>
> >> If I compile that with aarch64 gcc , it compiles just fine.
> >>
> >> If I compile the same thing with 32bit armv7ahf multilib gcc, the
> >> build fails on "unknown type name '__uint128_t'". This comes from
> >> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
> >> struct fpsimd_context {} . The signal.h includes that (through a few
> >> glibc headers) and that's what triggers the failure. __uint128_t is
> >> defined on aarch64 , but it is not on armv7a (32bit).
> >>  
> > 
> > which 32bit toolchain are you using?  
> 
> Multilib toolchain from poky 2.2 (gcc 6.2 , glibc 2.24).
> 
> > I didn't see any problems with 32bit armhf linaro toolchain, or the
> > toolchain in debian armhf or ubuntu  
> 
> Can you be more specific about the version ?
> 

the one I'm using:

Using built-in specs.
COLLECT_GCC=arm-linux-gnueabihf-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabihf/6/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-5' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libquadmath --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-armhf-cross/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-armhf-cross --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-armhf-cross --with-arch-directory=arm --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj --with-targ
 et-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=arm-linux-gnueabihf --program-prefix=arm-linux-gnueabihf- --includedir=/usr/arm-linux-gnueabihf/include
Thread model: posix
gcc version 6.3.0 20170124 (Debian 6.3.0-5) 

Aarch64 kernel with 32bit userspace question

[Ard Biesheuvel]

On 9 February 2017 at 10:14, Marek Vasut <marex@denx.de> wrote:
> Hi,
>
> I'm trying multilib userland on aarch64, but I'm running into a problem.
> I have a simple test code:
>
> -->8--
> #include <signal.h>
>
> int main(void) {
>     return 0;
> }
> --8<--
>
> If I compile that with aarch64 gcc , it compiles just fine.
>
> If I compile the same thing with 32bit armv7ahf multilib gcc, the
> build fails on "unknown type name '__uint128_t'". This comes from
> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
> struct fpsimd_context {} . The signal.h includes that (through a few
> glibc headers) and that's what triggers the failure. __uint128_t is
> defined on aarch64 , but it is not on armv7a (32bit).
>

This is a toolchain or glibc bug: the arm64 version of that header
should never be pulled in when using a AArch32 toolchain, regardless
of whether you run it on arm64, ARM (or x86, for that matter)



> Grepping through the kernel sources , __uint128_t is used in some three
> places on aarch64 (and in bluetooth/ecc.c , where it's defined as a
> struct of two u64 internally in that file, so irrelevant). I wonder
> whether the usage of __uint128_t is really correct in sigcontext.h and
> whether it shouldn't be replaced by some u64 array or such ?
>
> Or is there any other way to use aarch64 kernel (and kernel headers)
> with 32bit userspace ?
>
> Thanks!
>
> --
> Best regards,
> Marek Vasut
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Aarch64 kernel with 32bit userspace question

[Marek Vasut]

On 02/09/2017 11:51 AM, Ard Biesheuvel wrote:
> On 9 February 2017 at 10:14, Marek Vasut <marex@denx.de> wrote:
>> Hi,
>>
>> I'm trying multilib userland on aarch64, but I'm running into a problem.
>> I have a simple test code:
>>
>> -->8--
>> #include <signal.h>
>>
>> int main(void) {
>>     return 0;
>> }
>> --8<--
>>
>> If I compile that with aarch64 gcc , it compiles just fine.
>>
>> If I compile the same thing with 32bit armv7ahf multilib gcc, the
>> build fails on "unknown type name '__uint128_t'". This comes from
>> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
>> struct fpsimd_context {} . The signal.h includes that (through a few
>> glibc headers) and that's what triggers the failure. __uint128_t is
>> defined on aarch64 , but it is not on armv7a (32bit).
>>
> 
> This is a toolchain or glibc bug: the arm64 version of that header
> should never be pulled in when using a AArch32 toolchain, regardless
> of whether you run it on arm64, ARM (or x86, for that matter)

Aha. So which version of the header should be pulled in ? The one from
arch/arm/include/uapi/asm/sigcontext.h (that doesn't make much sense to
me) ?

-- 
Best regards,
Marek Vasut

Aarch64 kernel with 32bit userspace question

[Ard Biesheuvel]

On 9 February 2017 at 11:14, Marek Vasut <marex@denx.de> wrote:
> On 02/09/2017 11:51 AM, Ard Biesheuvel wrote:
>> On 9 February 2017 at 10:14, Marek Vasut <marex@denx.de> wrote:
>>> Hi,
>>>
>>> I'm trying multilib userland on aarch64, but I'm running into a problem.
>>> I have a simple test code:
>>>
>>> -->8--
>>> #include <signal.h>
>>>
>>> int main(void) {
>>>     return 0;
>>> }
>>> --8<--
>>>
>>> If I compile that with aarch64 gcc , it compiles just fine.
>>>
>>> If I compile the same thing with 32bit armv7ahf multilib gcc, the
>>> build fails on "unknown type name '__uint128_t'". This comes from
>>> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
>>> struct fpsimd_context {} . The signal.h includes that (through a few
>>> glibc headers) and that's what triggers the failure. __uint128_t is
>>> defined on aarch64 , but it is not on armv7a (32bit).
>>>
>>
>> This is a toolchain or glibc bug: the arm64 version of that header
>> should never be pulled in when using a AArch32 toolchain, regardless
>> of whether you run it on arm64, ARM (or x86, for that matter)
>
> Aha. So which version of the header should be pulled in ? The one from
> arch/arm/include/uapi/asm/sigcontext.h (that doesn't make much sense to
> me) ?
>

signal.h includes bits/sigcontext.h, of which several versions exists
on my system:

/usr/aarch64-linux-gnu/include/bits/sigcontext.h
/usr/arm-linux-gnueabi/include/bits/sigcontext.h
/usr/arm-linux-gnueabihf/include/bits/sigcontext.h
/usr/include/x86_64-linux-gnu/bits/sigcontext.h

each of which includes asm/sigcontext.h under the same root path.

It is up to the compiler to set the builtin include paths correctly:
arm-linux-gnueabi-gcc should never use the aarch64 or x86 one.

Aarch64 kernel with 32bit userspace question

[Russell King - ARM Linux]

On Thu, Feb 09, 2017 at 11:24:43AM +0000, Ard Biesheuvel wrote:
> On 9 February 2017 at 11:14, Marek Vasut <marex@denx.de> wrote:
> > On 02/09/2017 11:51 AM, Ard Biesheuvel wrote:
> >> On 9 February 2017 at 10:14, Marek Vasut <marex@denx.de> wrote:
> >>> Hi,
> >>>
> >>> I'm trying multilib userland on aarch64, but I'm running into a problem.
> >>> I have a simple test code:
> >>>
> >>> -->8--
> >>> #include <signal.h>
> >>>
> >>> int main(void) {
> >>>     return 0;
> >>> }
> >>> --8<--
> >>>
> >>> If I compile that with aarch64 gcc , it compiles just fine.
> >>>
> >>> If I compile the same thing with 32bit armv7ahf multilib gcc, the
> >>> build fails on "unknown type name '__uint128_t'". This comes from
> >>> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
> >>> struct fpsimd_context {} . The signal.h includes that (through a few
> >>> glibc headers) and that's what triggers the failure. __uint128_t is
> >>> defined on aarch64 , but it is not on armv7a (32bit).
> >>>
> >>
> >> This is a toolchain or glibc bug: the arm64 version of that header
> >> should never be pulled in when using a AArch32 toolchain, regardless
> >> of whether you run it on arm64, ARM (or x86, for that matter)
> >
> > Aha. So which version of the header should be pulled in ? The one from
> > arch/arm/include/uapi/asm/sigcontext.h (that doesn't make much sense to
> > me) ?
> >
> 
> signal.h includes bits/sigcontext.h, of which several versions exists
> on my system:
> 
> /usr/aarch64-linux-gnu/include/bits/sigcontext.h
> /usr/arm-linux-gnueabi/include/bits/sigcontext.h
> /usr/arm-linux-gnueabihf/include/bits/sigcontext.h
> /usr/include/x86_64-linux-gnu/bits/sigcontext.h
> 
> each of which includes asm/sigcontext.h under the same root path.
> 
> It is up to the compiler to set the builtin include paths correctly:
> arm-linux-gnueabi-gcc should never use the aarch64 or x86 one.

The issue may be where arch/arm*/include/uapi/asm/sigcontext.h has
been installed to.

GCC may be doing the right thing, so selecting the right include
directory, but they all include "asm/sigcontext.h".

If the ARM64 UAPI headers got dropped into /usr/include/asm, then
the compiler may be picking these up rather than the proper ones
in the appropriate directory.  For me:

	/usr/include/arm-linux-gnueabihf/bits/sigcontext.h

should be picking up on:

	/usr/include/arm-linux-gnueabihf/asm/sigcontext.h

and there should not be a /usr/include/asm directory.

The default kernel headers_install will want to place the headers into
<includepath>/asm along side <includepath>/asm-generic and
<includepath>/linux which is not what modern ARM distros want.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Aarch64 kernel with 32bit userspace question

[Marek Vasut]

On 02/09/2017 01:25 PM, Russell King - ARM Linux wrote:
> On Thu, Feb 09, 2017 at 11:24:43AM +0000, Ard Biesheuvel wrote:
>> On 9 February 2017 at 11:14, Marek Vasut <marex@denx.de> wrote:
>>> On 02/09/2017 11:51 AM, Ard Biesheuvel wrote:
>>>> On 9 February 2017 at 10:14, Marek Vasut <marex@denx.de> wrote:
>>>>> Hi,
>>>>>
>>>>> I'm trying multilib userland on aarch64, but I'm running into a problem.
>>>>> I have a simple test code:
>>>>>
>>>>> -->8--
>>>>> #include <signal.h>
>>>>>
>>>>> int main(void) {
>>>>>     return 0;
>>>>> }
>>>>> --8<--
>>>>>
>>>>> If I compile that with aarch64 gcc , it compiles just fine.
>>>>>
>>>>> If I compile the same thing with 32bit armv7ahf multilib gcc, the
>>>>> build fails on "unknown type name '__uint128_t'". This comes from
>>>>> arch/arm64/include/uapi/asm/sigcontext.h , which has __uint128_t in
>>>>> struct fpsimd_context {} . The signal.h includes that (through a few
>>>>> glibc headers) and that's what triggers the failure. __uint128_t is
>>>>> defined on aarch64 , but it is not on armv7a (32bit).
>>>>>
>>>>
>>>> This is a toolchain or glibc bug: the arm64 version of that header
>>>> should never be pulled in when using a AArch32 toolchain, regardless
>>>> of whether you run it on arm64, ARM (or x86, for that matter)
>>>
>>> Aha. So which version of the header should be pulled in ? The one from
>>> arch/arm/include/uapi/asm/sigcontext.h (that doesn't make much sense to
>>> me) ?
>>>
>>
>> signal.h includes bits/sigcontext.h, of which several versions exists
>> on my system:
>>
>> /usr/aarch64-linux-gnu/include/bits/sigcontext.h
>> /usr/arm-linux-gnueabi/include/bits/sigcontext.h
>> /usr/arm-linux-gnueabihf/include/bits/sigcontext.h
>> /usr/include/x86_64-linux-gnu/bits/sigcontext.h
>>
>> each of which includes asm/sigcontext.h under the same root path.
>>
>> It is up to the compiler to set the builtin include paths correctly:
>> arm-linux-gnueabi-gcc should never use the aarch64 or x86 one.
> 
> The issue may be where arch/arm*/include/uapi/asm/sigcontext.h has
> been installed to.
> 
> GCC may be doing the right thing, so selecting the right include
> directory, but they all include "asm/sigcontext.h".
> 
> If the ARM64 UAPI headers got dropped into /usr/include/asm, then
> the compiler may be picking these up rather than the proper ones
> in the appropriate directory.  For me:
> 
> 	/usr/include/arm-linux-gnueabihf/bits/sigcontext.h
> 
> should be picking up on:
> 
> 	/usr/include/arm-linux-gnueabihf/asm/sigcontext.h
> 
> and there should not be a /usr/include/asm directory.
> 
> The default kernel headers_install will want to place the headers into
> <includepath>/asm along side <includepath>/asm-generic and
> <includepath>/linux which is not what modern ARM distros want.

Thanks for all the help, it at least helped me get a better picture how
the multilib stuff works on arm/aarch64 .

It turns out that OE is not yet able to generate multilib sdk, while it
is able to generate multilib system image. Therefore, while I had two
different compilers, only one sysroot was installed (aarch64 one) and
both were pointing into it. The 32bit compiler used the wrong headers,
so I ran into this issue.  I'll have to drill into the OE further to
figure out the proper solution.

-- 
Best regards,
Marek Vasut

Aarch64 kernel with 32bit userspace question

[Arnd Bergmann]

On Thursday, February 9, 2017 9:38:41 PM CET Marek Vasut wrote:
> > 
> > The default kernel headers_install will want to place the headers into
> > <includepath>/asm along side <includepath>/asm-generic and
> > <includepath>/linux which is not what modern ARM distros want.
> 
> Thanks for all the help, it at least helped me get a better picture how
> the multilib stuff works on arm/aarch64 .
> 
> It turns out that OE is not yet able to generate multilib sdk, while it
> is able to generate multilib system image. Therefore, while I had two
> different compilers, only one sysroot was installed (aarch64 one) and
> both were pointing into it. The 32bit compiler used the wrong headers,
> so I ran into this issue.  I'll have to drill into the OE further to
> figure out the proper solution.

It seems the support for biarch headers was removed from the kernel
a few years before arm64 got merged: https://lkml.org/lkml/2008/6/30/467

You could still use "make headers_install_all" in place of
"make headers_install" in 
http://git.yoctoproject.org/cgit.cgi/poky/plain/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
but you still have to figure out how to find the right headers,
either by moving the two asm directories to /usr/{arm,aarch64}-linux-gnu/include/asm
or by doing the same trick that we had before the 2008 patch.

	Arnd

Aarch64 kernel with 32bit userspace question

[Marek Vasut]

On 02/10/2017 01:39 PM, Arnd Bergmann wrote:
> On Thursday, February 9, 2017 9:38:41 PM CET Marek Vasut wrote:
>>>
>>> The default kernel headers_install will want to place the headers into
>>> <includepath>/asm along side <includepath>/asm-generic and
>>> <includepath>/linux which is not what modern ARM distros want.
>>
>> Thanks for all the help, it at least helped me get a better picture how
>> the multilib stuff works on arm/aarch64 .
>>
>> It turns out that OE is not yet able to generate multilib sdk, while it
>> is able to generate multilib system image. Therefore, while I had two
>> different compilers, only one sysroot was installed (aarch64 one) and
>> both were pointing into it. The 32bit compiler used the wrong headers,
>> so I ran into this issue.  I'll have to drill into the OE further to
>> figure out the proper solution.
> 
> It seems the support for biarch headers was removed from the kernel
> a few years before arm64 got merged: https://lkml.org/lkml/2008/6/30/467
> 
> You could still use "make headers_install_all" in place of
> "make headers_install" in 

Whoa, I didn't know about this trick, nice :)

> http://git.yoctoproject.org/cgit.cgi/poky/plain/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> but you still have to figure out how to find the right headers,
> either by moving the two asm directories to /usr/{arm,aarch64}-linux-gnu/include/asm
> or by doing the same trick that we had before the 2008 patch.

Well, I think I will wait for the multilib support to be fixed in OE
(someone is already working on it). Until then, I will stick with 32bit
userspace, which should be good enough for now.

Thanks for the help!

-- 
Best regards,
Marek Vasut