Endian/__BYTE_ORDER question

Endian/__BYTE_ORDER question

[Joakim Tjernlund]

I am getting confused about on how to test for Endian in the kernel code. In user
space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN

I can see lots of kernel headers using this test too, but it doesn't seem
to be an arch specific file #defining __BYTE_ORDER. Instead I find files like:
 arch/alpha/math-emu/sfp-util.h
 arch/powerpc/include/asm/sfp-machine.h
 arch/s390/include/asm/sfp-util.h
 arch/sh/math-emu/sfp-util.h

How is this supposed to work?

    Jocke

Re: Endian/__BYTE_ORDER question

[Joakim Tjernlund]

>
> I am getting confused about on how to test for Endian in the kernel code. In user
> space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN
>
> I can see lots of kernel headers using this test too, but it doesn't seem
> to be an arch specific file #defining __BYTE_ORDER. Instead I find files like:
>  arch/alpha/math-emu/sfp-util.h
>  arch/powerpc/include/asm/sfp-machine.h
>  arch/s390/include/asm/sfp-util.h
>  arch/sh/math-emu/sfp-util.h
>
> How is this supposed to work?
>
>     Jocke

Figured I should mention that a simple:
#include <asm/byteorder.h>

#ifndef __BYTE_ORDER
# error __BYTE_ORDER is not defined
#endif

Doesn't work.

    Jocke

Re: Endian/__BYTE_ORDER question

[Adrian Reber]

On Thu, Feb 11, 2010 at 05:17:37PM +0100, Joakim Tjernlund wrote:
> I am getting confused about on how to test for Endian in the kernel code. In user
> space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN
> 
> I can see lots of kernel headers using this test too, but it doesn't seem
> to be an arch specific file #defining __BYTE_ORDER. Instead I find files like:
>  arch/alpha/math-emu/sfp-util.h
>  arch/powerpc/include/asm/sfp-machine.h
>  arch/s390/include/asm/sfp-util.h
>  arch/sh/math-emu/sfp-util.h
> 
> How is this supposed to work?

I have no idea how it is actually done in the kernel code... but gcc
defines it:

gcc -dM -E -x c - <<<'' | grep ENDIAN
#define __BIG_ENDIAN__ 1
#define _BIG_ENDIAN 1

		Adrian

Re: Endian/__BYTE_ORDER question

[Joakim Tjernlund]

Adrian Reber <adrian@lisas.de> wrote on 2010/02/11 17:33:29:
>
> On Thu, Feb 11, 2010 at 05:17:37PM +0100, Joakim Tjernlund wrote:
> > I am getting confused about on how to test for Endian in the kernel code. In user
> > space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN
> >
> > I can see lots of kernel headers using this test too, but it doesn't seem
> > to be an arch specific file #defining __BYTE_ORDER. Instead I find files like:
> >  arch/alpha/math-emu/sfp-util.h
> >  arch/powerpc/include/asm/sfp-machine.h
> >  arch/s390/include/asm/sfp-util.h
> >  arch/sh/math-emu/sfp-util.h
> >
> > How is this supposed to work?
>
> I have no idea how it is actually done in the kernel code... but gcc
> defines it:
>
> gcc -dM -E -x c - <<<'' | grep ENDIAN
> #define __BIG_ENDIAN__ 1
> #define _BIG_ENDIAN 1

That doesn't define __BYTE_ORDER. Try the same gcc command
on a file that #includes <stdlib.h> and you will get both
__BIG_ENDIAN and __LITTLE_ENDIAN

     Jocke

Re: Endian/__BYTE_ORDER question

[Wolfgang Denk]

Dear Joakim Tjernlund,

In message <OF918AA866.3ED427EB-ONC12576C7.005CBEE4-C12576C7.005CF730@transmode.se> you wrote:
>
> > I have no idea how it is actually done in the kernel code... but gcc
> > defines it:
> >
> > gcc -dM -E -x c - <<<'' | grep ENDIAN
> > #define __BIG_ENDIAN__ 1
> > #define _BIG_ENDIAN 1
> 
> That doesn't define __BYTE_ORDER. Try the same gcc command
> on a file that #includes <stdlib.h> and you will get both
> __BIG_ENDIAN and __LITTLE_ENDIAN

For me this appears to work:

On x86:

	$ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
	#define _ENDIAN_H 1
	#define PDP_ENDIAN __PDP_ENDIAN
	#define __PDP_ENDIAN 3412
	#define BIG_ENDIAN __BIG_ENDIAN
	#define __BYTE_ORDER __LITTLE_ENDIAN
	#define __LITTLE_ENDIAN 1234
	#define __BIG_ENDIAN 4321
	#define LITTLE_ENDIAN __LITTLE_ENDIAN

On PowerPC:

	$ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
	#define __BIG_ENDIAN__ 1
	#define __PDP_ENDIAN 3412
	#define __LITTLE_ENDIAN 1234
	#define BIG_ENDIAN __BIG_ENDIAN
	#define _BIG_ENDIAN 1
	#define __BYTE_ORDER __BIG_ENDIAN
	#define _ENDIAN_H 1
	#define __BIG_ENDIAN 4321
	#define PDP_ENDIAN __PDP_ENDIAN
	#define LITTLE_ENDIAN __LITTLE_ENDIAN

In both cases __BYTE_ORDER is set to a sane value.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Copy from one, it's plagiarism; copy from two, it's research.

Re: Endian/__BYTE_ORDER question

[Joakim Tjernlund]

Wolfgang Denk <wd@denx.de> wrote on 2010/02/11 22:39:00:
>
> Dear Joakim Tjernlund,
>
> In message <OF918AA866.3ED427EB-ONC12576C7.005CBEE4-C12576C7.
> 005CF730@transmode.se> you wrote:
> >
> > > I have no idea how it is actually done in the kernel code... but gcc
> > > defines it:
> > >
> > > gcc -dM -E -x c - <<<'' | grep ENDIAN
> > > #define __BIG_ENDIAN__ 1
> > > #define _BIG_ENDIAN 1
> >
> > That doesn't define __BYTE_ORDER. Try the same gcc command
> > on a file that #includes <stdlib.h> and you will get both
> > __BIG_ENDIAN and __LITTLE_ENDIAN
>
> For me this appears to work:
>
> On x86:
>
>    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
>    #define _ENDIAN_H 1
>    #define PDP_ENDIAN __PDP_ENDIAN
>    #define __PDP_ENDIAN 3412
>    #define BIG_ENDIAN __BIG_ENDIAN
>    #define __BYTE_ORDER __LITTLE_ENDIAN
>    #define __LITTLE_ENDIAN 1234
>    #define __BIG_ENDIAN 4321
>    #define LITTLE_ENDIAN __LITTLE_ENDIAN
>
> On PowerPC:
>
>    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
>    #define __BIG_ENDIAN__ 1
>    #define __PDP_ENDIAN 3412
>    #define __LITTLE_ENDIAN 1234
>    #define BIG_ENDIAN __BIG_ENDIAN
>    #define _BIG_ENDIAN 1
>    #define __BYTE_ORDER __BIG_ENDIAN
>    #define _ENDIAN_H 1
>    #define __BIG_ENDIAN 4321
>    #define PDP_ENDIAN __PDP_ENDIAN
>    #define LITTLE_ENDIAN __LITTLE_ENDIAN
>
> In both cases __BYTE_ORDER is set to a sane value.

Yes, but that was not what I meant. If you look closer you see that both
__BIG_ENDIAN and __LITTLE_ENDIAN are defined in each compiler. This means
you cannot use these two defines directly to tell if your arch is BE or LE.
Guess what the kernel uses :(

I think the kernel should #define __BYTE_ORDER properly so one
can use __BYTE_ORDER in the kernel too.
Just grep for __BYTE_ORDER in the linux tree to see some ugly
workarounds just because __BYTE_ORDER isn't defined.

 Jocke

Re: Endian/__BYTE_ORDER question

[Geert Uytterhoeven]

On Fri, 12 Feb 2010, Joakim Tjernlund wrote:
> Wolfgang Denk <wd@denx.de> wrote on 2010/02/11 22:39:00:
> > Dear Joakim Tjernlund,
> >
> > In message <OF918AA866.3ED427EB-ONC12576C7.005CBEE4-C12576C7.
> > 005CF730@transmode.se> you wrote:
> > >
> > > > I have no idea how it is actually done in the kernel code... but gc=
c
> > > > defines it:
> > > >
> > > > gcc -dM -E -x c - <<<'' | grep ENDIAN
> > > > #define __BIG_ENDIAN__ 1
> > > > #define _BIG_ENDIAN 1
> > >
> > > That doesn't define __BYTE_ORDER. Try the same gcc command
> > > on a file that #includes <stdlib.h> and you will get both
> > > __BIG_ENDIAN and __LITTLE_ENDIAN
> >
> > For me this appears to work:
> >
> > On x86:
> >
> >    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
> >    #define _ENDIAN_H 1
> >    #define PDP_ENDIAN __PDP_ENDIAN
> >    #define __PDP_ENDIAN 3412
> >    #define BIG_ENDIAN __BIG_ENDIAN
> >    #define __BYTE_ORDER __LITTLE_ENDIAN
> >    #define __LITTLE_ENDIAN 1234
> >    #define __BIG_ENDIAN 4321
> >    #define LITTLE_ENDIAN __LITTLE_ENDIAN
> >
> > On PowerPC:
> >
> >    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
> >    #define __BIG_ENDIAN__ 1
> >    #define __PDP_ENDIAN 3412
> >    #define __LITTLE_ENDIAN 1234
> >    #define BIG_ENDIAN __BIG_ENDIAN
> >    #define _BIG_ENDIAN 1
> >    #define __BYTE_ORDER __BIG_ENDIAN
> >    #define _ENDIAN_H 1
> >    #define __BIG_ENDIAN 4321
> >    #define PDP_ENDIAN __PDP_ENDIAN
> >    #define LITTLE_ENDIAN __LITTLE_ENDIAN
> >
> > In both cases __BYTE_ORDER is set to a sane value.
>=20
> Yes, but that was not what I meant. If you look closer you see that both
> __BIG_ENDIAN and __LITTLE_ENDIAN are defined in each compiler. This means
> you cannot use these two defines directly to tell if your arch is BE or L=
E.

The proper tests are:
  - #if __BYTE_ORDER =3D=3D __BIG_ENDIAN
  - #if __BYTE_ORDER =3D=3D __LITTLE_ENDIAN
i.e. don't just test for the presence of __BIG_ENDIAN or __LITTLE_ENDIAN.

With kind regards,

Geert Uytterhoeven
Software Architect
Techsoft Centre

Technology and Software Centre Europe
The Corporate Village =B7 Da Vincilaan 7-D1 =B7 B-1935 Zaventem =B7 Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 =B7 RPR Brussels
Fortis =B7 BIC GEBABEBB =B7 IBAN BE41293037680010

Re: Endian/__BYTE_ORDER question

[Joakim Tjernlund]

geert@sonytel.be wrote on 2010/02/12 11:33:02:
>
> On Fri, 12 Feb 2010, Joakim Tjernlund wrote:
> > Wolfgang Denk <wd@denx.de> wrote on 2010/02/11 22:39:00:
> > > Dear Joakim Tjernlund,
> > >
> > > In message <OF918AA866.3ED427EB-ONC12576C7.005CBEE4-C12576C7.
> > > 005CF730@transmode.se> you wrote:
> > > >
> > > > > I have no idea how it is actually done in the kernel code... but gcc
> > > > > defines it:
> > > > >
> > > > > gcc -dM -E -x c - <<<'' | grep ENDIAN
> > > > > #define __BIG_ENDIAN__ 1
> > > > > #define _BIG_ENDIAN 1
> > > >
> > > > That doesn't define __BYTE_ORDER. Try the same gcc command
> > > > on a file that #includes <stdlib.h> and you will get both
> > > > __BIG_ENDIAN and __LITTLE_ENDIAN
> > >
> > > For me this appears to work:
> > >
> > > On x86:
> > >
> > >    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
> > >    #define _ENDIAN_H 1
> > >    #define PDP_ENDIAN __PDP_ENDIAN
> > >    #define __PDP_ENDIAN 3412
> > >    #define BIG_ENDIAN __BIG_ENDIAN
> > >    #define __BYTE_ORDER __LITTLE_ENDIAN
> > >    #define __LITTLE_ENDIAN 1234
> > >    #define __BIG_ENDIAN 4321
> > >    #define LITTLE_ENDIAN __LITTLE_ENDIAN
> > >
> > > On PowerPC:
> > >
> > >    $ echo '#include <stdlib.h>' | gcc -dM -E -x c - | grep ENDIAN
> > >    #define __BIG_ENDIAN__ 1
> > >    #define __PDP_ENDIAN 3412
> > >    #define __LITTLE_ENDIAN 1234
> > >    #define BIG_ENDIAN __BIG_ENDIAN
> > >    #define _BIG_ENDIAN 1
> > >    #define __BYTE_ORDER __BIG_ENDIAN
> > >    #define _ENDIAN_H 1
> > >    #define __BIG_ENDIAN 4321
> > >    #define PDP_ENDIAN __PDP_ENDIAN
> > >    #define LITTLE_ENDIAN __LITTLE_ENDIAN
> > >
> > > In both cases __BYTE_ORDER is set to a sane value.
> >
> > Yes, but that was not what I meant. If you look closer you see that both
> > __BIG_ENDIAN and __LITTLE_ENDIAN are defined in each compiler. This means
> > you cannot use these two defines directly to tell if your arch is BE or LE.
>
> The proper tests are:
>   - #if __BYTE_ORDER == __BIG_ENDIAN
>   - #if __BYTE_ORDER == __LITTLE_ENDIAN
> i.e. don't just test for the presence of __BIG_ENDIAN or __LITTLE_ENDIAN.

Yes, but my point is that __BYTE_ORDER isn't defined in the kernel so
there the correct test actually is to test __BIG_ENDIAN/__LITTLE_ENDIAN
directly. Look at lib/crc32.c, it also includes some test code
so one can verify the algo. If you enable the test code and move
it all to user space(some minor hacks needed to do that) one fail
the test on BE machines because both __BIG_ENDIAN and __LITTLE_ENDIAN
are defined in user space so the crc code thinks it still is LE.

There is some ugly code that messes with __BYTE_ORDER in the kernel,
all just because the kernel doesn't define __BYTE_ORDER. Is there
some rule that prevents the kernel to define __BYTE_ORDER?

    Jocke
   Jocke

Re: Endian/__BYTE_ORDER question

[Andreas Schwab]

Joakim Tjernlund <joakim.tjernlund@transmode.se> writes:

> I am getting confused about on how to test for Endian in the kernel code. In user
> space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN
>
> I can see lots of kernel headers using this test too

Only outside of __KERNEL__.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Endian/__BYTE_ORDER question

[Joakim Tjernlund]

Andreas Schwab <schwab@linux-m68k.org> wrote on 2010/02/11 17:35:54:
>
> Joakim Tjernlund <joakim.tjernlund@transmode.se> writes:
>
> > I am getting confused about on how to test for Endian in the kernel code. In user
> > space one uses #if __BYTE_ORDER == __LITTLE_ENDIAN or #if __BYTE_ORDER == __BIG_ENDIAN
> >
> > I can see lots of kernel headers using this test too
>
> Only outside of __KERNEL__.

Ah, yes now I see. This is unfortunate as if one moves kernel
code to user space and still use the kernel way:
 #ifdef __LITTLE_ENDIAN
one will be in big trouble as including stdlib.h will define
both __LITTLE_ENDIAN and __BIG_ENDIAN. Spent quite a while
chasing that one down :(

      Jocke