Help with cross-endian bitfields?

Help with cross-endian bitfields?

[jlquinn]

Hi, all.  I'm working on getting the Advansys driver working and what
appears to be the final stumbling block is a difference in the layout of
bitfields between little-endian and big-endian systems.  In particular,
there are structs like the following:

struct blah {
  uchar a : 3;
  uchar b : 3;
  uchar c : 2;
}

On a little endian machine, this works like I expect.  (blah & 0x07) ==
blah.a is true.  On linuxppc (and apparently MacOS as well), this is NOT
true.  blah.a refers to the 3 most significant bits.  The linuxppc behavior
seems counterintuitive to me because I expected each successive member in a
struct to be the next physical piece of data I encounter.

Can someone tell me what's going on here?  Am I going to have to have
ifdef's on these structs, or resort to mask macros, or is there a
reasonable solution here.

Thanks,
Jerry Quinn

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[Dan Malek]

jlquinn@us.ibm.com wrote:

> Hi, all.  I'm working on getting the Advansys driver working and what
> appears to be the final stumbling block is a difference in the layout of
> bitfields between little-endian and big-endian systems.

IMHO, bit fields are not normally a good thing to be using if this is
describing a real device register.  The compiler is able to read/write
in ways that may not be proper for hardware access.  For device registers
and when this is a data structure in memory, the cross-platform method
of using macros for reading writing 16- and 32-bit values and then 'C'
operators for testing/setting bits has worked pretty well in the past.....


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[Josh Huber]

On Fri, May 05, 2000 at 04:05:52PM -0400, Dan Malek wrote:
> IMHO, bit fields are not normally a good thing to be using if this is
> describing a real device register.  The compiler is able to read/write
> in ways that may not be proper for hardware access.  For device registers
> and when this is a data structure in memory, the cross-platform method
> of using macros for reading writing 16- and 32-bit values and then 'C'
> operators for testing/setting bits has worked pretty well in the past.....
Yeah, I just looked in the PowerPC ABI for bitfield storage, and the
relevant text said:
* Bit-fields are allocated from right to left (least to most significant) on
Little-Endian implementations and from left to right (most to least
significant) on Big-Endian implementations.

Josh

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[jlquinn]

The data is read from the device as a complete byte, so that aspect is OK.
However, the byte contains multiple pieces of information that can be
conveniently accessed using bitfields.  It's less convenient to write
macros that mask and shift the appropriate piece, although it will be
obviously portable.

Jerry


Dan Malek <dan@netx4.com>@lists.linuxppc.org on 05/05/2000 04:05:52 PM




jlquinn@us.ibm.com wrote:

> Hi, all.  I'm working on getting the Advansys driver working and what
> appears to be the final stumbling block is a difference in the layout of
> bitfields between little-endian and big-endian systems.

IMHO, bit fields are not normally a good thing to be using if this is
describing a real device register.  The compiler is able to read/write
in ways that may not be proper for hardware access.  For device registers
and when this is a data structure in memory, the cross-platform method
of using macros for reading writing 16- and 32-bit values and then 'C'
operators for testing/setting bits has worked pretty well in the past.....


     -- Dan


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[Daniel Jacobowitz]

On Fri, May 05, 2000 at 03:27:21PM -0400, jlquinn@us.ibm.com wrote:
>
> Hi, all.  I'm working on getting the Advansys driver working and what
> appears to be the final stumbling block is a difference in the layout of
> bitfields between little-endian and big-endian systems.  In particular,
> there are structs like the following:
>
> struct blah {
>   uchar a : 3;
>   uchar b : 3;
>   uchar c : 2;
> }
>
> On a little endian machine, this works like I expect.  (blah & 0x07) ==
> blah.a is true.  On linuxppc (and apparently MacOS as well), this is NOT
> true.  blah.a refers to the 3 most significant bits.  The linuxppc behavior
> seems counterintuitive to me because I expected each successive member in a
> struct to be the next physical piece of data I encounter.
>
> Can someone tell me what's going on here?  Am I going to have to have
> ifdef's on these structs, or resort to mask macros, or is there a
> reasonable solution here.

As Dan Malek said, there is no defined ordering.  The compiler is free
to do whatever it wishes.  You can't assume that.  (I think.)

Also, by your intuition, the obvious thing IS happening.  Remember that
we are big-endian - the first thing in memory is big-endian.  Thus, it
follows logically that a should be in the first three (most
significant) bits.

Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[james woodyatt]

Daniel Jacobowitz <drow@false.org> wrote:
>On Fri, May 05, 2000 at 03:27:21PM -0400, jlquinn@us.ibm.com wrote:
>>
[...]
>> Can someone tell me what's going on here?  Am I going to have to have
>> ifdef's on these structs, or resort to mask macros, or is there a
>> reasonable solution here.
>
>As Dan Malek said, there is no defined ordering.  The compiler is free
>to do whatever it wishes.  You can't assume that.  (I think.)
[...]

It's absolutely true.

Not only is there no defined ordering, the ANSI standard says that it
doesn't even need to use bitfield operations in the machine code.  It's
allowed to pad arbitrarily between the bitfields.  It's allowed to
completely ignore the bitfield widths and treat the members like integers.
It's even allowed to order the very bits themselves in weird ways, if it
thinks the processor will like it better that way.

Bitfields are just not even remotely portable.

C structures should not be used to represent portable serializations of
external data.  Period.  Yes, they're convenient-- but your compiler and my
compiler don't have to agree on practically anything about them.


--
j h woodyatt <jhw@wetware.com>
http://www.wetware.com/jhw
"Yeah, Mercury's going into retrograde, but
at least we're not the Kennedys."  --sean poreda


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[Geert Uytterhoeven]

On Mon, 8 May 2000, james  woodyatt wrote:
> Daniel Jacobowitz <drow@false.org> wrote:
> >On Fri, May 05, 2000 at 03:27:21PM -0400, jlquinn@us.ibm.com wrote:
> >>
> [...]
> >> Can someone tell me what's going on here?  Am I going to have to have
> >> ifdef's on these structs, or resort to mask macros, or is there a
> >> reasonable solution here.
> >
> >As Dan Malek said, there is no defined ordering.  The compiler is free
> >to do whatever it wishes.  You can't assume that.  (I think.)
> [...]
>
> It's absolutely true.
>
> Not only is there no defined ordering, the ANSI standard says that it
> doesn't even need to use bitfield operations in the machine code.  It's
> allowed to pad arbitrarily between the bitfields.  It's allowed to
> completely ignore the bitfield widths and treat the members like integers.
> It's even allowed to order the very bits themselves in weird ways, if it
> thinks the processor will like it better that way.
>
> Bitfields are just not even remotely portable.
>
> C structures should not be used to represent portable serializations of
> external data.  Period.  Yes, they're convenient-- but your compiler and my
> compiler don't have to agree on practically anything about them.

And we all use gcc to compile Linux kernels... So the only remaining issue is
endianness.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- Linux/{m68k~Amiga,PPC~CHRP} -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[D.J. Barrow]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 2607 bytes --]


If you are using gcc using __attribute__(packed)
may help otherwise the compiler may optimise alignment
info gcc for more info.
struct blah
{
   uchar a : 3;
   uchar b : 3;
   uchar c : 2;
} __attribute__(packed);

to pack the array it also would be more correct to
use unsigned as opposed to uchar.

struct blah
{
   unsigned a : 3;
   unsigned b : 3;
   unsigned c : 2;
} __attribute__(packed);

I personally would be expecting to use ((*((u8
*)&blah[0]))&0x0e0)>>5 to get the info from blah.a on
both little & big endian machines.



--- Daniel Jacobowitz <drow@false.org> wrote:
>
> On Fri, May 05, 2000 at 03:27:21PM -0400,
> jlquinn@us.ibm.com wrote:
> >
> > Hi, all.  I'm working on getting the Advansys
> driver working and what
> > appears to be the final stumbling block is a
> difference in the layout of
> > bitfields between little-endian and big-endian
> systems.  In particular,
> > there are structs like the following:
> >
> > struct blah {
> >   uchar a : 3;
> >   uchar b : 3;
> >   uchar c : 2;
> > }
> >
> > On a little endian machine, this works like I
> expect.  (blah & 0x07) ==
> > blah.a is true.  On linuxppc (and apparently MacOS
> as well), this is NOT
> > true.  blah.a refers to the 3 most significant
> bits.  The linuxppc behavior
> > seems counterintuitive to me because I expected
> each successive member in a
> > struct to be the next physical piece of data I
> encounter.
> >
> > Can someone tell me what's going on here?  Am I
> going to have to have
> > ifdef's on these structs, or resort to mask
> macros, or is there a
> > reasonable solution here.
>
> As Dan Malek said, there is no defined ordering.
> The compiler is free
> to do whatever it wishes.  You can't assume that.
> (I think.)
>
> Also, by your intuition, the obvious thing IS
> happening.  Remember that
> we are big-endian - the first thing in memory is
> big-endian.  Thus, it
> follows logically that a should be in the first
> three (most
> significant) bits.
>
> Dan
>
> /--------------------------------\
> /--------------------------------\
> |       Daniel Jacobowitz        |__|        SCS
> Class of 2002       |
> |   Debian GNU/Linux Developer    __    Carnegie
> Mellon University   |
> |         dan@debian.org         |  |
> dmj+@andrew.cmu.edu      |
> \--------------------------------/
> \--------------------------------/
>
>

=====
D.J. Barrow Linux for S/390 kernel developer
eMail: djbarrow@de.ibm.com,barrow_dj@yahoo.com
Phone: +49-(0)7031-16-2583
IBM Germany Lab, Schönaicherstr. 220, 71032 Böblingen


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[Geert Uytterhoeven]

On Mon, 8 May 2000, D.J. Barrow wrote:
> struct blah
> {
>    unsigned a : 3;
>    unsigned b : 3;
>    unsigned c : 2;
> } __attribute__(packed);
>
> I personally would be expecting to use ((*((u8
> *)&blah[0]))&0x0e0)>>5 to get the info from blah.a on
> both little & big endian machines.

Which no longer works if a is longer than 8 bits and/or spans multiple bytes.
For PCI accesses, you can work around this by using {read,write}l(), which do
byte swapping on big-endian platforms as well.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- Linux/{m68k~Amiga,PPC~CHRP} -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Help with cross-endian bitfields?

[jlquinn]

In this particular instance, the data is read from the board a short at a
time (which I don't think I've got control over), but being placed into
data that we see as chars.  So I've got to undo the swap.  Then, the data
that has been returned is encoded within several bits inside a char.  In
this case, mask and shift will correctly access the data.  In general, the
problem is pretty disgusting, it seems.  At least I know how the data is
packed to start with.  Without that, there doesn't seem to be much hope for
portable solutions.

It's a shame, because otherwise, bitfields are an elegant way to deal with
packed data.

Jerry Quinn


Geert Uytterhoeven <geert@linux-m68k.org>@lists.linuxppc.org on 05/08/2000
10:19:59 AM
On Mon, 8 May 2000, D.J. Barrow wrote:
> struct blah
> {
>    unsigned a : 3;
>    unsigned b : 3;
>    unsigned c : 2;
> } __attribute__(packed);
>
> I personally would be expecting to use ((*((u8
> *)&blah[0]))&0x0e0)>>5 to get the info from blah.a on
> both little & big endian machines.

Which no longer works if a is longer than 8 bits and/or spans multiple
bytes.
For PCI accesses, you can work around this by using {read,write}l(), which
do
byte swapping on big-endian platforms as well.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/