[RFC] skge csum problems

[RFC] skge csum problems

[Al Viro]

	Both variants of skge (drivers/net and drivers/net/sk98lin/ resp.)
have the same problem with rx checksums.  They pick checksum from rx
descriptor and use it as-is.  Normally that would be the right thing to
do.  However, skge is told to byteswap descriptors on big-endian boxen.

Checksum is fixed-endian and we want it that way; IOW, what we end up
storing in skb->csum should be fixed-endian as well.  Unless the card
is smart enough to byteswap everything in rx descriptor _except_ the
checksum, we have a trouble - we get a value converted to host-endian
by the general byteswap in descriptor and we must convert it to fixed-endian
ourselves.

FWIW, FreeBSD sk_if sidesteps that mess by not telling the card to 
byteswap, so that's not too informative.  Datasheet on
http://people.freebsd.org/~wpaul/SysKonnect/xmacii_datasheet_rev_c_9-29.pdf
is not clear on what's going on with checksum in byteswapping mode either...

Could somebody with that sucker on a card (all instances I have here are
on-board ones in little-endian boxen) test what's really going on for
big-endian hosts with either driver?

Re: [RFC] skge csum problems

[Andi Kleen]

Al Viro <viro@ftp.linux.org.uk> writes:
>
> Checksum is fixed-endian and we want it that way; IOW, what we end up
> storing in skb->csum should be fixed-endian as well.

AFAIK skb->csum is always native endian because it normally
needs to be manipulated further even for RX.

-Andi

Re: [RFC] skge csum problems

[Al Viro]

On Mon, Dec 24, 2007 at 02:15:40PM +0100, Andi Kleen wrote:
> Al Viro <viro@ftp.linux.org.uk> writes:
> >
> > Checksum is fixed-endian and we want it that way; IOW, what we end up
> > storing in skb->csum should be fixed-endian as well.
> 
> AFAIK skb->csum is always native endian because it normally
> needs to be manipulated further even for RX.

No.  It needs to be manipulated, but that's exactly why it can't be
(and isn't) kept host-endian.  Large part of the reason why checksums are
done the way they are done (operations mod 0xffff, etc.) is that
they can be implemented via native arithmetics without any conversions;
e.g. if you do

add(u8 a[2], u8 b[2], u8 sum[2])
{
	u32 x = *(u16 *)a + *(u16 *)b;
	if (x > 0xffff)
		x -= 0xffff;
	*(u16 *)sum = x;
}

you will get the same behaviour on big- and little-endian boxen, even though
the intermediate integer values will be of course different.

skb->csum *must* be stored in the same order on l-e and b-e boxen; that
way you don't need to convert it or raw data when updating the sucker [*].

[*] it's slightly more complicated since skb->csum is 4-byte, not 2-byte
and the real invariant is "checksum of 4-octet array at &skb->csum must
not depend on host" (so e.g XX YY 00 00 and 00 00 XX YY are equivalent -
checksum doesn't change from reordering octet pairs; XX YY 00 00 and
00 00 YY XX are very definitely *NOT* equivalent; odd and even bytes
can't be exchanged).

Re: [RFC] skge csum problems

[Stephen Hemminger]

On Mon, 24 Dec 2007 18:39:50 +0000
Al Viro <viro@ZenIV.linux.org.uk> wrote:

> On Mon, Dec 24, 2007 at 02:15:40PM +0100, Andi Kleen wrote:
> > Al Viro <viro@ftp.linux.org.uk> writes:
> > >
> > > Checksum is fixed-endian and we want it that way; IOW, what we end up
> > > storing in skb->csum should be fixed-endian as well.
> > 
> > AFAIK skb->csum is always native endian because it normally
> > needs to be manipulated further even for RX.
> 
> No.  It needs to be manipulated, but that's exactly why it can't be
> (and isn't) kept host-endian.  Large part of the reason why checksums are
> done the way they are done (operations mod 0xffff, etc.) is that
> they can be implemented via native arithmetics without any conversions;
> e.g. if you do
> 
> add(u8 a[2], u8 b[2], u8 sum[2])
> {
> 	u32 x = *(u16 *)a + *(u16 *)b;
> 	if (x > 0xffff)
> 		x -= 0xffff;
> 	*(u16 *)sum = x;
> }
> 
> you will get the same behaviour on big- and little-endian boxen, even though
> the intermediate integer values will be of course different.
> 
> skb->csum *must* be stored in the same order on l-e and b-e boxen; that
> way you don't need to convert it or raw data when updating the sucker [*].
> 
> [*] it's slightly more complicated since skb->csum is 4-byte, not 2-byte
> and the real invariant is "checksum of 4-octet array at &skb->csum must
> not depend on host" (so e.g XX YY 00 00 and 00 00 XX YY are equivalent -
> checksum doesn't change from reordering octet pairs; XX YY 00 00 and
> 00 00 YY XX are very definitely *NOT* equivalent; odd and even bytes
> can't be exchanged).

Did you test this on real hardware?

Re: [RFC] skge csum problems

[Al Viro]

On Mon, Dec 24, 2007 at 11:36:38AM -0800, Stephen Hemminger wrote:
> > you will get the same behaviour on big- and little-endian boxen, even though
> > the intermediate integer values will be of course different.
> > 
> > skb->csum *must* be stored in the same order on l-e and b-e boxen; that
> > way you don't need to convert it or raw data when updating the sucker [*].
> > 
> > [*] it's slightly more complicated since skb->csum is 4-byte, not 2-byte
> > and the real invariant is "checksum of 4-octet array at &skb->csum must
> > not depend on host" (so e.g XX YY 00 00 and 00 00 XX YY are equivalent -
> > checksum doesn't change from reordering octet pairs; XX YY 00 00 and
> > 00 00 YY XX are very definitely *NOT* equivalent; odd and even bytes
> > can't be exchanged).
> 
> Did you test this on real hardware?

Test _what_ on real hardware?  That kernel expects skb->csum fixed-endian?
That csum_add() and friends work?  Yes to both.

If you are asking whether I'd tested what skge does to csum in its rx
descriptors when asked to byteswap - as I've said, all skge-handled stuff
I have is on-board in little-endian boxen.  Thus asking for folks who
could test it on big-endian and see what does that sucker actually do...

Re: [RFC] skge csum problems

[Stephen Hemminger]

On Mon, 24 Dec 2007 19:45:23 +0000
Al Viro <viro@ZenIV.linux.org.uk> wrote:

> On Mon, Dec 24, 2007 at 11:36:38AM -0800, Stephen Hemminger wrote:
> > > you will get the same behaviour on big- and little-endian boxen, even though
> > > the intermediate integer values will be of course different.
> > > 
> > > skb->csum *must* be stored in the same order on l-e and b-e boxen; that
> > > way you don't need to convert it or raw data when updating the sucker [*].
> > > 
> > > [*] it's slightly more complicated since skb->csum is 4-byte, not 2-byte
> > > and the real invariant is "checksum of 4-octet array at &skb->csum must
> > > not depend on host" (so e.g XX YY 00 00 and 00 00 XX YY are equivalent -
> > > checksum doesn't change from reordering octet pairs; XX YY 00 00 and
> > > 00 00 YY XX are very definitely *NOT* equivalent; odd and even bytes
> > > can't be exchanged).
> > 
> > Did you test this on real hardware?
> 
> Test _what_ on real hardware?  That kernel expects skb->csum fixed-endian?
> That csum_add() and friends work?  Yes to both.
> 
> If you are asking whether I'd tested what skge does to csum in its rx
> descriptors when asked to byteswap - as I've said, all skge-handled stuff
> I have is on-board in little-endian boxen.  Thus asking for folks who
> could test it on big-endian and see what does that sucker actually do...

I'll wait for 2.6.25 then. I am just concerned about how real hardware
interprets the byte swap PCI flag. I'll see if I can find some cost
competitive bigendian hardware.

-- 
Stephen Hemminger <stephen.hemminger@vyatta.com>