Re: Question about TCP checksum offload in Xen

[John Haxby <john.haxby@oracle.com>]

On 05/12/13 12:56, Balraj Singh wrote:
> Thanks very much.  This is fine and exactly what is observed (that the
> checksum is only calculated going to/from a real wire).  The engineering
> choice makes sense.  But this does mean that I need to know when the
> stack receives a pkt if the checksum should be verified for that packet.
>  I can't do a redundant verification because a bad chksum may be because
> the field was clobbered (not set) during earlier verification.  So, what
> I wanted to know was where to look for this flag - a snippet of code or
> a ptr to a file :).
> 

Look at the skb in the netfront: there's a flag in there to say whether
the checksum is valid.  Interestingly, if the checksum is valid then you
know that the data has come from outside the host (otherwise it would
have been dropped).

Why do you even care about the checksum?  Packets with bad checksums are
dropped by the NIC (with checksum offload) or by the NIC's driver
(without) -- the only packets you'll see with a bad checksum are those
that have come from another domain on the same host.

jch

> Thanks,
> 
> Balraj
> 
> 
> 
> On Thu, Dec 5, 2013 at 12:37 PM, John Haxby <john.haxby@oracle.com
> <mailto:john.haxby@oracle.com>> wrote:
> 
>     On 05/12/13 11:39, Ian Campbell wrote:
>     > On Thu, 2013-12-05 at 11:29 +0000, Anil Madhavapeddy wrote:
>     >> > On Tue, Dec 03, 2013 at 01:00:23PM +0000, Balraj Singh wrote:
>     >>> > > Hi,
>     >>> > >
>     >>> > > I'm working on verifying TCP checksums on incoming packets
>     in Mirage, but
>     >>> > > I've run into a bit of a problem.
>     >>> > >
>     >>> > > If TCP checksum offload is turned on on a virtual interface
>     (this is the
>     >>> > > default), and if the TCP connection is local to the machine,
>     it looks like
>     >>> > > Xen does not calculate the checksum at all.  This may be
>     valid because Xen
>     >>> > > may be providing a stronger guarantee, but it means that
>     incoming packets
>     >>> > > don't have a valid checksum in the header.  This then means
>     that in Mirage
>     >>> > > we can't just have checksum verification turned on all the
>     time.  This
>     >>> > > would have been the safe fall back option and detecting that
>     checksum
>     >>> > > offload is on, and then not duplicating the verification in
>     Mirage would
>     >>> > > have been an optimisation.  But it looks like this is not an
>     option.  Now I
>     >>> > > need to know for every incoming packet whether checksum
>     verification should
>     >>> > > be done or not.  It should ideally be for every packet since
>     chksum offload
>     >>> > > can be turned off and on on the VIF and existing tcp
>     connections should
>     >>> > > continue.  If not every packet, I need to get a notification
>     or efficiently
>     >>> > > detect right away that the setting is changed on the VIF.
>     >> >
>     >> > This is a question that seems to keep coming up even for Linux and
>     >> > Windows, as the combination of local<->local VMs vs
>     local<->off-host and
>     >> > the checksum offload is quite confusing.
>     >> >
>     >> > CCing xen-devel: is the appropriate behaviour for a guest VM
>     that wants to
>     >> > use checksum offloading in all situations documented anywhere?
>     > I don't understand the question/concern. If you have enabled checksum
>     > offload then of course you don't recalculate the checksum, that's the
>     > whole point of offloading it.
> 
>     I get this a lot.
> 
>     There are a few different cases:
> 
>       * domain to domain traffic
>       * domain to external traffic with egress from a NIC that does offload
>       * domain to external through a non-offloading NIC
> 
>     With xen checksum offloading, domain to domain traffic appears to be
>     received with a bad checksum.  This is OK, there is no point in
>     calculating a checksum if the packets are only going through memory.  If
>     your memory is going to randomly corrupt packets you have more bigger
>     problems to worry about.   However, this does upset at least Solaris: if
>     you're using a Solaris guest for NAT then the NAT module on Solaris gets
>     all upset if the checksum is wrong and drops the packets.  (This is
>     Solaris's NAT module being overly picky, it may need to recalculate or
>     at least invalidate the existing checksum, but it doesn't need to check
>     it as well.)
> 
>     The second two cases are of interest from the domain perspective.  A
>     domain has no way of knowing how any given packet is going to leave the
>     host (or even if it is) so it can't know ahead of time whether to
>     calculate any checksums: the skb's are just marked with "checksum
>     needed" as usual and either the egress NIC will do the job or dom0 will
>     do it.
> 
>     There is absolutely nothing wrong in any of this (Solaris
>     notwithstanding).   The difficulty is getting people to realise that
>     checksums are only calculated when a packet hits the cat-5.  It doesn't
>     need documenting, it just needs a little thought.   I got tired of
>     hammering the point home :)
> 
>     jch
> 
>