From mboxrd@z Thu Jan 1 00:00:00 1970 From: Balraj Singh Subject: Re: Question about TCP checksum offload in Xen Date: Thu, 5 Dec 2013 13:43:07 +0000 Message-ID: References: <20131205112952.GF14792@dark.recoil.org> <1386243546.20047.26.camel@kazak.uk.xensource.com> <52A073A1.4000102@oracle.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7326973945865001368==" Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VoZCv-0000Eb-OK for xen-devel@lists.xenproject.org; Thu, 05 Dec 2013 13:43:10 +0000 Received: by mail-we0-f172.google.com with SMTP id w62so11009948wes.31 for ; Thu, 05 Dec 2013 05:43:07 -0800 (PST) In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jon Crowcroft Cc: John Haxby , "xen-devel@lists.xenproject.org" , Ian Campbell , Mirage List , Anil Madhavapeddy List-Id: xen-devel@lists.xenproject.org --===============7326973945865001368== Content-Type: multipart/alternative; boundary=001a11c35300a84df404ecc9b585 --001a11c35300a84df404ecc9b585 Content-Type: text/plain; charset=ISO-8859-1 That could work too to signal in the packet itself that the checksum is ok. But I don't think the received packets had 0 in the chksum field. It was just some random value, though I can check again. Thanks, Balraj On Thu, Dec 5, 2013 at 1:15 PM, Jon Crowcroft wrote: > i thought a value of 0 in the tcp chcksum field indicated "no checksum" > and could be used in the cases you identify and ought not to trigger > problems in correct code? > > > On Thu, Dec 5, 2013 at 12:37 PM, John Haxby wrote: > >> On 05/12/13 11:39, Ian Campbell wrote: >> > On Thu, 2013-12-05 at 11:29 +0000, Anil Madhavapeddy wrote: >> >> > On Tue, Dec 03, 2013 at 01:00:23PM +0000, Balraj Singh wrote: >> >>> > > Hi, >> >>> > > >> >>> > > I'm working on verifying TCP checksums on incoming packets in >> Mirage, but >> >>> > > I've run into a bit of a problem. >> >>> > > >> >>> > > If TCP checksum offload is turned on on a virtual interface (this >> is the >> >>> > > default), and if the TCP connection is local to the machine, it >> looks like >> >>> > > Xen does not calculate the checksum at all. This may be valid >> because Xen >> >>> > > may be providing a stronger guarantee, but it means that incoming >> packets >> >>> > > don't have a valid checksum in the header. This then means that >> in Mirage >> >>> > > we can't just have checksum verification turned on all the time. >> This >> >>> > > would have been the safe fall back option and detecting that >> checksum >> >>> > > offload is on, and then not duplicating the verification in >> Mirage would >> >>> > > have been an optimisation. But it looks like this is not an >> option. Now I >> >>> > > need to know for every incoming packet whether checksum >> verification should >> >>> > > be done or not. It should ideally be for every packet since >> chksum offload >> >>> > > can be turned off and on on the VIF and existing tcp connections >> should >> >>> > > continue. If not every packet, I need to get a notification or >> efficiently >> >>> > > detect right away that the setting is changed on the VIF. >> >> > >> >> > This is a question that seems to keep coming up even for Linux and >> >> > Windows, as the combination of local<->local VMs vs local<->off-host >> and >> >> > the checksum offload is quite confusing. >> >> > >> >> > CCing xen-devel: is the appropriate behaviour for a guest VM that >> wants to >> >> > use checksum offloading in all situations documented anywhere? >> > I don't understand the question/concern. If you have enabled checksum >> > offload then of course you don't recalculate the checksum, that's the >> > whole point of offloading it. >> >> I get this a lot. >> >> There are a few different cases: >> >> * domain to domain traffic >> * domain to external traffic with egress from a NIC that does offload >> * domain to external through a non-offloading NIC >> >> With xen checksum offloading, domain to domain traffic appears to be >> received with a bad checksum. This is OK, there is no point in >> calculating a checksum if the packets are only going through memory. If >> your memory is going to randomly corrupt packets you have more bigger >> problems to worry about. However, this does upset at least Solaris: if >> you're using a Solaris guest for NAT then the NAT module on Solaris gets >> all upset if the checksum is wrong and drops the packets. (This is >> Solaris's NAT module being overly picky, it may need to recalculate or >> at least invalidate the existing checksum, but it doesn't need to check >> it as well.) >> >> The second two cases are of interest from the domain perspective. A >> domain has no way of knowing how any given packet is going to leave the >> host (or even if it is) so it can't know ahead of time whether to >> calculate any checksums: the skb's are just marked with "checksum >> needed" as usual and either the egress NIC will do the job or dom0 will >> do it. >> >> There is absolutely nothing wrong in any of this (Solaris >> notwithstanding). The difficulty is getting people to realise that >> checksums are only calculated when a packet hits the cat-5. It doesn't >> need documenting, it just needs a little thought. I got tired of >> hammering the point home :) >> >> jch >> >> > --001a11c35300a84df404ecc9b585 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
That could work too to signal in the packet itself that th= e checksum is ok. =A0But I don't think the received packets had 0 in th= e chksum field. =A0It was just some random value, though I can check again.=

Thanks,

Balraj


On Thu, Dec 5, 20= 13 at 1:15 PM, Jon Crowcroft <jon.crowcroft@cl.cam.ac.uk><= /span> wrote:
i thought a value of 0 in t= he tcp chcksum field indicated "no checksum" and could be used in= the cases you identify and ought not to trigger problems in correct code?<= /div>


On Thu, Dec 5, 2013 at= 12:37 PM, John Haxby <john.haxby@oracle.com> wrote:
=
On 05/12/13 11:39, Ian Campbell wrote:
> On Thu, 2013-12-05 at 11:29 +0000, Anil Madhavapeddy wrote:
>> > On Tue, Dec 03, 2013 at 01:00:23PM +0000, Balraj Singh wrote:=
>>> > > Hi,
>>> > >
>>> > > I'm working on verifying TCP checksums on incomi= ng packets in Mirage, but
>>> > > I've run into a bit of a problem.
>>> > >
>>> > > If TCP checksum offload is turned on on a virtual in= terface (this is the
>>> > > default), and if the TCP connection is local to the = machine, it looks like
>>> > > Xen does not calculate the checksum at all. =A0This = may be valid because Xen
>>> > > may be providing a stronger guarantee, but it means = that incoming packets
>>> > > don't have a valid checksum in the header. =A0Th= is then means that in Mirage
>>> > > we can't just have checksum verification turned = on all the time. =A0This
>>> > > would have been the safe fall back option and detect= ing that checksum
>>> > > offload is on, and then not duplicating the verifica= tion in Mirage would
>>> > > have been an optimisation. =A0But it looks like this= is not an option. =A0Now I
>>> > > need to know for every incoming packet whether check= sum verification should
>>> > > be done or not. =A0It should ideally be for every pa= cket since chksum offload
>>> > > can be turned off and on on the VIF and existing tcp= connections should
>>> > > continue. =A0If not every packet, I need to get a no= tification or efficiently
>>> > > detect right away that the setting is changed on the= VIF.
>> >
>> > This is a question that seems to keep coming up even for Linu= x and
>> > Windows, as the combination of local<->local VMs vs loc= al<->off-host and
>> > the checksum offload is quite confusing.
>> >
>> > CCing xen-devel: is the appropriate behaviour for a guest VM = that wants to
>> > use checksum offloading in all situations documented anywhere= ?
> I don't understand the question/concern. If you have enabled check= sum
> offload then of course you don't recalculate the checksum, that= 9;s the
> whole point of offloading it.

I get this a lot.

There are a few different cases:

=A0 * domain to domain traffic
=A0 * domain to external traffic with egress from a NIC that does offload =A0 * domain to external through a non-offloading NIC

With xen checksum offloading, domain to domain traffic appears to be
received with a bad checksum. =A0This is OK, there is no point in
calculating a checksum if the packets are only going through memory. =A0If<= br> your memory is going to randomly corrupt packets you have more bigger
problems to worry about. =A0 However, this does upset at least Solaris: if<= br> you're using a Solaris guest for NAT then the NAT module on Solaris get= s
all upset if the checksum is wrong and drops the packets. =A0(This is
Solaris's NAT module being overly picky, it may need to recalculate or<= br> at least invalidate the existing checksum, but it doesn't need to check=
it as well.)

The second two cases are of interest from the domain perspective. =A0A
domain has no way of knowing how any given packet is going to leave the
host (or even if it is) so it can't know ahead of time whether to
calculate any checksums: the skb's are just marked with "checksum<= br> needed" as usual and either the egress NIC will do the job or dom0 wil= l
do it.

There is absolutely nothing wrong in any of this (Solaris
notwithstanding). =A0 The difficulty is getting people to realise that
checksums are only calculated when a packet hits the cat-5. =A0It doesn'= ;t
need documenting, it just needs a little thought. =A0 I got tired of
hammering the point home :)

jch



--001a11c35300a84df404ecc9b585-- --===============7326973945865001368== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============7326973945865001368==--