netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Performance problem with bond interface
@ 2014-04-21 14:23 Venkat Venkatsubra
  2014-04-21 15:47 ` Eric Dumazet
  0 siblings, 1 reply; 4+ messages in thread
From: Venkat Venkatsubra @ 2014-04-21 14:23 UTC (permalink / raw)
  To: netdev
  Cc: davem, linux-kernel, Rama Nichanamatlu, Sergey Linetskiy,
	Vadim Makhervaks, Guangyu Sun

We see a performance problem when the slaves of the bond 
don't support checksum offload features. What we see is
tcp_sendmsg's skb_add_data_nocache ending up not using the
csum_and_copy_from_user which would have computed the
checksum while copying from user buffer to kernel buffer.
Instead it computes later in dev_hard_start_xmit when it
figures out the slave doesn't support checksum offload and
ends up expensive . 

The bonding interface's "features" has  NETIF_F_HW_CSUM 
(or NETIF_F_NO_CSUM in 2.6.39) set which makes the 
stack think checksum need not be computed in software.
/*
* Check whether we can use HW checksum.
*/
if (sk->sk_route_caps & NETIF_F_ALL_CSUM)
     skb->ip_summed = CHECKSUM_PARTIAL;

But later in dev_hard_start_xmit it finds out the slave does not
support checksumming and decides to compute in software.

/* If packet is not checksummed and device does not
* support checksumming for this protocol, complete
* checksumming here.
*/
if (skb->ip_summed == CHECKSUM_PARTIAL) {
         skb_set_transport_header(skb,
              skb_checksum_start_offset(skb));
        if (!(features & NETIF_F_ALL_CSUM) &&
             skb_checksum_help(skb))
               goto out_kfree_skb;
}

We see this problem after this commit:
commit 1742f183fc218798dab6fcf0ded25b6608fc0a48
Author: MichaÅ<82> MirosÅ<82>aw <mirq-linux@rere.qmqm.pl>
Date:   Fri Apr 22 06:31:16 2011 +0000

    net: fix netdev_increment_features()

    Simplify and fix netdev_increment_features() to conform to what is
    stated in netdevice.h comments about NETIF_F_ONE_FOR_ALL.
    Include FCoE segmentation and VLAN-challedged flags in computation.

    Signed-off-by: MichaÅ<82> MirosÅ<82>aw <mirq-linux@rere.qmqm.pl>
    Signed-off-by: David S. Miller <davem@davemloft.net>

Prior to that the below code in netdev_increment_features was helping in
turning off NETIF_F_NO_CSUM on bond when the slaves don't support it: 
/* If device needs checksumming, downgrade to it. */
if (all & NETIF_F_NO_CSUM && !(one & NETIF_F_NO_CSUM))
        all ^= NETIF_F_NO_CSUM | (one & NETIF_F_ALL_CSUM);

The slaves are Mellanox IB adapters. This is on x86_64 platform.

Please let us know if you need any additional information.

Thanks.

Venkat

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-04-22 17:04 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-04-21 14:23 Performance problem with bond interface Venkat Venkatsubra
2014-04-21 15:47 ` Eric Dumazet
2014-04-21 18:32   ` Venkat Venkatsubra
2014-04-22 17:04     ` Venkat Venkatsubra

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).