From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kieran Mansley Subject: Re: LRO/GSO interaction when packets are forwarded Date: Fri, 07 Mar 2008 17:06:15 +0000 Message-ID: <1204909575.4220.71.camel@moonstone.uk.level5networks.com> References: <1204898997.4220.41.camel@moonstone.uk.level5networks.com> <20080307082538.1a674ae1@extreme> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: netdev@vger.kernel.org To: Stephen Hemminger Return-path: Received: from 216-237-3-220.orange.nextweb.net ([216.237.3.220]:42124 "EHLO exchange.solarflare.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755542AbYCGRGU (ORCPT ); Fri, 7 Mar 2008 12:06:20 -0500 In-Reply-To: <20080307082538.1a674ae1@extreme> Sender: netdev-owner@vger.kernel.org List-ID: On Fri, 2008-03-07 at 08:25 -0800, Stephen Hemminger wrote: > On Fri, 07 Mar 2008 14:09:57 +0000 > Kieran Mansley wrote: > > > We've seen a couple of problems when using a bridge or IP forwarding > > combined with LRO packets generated by a network device driver. As you > > know, LRO packets can be either be page based (and passed up with > > lro_receive_page()) or use the skb frag_list (and passed up with > > lro_receive_skb()). In both cases it is likely that the device driver > > will have set CHECKSUM_UNNECESSARY to indicate that the packet has been > > checksummed by the device, and gso_size to mark it as an LRO packet and > > indicate the actual received MSS. > > First off, no hardware should ever do LRO on non-local packets. If the > hardware isn't smart enough to do this, I guess the bridge code to have > an API to turn it off. IP should also turn it off if ip_forwarding > is enabled on that device. If the only way to deal with this is to prevent LRO in those cases, having an API to turn if off would clearly be helpful: working out in the hardware or driver which packets can be forwarded or not is hard, and would probably be a layering violation in itself. However, if there were a way in which it could be made to work, that would in many ways be preferable. > > If this skb goes directly to the network stack everything is fine. The > > problem comes when this packet instead goes into a bridge and is then > > retransmitted on another device. The skb seems to pass through the > > bridge relatively unmodified and because it has gso_size set the > > transmit path will attempt to segment it. If page-based allocation has > > been used, this is fine, but if the skb frag_list has been used the > > transmit path BUGs in skb_gso_segment(): > > You can't do LRO with bridging, it is that simple, it is a protocol > layering violation. Agreed. LRO is a layering violation with or without bridging. If the consensus is that LRO with bridging is not viable, then that's straightforward for us, but there are a lot of people who will want this for performance. Bridging and virtualisation are common partners, and virtualisation and high performance networking are also rather popular at the moment. Given that there seem to be a small number of places where this is currently causing problems (LRO through a bridge to a stack is fine, it's only if the packet is forwarded that things go wrong), and that an LRO packet should in most cases be very easy to segment (it's already in network-sized fragments) I wonder if making it work would be easier/better than preventing it from happening. Kieran