[PATCH net-next v3 0/2] veth and GSO maximums

[PATCH net-next v3 0/2] veth and GSO maximums

[Stephen Hemminger]

This is the more general way to solving the issue of GSO limits
not being set correctly for containers on Azure. If a GSO packet
is sent to host that exceeds the limit (reported by NDIS), then
the host is forced to do segmentation in software which has noticeable
performance impact.

The core rtnetlink infrastructure already has the messages and
infrastructure to allow changing gso limits. With an updated iproute2
the following already works:
  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Stephen Hemminger (2):
  rtnetlink: allow GSO maximums to be set on device creation
  veth: set peer GSO values

 drivers/net/veth.c   |  3 +++
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

-- 
2.11.0

[PATCH net-next 1/2] rtnetlink: allow GSO maximums to be set on device creation

[Stephen Hemminger]

From: Stephen Hemminger <stephen@networkplumber.org>

Netlink device already allows changing GSO sizes with
ip set command. The part that is missing is allowing overriding
GSO settings on device creation.

Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>

allow set gso on create
---
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index a4faefd65006..412ebf0b09c6 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -1637,6 +1637,8 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
 	[IFLA_PROMISCUITY]	= { .type = NLA_U32 },
 	[IFLA_NUM_TX_QUEUES]	= { .type = NLA_U32 },
 	[IFLA_NUM_RX_QUEUES]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SEGS]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SIZE]	= { .type = NLA_U32 },
 	[IFLA_PHYS_PORT_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },
 	[IFLA_CARRIER_CHANGES]	= { .type = NLA_U32 },  /* ignored */
 	[IFLA_PHYS_SWITCH_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },
@@ -2287,6 +2289,34 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	if (tb[IFLA_GSO_MAX_SIZE]) {
+		u32 max_size = nla_get_u32(tb[IFLA_GSO_MAX_SIZE]);
+
+		if (max_size > GSO_MAX_SIZE) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_size ^ max_size) {
+			netif_set_gso_max_size(dev, max_size);
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
+	if (tb[IFLA_GSO_MAX_SEGS]) {
+		u32 max_segs = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
+
+		if (max_segs > GSO_MAX_SEGS) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_segs ^ max_segs) {
+			dev->gso_max_segs = max_segs;
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
 	if (tb[IFLA_OPERSTATE])
 		set_operstate(dev, nla_get_u8(tb[IFLA_OPERSTATE]));
 
@@ -2651,6 +2681,10 @@ struct net_device *rtnl_create_link(struct net *net,
 		dev->link_mode = nla_get_u8(tb[IFLA_LINKMODE]);
 	if (tb[IFLA_GROUP])
 		dev_set_group(dev, nla_get_u32(tb[IFLA_GROUP]));
+	if (tb[IFLA_GSO_MAX_SIZE])
+		netif_set_gso_max_size(dev, nla_get_u32(tb[IFLA_GSO_MAX_SIZE]));
+	if (tb[IFLA_GSO_MAX_SEGS])
+		dev->gso_max_size = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
 
 	return dev;
 }
-- 
2.11.0

[PATCH net-next 2/2] veth: set peer GSO values

[Stephen Hemminger]

When new veth is created, and GSO values have been configured
on one device, clone those values to the peer.

For example:
   # ip link add dev vm1 gso_max_size 65530 type veth peer name vm2

This should create vm1 <--> vm2 with both having GSO maximum
size set to 65530.

Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
---
 drivers/net/veth.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index f5438d0978ca..a69ad39ee57e 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;
-- 
2.11.0

Re: [PATCH net-next 2/2] veth: set peer GSO values

[David Ahern]

On 12/7/17 4:40 PM, Stephen Hemminger wrote:
> diff --git a/drivers/net/veth.c b/drivers/net/veth.c
> index f5438d0978ca..a69ad39ee57e 100644
> --- a/drivers/net/veth.c
> +++ b/drivers/net/veth.c
> @@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
>  	if (ifmp && (dev->ifindex != 0))
>  		peer->ifindex = ifmp->ifi_index;
>  
> +	peer->gso_max_size = dev->gso_max_size;
> +	peer->gso_max_segs = dev->gso_max_segs;
> +
>  	err = register_netdevice(peer);
>  	put_net(net);
>  	net = NULL;
> 

What if gso changes are made after device create? They are not
propagated to the peer device like they are on link create.

Re: [PATCH net-next 2/2] veth: set peer GSO values

[Solio Sarabia]

On Sat, Dec 09, 2017 at 09:41:25AM -0700, David Ahern wrote:
> On 12/7/17 4:40 PM, Stephen Hemminger wrote:
> > diff --git a/drivers/net/veth.c b/drivers/net/veth.c
> > index f5438d0978ca..a69ad39ee57e 100644
> > --- a/drivers/net/veth.c
> > +++ b/drivers/net/veth.c
> > @@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
> >  	if (ifmp && (dev->ifindex != 0))
> >  		peer->ifindex = ifmp->ifi_index;
> >  
> > +	peer->gso_max_size = dev->gso_max_size;
> > +	peer->gso_max_segs = dev->gso_max_segs;
> > +
> >  	err = register_netdevice(peer);
> >  	put_net(net);
> >  	net = NULL;
> > 
> 
> What if gso changes are made after device create? They are not
> propagated to the peer device like they are on link create.

This would be a nice addition after ongoing patches are merged,
since veth usually lives in another netns. For docker, it requires
a couple of extra commands to expose the peer's netns.

Re: [PATCH net-next v3 0/2] veth and GSO maximums

[David Miller]

From: Stephen Hemminger <stephen@networkplumber.org>
Date: Thu,  7 Dec 2017 15:40:18 -0800

> This is the more general way to solving the issue of GSO limits
> not being set correctly for containers on Azure. If a GSO packet
> is sent to host that exceeds the limit (reported by NDIS), then
> the host is forced to do segmentation in software which has noticeable
> performance impact.
> 
> The core rtnetlink infrastructure already has the messages and
> infrastructure to allow changing gso limits. With an updated iproute2
> the following already works:
>   # ip li set dev dummy0 gso_max_size 30000
> 
> These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

Re: [PATCH net-next v3 0/2] veth and GSO maximums

[Solio Sarabia]

On Fri, Dec 08, 2017 at 02:23:23PM -0500, David Miller wrote:
> From: Stephen Hemminger <stephen@networkplumber.org>
> Date: Thu,  7 Dec 2017 15:40:18 -0800
> 
> > This is the more general way to solving the issue of GSO limits
> > not being set correctly for containers on Azure. If a GSO packet
> > is sent to host that exceeds the limit (reported by NDIS), then
> > the host is forced to do segmentation in software which has noticeable
> > performance impact.
> > 
> > The core rtnetlink infrastructure already has the messages and
> > infrastructure to allow changing gso limits. With an updated iproute2
> > the following already works:
> >   # ip li set dev dummy0 gso_max_size 30000
> > 
> > These patches are about making it easier with veth.
> 
> Ok, this is definitely a step forward.
> 
> Series applied, thanks Stephen.

Thanks.
Still not seeing the iproute2 patch though, either master or net-next.

Re: [PATCH net-next v3 0/2] veth and GSO maximums

[Stephen Hemminger]

On Fri, 8 Dec 2017 15:50:25 -0800
Solio Sarabia <solio.sarabia@intel.com> wrote:

> On Fri, Dec 08, 2017 at 02:23:23PM -0500, David Miller wrote:
> > From: Stephen Hemminger <stephen@networkplumber.org>
> > Date: Thu,  7 Dec 2017 15:40:18 -0800
> >   
> > > This is the more general way to solving the issue of GSO limits
> > > not being set correctly for containers on Azure. If a GSO packet
> > > is sent to host that exceeds the limit (reported by NDIS), then
> > > the host is forced to do segmentation in software which has noticeable
> > > performance impact.
> > > 
> > > The core rtnetlink infrastructure already has the messages and
> > > infrastructure to allow changing gso limits. With an updated iproute2
> > > the following already works:
> > >   # ip li set dev dummy0 gso_max_size 30000
> > > 
> > > These patches are about making it easier with veth.  
> > 
> > Ok, this is definitely a step forward.
> > 
> > Series applied, thanks Stephen.  
> 
> Thanks.
> Still not seeing the iproute2 patch though, either master or net-next.

I held off on iproute2 until kernel change is accepted upstream.