Layer 2 acceleration vs GSO

Layer 2 acceleration vs GSO

[Ben Hutchings]

What happens when an skb to be sent through ndo_dfwd_start_xmit()
requires software GSO?

dev_hard_start_xmit() will segment it and then submit each segment, and
then:

> gso:
> 	do {
[...]
> 		if (accel_priv)
> 			rc = ops->ndo_dfwd_start_xmit(nskb, dev, accel_priv);
> 		else
> 			rc = ops->ndo_start_xmit(nskb, dev);
> 		trace_net_dev_xmit(nskb, rc, dev, skb_len);
[...]
> 		txq_trans_update(txq);

Oops, txq is NULL.  And once we add the obvious condition to that,

> 		if (unlikely(netif_xmit_stopped(txq) && skb->next))
> 			return NETDEV_TX_BUSY;

How can we tell if the hardware transmit queue filled up?

It seems like this feature currently relies on the driver returning
NETDEV_TX_BUSY when the TX queue is already full, like ixgbe does.  But
that is exactly what drivers are *not* supposed to do.

Ben.

> 	} while (skb->next);

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: Layer 2 acceleration vs GSO

[Neil Horman]

On Fri, Jan 10, 2014 at 08:05:39PM +0000, Ben Hutchings wrote:
> What happens when an skb to be sent through ndo_dfwd_start_xmit()
> requires software GSO?
> 
> dev_hard_start_xmit() will segment it and then submit each segment, and
> then:
> 
> > gso:
> > 	do {
> [...]
> > 		if (accel_priv)
> > 			rc = ops->ndo_dfwd_start_xmit(nskb, dev, accel_priv);
> > 		else
> > 			rc = ops->ndo_start_xmit(nskb, dev);
> > 		trace_net_dev_xmit(nskb, rc, dev, skb_len);
> [...]
> > 		txq_trans_update(txq);
> 
> Oops, txq is NULL.  And once we add the obvious condition to that,
> 
> > 		if (unlikely(netif_xmit_stopped(txq) && skb->next))
> > 			return NETDEV_TX_BUSY;
> 
> How can we tell if the hardware transmit queue filled up?
> 
> It seems like this feature currently relies on the driver returning
> NETDEV_TX_BUSY when the TX queue is already full, like ixgbe does.  But
> that is exactly what drivers are *not* supposed to do.
> 
> Ben.
> 
> > 	} while (skb->next);
> 
Dave just to a fix to deal with this (among some other issues) here today:
http://marc.info/?l=linux-kernel&m=138934276507518&w=2
Neil

> -- 
> Ben Hutchings, Staff Engineer, Solarflare
> Not speaking for my employer; that's the marketing department's job.
> They asked us to note that Solarflare product names are trademarked.
> 
> 

Re: Layer 2 acceleration vs GSO

[Ben Hutchings]

On Fri, 2014-01-10 at 15:32 -0500, Neil Horman wrote:
> On Fri, Jan 10, 2014 at 08:05:39PM +0000, Ben Hutchings wrote:
> > What happens when an skb to be sent through ndo_dfwd_start_xmit()
> > requires software GSO?
> > 
> > dev_hard_start_xmit() will segment it and then submit each segment, and
> > then:
> > 
> > > gso:
> > > 	do {
> > [...]
> > > 		if (accel_priv)
> > > 			rc = ops->ndo_dfwd_start_xmit(nskb, dev, accel_priv);
> > > 		else
> > > 			rc = ops->ndo_start_xmit(nskb, dev);
> > > 		trace_net_dev_xmit(nskb, rc, dev, skb_len);
> > [...]
> > > 		txq_trans_update(txq);
> > 
> > Oops, txq is NULL.  And once we add the obvious condition to that,
> > 
> > > 		if (unlikely(netif_xmit_stopped(txq) && skb->next))
> > > 			return NETDEV_TX_BUSY;
> > 
> > How can we tell if the hardware transmit queue filled up?
> > 
> > It seems like this feature currently relies on the driver returning
> > NETDEV_TX_BUSY when the TX queue is already full, like ixgbe does.  But
> > that is exactly what drivers are *not* supposed to do.
> > 
> > Ben.
> > 
> > > 	} while (skb->next);
> > 
> Dave just to a fix to deal with this (among some other issues) here today:
> http://marc.info/?l=linux-kernel&m=138934276507518&w=2
> Neil

Sorry, I didn't think to check the net tree as I this feature was in
net-next only.  Good to know it's fixed.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.