Re: [PATCH 2/2] ixgbe: support skb->xmit_more in netdev_ops->ndo_start_xmit()

[Daniel Borkmann <dborkman@redhat.com>]

On 08/26/2014 05:40 PM, Hannes Frederic Sowa wrote:
> On Di, 2014-08-26 at 08:00 -0700, Alexander Duyck wrote:
>> On 08/25/2014 04:35 PM, David Miller wrote:
>>>
>>> From: Daniel Borkmann <dborkman@redhat.com>
>>>
>>> This implements the deferred tail pointer flush API for the ixgbe
>>> driver. Similar version also proposed longer time ago by Alexander Duyck.
>>>
>>> Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
>>> Signed-off-by: David S. Miller <davem@davemloft.net>
>>> ---
>>>   drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 7 ++++---
>>>   1 file changed, 4 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>>> index 87bd53f..ba9ceaa 100644
>>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>>> @@ -6958,9 +6958,10 @@ static void ixgbe_tx_map(struct ixgbe_ring *tx_ring,
>>>
>>>   	tx_ring->next_to_use = i;
>>>
>>> -	/* notify HW of packet */
>>> -	ixgbe_write_tail(tx_ring, i);
>>> -
>>> +	if (!skb->xmit_more) {
>>> +		/* notify HW of packet */
>>> +		ixgbe_write_tail(tx_ring, i);
>>> +	}
>>>   	return;
>>>   dma_error:
>>>   	dev_err(tx_ring->dev, "TX DMA map failed\n");
>>>
>>
>> It might help to add some handling for the case where xmit_more is set,
>> but the ring has become full.  This current implementation introduces
>> the risk of triggering a Tx hang.
>
> IMHO this should be done before the patch lands in the driver.

I think that xmit_more=1 should be seen as an indication/hint to the
driver so it can decide what to do with it, e.g. flush nevertheless
when it runs under pressure.

>> My advice would be to pull the ixgbe_maybe_stop_tx code at the end of
>> xmit_frame into the if check here, and perhaps look into adding an
>> additional check to see if BQL has stopped the ring as well.
>
> I would like to have the BQL check not in the driver but in the generic
> code steering xmit_more.
>
> Otherwise I'll like the API.

I agree with you that the logic when to set/unset the hint should
come from a generic core part. I think the xmit_more API seems so far
lightweight but lets see how it evolves. :) So far we don't have a
flush-only part (as before), where one could trigger that outside of
a fast-path to flush remaining skbs, but perhaps not having it is not
too bad either.

> David, because of debugging and driver bugs, can we have an interface
> flag for that, so we can e.g. switch xmit_more to 0 permanently? It
> would also be nice to have for documentation purposes, so ethtool e.g.
> can discover feature-set of a driver.
>
> Thanks,
> Hannes