Re: [PATCH net-next 1/3] net: provide macros for commonly copied lockless queue stop/wake code

[Jakub Kicinski <kuba@kernel.org>]

On Sat, 1 Apr 2023 17:18:12 +0200 Heiner Kallweit wrote:
> > +#define __netif_tx_queue_maybe_wake(txq, get_desc, start_thrs, down_cond) \
> > +	({								\
> > +		int _res;						\
> > +									\
> > +		_res = -1;						\  
> 
> One more question: Don't we need a read memory barrier here to ensure
> get_desc is up-to-date?

CC: Alex, maybe I should not be posting after 10pm, with the missing v2
and sparse CC list.. :|

I was thinking about this too yesterday. AFAICT this implementation
could indeed result in waking even tho the queue is full on non-x86.
That's why the drivers have an extra check at the start of .xmit? :(

I *think* that the right ordering would be:

WRITE cons
mb()  # A
READ stopped
rmb() # C
READ prod, cons

And on the producer side (existing):

WRITE prod
READ prod, cons
mb()  # B
WRITE stopped
READ prod, cons

But I'm slightly afraid to change it, it's been working for over 
a decade :D

One neat thing that I noticed, which we could potentially exploit 
if we were to touch this code is that BQL already has a smp_mb() 
on the consumer side. So on any kernel config and driver which support
BQL we can use that instead of adding another barrier at #A.

It would actually be a neat optimization because right now, AFAICT,
completion will fire the # A -like barrier almost every time.

> > +		if (likely(get_desc > start_thrs))			\
> > +			_res = __netif_tx_queue_try_wake(txq, get_desc,	\
> > +							 start_thrs,	\
> > +							 down_cond);	\
> > +		_res;							\
> > +	})