Re: Proposal using RCU for callbacks

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 21 Oct 2026 12:51:11 +0100
Konstantin Ananyev <konstantin.v.ananyev@yandex.ru> wrote:

> In particular, regarding membarrier() - consider using it for BPF CBs
> instead of smb_mb, but decided not to. The main reasons:
> - with membarrier() - all threads in the process will be affected.
>    Also, AFAIK it is implemented via IPI - which is quite heavy.
> - current smp_mb (at least on modern x86_64) is quite fast
> (it uses lock-ed instruction on thread's stack, and last time I measured it,
> takes ¬30-50 cycles). As we usually doing it for a bulk (32+) of packets,
> the overhead is quite small: 1-1.5 cycle per packet.
> 
> But again, if you have some smart ideas how to improve our RCU lib with 
> membarrier() -
> would be really interested to hear.

There is a version in current wireshark extcap patch series.
The membarrier is expensive but only happens when capture is stopped.
We can afford a system call there.

The read side (in callback) is:

static inline void
capture_rcu_enter(unsigned int slot)
{
	struct capture_rcu_reader *r = &capture_rcu.reader[slot];
	uint64_t t;

	if (r->depth++ != 0)
		return;

	t = rte_atomic_load_explicit(&capture_rcu.token, rte_memory_order_relaxed);
	rte_atomic_store_explicit(&r->ctr, t, rte_memory_order_relaxed);

	/*
	 * The loads that follow must not move above the store of ctr. The
	 * writer forces that on every CPU with membarrier, so no fence is
	 * needed here; capture is refused if membarrier does not work.
	 */
	rte_compiler_barrier();
}

static inline void
capture_rcu_exit(unsigned int slot)
{
	struct capture_rcu_reader *r = &capture_rcu.reader[slot];

	if (--r->depth != 0)
		return;

	/* release: prior loads complete before the slot reads idle */
	rte_atomic_store_explicit(&r->ctr, 0, rte_memory_order_release);
}