Re: [PATCH v5] rfs: Receive Flow Steering

[Andi Kleen <andi@halobates.de>]

Tom Herbert <therbert@google.com> writes:
> +
> +		/*
> +		 * If the desired CPU (where last recvmsg was done) is
> +		 * different from current CPU (one in the rx-queue flow
> +		 * table entry), switch if one of the following holds:
> +		 *   - Current CPU is unset (equal to RPS_NO_CPU).
> +		 *   - Current CPU is offline.
> +		 *   - The current CPU's queue tail has advanced beyond the
> +		 *     last packet that was enqueued using this table entry.
> +		 *     This guarantees that all previous packets for the flow
> +		 *     have been dequeued, thus preserving in order delivery.
> +		 */
> +		if (unlikely(tcpu != next_cpu) &&
> +		    (tcpu == RPS_NO_CPU || !cpu_online(tcpu) ||
> +		     ((int)(per_cpu(softnet_data, tcpu).input_queue_head -

One thing I've been wondering while reading if this should be made
socket or SMT aware.

If you're on a hyperthreaded system and sending a IPI
to your core sibling, which has a completely shared cache hierarchy,
might not be the best use of cycles.

The same could potentially true for shared L2 or shared L3 cache
(e.g. only redirect flows between different sockets)

Have you ever considered that?

This is of course something that could be addressed post-merge, not
a blocker.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.