Re: [PATCH net-next v2 2/2] net/smc: reduce TX slot contention with exclusive wait

[Mahanta Jambigi <mjambigi@linux.ibm.com>]

On 21/08/26 2:47 pm, D. Wythe wrote:
> smc_wr_tx_get_free_slot() waits for a free TX slot with
> wait_event_interruptible_timeout(). Since the wait_event family
> enqueues waiters as non-exclusive, wake_up() may wake multiple
> waiters even though only one can use the slot, causing
> thundering-herd contention when slots are scarce.
> 
> Use an exclusive wait loop with prepare_to_wait_exclusive() so
> wake_up() wakes only one waiter per freed slot.
> smc_wr_wakeup_tx_wait() still uses wake_up_all() during link
> teardown, so teardown behavior is unchanged.
> 
> This also corrects the return value on a pending signal: the previous
> wait_event_interruptible_timeout() path fell through to the "no free
> slot" case and returned -EPIPE, masking the signal as a connection
> error. The open-coded loop now returns -ERESTARTSYS, matching the
> standard interruptible-wait semantics and letting the syscall restart
> machinery handle it.
> 
> Performance
> ===========
> 
> Measured with uperf between two peers over SMC-R. The benefit depends
> on how often the TX slot wait path is actually taken.
> 
> With the default settings, where many connections share a link group
> and the send queue is small, slots are scarce and the wait path is hot:
> 
>   net.smc.smcr_max_conns_per_lgr = 255
>   net.smc.smcr_max_send_wr       = 16
>   net.smc.smcr_max_recv_wr       = 48
> 
>   workload                 baseline      patched     delta
>   ---------------------------------------------------------
>   rr1c-200x1000-50.xml     655.06 Mb/s   1.53 Gb/s   +134%
>   rr1c-1x1-250.xml         371.03 Kb/s   2.07 Mb/s   +458%

Overall, I am also seeing significant performance improvements with this
patch, ranging from approximately 20% to over 150% across several
workloads. Do we expect to see further improvements once the CQ pooling
patch is applied?

> Signed-off-by: D. Wythe <alibuda@linux.alibaba.com>
> Reviewed-by: Wen Gu <guwen@linux.alibaba.com>

Reviewed-by: Mahanta Jambigi <mjambigi@linux.ibm.com>