Re: [PATCH net-next 1/2] net/smc: make wr buffer count configurable

[Dust Li <dust.li@linux.alibaba.com>]

On 2025-09-05 14:01:35, Halil Pasic wrote:
>On Fri, 5 Sep 2025 11:00:59 +0200
>Halil Pasic <pasic@linux.ibm.com> wrote:
>
>> > 1. What if the two sides have different max_send_wr/max_recv_wr configurations?
>> > IIUC, For example, if the client sets max_send_wr to 64, but the server sets
>> > max_recv_wr to 16, the client might overflow the server's QP receive
>> > queue, potentially causing an RNR (Receiver Not Ready) error.  
>>
>> I don't think the 16 is spec-ed anywhere and if the client and the server
>> need to agree on the same value it should either be speced, or a
>> protocol mechanism for negotiating it needs to exist. So what is your
>> take on this as an SMC maintainer?

Right — I didn't realize that either until I saw this patch today :)
But since the implementation's been set to 16 since day one, bumping it
up might break things.

>>
>> I think, we have tested heterogeneous setups and didn't see any grave
>> issues. But let me please do a follow up on this. Maybe the other
>> maintainers can chime in as well.

I'm glad to hear from others.

>
>Did some research and some thinking. Are you concerned about a
>performance regression for e.g. 64 -> 16 compared to 16 -> 16? According
>to my current understanding the RNR must not lead to a catastrophic
>failure, but the RDMA/IB stack is supposed to handle that.

No, it's not just a performance regression.
If we get an RNR when going from 64 -> 16, the whole link group gets
torn down — and all SMC connections inside it break.
So from the user’s point of view, connections will just randomly drop
out of nowhere.

>
>I would like to also point out that bumping SMC_WR_BUF_CNT basically has
>the same problem, although admittedly to a smaller extent because it is
>only between "old" and "new".

Right.

>
>Assuming that my understanding is correct, I believe that the problem of
>the potential RNR is inherent to the objective of the series, and
>probably one that can be lived with. Given this entire EID business, I
>think the SMC-R setup is likely to happen in a coordinated fashion for
>all potential peers, and I hope whoever tweaks those values has a
>sufficent understanding or empiric evidence to justify the tweaks.
>
>Assuming my understanding is not utterly wrong, I would very much like
>to know what would you want me to do with this?

In my opinion, the best way to support different SMC_WR_BUF_CNT values
is: First, define it in the spec - then do a handshake to agree on the
value.
If the peer doesn’t support this feature, just fall back to 16.
If both sides support it, use the smaller (MIN) of the two values.

Best regards,
Dust