Re: [PATCH net-next v3 1/2] net/smc: make wr buffer count configurable

Paolo Abeni
2025-09-25T09:27:43Z

On 9/21/25 11:44 PM, Halil Pasic wrote:
> diff --git a/Documentation/networking/smc-sysctl.rst b/Documentation/networking/smc-sysctl.rst
> index a874d007f2db..c94d750c7c84 100644
> --- a/Documentation/networking/smc-sysctl.rst
> +++ b/Documentation/networking/smc-sysctl.rst
> @@ -71,3 +71,39 @@ smcr_max_conns_per_lgr - INTEGER
> acceptable value ranges from 16 to 255. Only for SMC-R v2.1 and later.
>
> Default: 255
> +
> +smcr_max_send_wr - INTEGER
> + So called work request buffers are SMCR link (and RDMA queue pair) level
> + resources necessary for performing RDMA operations. Since up to 255
> + connections can share a link group and thus also a link and the number
> + of the work request buffers is decided when the link is allocated,
> + depending on the workload it can a bottleneck in a sense that threads

missing 'be' or 'become' here^^

> + have to wait for work request buffers to become available. Before the
> + introduction of this control the maximal number of work request buffers
> + available on the send path used to be hard coded to 16. With this control
> + it becomes configurable. The acceptable range is between 2 and 2048.
> +
> + Please be aware that all the buffers need to be allocated as a physically
> + continuous array in which each element is a single buffer and has the size
> + of SMC_WR_BUF_SIZE (48) bytes. If the allocation fails we give up much
> + like before having this control.
> +
> + Default: 16
> +
> +smcr_max_recv_wr - INTEGER
> + So called work request buffers are SMCR link (and RDMA queue pair) level
> + resources necessary for performing RDMA operations. Since up to 255
> + connections can share a link group and thus also a link and the number
> + of the work request buffers is decided when the link is allocated,
> + depending on the workload it can a bottleneck in a sense that threads

same here^^

[...]
> @@ -683,6 +678,8 @@ int smc_ib_create_queue_pair(struct smc_link *lnk)
> };
> int rc;
>
> + qp_attr.cap.max_send_wr = 3 * lnk->lgr->max_send_wr;
> + qp_attr.cap.max_recv_wr = lnk->lgr->max_recv_wr;

Possibly:

cap = max(3 * lnk->lgr->max_send_wr, lnk->lgr->max_recv_wr);
qp_attr.cap.max_send_wr = cap;
qp_attr.cap.max_recv_wr = cap

to avoid assumption on `max_send_wr`, `max_recv_wr` relative values.

[...]
> diff --git a/net/smc/smc_sysctl.h b/net/smc/smc_sysctl.h
> index eb2465ae1e15..8538915af7af 100644
> --- a/net/smc/smc_sysctl.h
> +++ b/net/smc/smc_sysctl.h
> @@ -25,6 +25,8 @@ static inline int smc_sysctl_net_init(struct net *net)
> net->smc.sysctl_autocorking_size = SMC_AUTOCORKING_DEFAULT_SIZE;
> net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER;
> net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER;
> + net->smc.sysctl_smcr_max_send_wr = SMCR_MAX_SEND_WR_DEF;
> + net->smc.sysctl_smcr_max_recv_wr = SMCR_MAX_RECV_WR_DEF;
> return 0;
> }
>
> diff --git a/net/smc/smc_wr.c b/net/smc/smc_wr.c
> index b04a21b8c511..f5b2772414fd 100644
> --- a/net/smc/smc_wr.c
> +++ b/net/smc/smc_wr.c
> @@ -34,6 +34,7 @@
> #define SMC_WR_MAX_POLL_CQE 10 /* max. # of compl. queue elements in 1 poll */
>
> #define SMC_WR_RX_HASH_BITS 4
> +

Please avoid unrelated whitespace only changes.

Thanks,

Paolo