[PATCH] net/mlx5: fix shared Rx queue limitations

[Dariusz Sosnowski <dsosnowski@nvidia.com>]

Affected patch introduced a new Rx burst function to mlx5 PMD
for handling out of order completions which are possible
on shared Rx queues.
This burst function is selected if and only if at least one of
the Rx queues on the device is configured as shared.

This burst function contains only non-vectorized code and is not
compatible with multi packet Rx (MPRQ). Although the function selection
worked correctly, if either vectorized Rx datapath was supported
or MPRQ was enabled, the relevant resources were still initialized.
For example, additional fake mbufs were set up for vectorized Rx
leading to memory leaks.

This patch fixes that by adding additional checks for shared Rx queues
to vectorized and MPRQ support checks.
Shared Rx queue limitations and Rx burst functions info is also amended
in mlx5 PMD documentation.

Fixes: 5f9223611f35 ("net/mlx5: fix out-of-order completions in ordinary Rx burst")
Cc: viacheslavo@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/nics/mlx5.rst         | 41 +++++++++++++++++++-------------
 drivers/net/mlx5/mlx5_rx.h       | 36 +++++++++++++++-------------
 drivers/net/mlx5/mlx5_rxtx_vec.c |  2 ++
 3 files changed, 46 insertions(+), 33 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 2529c2f4c8..a1a428dd69 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -956,27 +956,32 @@ The Rx function is selected based on multiple parameters:
 - :ref:`multi-packet Rx queues (MPRQ) <mlx5_mprq_params>`
 - :ref:`vectorized Rx datapath <mlx5_rx_vec_param>`
 
-This parameter may also have an impact on the behavior:
+These configurations may also have an impact on the behavior:
 
 - :ref:`packet descriptor (CQE) compression <mlx5_cqe_comp_param>`
+- :ref:`shared Rx queue <mlx5_shared_rx>`
 
 .. table:: Rx burst functions
 
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   || Function Name    || Parameters to Enable  || Scatter|| Error Recovery || CQE || Large|
-   |                   |                        |         |                 || comp|| MTU  |
-   +===================+========================+=========+=================+======+=======+
-   | rx_burst          | rx_vec_en=0            |   Yes   | Yes             |  Yes |  Yes  |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_vec      | rx_vec_en=1 (default)  |   No    | if CQE comp off |  Yes |  No   |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_mprq     || mprq_en=1             |   No    | Yes             |  Yes |  Yes  |
-   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |
-   +-------------------+------------------------+---------+-----------------+------+-------+
-   | rx_burst_mprq_vec || rx_vec_en=1 (default) |   No    | if CQE comp off |  Yes |  Yes  |
-   |                   || mprq_en=1             |         |                 |      |       |
-   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |
-   +-------------------+------------------------+---------+-----------------+------+-------+
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   || Function Name    || Parameters to Enable  || Scatter|| Error Recovery || CQE || Large|| Shared |
+   |                   |                        |         |                 || comp|| MTU  |  RXQ    |
+   +===================+========================+=========+=================+======+=======+=========+
+   | rx_burst          | rx_vec_en=0            |   Yes   | Yes             |  Yes |  Yes  | No      |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_vec      | rx_vec_en=1 (default)  |   No    | if CQE comp off |  Yes |  No   | No      |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_mprq     || mprq_en=1             |   No    | Yes             |  Yes |  Yes  | No      |
+   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst_mprq_vec || rx_vec_en=1 (default) |   No    | if CQE comp off |  Yes |  Yes  | No      |
+   |                   || mprq_en=1             |         |                 |      |       |         |
+   |                   || RxQs >= rxqs_min_mprq |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
+   | rx_burst (OoO)    | at least one Rx queue  |   Yes   | Yes             |  Yes |  Yes  | Yes     |
+   |                   | on the device          |         |                 |      |       |         |
+   |                   | is shared              |         |                 |      |       |         |
+   +-------------------+------------------------+---------+-----------------+------+-------+---------+
 
 
 Rx/Tx Tuning
@@ -1829,6 +1834,10 @@ Shared Rx Queue
 Limitations
 ^^^^^^^^^^^
 
+#. Shared Rx queue is not compatible with both vectorized Rx datapath and multi-packet Rx queues.
+   Any configuration related to these features passed through device arguments
+   will be ignored.
+
 #. Counters of received packets and bytes of devices in the same share group are same.
 
 #. Counters of received packets and bytes of queues in the same group and queue ID are same.
diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h
index 86636d598f..dffab3955b 100644
--- a/drivers/net/mlx5/mlx5_rx.h
+++ b/drivers/net/mlx5/mlx5_rx.h
@@ -602,6 +602,23 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Shared RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_shared_rq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	return !LIST_EMPTY(&priv->sh->shared_rxqs);
+}
+
 /**
  * Check whether Multi-Packet RQ can be enabled for the device.
  *
@@ -616,6 +633,8 @@ mlx5_check_mprq_support(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 
+	if (mlx5_shared_rq_enabled(dev))
+		return -ENOTSUP;
 	if (priv->config.mprq.enabled &&
 	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
 		return 1;
@@ -671,23 +690,6 @@ mlx5_mprq_enabled(struct rte_eth_dev *dev)
 	return n == n_ibv;
 }
 
-/**
- * Check whether Shared RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-static __rte_always_inline int
-mlx5_shared_rq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	return !LIST_EMPTY(&priv->sh->shared_rxqs);
-}
-
 /**
  * Check whether given RxQ is external.
  *
diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.c b/drivers/net/mlx5/mlx5_rxtx_vec.c
index 1b701801c5..79e92811d5 100644
--- a/drivers/net/mlx5/mlx5_rxtx_vec.c
+++ b/drivers/net/mlx5/mlx5_rxtx_vec.c
@@ -606,6 +606,8 @@ mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq)
 
 	if (!RXQ_PORT(ctrl)->config.rx_vec_en || rxq->sges_n != 0)
 		return -ENOTSUP;
+	if (mlx5_shared_rq_enabled(RXQ_DEV(ctrl)))
+		return -ENOTSUP;
 	if (rxq->lro)
 		return -ENOTSUP;
 	return 1;
-- 
2.47.3