[PATCH net-next v8 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels

[Mingming Cao <mmc@linux.ibm.com>]

Report MQ-aware channel limits and counts from get_channels():

  - max_rx is IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled and
    mq_fallback is not set, else the live rx_count
  - rx_count tracks the driver's published num_rx_queues

Previously get_channels() mirrored netdev->real_num_rx_queues for both
fields, which under-reports max_rx on MQ firmware. Here every publish
of num_rx_queues is paired with a matching real_num, so rx_count does
not change yet; reading the published count keeps it right once the
resize patches can publish a count that real_num fails to reach.

This patch only reports. The set_channels() rx_count reject lives in
the MQ enable patch; the live RX set lands with the incremental resize
patch. Once mq_fallback is set but not yet applied, rx_count is still N
until the next open; max_rx follows rx_count rather than dropping
to 1, because the ethtool core rejects a TX-only -L that
resubmits rx_count above max_rx.

mq_fallback is set from the replenish path without RTNL and is now
read under RTNL here, so its accesses use READ_ONCE()/WRITE_ONCE(),
including the two writers the MQ enable patch added.

Keep this out of the stats patch: channel reporting is ethtool -l ABI,
independent of per-queue counters, so it can be reviewed and blamed on
its own.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v8:
- get_channels max_rx is the live rx_count when single queue or once
  mq_fallback is set, not a fixed 1 (the v7 patch 15 cap moves here)
- mq_fallback uses READ_ONCE()/WRITE_ONCE(): replenish writes it
  without RTNL
- the set_channels rx_count reject is now in patch 10

Changes in v7:
- reject rx_count change with -EOPNOTSUPP in
  set_channels() until patch 14 implements resize
- noted: TX-only read-modify-write still passes;
  max_rx stays here (reject, not reorder);
  get_channels uses the adapter count,
  get_base_stats uses real_num; RX max is
  MAX_RX_QUEUES, not the TX CPU cap; P14
  wires live rx_count resize; P15 is the
  tip ABI (live rx_count, max_rx cap on
  mq_fallback, max_tx at least the live tx_count)

Changes in v5:
- get_channels rx_count via get_num_rx_queues() (READ_ONCE consistency)
- New peel: get_channels from v4 stats -> tip P12 (14->15)
- get_channels: max_rx = MAX_RX_QUEUES when MQ else 1; rx_count =
  get_num_rx_queues() (was mirroring real_num_rx_queues for both)

 drivers/net/ethernet/ibm/ibmveth.c | 17 +++++++++++------
 drivers/net/ethernet/ibm/ibmveth.h |  1 +
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 45adf6e9be09..1e2e845f8011 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1306,7 +1306,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
 		dev_err_ratelimited(&adapter->netdev->dev,
 				    "MQ buffer add H_FUNCTION (q=%d, batch=%u), reset\n",
 				    queue_index, fail.batch);
-		adapter->mq_fallback = true;
+		WRITE_ONCE(adapter->mq_fallback, true);
 		schedule_work(&adapter->work);
 	}
 
@@ -2152,7 +2152,7 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address,
 		if (rc) {
 			/* Firmware MQ gone: fall back to SQ on next open. */
 			if (rc == -EOPNOTSUPP)
-				adapter->mq_fallback = true;
+				WRITE_ONCE(adapter->mq_fallback, true);
 			goto err_unregister;
 		}
 	}
@@ -2192,10 +2192,10 @@ static void ibmveth_apply_mq_fallback(struct ibmveth_adapter *adapter)
 {
 	struct net_device *netdev = adapter->netdev;
 
-	if (!adapter->mq_fallback)
+	if (!READ_ONCE(adapter->mq_fallback))
 		return;
 
-	adapter->mq_fallback = false;
+	WRITE_ONCE(adapter->mq_fallback, false);
 
 	if (!adapter->multi_queue && ibmveth_get_num_rx_queues(adapter) == 1)
 		return;
@@ -2756,11 +2756,16 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev,
 static void ibmveth_get_channels(struct net_device *netdev,
 				 struct ethtool_channels *channels)
 {
+	struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
 	channels->max_tx = ibmveth_real_max_tx_queues();
 	channels->tx_count = netdev->real_num_tx_queues;
 
-	channels->max_rx = netdev->real_num_rx_queues;
-	channels->rx_count = netdev->real_num_rx_queues;
+	channels->rx_count = ibmveth_get_num_rx_queues(adapter);
+	if (adapter->multi_queue && !READ_ONCE(adapter->mq_fallback))
+		channels->max_rx = IBMVETH_MAX_RX_QUEUES;
+	else
+		channels->max_rx = channels->rx_count;
 }
 
 static int ibmveth_set_channels(struct net_device *netdev,
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 76640bd5598e..66288f8e8349 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -374,6 +374,7 @@ struct ibmveth_adapter {
 	char queue_irq_name[IBMVETH_MAX_RX_QUEUES][32];
 	bool multi_queue;
 	unsigned int num_rx_queues;
+	/* Set from replenish without RTNL: READ_ONCE()/WRITE_ONCE(). */
 	bool mq_fallback;
 	/* Lifetime: true while RX IRQ handlers / NAPI are installed. */
 	bool rx_irq_setup;
-- 
2.50.1 (Apple Git-155)