[PATCH net-next v1 6/6] ibmvnic: change the mtu without a reset where the buffers allow it

[Mingming Cao <mmc@linux.ibm.com>]

With the earlier patches in this series, in-range mtus are honoured
instead of snapped to 1500 or 9000. Every such change still cycles the
adapter through wait_for_reset(), which tears down the CRQ, logs in
again and rebuilds every queue. The link goes down for around two
seconds each time, even when the change needs nothing from the VIOS.

Most of them do not. The backing device runs at one of a small set of
fixed sizes and the vnicserver reports that size when it answers
REQ_MTU, so any mtu within it is already carried end to end. Record it
as adapter->backing_mtu (set when the covering PARTIALSUCCESS path from
the honour-mtu patch accepts a request) and skip the reset when the new
mtu is within it and the tx buffers are large enough as they are, which
covers every decrease and the increases that stay inside the current
buffer size.

Both conditions are needed. Buffer sizes are rounded up to a cache
line, so buffers alone would also admit an mtu somewhat past what the
backing device carries: with a 1514 backing mtu, everything up to 1532
aligns to the same 1536-byte buffer. Those frames would leave the
partition and be dropped by a backing device still configured for the
smaller size.

An increase past backing_mtu still resets, since it has to be put to
the vnicserver rather than assumed. Whatever comes back becomes the new
backing_mtu, so if the device does move up, later changes within the
larger size settle without a reset, and if it does not, the mtu is
renegotiated down to what the device carries and the fast path keeps
measuring against the truth.

Buffers are left at their existing size on a decrease rather than
shrunk. reuse_tx_pools() compares prev_mtu against req_mtu, so the next
reset for any reason reallocates them.

Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Vaishnavi Bhat <vaishnavi@linux.ibm.com>
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
---
 drivers/net/ethernet/ibm/ibmvnic.c | 35 +++++++++++++++++++++++++++---
 drivers/net/ethernet/ibm/ibmvnic.h |  1 +
 2 files changed, 33 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 88d0c231a74f..1d18a0e13cad 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3705,8 +3705,34 @@ out:
 static int ibmvnic_change_mtu(struct net_device *netdev, int new_mtu)
 {
 	struct ibmvnic_adapter *adapter = netdev_priv(netdev);
+	u64 new_mtu_with_hdr = new_mtu + ETH_HLEN;
+	u64 old_buff_size, new_buff_size;
+
+	if (adapter->req_mtu == new_mtu_with_hdr)
+		return 0;
+
+	old_buff_size = ALIGN(adapter->prev_mtu + VLAN_HLEN, L1_CACHE_BYTES);
+	new_buff_size = ALIGN(new_mtu_with_hdr + VLAN_HLEN, L1_CACHE_BYTES);
+
+	/* Skip the reset when backing_mtu and the current buffers already
+	 * cover the new mtu. Keep desired.mtu in sync with req_mtu.
+	 */
+	if (new_mtu_with_hdr <= adapter->backing_mtu &&
+	    new_buff_size <= old_buff_size) {
+		netdev_dbg(netdev, "mtu %u->%d without reset\n",
+			   netdev->mtu, new_mtu);
+
+		WRITE_ONCE(netdev->mtu, new_mtu);
+		adapter->req_mtu = new_mtu_with_hdr;
+		adapter->desired.mtu = new_mtu_with_hdr;
+
+		return 0;
+	}
+
+	netdev_dbg(netdev, "mtu %u->%d needs larger buffers, resetting\n",
+		   netdev->mtu, new_mtu);
 
-	adapter->desired.mtu = new_mtu + ETH_HLEN;
+	adapter->desired.mtu = new_mtu_with_hdr;
 
 	return wait_for_reset(adapter);
 }
@@ -5558,14 +5584,17 @@ static void handle_request_cap_rsp(union ibmvnic_crq *crq,
 
 	switch (crq->request_capability_rsp.rc.code) {
 	case SUCCESS:
+		if (cap == REQ_MTU)
+			adapter->backing_mtu = *req_value;
 		break;
 	case PARTIALSUCCESS:
 		rsp_value = be64_to_cpu(crq->request_capability_rsp.number);
 
-		/* Covering PARTIALSUCCESS: keep the request. Otherwise
-		 * retry with rsp_value.
+		/* Covering PARTIALSUCCESS: keep the request and record
+		 * backing_mtu. Otherwise retry with rsp_value.
 		 */
 		if (cap == REQ_MTU && rsp_value >= *req_value) {
+			adapter->backing_mtu = rsp_value;
 			netdev_dbg(adapter->netdev,
 				   "backing mtu %llu covers requested %llu\n",
 				   rsp_value, *req_value);
diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h
index 4cfedae5d89d..80e1b2f6ede5 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.h
+++ b/drivers/net/ethernet/ibm/ibmvnic.h
@@ -1020,6 +1020,7 @@ struct ibmvnic_adapter {
 	u64 max_mtu;
 	u64 req_mtu;
 	u64 prev_mtu;
+	u64 backing_mtu; /* mtu the backing device currently carries */
 	u64 max_multicast_filters;
 	u64 vlan_header_insertion;
 	u64 rx_vlan_header_insertion;
-- 
2.50.1 (Apple Git-155)