[PATCH net 6/6] iavf: cap advertised max_pkt_size at the single-buffer HW limit

[Tony Nguyen <anthony.l.nguyen@intel.com>]

From: Dave Butler <david.butler@appgate.com>

Since commit 5fa4caff59f2 ("iavf: switch to Page Pool")
iavf_configure_queues() advertises max_pkt_size to the PF as:

	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);

LIBIE_MAX_RX_FRM_LEN (16382) is the multi-descriptor scatter/gather frame
ceiling, not a single-queue value, and it exceeds the E810 MAC frame size
maximum of 9728. Per the E810 datasheet (613875-009 section 13.2.2.17.1)
the Tx frame-size register PRTDCB_TDPUC.MAX_TXFRAME has a maximum of
0x2600 (9728); larger frames are discarded. The in-tree ice driver encodes
the same value as ICE_AQ_SET_MAC_FRAME_SIZE_MAX (== LIBIE_MAX_RX_BUF_LEN ==
9728), and the VF clamped max_frame to IAVF_MAX_RXBUFFER (9728) before this
commit.

When the PF advertises vf_res->max_mtu as 0, min_not_zero() leaves
max_frame at 16382. The Linux ice PF advertises max_mtu = port MAC frame
size (<= 9728), so a VF behind ice never sends more than that. The ESXi
"icen" PF on E810 advertises max_mtu as 0, so the VF sends
max_pkt_size = 16382, which icen rejects while programming the queue
context for VIRTCHNL_OP_CONFIG_VSI_QUEUES (opcode 6):

	icen_ConfigureTxQueue: VSI 8: Failed to set LAN Tx queue context for
	                       absolute Tx queue 64, Error: ICE_ERR_PARAM
	indrv_SendMsgToVf: VF 0: Failed opcode 6, Error -5

	iavf 0000:03:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
	iavf 0000:03:00.0 ethX: NETDEV WATCHDOG: transmit queue N timed out

The VF's queues never come up; under SR-IOV passthrough the mis-programmed
queue can also trigger a fatal IOMMU fault in the guest. Forcing only
max_pkt_size back to 9728 (and leaving the Page Pool rx_buf_len/
databuffer_size untouched) makes the VF come up; databuffer_size is not
involved. This was confirmed on two E810 NVM revisions (3.00 and 4.51) and
two icen versions (1.14.2.0 and the latest 2.3.3.0): all reject the
unpatched VF and accept the patched one, so the trigger is the icen PF
behaviour, not the firmware or icen revision. Reported by several users on
E810 + ESXi icen with v6.10+ guests:

Link: https://community.intel.com/t5/Ethernet-Products/E810-C-iavf-driver-issue-on-Linux-6-12/m-p/1737490
Link: https://access.redhat.com/solutions/6973766
Link: https://knowledge.broadcom.com/external/article/404315/sriov-enabled-vms-network-adaptor-goes-d.html

Cap max_frame at the single-buffer hardware limit, restoring the
pre-Page-Pool behaviour while keeping the Page Pool rx_buf_len unchanged.

Cc: stable@vger.kernel.org # v6.10+
Fixes: 5fa4caff59f2 ("iavf: switch to Page Pool")
Signed-off-by: Dave Butler <david.butler@appgate.com>
Acked-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
---
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index ec234cc8bd9d..680a28a739bf 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -382,6 +382,14 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
 
 	max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
 	max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);
+	/* The PF programs max_pkt_size into the per-queue Rx context "rxmax".
+	 * LIBIE_MAX_RX_FRM_LEN is the multi-descriptor (S/G) frame ceiling
+	 * (16382), but that exceeds the E810 max MAC frame size (9728); some
+	 * PFs reject the out-of-range value with VIRTCHNL_STATUS_ERR_PARAM.
+	 * Cap it at the single-buffer HW limit (== the MAC frame max),
+	 * restoring the pre-Page-Pool behaviour.
+	 */
+	max_frame = min(max_frame, LIBIE_MAX_RX_BUF_LEN);
 
 	if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
 		/* bail because we already have a command pending */
-- 
2.47.1