[PATCH net] bnx2x: skip invalid DCBX priorities in bitmaps and FW config

["Lâm Trần" <admin@rezlabs.io>]

When the DCBX application feature is disabled, bnx2x stores
INVALID_TRAFFIC_TYPE_PRIORITY (0xffffffff) in
traffic_type_priority[]. bnx2x_dcbx_map_nw() uses that value as a
shift count, which UBSAN reports as:

  shift exponent 4294967295 is too large for 32-bit type 'int'

bnx2x_dcbx_fw_struct() assigns the same sentinel to a u8 before
shifting, so UBSAN reports exponent 255.

Skip priorities outside 0..MAX_PFC_PRIORITIES-1 when building COS
bitmasks and when filling the firmware TX-start structure. Keep the
stored sentinel so disabled-app state remains visible; skipped
firmware entries stay zero from the existing memset.

Valid 802.1p priorities 0..7 keep the previous mapping and firmware
encoding.

A userspace UBSAN extraction of these two functions reproduces the
unpatched shifts and passes after this change. This is not a kernel
or hardware test; the driver was not rebuilt or loaded.

Fixes: 09b775e7ec08 ("bnx2x: dcb - send all unmapped priorities to same COS as L2")
Fixes: e4901dde12d9 ("bnx2x: add DCB support")
Assisted-by: LLM
Signed-off-by: Lâm Trần <admin@rezlabs.io>
---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c
index 9af8163..e429e64 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c
@@ -350,13 +350,20 @@ static void bnx2x_dcbx_map_nw(struct bnx2x *bp)
 	int i;
 	u32 unmapped = (1 << MAX_PFC_PRIORITIES) - 1; /* all ones */
 	u32 *ttp = bp->dcbx_port_params.app.traffic_type_priority;
-	u32 nw_prio = 1 << ttp[LLFC_TRAFFIC_TYPE_NW];
+	u32 nw_prio;
 	struct bnx2x_dcbx_cos_params *cos_params =
 			bp->dcbx_port_params.ets.cos_params;
 
+	if (ttp[LLFC_TRAFFIC_TYPE_NW] >= MAX_PFC_PRIORITIES)
+		return;
+
+	nw_prio = 1 << ttp[LLFC_TRAFFIC_TYPE_NW];
+
 	/* get unmapped priorities by clearing mapped bits */
-	for (i = 0; i < LLFC_DRIVER_TRAFFIC_TYPE_MAX; i++)
-		unmapped &= ~(1 << ttp[i]);
+	for (i = 0; i < LLFC_DRIVER_TRAFFIC_TYPE_MAX; i++) {
+		if (ttp[i] < MAX_PFC_PRIORITIES)
+			unmapped &= ~(1 << ttp[i]);
+	}
 
 	/* find cos for nw prio and extend it with unmapped */
 	for (i = 0; i < ARRAY_SIZE(bp->dcbx_port_params.ets.cos_params); i++) {
@@ -1871,6 +1878,9 @@ static void bnx2x_dcbx_fw_struct(struct bnx2x *bp,
 
 	/* Fill priority parameters */
 	for (pri = 0; pri < LLFC_DRIVER_TRAFFIC_TYPE_MAX; pri++) {
+		if (ttp[pri] >= MAX_PFC_PRIORITIES)
+			continue;
+
 		tt2cos[pri].priority = ttp[pri];
 		pri_bit = 1 << tt2cos[pri].priority;
 
-- 
2.50.1 (Apple Git-155)