[net PATCH v4 00/10] octeontx2: Miscellaneous fixes

[net PATCH v4 00/10] octeontx2: Miscellaneous fixes

[Sai Krishna]

This patchset includes following fixes.

Patch #1 Fix for the race condition while updating APR table 
 
Patch #2 Fix end bit position in NPC scan config 

Patch #3 Fix depth of CAM, MEM table entries

Patch #4 Fix in increase the size of DMAC filter flows

Patch #5 Fix driver crash resulting from invalid interface type
information retrieved from firmware

Patch #6 Fix incorrect mask used while installing filters involving
fragmented packets

Patch #7 Fixes for NPC field hash extract w.r.t IPV6 hash reduction,
         IPV6 filed hash configuration.

Patch #8 Fix for NPC hardware parser configuration destination 
         address hash, IPV6 endianness issues.

Patch #9 Fix for skipping mbox initialization for PFs disabled by firmware.

Patch #10 Fix disabling packet I/O in case of mailbox timeout.

Geetha sowjanya (1):
  octeontx2-af: Secure APR table update with the lock

Hariprasad Kelam (1):
  octeontx2-af: Add validation for lmac type

Ratheesh Kannoth (6):
  octeontx2-af: Fix start and end bit for scan config
  octeontx2-af: Fix depth of cam and mem table.
  octeontx2-pf: Increase the size of dmac filter flows
  octeontx2-af: Update/Fix NPC field hash extract feature
  octeontx2-af: Fix issues with NPC field hash extract
  octeontx2-af: Skip PFs if not enabled

Subbaraya Sundeep (1):
  octeontx2-pf: Disable packet I/O for graceful exit

Suman Ghosh (1):
  octeontx2-af: Update correct mask to filter IPv4 fragments

---
v4 changes:
	Fixed review comments given by Simon Horman
        1. Replaced kcalloc() with bitmap_zalloc().
	2. Modified/Elaborated commit messages.
        3. Fixed end bit position in NPC exact match bitmap enable.

v3 changes:
	Fixed review comments given by Simon Horman
        1. Split the patches
        2. Replaced devm_kcalloc() with kcalloc.
        3. Remove un-necessary validation before free_percpu
	4. Modified/Elaborated commit message
        5. Move the lock to inner function "rvu_get_lmtaddr()" to
           avoid synchronization issues.

v2 changes:
	Fixed review comments given by Leon Romanovsky
	1. Updated lmac_type in case of invalid lmac
	2. Modified commit message

 .../net/ethernet/marvell/octeontx2/af/cgx.c   |   8 ++
 .../net/ethernet/marvell/octeontx2/af/mbox.c  |   5 +-
 .../net/ethernet/marvell/octeontx2/af/mbox.h  |  19 ++-
 .../net/ethernet/marvell/octeontx2/af/rvu.c   |  44 +++++-
 .../ethernet/marvell/octeontx2/af/rvu_cn10k.c |  13 +-
 .../marvell/octeontx2/af/rvu_npc_fs.c         |  26 ++--
 .../marvell/octeontx2/af/rvu_npc_fs.h         |   4 +
 .../marvell/octeontx2/af/rvu_npc_hash.c       | 125 ++++++++++--------
 .../marvell/octeontx2/af/rvu_npc_hash.h       |  10 +-
 .../marvell/octeontx2/nic/otx2_common.h       |   4 +-
 .../ethernet/marvell/octeontx2/nic/otx2_pf.c  |  11 +-
 .../ethernet/marvell/octeontx2/nic/otx2_tc.c  |   2 +-
 .../ethernet/marvell/octeontx2/nic/otx2_vf.c  |   2 +-
 13 files changed, 185 insertions(+), 88 deletions(-)

-- 
2.25.1

[net PATCH v4 01/10] octeontx2-af: Secure APR table update with the lock

[Sai Krishna]

From: Geetha sowjanya <gakula@marvell.com>

APR table contains the lmtst base address of PF/VFs. These entries
are updated by the PF/VF during the device probe. The lmtst address
is fetched from HW using "TXN_REQ" and "ADDR_RSP_STS" registers.
The lock tries to protect these registers from getting overwritten
when multiple PFs invokes rvu_get_lmtaddr() simultaneously.

For example, if PF1 submit the request and got permitted before it
reads the response and PF2 got scheduled submit the request then the
response of PF1 is overwritten by the PF2 response.

Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable LMTST regions")
Signed-off-by: Geetha sowjanya <gakula@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 .../net/ethernet/marvell/octeontx2/af/rvu_cn10k.c   | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
index 4ad9ff025c96..0e74c5a2231e 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
@@ -60,13 +60,14 @@ static int rvu_get_lmtaddr(struct rvu *rvu, u16 pcifunc,
 			   u64 iova, u64 *lmt_addr)
 {
 	u64 pa, val, pf;
-	int err;
+	int err = 0;
 
 	if (!iova) {
 		dev_err(rvu->dev, "%s Requested Null address for transulation\n", __func__);
 		return -EINVAL;
 	}
 
+	mutex_lock(&rvu->rsrc_lock);
 	rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_REQ, iova);
 	pf = rvu_get_pf(pcifunc) & 0x1F;
 	val = BIT_ULL(63) | BIT_ULL(14) | BIT_ULL(13) | pf << 8 |
@@ -76,12 +77,13 @@ static int rvu_get_lmtaddr(struct rvu *rvu, u16 pcifunc,
 	err = rvu_poll_reg(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_RSP_STS, BIT_ULL(0), false);
 	if (err) {
 		dev_err(rvu->dev, "%s LMTLINE iova transulation failed\n", __func__);
-		return err;
+		goto exit;
 	}
 	val = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_RSP_STS);
 	if (val & ~0x1ULL) {
 		dev_err(rvu->dev, "%s LMTLINE iova transulation failed err:%llx\n", __func__, val);
-		return -EIO;
+		err = -EIO;
+		goto exit;
 	}
 	/* PA[51:12] = RVU_AF_SMMU_TLN_FLIT0[57:18]
 	 * PA[11:0] = IOVA[11:0]
@@ -89,8 +91,9 @@ static int rvu_get_lmtaddr(struct rvu *rvu, u16 pcifunc,
 	pa = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_TLN_FLIT0) >> 18;
 	pa &= GENMASK_ULL(39, 0);
 	*lmt_addr = (pa << 12) | (iova  & 0xFFF);
-
-	return 0;
+exit:
+	mutex_unlock(&rvu->rsrc_lock);
+	return err;
 }
 
 static int rvu_update_lmtaddr(struct rvu *rvu, u16 pcifunc, u64 lmt_addr)
-- 
2.25.1

[net PATCH v4 02/10] octeontx2-af: Fix start and end bit for scan config

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

In the current driver, NPC exact match feature was not getting
enabled as configured bit was not read properly.
for_each_set_bit_from() need end bit as one bit post
position in the bit map to read NPC exact nibble enable
bits properly. This patch fixes the same.

Fixes: b747923afff8 ("octeontx2-af: Exact match support")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
---
 drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
index 006beb5cf98d..f15efd41972e 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
@@ -594,8 +594,7 @@ static int npc_scan_kex(struct rvu *rvu, int blkaddr, u8 intf)
 	 */
 	masked_cfg = cfg & NPC_EXACT_NIBBLE;
 	bitnr = NPC_EXACT_NIBBLE_START;
-	for_each_set_bit_from(bitnr, (unsigned long *)&masked_cfg,
-			      NPC_EXACT_NIBBLE_START) {
+	for_each_set_bit_from(bitnr, (unsigned long *)&masked_cfg, NPC_EXACT_NIBBLE_END + 1) {
 		npc_scan_exact_result(mcam, bitnr, key_nibble, intf);
 		key_nibble++;
 	}
-- 
2.25.1

[net PATCH v4 03/10] octeontx2-af: Fix depth of cam and mem table.

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

In current driver, NPC cam and mem table sizes are read from wrong
register offset. This patch fixes the register offset so that correct
values are populated on read.

Fixes: b747923afff8 ("octeontx2-af: Exact match support")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
index 20ebb9c95c73..6597af84aa36 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
@@ -1868,9 +1868,9 @@ int rvu_npc_exact_init(struct rvu *rvu)
 	rvu->hw->table = table;
 
 	/* Read table size, ways and depth */
-	table->mem_table.depth = FIELD_GET(GENMASK_ULL(31, 24), npc_const3);
 	table->mem_table.ways = FIELD_GET(GENMASK_ULL(19, 16), npc_const3);
-	table->cam_table.depth = FIELD_GET(GENMASK_ULL(15, 0), npc_const3);
+	table->mem_table.depth = FIELD_GET(GENMASK_ULL(15, 0), npc_const3);
+	table->cam_table.depth = FIELD_GET(GENMASK_ULL(31, 24), npc_const3);
 
 	dev_dbg(rvu->dev, "%s: NPC exact match 4way_2k table(ways=%d, depth=%d)\n",
 		__func__,  table->mem_table.ways, table->cam_table.depth);
-- 
2.25.1

[net PATCH v4 04/10] octeontx2-pf: Increase the size of dmac filter flows

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

CN10kb supports large number of dmac filter flows to be
inserted. Increase the field size to accommodate the same

Fixes: b747923afff8 ("octeontx2-af: Exact match support")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
index 3d22cc6a2804..99cdc871b59c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
@@ -335,11 +335,11 @@ struct otx2_flow_config {
 #define OTX2_PER_VF_VLAN_FLOWS	2 /* Rx + Tx per VF */
 #define OTX2_VF_VLAN_RX_INDEX	0
 #define OTX2_VF_VLAN_TX_INDEX	1
-	u16			max_flows;
-	u8			dmacflt_max_flows;
 	u32			*bmap_to_dmacindex;
 	unsigned long		*dmacflt_bmap;
 	struct list_head	flow_list;
+	u32			dmacflt_max_flows;
+	u16                     max_flows;
 };
 
 struct otx2_tc_info {
-- 
2.25.1

[net PATCH v4 05/10] octeontx2-af: Add validation for lmac type

[Sai Krishna]

From: Hariprasad Kelam <hkelam@marvell.com>

Upon physical link change, firmware reports to the kernel about the
change along with the details like speed, lmac_type_id, etc.
Kernel derives lmac_type based on lmac_type_id received from firmware.

In a few scenarios, firmware returns an invalid lmac_type_id, which
is resulting in below kernel panic. This patch adds the missing
validation of the lmac_type_id field.

Internal error: Oops: 96000005 [#1] PREEMPT SMP
[   35.321595] Modules linked in:
[   35.328982] CPU: 0 PID: 31 Comm: kworker/0:1 Not tainted
5.4.210-g2e3169d8e1bc-dirty #17
[   35.337014] Hardware name: Marvell CN103XX board (DT)
[   35.344297] Workqueue: events work_for_cpu_fn
[   35.352730] pstate: 40400089 (nZcv daIf +PAN -UAO)
[   35.360267] pc : strncpy+0x10/0x30
[   35.366595] lr : cgx_link_change_handler+0x90/0x180

Fixes: 61071a871ea6 ("octeontx2-af: Forward CGX link notifications to PFs")
Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 drivers/net/ethernet/marvell/octeontx2/af/cgx.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cgx.c b/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
index 724df6398bbe..bd77152bb8d7 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
@@ -1231,6 +1231,14 @@ static inline void link_status_user_format(u64 lstat,
 	linfo->an = FIELD_GET(RESP_LINKSTAT_AN, lstat);
 	linfo->fec = FIELD_GET(RESP_LINKSTAT_FEC, lstat);
 	linfo->lmac_type_id = FIELD_GET(RESP_LINKSTAT_LMAC_TYPE, lstat);
+
+	if (linfo->lmac_type_id >= LMAC_MODE_MAX) {
+		dev_err(&cgx->pdev->dev, "Unknown lmac_type_id %d reported by firmware on cgx port%d:%d",
+			linfo->lmac_type_id, cgx->cgx_id, lmac_id);
+		strncpy(linfo->lmac_type, "Unknown", LMACTYPE_STR_LEN - 1);
+		return;
+	}
+
 	lmac_string = cgx_lmactype_string[linfo->lmac_type_id];
 	strncpy(linfo->lmac_type, lmac_string, LMACTYPE_STR_LEN - 1);
 }
-- 
2.25.1

[net PATCH v4 06/10] octeontx2-af: Update correct mask to filter IPv4 fragments

[Sai Krishna]

From: Suman Ghosh <sumang@marvell.com>

During the initial design, the IPv4 ip_flag mask was set to 0xff.
Which results to filter only fragmets with (fragment_offset == 0).
As part of the fix, updated the mask to 0x20 to filter all the
fragmented packets irrespective of the fragment_offset value.

Fixes: c672e3727989 ("octeontx2-pf: Add support to filter packet based on IP fragment")
Signed-off-by: Suman Ghosh <sumang@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c
index 044cc211424e..8392f63e433f 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c
@@ -544,7 +544,7 @@ static int otx2_tc_prepare_flow(struct otx2_nic *nic, struct otx2_tc_flow *node,
 		if (match.mask->flags & FLOW_DIS_IS_FRAGMENT) {
 			if (ntohs(flow_spec->etype) == ETH_P_IP) {
 				flow_spec->ip_flag = IPV4_FLAG_MORE;
-				flow_mask->ip_flag = 0xff;
+				flow_mask->ip_flag = IPV4_FLAG_MORE;
 				req->features |= BIT_ULL(NPC_IPFRAG_IPV4);
 			} else if (ntohs(flow_spec->etype) == ETH_P_IPV6) {
 				flow_spec->next_header = IPPROTO_FRAGMENT;
-- 
2.25.1

[net PATCH v4 07/10] octeontx2-af: Update/Fix NPC field hash extract feature

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

1. As per previous implementation, mask and control parameter to
generate the field hash value was not passed to the caller program.
Updated the secret key mbox to share that information as well,
as a part of the fix.
2. Earlier implementation did not consider hash reduction of both
source and destination IPv6 addresses. Only source IPv6 address
was considered. This fix solves that and provides option to hash

Fixes: 56d9f5fd2246 ("octeontx2-af: Use hashed field in MCAM key")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 .../net/ethernet/marvell/octeontx2/af/mbox.h  | 16 +++++---
 .../marvell/octeontx2/af/rvu_npc_hash.c       | 37 ++++++++++++-------
 .../marvell/octeontx2/af/rvu_npc_hash.h       |  6 +++
 3 files changed, 41 insertions(+), 18 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
index 5727d67e0259..0ce533848536 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
@@ -245,9 +245,9 @@ M(NPC_MCAM_READ_BASE_RULE, 0x6011, npc_read_base_steer_rule,            \
 M(NPC_MCAM_GET_STATS, 0x6012, npc_mcam_entry_stats,                     \
 				   npc_mcam_get_stats_req,              \
 				   npc_mcam_get_stats_rsp)              \
-M(NPC_GET_SECRET_KEY, 0x6013, npc_get_secret_key,                     \
-				   npc_get_secret_key_req,              \
-				   npc_get_secret_key_rsp)              \
+M(NPC_GET_FIELD_HASH_INFO, 0x6013, npc_get_field_hash_info,                     \
+				   npc_get_field_hash_info_req,              \
+				   npc_get_field_hash_info_rsp)              \
 M(NPC_GET_FIELD_STATUS, 0x6014, npc_get_field_status,                     \
 				   npc_get_field_status_req,              \
 				   npc_get_field_status_rsp)              \
@@ -1524,14 +1524,20 @@ struct npc_mcam_get_stats_rsp {
 	u8 stat_ena; /* enabled */
 };
 
-struct npc_get_secret_key_req {
+struct npc_get_field_hash_info_req {
 	struct mbox_msghdr hdr;
 	u8 intf;
 };
 
-struct npc_get_secret_key_rsp {
+struct npc_get_field_hash_info_rsp {
 	struct mbox_msghdr hdr;
 	u64 secret_key[3];
+#define NPC_MAX_HASH 2
+#define NPC_MAX_HASH_MASK 2
+	/* NPC_AF_INTF(0..1)_HASH(0..1)_MASK(0..1) */
+	u64 hash_mask[NPC_MAX_INTF][NPC_MAX_HASH][NPC_MAX_HASH_MASK];
+	/* NPC_AF_INTF(0..1)_HASH(0..1)_RESULT_CTRL */
+	u64 hash_ctrl[NPC_MAX_INTF][NPC_MAX_HASH];
 };
 
 enum ptp_op {
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
index 6597af84aa36..68f813040363 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
@@ -110,8 +110,8 @@ static u64 npc_update_use_hash(int lt, int ld)
 		 * in KEX_LD_CFG
 		 */
 		cfg = KEX_LD_CFG_USE_HASH(0x1, 0x03,
-					  ld ? 0x8 : 0x18,
-					  0x1, 0x0, 0x10);
+					  ld ? 0x18 : 0x8,
+					  0x1, 0x0, ld ? 0x14 : 0x10);
 		break;
 	}
 
@@ -134,7 +134,6 @@ static void npc_program_mkex_hash_rx(struct rvu *rvu, int blkaddr,
 				if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][ld]) {
 					u64 cfg = npc_update_use_hash(lt, ld);
 
-					hash_cnt++;
 					if (hash_cnt == NPC_MAX_HASH)
 						return;
 
@@ -149,6 +148,8 @@ static void npc_program_mkex_hash_rx(struct rvu *rvu, int blkaddr,
 							     mkex_hash->hash_mask[intf][ld][1]);
 					SET_KEX_LD_HASH_CTRL(intf, ld,
 							     mkex_hash->hash_ctrl[intf][ld]);
+
+					hash_cnt++;
 				}
 			}
 		}
@@ -171,7 +172,6 @@ static void npc_program_mkex_hash_tx(struct rvu *rvu, int blkaddr,
 				if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][ld]) {
 					u64 cfg = npc_update_use_hash(lt, ld);
 
-					hash_cnt++;
 					if (hash_cnt == NPC_MAX_HASH)
 						return;
 
@@ -187,8 +187,6 @@ static void npc_program_mkex_hash_tx(struct rvu *rvu, int blkaddr,
 					SET_KEX_LD_HASH_CTRL(intf, ld,
 							     mkex_hash->hash_ctrl[intf][ld]);
 					hash_cnt++;
-					if (hash_cnt == NPC_MAX_HASH)
-						return;
 				}
 		}
 	}
@@ -238,8 +236,8 @@ void npc_update_field_hash(struct rvu *rvu, u8 intf,
 			   struct flow_msg *omask)
 {
 	struct npc_mcam_kex_hash *mkex_hash = rvu->kpu.mkex_hash;
-	struct npc_get_secret_key_req req;
-	struct npc_get_secret_key_rsp rsp;
+	struct npc_get_field_hash_info_req req;
+	struct npc_get_field_hash_info_rsp rsp;
 	u64 ldata[2], cfg;
 	u32 field_hash;
 	u8 hash_idx;
@@ -250,7 +248,7 @@ void npc_update_field_hash(struct rvu *rvu, u8 intf,
 	}
 
 	req.intf = intf;
-	rvu_mbox_handler_npc_get_secret_key(rvu, &req, &rsp);
+	rvu_mbox_handler_npc_get_field_hash_info(rvu, &req, &rsp);
 
 	for (hash_idx = 0; hash_idx < NPC_MAX_HASH; hash_idx++) {
 		cfg = rvu_read64(rvu, blkaddr, NPC_AF_INTFX_HASHX_CFG(intf, hash_idx));
@@ -311,13 +309,13 @@ void npc_update_field_hash(struct rvu *rvu, u8 intf,
 	}
 }
 
-int rvu_mbox_handler_npc_get_secret_key(struct rvu *rvu,
-					struct npc_get_secret_key_req *req,
-					struct npc_get_secret_key_rsp *rsp)
+int rvu_mbox_handler_npc_get_field_hash_info(struct rvu *rvu,
+					     struct npc_get_field_hash_info_req *req,
+					     struct npc_get_field_hash_info_rsp *rsp)
 {
 	u64 *secret_key = rsp->secret_key;
 	u8 intf = req->intf;
-	int blkaddr;
+	int i, j, blkaddr;
 
 	blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
 	if (blkaddr < 0) {
@@ -329,6 +327,19 @@ int rvu_mbox_handler_npc_get_secret_key(struct rvu *rvu,
 	secret_key[1] = rvu_read64(rvu, blkaddr, NPC_AF_INTFX_SECRET_KEY1(intf));
 	secret_key[2] = rvu_read64(rvu, blkaddr, NPC_AF_INTFX_SECRET_KEY2(intf));
 
+	for (i = 0; i < NPC_MAX_HASH; i++) {
+		for (j = 0; j < NPC_MAX_HASH_MASK; j++) {
+			rsp->hash_mask[NIX_INTF_RX][i][j] =
+				GET_KEX_LD_HASH_MASK(NIX_INTF_RX, i, j);
+			rsp->hash_mask[NIX_INTF_TX][i][j] =
+				GET_KEX_LD_HASH_MASK(NIX_INTF_TX, i, j);
+		}
+	}
+
+	for (i = 0; i < NPC_MAX_INTF; i++)
+		for (j = 0; j < NPC_MAX_HASH; j++)
+			rsp->hash_ctrl[i][j] = GET_KEX_LD_HASH_CTRL(i, j);
+
 	return 0;
 }
 
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
index 3efeb09c58de..65936f4aeaac 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
@@ -31,6 +31,12 @@
 	rvu_write64(rvu, blkaddr,	\
 		    NPC_AF_INTFX_HASHX_MASKX(intf, ld, mask_idx), cfg)
 
+#define GET_KEX_LD_HASH_CTRL(intf, ld)	\
+	rvu_read64(rvu, blkaddr, NPC_AF_INTFX_HASHX_RESULT_CTRL(intf, ld))
+
+#define GET_KEX_LD_HASH_MASK(intf, ld, mask_idx)	\
+	rvu_read64(rvu, blkaddr, NPC_AF_INTFX_HASHX_MASKX(intf, ld, mask_idx))
+
 #define SET_KEX_LD_HASH_CTRL(intf, ld, cfg) \
 	rvu_write64(rvu, blkaddr,	\
 		    NPC_AF_INTFX_HASHX_RESULT_CTRL(intf, ld), cfg)
-- 
2.25.1

[net PATCH v4 08/10] octeontx2-af: Fix issues with NPC field hash extract

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

1. Allow field hash configuration for both source and destination IPv6.
2. Configure hardware parser based on hash extract feature enable flag
   for IPv6.
3. Fix IPv6 endianness issue while updating the source/destination IP
   address via ntuple rule.

Fixes: 56d9f5fd2246 ("octeontx2-af: Use hashed field in MCAM key")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
---
 .../marvell/octeontx2/af/rvu_npc_fs.c         | 23 +++--
 .../marvell/octeontx2/af/rvu_npc_fs.h         |  4 +
 .../marvell/octeontx2/af/rvu_npc_hash.c       | 88 ++++++++++---------
 .../marvell/octeontx2/af/rvu_npc_hash.h       |  4 +-
 4 files changed, 69 insertions(+), 50 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
index f15efd41972e..952319453701 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c
@@ -13,11 +13,6 @@
 #include "rvu_npc_fs.h"
 #include "rvu_npc_hash.h"
 
-#define NPC_BYTESM		GENMASK_ULL(19, 16)
-#define NPC_HDR_OFFSET		GENMASK_ULL(15, 8)
-#define NPC_KEY_OFFSET		GENMASK_ULL(5, 0)
-#define NPC_LDATA_EN		BIT_ULL(7)
-
 static const char * const npc_flow_names[] = {
 	[NPC_DMAC]	= "dmac",
 	[NPC_SMAC]	= "smac",
@@ -442,6 +437,7 @@ static void npc_handle_multi_layer_fields(struct rvu *rvu, int blkaddr, u8 intf)
 static void npc_scan_ldata(struct rvu *rvu, int blkaddr, u8 lid,
 			   u8 lt, u64 cfg, u8 intf)
 {
+	struct npc_mcam_kex_hash *mkex_hash = rvu->kpu.mkex_hash;
 	struct npc_mcam *mcam = &rvu->hw->mcam;
 	u8 hdr, key, nr_bytes, bit_offset;
 	u8 la_ltype, la_start;
@@ -490,8 +486,21 @@ do {									       \
 	NPC_SCAN_HDR(NPC_SIP_IPV4, NPC_LID_LC, NPC_LT_LC_IP, 12, 4);
 	NPC_SCAN_HDR(NPC_DIP_IPV4, NPC_LID_LC, NPC_LT_LC_IP, 16, 4);
 	NPC_SCAN_HDR(NPC_IPFRAG_IPV6, NPC_LID_LC, NPC_LT_LC_IP6_EXT, 6, 1);
-	NPC_SCAN_HDR(NPC_SIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 8, 16);
-	NPC_SCAN_HDR(NPC_DIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 24, 16);
+	if (rvu->hw->cap.npc_hash_extract) {
+		if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][0])
+			NPC_SCAN_HDR(NPC_SIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 8, 4);
+		else
+			NPC_SCAN_HDR(NPC_SIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 8, 16);
+
+		if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][1])
+			NPC_SCAN_HDR(NPC_DIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 24, 4);
+		else
+			NPC_SCAN_HDR(NPC_DIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 24, 16);
+	} else {
+		NPC_SCAN_HDR(NPC_SIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 8, 16);
+		NPC_SCAN_HDR(NPC_DIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 24, 16);
+	}
+
 	NPC_SCAN_HDR(NPC_SPORT_UDP, NPC_LID_LD, NPC_LT_LD_UDP, 0, 2);
 	NPC_SCAN_HDR(NPC_DPORT_UDP, NPC_LID_LD, NPC_LT_LD_UDP, 2, 2);
 	NPC_SCAN_HDR(NPC_SPORT_TCP, NPC_LID_LD, NPC_LT_LD_TCP, 0, 2);
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.h
index bdd65ce56a32..3f5c9042d10e 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.h
@@ -9,6 +9,10 @@
 #define __RVU_NPC_FS_H
 
 #define IPV6_WORDS	4
+#define NPC_BYTESM	GENMASK_ULL(19, 16)
+#define NPC_HDR_OFFSET	GENMASK_ULL(15, 8)
+#define NPC_KEY_OFFSET	GENMASK_ULL(5, 0)
+#define NPC_LDATA_EN	BIT_ULL(7)
 
 void npc_update_entry(struct rvu *rvu, enum key_fields type,
 		      struct mcam_entry *entry, u64 val_lo,
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
index 68f813040363..51209119f0f2 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.c
@@ -78,42 +78,43 @@ static u32 rvu_npc_toeplitz_hash(const u64 *data, u64 *key, size_t data_bit_len,
 	return hash_out;
 }
 
-u32 npc_field_hash_calc(u64 *ldata, struct npc_mcam_kex_hash *mkex_hash,
-			u64 *secret_key, u8 intf, u8 hash_idx)
+u32 npc_field_hash_calc(u64 *ldata, struct npc_get_field_hash_info_rsp rsp,
+			u8 intf, u8 hash_idx)
 {
 	u64 hash_key[3];
 	u64 data_padded[2];
 	u32 field_hash;
 
-	hash_key[0] = secret_key[1] << 31;
-	hash_key[0] |= secret_key[2];
-	hash_key[1] = secret_key[1] >> 33;
-	hash_key[1] |= secret_key[0] << 31;
-	hash_key[2] = secret_key[0] >> 33;
+	hash_key[0] = rsp.secret_key[1] << 31;
+	hash_key[0] |= rsp.secret_key[2];
+	hash_key[1] = rsp.secret_key[1] >> 33;
+	hash_key[1] |= rsp.secret_key[0] << 31;
+	hash_key[2] = rsp.secret_key[0] >> 33;
 
-	data_padded[0] = mkex_hash->hash_mask[intf][hash_idx][0] & ldata[0];
-	data_padded[1] = mkex_hash->hash_mask[intf][hash_idx][1] & ldata[1];
+	data_padded[0] = rsp.hash_mask[intf][hash_idx][0] & ldata[0];
+	data_padded[1] = rsp.hash_mask[intf][hash_idx][1] & ldata[1];
 	field_hash = rvu_npc_toeplitz_hash(data_padded, hash_key, 128, 159);
 
-	field_hash &= mkex_hash->hash_ctrl[intf][hash_idx] >> 32;
-	field_hash |= mkex_hash->hash_ctrl[intf][hash_idx];
+	field_hash &= FIELD_GET(GENMASK(63, 32), rsp.hash_ctrl[intf][hash_idx]);
+	field_hash += FIELD_GET(GENMASK(31, 0), rsp.hash_ctrl[intf][hash_idx]);
 	return field_hash;
 }
 
-static u64 npc_update_use_hash(int lt, int ld)
+static u64 npc_update_use_hash(struct rvu *rvu, int blkaddr,
+			       u8 intf, int lid, int lt, int ld)
 {
-	u64 cfg = 0;
-
-	switch (lt) {
-	case NPC_LT_LC_IP6:
-		/* Update use_hash(bit-20) and bytesm1 (bit-16:19)
-		 * in KEX_LD_CFG
-		 */
-		cfg = KEX_LD_CFG_USE_HASH(0x1, 0x03,
-					  ld ? 0x18 : 0x8,
-					  0x1, 0x0, ld ? 0x14 : 0x10);
-		break;
-	}
+	u8 hdr, key;
+	u64 cfg;
+
+	cfg = rvu_read64(rvu, blkaddr, NPC_AF_INTFX_LIDX_LTX_LDX_CFG(intf, lid, lt, ld));
+	hdr = FIELD_GET(NPC_HDR_OFFSET, cfg);
+	key = FIELD_GET(NPC_KEY_OFFSET, cfg);
+
+	/* Update use_hash(bit-20) to 'true' and
+	 * bytesm1(bit-16:19) to '0x3' in KEX_LD_CFG
+	 */
+	cfg = KEX_LD_CFG_USE_HASH(0x1, 0x03,
+				  hdr, 0x1, 0x0, key);
 
 	return cfg;
 }
@@ -132,11 +133,13 @@ static void npc_program_mkex_hash_rx(struct rvu *rvu, int blkaddr,
 		for (lt = 0; lt < NPC_MAX_LT; lt++) {
 			for (ld = 0; ld < NPC_MAX_LD; ld++) {
 				if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][ld]) {
-					u64 cfg = npc_update_use_hash(lt, ld);
+					u64 cfg;
 
 					if (hash_cnt == NPC_MAX_HASH)
 						return;
 
+					cfg = npc_update_use_hash(rvu, blkaddr,
+								  intf, lid, lt, ld);
 					/* Set updated KEX configuration */
 					SET_KEX_LD(intf, lid, lt, ld, cfg);
 					/* Set HASH configuration */
@@ -170,11 +173,13 @@ static void npc_program_mkex_hash_tx(struct rvu *rvu, int blkaddr,
 		for (lt = 0; lt < NPC_MAX_LT; lt++) {
 			for (ld = 0; ld < NPC_MAX_LD; ld++)
 				if (mkex_hash->lid_lt_ld_hash_en[intf][lid][lt][ld]) {
-					u64 cfg = npc_update_use_hash(lt, ld);
+					u64 cfg;
 
 					if (hash_cnt == NPC_MAX_HASH)
 						return;
 
+					cfg = npc_update_use_hash(rvu, blkaddr,
+								  intf, lid, lt, ld);
 					/* Set updated KEX configuration */
 					SET_KEX_LD(intf, lid, lt, ld, cfg);
 					/* Set HASH configuration */
@@ -264,44 +269,45 @@ void npc_update_field_hash(struct rvu *rvu, u8 intf,
 				 * is hashed to 32 bit value.
 				 */
 				case NPC_LT_LC_IP6:
-					if (features & BIT_ULL(NPC_SIP_IPV6)) {
+					/* ld[0] == hash_idx[0] == Source IPv6
+					 * ld[1] == hash_idx[1] == Destination IPv6
+					 */
+					if ((features & BIT_ULL(NPC_SIP_IPV6)) && !hash_idx) {
 						u32 src_ip[IPV6_WORDS];
 
 						be32_to_cpu_array(src_ip, pkt->ip6src, IPV6_WORDS);
-						ldata[0] = (u64)src_ip[0] << 32 | src_ip[1];
-						ldata[1] = (u64)src_ip[2] << 32 | src_ip[3];
+						ldata[1] = (u64)src_ip[0] << 32 | src_ip[1];
+						ldata[0] = (u64)src_ip[2] << 32 | src_ip[3];
 						field_hash = npc_field_hash_calc(ldata,
-										 mkex_hash,
-										 rsp.secret_key,
+										 rsp,
 										 intf,
 										 hash_idx);
 						npc_update_entry(rvu, NPC_SIP_IPV6, entry,
-								 field_hash, 0, 32, 0, intf);
+								 field_hash, 0,
+								 GENMASK(31, 0), 0, intf);
 						memcpy(&opkt->ip6src, &pkt->ip6src,
 						       sizeof(pkt->ip6src));
 						memcpy(&omask->ip6src, &mask->ip6src,
 						       sizeof(mask->ip6src));
-						break;
-					}
-
-					if (features & BIT_ULL(NPC_DIP_IPV6)) {
+					} else if ((features & BIT_ULL(NPC_DIP_IPV6)) && hash_idx) {
 						u32 dst_ip[IPV6_WORDS];
 
 						be32_to_cpu_array(dst_ip, pkt->ip6dst, IPV6_WORDS);
-						ldata[0] = (u64)dst_ip[0] << 32 | dst_ip[1];
-						ldata[1] = (u64)dst_ip[2] << 32 | dst_ip[3];
+						ldata[1] = (u64)dst_ip[0] << 32 | dst_ip[1];
+						ldata[0] = (u64)dst_ip[2] << 32 | dst_ip[3];
 						field_hash = npc_field_hash_calc(ldata,
-										 mkex_hash,
-										 rsp.secret_key,
+										 rsp,
 										 intf,
 										 hash_idx);
 						npc_update_entry(rvu, NPC_DIP_IPV6, entry,
-								 field_hash, 0, 32, 0, intf);
+								 field_hash, 0,
+								 GENMASK(31, 0), 0, intf);
 						memcpy(&opkt->ip6dst, &pkt->ip6dst,
 						       sizeof(pkt->ip6dst));
 						memcpy(&omask->ip6dst, &mask->ip6dst,
 						       sizeof(mask->ip6dst));
 					}
+
 					break;
 				}
 			}
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
index 65936f4aeaac..a1c3d987b804 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_hash.h
@@ -62,8 +62,8 @@ void npc_update_field_hash(struct rvu *rvu, u8 intf,
 			   struct flow_msg *omask);
 void npc_config_secret_key(struct rvu *rvu, int blkaddr);
 void npc_program_mkex_hash(struct rvu *rvu, int blkaddr);
-u32 npc_field_hash_calc(u64 *ldata, struct npc_mcam_kex_hash *mkex_hash,
-			u64 *secret_key, u8 intf, u8 hash_idx);
+u32 npc_field_hash_calc(u64 *ldata, struct npc_get_field_hash_info_rsp rsp,
+			u8 intf, u8 hash_idx);
 
 static struct npc_mcam_kex_hash npc_mkex_hash_default __maybe_unused = {
 	.lid_lt_ld_hash_en = {
-- 
2.25.1

[net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled

[Sai Krishna]

From: Ratheesh Kannoth <rkannoth@marvell.com>

Fiwmware enables set of PFs and allocate mbox resources based on the
number of enabled PFs. Current driver tries to initialize the mbox
resources even for disabled PFs, which may waste the resources.
This patch fixes the issue by skipping the mbox initialization of
disabled PFs.

Fixes: 9bdc47a6e328 ("octeontx2-af: Mbox communication support btw AF and it's VFs")
Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
---
 .../net/ethernet/marvell/octeontx2/af/mbox.c  |  5 ++-
 .../net/ethernet/marvell/octeontx2/af/mbox.h  |  3 +-
 .../net/ethernet/marvell/octeontx2/af/rvu.c   | 44 ++++++++++++++++---
 3 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
index 2898931d5260..9690ac01f02c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
@@ -157,7 +157,7 @@ EXPORT_SYMBOL(otx2_mbox_init);
  */
 int otx2_mbox_regions_init(struct otx2_mbox *mbox, void **hwbase,
 			   struct pci_dev *pdev, void *reg_base,
-			   int direction, int ndevs)
+			   int direction, int ndevs, unsigned long *pf_bmap)
 {
 	struct otx2_mbox_dev *mdev;
 	int devid, err;
@@ -169,6 +169,9 @@ int otx2_mbox_regions_init(struct otx2_mbox *mbox, void **hwbase,
 	mbox->hwbase = hwbase[0];
 
 	for (devid = 0; devid < ndevs; devid++) {
+		if (!test_bit(devid, pf_bmap))
+			continue;
+
 		mdev = &mbox->dev[devid];
 		mdev->mbase = hwbase[devid];
 		mdev->hwbase = hwbase[devid];
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
index 0ce533848536..26636a4d7dcc 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
@@ -96,9 +96,10 @@ void otx2_mbox_destroy(struct otx2_mbox *mbox);
 int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
 		   struct pci_dev *pdev, void __force *reg_base,
 		   int direction, int ndevs);
+
 int otx2_mbox_regions_init(struct otx2_mbox *mbox, void __force **hwbase,
 			   struct pci_dev *pdev, void __force *reg_base,
-			   int direction, int ndevs);
+			   int direction, int ndevs, unsigned long *bmap);
 void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
index 8683ce57ed3f..242089b6f199 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
@@ -2282,7 +2282,7 @@ static inline void rvu_afvf_mbox_up_handler(struct work_struct *work)
 }
 
 static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
-				int num, int type)
+				int num, int type, unsigned long *pf_bmap)
 {
 	struct rvu_hwinfo *hw = rvu->hw;
 	int region;
@@ -2294,6 +2294,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
 	 */
 	if (type == TYPE_AFVF) {
 		for (region = 0; region < num; region++) {
+			if (!test_bit(region, pf_bmap))
+				continue;
+
 			if (hw->cap.per_pf_mbox_regs) {
 				bar4 = rvu_read64(rvu, BLKADDR_RVUM,
 						  RVU_AF_PFX_BAR4_ADDR(0)) +
@@ -2315,6 +2318,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
 	 * RVU_AF_PF_BAR4_ADDR register.
 	 */
 	for (region = 0; region < num; region++) {
+		if (!test_bit(region, pf_bmap))
+			continue;
+
 		if (hw->cap.per_pf_mbox_regs) {
 			bar4 = rvu_read64(rvu, BLKADDR_RVUM,
 					  RVU_AF_PFX_BAR4_ADDR(region));
@@ -2343,12 +2349,33 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 	int err = -EINVAL, i, dir, dir_up;
 	void __iomem *reg_base;
 	struct rvu_work *mwork;
+	unsigned long *pf_bmap;
 	void **mbox_regions;
 	const char *name;
+	u64 cfg;
+
+	pf_bmap = bitmap_zalloc(num, GFP_KERNEL);
+	if (!pf_bmap)
+		return -ENOMEM;
+
+	/* RVU VFs */
+	if (type == TYPE_AFVF)
+		bitmap_set(pf_bmap, 0, num);
+
+	if (type == TYPE_AFPF) {
+		/* Mark enabled PFs in bitmap */
+		for (i = 0; i < num; i++) {
+			cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(i));
+			if (cfg & BIT_ULL(20))
+				set_bit(i, pf_bmap);
+		}
+	}
 
 	mbox_regions = kcalloc(num, sizeof(void *), GFP_KERNEL);
-	if (!mbox_regions)
+	if (!mbox_regions) {
+		bitmap_free(pf_bmap);
 		return -ENOMEM;
+	}
 
 	switch (type) {
 	case TYPE_AFPF:
@@ -2356,7 +2383,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 		dir = MBOX_DIR_AFPF;
 		dir_up = MBOX_DIR_AFPF_UP;
 		reg_base = rvu->afreg_base;
-		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFPF);
+		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFPF, pf_bmap);
 		if (err)
 			goto free_regions;
 		break;
@@ -2365,7 +2392,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 		dir = MBOX_DIR_PFVF;
 		dir_up = MBOX_DIR_PFVF_UP;
 		reg_base = rvu->pfreg_base;
-		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFVF);
+		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFVF, pf_bmap);
 		if (err)
 			goto free_regions;
 		break;
@@ -2396,16 +2423,19 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 	}
 
 	err = otx2_mbox_regions_init(&mw->mbox, mbox_regions, rvu->pdev,
-				     reg_base, dir, num);
+				     reg_base, dir, num, pf_bmap);
 	if (err)
 		goto exit;
 
 	err = otx2_mbox_regions_init(&mw->mbox_up, mbox_regions, rvu->pdev,
-				     reg_base, dir_up, num);
+				     reg_base, dir_up, num, pf_bmap);
 	if (err)
 		goto exit;
 
 	for (i = 0; i < num; i++) {
+		if (!test_bit(i, pf_bmap))
+			continue;
+
 		mwork = &mw->mbox_wrk[i];
 		mwork->rvu = rvu;
 		INIT_WORK(&mwork->work, mbox_handler);
@@ -2415,6 +2445,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 		INIT_WORK(&mwork->work, mbox_up_handler);
 	}
 	kfree(mbox_regions);
+	bitmap_free(pf_bmap);
 	return 0;
 
 exit:
@@ -2424,6 +2455,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
 		iounmap((void __iomem *)mbox_regions[num]);
 free_regions:
 	kfree(mbox_regions);
+	bitmap_free(pf_bmap);
 	return err;
 }
 
-- 
2.25.1

Re: [net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled

[Simon Horman]

On Wed, Apr 26, 2023 at 01:13:44PM +0530, Sai Krishna wrote:
> From: Ratheesh Kannoth <rkannoth@marvell.com>
> 
> Fiwmware enables set of PFs and allocate mbox resources based on the
> number of enabled PFs. Current driver tries to initialize the mbox
> resources even for disabled PFs, which may waste the resources.
> This patch fixes the issue by skipping the mbox initialization of
> disabled PFs.

FWIIW, this feels more like an enhancement than a fix to me.

> Fixes: 9bdc47a6e328 ("octeontx2-af: Mbox communication support btw AF and it's VFs")
> Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> ---
>  .../net/ethernet/marvell/octeontx2/af/mbox.c  |  5 ++-
>  .../net/ethernet/marvell/octeontx2/af/mbox.h  |  3 +-
>  .../net/ethernet/marvell/octeontx2/af/rvu.c   | 44 ++++++++++++++++---
>  3 files changed, 44 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> index 2898931d5260..9690ac01f02c 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> @@ -157,7 +157,7 @@ EXPORT_SYMBOL(otx2_mbox_init);
>   */
>  int otx2_mbox_regions_init(struct otx2_mbox *mbox, void **hwbase,
>  			   struct pci_dev *pdev, void *reg_base,
> -			   int direction, int ndevs)
> +			   int direction, int ndevs, unsigned long *pf_bmap)
>  {
>  	struct otx2_mbox_dev *mdev;
>  	int devid, err;
> @@ -169,6 +169,9 @@ int otx2_mbox_regions_init(struct otx2_mbox *mbox, void **hwbase,
>  	mbox->hwbase = hwbase[0];
>  
>  	for (devid = 0; devid < ndevs; devid++) {
> +		if (!test_bit(devid, pf_bmap))
> +			continue;
> +
>  		mdev = &mbox->dev[devid];
>  		mdev->mbase = hwbase[devid];
>  		mdev->hwbase = hwbase[devid];
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> index 0ce533848536..26636a4d7dcc 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> @@ -96,9 +96,10 @@ void otx2_mbox_destroy(struct otx2_mbox *mbox);
>  int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
>  		   struct pci_dev *pdev, void __force *reg_base,
>  		   int direction, int ndevs);
> +
>  int otx2_mbox_regions_init(struct otx2_mbox *mbox, void __force **hwbase,
>  			   struct pci_dev *pdev, void __force *reg_base,
> -			   int direction, int ndevs);
> +			   int direction, int ndevs, unsigned long *bmap);
>  void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
>  int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
>  int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> index 8683ce57ed3f..242089b6f199 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> @@ -2282,7 +2282,7 @@ static inline void rvu_afvf_mbox_up_handler(struct work_struct *work)
>  }
>  
>  static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
> -				int num, int type)
> +				int num, int type, unsigned long *pf_bmap)
>  {
>  	struct rvu_hwinfo *hw = rvu->hw;
>  	int region;
> @@ -2294,6 +2294,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
>  	 */
>  	if (type == TYPE_AFVF) {
>  		for (region = 0; region < num; region++) {
> +			if (!test_bit(region, pf_bmap))
> +				continue;
> +
>  			if (hw->cap.per_pf_mbox_regs) {
>  				bar4 = rvu_read64(rvu, BLKADDR_RVUM,
>  						  RVU_AF_PFX_BAR4_ADDR(0)) +
> @@ -2315,6 +2318,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
>  	 * RVU_AF_PF_BAR4_ADDR register.
>  	 */
>  	for (region = 0; region < num; region++) {
> +		if (!test_bit(region, pf_bmap))
> +			continue;
> +
>  		if (hw->cap.per_pf_mbox_regs) {
>  			bar4 = rvu_read64(rvu, BLKADDR_RVUM,
>  					  RVU_AF_PFX_BAR4_ADDR(region));
> @@ -2343,12 +2349,33 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  	int err = -EINVAL, i, dir, dir_up;
>  	void __iomem *reg_base;
>  	struct rvu_work *mwork;
> +	unsigned long *pf_bmap;
>  	void **mbox_regions;
>  	const char *name;
> +	u64 cfg;
> +
> +	pf_bmap = bitmap_zalloc(num, GFP_KERNEL);
> +	if (!pf_bmap)
> +		return -ENOMEM;
> +
> +	/* RVU VFs */
> +	if (type == TYPE_AFVF)
> +		bitmap_set(pf_bmap, 0, num);
> +
> +	if (type == TYPE_AFPF) {
> +		/* Mark enabled PFs in bitmap */
> +		for (i = 0; i < num; i++) {
> +			cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(i));
> +			if (cfg & BIT_ULL(20))
> +				set_bit(i, pf_bmap);
> +		}
> +	}
>  
>  	mbox_regions = kcalloc(num, sizeof(void *), GFP_KERNEL);
> -	if (!mbox_regions)
> +	if (!mbox_regions) {
> +		bitmap_free(pf_bmap);
>  		return -ENOMEM;

Maybe it is more idiomatic to use

	goto free_bitmap;

> +	}
>  
>  	switch (type) {
>  	case TYPE_AFPF:
> @@ -2356,7 +2383,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  		dir = MBOX_DIR_AFPF;
>  		dir_up = MBOX_DIR_AFPF_UP;
>  		reg_base = rvu->afreg_base;
> -		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFPF);
> +		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFPF, pf_bmap);
>  		if (err)
>  			goto free_regions;
>  		break;
> @@ -2365,7 +2392,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  		dir = MBOX_DIR_PFVF;
>  		dir_up = MBOX_DIR_PFVF_UP;
>  		reg_base = rvu->pfreg_base;
> -		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFVF);
> +		err = rvu_get_mbox_regions(rvu, mbox_regions, num, TYPE_AFVF, pf_bmap);
>  		if (err)
>  			goto free_regions;
>  		break;
> @@ -2396,16 +2423,19 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  	}
>  
>  	err = otx2_mbox_regions_init(&mw->mbox, mbox_regions, rvu->pdev,
> -				     reg_base, dir, num);
> +				     reg_base, dir, num, pf_bmap);
>  	if (err)
>  		goto exit;
>  
>  	err = otx2_mbox_regions_init(&mw->mbox_up, mbox_regions, rvu->pdev,
> -				     reg_base, dir_up, num);
> +				     reg_base, dir_up, num, pf_bmap);
>  	if (err)
>  		goto exit;
>  
>  	for (i = 0; i < num; i++) {
> +		if (!test_bit(i, pf_bmap))
> +			continue;
> +
>  		mwork = &mw->mbox_wrk[i];
>  		mwork->rvu = rvu;
>  		INIT_WORK(&mwork->work, mbox_handler);
> @@ -2415,6 +2445,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  		INIT_WORK(&mwork->work, mbox_up_handler);
>  	}
>  	kfree(mbox_regions);
> +	bitmap_free(pf_bmap);
>  	return 0;

Also, this could avoid duplication using:

	goto free_regions;

>  exit:
> @@ -2424,6 +2455,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
>  		iounmap((void __iomem *)mbox_regions[num]);
>  free_regions:
>  	kfree(mbox_regions);

free_bitmap:

> +	bitmap_free(pf_bmap);
>  	return err;
>  }
>  
> -- 
> 2.25.1
> 

RE: [EXT] Re: [net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled

[Ratheesh Kannoth]

Hi Simon,

Thanks for your review. Please find replies inline. 

-Ratheesh

> -----Original Message-----
> From: Simon Horman <simon.horman@corigine.com>
> Sent: Wednesday, April 26, 2023 3:31 PM
> To: Sai Krishna Gajula <saikrishnag@marvell.com>
> Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> pabeni@redhat.com; netdev@vger.kernel.org; linux-
> kernel@vger.kernel.org; leon@kernel.org; Sunil Kovvuri Goutham
> <sgoutham@marvell.com>; Geethasowjanya Akula <gakula@marvell.com>;
> Linu Cherian <lcherian@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Hariprasad Kelam <hkelam@marvell.com>;
> Subbaraya Sundeep Bhatta <sbhatta@marvell.com>; Ratheesh Kannoth
> <rkannoth@marvell.com>
> Subject: [EXT] Re: [net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled
> 
> External Email
> 
> ----------------------------------------------------------------------
> On Wed, Apr 26, 2023 at 01:13:44PM +0530, Sai Krishna wrote:
> > From: Ratheesh Kannoth <rkannoth@marvell.com>
> >
> > Fiwmware enables set of PFs and allocate mbox resources based on the
> > number of enabled PFs. Current driver tries to initialize the mbox
> > resources even for disabled PFs, which may waste the resources.
> > This patch fixes the issue by skipping the mbox initialization of
> > disabled PFs.
> 
> FWIIW, this feels more like an enhancement than a fix to me.
[Ratheesh Kannoth] I agree, commit message convey this change as enhancement. But this 
Code change fixes a crash in driver.  We will modify the commit message as below in next 
Patchset version. 
" Firmware enables PFs and allocate mbox resources for each of the PFs.
Currently PF driver configures mbox resources without checking whether
PF is enabled or not. This results in crash. This patch fixes this issue
by skipping disabled PF's mbox initialization. "
> 
> > Fixes: 9bdc47a6e328 ("octeontx2-af: Mbox communication support btw AF
> > and it's VFs")
> > Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>
> > Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> > Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> > ---
> >  .../net/ethernet/marvell/octeontx2/af/mbox.c  |  5 ++-
> > .../net/ethernet/marvell/octeontx2/af/mbox.h  |  3 +-
> >  .../net/ethernet/marvell/octeontx2/af/rvu.c   | 44 ++++++++++++++++---
> >  3 files changed, 44 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> > b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> > index 2898931d5260..9690ac01f02c 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> > +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c
> > @@ -157,7 +157,7 @@ EXPORT_SYMBOL(otx2_mbox_init);
> >   */
> >  int otx2_mbox_regions_init(struct otx2_mbox *mbox, void **hwbase,
> >  			   struct pci_dev *pdev, void *reg_base,
> > -			   int direction, int ndevs)
> > +			   int direction, int ndevs, unsigned long *pf_bmap)
> >  {
> >  	struct otx2_mbox_dev *mdev;
> >  	int devid, err;
> > @@ -169,6 +169,9 @@ int otx2_mbox_regions_init(struct otx2_mbox
> *mbox, void **hwbase,
> >  	mbox->hwbase = hwbase[0];
> >
> >  	for (devid = 0; devid < ndevs; devid++) {
> > +		if (!test_bit(devid, pf_bmap))
> > +			continue;
> > +
> >  		mdev = &mbox->dev[devid];
> >  		mdev->mbase = hwbase[devid];
> >  		mdev->hwbase = hwbase[devid];
> > diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> > b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> > index 0ce533848536..26636a4d7dcc 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> > +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h
> > @@ -96,9 +96,10 @@ void otx2_mbox_destroy(struct otx2_mbox *mbox);
> > int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
> >  		   struct pci_dev *pdev, void __force *reg_base,
> >  		   int direction, int ndevs);
> > +
> >  int otx2_mbox_regions_init(struct otx2_mbox *mbox, void __force
> **hwbase,
> >  			   struct pci_dev *pdev, void __force *reg_base,
> > -			   int direction, int ndevs);
> > +			   int direction, int ndevs, unsigned long *bmap);
> >  void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);  int
> > otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);  int
> > otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid); diff
> > --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> > b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> > index 8683ce57ed3f..242089b6f199 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> > +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
> > @@ -2282,7 +2282,7 @@ static inline void
> > rvu_afvf_mbox_up_handler(struct work_struct *work)  }
> >
> >  static int rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr,
> > -				int num, int type)
> > +				int num, int type, unsigned long *pf_bmap)
> >  {
> >  	struct rvu_hwinfo *hw = rvu->hw;
> >  	int region;
> > @@ -2294,6 +2294,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu,
> void **mbox_addr,
> >  	 */
> >  	if (type == TYPE_AFVF) {
> >  		for (region = 0; region < num; region++) {
> > +			if (!test_bit(region, pf_bmap))
> > +				continue;
> > +
> >  			if (hw->cap.per_pf_mbox_regs) {
> >  				bar4 = rvu_read64(rvu, BLKADDR_RVUM,
> >
> RVU_AF_PFX_BAR4_ADDR(0)) +
> > @@ -2315,6 +2318,9 @@ static int rvu_get_mbox_regions(struct rvu *rvu,
> void **mbox_addr,
> >  	 * RVU_AF_PF_BAR4_ADDR register.
> >  	 */
> >  	for (region = 0; region < num; region++) {
> > +		if (!test_bit(region, pf_bmap))
> > +			continue;
> > +
> >  		if (hw->cap.per_pf_mbox_regs) {
> >  			bar4 = rvu_read64(rvu, BLKADDR_RVUM,
> >  					  RVU_AF_PFX_BAR4_ADDR(region));
> @@ -2343,12 +2349,33 @@ static
> > int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
> >  	int err = -EINVAL, i, dir, dir_up;
> >  	void __iomem *reg_base;
> >  	struct rvu_work *mwork;
> > +	unsigned long *pf_bmap;
> >  	void **mbox_regions;
> >  	const char *name;
> > +	u64 cfg;
> > +
> > +	pf_bmap = bitmap_zalloc(num, GFP_KERNEL);
> > +	if (!pf_bmap)
> > +		return -ENOMEM;
> > +
> > +	/* RVU VFs */
> > +	if (type == TYPE_AFVF)
> > +		bitmap_set(pf_bmap, 0, num);
> > +
> > +	if (type == TYPE_AFPF) {
> > +		/* Mark enabled PFs in bitmap */
> > +		for (i = 0; i < num; i++) {
> > +			cfg = rvu_read64(rvu, BLKADDR_RVUM,
> RVU_PRIV_PFX_CFG(i));
> > +			if (cfg & BIT_ULL(20))
> > +				set_bit(i, pf_bmap);
> > +		}
> > +	}
> >
> >  	mbox_regions = kcalloc(num, sizeof(void *), GFP_KERNEL);
> > -	if (!mbox_regions)
> > +	if (!mbox_regions) {
> > +		bitmap_free(pf_bmap);
> >  		return -ENOMEM;
> 
> Maybe it is more idiomatic to use
> 
> 	goto free_bitmap;
> 
[Ratheesh Kannoth] Agree. Will update the code. 
> > +	}
> >
> >  	switch (type) {
> >  	case TYPE_AFPF:
> > @@ -2356,7 +2383,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct
> mbox_wq_info *mw,
> >  		dir = MBOX_DIR_AFPF;
> >  		dir_up = MBOX_DIR_AFPF_UP;
> >  		reg_base = rvu->afreg_base;
> > -		err = rvu_get_mbox_regions(rvu, mbox_regions, num,
> TYPE_AFPF);
> > +		err = rvu_get_mbox_regions(rvu, mbox_regions, num,
> TYPE_AFPF,
> > +pf_bmap);
> >  		if (err)
> >  			goto free_regions;
> >  		break;
> > @@ -2365,7 +2392,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct
> mbox_wq_info *mw,
> >  		dir = MBOX_DIR_PFVF;
> >  		dir_up = MBOX_DIR_PFVF_UP;
> >  		reg_base = rvu->pfreg_base;
> > -		err = rvu_get_mbox_regions(rvu, mbox_regions, num,
> TYPE_AFVF);
> > +		err = rvu_get_mbox_regions(rvu, mbox_regions, num,
> TYPE_AFVF,
> > +pf_bmap);
> >  		if (err)
> >  			goto free_regions;
> >  		break;
> > @@ -2396,16 +2423,19 @@ static int rvu_mbox_init(struct rvu *rvu, struct
> mbox_wq_info *mw,
> >  	}
> >
> >  	err = otx2_mbox_regions_init(&mw->mbox, mbox_regions, rvu-
> >pdev,
> > -				     reg_base, dir, num);
> > +				     reg_base, dir, num, pf_bmap);
> >  	if (err)
> >  		goto exit;
> >
> >  	err = otx2_mbox_regions_init(&mw->mbox_up, mbox_regions, rvu-
> >pdev,
> > -				     reg_base, dir_up, num);
> > +				     reg_base, dir_up, num, pf_bmap);
> >  	if (err)
> >  		goto exit;
> >
> >  	for (i = 0; i < num; i++) {
> > +		if (!test_bit(i, pf_bmap))
> > +			continue;
> > +
> >  		mwork = &mw->mbox_wrk[i];
> >  		mwork->rvu = rvu;
> >  		INIT_WORK(&mwork->work, mbox_handler); @@ -2415,6
> +2445,7 @@ static
> > int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw,
> >  		INIT_WORK(&mwork->work, mbox_up_handler);
> >  	}
> >  	kfree(mbox_regions);
> > +	bitmap_free(pf_bmap);
> >  	return 0;
> 
> Also, this could avoid duplication using:
> 
> 	goto free_regions;
> 
> >  exit:
> > @@ -2424,6 +2455,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct
> mbox_wq_info *mw,
> >  		iounmap((void __iomem *)mbox_regions[num]);
> >  free_regions:
> >  	kfree(mbox_regions);
> 
> free_bitmap:
> 
> > +	bitmap_free(pf_bmap);
> >  	return err;
> >  }
> >
> > --
> > 2.25.1
> >

Re: [EXT] Re: [net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled

[Simon Horman]

On Tue, May 02, 2023 at 03:10:53AM +0000, Ratheesh Kannoth wrote:
> Hi Simon,
> 
> Thanks for your review. Please find replies inline. 
> 
> -Ratheesh
> 
> > -----Original Message-----
> > From: Simon Horman <simon.horman@corigine.com>
> > Sent: Wednesday, April 26, 2023 3:31 PM
> > To: Sai Krishna Gajula <saikrishnag@marvell.com>
> > Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> > pabeni@redhat.com; netdev@vger.kernel.org; linux-
> > kernel@vger.kernel.org; leon@kernel.org; Sunil Kovvuri Goutham
> > <sgoutham@marvell.com>; Geethasowjanya Akula <gakula@marvell.com>;
> > Linu Cherian <lcherian@marvell.com>; Jerin Jacob Kollanukkaran
> > <jerinj@marvell.com>; Hariprasad Kelam <hkelam@marvell.com>;
> > Subbaraya Sundeep Bhatta <sbhatta@marvell.com>; Ratheesh Kannoth
> > <rkannoth@marvell.com>
> > Subject: [EXT] Re: [net PATCH v4 09/10] octeontx2-af: Skip PFs if not enabled
> > 
> > External Email
> > 
> > ----------------------------------------------------------------------
> > On Wed, Apr 26, 2023 at 01:13:44PM +0530, Sai Krishna wrote:
> > > From: Ratheesh Kannoth <rkannoth@marvell.com>
> > >
> > > Fiwmware enables set of PFs and allocate mbox resources based on the
> > > number of enabled PFs. Current driver tries to initialize the mbox
> > > resources even for disabled PFs, which may waste the resources.
> > > This patch fixes the issue by skipping the mbox initialization of
> > > disabled PFs.
> > 
> > FWIIW, this feels more like an enhancement than a fix to me.
>
> [Ratheesh Kannoth] I agree, commit message convey this change as
> enhancement. But this Code change fixes a crash in driver.  We will
> modify the commit message as below in next Patchset version.  " Firmware
> enables PFs and allocate mbox resources for each of the PFs.  Currently
> PF driver configures mbox resources without checking whether PF is
> enabled or not. This results in crash. This patch fixes this issue by
> skipping disabled PF's mbox initialization. "

Thanks, much appreciated.

[net PATCH v4 10/10] octeontx2-pf: Disable packet I/O for graceful exit

[Sai Krishna]

From: Subbaraya Sundeep <sbhatta@marvell.com>

At the stage of enabling packet I/O in otx2_open, If mailbox
timeout occurs then interface ends up in down state where as
hardware packet I/O is enabled. Hence disable packet I/O also
before bailing out.

As per earlier implementation, when the VF device probe fails due
to error in MSIX vector allocation, the LF resources, NIX and NPA
LF were not detached. This patch fixes in releasing these
resources, when ever this MSIX vector allocation/VF probe fails.

Fixes: 1ea0166da050 ("octeontx2-pf: Fix the device state on error")
Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
---
 drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 11 ++++++++++-
 drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c |  2 +-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index 179433d0a54a..52a57d2493dc 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -1835,13 +1835,22 @@ int otx2_open(struct net_device *netdev)
 		otx2_dmacflt_reinstall_flows(pf);
 
 	err = otx2_rxtx_enable(pf, true);
-	if (err)
+	/* If a mbox communication error happens at this point then interface
+	 * will end up in a state such that it is in down state but hardware
+	 * mcam entries are enabled to receive the packets. Hence disable the
+	 * packet I/O.
+	 */
+	if (err == EIO)
+		goto err_disable_rxtx;
+	else if (err)
 		goto err_tx_stop_queues;
 
 	otx2_do_set_rx_mode(pf);
 
 	return 0;
 
+err_disable_rxtx:
+	otx2_rxtx_enable(pf, false);
 err_tx_stop_queues:
 	netif_tx_stop_all_queues(netdev);
 	netif_carrier_off(netdev);
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
index ab126f8706c7..53366dbfbf27 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
@@ -621,7 +621,7 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	err = otx2vf_realloc_msix_vectors(vf);
 	if (err)
-		goto err_mbox_destroy;
+		goto err_detach_rsrc;
 
 	err = otx2_set_real_num_queues(netdev, qcount, qcount);
 	if (err)
-- 
2.25.1

Re: [net PATCH v4 10/10] octeontx2-pf: Disable packet I/O for graceful exit

[Simon Horman]

On Wed, Apr 26, 2023 at 01:13:45PM +0530, Sai Krishna wrote:
> From: Subbaraya Sundeep <sbhatta@marvell.com>
> 
> At the stage of enabling packet I/O in otx2_open, If mailbox
> timeout occurs then interface ends up in down state where as
> hardware packet I/O is enabled. Hence disable packet I/O also
> before bailing out.
> 
> As per earlier implementation, when the VF device probe fails due
> to error in MSIX vector allocation, the LF resources, NIX and NPA
> LF were not detached. This patch fixes in releasing these
> resources, when ever this MSIX vector allocation/VF probe fails.

It seems to me that the issue in the 2nd paragraph / hunk is
different to that in the first. And thus it seems to be appropriate
for it to be in a separate patch, possibly with.

Fixes: 3184fb5ba96e ("octeontx2-vf: Virtual function driver support")

> Fixes: 1ea0166da050 ("octeontx2-pf: Fix the device state on error")
> Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> ---
>  drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 11 ++++++++++-
>  drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c |  2 +-
>  2 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> index 179433d0a54a..52a57d2493dc 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> @@ -1835,13 +1835,22 @@ int otx2_open(struct net_device *netdev)
>  		otx2_dmacflt_reinstall_flows(pf);
>  
>  	err = otx2_rxtx_enable(pf, true);
> -	if (err)
> +	/* If a mbox communication error happens at this point then interface
> +	 * will end up in a state such that it is in down state but hardware
> +	 * mcam entries are enabled to receive the packets. Hence disable the
> +	 * packet I/O.
> +	 */
> +	if (err == EIO)
> +		goto err_disable_rxtx;
> +	else if (err)
>  		goto err_tx_stop_queues;
>  
>  	otx2_do_set_rx_mode(pf);
>  
>  	return 0;
>  
> +err_disable_rxtx:
> +	otx2_rxtx_enable(pf, false);
>  err_tx_stop_queues:
>  	netif_tx_stop_all_queues(netdev);
>  	netif_carrier_off(netdev);
> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> index ab126f8706c7..53366dbfbf27 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> @@ -621,7 +621,7 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>  
>  	err = otx2vf_realloc_msix_vectors(vf);
>  	if (err)
> -		goto err_mbox_destroy;
> +		goto err_detach_rsrc;
>  
>  	err = otx2_set_real_num_queues(netdev, qcount, qcount);
>  	if (err)
> -- 
> 2.25.1
> 

Re: [net PATCH v4 10/10] octeontx2-pf: Disable packet I/O for graceful exit

[Sai Krishna Gajula]

> -----Original Message-----
> From: Simon Horman <simon.horman@corigine.com>
> Sent: Wednesday, April 26, 2023 3:36 PM
> To: Sai Krishna Gajula <saikrishnag@marvell.com>
> Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> pabeni@redhat.com; netdev@vger.kernel.org; linux-
> kernel@vger.kernel.org; leon@kernel.org; Sunil Kovvuri Goutham
> <sgoutham@marvell.com>; Geethasowjanya Akula <gakula@marvell.com>;
> Linu Cherian <lcherian@marvell.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>; Hariprasad Kelam <hkelam@marvell.com>;
> Subbaraya Sundeep Bhatta <sbhatta@marvell.com>
> Subject: Re: [net PATCH v4 10/10] octeontx2-pf: Disable packet I/O for
> graceful exit
> 
> On Wed, Apr 26, 2023 at 01:13:45PM +0530, Sai Krishna wrote:
> > From: Subbaraya Sundeep <sbhatta@marvell.com>
> >
> > At the stage of enabling packet I/O in otx2_open, If mailbox timeout
> > occurs then interface ends up in down state where as hardware packet
> > I/O is enabled. Hence disable packet I/O also before bailing out.
> >
> > As per earlier implementation, when the VF device probe fails due to
> > error in MSIX vector allocation, the LF resources, NIX and NPA LF were
> > not detached. This patch fixes in releasing these resources, when ever
> > this MSIX vector allocation/VF probe fails.
> 
> It seems to me that the issue in the 2nd paragraph / hunk is different to that
> in the first. And thus it seems to be appropriate for it to be in a separate
> patch, possibly with.
> 
> Fixes: 3184fb5ba96e ("octeontx2-vf: Virtual function driver support")

We will separate this VF patch and send it in V5 patch series or as a separate patch.

> 
> > Fixes: 1ea0166da050 ("octeontx2-pf: Fix the device state on error")
> > Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
> > Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> > Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> > ---
> >  drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 11 ++++++++++-
> > drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c |  2 +-
> >  2 files changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> > b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> > index 179433d0a54a..52a57d2493dc 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
> > @@ -1835,13 +1835,22 @@ int otx2_open(struct net_device *netdev)
> >  		otx2_dmacflt_reinstall_flows(pf);
> >
> >  	err = otx2_rxtx_enable(pf, true);
> > -	if (err)
> > +	/* If a mbox communication error happens at this point then
> interface
> > +	 * will end up in a state such that it is in down state but hardware
> > +	 * mcam entries are enabled to receive the packets. Hence disable
> the
> > +	 * packet I/O.
> > +	 */
> > +	if (err == EIO)
> > +		goto err_disable_rxtx;
> > +	else if (err)
> >  		goto err_tx_stop_queues;
> >
> >  	otx2_do_set_rx_mode(pf);
> >
> >  	return 0;
> >
> > +err_disable_rxtx:
> > +	otx2_rxtx_enable(pf, false);
> >  err_tx_stop_queues:
> >  	netif_tx_stop_all_queues(netdev);
> >  	netif_carrier_off(netdev);
> > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> > b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> > index ab126f8706c7..53366dbfbf27 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
> > @@ -621,7 +621,7 @@ static int otx2vf_probe(struct pci_dev *pdev,
> > const struct pci_device_id *id)
> >
> >  	err = otx2vf_realloc_msix_vectors(vf);
> >  	if (err)
> > -		goto err_mbox_destroy;
> > +		goto err_detach_rsrc;
> >
> >  	err = otx2_set_real_num_queues(netdev, qcount, qcount);
> >  	if (err)
> > --
> > 2.25.1
> >