[PATCH net-next v5 0/5] bnxt_fwctl: fwctl for Broadcom Netxtreme devices

[PATCH net-next v5 0/5] bnxt_fwctl: fwctl for Broadcom Netxtreme devices

[Pavan Chebbi]

Introducing bnxt_fwctl which follows along Jason's work [1].
It is an aux bus driver that enables fwctl for Broadcom
NetXtreme 574xx, 575xx and 576xx series chipsets by using
bnxt driver's capability to talk to devices' firmware.

The first patch moves the ULP definitions to a common place
inside include/linux/bnxt/. The second and third patches
refactor and extend the existing bnxt aux bus functions to
be able to add more than one auxiliary device. The last three
patches create an additional bnxt aux device, add bnxt_fwctl,
and the documentation.

[1] https://lore.kernel.org/netdev/0-v5-642aa0c94070+4447f-fwctl_jgg@nvidia.com/

v5: Addressed the v4's review comments as below:
Patch #2 and #3: Simplified aux bus device creation logic by
having the core maintain arrays of pointers to aux devices and
their contexts, thereby avoiding function calls from aux dev.
[thanks Leon]
Patch #4: Used memdup_user() as suggested by cocci. Addressed
additional review comments from Jonathon and Dave. Collected
Rb tags from Dave.

v4: In patch #4, added the missing kfree on error for response
buffer. Improved documentation in patch #5 based on comments
from Dave.

v3: Addressed the review comments as below
Patch #1: Removed redundant common.h [thanks Saeed]
Patch #2 and #3 merged into a single patch [thanks Jonathan]
Patch #3: Addressed comments from Jonathan
Patch #4 and #5: Addressed comments from Jonathan and Dave

v2: In patch #5, fixed a sparse warning where a __le16 was
degraded to an integer. Also addressed kdoc warnings for
include/uapi/fwctl/bnxt.h in the same patch.

v1: https://lore.kernel.org/netdev/20250922090851.719913-1-pavan.chebbi@broadcom.com/

The following are changes since commit 3a8660878839faadb4f1a6dd72c3179c1df56787:
  Linux 6.18-rc1
and are available in the git repository at:
  https://github.com/pavanchebbi/linux/tree/bnxt_fwctl_v5

Pavan Chebbi (5):
  bnxt_en: Move common definitions to include/linux/bnxt/
  bnxt_en: Refactor aux bus functions to be more generic
  bnxt_en: Create an aux device for fwctl
  bnxt_fwctl: Add bnxt fwctl device
  bnxt_fwctl: Add documentation entries

 .../userspace-api/fwctl/bnxt_fwctl.rst        |  78 +++
 Documentation/userspace-api/fwctl/fwctl.rst   |   1 +
 Documentation/userspace-api/fwctl/index.rst   |   1 +
 MAINTAINERS                                   |   6 +
 drivers/fwctl/Kconfig                         |  11 +
 drivers/fwctl/Makefile                        |   1 +
 drivers/fwctl/bnxt/Makefile                   |   4 +
 drivers/fwctl/bnxt/main.c                     | 453 ++++++++++++++++++
 drivers/infiniband/hw/bnxt_re/debugfs.c       |   2 +-
 drivers/infiniband/hw/bnxt_re/main.c          |   2 +-
 drivers/infiniband/hw/bnxt_re/qplib_fp.c      |   2 +-
 drivers/infiniband/hw/bnxt_re/qplib_res.h     |   2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  37 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  13 +-
 .../net/ethernet/broadcom/bnxt/bnxt_devlink.c |   2 +-
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |   4 +-
 .../net/ethernet/broadcom/bnxt/bnxt_sriov.c   |   2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 250 ++++++----
 .../bnxt_ulp.h => include/linux/bnxt/ulp.h    |  24 +-
 include/uapi/fwctl/bnxt.h                     |  64 +++
 include/uapi/fwctl/fwctl.h                    |   1 +
 21 files changed, 826 insertions(+), 134 deletions(-)
 create mode 100644 Documentation/userspace-api/fwctl/bnxt_fwctl.rst
 create mode 100644 drivers/fwctl/bnxt/Makefile
 create mode 100644 drivers/fwctl/bnxt/main.c
 rename drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h => include/linux/bnxt/ulp.h (85%)
 create mode 100644 include/uapi/fwctl/bnxt.h

-- 
2.39.1

[PATCH net-next v5 1/5] bnxt_en: Move common definitions to include/linux/bnxt/

[Pavan Chebbi]

We have common definitions that are now going to be used
by more than one component outside of bnxt (bnxt_re and
fwctl)

Move bnxt_ulp.h to include/linux/bnxt/ as ulp.h.

Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Cc: linux-rdma@vger.kernel.org
Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/debugfs.c                         | 2 +-
 drivers/infiniband/hw/bnxt_re/main.c                            | 2 +-
 drivers/infiniband/hw/bnxt_re/qplib_fp.c                        | 2 +-
 drivers/infiniband/hw/bnxt_re/qplib_res.h                       | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.c                       | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c               | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c               | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c                 | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c                   | 2 +-
 .../broadcom/bnxt/bnxt_ulp.h => include/linux/bnxt/ulp.h        | 0
 10 files changed, 9 insertions(+), 9 deletions(-)
 rename drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h => include/linux/bnxt/ulp.h (100%)

diff --git a/drivers/infiniband/hw/bnxt_re/debugfs.c b/drivers/infiniband/hw/bnxt_re/debugfs.c
index be5e9b5ca2f0..d72c1693c57f 100644
--- a/drivers/infiniband/hw/bnxt_re/debugfs.c
+++ b/drivers/infiniband/hw/bnxt_re/debugfs.c
@@ -10,8 +10,8 @@
 #include <linux/pci.h>
 #include <linux/seq_file.h>
 #include <rdma/ib_addr.h>
+#include <linux/bnxt/ulp.h>
 
-#include "bnxt_ulp.h"
 #include "roce_hsi.h"
 #include "qplib_res.h"
 #include "qplib_sp.h"
diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
index b13810572c2e..0fca523873b8 100644
--- a/drivers/infiniband/hw/bnxt_re/main.c
+++ b/drivers/infiniband/hw/bnxt_re/main.c
@@ -55,8 +55,8 @@
 #include <rdma/ib_umem.h>
 #include <rdma/ib_addr.h>
 #include <linux/hashtable.h>
+#include <linux/bnxt/ulp.h>
 
-#include "bnxt_ulp.h"
 #include "roce_hsi.h"
 #include "qplib_res.h"
 #include "qplib_sp.h"
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
index ce90d3d834d4..9e97d67de1be 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
@@ -46,6 +46,7 @@
 #include <linux/delay.h>
 #include <linux/prefetch.h>
 #include <linux/if_ether.h>
+#include <linux/bnxt/ulp.h>
 #include <rdma/ib_mad.h>
 
 #include "roce_hsi.h"
@@ -55,7 +56,6 @@
 #include "qplib_sp.h"
 #include "qplib_fp.h"
 #include <rdma/ib_addr.h>
-#include "bnxt_ulp.h"
 #include "bnxt_re.h"
 #include "ib_verbs.h"
 
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h
index 2ea3b7f232a3..ebe8893937f6 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h
@@ -39,7 +39,7 @@
 #ifndef __BNXT_QPLIB_RES_H__
 #define __BNXT_QPLIB_RES_H__
 
-#include "bnxt_ulp.h"
+#include <linux/bnxt/ulp.h>
 
 extern const struct bnxt_qplib_gid bnxt_qplib_gid_zero;
 
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 3fc33b1b4dfb..34fef6b15ce1 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -59,10 +59,10 @@
 #include <net/netdev_rx_queue.h>
 #include <linux/pci-tph.h>
 #include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
 
 #include "bnxt.h"
 #include "bnxt_hwrm.h"
-#include "bnxt_ulp.h"
 #include "bnxt_sriov.h"
 #include "bnxt_ethtool.h"
 #include "bnxt_dcb.h"
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
index 02961d93ed35..cfcd3335a2d3 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
@@ -13,12 +13,12 @@
 #include <net/devlink.h>
 #include <net/netdev_lock.h>
 #include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
 #include "bnxt.h"
 #include "bnxt_hwrm.h"
 #include "bnxt_vfr.h"
 #include "bnxt_devlink.h"
 #include "bnxt_ethtool.h"
-#include "bnxt_ulp.h"
 #include "bnxt_ptp.h"
 #include "bnxt_coredump.h"
 #include "bnxt_nvm_defs.h"
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 41686a6f84b5..818bd0fa0a7d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -27,9 +27,9 @@
 #include <net/netdev_queues.h>
 #include <net/netlink.h>
 #include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
 #include "bnxt.h"
 #include "bnxt_hwrm.h"
-#include "bnxt_ulp.h"
 #include "bnxt_xdp.h"
 #include "bnxt_ptp.h"
 #include "bnxt_ethtool.h"
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
index 80fed2c07b9e..84c43f83193a 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
@@ -17,9 +17,9 @@
 #include <linux/etherdevice.h>
 #include <net/dcbnl.h>
 #include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
 #include "bnxt.h"
 #include "bnxt_hwrm.h"
-#include "bnxt_ulp.h"
 #include "bnxt_sriov.h"
 #include "bnxt_vfr.h"
 #include "bnxt_ethtool.h"
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index f8c2c72b382d..56a35c65d193 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -22,10 +22,10 @@
 #include <linux/auxiliary_bus.h>
 #include <net/netdev_lock.h>
 #include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
 
 #include "bnxt.h"
 #include "bnxt_hwrm.h"
-#include "bnxt_ulp.h"
 
 static DEFINE_IDA(bnxt_aux_dev_ids);
 
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/include/linux/bnxt/ulp.h
similarity index 100%
rename from drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h
rename to include/linux/bnxt/ulp.h
-- 
2.39.1

[PATCH net-next v5 2/5] bnxt_en: Refactor aux bus functions to be more generic

[Pavan Chebbi]

Up until now there was only one auxiliary device that bnxt
created and that was for RoCE driver. bnxt fwctl is also
going to use an aux bus device that bnxt should create.
This requires some nomenclature changes and refactoring of
the existing bnxt aux dev functions.

Convert 'aux_priv' and 'edev' members of struct bnxt into
arrays where each element contains supported auxbus device's
data. Move struct bnxt_aux_priv from bnxt.h to ulp.h because
that is where it belongs. Make aux bus init/uninit/add/del
functions more generic which will accept aux device type as
a parameter. Make bnxt_ulp_start/stop functions (the only
other common functions applicable to any aux device) loop
through the aux devices to update their config and states.

Also, as an improvement in code, bnxt_register_dev() can skip
unnecessary dereferencing of edev from bp, instead use the
edev pointer from the function parameter.

Future patches will reuse these functions to add an aux bus
device for fwctl.

Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  29 ++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  13 +-
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |   2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 238 ++++++++++--------
 include/linux/bnxt/ulp.h                      |  23 +-
 5 files changed, 181 insertions(+), 124 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 34fef6b15ce1..dcbb321cee3a 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -6848,7 +6848,8 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 #endif
 	if ((bp->flags & BNXT_FLAG_STRIP_VLAN) || def_vlan)
 		req->flags |= cpu_to_le32(VNIC_CFG_REQ_FLAGS_VLAN_STRIP_MODE);
-	if (vnic->vnic_id == BNXT_VNIC_DEFAULT && bnxt_ulp_registered(bp->edev))
+	if (vnic->vnic_id == BNXT_VNIC_DEFAULT &&
+	    bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA]))
 		req->flags |= cpu_to_le32(bnxt_get_roce_vnic_mode(bp));
 
 	return hwrm_req_send(bp, req);
@@ -7963,6 +7964,7 @@ static int bnxt_get_avail_msix(struct bnxt *bp, int num);
 
 static int __bnxt_reserve_rings(struct bnxt *bp)
 {
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	struct bnxt_hw_rings hwr = {0};
 	int rx_rings, old_rx_rings, rc;
 	int cp = bp->cp_nr_rings;
@@ -7973,7 +7975,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
 	if (!bnxt_need_reserve_rings(bp))
 		return 0;
 
-	if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) {
+	if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) {
 		ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want);
 		if (!ulp_msix)
 			bnxt_set_ulp_stat_ctxs(bp, 0);
@@ -8024,8 +8026,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
 	}
 	rx_rings = min_t(int, rx_rings, hwr.grp);
 	hwr.cp = min_t(int, hwr.cp, bp->cp_nr_rings);
-	if (bnxt_ulp_registered(bp->edev) &&
-	    hwr.stat > bnxt_get_ulp_stat_ctxs(bp))
+	if (bnxt_ulp_registered(edev) && hwr.stat > bnxt_get_ulp_stat_ctxs(bp))
 		hwr.stat -= bnxt_get_ulp_stat_ctxs(bp);
 	hwr.cp = min_t(int, hwr.cp, hwr.stat);
 	rc = bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh);
@@ -8064,7 +8065,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
 	    !netif_is_rxfh_configured(bp->dev))
 		bnxt_set_dflt_rss_indir_tbl(bp, NULL);
 
-	if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) {
+	if (!bnxt_ulp_registered(edev) && BNXT_NEW_RM(bp)) {
 		int resv_msix, resv_ctx, ulp_ctxs;
 		struct bnxt_hw_resc *hw_resc;
 
@@ -11412,6 +11413,7 @@ static void bnxt_clear_int_mode(struct bnxt *bp)
 
 int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
 {
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	bool irq_cleared = false;
 	bool irq_change = false;
 	int tcs = bp->num_tc;
@@ -11421,7 +11423,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
 	if (!bnxt_need_reserve_rings(bp))
 		return 0;
 
-	if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) {
+	if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) {
 		int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want);
 
 		if (ulp_msix > bp->ulp_num_msix_want)
@@ -14653,7 +14655,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
 		hwr.cp_p5 = hwr.tx + rx;
 	rc = bnxt_hwrm_check_rings(bp, &hwr);
 	if (!rc && pci_msix_can_alloc_dyn(bp->pdev)) {
-		if (!bnxt_ulp_registered(bp->edev)) {
+		if (!bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) {
 			hwr.cp += bnxt_get_ulp_msix_num(bp);
 			hwr.cp = min_t(int, hwr.cp, bnxt_get_max_func_irqs(bp));
 		}
@@ -16185,12 +16187,13 @@ static void bnxt_remove_one(struct pci_dev *pdev)
 	if (BNXT_PF(bp))
 		__bnxt_sriov_disable(bp);
 
-	bnxt_rdma_aux_device_del(bp);
+	bnxt_aux_device_del(bp, BNXT_AUXDEV_RDMA);
 
 	unregister_netdev(dev);
 	bnxt_ptp_clear(bp);
 
-	bnxt_rdma_aux_device_uninit(bp);
+	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_RDMA);
+	bnxt_auxdev_id_free(bp, bp->auxdev_id);
 
 	bnxt_free_l2_filters(bp, true);
 	bnxt_free_ntp_fltrs(bp, true);
@@ -16776,7 +16779,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	bnxt_set_tpa_flags(bp);
 	bnxt_init_ring_params(bp);
 	bnxt_set_ring_params(bp);
-	bnxt_rdma_aux_device_init(bp);
+	if (!bnxt_auxdev_id_alloc(bp))
+		bnxt_aux_device_init(bp, BNXT_AUXDEV_RDMA);
 	rc = bnxt_set_dflt_rings(bp, true);
 	if (rc) {
 		if (BNXT_VF(bp) && rc == -ENODEV) {
@@ -16840,7 +16844,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	bnxt_dl_fw_reporters_create(bp);
 
-	bnxt_rdma_aux_device_add(bp);
+	bnxt_aux_device_add(bp, BNXT_AUXDEV_RDMA);
 
 	bnxt_print_device_info(bp);
 
@@ -16848,7 +16852,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	return 0;
 init_err_cleanup:
-	bnxt_rdma_aux_device_uninit(bp);
+	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_RDMA);
+	bnxt_auxdev_id_free(bp, bp->auxdev_id);
 	bnxt_dl_unregister(bp);
 init_err_dl:
 	bnxt_shutdown_tc(bp);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 741b2d854789..e0c3ad6a76bf 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -24,12 +24,12 @@
 #include <linux/interrupt.h>
 #include <linux/rhashtable.h>
 #include <linux/crash_dump.h>
-#include <linux/auxiliary_bus.h>
 #include <net/devlink.h>
 #include <net/dst_metadata.h>
 #include <net/xdp.h>
 #include <linux/dim.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/bnxt/ulp.h>
 #ifdef CONFIG_TEE_BNXT_FW
 #include <linux/firmware/broadcom/tee_bnxt_fw.h>
 #endif
@@ -2075,12 +2075,6 @@ struct bnxt_fw_health {
 #define BNXT_FW_IF_RETRY		10
 #define BNXT_FW_SLOT_RESET_RETRY	4
 
-struct bnxt_aux_priv {
-	struct auxiliary_device aux_dev;
-	struct bnxt_en_dev *edev;
-	int id;
-};
-
 enum board_idx {
 	BCM57301,
 	BCM57302,
@@ -2340,8 +2334,8 @@ struct bnxt {
 #define BNXT_CHIP_P5_AND_MINUS(bp)		\
 	(BNXT_CHIP_P3(bp) || BNXT_CHIP_P4(bp) || BNXT_CHIP_P5(bp))
 
-	struct bnxt_aux_priv	*aux_priv;
-	struct bnxt_en_dev	*edev;
+	struct bnxt_aux_priv	*aux_priv[__BNXT_AUXDEV_MAX];
+	struct bnxt_en_dev	*edev[__BNXT_AUXDEV_MAX];
 
 	struct bnxt_napi	**bnapi;
 
@@ -2748,6 +2742,7 @@ struct bnxt {
 	struct bnxt_ctx_pg_info	*fw_crash_mem;
 	u32			fw_crash_len;
 	struct bnxt_bs_trace_info bs_trace[BNXT_TRACE_MAX];
+	int			auxdev_id;
 };
 
 #define BNXT_NUM_RX_RING_STATS			8
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 818bd0fa0a7d..57f7303e3465 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -5086,7 +5086,7 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest,
 
 	memset(buf, 0, sizeof(u64) * bp->num_tests);
 	if (etest->flags & ETH_TEST_FL_OFFLINE &&
-	    bnxt_ulp_registered(bp->edev)) {
+	    bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) {
 		etest->flags |= ETH_TEST_FL_FAILED;
 		netdev_warn(dev, "Offline tests cannot be run with RoCE driver loaded\n");
 		return;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index 56a35c65d193..e15cf4774e9b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -29,9 +29,17 @@
 
 static DEFINE_IDA(bnxt_aux_dev_ids);
 
+struct bnxt_aux_device {
+	const char *name;
+};
+
+static struct bnxt_aux_device bnxt_aux_devices[__BNXT_AUXDEV_MAX] = {{
+	.name		= "rdma",
+}};
+
 static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent)
 {
-	struct bnxt_en_dev *edev = bp->edev;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	int num_msix, i;
 
 	if (!edev->ulp_tbl->msix_requested) {
@@ -51,61 +59,75 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent)
 
 int bnxt_get_ulp_msix_num(struct bnxt *bp)
 {
-	if (bp->edev)
-		return bp->edev->ulp_num_msix_vec;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (edev)
+		return edev->ulp_num_msix_vec;
 	return 0;
 }
 
 void bnxt_set_ulp_msix_num(struct bnxt *bp, int num)
 {
-	if (bp->edev)
-		bp->edev->ulp_num_msix_vec = num;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (edev)
+		edev->ulp_num_msix_vec = num;
 }
 
 int bnxt_get_ulp_msix_num_in_use(struct bnxt *bp)
 {
-	if (bnxt_ulp_registered(bp->edev))
-		return bp->edev->ulp_num_msix_vec;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (bnxt_ulp_registered(edev))
+		return edev->ulp_num_msix_vec;
 	return 0;
 }
 
 int bnxt_get_ulp_stat_ctxs(struct bnxt *bp)
 {
-	if (bp->edev)
-		return bp->edev->ulp_num_ctxs;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (edev)
+		return edev->ulp_num_ctxs;
 	return 0;
 }
 
 void bnxt_set_ulp_stat_ctxs(struct bnxt *bp, int num_ulp_ctx)
 {
-	if (bp->edev)
-		bp->edev->ulp_num_ctxs = num_ulp_ctx;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (edev)
+		edev->ulp_num_ctxs = num_ulp_ctx;
 }
 
 int bnxt_get_ulp_stat_ctxs_in_use(struct bnxt *bp)
 {
-	if (bnxt_ulp_registered(bp->edev))
-		return bp->edev->ulp_num_ctxs;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (bnxt_ulp_registered(edev))
+		return edev->ulp_num_ctxs;
 	return 0;
 }
 
 void bnxt_set_dflt_ulp_stat_ctxs(struct bnxt *bp)
 {
-	if (bp->edev) {
-		bp->edev->ulp_num_ctxs = BNXT_MIN_ROCE_STAT_CTXS;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+	if (edev) {
+		edev->ulp_num_ctxs = BNXT_MIN_ROCE_STAT_CTXS;
 		/* Reserve one additional stat_ctx for PF0 (except
 		 * on 1-port NICs) as it also creates one stat_ctx
 		 * for PF1 in case of RoCE bonding.
 		 */
 		if (BNXT_PF(bp) && !bp->pf.port_id &&
 		    bp->port_count > 1)
-			bp->edev->ulp_num_ctxs++;
+			edev->ulp_num_ctxs++;
 
 		/* Reserve one additional stat_ctx when the device is capable
 		 * of supporting port mirroring on RDMA device.
 		 */
 		if (BNXT_MIRROR_ON_ROCE_CAP(bp))
-			bp->edev->ulp_num_ctxs++;
+			edev->ulp_num_ctxs++;
 	}
 }
 
@@ -141,7 +163,7 @@ int bnxt_register_dev(struct bnxt_en_dev *edev,
 
 	edev->ulp_tbl->msix_requested = bnxt_get_ulp_msix_num(bp);
 
-	bnxt_fill_msix_vecs(bp, bp->edev->msix_entries);
+	bnxt_fill_msix_vecs(bp, edev->msix_entries);
 	edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED;
 exit:
 	mutex_unlock(&edev->en_dev_lock);
@@ -230,78 +252,88 @@ EXPORT_SYMBOL(bnxt_send_msg);
 
 void bnxt_ulp_stop(struct bnxt *bp)
 {
-	struct bnxt_aux_priv *aux_priv = bp->aux_priv;
-	struct bnxt_en_dev *edev = bp->edev;
+	int i;
 
-	if (!edev)
-		return;
+	for (i = 0; i < __BNXT_AUXDEV_MAX; i++) {
+		struct bnxt_aux_priv *aux_priv = bp->aux_priv[i];
+		struct bnxt_en_dev *edev = bp->edev[i];
 
-	mutex_lock(&edev->en_dev_lock);
-	if (!bnxt_ulp_registered(edev) ||
-	    (edev->flags & BNXT_EN_FLAG_ULP_STOPPED))
-		goto ulp_stop_exit;
-
-	edev->flags |= BNXT_EN_FLAG_ULP_STOPPED;
-	if (aux_priv) {
-		struct auxiliary_device *adev;
-
-		adev = &aux_priv->aux_dev;
-		if (adev->dev.driver) {
-			const struct auxiliary_driver *adrv;
-			pm_message_t pm = {};
-
-			adrv = to_auxiliary_drv(adev->dev.driver);
-			edev->en_state = bp->state;
-			adrv->suspend(adev, pm);
+		if (!edev)
+			continue;
+
+		mutex_lock(&edev->en_dev_lock);
+		if (!bnxt_ulp_registered(edev) ||
+		    (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) {
+			mutex_unlock(&edev->en_dev_lock);
+			continue;
 		}
+
+		edev->flags |= BNXT_EN_FLAG_ULP_STOPPED;
+		if (aux_priv) {
+			struct auxiliary_device *adev;
+
+			adev = &aux_priv->aux_dev;
+			if (adev->dev.driver) {
+				const struct auxiliary_driver *adrv;
+				pm_message_t pm = {};
+
+				adrv = to_auxiliary_drv(adev->dev.driver);
+				edev->en_state = bp->state;
+				adrv->suspend(adev, pm);
+			}
+		}
+		mutex_unlock(&edev->en_dev_lock);
 	}
-ulp_stop_exit:
-	mutex_unlock(&edev->en_dev_lock);
 }
 
 void bnxt_ulp_start(struct bnxt *bp, int err)
 {
-	struct bnxt_aux_priv *aux_priv = bp->aux_priv;
-	struct bnxt_en_dev *edev = bp->edev;
+	int i;
 
-	if (!edev || err)
-		return;
+	for (i = 0; i < __BNXT_AUXDEV_MAX; i++) {
+		struct bnxt_aux_priv *aux_priv = bp->aux_priv[i];
+		struct bnxt_en_dev *edev = bp->edev[i];
 
-	mutex_lock(&edev->en_dev_lock);
-	if (!bnxt_ulp_registered(edev) ||
-	    !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED))
-		goto ulp_start_exit;
+		if (!edev || err)
+			return;
 
-	if (edev->ulp_tbl->msix_requested)
-		bnxt_fill_msix_vecs(bp, edev->msix_entries);
+		mutex_lock(&edev->en_dev_lock);
+		if (!bnxt_ulp_registered(edev) ||
+		    !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) {
+			goto clear_flag_continue;
+		}
 
-	if (aux_priv) {
-		struct auxiliary_device *adev;
+		if (edev->ulp_tbl->msix_requested)
+			bnxt_fill_msix_vecs(bp, edev->msix_entries);
 
-		adev = &aux_priv->aux_dev;
-		if (adev->dev.driver) {
-			const struct auxiliary_driver *adrv;
+		if (aux_priv) {
+			struct auxiliary_device *adev;
 
-			adrv = to_auxiliary_drv(adev->dev.driver);
-			edev->en_state = bp->state;
-			adrv->resume(adev);
+			adev = &aux_priv->aux_dev;
+			if (adev->dev.driver) {
+				const struct auxiliary_driver *adrv;
+
+				adrv = to_auxiliary_drv(adev->dev.driver);
+				edev->en_state = bp->state;
+				adrv->resume(adev);
+			}
 		}
+clear_flag_continue:
+		edev->flags &= ~BNXT_EN_FLAG_ULP_STOPPED;
+		mutex_unlock(&edev->en_dev_lock);
 	}
-ulp_start_exit:
-	edev->flags &= ~BNXT_EN_FLAG_ULP_STOPPED;
-	mutex_unlock(&edev->en_dev_lock);
 }
 
 void bnxt_ulp_irq_stop(struct bnxt *bp)
 {
-	struct bnxt_en_dev *edev = bp->edev;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	struct bnxt_ulp_ops *ops;
 	bool reset = false;
 
 	if (!edev || !(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED))
 		return;
 
-	if (bnxt_ulp_registered(bp->edev)) {
+	if (bnxt_ulp_registered(edev)) {
 		struct bnxt_ulp *ulp = edev->ulp_tbl;
 
 		if (!ulp->msix_requested)
@@ -318,13 +350,13 @@ void bnxt_ulp_irq_stop(struct bnxt *bp)
 
 void bnxt_ulp_irq_restart(struct bnxt *bp, int err)
 {
-	struct bnxt_en_dev *edev = bp->edev;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	struct bnxt_ulp_ops *ops;
 
 	if (!edev || !(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED))
 		return;
 
-	if (bnxt_ulp_registered(bp->edev)) {
+	if (bnxt_ulp_registered(edev)) {
 		struct bnxt_ulp *ulp = edev->ulp_tbl;
 		struct bnxt_msix_entry *ent = NULL;
 
@@ -350,7 +382,7 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err)
 void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl)
 {
 	u16 event_id = le16_to_cpu(cmpl->event_id);
-	struct bnxt_en_dev *edev = bp->edev;
+	struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
 	struct bnxt_ulp_ops *ops;
 	struct bnxt_ulp *ulp;
 
@@ -391,16 +423,16 @@ void bnxt_register_async_events(struct bnxt_en_dev *edev,
 }
 EXPORT_SYMBOL(bnxt_register_async_events);
 
-void bnxt_rdma_aux_device_uninit(struct bnxt *bp)
+void bnxt_aux_device_uninit(struct bnxt *bp, enum bnxt_auxdev_type idx)
 {
 	struct bnxt_aux_priv *aux_priv;
 	struct auxiliary_device *adev;
 
 	/* Skip if no auxiliary device init was done. */
-	if (!bp->aux_priv)
+	if (!bp->aux_priv[idx])
 		return;
 
-	aux_priv = bp->aux_priv;
+	aux_priv = bp->aux_priv[idx];
 	adev = &aux_priv->aux_dev;
 	auxiliary_device_uninit(adev);
 }
@@ -411,20 +443,19 @@ static void bnxt_aux_dev_release(struct device *dev)
 		container_of(dev, struct bnxt_aux_priv, aux_dev.dev);
 	struct bnxt *bp = netdev_priv(aux_priv->edev->net);
 
-	ida_free(&bnxt_aux_dev_ids, aux_priv->id);
 	kfree(aux_priv->edev->ulp_tbl);
-	bp->edev = NULL;
+	bp->edev[aux_priv->id] = NULL;
 	kfree(aux_priv->edev);
 	kfree(aux_priv);
-	bp->aux_priv = NULL;
+	bp->aux_priv[aux_priv->id] = NULL;
 }
 
-void bnxt_rdma_aux_device_del(struct bnxt *bp)
+void bnxt_aux_device_del(struct bnxt *bp, enum bnxt_auxdev_type idx)
 {
-	if (!bp->edev)
+	if (!bp->edev[idx])
 		return;
 
-	auxiliary_device_delete(&bp->aux_priv->aux_dev);
+	auxiliary_device_delete(&bp->aux_priv[idx]->aux_dev);
 }
 
 static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp)
@@ -454,59 +485,52 @@ static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp)
 	edev->bar0 = bp->bar0;
 }
 
-void bnxt_rdma_aux_device_add(struct bnxt *bp)
+void bnxt_aux_device_add(struct bnxt *bp, enum bnxt_auxdev_type idx)
 {
 	struct auxiliary_device *aux_dev;
 	int rc;
 
-	if (!bp->edev)
+	if (!bp->edev[idx])
 		return;
 
-	aux_dev = &bp->aux_priv->aux_dev;
+	aux_dev = &bp->aux_priv[idx]->aux_dev;
 	rc = auxiliary_device_add(aux_dev);
 	if (rc) {
 		netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n");
 		auxiliary_device_uninit(aux_dev);
-		bp->flags &= ~BNXT_FLAG_ROCE_CAP;
+		if (idx == BNXT_AUXDEV_RDMA)
+			bp->flags &= ~BNXT_FLAG_ROCE_CAP;
 	}
 }
 
-void bnxt_rdma_aux_device_init(struct bnxt *bp)
+void bnxt_aux_device_init(struct bnxt *bp, enum bnxt_auxdev_type idx)
 {
+	const char *name = bnxt_aux_devices[idx].name;
 	struct auxiliary_device *aux_dev;
 	struct bnxt_aux_priv *aux_priv;
 	struct bnxt_en_dev *edev;
 	struct bnxt_ulp *ulp;
 	int rc;
 
-	if (!(bp->flags & BNXT_FLAG_ROCE_CAP))
+	if (idx == BNXT_AUXDEV_RDMA && !(bp->flags & BNXT_FLAG_ROCE_CAP))
 		return;
 
-	aux_priv = kzalloc(sizeof(*bp->aux_priv), GFP_KERNEL);
+	aux_priv = kzalloc(sizeof(**bp->aux_priv), GFP_KERNEL);
 	if (!aux_priv)
 		goto exit;
 
-	aux_priv->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL);
-	if (aux_priv->id < 0) {
-		netdev_warn(bp->dev,
-			    "ida alloc failed for ROCE auxiliary device\n");
-		kfree(aux_priv);
-		goto exit;
-	}
-
 	aux_dev = &aux_priv->aux_dev;
-	aux_dev->id = aux_priv->id;
-	aux_dev->name = "rdma";
+	aux_dev->id = bp->auxdev_id;
+	aux_dev->name = name;
 	aux_dev->dev.parent = &bp->pdev->dev;
 	aux_dev->dev.release = bnxt_aux_dev_release;
 
 	rc = auxiliary_device_init(aux_dev);
 	if (rc) {
-		ida_free(&bnxt_aux_dev_ids, aux_priv->id);
 		kfree(aux_priv);
 		goto exit;
 	}
-	bp->aux_priv = aux_priv;
+	bp->aux_priv[idx] = aux_priv;
 
 	/* From this point, all cleanup will happen via the .release callback &
 	 * any error unwinding will need to include a call to
@@ -523,14 +547,32 @@ void bnxt_rdma_aux_device_init(struct bnxt *bp)
 		goto aux_dev_uninit;
 
 	edev->ulp_tbl = ulp;
-	bp->edev = edev;
+	bp->edev[idx] = edev;
 	bnxt_set_edev_info(edev, bp);
-	bp->ulp_num_msix_want = bnxt_set_dflt_ulp_msix(bp);
+	if (idx == BNXT_AUXDEV_RDMA)
+		bp->ulp_num_msix_want = bnxt_set_dflt_ulp_msix(bp);
+	aux_priv->id = idx;
 
 	return;
 
 aux_dev_uninit:
 	auxiliary_device_uninit(aux_dev);
 exit:
-	bp->flags &= ~BNXT_FLAG_ROCE_CAP;
+	if (idx == BNXT_AUXDEV_RDMA)
+		bp->flags &= ~BNXT_FLAG_ROCE_CAP;
+}
+
+int bnxt_auxdev_id_alloc(struct bnxt *bp)
+{
+	bp->auxdev_id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL);
+	if (bp->auxdev_id < 0)
+		return bp->auxdev_id;
+
+	return 0;
+}
+
+void bnxt_auxdev_id_free(struct bnxt *bp, int id)
+{
+	if (bp->auxdev_id >= 0)
+		ida_free(&bnxt_aux_dev_ids, id);
 }
diff --git a/include/linux/bnxt/ulp.h b/include/linux/bnxt/ulp.h
index 7b9dd8ebe4bc..99cd872f6605 100644
--- a/include/linux/bnxt/ulp.h
+++ b/include/linux/bnxt/ulp.h
@@ -10,6 +10,8 @@
 #ifndef BNXT_ULP_H
 #define BNXT_ULP_H
 
+#include <linux/auxiliary_bus.h>
+
 #define BNXT_MIN_ROCE_CP_RINGS	2
 #define BNXT_MIN_ROCE_STAT_CTXS	1
 
@@ -20,6 +22,17 @@
 struct hwrm_async_event_cmpl;
 struct bnxt;
 
+enum bnxt_auxdev_type {
+	BNXT_AUXDEV_RDMA = 0,
+	__BNXT_AUXDEV_MAX
+};
+
+struct bnxt_aux_priv {
+	struct auxiliary_device aux_dev;
+	struct bnxt_en_dev *edev;
+	int id;
+};
+
 struct bnxt_msix_entry {
 	u32	vector;
 	u32	ring_idx;
@@ -116,14 +129,16 @@ void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs);
 void bnxt_ulp_irq_stop(struct bnxt *bp);
 void bnxt_ulp_irq_restart(struct bnxt *bp, int err);
 void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl);
-void bnxt_rdma_aux_device_uninit(struct bnxt *bp);
-void bnxt_rdma_aux_device_del(struct bnxt *bp);
-void bnxt_rdma_aux_device_add(struct bnxt *bp);
-void bnxt_rdma_aux_device_init(struct bnxt *bp);
+void bnxt_aux_device_uninit(struct bnxt *bp, enum bnxt_auxdev_type idx);
+void bnxt_aux_device_del(struct bnxt *bp, enum bnxt_auxdev_type idx);
+void bnxt_aux_device_add(struct bnxt *bp, enum bnxt_auxdev_type idx);
+void bnxt_aux_device_init(struct bnxt *bp, enum bnxt_auxdev_type idx);
 int bnxt_register_dev(struct bnxt_en_dev *edev, struct bnxt_ulp_ops *ulp_ops,
 		      void *handle);
 void bnxt_unregister_dev(struct bnxt_en_dev *edev);
 int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg);
 void bnxt_register_async_events(struct bnxt_en_dev *edev,
 				unsigned long *events_bmap, u16 max_id);
+int bnxt_auxdev_id_alloc(struct bnxt *bp);
+void bnxt_auxdev_id_free(struct bnxt *bp, int id);
 #endif
-- 
2.39.1

[PATCH net-next v5 3/5] bnxt_en: Create an aux device for fwctl

[Pavan Chebbi]

Create an additional auxiliary device to support fwctl.
The next patch will create bnxt_fwctl and bind to this
device.

Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  8 +++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 12 ++++++++++--
 include/linux/bnxt/ulp.h                      |  1 +
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index dcbb321cee3a..5244deb15ecc 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -16188,11 +16188,13 @@ static void bnxt_remove_one(struct pci_dev *pdev)
 		__bnxt_sriov_disable(bp);
 
 	bnxt_aux_device_del(bp, BNXT_AUXDEV_RDMA);
+	bnxt_aux_device_del(bp, BNXT_AUXDEV_FWCTL);
 
 	unregister_netdev(dev);
 	bnxt_ptp_clear(bp);
 
 	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_RDMA);
+	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_FWCTL);
 	bnxt_auxdev_id_free(bp, bp->auxdev_id);
 
 	bnxt_free_l2_filters(bp, true);
@@ -16779,8 +16781,10 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	bnxt_set_tpa_flags(bp);
 	bnxt_init_ring_params(bp);
 	bnxt_set_ring_params(bp);
-	if (!bnxt_auxdev_id_alloc(bp))
+	if (!bnxt_auxdev_id_alloc(bp)) {
 		bnxt_aux_device_init(bp, BNXT_AUXDEV_RDMA);
+		bnxt_aux_device_init(bp, BNXT_AUXDEV_FWCTL);
+	}
 	rc = bnxt_set_dflt_rings(bp, true);
 	if (rc) {
 		if (BNXT_VF(bp) && rc == -ENODEV) {
@@ -16845,6 +16849,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	bnxt_dl_fw_reporters_create(bp);
 
 	bnxt_aux_device_add(bp, BNXT_AUXDEV_RDMA);
+	bnxt_aux_device_add(bp, BNXT_AUXDEV_FWCTL);
 
 	bnxt_print_device_info(bp);
 
@@ -16853,6 +16858,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	return 0;
 init_err_cleanup:
 	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_RDMA);
+	bnxt_aux_device_uninit(bp, BNXT_AUXDEV_FWCTL);
 	bnxt_auxdev_id_free(bp, bp->auxdev_id);
 	bnxt_dl_unregister(bp);
 init_err_dl:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index e15cf4774e9b..2e06cf5e3604 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -35,6 +35,8 @@ struct bnxt_aux_device {
 
 static struct bnxt_aux_device bnxt_aux_devices[__BNXT_AUXDEV_MAX] = {{
 	.name		= "rdma",
+}, {
+	.name		= "fwctl",
 }};
 
 static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent)
@@ -262,6 +264,11 @@ void bnxt_ulp_stop(struct bnxt *bp)
 			continue;
 
 		mutex_lock(&edev->en_dev_lock);
+		if (i == BNXT_AUXDEV_FWCTL) {
+			edev->flags |= BNXT_EN_FLAG_ULP_STOPPED;
+			mutex_unlock(&edev->en_dev_lock);
+			continue;
+		}
 		if (!bnxt_ulp_registered(edev) ||
 		    (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) {
 			mutex_unlock(&edev->en_dev_lock);
@@ -298,7 +305,7 @@ void bnxt_ulp_start(struct bnxt *bp, int err)
 			return;
 
 		mutex_lock(&edev->en_dev_lock);
-		if (!bnxt_ulp_registered(edev) ||
+		if (i == BNXT_AUXDEV_FWCTL || !bnxt_ulp_registered(edev) ||
 		    !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) {
 			goto clear_flag_continue;
 		}
@@ -496,7 +503,8 @@ void bnxt_aux_device_add(struct bnxt *bp, enum bnxt_auxdev_type idx)
 	aux_dev = &bp->aux_priv[idx]->aux_dev;
 	rc = auxiliary_device_add(aux_dev);
 	if (rc) {
-		netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n");
+		netdev_warn(bp->dev, "Failed to add auxiliary device for auxdev type %d\n",
+			    idx);
 		auxiliary_device_uninit(aux_dev);
 		if (idx == BNXT_AUXDEV_RDMA)
 			bp->flags &= ~BNXT_FLAG_ROCE_CAP;
diff --git a/include/linux/bnxt/ulp.h b/include/linux/bnxt/ulp.h
index 99cd872f6605..3cd9d128e5d1 100644
--- a/include/linux/bnxt/ulp.h
+++ b/include/linux/bnxt/ulp.h
@@ -24,6 +24,7 @@ struct bnxt;
 
 enum bnxt_auxdev_type {
 	BNXT_AUXDEV_RDMA = 0,
+	BNXT_AUXDEV_FWCTL,
 	__BNXT_AUXDEV_MAX
 };
 
-- 
2.39.1

[PATCH net-next v5 4/5] bnxt_fwctl: Add bnxt fwctl device

[Pavan Chebbi]

Create bnxt_fwctl device. This will bind to bnxt's aux device.
On the upper edge, it will register with the fwctl subsystem.
It will make use of bnxt's ULP functions to send FW commands.

Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
---
 MAINTAINERS                 |   6 +
 drivers/fwctl/Kconfig       |  11 +
 drivers/fwctl/Makefile      |   1 +
 drivers/fwctl/bnxt/Makefile |   4 +
 drivers/fwctl/bnxt/main.c   | 453 ++++++++++++++++++++++++++++++++++++
 include/uapi/fwctl/bnxt.h   |  64 +++++
 include/uapi/fwctl/fwctl.h  |   1 +
 7 files changed, 540 insertions(+)
 create mode 100644 drivers/fwctl/bnxt/Makefile
 create mode 100644 drivers/fwctl/bnxt/main.c
 create mode 100644 include/uapi/fwctl/bnxt.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 3a27901781c2..a5ab77d4ea67 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10282,6 +10282,12 @@ L:	linux-kernel@vger.kernel.org
 S:	Maintained
 F:	drivers/fwctl/pds/
 
+FWCTL BNXT DRIVER
+M:	Pavan Chebbi <pavan.chebbi@broadcom.com>
+L:	linux-kernel@vger.kernel.org
+S:	Maintained
+F:	drivers/fwctl/bnxt/
+
 GALAXYCORE GC0308 CAMERA SENSOR DRIVER
 M:	Sebastian Reichel <sre@kernel.org>
 L:	linux-media@vger.kernel.org
diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig
index b5583b12a011..b3795a17f8f2 100644
--- a/drivers/fwctl/Kconfig
+++ b/drivers/fwctl/Kconfig
@@ -29,5 +29,16 @@ config FWCTL_PDS
 	  to access the debug and configuration information of the AMD/Pensando
 	  DSC hardware family.
 
+	  If you don't know what to do here, say N.
+
+config FWCTL_BNXT
+	tristate "bnxt control fwctl driver"
+	depends on BNXT
+	help
+	  BNXT provides interface for the user process to access the debug and
+	  configuration registers of the Broadcom NIC hardware family.
+	  This will allow configuration and debug tools to work out of the box on
+	  mainstream kernel.
+
 	  If you don't know what to do here, say N.
 endif
diff --git a/drivers/fwctl/Makefile b/drivers/fwctl/Makefile
index c093b5f661d6..fdd46f3a0e4e 100644
--- a/drivers/fwctl/Makefile
+++ b/drivers/fwctl/Makefile
@@ -2,5 +2,6 @@
 obj-$(CONFIG_FWCTL) += fwctl.o
 obj-$(CONFIG_FWCTL_MLX5) += mlx5/
 obj-$(CONFIG_FWCTL_PDS) += pds/
+obj-$(CONFIG_FWCTL_BNXT) += bnxt/
 
 fwctl-y += main.o
diff --git a/drivers/fwctl/bnxt/Makefile b/drivers/fwctl/bnxt/Makefile
new file mode 100644
index 000000000000..b47172761f1e
--- /dev/null
+++ b/drivers/fwctl/bnxt/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_FWCTL_BNXT) += bnxt_fwctl.o
+
+bnxt_fwctl-y += main.o
diff --git a/drivers/fwctl/bnxt/main.c b/drivers/fwctl/bnxt/main.c
new file mode 100644
index 000000000000..b31f34c1cc3c
--- /dev/null
+++ b/drivers/fwctl/bnxt/main.c
@@ -0,0 +1,453 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025, Broadcom Corporation
+ */
+
+#include <linux/kernel.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/slab.h>
+#include <linux/pci.h>
+#include <linux/fwctl.h>
+#include <uapi/fwctl/fwctl.h>
+#include <uapi/fwctl/bnxt.h>
+#include <linux/bnxt/hsi.h>
+#include <linux/bnxt/ulp.h>
+
+struct bnxtctl_uctx {
+	struct fwctl_uctx uctx;
+	u32 uctx_caps;
+};
+
+struct bnxtctl_dev {
+	struct fwctl_device fwctl;
+	struct bnxt_aux_priv *aux_priv;
+};
+
+DEFINE_FREE(bnxtctl, struct bnxtctl_dev *, if (_T) fwctl_put(&_T->fwctl))
+
+static int bnxtctl_open_uctx(struct fwctl_uctx *uctx)
+{
+	struct bnxtctl_uctx *bnxtctl_uctx =
+		container_of(uctx, struct bnxtctl_uctx, uctx);
+
+	bnxtctl_uctx->uctx_caps = BIT(FWCTL_BNXT_QUERY_COMMANDS) |
+				  BIT(FWCTL_BNXT_SEND_COMMAND);
+	return 0;
+}
+
+static void bnxtctl_close_uctx(struct fwctl_uctx *uctx)
+{
+}
+
+static void *bnxtctl_info(struct fwctl_uctx *uctx, size_t *length)
+{
+	struct bnxtctl_uctx *bnxtctl_uctx =
+		container_of(uctx, struct bnxtctl_uctx, uctx);
+	struct fwctl_info_bnxt *info;
+
+	info = kzalloc(sizeof(*info), GFP_KERNEL);
+	if (!info)
+		return ERR_PTR(-ENOMEM);
+
+	info->uctx_caps = bnxtctl_uctx->uctx_caps;
+
+	*length = sizeof(*info);
+	return info;
+}
+
+static bool bnxtctl_validate_rpc(struct bnxt_en_dev *edev,
+				 struct bnxt_fw_msg *hwrm_in,
+				 enum fwctl_rpc_scope scope)
+{
+	struct input *req = (struct input *)hwrm_in->msg;
+
+	guard(mutex)(&edev->en_dev_lock);
+	if (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)
+		return false;
+
+	switch (le16_to_cpu(req->req_type)) {
+	case HWRM_FUNC_VF_CFG:
+	case HWRM_FUNC_RESET:
+	case HWRM_FUNC_CFG:
+	case HWRM_PORT_PHY_CFG:
+	case HWRM_PORT_MAC_CFG:
+	case HWRM_PORT_CLR_STATS:
+	case HWRM_QUEUE_PRI2COS_CFG:
+	case HWRM_QUEUE_COS2BW_CFG:
+	case HWRM_QUEUE_DSCP2PRI_CFG:
+	case HWRM_QUEUE_ADPTV_QOS_RX_FEATURE_CFG:
+	case HWRM_QUEUE_ADPTV_QOS_TX_FEATURE_CFG:
+	case HWRM_QUEUE_ADPTV_QOS_RX_TUNING_CFG:
+	case HWRM_VNIC_RSS_CFG:
+	case HWRM_TUNNEL_DST_PORT_ALLOC:
+	case HWRM_TUNNEL_DST_PORT_FREE:
+	case HWRM_QUEUE_ADPTV_QOS_TX_TUNING_CFG:
+	case HWRM_PORT_TX_FIR_CFG:
+	case HWRM_FW_SET_STRUCTURED_DATA:
+	case HWRM_PORT_PRBS_TEST:
+	case HWRM_PORT_EP_TX_CFG:
+	case HWRM_CFA_REDIRECT_TUNNEL_TYPE_INFO:
+	case HWRM_CFA_FLOW_FLUSH:
+	case HWRM_CFA_L2_FILTER_ALLOC:
+	case HWRM_CFA_NTUPLE_FILTER_FREE:
+	case HWRM_CFA_REDIRECT_TUNNEL_TYPE_ALLOC:
+	case HWRM_CFA_REDIRECT_TUNNEL_TYPE_FREE:
+	case HWRM_FW_LIVEPATCH:
+	case HWRM_FW_RESET:
+	case HWRM_FW_SYNC:
+	case HWRM_FW_SET_TIME:
+	case HWRM_PORT_CFG:
+	case HWRM_FUNC_PTP_PIN_CFG:
+	case HWRM_FUNC_PTP_CFG:
+	case HWRM_FUNC_PTP_EXT_CFG:
+	case HWRM_FUNC_SYNCE_CFG:
+	case HWRM_MFG_OTP_CFG:
+	case HWRM_MFG_TESTS:
+	case HWRM_UDCC_CFG:
+	case HWRM_DBG_SERDES_TEST:
+	case HWRM_DBG_LOG_BUFFER_FLUSH:
+	case HWRM_DBG_DUMP:
+	case HWRM_DBG_ERASE_NVM:
+	case HWRM_DBG_CFG:
+	case HWRM_DBG_COREDUMP_LIST:
+	case HWRM_DBG_COREDUMP_INITIATE:
+	case HWRM_DBG_COREDUMP_RETRIEVE:
+	case HWRM_DBG_CRASHDUMP_HEADER:
+	case HWRM_DBG_CRASHDUMP_ERASE:
+	case HWRM_DBG_PTRACE:
+	case HWRM_DBG_TOKEN_CFG:
+	case HWRM_NVM_DEFRAG:
+	case HWRM_NVM_FACTORY_DEFAULTS:
+	case HWRM_NVM_FLUSH:
+	case HWRM_NVM_INSTALL_UPDATE:
+	case HWRM_NVM_MODIFY:
+	case HWRM_NVM_VERIFY_UPDATE:
+	case HWRM_NVM_ERASE_DIR_ENTRY:
+	case HWRM_NVM_MOD_DIR_ENTRY:
+	case HWRM_NVM_FIND_DIR_ENTRY:
+	case HWRM_NVM_RAW_DUMP:
+		return scope >= FWCTL_RPC_CONFIGURATION;
+
+	case HWRM_VER_GET:
+	case HWRM_FW_GET_STRUCTURED_DATA:
+	case HWRM_ERROR_RECOVERY_QCFG:
+	case HWRM_FUNC_QCAPS:
+	case HWRM_FUNC_QCFG:
+	case HWRM_FUNC_QSTATS:
+	case HWRM_PORT_QSTATS:
+	case HWRM_PORT_PHY_QCFG:
+	case HWRM_PORT_MAC_QCFG:
+	case HWRM_PORT_PHY_QCAPS:
+	case HWRM_PORT_PHY_I2C_READ:
+	case HWRM_PORT_PHY_MDIO_READ:
+	case HWRM_QUEUE_PRI2COS_QCFG:
+	case HWRM_QUEUE_COS2BW_QCFG:
+	case HWRM_QUEUE_DSCP2PRI_QCFG:
+	case HWRM_VNIC_RSS_QCFG:
+	case HWRM_QUEUE_GLOBAL_QCFG:
+	case HWRM_QUEUE_ADPTV_QOS_RX_FEATURE_QCFG:
+	case HWRM_QUEUE_ADPTV_QOS_TX_FEATURE_QCFG:
+	case HWRM_QUEUE_QCAPS:
+	case HWRM_QUEUE_ADPTV_QOS_RX_TUNING_QCFG:
+	case HWRM_QUEUE_ADPTV_QOS_TX_TUNING_QCFG:
+	case HWRM_TUNNEL_DST_PORT_QUERY:
+	case HWRM_PORT_QSTATS_EXT:
+	case HWRM_PORT_TX_FIR_QCFG:
+	case HWRM_FW_LIVEPATCH_QUERY:
+	case HWRM_FW_QSTATUS:
+	case HWRM_FW_HEALTH_CHECK:
+	case HWRM_FW_GET_TIME:
+	case HWRM_PORT_DSC_DUMP:
+	case HWRM_PORT_EP_TX_QCFG:
+	case HWRM_PORT_QCFG:
+	case HWRM_PORT_MAC_QCAPS:
+	case HWRM_TEMP_MONITOR_QUERY:
+	case HWRM_REG_POWER_QUERY:
+	case HWRM_CORE_FREQUENCY_QUERY:
+	case HWRM_STAT_QUERY_ROCE_STATS:
+	case HWRM_STAT_QUERY_ROCE_STATS_EXT:
+	case HWRM_CFA_REDIRECT_QUERY_TUNNEL_TYPE:
+	case HWRM_CFA_FLOW_INFO:
+	case HWRM_CFA_ADV_FLOW_MGNT_QCAPS:
+	case HWRM_FUNC_RESOURCE_QCAPS:
+	case HWRM_FUNC_BACKING_STORE_QCAPS:
+	case HWRM_FUNC_BACKING_STORE_QCFG:
+	case HWRM_FUNC_QSTATS_EXT:
+	case HWRM_FUNC_PTP_PIN_QCFG:
+	case HWRM_FUNC_PTP_EXT_QCFG:
+	case HWRM_FUNC_BACKING_STORE_QCFG_V2:
+	case HWRM_FUNC_BACKING_STORE_QCAPS_V2:
+	case HWRM_FUNC_SYNCE_QCFG:
+	case HWRM_FUNC_TTX_PACING_RATE_PROF_QUERY:
+	case HWRM_PCIE_QSTATS:
+	case HWRM_MFG_OTP_QCFG:
+	case HWRM_MFG_FRU_EEPROM_READ:
+	case HWRM_MFG_GET_NVM_MEASUREMENT:
+	case HWRM_STAT_GENERIC_QSTATS:
+	case HWRM_PORT_PHY_FDRSTAT:
+	case HWRM_UDCC_QCAPS:
+	case HWRM_UDCC_QCFG:
+	case HWRM_UDCC_SESSION_QCFG:
+	case HWRM_UDCC_SESSION_QUERY:
+	case HWRM_UDCC_COMP_QCFG:
+	case HWRM_UDCC_COMP_QUERY:
+	case HWRM_QUEUE_ADPTV_QOS_RX_QCFG:
+	case HWRM_QUEUE_ADPTV_QOS_TX_QCFG:
+	case HWRM_TF_RESC_USAGE_QUERY:
+	case HWRM_TFC_RESC_USAGE_QUERY:
+	case HWRM_DBG_READ_DIRECT:
+	case HWRM_DBG_READ_INDIRECT:
+	case HWRM_DBG_RING_INFO_GET:
+	case HWRM_DBG_QCAPS:
+	case HWRM_DBG_QCFG:
+	case HWRM_DBG_USEQ_FLUSH:
+	case HWRM_DBG_USEQ_QCAPS:
+	case HWRM_DBG_SIM_CABLE_STATE:
+	case HWRM_DBG_TOKEN_QUERY_AUTH_IDS:
+	case HWRM_NVM_GET_VARIABLE:
+	case HWRM_NVM_GET_DEV_INFO:
+	case HWRM_NVM_GET_DIR_ENTRIES:
+	case HWRM_NVM_GET_DIR_INFO:
+	case HWRM_NVM_READ:
+	case HWRM_SELFTEST_QLIST:
+	case HWRM_SELFTEST_RETRIEVE_SERDES_DATA:
+		return scope >= FWCTL_RPC_DEBUG_READ_ONLY;
+
+	case HWRM_PORT_PHY_I2C_WRITE:
+	case HWRM_MFG_FRU_WRITE_CONTROL:
+	case HWRM_MFG_FRU_EEPROM_WRITE:
+	case HWRM_DBG_WRITE_DIRECT:
+	case HWRM_NVM_SET_VARIABLE:
+	case HWRM_NVM_WRITE:
+	case HWRM_NVM_RAW_WRITE_BLK:
+	case HWRM_PORT_PHY_MDIO_WRITE:
+		return scope >= FWCTL_RPC_DEBUG_WRITE;
+
+	default:
+		return false;
+	}
+}
+
+static int bnxt_fw_setup_input_dma(struct bnxtctl_dev *bnxt_dev,
+				   struct device *dev,
+				   struct fwctl_dma_info_bnxt *msg,
+				   struct bnxt_fw_msg *fw_msg,
+				   int num_dma,
+				   void **dma_virt_addr,
+				   dma_addr_t *dma_addr)
+{
+	u8 i, num_allocated = 0;
+	void *dma_ptr;
+	int rc;
+
+	for (i = 0; i < num_dma; i++) {
+		if (msg->len == 0 || msg->len > MAX_DMA_MEM_SIZE) {
+			rc = -EINVAL;
+			goto err;
+		}
+		dma_virt_addr[i] = dma_alloc_coherent(dev->parent, msg->len,
+						      &dma_addr[i], GFP_KERNEL);
+		if (!dma_virt_addr[i]) {
+			rc = -ENOMEM;
+			goto err;
+		}
+		num_allocated++;
+		if (msg->dma_direction == DEVICE_WRITE) {
+			if (copy_from_user(dma_virt_addr[i],
+					   u64_to_user_ptr(msg->data),
+					   msg->len)) {
+				rc = -EFAULT;
+				goto err;
+			}
+		}
+		dma_ptr = fw_msg->msg + msg->offset;
+
+		if ((PTR_ALIGN(dma_ptr, 8) == dma_ptr) &&
+		    msg->offset < fw_msg->msg_len) {
+			__le64 *dmap = dma_ptr;
+
+			*dmap = cpu_to_le64(dma_addr[i]);
+		} else {
+			rc = -EINVAL;
+			goto err;
+		}
+		msg += 1;
+	}
+
+	return 0;
+err:
+	for (i = 0; i < num_allocated; i++)
+		dma_free_coherent(dev->parent, msg->len, dma_virt_addr[i],
+				  dma_addr[i]);
+
+	return rc;
+}
+
+static void *bnxtctl_fw_rpc(struct fwctl_uctx *uctx,
+			    enum fwctl_rpc_scope scope,
+			    void *in, size_t in_len, size_t *out_len)
+{
+	struct bnxtctl_dev *bnxtctl =
+		container_of(uctx->fwctl, struct bnxtctl_dev, fwctl);
+	struct bnxt_aux_priv *bnxt_aux_priv = bnxtctl->aux_priv;
+	void *dma_virt_addr[MAX_NUM_DMA_INDICATIONS];
+	dma_addr_t dma_addr[MAX_NUM_DMA_INDICATIONS];
+	struct fwctl_dma_info_bnxt *dma_buf = NULL;
+	struct device *dev = &uctx->fwctl->dev;
+	struct fwctl_rpc_bnxt *msg = in;
+	struct bnxt_fw_msg rpc_in;
+	int i, rc, err = 0;
+
+	rpc_in.msg = memdup_user(u64_to_user_ptr(msg->req), msg->req_len);
+	if (IS_ERR(rpc_in.msg))
+		return rpc_in.msg;
+
+	if (!bnxtctl_validate_rpc(bnxt_aux_priv->edev, &rpc_in, scope)) {
+		err = -EPERM;
+		goto free_msg_out;
+	}
+
+	rpc_in.msg_len = msg->req_len;
+	rpc_in.resp = kzalloc(*out_len, GFP_KERNEL);
+	if (!rpc_in.resp) {
+		err = -ENOMEM;
+		goto free_msg_out;
+	}
+
+	rpc_in.resp_max_len = *out_len;
+	if (!msg->timeout)
+		rpc_in.timeout = DFLT_HWRM_CMD_TIMEOUT;
+	else
+		rpc_in.timeout = msg->timeout;
+
+	if (msg->num_dma) {
+		if (msg->num_dma > MAX_NUM_DMA_INDICATIONS) {
+			dev_err(dev, "DMA buffers exceed the number supported\n");
+			err = -EINVAL;
+			goto free_msg_out;
+		}
+
+		dma_buf = kcalloc(msg->num_dma, sizeof(*dma_buf), GFP_KERNEL);
+		if (!dma_buf) {
+			err = -ENOMEM;
+			goto free_msg_out;
+		}
+
+		if (copy_from_user(dma_buf, u64_to_user_ptr(msg->payload),
+				   msg->num_dma * sizeof(*dma_buf))) {
+			dev_dbg(dev, "Failed to copy payload from user\n");
+			err = -EFAULT;
+			goto free_dmabuf_out;
+		}
+
+		err = bnxt_fw_setup_input_dma(bnxtctl, dev, dma_buf, &rpc_in,
+					      msg->num_dma, &dma_virt_addr[0],
+					      &dma_addr[0]);
+		if (err)
+			goto free_dmabuf_out;
+	}
+
+	rc = bnxt_send_msg(bnxt_aux_priv->edev, &rpc_in);
+	if (rc) {
+		struct output *resp = rpc_in.resp;
+
+		/* Copy the response to user always, as it contains
+		 * detailed status of the command failure
+		 */
+		if (!resp->error_code)
+			/* bnxt_send_msg() returned much before FW
+			 * received the command.
+			 */
+			resp->error_code = rc;
+
+		goto free_dma_out;
+	}
+
+	for (i = 0; i < msg->num_dma; i++) {
+		if (dma_buf[i].dma_direction == DEVICE_READ) {
+			if (copy_to_user(u64_to_user_ptr(dma_buf[i].data),
+					 dma_virt_addr[i],
+					 dma_buf[i].len)) {
+				dev_dbg(dev, "Failed to copy resp to user\n");
+				err = -EFAULT;
+				break;
+			}
+		}
+	}
+free_dma_out:
+	for (i = 0; i < msg->num_dma; i++)
+		dma_free_coherent(dev->parent, dma_buf[i].len, dma_virt_addr[i],
+				  dma_addr[i]);
+free_dmabuf_out:
+	kfree(dma_buf);
+free_msg_out:
+	kfree(rpc_in.msg);
+
+	if (err) {
+		kfree(rpc_in.resp);
+		return ERR_PTR(err);
+	}
+
+	return rpc_in.resp;
+}
+
+static const struct fwctl_ops bnxtctl_ops = {
+	.device_type = FWCTL_DEVICE_TYPE_BNXT,
+	.uctx_size = sizeof(struct bnxtctl_uctx),
+	.open_uctx = bnxtctl_open_uctx,
+	.close_uctx = bnxtctl_close_uctx,
+	.info = bnxtctl_info,
+	.fw_rpc = bnxtctl_fw_rpc,
+};
+
+static int bnxtctl_probe(struct auxiliary_device *adev,
+			 const struct auxiliary_device_id *id)
+{
+	struct bnxt_aux_priv *aux_priv =
+		container_of(adev, struct bnxt_aux_priv, aux_dev);
+	struct bnxtctl_dev *bnxtctl __free(bnxtctl) =
+		fwctl_alloc_device(&aux_priv->edev->pdev->dev, &bnxtctl_ops,
+				   struct bnxtctl_dev, fwctl);
+	int rc;
+
+	if (!bnxtctl)
+		return -ENOMEM;
+
+	bnxtctl->aux_priv = aux_priv;
+
+	rc = fwctl_register(&bnxtctl->fwctl);
+	if (rc)
+		return rc;
+
+	auxiliary_set_drvdata(adev, no_free_ptr(bnxtctl));
+	return 0;
+}
+
+static void bnxtctl_remove(struct auxiliary_device *adev)
+{
+	struct bnxtctl_dev *ctldev = auxiliary_get_drvdata(adev);
+
+	fwctl_unregister(&ctldev->fwctl);
+	fwctl_put(&ctldev->fwctl);
+}
+
+static const struct auxiliary_device_id bnxtctl_id_table[] = {
+	{ .name = "bnxt_en.fwctl", },
+	{}
+};
+MODULE_DEVICE_TABLE(auxiliary, bnxtctl_id_table);
+
+static struct auxiliary_driver bnxtctl_driver = {
+	.name = "bnxt_fwctl",
+	.probe = bnxtctl_probe,
+	.remove = bnxtctl_remove,
+	.id_table = bnxtctl_id_table,
+};
+
+module_auxiliary_driver(bnxtctl_driver);
+
+MODULE_IMPORT_NS("FWCTL");
+MODULE_DESCRIPTION("BNXT fwctl driver");
+MODULE_AUTHOR("Pavan Chebbi <pavan.chebbi@broadcom.com>");
+MODULE_AUTHOR("Andy Gospodarek <gospo@broadcom.com>");
+MODULE_LICENSE("GPL");
diff --git a/include/uapi/fwctl/bnxt.h b/include/uapi/fwctl/bnxt.h
new file mode 100644
index 000000000000..0fb20972b476
--- /dev/null
+++ b/include/uapi/fwctl/bnxt.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2025, Broadcom Inc
+ */
+
+#ifndef _UAPI_FWCTL_BNXT_H_
+#define _UAPI_FWCTL_BNXT_H_
+
+#include <linux/types.h>
+
+#define MAX_DMA_MEM_SIZE		0x10000 /*64K*/
+#define DFLT_HWRM_CMD_TIMEOUT		500
+#define DEVICE_WRITE			0
+#define DEVICE_READ			1
+
+enum fwctl_bnxt_commands {
+	FWCTL_BNXT_QUERY_COMMANDS = 0,
+	FWCTL_BNXT_SEND_COMMAND,
+};
+
+/**
+ * struct fwctl_info_bnxt - ioctl(FWCTL_INFO) out_device_data
+ * @uctx_caps: The command capabilities driver accepts.
+ *
+ * Return basic information about the FW interface available.
+ */
+struct fwctl_info_bnxt {
+	__u32 uctx_caps;
+};
+
+#define MAX_NUM_DMA_INDICATIONS 10
+
+/**
+ * struct fwctl_dma_info_bnxt - describe the buffer that should be DMAed
+ * @data: DMA-intended buffer
+ * @len: length of the @data
+ * @offset: offset at which FW (HWRM) input structure needs DMA address
+ * @dma_direction: DMA direction, DEVICE_READ or DEVICE_WRITE
+ * @unused: pad
+ */
+struct fwctl_dma_info_bnxt {
+	__aligned_u64 data;
+	__u32 len;
+	__u16 offset;
+	__u8 dma_direction;
+	__u8 unused;
+};
+
+/**
+ * struct fwctl_rpc_bnxt - describe the fwctl message for bnxt
+ * @req: FW (HWRM) command input structure
+ * @req_len: length of @req
+ * @timeout: if the user wants to override the driver's default, 0 otherwise
+ * @num_dma: number of DMA buffers to be added to @req
+ * @payload: DMA buffer details in struct fwctl_dma_info_bnxt format
+ */
+struct fwctl_rpc_bnxt {
+	__aligned_u64 req;
+	__u32 req_len;
+	__u32 timeout;
+	__u32 num_dma;
+	__aligned_u64 payload;
+};
+#endif
diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h
index 716ac0eee42d..2d6d4049c205 100644
--- a/include/uapi/fwctl/fwctl.h
+++ b/include/uapi/fwctl/fwctl.h
@@ -44,6 +44,7 @@ enum fwctl_device_type {
 	FWCTL_DEVICE_TYPE_ERROR = 0,
 	FWCTL_DEVICE_TYPE_MLX5 = 1,
 	FWCTL_DEVICE_TYPE_CXL = 2,
+	FWCTL_DEVICE_TYPE_BNXT = 3,
 	FWCTL_DEVICE_TYPE_PDS = 4,
 };
 
-- 
2.39.1

[PATCH net-next v5 5/5] bnxt_fwctl: Add documentation entries

[Pavan Chebbi]

Add bnxt_fwctl to the driver and fwctl documentation pages.

Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
---
 .../userspace-api/fwctl/bnxt_fwctl.rst        | 78 +++++++++++++++++++
 Documentation/userspace-api/fwctl/fwctl.rst   |  1 +
 Documentation/userspace-api/fwctl/index.rst   |  1 +
 3 files changed, 80 insertions(+)
 create mode 100644 Documentation/userspace-api/fwctl/bnxt_fwctl.rst

diff --git a/Documentation/userspace-api/fwctl/bnxt_fwctl.rst b/Documentation/userspace-api/fwctl/bnxt_fwctl.rst
new file mode 100644
index 000000000000..cbf6be4410cc
--- /dev/null
+++ b/Documentation/userspace-api/fwctl/bnxt_fwctl.rst
@@ -0,0 +1,78 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+fwctl bnxt driver
+=================
+
+:Author: Pavan Chebbi
+
+Overview
+========
+
+BNXT driver makes a fwctl service available through an auxiliary_device.
+The bnxt_fwctl driver binds to this device and registers itself with the
+fwctl subsystem.
+
+The bnxt_fwctl driver is agnostic to the device firmware internals. It
+uses the Upper Layer Protocol (ULP) conduit provided by bnxt to send
+HardWare Resource Manager (HWRM) commands to firmware.
+
+These commands can query or change firmware driven device configurations
+and read/write registers that are useful for debugging.
+
+bnxt_fwctl User API
+===================
+
+Each RPC request contains a message request structure (HWRM input),
+its length, optional request timeout, and dma buffers' information
+if the command needs any DMA. The request is then put together with
+the request data and sent through bnxt's message queue to the firmware,
+and the results are returned to the caller.
+
+A typical user application can send a FWCTL_INFO command using ioctl()
+to discover bnxt_fwctl's RPC capabilities as shown below:
+
+        ioctl(fd, FWCTL_INFO, &fwctl_info_msg);
+
+where fwctl_info_msg (of type struct fwctl_info) describes bnxt_info_msg
+(of type struct fwctl_info_bnxt). fwctl_info_msg is set up as follows:
+
+        size = sizeof(struct fwctl_info);
+        flags = 0;
+        device_data_len = sizeof(bnxt_info_msg);
+        out_device_data = (__aligned_u64)&bnxt_info_msg;
+
+The uctx_caps of bnxt_info_msg represents the capabilities as described
+in fwctl_bnxt_commands of include/uapi/fwctl/bnxt.h
+
+The FW RPC itself, FWCTL_RPC can be sent using ioctl() as:
+
+        ioctl(fd, FWCTL_RPC, &fwctl_rpc_msg);
+
+where fwctl_rpc_msg (of type struct fwctl_rpc) encapsulates fwctl_rpc_bnxt
+(see bnxt_rpc_msg below). fwctl_rpc_bnxt members are set up as per the
+requirements of specific HWRM commands described in include/bnxt/hsi.h.
+An example for HWRM_VER_GET is shown below:
+
+        struct fwctl_rpc_bnxt bnxt_rpc_msg;
+        struct hwrm_ver_get_output resp;
+        struct fwctl_rpc fwctl_rpc_msg;
+        struct hwrm_ver_get_input req;
+
+        req.req_type = HWRM_VER_GET;
+        req.hwrm_intf_maj = HWRM_VERSION_MAJOR;
+        req.hwrm_intf_min = HWRM_VERSION_MINOR;
+        req.hwrm_intf_upd = HWRM_VERSION_UPDATE;
+        req.cmpl_ring = -1;
+        req.target_id = -1;
+
+        bnxt_rpc_msg.req_len = sizeof(struct hwrm_ver_get_input);
+        bnxt_rpc_msg.num_dma = 0;
+        bnxt_rpc_msg.req = (__aligned_u64)&req;
+
+        fwctl_rpc_msg.size = sizeof(struct fwctl_rpc);
+        fwctl_rpc_msg.scope = FWCTL_RPC_DEBUG_READ_ONLY;
+        fwctl_rpc_msg.in_len = sizeof(bnxt_rpc_msg) + sizeof(req);
+        fwctl_rpc_msg.out_len = sizeof(struct hwrm_ver_get_output);
+        fwctl_rpc_msg.in = (__aligned_u64)&bnxt_rpc_msg;
+        fwctl_rpc_msg.out = (__aligned_u64)&resp;
diff --git a/Documentation/userspace-api/fwctl/fwctl.rst b/Documentation/userspace-api/fwctl/fwctl.rst
index a74eab8d14c6..826817bfd54d 100644
--- a/Documentation/userspace-api/fwctl/fwctl.rst
+++ b/Documentation/userspace-api/fwctl/fwctl.rst
@@ -148,6 +148,7 @@ area resulting in clashes will be resolved in favour of a kernel implementation.
 fwctl User API
 ==============
 
+.. kernel-doc:: include/uapi/fwctl/bnxt.h
 .. kernel-doc:: include/uapi/fwctl/fwctl.h
 .. kernel-doc:: include/uapi/fwctl/mlx5.h
 .. kernel-doc:: include/uapi/fwctl/pds.h
diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst
index 316ac456ad3b..8062f7629654 100644
--- a/Documentation/userspace-api/fwctl/index.rst
+++ b/Documentation/userspace-api/fwctl/index.rst
@@ -10,5 +10,6 @@ to securely construct and execute RPCs inside device firmware.
    :maxdepth: 1
 
    fwctl
+   bnxt_fwctl
    fwctl-cxl
    pds_fwctl
-- 
2.39.1

Re: [PATCH net-next v5 4/5] bnxt_fwctl: Add bnxt fwctl device

[Jakub Kicinski]

On Tue, 14 Oct 2025 01:10:32 -0700 Pavan Chebbi wrote:
> +	case HWRM_VNIC_RSS_CFG:

Gotta be able to configure basic netdev functions for complex debug!
Or whatever the pretend reasons for fwctl were.

And AFAIK all Broadcom NIC tooling is aggressively closed source.

Frankly I have no idea what the rules are supposed to be for merging
code in this joke of a "subsystem". Please stop tagging this with
net-next, it's definitely no going via our tree.

Re: [PATCH net-next v5 2/5] bnxt_en: Refactor aux bus functions to be more generic

[Leon Romanovsky]

On Tue, Oct 14, 2025 at 01:10:30AM -0700, Pavan Chebbi wrote:
> Up until now there was only one auxiliary device that bnxt
> created and that was for RoCE driver. bnxt fwctl is also
> going to use an aux bus device that bnxt should create.
> This requires some nomenclature changes and refactoring of
> the existing bnxt aux dev functions.
> 
> Convert 'aux_priv' and 'edev' members of struct bnxt into
> arrays where each element contains supported auxbus device's
> data. Move struct bnxt_aux_priv from bnxt.h to ulp.h because
> that is where it belongs. Make aux bus init/uninit/add/del
> functions more generic which will accept aux device type as
> a parameter. Make bnxt_ulp_start/stop functions (the only
> other common functions applicable to any aux device) loop
> through the aux devices to update their config and states.
> 
> Also, as an improvement in code, bnxt_register_dev() can skip
> unnecessary dereferencing of edev from bp, instead use the
> edev pointer from the function parameter.
> 
> Future patches will reuse these functions to add an aux bus
> device for fwctl.
> 
> Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
> ---
>  drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  29 ++-
>  drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  13 +-
>  .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |   2 +-
>  drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 238 ++++++++++--------
>  include/linux/bnxt/ulp.h                      |  23 +-
>  5 files changed, 181 insertions(+), 124 deletions(-)

<...>

> -void bnxt_rdma_aux_device_uninit(struct bnxt *bp)
> +void bnxt_aux_device_uninit(struct bnxt *bp, enum bnxt_auxdev_type idx)
>  {
>  	struct bnxt_aux_priv *aux_priv;
>  	struct auxiliary_device *adev;
>  
>  	/* Skip if no auxiliary device init was done. */
> -	if (!bp->aux_priv)
> +	if (!bp->aux_priv[idx])
>  		return;

<...>

> -void bnxt_rdma_aux_device_del(struct bnxt *bp)
> +void bnxt_aux_device_del(struct bnxt *bp, enum bnxt_auxdev_type idx)
>  {
> -	if (!bp->edev)
> +	if (!bp->edev[idx])
>  		return;

You are not supposed to call these functions if you didn't initialize
auxdev for this idx first. Please don't use defensive programming style
for in-kernel API.

Thanks

Re: [PATCH net-next v5 4/5] bnxt_fwctl: Add bnxt fwctl device

[Jonathan Cameron]

On Tue, 14 Oct 2025 01:10:32 -0700
Pavan Chebbi <pavan.chebbi@broadcom.com> wrote:

> Create bnxt_fwctl device. This will bind to bnxt's aux device.
> On the upper edge, it will register with the fwctl subsystem.
> It will make use of bnxt's ULP functions to send FW commands.
> 
> Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
A few things inline.


> diff --git a/drivers/fwctl/bnxt/main.c b/drivers/fwctl/bnxt/main.c
> new file mode 100644
> index 000000000000..b31f34c1cc3c
> --- /dev/null
> +++ b/drivers/fwctl/bnxt/main.c
> @@ -0,0 +1,453 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025, Broadcom Corporation
> + */
> +
> +#include <linux/kernel.h>

Anything actually in kernel.h used in here?  There is a considerable
effort going on to not include that unless absolutely necessary. Instead
figure out which actual headers are needed.  In general follow 
include what you use (IWYU) principles.  A few things obviously missing
are cleanup.h and mutex.h for the guard(mutex). 


> +#include <linux/auxiliary_bus.h>
Pick an order.
> +#include <linux/slab.h>
> +#include <linux/pci.h>
> +#include <linux/fwctl.h>
> +#include <uapi/fwctl/fwctl.h>
> +#include <uapi/fwctl/bnxt.h>

I'd put the uapi after, but not sure if this is a convention for some kernel code.

> +#include <linux/bnxt/hsi.h>
> +#include <linux/bnxt/ulp.h>


> +static int bnxt_fw_setup_input_dma(struct bnxtctl_dev *bnxt_dev,
> +				   struct device *dev,
> +				   struct fwctl_dma_info_bnxt *msg,
> +				   struct bnxt_fw_msg *fw_msg,
> +				   int num_dma,
> +				   void **dma_virt_addr,
> +				   dma_addr_t *dma_addr)
> +{
> +	u8 i, num_allocated = 0;
> +	void *dma_ptr;
> +	int rc;
> +
> +	for (i = 0; i < num_dma; i++) {
> +		if (msg->len == 0 || msg->len > MAX_DMA_MEM_SIZE) {
> +			rc = -EINVAL;
> +			goto err;
> +		}
> +		dma_virt_addr[i] = dma_alloc_coherent(dev->parent, msg->len,
> +						      &dma_addr[i], GFP_KERNEL);
> +		if (!dma_virt_addr[i]) {
> +			rc = -ENOMEM;
> +			goto err;
> +		}
> +		num_allocated++;
> +		if (msg->dma_direction == DEVICE_WRITE) {
> +			if (copy_from_user(dma_virt_addr[i],
> +					   u64_to_user_ptr(msg->data),
> +					   msg->len)) {
> +				rc = -EFAULT;
> +				goto err;
> +			}
> +		}
> +		dma_ptr = fw_msg->msg + msg->offset;
> +
> +		if ((PTR_ALIGN(dma_ptr, 8) == dma_ptr) &&
> +		    msg->offset < fw_msg->msg_len) {
> +			__le64 *dmap = dma_ptr;
> +
> +			*dmap = cpu_to_le64(dma_addr[i]);
> +		} else {
I would flip the error condition so that only that is out of line as
generally that makes for easier flow.

		if (!(PTR_ALIGN(dma_ptr, 8) == dma_ptr) ||
		      msg->offset >= fw_msg->msg_len)) {
			rc = -EINVAL;
			goto err;
		}

		*(__le64)(dmap_ptr) = cpu_to_le64(dma_addr[i]);

or something like that.

> +			rc = -EINVAL;
> +			goto err;
> +		}
> +		msg += 1;
		msg++;
or do that in the loop update.


> +	}
> +
> +	return 0;
> +err:
> +	for (i = 0; i < num_allocated; i++)
> +		dma_free_coherent(dev->parent, msg->len, dma_virt_addr[i],
> +				  dma_addr[i]);
> +
> +	return rc;
> +}
> +
> +static void *bnxtctl_fw_rpc(struct fwctl_uctx *uctx,
> +			    enum fwctl_rpc_scope scope,
> +			    void *in, size_t in_len, size_t *out_len)
> +{
> +	struct bnxtctl_dev *bnxtctl =
> +		container_of(uctx->fwctl, struct bnxtctl_dev, fwctl);
> +	struct bnxt_aux_priv *bnxt_aux_priv = bnxtctl->aux_priv;
> +	void *dma_virt_addr[MAX_NUM_DMA_INDICATIONS];
> +	dma_addr_t dma_addr[MAX_NUM_DMA_INDICATIONS];
> +	struct fwctl_dma_info_bnxt *dma_buf = NULL;
> +	struct device *dev = &uctx->fwctl->dev;
> +	struct fwctl_rpc_bnxt *msg = in;
> +	struct bnxt_fw_msg rpc_in;
> +	int i, rc, err = 0;
> +
> +	rpc_in.msg = memdup_user(u64_to_user_ptr(msg->req), msg->req_len);
> +	if (IS_ERR(rpc_in.msg))
> +		return rpc_in.msg;
> +
> +	if (!bnxtctl_validate_rpc(bnxt_aux_priv->edev, &rpc_in, scope)) {
> +		err = -EPERM;
> +		goto free_msg_out;

If you follow this path, you will free rpc_in.resp which hasn't
been allocated yet.  Likely not a bug, but definitely a problem for
easy understanding. 

I can understand why you want to keep the if (err) freeing of rpc_in.resp
for the end of the exit path, but if so move 
	rpc_in.resp = kzalloc();
to be the first thing we need to unwind, before rpc_in.msg is allocated above.


> +	}
> +
> +	rpc_in.msg_len = msg->req_len;
> +	rpc_in.resp = kzalloc(*out_len, GFP_KERNEL);
> +	if (!rpc_in.resp) {
> +		err = -ENOMEM;
> +		goto free_msg_out;
> +	}
> +
> +	rpc_in.resp_max_len = *out_len;
> +	if (!msg->timeout)
> +		rpc_in.timeout = DFLT_HWRM_CMD_TIMEOUT;
> +	else
> +		rpc_in.timeout = msg->timeout;
> +
> +	if (msg->num_dma) {
> +		if (msg->num_dma > MAX_NUM_DMA_INDICATIONS) {
> +			dev_err(dev, "DMA buffers exceed the number supported\n");
> +			err = -EINVAL;
> +			goto free_msg_out;
> +		}
> +
> +		dma_buf = kcalloc(msg->num_dma, sizeof(*dma_buf), GFP_KERNEL);
> +		if (!dma_buf) {
> +			err = -ENOMEM;
> +			goto free_msg_out;
> +		}
> +
> +		if (copy_from_user(dma_buf, u64_to_user_ptr(msg->payload),
> +				   msg->num_dma * sizeof(*dma_buf))) {
> +			dev_dbg(dev, "Failed to copy payload from user\n");
> +			err = -EFAULT;
> +			goto free_dmabuf_out;
> +		}
> +
> +		err = bnxt_fw_setup_input_dma(bnxtctl, dev, dma_buf, &rpc_in,
> +					      msg->num_dma, &dma_virt_addr[0],
> +					      &dma_addr[0]);
> +		if (err)
> +			goto free_dmabuf_out;
> +	}
> +
> +	rc = bnxt_send_msg(bnxt_aux_priv->edev, &rpc_in);
> +	if (rc) {
> +		struct output *resp = rpc_in.resp;
> +
> +		/* Copy the response to user always, as it contains
> +		 * detailed status of the command failure
> +		 */
> +		if (!resp->error_code)
> +			/* bnxt_send_msg() returned much before FW
> +			 * received the command.
> +			 */
> +			resp->error_code = rc;
> +
> +		goto free_dma_out;
> +	}
> +
> +	for (i = 0; i < msg->num_dma; i++) {
> +		if (dma_buf[i].dma_direction == DEVICE_READ) {

Consider flipping this to reduce indent and slightly improve readability.

		if (dma_buf[i].dma_direction != DEVICE_READ)
			continue;

		if (copy_to_user(...)

> +			if (copy_to_user(u64_to_user_ptr(dma_buf[i].data),
> +					 dma_virt_addr[i],
> +					 dma_buf[i].len)) {
> +				dev_dbg(dev, "Failed to copy resp to user\n");
> +				err = -EFAULT;
> +				break;
> +			}
> +		}
> +	}
> +free_dma_out:
> +	for (i = 0; i < msg->num_dma; i++)
> +		dma_free_coherent(dev->parent, dma_buf[i].len, dma_virt_addr[i],
> +				  dma_addr[i]);

I'd prefer a little helper function with this to make it clear it's
underdoing stuff in bnxt_fw_setup_input_dma()  Probably only call that
if (msg->num_dma)
to simplify reasoning on whether this is always safe or not.

> +free_dmabuf_out:
> +	kfree(dma_buf);
> +free_msg_out:
> +	kfree(rpc_in.msg);
> +
> +	if (err) {
> +		kfree(rpc_in.resp);
> +		return ERR_PTR(err);
> +	}
> +
> +	return rpc_in.resp;
> +}



> +static const struct auxiliary_device_id bnxtctl_id_table[] = {
> +	{ .name = "bnxt_en.fwctl", },
> +	{}
> +};
> +MODULE_DEVICE_TABLE(auxiliary, bnxtctl_id_table);
> +
> +static struct auxiliary_driver bnxtctl_driver = {
> +	.name = "bnxt_fwctl",
> +	.probe = bnxtctl_probe,
> +	.remove = bnxtctl_remove,
> +	.id_table = bnxtctl_id_table,
> +};
> +
> +module_auxiliary_driver(bnxtctl_driver);
> +
> +MODULE_IMPORT_NS("FWCTL");
> +MODULE_DESCRIPTION("BNXT fwctl driver");
> +MODULE_AUTHOR("Pavan Chebbi <pavan.chebbi@broadcom.com>");
> +MODULE_AUTHOR("Andy Gospodarek <gospo@broadcom.com>");
> +MODULE_LICENSE("GPL");

Re: [PATCH net-next v5 5/5] bnxt_fwctl: Add documentation entries

[Jonathan Cameron]

On Tue, 14 Oct 2025 01:10:33 -0700
Pavan Chebbi <pavan.chebbi@broadcom.com> wrote:

> Add bnxt_fwctl to the driver and fwctl documentation pages.
> 
> Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>

Would be useful to provide a reference to userspace code that is
making use of this.

Jason / others, did we ever get the central repo for user space code
set up?


> ---
>  .../userspace-api/fwctl/bnxt_fwctl.rst        | 78 +++++++++++++++++++
>  Documentation/userspace-api/fwctl/fwctl.rst   |  1 +
>  Documentation/userspace-api/fwctl/index.rst   |  1 +
>  3 files changed, 80 insertions(+)
>  create mode 100644 Documentation/userspace-api/fwctl/bnxt_fwctl.rst
> 
> diff --git a/Documentation/userspace-api/fwctl/bnxt_fwctl.rst b/Documentation/userspace-api/fwctl/bnxt_fwctl.rst
> new file mode 100644
> index 000000000000..cbf6be4410cc
> --- /dev/null
> +++ b/Documentation/userspace-api/fwctl/bnxt_fwctl.rst
> @@ -0,0 +1,78 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=================
> +fwctl bnxt driver
> +=================
> +
> +:Author: Pavan Chebbi
> +
> +Overview
> +========
> +
> +BNXT driver makes a fwctl service available through an auxiliary_device.
> +The bnxt_fwctl driver binds to this device and registers itself with the
> +fwctl subsystem.
> +
> +The bnxt_fwctl driver is agnostic to the device firmware internals. It
> +uses the Upper Layer Protocol (ULP) conduit provided by bnxt to send
> +HardWare Resource Manager (HWRM) commands to firmware.
> +
> +These commands can query or change firmware driven device configurations
> +and read/write registers that are useful for debugging.
> +
> +bnxt_fwctl User API
> +===================
> +
> +Each RPC request contains a message request structure (HWRM input),
> +its length, optional request timeout, and dma buffers' information
> +if the command needs any DMA. The request is then put together with
> +the request data and sent through bnxt's message queue to the firmware,
> +and the results are returned to the caller.
> +
> +A typical user application can send a FWCTL_INFO command using ioctl()
> +to discover bnxt_fwctl's RPC capabilities as shown below:
> +
> +        ioctl(fd, FWCTL_INFO, &fwctl_info_msg);
> +
> +where fwctl_info_msg (of type struct fwctl_info) describes bnxt_info_msg
> +(of type struct fwctl_info_bnxt). fwctl_info_msg is set up as follows:
> +
> +        size = sizeof(struct fwctl_info);
> +        flags = 0;
> +        device_data_len = sizeof(bnxt_info_msg);
> +        out_device_data = (__aligned_u64)&bnxt_info_msg;
> +
> +The uctx_caps of bnxt_info_msg represents the capabilities as described
> +in fwctl_bnxt_commands of include/uapi/fwctl/bnxt.h
> +
> +The FW RPC itself, FWCTL_RPC can be sent using ioctl() as:
> +
> +        ioctl(fd, FWCTL_RPC, &fwctl_rpc_msg);
> +
> +where fwctl_rpc_msg (of type struct fwctl_rpc) encapsulates fwctl_rpc_bnxt
> +(see bnxt_rpc_msg below). fwctl_rpc_bnxt members are set up as per the
> +requirements of specific HWRM commands described in include/bnxt/hsi.h.
> +An example for HWRM_VER_GET is shown below:
> +
> +        struct fwctl_rpc_bnxt bnxt_rpc_msg;
> +        struct hwrm_ver_get_output resp;
> +        struct fwctl_rpc fwctl_rpc_msg;
> +        struct hwrm_ver_get_input req;
> +
> +        req.req_type = HWRM_VER_GET;
> +        req.hwrm_intf_maj = HWRM_VERSION_MAJOR;
> +        req.hwrm_intf_min = HWRM_VERSION_MINOR;
> +        req.hwrm_intf_upd = HWRM_VERSION_UPDATE;
> +        req.cmpl_ring = -1;
> +        req.target_id = -1;
> +
> +        bnxt_rpc_msg.req_len = sizeof(struct hwrm_ver_get_input);
> +        bnxt_rpc_msg.num_dma = 0;
> +        bnxt_rpc_msg.req = (__aligned_u64)&req;
> +
> +        fwctl_rpc_msg.size = sizeof(struct fwctl_rpc);
> +        fwctl_rpc_msg.scope = FWCTL_RPC_DEBUG_READ_ONLY;
> +        fwctl_rpc_msg.in_len = sizeof(bnxt_rpc_msg) + sizeof(req);
> +        fwctl_rpc_msg.out_len = sizeof(struct hwrm_ver_get_output);
> +        fwctl_rpc_msg.in = (__aligned_u64)&bnxt_rpc_msg;
> +        fwctl_rpc_msg.out = (__aligned_u64)&resp;
> diff --git a/Documentation/userspace-api/fwctl/fwctl.rst b/Documentation/userspace-api/fwctl/fwctl.rst
> index a74eab8d14c6..826817bfd54d 100644
> --- a/Documentation/userspace-api/fwctl/fwctl.rst
> +++ b/Documentation/userspace-api/fwctl/fwctl.rst
> @@ -148,6 +148,7 @@ area resulting in clashes will be resolved in favour of a kernel implementation.
>  fwctl User API
>  ==============
>  
> +.. kernel-doc:: include/uapi/fwctl/bnxt.h
>  .. kernel-doc:: include/uapi/fwctl/fwctl.h
>  .. kernel-doc:: include/uapi/fwctl/mlx5.h
>  .. kernel-doc:: include/uapi/fwctl/pds.h
> diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst
> index 316ac456ad3b..8062f7629654 100644
> --- a/Documentation/userspace-api/fwctl/index.rst
> +++ b/Documentation/userspace-api/fwctl/index.rst
> @@ -10,5 +10,6 @@ to securely construct and execute RPCs inside device firmware.
>     :maxdepth: 1
>  
>     fwctl
> +   bnxt_fwctl
>     fwctl-cxl
>     pds_fwctl

Re: [PATCH net-next v5 5/5] bnxt_fwctl: Add documentation entries

[Jason Gunthorpe]

On Tue, Oct 28, 2025 at 04:46:51PM +0000, Jonathan Cameron wrote:
> On Tue, 14 Oct 2025 01:10:33 -0700
> Pavan Chebbi <pavan.chebbi@broadcom.com> wrote:
> 
> > Add bnxt_fwctl to the driver and fwctl documentation pages.
> > 
> > Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> > Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> > Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
> 
> Would be useful to provide a reference to userspace code that is
> making use of this.
> 
> Jason / others, did we ever get the central repo for user space code
> set up?

No, we have some fragemented userspace repos at the moment only. I've
been looking around for someone who'd like to take on the challenge

Jason

Re: [PATCH net-next v5 5/5] bnxt_fwctl: Add documentation entries

[Leon Romanovsky]

On Wed, Oct 29, 2025 at 10:36:35AM -0300, Jason Gunthorpe wrote:
> On Tue, Oct 28, 2025 at 04:46:51PM +0000, Jonathan Cameron wrote:
> > On Tue, 14 Oct 2025 01:10:33 -0700
> > Pavan Chebbi <pavan.chebbi@broadcom.com> wrote:
> > 
> > > Add bnxt_fwctl to the driver and fwctl documentation pages.
> > > 
> > > Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
> > > Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> > > Signed-off-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
> > 
> > Would be useful to provide a reference to userspace code that is
> > making use of this.
> > 
> > Jason / others, did we ever get the central repo for user space code
> > set up?
> 
> No, we have some fragemented userspace repos at the moment only. I've
> been looking around for someone who'd like to take on the challenge

Once you find that person, contact me, I'll transfer an ownership
for https://github.com/linux-fwctl to him.

Thanks

> 
> Jason