[PATCH 00/13] raw/ntb: post-merge fixes

[PATCH 00/13] raw/ntb: post-merge fixes

[Raghavendra Ningoji]

This series addresses the post-merge review of the AMD NTB support
by Stephen. Some of the issues also exist in the Intel code and in
released versions; those patches are tagged for stable.

- Patches 1-4 fix the type confusion in rte_pmd_ntb_get_mem_align(),
  the unbounded peer memory window count, the off-by-one in the
  address remapping and the memory window leaks.

- Patch 5 fixes the doorbell interrupts with vfio-pci: only MSI-X
  vector 0 (DB0) was serviced, so the DB1/DB2 exchange of dev_stop
  never completed and the stop waited for its 1s timeout.

- Patch 7 removes the AMD hardware event handling, which could not run
  as wired, and documents that hardware events are not handled.

- Patches 8-9 fix the Rx path: a partially consumed packet was leaked
  and desynchronized the ring, and an mbuf owned by the application
  could be freed again on device stop.

- The other patches are documentation and cleanup.

The peer memory window size check suggested for the AMD handshake is
not added: all the AMD scratchpads are in use, and the window limit
register programmed with the local memzone length already bounds what
the peer can reach.

Tested with vfio-pci on two AMD EPYC Embedded hosts: handshake,
file transfer, and stop/quit with DB1/DB2 exchange.

Raghavendra Ningoji (13):
  raw/ntb: fix device check in memory alignment query
  raw/ntb: fix peer memory window count check
  raw/ntb: fix memory window bound in address remapping
  raw/ntb: fix memory leaks
  raw/ntb: fix doorbell interrupts with VFIO MSI-X
  raw/ntb: fix AMD processor names
  raw/ntb: remove AMD hardware event handling
  raw/ntb: fix Rx of incomplete packets
  raw/ntb: fix double free of Rx mbufs on stop
  examples/ntb: fix raw/ntb library linking
  doc: add NTB device info change to ABI section
  doc: add IOVA requirement for NTB
  raw/ntb: remove unused AMD register definitions

 doc/guides/rawdevs/ntb.rst             |  12 ++-
 doc/guides/rel_notes/release_26_11.rst |   6 +-
 drivers/raw/ntb/ntb.c                  | 136 ++++++++++++++++++++-----
 drivers/raw/ntb/ntb.h                  |   7 +-
 drivers/raw/ntb/ntb_hw_amd.c           |  54 ++--------
 drivers/raw/ntb/ntb_hw_amd.h           |  25 +----
 drivers/raw/ntb/ntb_hw_intel.c         |   9 +-
 examples/ntb/Makefile                  |   3 +-
 8 files changed, 149 insertions(+), 103 deletions(-)

-- 
2.34.1

[PATCH 01/13] raw/ntb: fix device check in memory alignment query

[Raghavendra Ningoji]

rte_pmd_ntb_get_mem_align() only checked the device index range and
the presence of private data before casting it to struct ntb_hw.
Passing the id of a detached slot or of a rawdev from another driver
would then call through a garbage ops pointer.

Check that the device is attached and is driven by this PMD.

Also return 0 for an out of range memory window index, as documented,
instead of a cache line size.

Fixes: 5ce50c234ad1 ("raw/ntb: generalize framework for multiple vendors")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index fe10782ecf..e42af0b9c5 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -33,6 +33,8 @@ static const struct rte_pci_id pci_id_ntb_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static const struct rte_rawdev_ops ntb_ops;
+
 /* Align with enum ntb_xstats_idx */
 static struct rte_rawdev_xstats_name ntb_xstats_names[] = {
 	{"Tx-packets"},
@@ -722,15 +724,15 @@ rte_pmd_ntb_get_mem_align(uint16_t dev_id, uint32_t mw_id, uint64_t mw_len)
 	struct rte_rawdev *dev;
 	struct ntb_hw *hw;
 
-	if (dev_id >= RTE_RAWDEV_MAX_DEVS)
+	if (dev_id >= RTE_RAWDEV_MAX_DEVS || !rte_rawdev_pmd_is_valid_dev(dev_id))
 		return 0;
 	dev = rte_rawdev_pmd_get_dev(dev_id);
-	if (dev->dev_private == NULL)
+	if (dev->dev_ops != &ntb_ops)
 		return 0;
 
 	hw = dev->dev_private;
 	if (mw_id >= hw->mw_cnt)
-		return RTE_CACHE_LINE_SIZE;
+		return 0;
 
 	return (*hw->ntb_ops->mem_align)(dev, mw_id, mw_len);
 }
-- 
2.34.1

[PATCH 02/13] raw/ntb: fix peer memory window count check

[Raghavendra Ningoji]

The number of memory windows used by the peer is read from a
scratchpad register written by the peer, and is then used as the loop
bound to fill peer_mw_base[], which is allocated with mw_cnt entries.
A peer advertising more windows (bug, version mismatch, or a stale
scratchpad) makes the driver write past the end of this array.
On Intel, the 32-bit register value was also silently truncated into
the 8-bit peer_used_mws field.

Reject a peer configuration using more windows than the local device
has.

Fixes: c39d1e082a4b ("raw/ntb: setup queues")
Fixes: d6dc5a7f8b06 ("raw/ntb: support AMD NTB")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb_hw_amd.c   | 10 ++++++++--
 drivers/raw/ntb/ntb_hw_intel.c |  7 ++++++-
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/raw/ntb/ntb_hw_amd.c b/drivers/raw/ntb/ntb_hw_amd.c
index ab78675218..dba5b03671 100644
--- a/drivers/raw/ntb/ntb_hw_amd.c
+++ b/drivers/raw/ntb/ntb_hw_amd.c
@@ -502,7 +502,7 @@ static int
 amd_ntb_read_peer_config(const struct rte_rawdev *dev)
 {
 	struct ntb_hw *ntb = dev->dev_private;
-	uint32_t info, peer_qps, peer_qsz, lo, hi;
+	uint32_t info, peer_qps, peer_qsz, peer_used_mws, lo, hi;
 	int i;
 
 	info = amd_ntb_spad_read(dev, AMD_SPAD_CNT_INFO, 0);
@@ -520,7 +520,13 @@ amd_ntb_read_peer_config(const struct rte_rawdev *dev)
 		return -EINVAL;
 	}
 
-	ntb->peer_used_mws = (info >> 16) & 0xff;
+	peer_used_mws = (info >> 16) & 0xff;
+	if (peer_used_mws > ntb->mw_cnt) {
+		NTB_LOG(ERR, "Peer used mws %u > mw cnt %u.", peer_used_mws, ntb->mw_cnt);
+		return -EINVAL;
+	}
+	ntb->peer_used_mws = peer_used_mws;
+
 	for (i = 0; i < ntb->peer_used_mws; i++) {
 		lo = amd_ntb_spad_read(dev, AMD_SPAD_MW0_BA_L + 2 * i, 0);
 		hi = amd_ntb_spad_read(dev, AMD_SPAD_MW0_BA_H + 2 * i, 0);
diff --git a/drivers/raw/ntb/ntb_hw_intel.c b/drivers/raw/ntb/ntb_hw_intel.c
index 1b784055ca..78fce9ffac 100644
--- a/drivers/raw/ntb/ntb_hw_intel.c
+++ b/drivers/raw/ntb/ntb_hw_intel.c
@@ -724,7 +724,12 @@ intel_ntb_read_peer_config(const struct rte_rawdev *dev)
 		return -EINVAL;
 	}
 
-	hw->peer_used_mws = intel_ntb_spad_read(dev, SPAD_USED_MWS, 0);
+	peer_val = intel_ntb_spad_read(dev, SPAD_USED_MWS, 0);
+	if (peer_val > hw->mw_cnt) {
+		NTB_LOG(ERR, "Peer used mws %u > mw cnt %u.", peer_val, hw->mw_cnt);
+		return -EINVAL;
+	}
+	hw->peer_used_mws = peer_val;
 
 	for (i = 0; i < hw->peer_used_mws; i++) {
 		peer_base_h = intel_ntb_spad_read(dev, SPAD_MW0_BA_H + 2 * i, 0);
-- 
2.34.1

[PATCH 03/13] raw/ntb: fix memory window bound in address remapping

[Raghavendra Ningoji]

When translating a peer address into the local mapping of a memory
window, the end of the window was included in the accepted range,
so an address one byte past the window was mapped.

Exclude the end of the window.

Fixes: 11b5c7daf019 ("raw/ntb: add enqueue and dequeue functions")
Fixes: d6dc5a7f8b06 ("raw/ntb: support AMD NTB")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb_hw_amd.c   | 2 +-
 drivers/raw/ntb/ntb_hw_intel.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/raw/ntb/ntb_hw_amd.c b/drivers/raw/ntb/ntb_hw_amd.c
index dba5b03671..64075501c2 100644
--- a/drivers/raw/ntb/ntb_hw_amd.c
+++ b/drivers/raw/ntb/ntb_hw_amd.c
@@ -166,7 +166,7 @@ amd_ntb_ioremap(const struct rte_rawdev *dev, uint64_t addr)
 
 	for (i = 0; i < ntb->peer_used_mws; i++) {
 		if (addr >= ntb->peer_mw_base[i] &&
-				addr <= ntb->peer_mw_base[i] + ntb->mw_size[i]) {
+				addr < ntb->peer_mw_base[i] + ntb->mw_size[i]) {
 			base = amd_ntb_get_peer_mw_addr(dev, i);
 			mapped = (void *)(size_t)(addr - ntb->peer_mw_base[i] + (size_t)base);
 			break;
diff --git a/drivers/raw/ntb/ntb_hw_intel.c b/drivers/raw/ntb/ntb_hw_intel.c
index 78fce9ffac..47d5d39339 100644
--- a/drivers/raw/ntb/ntb_hw_intel.c
+++ b/drivers/raw/ntb/ntb_hw_intel.c
@@ -312,7 +312,7 @@ intel_ntb_ioremap(const struct rte_rawdev *dev, uint64_t addr)
 
 	for (i = 0; i < hw->peer_used_mws; i++) {
 		if (addr >= hw->peer_mw_base[i] &&
-		    addr <= hw->peer_mw_base[i] + hw->mw_size[i]) {
+		    addr < hw->peer_mw_base[i] + hw->mw_size[i]) {
 			base = intel_ntb_get_peer_mw_addr(dev, i);
 			mapped = (void *)(size_t)(addr - hw->peer_mw_base[i] +
 				 (size_t)base);
-- 
2.34.1

[PATCH 04/13] raw/ntb: fix memory leaks

[Raghavendra Ningoji]

The memory window size array allocated by the vendor init at probe
time was never freed.
The peer memory window base array allocated at each device start was
only freed if the start failed, so it leaked on every stop/start cycle.

Free the peer memory window base array when stopping the device, and
the memory window size array when closing it (which also happens on
device removal and on probe failure).

Fixes: c39d1e082a4b ("raw/ntb: setup queues")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index e42af0b9c5..af3fcd832b 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -820,6 +820,8 @@ ntb_dev_start(struct rte_rawdev *dev)
 
 err_up:
 	rte_free(hw->peer_mw_base);
+	hw->peer_mw_base = NULL;
+	hw->peer_used_mws = 0;
 err_q_init:
 	for (i = 0; i < hw->queue_pairs; i++) {
 		ntb_rxq_release_mbufs(hw->rx_queues[i]);
@@ -872,6 +874,10 @@ ntb_dev_stop(struct rte_rawdev *dev)
 		ntb_txq_release_mbufs(hw->tx_queues[i]);
 	}
 
+	rte_free(hw->peer_mw_base);
+	hw->peer_mw_base = NULL;
+	hw->peer_used_mws = 0;
+
 	dev->started = 0;
 }
 
@@ -890,6 +896,9 @@ ntb_dev_close(struct rte_rawdev *dev)
 		ntb_queue_release(dev, i);
 	hw->queue_pairs = 0;
 
+	rte_free(hw->mw_size);
+	hw->mw_size = NULL;
+
 	intr_handle = hw->pci_dev->intr_handle;
 	/* Disable interrupt only once */
 	if (!rte_intr_nb_efd_get(intr_handle) &&
-- 
2.34.1

[PATCH 05/13] raw/ntb: fix doorbell interrupts with VFIO MSI-X

[Raghavendra Ningoji]

With VFIO MSI-X, each doorbell raises its own vector. Vector 0 is
signalled on the PCI device interrupt handle while the others are
signalled on the eventfds created by rte_intr_efd_enable(), but the
interrupt handler was only registered on the device interrupt handle.

As a result, doorbells other than DB0 were only noticed when DB0 fired
at the same time. In particular, the DB1 rung by the peer in its
dev_stop was never handled, the peer never got the DB2 answer and had
to wait for the 1s timeout.

Register the interrupt handler on each doorbell eventfd as well.

Fixes: 62012a76811e ("raw/ntb: add handshake process")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb.c | 77 ++++++++++++++++++++++++++++++++++++++++++-
 drivers/raw/ntb/ntb.h |  4 +++
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index af3fcd832b..a1993121dd 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -11,6 +11,7 @@
 #include <rte_lcore.h>
 #include <rte_cycles.h>
 #include <rte_eal.h>
+#include <rte_errno.h>
 #include <rte_log.h>
 #include <rte_pci.h>
 #include <rte_mbuf.h>
@@ -124,6 +125,74 @@ ntb_dev_intr_handler(void *param)
 		(*hw->ntb_ops->handle_hw_event)(dev);
 }
 
+static void
+ntb_db_intr_unregister(struct rte_rawdev *dev)
+{
+	struct ntb_hw *hw = dev->dev_private;
+	struct rte_intr_handle *handle;
+	uint16_t i;
+
+	for (i = 0; i < hw->nb_db_intr_handles; i++) {
+		handle = hw->db_intr_handles[i];
+		rte_intr_callback_unregister_sync(handle,
+						  ntb_dev_intr_handler, dev);
+		/* The eventfd is owned by the PCI device interrupt handle. */
+		rte_intr_fd_set(handle, -1);
+		rte_intr_instance_free(handle);
+	}
+	rte_free(hw->db_intr_handles);
+	hw->db_intr_handles = NULL;
+	hw->nb_db_intr_handles = 0;
+}
+
+/*
+ * With MSI-X, only vector 0 is serviced through the PCI device interrupt
+ * handle. Each other doorbell raises its own vector, signalled on the
+ * eventfds created by rte_intr_efd_enable(): register the handler on them.
+ */
+static int
+ntb_db_intr_register(struct rte_rawdev *dev,
+		     struct rte_intr_handle *intr_handle)
+{
+	struct ntb_hw *hw = dev->dev_private;
+	int nb_efd = rte_intr_nb_efd_get(intr_handle);
+	struct rte_intr_handle *handle;
+	int i, ret;
+
+	hw->db_intr_handles = rte_zmalloc("ntb_db_intr",
+				nb_efd * sizeof(*hw->db_intr_handles), 0);
+	if (hw->db_intr_handles == NULL)
+		return -ENOMEM;
+
+	for (i = 0; i < nb_efd; i++) {
+		handle = rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE);
+		if (handle == NULL) {
+			ret = -ENOMEM;
+			goto err;
+		}
+		hw->db_intr_handles[hw->nb_db_intr_handles++] = handle;
+
+		if (rte_intr_type_set(handle, RTE_INTR_HANDLE_VFIO_MSIX) ||
+		    rte_intr_fd_set(handle,
+				    rte_intr_efds_index_get(intr_handle, i))) {
+			ret = -rte_errno;
+			goto err;
+		}
+
+		ret = rte_intr_callback_register(handle,
+						 ntb_dev_intr_handler, dev);
+		if (ret)
+			goto err;
+	}
+
+	return 0;
+
+err:
+	NTB_LOG(ERR, "Failed to register doorbell interrupt %d.", i + 1);
+	ntb_db_intr_unregister(dev);
+	return ret;
+}
+
 static int
 ntb_queue_conf_get(struct rte_rawdev *dev,
 		   uint16_t queue_id,
@@ -905,6 +974,8 @@ ntb_dev_close(struct rte_rawdev *dev)
 	    !rte_intr_max_intr_get(intr_handle))
 		return 0;
 
+	ntb_db_intr_unregister(dev);
+
 	/* Clean datapath event and vec mapping */
 	rte_intr_efd_disable(intr_handle);
 	rte_intr_vec_list_free(intr_handle);
@@ -1308,7 +1379,11 @@ ntb_init_hw(struct rte_rawdev *dev, struct rte_pci_device *pci_dev)
 	 * by default for intel gen3. They are mapped to msix vec 1-32,
 	 * and hardware intr is mapped to 0. Map all to 0 for uio.
 	 */
-	if (!rte_intr_cap_multiple(intr_handle)) {
+	if (rte_intr_cap_multiple(intr_handle)) {
+		ret = ntb_db_intr_register(dev, intr_handle);
+		if (ret)
+			return ret;
+	} else {
 		for (i = 0; i < hw->db_cnt; i++) {
 			ret = (*hw->ntb_ops->vector_bind)(dev, i, 0);
 			if (ret)
diff --git a/drivers/raw/ntb/ntb.h b/drivers/raw/ntb/ntb.h
index 0181bec5c9..2787883160 100644
--- a/drivers/raw/ntb/ntb.h
+++ b/drivers/raw/ntb/ntb.h
@@ -219,6 +219,10 @@ struct ntb_hw {
 	struct rte_pci_device *pci_dev;
 	char *hw_addr;
 
+	/* Interrupt handles for the MSI-X doorbell vectors besides vector 0. */
+	struct rte_intr_handle **db_intr_handles;
+	uint16_t nb_db_intr_handles;
+
 	uint8_t peer_dev_up;
 	uint64_t *mw_size;
 	/* remote mem base addr */
-- 
2.34.1

[PATCH 06/13] raw/ntb: fix AMD processor names

[Raghavendra Ningoji]

Refer to the AMD processors integrating the NTB endpoints by their
EPYC Embedded 9004/9005/8004 series numbers rather than code names.

Fixes: d6dc5a7f8b06 ("raw/ntb: support AMD NTB")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 doc/guides/rawdevs/ntb.rst             | 4 ++--
 doc/guides/rel_notes/release_26_11.rst | 4 ++--
 drivers/raw/ntb/ntb_hw_amd.h           | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/doc/guides/rawdevs/ntb.rst b/doc/guides/rawdevs/ntb.rst
index 99474745d6..37328cd9e6 100644
--- a/doc/guides/rawdevs/ntb.rst
+++ b/doc/guides/rawdevs/ntb.rst
@@ -41,7 +41,7 @@ AMD EPYC Embedded NTB
 ---------------------
 
 The driver also supports the NTB endpoints integrated in AMD EPYC Embedded
-"Turin", "Genoa" and "Siena" processors.
+9004/9005/8004 processors.
 These use a primary/secondary topology rather than the Intel back-to-back topology:
 one endpoint is enumerated as the primary (device ID ``0x14c0``) and
 the other as the secondary (device ID ``0x14c3``).
@@ -179,7 +179,7 @@ Limitation
 
 This PMD is supported on:
 
-- AMD EPYC Embedded "Turin", "Genoa" and "Siena" processors.
+- AMD EPYC Embedded 9004/9005/8004 processors.
 - 4th Generation Intel® Xeon® Scalable Processors.
 - 3rd Generation Intel® Xeon® Scalable Processors.
 - 2nd Generation Intel® Xeon® Scalable Processors.
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 8ca8379cc6..6c76c60139 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -306,8 +306,8 @@ New Features
 
 * **Added AMD NTB support to the NTB rawdev driver.**
 
-  Added support for the NTB endpoints integrated in AMD EPYC Embedded "Turin", "Genoa"
-  and "Siena" processors to the ``raw/ntb`` driver.
+  Added support for the NTB endpoints integrated in AMD EPYC Embedded
+  9004/9005/8004 processors to the ``raw/ntb`` driver.
   The NTB rawdev framework was generalized to support multiple vendors,
   with AMD-specific hardware access, a primary/secondary topology
   and a packed scratchpad handshake.
diff --git a/drivers/raw/ntb/ntb_hw_amd.h b/drivers/raw/ntb/ntb_hw_amd.h
index 48f0db1ad4..b03aed7d08 100644
--- a/drivers/raw/ntb/ntb_hw_amd.h
+++ b/drivers/raw/ntb/ntb_hw_amd.h
@@ -7,7 +7,7 @@
 
 #include <stdint.h>
 
-/* NTB vendor and device IDs (EPYC Embedded Turin/Genoa/Siena). */
+/* NTB vendor and device IDs (EPYC Embedded 9004/9005/8004). */
 #define NTB_AMD_VENDOR_ID		0x1022
 #define NTB_AMD_DEV_ID_PRI		0x14c0	/* Primary NTB endpoint. */
 #define NTB_AMD_DEV_ID_SEC		0x14c3	/* Secondary NTB endpoint. */
-- 
2.34.1

[PATCH 07/13] raw/ntb: remove AMD hardware event handling

[Raghavendra Ningoji]

The AMD hardware event handler could never run: the event interrupts
were never unmasked, their MSI-X vectors were never enabled and the
handler was only reached when no known doorbell was pending.
It also mishandled several simultaneous events and never cleared
the interrupt status.

Remove it, together with the now unused vendor operation and
register definitions, and document that hardware events are not
handled.

Fixes: d6dc5a7f8b06 ("raw/ntb: support AMD NTB")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 doc/guides/rawdevs/ntb.rst   |  4 ++++
 drivers/raw/ntb/ntb.c        |  4 ----
 drivers/raw/ntb/ntb.h        |  3 ---
 drivers/raw/ntb/ntb_hw_amd.c | 42 +-----------------------------------
 drivers/raw/ntb/ntb_hw_amd.h | 21 ------------------
 5 files changed, 5 insertions(+), 69 deletions(-)

diff --git a/doc/guides/rawdevs/ntb.rst b/doc/guides/rawdevs/ntb.rst
index 37328cd9e6..c4ad2f05c4 100644
--- a/doc/guides/rawdevs/ntb.rst
+++ b/doc/guides/rawdevs/ntb.rst
@@ -53,6 +53,10 @@ The AMD NTB hardware exposes two memory windows (BAR23 and BAR45),
 The scratchpad bank is split into two disjoint 8-register sets, one owned by each side,
 so the driver uses a packed handshake layout that fits within 8 registers.
 
+The AMD NTB hardware events (link up/down, peer reset, power state changes)
+are not handled: a peer going away is only detected through the doorbell
+handshake done when the peer device is stopped.
+
 The AMD NTB uses an outbound translation window:
 a write to a memory window offset is forwarded to ``(base | offset)`` in the peer's memory
 rather than ``(base + offset)``.
diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index a1993121dd..d514ed669f 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -119,10 +119,6 @@ ntb_dev_intr_handler(void *param)
 
 	/* Clear other received doorbells. */
 	(*hw->ntb_ops->db_clear)(dev, db_bits);
-
-	/* Let the vendor process any hardware-specific events. */
-	if (hw->ntb_ops->handle_hw_event != NULL)
-		(*hw->ntb_ops->handle_hw_event)(dev);
 }
 
 static void
diff --git a/drivers/raw/ntb/ntb.h b/drivers/raw/ntb/ntb.h
index 2787883160..2434997e84 100644
--- a/drivers/raw/ntb/ntb.h
+++ b/drivers/raw/ntb/ntb.h
@@ -102,8 +102,6 @@ enum ntb_spad_idx {
  * @link_cleanup: Tear down the local scratchpad/translation state so the
  * peer can no longer reach local memory.
  * @mem_align: Base-address alignment required for a memory window memzone of a given length.
- * @handle_hw_event: Optional. Process vendor-specific hardware events from
- * the common interrupt handler. May be NULL.
  */
 struct ntb_dev_ops {
 	int (*ntb_dev_init)(const struct rte_rawdev *dev);
@@ -126,7 +124,6 @@ struct ntb_dev_ops {
 	int (*read_peer_config)(const struct rte_rawdev *dev);
 	void (*link_cleanup)(const struct rte_rawdev *dev);
 	uint64_t (*mem_align)(const struct rte_rawdev *dev, uint32_t mw_id, uint64_t mw_len);
-	void (*handle_hw_event)(const struct rte_rawdev *dev);
 };
 
 struct ntb_desc {
diff --git a/drivers/raw/ntb/ntb_hw_amd.c b/drivers/raw/ntb/ntb_hw_amd.c
index 64075501c2..47e4979619 100644
--- a/drivers/raw/ntb/ntb_hw_amd.c
+++ b/drivers/raw/ntb/ntb_hw_amd.c
@@ -445,7 +445,7 @@ static int
 amd_ntb_vector_bind(const struct rte_rawdev *dev __rte_unused, uint8_t intr __rte_unused,
 		uint8_t msix __rte_unused)
 {
-	/* Each doorbell/event maps to its MSI-X vector by default. */
+	/* Each doorbell maps to its MSI-X vector by default. */
 	return 0;
 }
 
@@ -569,45 +569,6 @@ amd_ntb_link_cleanup(const struct rte_rawdev *dev)
 		amd_ntb_mw_set_trans(dev, i, 0, 0);
 }
 
-/* Handle link/power-management events and acknowledge the SMU. Invoked from
- * the common interrupt handler when no inter-device doorbell is pending.
- */
-static void
-amd_ntb_handle_hw_event(const struct rte_rawdev *dev)
-{
-	struct ntb_hw *ntb = dev->dev_private;
-	uint32_t event, ack;
-
-	event = rte_read32((char *)ntb->hw_addr + AMD_INTSTAT_OFFSET);
-	event &= AMD_EVENT_INTMASK;
-	if (event == 0)
-		return;
-
-	switch (event) {
-	case AMD_PEER_FLUSH_EVENT:
-		NTB_LOG(INFO, "Peer flush event.");
-		break;
-	case AMD_PEER_D0_EVENT:
-		ack = rte_read32((char *)ntb->hw_addr + AMD_PMESTAT_OFFSET);
-		if (ack & 0x1)
-			NTB_LOG(INFO, "D0 wakeup completed for NTB.");
-		/* fall through to ack the SMU */
-		/* Falls through. */
-	case AMD_PEER_RESET_EVENT:
-	case AMD_LINK_DOWN_EVENT:
-	case AMD_PEER_D3_EVENT:
-	case AMD_PEER_PMETO_EVENT:
-	case AMD_LINK_UP_EVENT:
-		ack = rte_read32((char *)ntb->hw_addr + AMD_SMUACK_OFFSET);
-		ack |= event;
-		rte_write32(ack, (char *)ntb->hw_addr + AMD_SMUACK_OFFSET);
-		break;
-	default:
-		NTB_LOG(ERR, "Unknown interrupt event 0x%" PRIx32, event);
-		break;
-	}
-}
-
 static uint64_t
 amd_ntb_get_mem_align(const struct rte_rawdev *dev __rte_unused, uint32_t mw_id __rte_unused,
 		uint64_t mw_len)
@@ -634,5 +595,4 @@ const struct ntb_dev_ops amd_ntb_ops = {
 	.read_peer_config	= amd_ntb_read_peer_config,
 	.link_cleanup		= amd_ntb_link_cleanup,
 	.mem_align		= amd_ntb_get_mem_align,
-	.handle_hw_event	= amd_ntb_handle_hw_event,
 };
diff --git a/drivers/raw/ntb/ntb_hw_amd.h b/drivers/raw/ntb/ntb_hw_amd.h
index b03aed7d08..8fffa24348 100644
--- a/drivers/raw/ntb/ntb_hw_amd.h
+++ b/drivers/raw/ntb/ntb_hw_amd.h
@@ -16,7 +16,6 @@
 #define AMD_MW_COUNT			2
 #define AMD_DB_COUNT			16
 #define AMD_SPAD_COUNT			16
-#define AMD_MSIX_VECTOR_COUNT		24
 
 /* Peer register window: peer register = primary offset + 0x400. */
 #define AMD_PEER_OFFSET			0x400
@@ -37,10 +36,6 @@
 #define AMD_DBREQ_OFFSET		0x454	/* Doorbell request. */
 #define AMD_DBMASK_OFFSET		0x45C	/* Doorbell mask. */
 #define AMD_DBSTAT_OFFSET		0x460	/* Doorbell status. */
-#define AMD_INTMASK_OFFSET		0x470	/* Interrupt mask. */
-#define AMD_INTSTAT_OFFSET		0x474	/* Interrupt status. */
-#define AMD_PMESTAT_OFFSET		0x480	/* PME status. */
-#define AMD_SMUACK_OFFSET		0x4A0	/* SMU control (PSMU_ACK). */
 
 /* Link Status Register (0x408) bits, spec Table 5. */
 #define AMD_SIDE_MASK			(1 << 0) /* 0: primary, 1: secondary. */
@@ -50,22 +45,6 @@
 #define AMD_SMM_REG_CTL			(1 << 20)
 #define AMD_PMM_REG_CTL			(1 << 21)
 
-/* Interrupt Status/Mask event bits, spec Table 7. */
-#define AMD_PEER_FLUSH_EVENT		(1 << 0)
-#define AMD_PEER_RESET_EVENT		(1 << 1)
-#define AMD_PEER_D3_EVENT		(1 << 2)
-#define AMD_PEER_PMETO_EVENT		(1 << 3)
-#define AMD_PEER_D0_EVENT		(1 << 4)
-#define AMD_LINK_UP_EVENT		(1 << 5)
-#define AMD_LINK_DOWN_EVENT		(1 << 6)
-#define AMD_EVENT_INTMASK		(AMD_PEER_FLUSH_EVENT | \
-					 AMD_PEER_RESET_EVENT | \
-					 AMD_PEER_D3_EVENT | \
-					 AMD_PEER_PMETO_EVENT | \
-					 AMD_PEER_D0_EVENT | \
-					 AMD_LINK_UP_EVENT | \
-					 AMD_LINK_DOWN_EVENT)
-
 /* PCIe link status decoding (link speed/width). */
 #define AMD_LNK_STA_SPEED_MASK		0x000f
 #define AMD_LNK_STA_WIDTH_MASK		0x03f0
-- 
2.34.1

[PATCH 08/13] raw/ntb: fix Rx of incomplete packets

[Raghavendra Ningoji]

When a packet had no EOP descriptor among the ones walked, or when a
refill mbuf could not be allocated in the middle of a packet, the Rx
burst stopped with the packet partially consumed: the ring indexes had
moved, but the mbuf chain was neither returned nor freed, and the
refilled descriptors were not published to the peer if no packet had
been completed before.

The EOP guard also bounded the walk on the ring size rather than on
the descriptors actually published by the peer.

Only take a packet once all of its descriptors have been published,
and allocate its refill mbufs before consuming it, so that the ring
indexes are only advanced for complete packets.

Fixes: 11b5c7daf019 ("raw/ntb: add enqueue and dequeue functions")
Fixes: 5ce50c234ad1 ("raw/ntb: generalize framework for multiple vendors")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb.c | 39 +++++++++++++++++++++------------------
 1 file changed, 21 insertions(+), 18 deletions(-)

diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index d514ed669f..87a35c796c 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -650,13 +650,14 @@ ntb_dequeue_bufs(struct rte_rawdev *dev,
 	struct ntb_rx_queue *rxq = hw->rx_queues[(size_t)context];
 	struct ntb_rx_entry *sw_ring = rxq->sw_ring;
 	struct ntb_desc rx_desc[NTB_MAX_DESC_SIZE];
-	struct rte_mbuf *first, *rxm_t;
+	struct rte_mbuf *new_mbufs[NTB_MAX_DESC_SIZE];
+	struct rte_mbuf *first = NULL, *rxm_t;
 	struct rte_mbuf *prev = NULL;
 	volatile struct ntb_used *rx_item;
 	uint16_t nb_mbufs = 0;
 	uint16_t nb_rx = 0;
 	uint64_t bytes = 0;
-	uint16_t off, last_avail, used_cnt, used_nb;
+	uint16_t off, last_avail, used_cnt, used_nb, nb_segs;
 	int i;
 
 	if (unlikely(dev->started == 0 || hw->peer_dev_up == 0)) {
@@ -673,13 +674,24 @@ ntb_dequeue_bufs(struct rte_rawdev *dev,
 	used_nb = (used_cnt - rxq->last_used) & (rxq->nb_rx_desc - 1);
 	count = RTE_MIN(count, used_nb);
 	for (nb_rx = 0; nb_rx < count; nb_rx++) {
-		i = 0;
-		while (true) {
-			if (unlikely(nb_mbufs >= rxq->nb_rx_desc)) {
-				NTB_LOG(ERR, "Malformed rx stream (no EOP); "
-					"aborting to avoid desc overflow.");
+		/* Only take a packet once the peer has published all of it. */
+		for (nb_segs = 1; ; nb_segs++) {
+			if (nb_segs > used_nb)
 				goto end_of_rx;
-			}
+			rx_item = rxq->rx_used_ring + ((rxq->last_used + nb_segs - 1) &
+						       (rxq->nb_rx_desc - 1));
+			if (rx_item->flags & NTB_FLAG_EOP)
+				break;
+		}
+
+		/* Get the refill mbufs first so a packet is never partially consumed. */
+		if (unlikely(rte_mbuf_raw_alloc_bulk(rxq->mpool, new_mbufs, nb_segs))) {
+			NTB_LOG(ERR, "recv alloc mbuf failed.");
+			goto end_of_rx;
+		}
+		used_nb -= nb_segs;
+
+		for (i = 0; i < nb_segs; i++) {
 			rx_item = rxq->rx_used_ring + rxq->last_used;
 			rxm_t = sw_ring[rxq->last_used].mbuf;
 			rxm_t->data_len = rx_item->len;
@@ -701,15 +713,9 @@ ntb_dequeue_bufs(struct rte_rawdev *dev,
 			rxq->last_used = (rxq->last_used + 1) &
 					 (rxq->nb_rx_desc - 1);
 
-			/* alloc new mbuf */
-			rxm_t = rte_mbuf_raw_alloc(rxq->mpool);
-			if (unlikely(rxm_t == NULL)) {
-				NTB_LOG(ERR, "recv alloc mbuf failed.");
-				goto end_of_rx;
-			}
+			rxm_t = new_mbufs[i];
 			rxm_t->port = rxq->port_id;
 			sw_ring[rxq->last_avail].mbuf = rxm_t;
-			i++;
 
 			/* fill new desc */
 			rx_desc[nb_mbufs].addr =
@@ -718,9 +724,6 @@ ntb_dequeue_bufs(struct rte_rawdev *dev,
 						  RTE_PKTMBUF_HEADROOM;
 			rxq->last_avail = (rxq->last_avail + 1) &
 					  (rxq->nb_rx_desc - 1);
-
-			if (rx_item->flags & NTB_FLAG_EOP)
-				break;
 		}
 		/* update stats */
 		bytes += first->pkt_len;
-- 
2.34.1

[PATCH 09/13] raw/ntb: fix double free of Rx mbufs on stop

[Raghavendra Ningoji]

Once an Rx mbuf is handed to the application, its software ring entry
still references it until the slot is refilled. On device stop, all
referenced mbufs are freed, including the one now owned by the
application.

Clear the ring entry when the mbuf is taken.

Fixes: 11b5c7daf019 ("raw/ntb: add enqueue and dequeue functions")
Cc: stable@dpdk.org

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c
index 87a35c796c..d4c2de60bd 100644
--- a/drivers/raw/ntb/ntb.c
+++ b/drivers/raw/ntb/ntb.c
@@ -694,6 +694,7 @@ ntb_dequeue_bufs(struct rte_rawdev *dev,
 		for (i = 0; i < nb_segs; i++) {
 			rx_item = rxq->rx_used_ring + rxq->last_used;
 			rxm_t = sw_ring[rxq->last_used].mbuf;
+			sw_ring[rxq->last_used].mbuf = NULL;
 			rxm_t->data_len = rx_item->len;
 			rxm_t->data_off = RTE_PKTMBUF_HEADROOM;
 			rxm_t->port = rxq->port_id;
-- 
2.34.1

[PATCH 10/13] examples/ntb: fix raw/ntb library linking

[Raghavendra Ningoji]

The raw/ntb driver library was added to the linker flags of both the
shared and static targets. For static linking, it is already part of
the libdpdk pkg-config output, so only add it to the shared target,
as done in other examples.

Fixes: 5ce50c234ad1 ("raw/ntb: generalize framework for multiple vendors")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 examples/ntb/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/examples/ntb/Makefile b/examples/ntb/Makefile
index 6c0307c697..a074251646 100644
--- a/examples/ntb/Makefile
+++ b/examples/ntb/Makefile
@@ -23,11 +23,12 @@ static: build/$(APP)-static
 	ln -sf $(APP)-static build/$(APP)
 
 CFLAGS += -D_FILE_OFFSET_BITS=64
-LDFLAGS += -pthread -lrte_raw_ntb
+LDFLAGS += -pthread
 
 PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null)
 CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) -I build/
 LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
+LDFLAGS_SHARED += -lrte_raw_ntb
 LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk)
 
 build/commands.h: commands.list Makefile
-- 
2.34.1

[PATCH 11/13] doc: add NTB device info change to ABI section

[Raghavendra Ningoji]

Removing the mw_size_align field changed the layout of the installed
struct ntb_dev_info. Mention it in the ABI changes as well.

Fixes: 5ce50c234ad1 ("raw/ntb: generalize framework for multiple vendors")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 doc/guides/rel_notes/release_26_11.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 6c76c60139..23980d5364 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -689,6 +689,8 @@ ABI Changes
 * cryptodev: The struct ``rte_crypto_rsa_op_param``
   was updated to include ``pss_salt``.
 
+* raw/ntb: The ``mw_size_align`` field was removed from ``struct ntb_dev_info``.
+
 
 Known Issues
 ------------
-- 
2.34.1

[PATCH 12/13] doc: add IOVA requirement for NTB

[Raghavendra Ningoji]

The memory window memzone is reserved with the alignment required by
the NTB hardware, but this alignment applies to the virtual address.
With IOVA as PA and a memzone larger than the hugepage size, the IOVA
may not be aligned and the device configuration fails.

Document that IOVA as VA mode or large enough hugepages are needed.

Fixes: 5ce50c234ad1 ("raw/ntb: generalize framework for multiple vendors")

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 doc/guides/rawdevs/ntb.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/guides/rawdevs/ntb.rst b/doc/guides/rawdevs/ntb.rst
index c4ad2f05c4..8332c27de9 100644
--- a/doc/guides/rawdevs/ntb.rst
+++ b/doc/guides/rawdevs/ntb.rst
@@ -66,6 +66,10 @@ Different NTB hardware imposes different alignment constraints,
 so applications should not assume any particular value:
 reserve each memory window memzone with the alignment returned by ``rte_pmd_ntb_get_mem_align()``
 for that window and length. The ``ntb_fwd`` example shows this usage.
+The memzone alignment applies to its virtual address.
+With IOVA as PA, the IOVA of a memzone larger than the hugepage size
+is not guaranteed to be aligned, and the device configuration fails:
+use IOVA as VA mode, or hugepages at least as large as the memory window.
 
 Device Setup
 ------------
-- 
2.34.1

[PATCH 13/13] raw/ntb: remove unused AMD register definitions

[Raghavendra Ningoji]

The scratchpad mutex and doorbell flush mode registers are not used
by the driver.

Signed-off-by: Raghavendra Ningoji <raghavendra.ningoji@amd.com>
---
 drivers/raw/ntb/ntb_hw_amd.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/raw/ntb/ntb_hw_amd.h b/drivers/raw/ntb/ntb_hw_amd.h
index 8fffa24348..eec2041ad2 100644
--- a/drivers/raw/ntb/ntb_hw_amd.h
+++ b/drivers/raw/ntb/ntb_hw_amd.h
@@ -25,14 +25,12 @@
 
 /* Primary-side MMIO register offsets (BAR0), spec Table 1. */
 #define AMD_CNTL_OFFSET			0x200	/* Link Control. */
-#define AMD_SPADMUTEX_OFFSET		0x20C	/* Scratchpad mutex. */
 #define AMD_SPAD_OFFSET			0x210	/* Scratchpad registers. */
 #define AMD_SIDEINFO_OFFSET		0x408	/* Link Status (PSIDE_INFO). */
 #define AMD_BAR23_LIMIT_OFFSET		0x418	/* MW BAR23 size. */
 #define AMD_BAR45_LIMIT_OFFSET		0x420	/* MW BAR45 size. */
 #define AMD_BAR23_XLAT_OFFSET		0x438	/* MW BAR23 translation. */
 #define AMD_BAR45_XLAT_OFFSET		0x440	/* MW BAR45 translation. */
-#define AMD_DBFM_OFFSET			0x450	/* Doorbell flush mode. */
 #define AMD_DBREQ_OFFSET		0x454	/* Doorbell request. */
 #define AMD_DBMASK_OFFSET		0x45C	/* Doorbell mask. */
 #define AMD_DBSTAT_OFFSET		0x460	/* Doorbell status. */
-- 
2.34.1

Re: [PATCH 00/13] raw/ntb: post-merge fixes

[Stephen Hemminger]

On Wed, 4 Nov 2026 16:49:48 +0530
Raghavendra Ningoji <raghavendra.ningoji@amd.com> wrote:

> This series addresses the post-merge review of the AMD NTB support
> by Stephen. Some of the issues also exist in the Intel code and in
> released versions; those patches are tagged for stable.
> 
> - Patches 1-4 fix the type confusion in rte_pmd_ntb_get_mem_align(),
>   the unbounded peer memory window count, the off-by-one in the
>   address remapping and the memory window leaks.
> 
> - Patch 5 fixes the doorbell interrupts with vfio-pci: only MSI-X
>   vector 0 (DB0) was serviced, so the DB1/DB2 exchange of dev_stop
>   never completed and the stop waited for its 1s timeout.
> 
> - Patch 7 removes the AMD hardware event handling, which could not run
>   as wired, and documents that hardware events are not handled.
> 
> - Patches 8-9 fix the Rx path: a partially consumed packet was leaked
>   and desynchronized the ring, and an mbuf owned by the application
>   could be freed again on device stop.
> 
> - The other patches are documentation and cleanup.
> 
> The peer memory window size check suggested for the AMD handshake is
> not added: all the AMD scratchpads are in use, and the window limit
> register programmed with the local memzone length already bounds what
> the peer can reach.
> 
> Tested with vfio-pci on two AMD EPYC Embedded hosts: handshake,
> file transfer, and stop/quit with DB1/DB2 exchange.
> 
> Raghavendra Ningoji (13):
>   raw/ntb: fix device check in memory alignment query
>   raw/ntb: fix peer memory window count check
>   raw/ntb: fix memory window bound in address remapping
>   raw/ntb: fix memory leaks
>   raw/ntb: fix doorbell interrupts with VFIO MSI-X
>   raw/ntb: fix AMD processor names
>   raw/ntb: remove AMD hardware event handling
>   raw/ntb: fix Rx of incomplete packets
>   raw/ntb: fix double free of Rx mbufs on stop
>   examples/ntb: fix raw/ntb library linking
>   doc: add NTB device info change to ABI section
>   doc: add IOVA requirement for NTB
>   raw/ntb: remove unused AMD register definitions
> 
>  doc/guides/rawdevs/ntb.rst             |  12 ++-
>  doc/guides/rel_notes/release_26_11.rst |   6 +-
>  drivers/raw/ntb/ntb.c                  | 136 ++++++++++++++++++++-----
>  drivers/raw/ntb/ntb.h                  |   7 +-
>  drivers/raw/ntb/ntb_hw_amd.c           |  54 ++--------
>  drivers/raw/ntb/ntb_hw_amd.h           |  25 +----
>  drivers/raw/ntb/ntb_hw_intel.c         |   9 +-
>  examples/ntb/Makefile                  |   3 +-
>  8 files changed, 149 insertions(+), 103 deletions(-)
> 

Lots of AI reported issues here:

Subject: Re: [PATCH 00/13] raw/ntb: post-merge fixes

Series applies to main (26.11-rc2) and builds at every commit with
-Dwerror=true (raw/ntb + examples/ntb).

Patch 04/13 raw/ntb: fix memory leaks

Warning: mw_size is freed in dev_close, but rte_rawdev_close() does not
detach the device; it stays attached and its ops remain reachable.

  +	rte_free(hw->mw_size);
  +	hw->mw_size = NULL;

After close, rte_pmd_ntb_get_mem_align() passes the new checks from
patch 1 (attached, dev_ops == &ntb_ops, mw_id < mw_cnt) and on Intel
calls intel_ntb_get_mem_align(), which does

	return hw->mw_size[mw_id];

i.e. a NULL dereference. A configure after close does the same in
intel_ntb_dev_handshake() (hw->mw_size[i]), and ntb_dev_info_get()
now hands the application mw_cnt = 2 with mw_size = NULL; ntb_fwd
indexes ntb_info.mw_size[0] unconditionally.

Both vendors have a fixed count of 2 windows (XEON_MW_COUNT,
AMD_MW_COUNT), so drop the allocation and embed the array in
struct ntb_hw. It then lives exactly as long as dev_private, the
info->mw_size pointer stays valid, and the leak goes away with no
free at all:

	#define NTB_MAX_MW	2
	...
	uint64_t mw_size[NTB_MAX_MW];

with RTE_BUILD_BUG_ON(XEON_MW_COUNT > NTB_MAX_MW) (and AMD) in the
vendor init, and the rte_zmalloc()/NULL check removed there.

Patch 05/13 raw/ntb: fix doorbell interrupts with VFIO MSI-X

Warning: db_intr_handles is an array of process-private pointers
(handles come from rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE))
used only by the control path. No reason to put it in hugepage memory.

  +	hw->db_intr_handles = rte_zmalloc("ntb_db_intr",
  +				nb_efd * sizeof(*hw->db_intr_handles), 0);

Use calloc(nb_efd, sizeof(*hw->db_intr_handles)) and free() in
ntb_db_intr_unregister().

Info (pre-existing, not introduced by this patch): peer_dev_up is a
plain uint8_t written by the interrupt thread (DB1/DB2 handlers,
"hw->peer_dev_up = 0;") and polled by ntb_dev_stop():

	while (hw->peer_dev_up && time_out) {

Now that DB2 is actually delivered, this loop is the normal stop path
rather than the timeout. It only works because rte_delay_us() is an
opaque call. Should be rte_atomic_store_explicit()/load with
release/acquire, as a separate patch.

Patch 08/13 raw/ntb: fix Rx of incomplete packets

Info (pre-existing, not introduced by this patch): the new on-stack

  +	struct rte_mbuf *new_mbufs[NTB_MAX_DESC_SIZE];

like the existing rx_desc[], is only bounded because nb_rx_desc is
assumed <= NTB_MAX_DESC_SIZE. ntb_rxq_setup() takes
"rxq->nb_rx_desc = rxq_conf->nb_desc;" without checking that, or that
it is a power of two (the ring indexes use & (nb_rx_desc - 1)); only
the ntb_fwd example range-checks it. The same function also leaks rxq
when rx_mp is NULL. Worth a separate fix:

	if (rxq_conf->rx_mp == NULL ||
	    rxq_conf->nb_desc < NTB_MIN_DESC_SIZE ||
	    rxq_conf->nb_desc > NTB_MAX_DESC_SIZE ||
	    !rte_is_power_of_2(rxq_conf->nb_desc))
		return -EINVAL;

placed before the rte_zmalloc_socket().

Review-Result: WARNING