Netdev List
 help / color / mirror / Atom feed
From: Koichiro Den <den@valinux.co.jp>
To: Jon Mason <jdmason@kudzu.us>, Dave Jiang <dave.jiang@intel.com>,
	Frank Li <Frank.Li@kernel.org>, Allen Hubbe <allenbh@gmail.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Niklas Cassel <cassel@kernel.org>,
	Nicholas Bellinger <nab@linux-iscsi.org>
Cc: ntb@lists.linux.dev, netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: [PATCH 06/16] NTB: ntb_transport: Define direct-DMA shared state
Date: Tue, 11 Aug 2026 01:51:25 +0900	[thread overview]
Message-ID: <20260810165136.2292436-7-den@valinux.co.jp> (raw)
In-Reply-To: <20260810165136.2292436-1-den@valinux.co.jp>

Define the per-QP shared layout for session state, RX buffer
publications, completion reporting, and teardown boundaries.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/ntb/ntb_transport.c | 56 +++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index 463ee6f17fcb..44957c11d1c8 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -55,6 +55,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/kthread.h>
+#include <linux/limits.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 #include <linux/slab.h>
@@ -264,6 +265,61 @@ enum {
 	LINK_DOWN_FLAG = BIT(1),
 };
 
+/* RX publications are pushed into the sender's local copy. */
+struct ntb_direct_pub {
+	u32 addr_lo;
+	u32 addr_hi;
+	u32 len;
+};
+
+/* During a session, the peer writes this area and the local CPU reads it. */
+struct ntb_direct_shared {
+	/* Peer session ID and the peer's ack of the local session. */
+	u32 session;
+	u32 session_ack;
+
+	/*
+	 * Tearing down a direct-DMA session must keep RX mappings valid until
+	 * completions for all in-flight transfers have been consumed. Each peer
+	 * publishes its final issued TX boundary, followed by a session-tagged
+	 * quiesce marker, using CPU MMIO writes to the peer's memory window.
+	 * The marker may arrive before earlier DMA writes because the CPU and
+	 * DMA paths may use different PCIe ordering domains. Quiesce is
+	 * therefore acknowledged only after completions through the published
+	 * boundary have been consumed:
+	 * - quiesce_issue: exclusive boundary of the peer's issued TX work that
+	 *                  local RX must consume.
+	 * - quiesce: local session ID written by the peer after quiesce_issue,
+	 *            validating that boundary for the current session.
+	 * - quiesce_ack: local session ID confirming that the peer consumed
+	 *                local TX through its final boundary.
+	 */
+	u32 quiesce_issue;
+	u32 quiesce;
+	u32 quiesce_ack;
+
+	/* Base address of the peer's RX completion array. */
+	u32 cpl_addr_lo;
+	u32 cpl_addr_hi;
+
+	/* Exclusive producer boundary for the peer RX publications below. */
+	u32 pub_head;
+
+	/* Peer RX buffer publications in ring order. */
+	struct ntb_direct_pub pub[];
+};
+
+/* Bound per-QP completion arrays and software queue allocations. */
+#define NTB_DIRECT_MAX_RING_ENTRIES	4096
+
+/* Zero is pending, U32_MAX is an error, and other values are lengths. */
+#define NTB_DIRECT_CPL_ERROR		U32_MAX
+
+static inline size_t ntb_direct_shared_size(unsigned int entries)
+{
+	return struct_size_t(struct ntb_direct_shared, pub, entries);
+}
+
 struct ntb_payload_header {
 	unsigned int ver;
 	unsigned int len;
-- 
2.51.0


  parent reply	other threads:[~2026-08-10 16:51 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-10 16:51 [PATCH 00/16] NTB: Add direct TX/RX using PCI endpoint DMA Koichiro Den
2026-08-10 16:51 ` [PATCH 01/16] NTB: ntb_transport: Abort link setup on QP MW allocation failure Koichiro Den
2026-08-10 18:41   ` Frank Li
2026-08-10 16:51 ` [PATCH 02/16] NTB: ntb_transport: Reject oversized TX buffers Koichiro Den
2026-08-10 16:51 ` [PATCH 03/16] NTB: ntb_transport: Start TX offload thread after queue setup Koichiro Den
2026-08-10 16:51 ` [PATCH 04/16] NTB: ntb_transport: Stop QP work before freeing a queue Koichiro Den
2026-08-10 16:51 ` [PATCH 05/16] NTB: ntb_transport: Run RX processing on system workqueue Koichiro Den
2026-08-10 16:51 ` Koichiro Den [this message]
2026-08-10 16:51 ` [PATCH 07/16] NTB: ntb_transport: Negotiate direct-DMA queue layout Koichiro Den
2026-08-10 16:51 ` [PATCH 08/16] NTB: ntb_transport: Add opt-in direct-DMA channel reservation Koichiro Den
2026-08-10 16:51 ` [PATCH 09/16] NTB: ntb_transport: Allocate direct-DMA queue state Koichiro Den
2026-08-10 16:51 ` [PATCH 10/16] NTB: ntb_transport: Implement direct-DMA QP session handshake Koichiro Den
2026-08-10 16:51 ` [PATCH 11/16] NTB: ntb_transport: Implement direct-DMA RX buffer publication Koichiro Den
2026-08-10 16:51 ` [PATCH 12/16] NTB: ntb_transport: Implement direct-DMA TX submission Koichiro Den
2026-08-10 16:51 ` [PATCH 13/16] NTB: ntb_transport: Implement safe direct-DMA teardown Koichiro Den
2026-08-10 16:51 ` [PATCH 14/16] NTB: ntb_transport: Enable direct-DMA queues Koichiro Den
2026-08-10 17:04   ` Koichiro Den
2026-08-10 16:51 ` [PATCH 15/16] NTB: ntb_transport: Report the direct-DMA payload limit Koichiro Den
2026-08-10 16:51 ` [PATCH 16/16] NTB: ntb_transport: Add optional polling for direct-DMA RX Koichiro Den

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260810165136.2292436-7-den@valinux.co.jp \
    --to=den@valinux.co.jp \
    --cc=Frank.Li@kernel.org \
    --cc=allenbh@gmail.com \
    --cc=cassel@kernel.org \
    --cc=dave.jiang@intel.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jdmason@kudzu.us \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nab@linux-iscsi.org \
    --cc=netdev@vger.kernel.org \
    --cc=ntb@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox