From: Frank Li <Frank.li@oss.nxp.com>
To: Koichiro Den <den@valinux.co.jp>
Cc: Jon Mason <jdmason@kudzu.us>, Dave Jiang <dave.jiang@intel.com>,
Frank Li <Frank.Li@kernel.org>, Allen Hubbe <allenbh@gmail.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Niklas Cassel <cassel@kernel.org>,
Nicholas Bellinger <nab@linux-iscsi.org>,
ntb@lists.linux.dev, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 01/16] NTB: ntb_transport: Abort link setup on QP MW allocation failure
Date: Wed, 12 Aug 2026 09:59:58 -0500 [thread overview]
Message-ID: <anyKbq3mpLG4y7rb@SMW015318> (raw)
In-Reply-To: <kdcjmsy7gqy2iolfom3tzigocecmkquzepbhusgprvzbfnfg6h@acqxas2tvpxb>
On Wed, Aug 12, 2026 at 11:15:37AM +0900, Koichiro Den wrote:
> On Mon, Aug 10, 2026 at 01:41:59PM -0500, Frank Li wrote:
> > On Tue, Aug 11, 2026 at 01:51:20AM +0900, Koichiro Den wrote:
> > > ntb_transport_setup_qp_mw() can fail while growing a QP's RX entry pool,
> > > but the link worker ignores that error. The worker can then publish a QP
> > > whose memory-window state is only partly initialized, and later work can
> > > use stale or incomplete pointers.
> > >
> > > Set up every QP memory window before publishing the transport link. On
> > > failure, clear the QP pointers before releasing its MW backing and leave
> > > the link down.
> > >
> > > Fixes: a754a8fcaf38 ("NTB: allocate number transport entries depending on size of ring size")
> > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > ---
> > > drivers/ntb/ntb_transport.c | 20 ++++++++++++++++----
> > > 1 file changed, 16 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
> > > index f59f926d4bfa..3efc50955253 100644
> > > --- a/drivers/ntb/ntb_transport.c
> > > +++ b/drivers/ntb/ntb_transport.c
> > > @@ -1084,14 +1084,19 @@ static void ntb_transport_link_work(struct work_struct *work)
> > > goto out1;
> > > }
> > >
> > > - nt->link_is_up = true;
> > > + nt->link_is_up = false;
> > > + for (i = 0; i < nt->qp_count; i++) {
> > > + rc = ntb_transport_setup_qp_mw(nt, i);
> > > + if (rc)
> > > + goto out1;
> > > + ntb_transport_setup_qp_peer_msi(nt, i);
> > > + }
> > >
> > > + /* Publish the link only after every QP has been set up. */
> > > + nt->link_is_up = true;
> >
> > Not sure if need WRITE_ONCE() or other memory barrier to make sure
> > ntb_transport_setup_qp_mw() and ntb_transport_setup_qp_peer_msi() actually
> > complete before set this flag.
>
> In that sense, I think we need a compiler barrier + WRITE_ONCE to avoid
> use_msi=true <-> nt->link_is_up=true reordering in case dev_info becomes no-op.
>
> But if we want this nt->link_is_up read/write more robust, here I would choose
> smp_store_release(&nt->link_is_up, true), paired with smp_load_acquire() in
> ntb_transport_link_up(), because another CPU can possibly observe
> link_is_up=true there and queue the work.
>
> I'm not sure this compiler or memory barrier issue should be folded into this
> small patch, but if preferred, I'll do so.
It should be seperate patch to fix it.
Frank
>
> Thanks for the review,
> Koichiro
>
> >
> > Frank
> >
> > > for (i = 0; i < nt->qp_count; i++) {
> > > struct ntb_transport_qp *qp = &nt->qp_vec[i];
> > >
> > > - ntb_transport_setup_qp_mw(nt, i);
> > > - ntb_transport_setup_qp_peer_msi(nt, i);
> > > -
> > > if (qp->client_ready)
> > > schedule_delayed_work(&qp->link_work, 0);
> > > }
> > > @@ -1099,6 +1104,13 @@ static void ntb_transport_link_work(struct work_struct *work)
> > > return;
> > >
> > > out1:
> > > + for (i = 0; i < nt->qp_count; i++) {
> > > + struct ntb_transport_qp *qp = &nt->qp_vec[i];
> > > +
> > > + qp->rx_buff = NULL;
> > > + qp->remote_rx_info = NULL;
> > > + }
> > > +
> > > for (i = 0; i < nt->mw_count; i++)
> > > ntb_free_mw(nt, i);
> > >
> > > --
> > > 2.51.0
> > >
next prev parent reply other threads:[~2026-08-12 15:00 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-10 16:51 [PATCH 00/16] NTB: Add direct TX/RX using PCI endpoint DMA Koichiro Den
2026-08-10 16:51 ` [PATCH 01/16] NTB: ntb_transport: Abort link setup on QP MW allocation failure Koichiro Den
2026-08-10 18:41 ` Frank Li
2026-08-12 2:15 ` Koichiro Den
2026-08-12 14:59 ` Frank Li [this message]
2026-08-10 16:51 ` [PATCH 02/16] NTB: ntb_transport: Reject oversized TX buffers Koichiro Den
2026-08-10 16:51 ` [PATCH 03/16] NTB: ntb_transport: Start TX offload thread after queue setup Koichiro Den
2026-08-10 16:51 ` [PATCH 04/16] NTB: ntb_transport: Stop QP work before freeing a queue Koichiro Den
2026-08-10 16:51 ` [PATCH 05/16] NTB: ntb_transport: Run RX processing on system workqueue Koichiro Den
2026-08-10 16:51 ` [PATCH 06/16] NTB: ntb_transport: Define direct-DMA shared state Koichiro Den
2026-08-10 16:51 ` [PATCH 07/16] NTB: ntb_transport: Negotiate direct-DMA queue layout Koichiro Den
2026-08-10 16:51 ` [PATCH 08/16] NTB: ntb_transport: Add opt-in direct-DMA channel reservation Koichiro Den
2026-08-10 16:51 ` [PATCH 09/16] NTB: ntb_transport: Allocate direct-DMA queue state Koichiro Den
2026-08-10 16:51 ` [PATCH 10/16] NTB: ntb_transport: Implement direct-DMA QP session handshake Koichiro Den
2026-08-10 16:51 ` [PATCH 11/16] NTB: ntb_transport: Implement direct-DMA RX buffer publication Koichiro Den
2026-08-10 16:51 ` [PATCH 12/16] NTB: ntb_transport: Implement direct-DMA TX submission Koichiro Den
2026-08-10 16:51 ` [PATCH 13/16] NTB: ntb_transport: Implement safe direct-DMA teardown Koichiro Den
2026-08-10 16:51 ` [PATCH 14/16] NTB: ntb_transport: Enable direct-DMA queues Koichiro Den
2026-08-10 17:04 ` Koichiro Den
2026-08-10 16:51 ` [PATCH 15/16] NTB: ntb_transport: Report the direct-DMA payload limit Koichiro Den
2026-08-10 16:51 ` [PATCH 16/16] NTB: ntb_transport: Add optional polling for direct-DMA RX Koichiro Den
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=anyKbq3mpLG4y7rb@SMW015318 \
--to=frank.li@oss.nxp.com \
--cc=Frank.Li@kernel.org \
--cc=allenbh@gmail.com \
--cc=cassel@kernel.org \
--cc=dave.jiang@intel.com \
--cc=den@valinux.co.jp \
--cc=gregkh@linuxfoundation.org \
--cc=jdmason@kudzu.us \
--cc=linux-kernel@vger.kernel.org \
--cc=nab@linux-iscsi.org \
--cc=netdev@vger.kernel.org \
--cc=ntb@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox