From: Niklas Cassel <cassel@kernel.org>
To: Koichiro Den <den@valinux.co.jp>
Cc: "Manivannan Sadhasivam" <mani@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Frank Li" <Frank.Li@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Jingoo Han" <jingoohan1@gmail.com>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Rob Herring" <robh@kernel.org>, "Aksh Garg" <a-garg7@ti.com>,
"Christoph Hellwig" <hch@lst.de>,
"Sagi Grimberg" <sagi@grimberg.me>,
"Chaitanya Kulkarni" <kch@nvidia.com>,
"Jon Mason" <jdmason@kudzu.us>,
"Dave Jiang" <dave.jiang@intel.com>,
"Allen Hubbe" <allenbh@gmail.com>,
"Heiko Stuebner" <heiko@sntech.de>,
"Shawn Lin" <shawn.lin@rock-chips.com>,
"Manikanta Maddireddy" <mmaddireddy@nvidia.com>,
"Shin'ichiro Kawasaki" <shinichiro.kawasaki@wdc.com>,
linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org,
ntb@lists.linux.dev, linux-rockchip@lists.infradead.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA
Date: Wed, 2 Sep 2026 09:25:13 +0200 [thread overview]
Message-ID: <apfPWc09CY2o-P5s@ryzen> (raw)
In-Reply-To: <sor2mm4jzgeprhn3fjw4jmcaowwkhhuus6exi4k5kum6frqvug@gatkk7r4qyxd>
On Wed, Sep 02, 2026 at 11:14:00AM +0900, Koichiro Den wrote:
>
> This still sounds really odd, and it does not obviously explain why both DWORDs
> at 0x4000 and 0x4004 are affected. I may be missing something important. If
> anything similar involving iMSIX-TX Table RAM entry #0 has been discussed
> before, please point me to the relevant thread.
>
> (Also, I can share my current WIP v3 series, which makes pci-epf-test and
> nvmet-pci-epf use the hardware-owned layout when available, while leaving
> pci-epf-ntb with its EPF-owned layout. If you have any setup where you could
> test it, please let me know.)
Some time ago, I did verify that I could raise an IRQ using
dw_pcie_ep_raise_msix_irq_doorbell(). If I remember correctly, I did so using
pci-epf-test and not nvmet-pci-epf.
Did pci-epf-test work? (From what you explain, it seems to be related to if
MSI-X Table entry #0 is used or not.)
Please share your branch. I should have time to do a quick test.
>
> >
> >
> > What I suggest that you instead do something like:
> > Patch 1/7: pci-epf-vntb: Fix to actually use MSI-X when requested.
> > This probably includes adding the code that is inside the
> > if (ret == -ENOENT) { }.
> > Patch 2/7: PCI: endpoint: Support hardware-owned MSI-X table and PBA
> > Patch 3/7: PCI: dw-rockchip: Support fixed MSI-X table and PBA on RK3588
> > Patch 4/7: pci-epf-vntb: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 5/7: pci-epf-test: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 6/7: nvmet-pci-epf: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 7/7: pci-epf-ntb: Add comment that explains why it cannot call
> > pci_epc_get_hw_msix_layout() like all other EPF drivers that
> > support MSI-X.
> >
> >
> > This way, patch 1/7 can be backported, and will help all users vNTB users,
> > even the users that do not use RK3588 as an EPC.
>
> That split makes sense. However, given the issue I described above, perhaps I
> should send patch 1/7 as a standalone fix and hold off on the rest until the
> issue is understood and resolved.
Agreed, a standalone 1/7 patch could be picked up without the need to wait for
any other changes.
>
> Testing vNTB alone (ie. this v2 way) did not expose it because its first data
> doorbell uses MSI-X Table entry #2. I only found the issue because your
> suggestion led me to convert and test nvmet-pci-epf, thank you!
I'm sorry that we have a problem.
At least it is better to discover it now, rather than discovering it after
the patches had been merged.
Kind regards,
Niklas
WARNING: multiple messages have this Message-ID (diff)
From: Niklas Cassel <cassel@kernel.org>
To: Koichiro Den <den@valinux.co.jp>
Cc: "Manivannan Sadhasivam" <mani@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Frank Li" <Frank.Li@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Jingoo Han" <jingoohan1@gmail.com>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Rob Herring" <robh@kernel.org>, "Aksh Garg" <a-garg7@ti.com>,
"Christoph Hellwig" <hch@lst.de>,
"Sagi Grimberg" <sagi@grimberg.me>,
"Chaitanya Kulkarni" <kch@nvidia.com>,
"Jon Mason" <jdmason@kudzu.us>,
"Dave Jiang" <dave.jiang@intel.com>,
"Allen Hubbe" <allenbh@gmail.com>,
"Heiko Stuebner" <heiko@sntech.de>,
"Shawn Lin" <shawn.lin@rock-chips.com>,
"Manikanta Maddireddy" <mmaddireddy@nvidia.com>,
"Shin'ichiro Kawasaki" <shinichiro.kawasaki@wdc.com>,
linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org,
ntb@lists.linux.dev, linux-rockchip@lists.infradead.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA
Date: Wed, 2 Sep 2026 09:25:13 +0200 [thread overview]
Message-ID: <apfPWc09CY2o-P5s@ryzen> (raw)
In-Reply-To: <sor2mm4jzgeprhn3fjw4jmcaowwkhhuus6exi4k5kum6frqvug@gatkk7r4qyxd>
On Wed, Sep 02, 2026 at 11:14:00AM +0900, Koichiro Den wrote:
>
> This still sounds really odd, and it does not obviously explain why both DWORDs
> at 0x4000 and 0x4004 are affected. I may be missing something important. If
> anything similar involving iMSIX-TX Table RAM entry #0 has been discussed
> before, please point me to the relevant thread.
>
> (Also, I can share my current WIP v3 series, which makes pci-epf-test and
> nvmet-pci-epf use the hardware-owned layout when available, while leaving
> pci-epf-ntb with its EPF-owned layout. If you have any setup where you could
> test it, please let me know.)
Some time ago, I did verify that I could raise an IRQ using
dw_pcie_ep_raise_msix_irq_doorbell(). If I remember correctly, I did so using
pci-epf-test and not nvmet-pci-epf.
Did pci-epf-test work? (From what you explain, it seems to be related to if
MSI-X Table entry #0 is used or not.)
Please share your branch. I should have time to do a quick test.
>
> >
> >
> > What I suggest that you instead do something like:
> > Patch 1/7: pci-epf-vntb: Fix to actually use MSI-X when requested.
> > This probably includes adding the code that is inside the
> > if (ret == -ENOENT) { }.
> > Patch 2/7: PCI: endpoint: Support hardware-owned MSI-X table and PBA
> > Patch 3/7: PCI: dw-rockchip: Support fixed MSI-X table and PBA on RK3588
> > Patch 4/7: pci-epf-vntb: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 5/7: pci-epf-test: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 6/7: nvmet-pci-epf: Add code that calls pci_epc_get_hw_msix_layout()
> > and uses that layout, if the function returned success.
> > Patch 7/7: pci-epf-ntb: Add comment that explains why it cannot call
> > pci_epc_get_hw_msix_layout() like all other EPF drivers that
> > support MSI-X.
> >
> >
> > This way, patch 1/7 can be backported, and will help all users vNTB users,
> > even the users that do not use RK3588 as an EPC.
>
> That split makes sense. However, given the issue I described above, perhaps I
> should send patch 1/7 as a standalone fix and hold off on the rest until the
> issue is understood and resolved.
Agreed, a standalone 1/7 patch could be picked up without the need to wait for
any other changes.
>
> Testing vNTB alone (ie. this v2 way) did not expose it because its first data
> doorbell uses MSI-X Table entry #2. I only found the issue because your
> suggestion led me to convert and test nvmet-pci-epf, thank you!
I'm sorry that we have a problem.
At least it is better to discover it now, rather than discovering it after
the patches had been merged.
Kind regards,
Niklas
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
next prev parent reply other threads:[~2026-09-02 7:25 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-30 15:19 [PATCH v2 0/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA Koichiro Den
2026-08-30 15:19 ` Koichiro Den
2026-08-30 15:19 ` [PATCH v2 1/3] " Koichiro Den
2026-08-30 15:19 ` Koichiro Den
2026-08-30 15:32 ` sashiko-bot
2026-08-30 15:19 ` [PATCH v2 2/3] PCI: dw-rockchip: Support fixed MSI-X table and PBA on RK3588 Koichiro Den
2026-08-30 15:19 ` Koichiro Den
2026-08-30 15:28 ` sashiko-bot
2026-08-30 15:19 ` [PATCH v2 3/3] PCI: endpoint: pci-epf-vntb: Honor MSI-X selection Koichiro Den
2026-08-30 15:19 ` Koichiro Den
2026-08-30 15:36 ` sashiko-bot
2026-08-31 10:24 ` [PATCH v2 0/3] PCI: endpoint: Support hardware-owned MSI-X table and PBA Niklas Cassel
2026-08-31 10:24 ` Niklas Cassel
2026-08-31 16:11 ` Koichiro Den
2026-08-31 16:11 ` Koichiro Den
2026-08-31 20:01 ` Niklas Cassel
2026-08-31 20:01 ` Niklas Cassel
2026-09-02 2:14 ` Koichiro Den
2026-09-02 2:14 ` Koichiro Den
2026-09-02 7:25 ` Niklas Cassel [this message]
2026-09-02 7:25 ` Niklas Cassel
2026-09-03 7:56 ` Koichiro Den
2026-09-03 7:56 ` Koichiro Den
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=apfPWc09CY2o-P5s@ryzen \
--to=cassel@kernel.org \
--cc=Frank.Li@kernel.org \
--cc=a-garg7@ti.com \
--cc=allenbh@gmail.com \
--cc=bhelgaas@google.com \
--cc=dave.jiang@intel.com \
--cc=den@valinux.co.jp \
--cc=hch@lst.de \
--cc=heiko@sntech.de \
--cc=jdmason@kudzu.us \
--cc=jingoohan1@gmail.com \
--cc=kch@nvidia.com \
--cc=kishon@kernel.org \
--cc=kwilczynski@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=mmaddireddy@nvidia.com \
--cc=ntb@lists.linux.dev \
--cc=robh@kernel.org \
--cc=sagi@grimberg.me \
--cc=shawn.lin@rock-chips.com \
--cc=shinichiro.kawasaki@wdc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.