[PATCH 0/3] PCI: endpoint: Support vNTB as a non-first EPF

[PATCH 0/3] PCI: endpoint: Support vNTB as a non-first EPF

[Koichiro Den]

Hi,

This series allows a vNTB endpoint function to be instantiated as PF1+.

vntb_epf_mw_set_trans() programs the memory-window BAR with hardcoded
function numbers (0, 0), so binding the EPF to any other function
programs the wrong one. Patch 1 resolves it.

The platform-MSI doorbell path can currently populate messages only for
the first EPF attached to the EPC. Supporting multiple EPFs in that path
requires separate work. This series leaves it unchanged and lets a
non-first EPF use the embedded doorbell instead.

The embedded-doorbell auxiliary-resource interface is
vendor-independent. Its current DesignWare provider exposes one EPC-wide
source. Since the generic interface cannot identify which EPF's peer
issued a write to a shared source, patch 2 makes the IRQ exclusive.
Patch 3 lets non-first EPFs try the embedded doorbell. If another EPF
has already requested the IRQ, vNTB retains its polling fallback.

I tested the embedded doorbell path on R-Car S4 with the proposed DMA
endpoint function as PF0 and vNTB as PF1.

Best regards,
Koichiro

Koichiro Den (3):
  PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming
  PCI: endpoint: pci-ep-msi: Make embedded doorbell IRQ exclusive
  PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells

 drivers/pci/endpoint/functions/pci-epf-vntb.c |  3 +-
 drivers/pci/endpoint/pci-ep-msi.c             | 40 ++++++++++---------
 include/linux/pci-epf.h                       |  3 +-
 3 files changed, 26 insertions(+), 20 deletions(-)

base-commit: e6ff30150738af5c7aa20fad08b74cbf9b86d3a2
-- 
2.51.0

[PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Koichiro Den]

vntb_epf_mw_set_trans() programs the memory-window BAR through
pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
lands on the wrong function whenever the vNTB EPF is bound to anything
but PF0. The other BAR programming sites in the vNTB driver already pass
the EPF's own numbers.

Pass the EPF's own func_no/vfunc_no here as well.

Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index c3caec927d74..fba65abfb6b2 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
 	epf_bar->barno = barno;
 	epf_bar->size = size;
 
-	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
+	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
+			      ntb->epf->vfunc_no, epf_bar);
 	if (ret) {
 		dev_err(dev, "failure set mw trans\n");
 		return ret;
-- 
2.51.0

[PATCH 2/3] PCI: endpoint: pci-ep-msi: Make embedded doorbell IRQ exclusive

[Koichiro Den]

Embedded doorbells may use the same virq for several messages. This does
not mean the IRQ itself should be shared. Consumers only need to request
each distinct virq once, as vNTB and pci-epf-test already do.

IRQF_SHARED would let different EPFs register handlers for the same IRQ.
The embedded doorbell interface cannot tell which EPF a write targets,
so every handler would handle it.

Only the first EPF can allocate doorbells today, so this cannot happen
yet. Drop IRQF_SHARED before extending the embedded backend to non-first
EPFs. A second request for the same IRQ will then fail with -EBUSY
instead of attaching another handler.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/pci/endpoint/pci-ep-msi.c | 9 +++++----
 include/linux/pci-epf.h           | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c
index 0855c7930abb..a123a189b4e6 100644
--- a/drivers/pci/endpoint/pci-ep-msi.c
+++ b/drivers/pci/endpoint/pci-ep-msi.c
@@ -10,7 +10,6 @@
 #include <linux/cleanup.h>
 #include <linux/device.h>
 #include <linux/export.h>
-#include <linux/interrupt.h>
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/msi.h>
@@ -112,8 +111,11 @@ static int pci_epf_alloc_doorbell_embedded(struct pci_epf *epf, u16 num_db)
 	 *
 	 * Still, pci_epf_alloc_doorbell() allows requesting multiple doorbells.
 	 * For such backends we replicate the same address/data for each entry
-	 * and mark the IRQ as shared (IRQF_SHARED). Consumers must treat them
-	 * as equivalent "kick" doorbells.
+	 * as equivalent "kick" doorbells. Consumers must request each distinct
+	 * IRQ only once.
+	 *
+	 * Keep the IRQ exclusive because the source cannot identify which EPF
+	 * rang the doorbell.
 	 */
 	for (i = 0; i < num_db; i++)
 		msg[i] = (struct pci_epf_doorbell_msg) {
@@ -121,7 +123,6 @@ static int pci_epf_alloc_doorbell_embedded(struct pci_epf *epf, u16 num_db)
 			.msg.address_hi = (u32)(addr >> 32),
 			.msg.data = doorbell->u.db_mmio.data,
 			.virq = doorbell->u.db_mmio.irq,
-			.irq_flags = IRQF_SHARED,
 			.type = PCI_EPF_DOORBELL_EMBEDDED,
 			.bar = doorbell->bar,
 			.offset = (doorbell->bar == NO_BAR) ? 0 :
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index 8a6c64a35890..e5b9a79beb93 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -163,7 +163,8 @@ enum pci_epf_doorbell_type {
  *       For MSI-backed doorbells this is the MSI message, while for
  *       "embedded" doorbells this represents an MMIO write that asserts
  *       an interrupt on the EP side.
- * @virq: IRQ number of this doorbell message
+ * @virq: IRQ number of this doorbell message. Multiple messages may use the
+ *        same IRQ; consumers must request each distinct IRQ only once.
  * @irq_flags: Required flags for request_irq()/request_threaded_irq().
  *             Callers may OR-in additional flags (e.g. IRQF_ONESHOT).
  * @type: Doorbell type.
-- 
2.51.0

[PATCH 3/3] PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells

[Koichiro Den]

pci_epf_alloc_doorbell() rejects every EPF that is not the first one
attached to the EPC because the MSI-backed path can populate doorbell
messages only for that EPF. This also prevents non-first EPFs from using
the embedded doorbell backend.

Keep the MSI-backed path limited to the first attached EPF. Let
non-first EPFs skip it and try the embedded doorbell directly. The
embedded IRQ is exclusive, so a second embedded user fails to request
the IRQ instead of receiving another EPF's notifications.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/pci/endpoint/pci-ep-msi.c | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c
index a123a189b4e6..5ea7eef9794f 100644
--- a/drivers/pci/endpoint/pci-ep-msi.c
+++ b/drivers/pci/endpoint/pci-ep-msi.c
@@ -199,28 +199,31 @@ static int pci_epf_alloc_doorbell_msi(struct pci_epf *epf, u16 num_db)
 int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 num_db)
 {
 	struct pci_epc *epc = epf->epc;
+	struct pci_epf *first_epf;
 	struct device *dev = &epf->dev;
 	int ret;
 
-	/* TODO: Multi-EPF support */
-	if (list_first_entry_or_null(&epc->pci_epf, struct pci_epf, list) != epf) {
-		dev_err(dev, "Doorbell doesn't support multiple EPF\n");
-		return -EINVAL;
-	}
-
 	if (epf->db_msg)
 		return -EBUSY;
 
-	ret = pci_epf_alloc_doorbell_msi(epf, num_db);
-	if (!ret)
-		return 0;
-
 	/*
-	 * Fall back to embedded doorbell only when platform MSI is unavailable
-	 * for this EPC.
+	 * The MSI-backed doorbell path currently targets the first EPF attached
+	 * to the EPC. Let non-first EPFs try the embedded doorbell instead.
 	 */
-	if (ret != -ENODEV)
-		return ret;
+	first_epf = list_first_entry_or_null(&epc->pci_epf, struct pci_epf,
+					     list);
+	if (first_epf == epf) {
+		ret = pci_epf_alloc_doorbell_msi(epf, num_db);
+		if (!ret)
+			return 0;
+
+		/*
+		 * Fall back to embedded doorbell only when platform MSI is
+		 * unavailable for this EPC.
+		 */
+		if (ret != -ENODEV)
+			return ret;
+	}
 
 	ret = pci_epf_alloc_doorbell_embedded(epf, num_db);
 	if (ret) {
-- 
2.51.0

Re: [PATCH 2/3] PCI: endpoint: pci-ep-msi: Make embedded doorbell IRQ exclusive

[sashiko-bot]

> Embedded doorbells may use the same virq for several messages. This does
> not mean the IRQ itself should be shared. Consumers only need to request
> each distinct virq once, as vNTB and pci-epf-test already do.
> 
> IRQF_SHARED would let different EPFs register handlers for the same IRQ.
> The embedded doorbell interface cannot tell which EPF a write targets,
> so every handler would handle it.
> 
> Only the first EPF can allocate doorbells today, so this cannot happen
> yet. Drop IRQF_SHARED before extending the embedded backend to non-first
> EPFs. A second request for the same IRQ will then fail with -EBUSY
> instead of attaching another handler.
> 
> Signed-off-by: Koichiro Den <den@valinux.co.jp>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260728172306.2751813-1-den@valinux.co.jp?part=2

Re: [PATCH 3/3] PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing cleanup of doorbell IRQ and message memory in `pci_epf_test_unbind` leads to a Use-After-Free.
--

--- Patch [3]: [PATCH 3/3] PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells ---
commit 5b37917c476823bd8dce57c08e171daf54ed33f5
Author: Koichiro Den <den@valinux.co.jp>

PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells

This patch allows non-first endpoint functions attached to an endpoint
controller to utilize embedded doorbells. It removes the restriction that
only the first endpoint function can allocate doorbells, falling back to
the embedded doorbell path directly for subsequent functions.

> diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c
> index a123a189b4e65..5ea7eef9794f7 100644
> --- a/drivers/pci/endpoint/pci-ep-msi.c
> +++ b/drivers/pci/endpoint/pci-ep-msi.c
> @@ -199,28 +199,31 @@ static int pci_epf_alloc_doorbell_msi(struct pci_epf *epf, u16 num_db)
>  int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 num_db)
>  {

[ ... ]

>  	/*
> -	 * Fall back to embedded doorbell only when platform MSI is unavailable
> -	 * for this EPC.
> +	 * The MSI-backed doorbell path currently targets the first EPF attached
> +	 * to the EPC. Let non-first EPFs try the embedded doorbell instead.
>  	 */
> -	if (ret != -ENODEV)
> -		return ret;
> +	first_epf = list_first_entry_or_null(&epc->pci_epf, struct pci_epf,
> +					     list);
> +	if (first_epf == epf) {
> +		ret = pci_epf_alloc_doorbell_msi(epf, num_db);
> +		if (!ret)
> +			return 0;
> +
> +		/*
> +		 * Fall back to embedded doorbell only when platform MSI is
> +		 * unavailable for this EPC.
> +		 */
> +		if (ret != -ENODEV)
> +			return ret;
> +	}
>  
>  	ret = pci_epf_alloc_doorbell_embedded(epf, num_db);
>  	if (ret) {

[Severity: High]
This is a pre-existing issue, but as this patch makes it possible for more
devices to allocate doorbells, can this lead to a use-after-free when an
endpoint is unbound while a doorbell is active?

Looking at pci_epf_test_unbind() in drivers/pci/endpoint/functions/pci-epf-test.c:

drivers/pci/endpoint/functions/pci-epf-test.c:pci_epf_test_unbind() {
    struct pci_epf_test *epf_test = epf_get_drvdata(epf);
    struct pci_epc *epc = epf->epc;

    cancel_delayed_work_sync(&epf_test->cmd_handler);
    if (epc->init_complete) {
        pci_epf_test_clean_dma_chan(epf_test);
        pci_epf_test_clear_bar(epf);
    }
    pci_epf_test_free_space(epf);
}

If the host sends a command to enable the doorbell, a doorbell is allocated
and an IRQ is requested via request_threaded_irq().

If the user then unbinds the driver via configfs, pci_epf_test_unbind()
does not check for or clean up the active doorbell. This means free_irq()
and pci_epf_free_doorbell() are not called.

When the endpoint function device is later destroyed via configfs rmdir,
the epf_test structure is freed. However, the IRQ handler remains
registered with the freed epf_test pointer.

Does this allow a remote host to trigger a use-after-free kernel panic by
sending an MSI doorbell interrupt to the endpoint after unbind?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260728172306.2751813-1-den@valinux.co.jp?part=3

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] An off-by-one loop initialization error in `epf_ntb_init_epc_bar()` leaves memory window BARs beyond the first one unassigned (`NO_BAR` or `-1`). This results in negative array indexing and out-of-bounds memory accesses when `epf_ntb_mw_bar_init()` and `vntb_epf_mw_set_trans()` index into `ntb->epf->bar[barno]`.
- [High] Missing NULL checks for `epc_features` returned by `pci_epc_get_features()`, leading to a kernel panic if the underlying EPC driver does not implement the `.get_features` callback.
- [High] The virtual PCI driver `vntb_pci_driver` lacks a `.remove` callback, resulting in resource leaks of the virtual PCI bus and Use-After-Free of the NTB device structures upon module unload or EPF unbind.
--

--- Patch [1]: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming ---
commit fc9ce298c179bf87e88da3f2964f2fbbd870806a
Author: Koichiro Den <den@valinux.co.jp>
PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

This commit updates vntb_epf_mw_set_trans() to pass the endpoint function's
own func_no and vfunc_no instead of hardcoded zeros when programming the
memory-window BAR. This ensures the BAR is correctly configured when the
virtual NTB endpoint function is bound to a function other than PF0.

> diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> index c3caec927d748..fba65abfb6b21 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
>  	epf_bar->barno = barno;
>  	epf_bar->size = size;

[Severity: High]
This isn't a bug introduced by this patch, but is it possible for barno to
be -1 here, leading to an out-of-bounds write?

Looking at epf_ntb_init_epc_bar(), the loop iterating over optional memory
window BARs incorrectly starts from BAR_MW1 instead of BAR_MW2:

drivers/pci/endpoint/functions/pci-epf-vntb.c:epf_ntb_init_epc_bar() {
	...
	for (bar = BAR_MW1, i = 1; i < num_mws; bar++, i++) {
	...
}

If a user configures num_mws > 1, this leaves BAR_MW2 unassigned (-1). Later,
when vntb_epf_mw_set_trans() is called for idx = 1, barno is evaluated as -1.
Will epf_bar = &ntb->epf->bar[barno] then access out of bounds and corrupt
memory?

> -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> +			      ntb->epf->vfunc_no, epf_bar);
>  	if (ret) {
>  		dev_err(dev, "failure set mw trans\n");
>  		return ret;

Also, while reviewing this file, I noticed a few other pre-existing issues:

[Severity: High]
This is a pre-existing issue, but how is the case where epc_features returns
NULL handled?

drivers/pci/endpoint/functions/pci-epf-vntb.c:epf_ntb_configure_interrupt() {
	...
	epc_features = pci_epc_get_features(ntb->epf->epc, ntb->epf->func_no, ntb->epf->vfunc_no);

	if (!(epc_features->msix_capable || epc_features->msi_capable)) {
	...
}

If the underlying EPC driver lacks the .get_features callback,
pci_epc_get_features() returns NULL, which would cause a NULL pointer
dereference here. A similar dereference happens in
epf_ntb_config_spad_bar_alloc().


[Severity: High]
This is a pre-existing issue, but does vntb_pci_driver intentionally omit a
.remove callback?

drivers/pci/endpoint/functions/pci-epf-vntb.c:
	static struct pci_driver vntb_pci_driver = {
		.name           = "pci-vntb",
		.id_table       = pci_vntb_table,
		.probe          = pci_vntb_probe,
	};

When epf_ntb_unbind() unregisters this driver:

drivers/pci/endpoint/functions/pci-epf-vntb.c:epf_ntb_unbind() {
	...
	pci_unregister_driver(&vntb_pci_driver);
}

Since there is no .remove callback, ntb_unregister_device() is never called to
clean up the virtual PCI bus and its associated devices. Could this lead to a
resource leak and dangling sysdata pointers upon module unload or EPF unbind?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260728172306.2751813-1-den@valinux.co.jp?part=1

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Frank Li]

On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> vntb_epf_mw_set_trans() programs the memory-window BAR through
> pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> lands on the wrong function whenever the vNTB EPF is bound to anything
> but PF0. The other BAR programming sites in the vNTB driver already pass
> the EPF's own numbers.
>
> Pass the EPF's own func_no/vfunc_no here as well.
>
> Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---

Reviewed-by: Frank Li <Frank.Li@nxp.com>

>  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> index c3caec927d74..fba65abfb6b2 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
>  	epf_bar->barno = barno;
>  	epf_bar->size = size;
>
> -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> +			      ntb->epf->vfunc_no, epf_bar);
>  	if (ret) {
>  		dev_err(dev, "failure set mw trans\n");
>  		return ret;
> --
> 2.51.0
>

Re: [PATCH 2/3] PCI: endpoint: pci-ep-msi: Make embedded doorbell IRQ exclusive

[Frank Li]

On Wed, Jul 29, 2026 at 02:23:05AM +0900, Koichiro Den wrote:
> Embedded doorbells may use the same virq for several messages. This does
> not mean the IRQ itself should be shared. Consumers only need to request
> each distinct virq once, as vNTB and pci-epf-test already do.
>
> IRQF_SHARED would let different EPFs register handlers for the same IRQ.
> The embedded doorbell interface cannot tell which EPF a write targets,
> so every handler would handle it.
>
> Only the first EPF can allocate doorbells today, so this cannot happen
> yet. Drop IRQF_SHARED before extending the embedded backend to non-first
> EPFs. A second request for the same IRQ will then fail with -EBUSY
> instead of attaching another handler.
>
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---

Reviewed-by: Frank Li <Frank.Li@nxp.com>

>  drivers/pci/endpoint/pci-ep-msi.c | 9 +++++----
>  include/linux/pci-epf.h           | 3 ++-
>  2 files changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c
> index 0855c7930abb..a123a189b4e6 100644
> --- a/drivers/pci/endpoint/pci-ep-msi.c
> +++ b/drivers/pci/endpoint/pci-ep-msi.c
> @@ -10,7 +10,6 @@
>  #include <linux/cleanup.h>
>  #include <linux/device.h>
>  #include <linux/export.h>
> -#include <linux/interrupt.h>
>  #include <linux/irqdomain.h>
>  #include <linux/module.h>
>  #include <linux/msi.h>
> @@ -112,8 +111,11 @@ static int pci_epf_alloc_doorbell_embedded(struct pci_epf *epf, u16 num_db)
>  	 *
>  	 * Still, pci_epf_alloc_doorbell() allows requesting multiple doorbells.
>  	 * For such backends we replicate the same address/data for each entry
> -	 * and mark the IRQ as shared (IRQF_SHARED). Consumers must treat them
> -	 * as equivalent "kick" doorbells.
> +	 * as equivalent "kick" doorbells. Consumers must request each distinct
> +	 * IRQ only once.
> +	 *
> +	 * Keep the IRQ exclusive because the source cannot identify which EPF
> +	 * rang the doorbell.
>  	 */
>  	for (i = 0; i < num_db; i++)
>  		msg[i] = (struct pci_epf_doorbell_msg) {
> @@ -121,7 +123,6 @@ static int pci_epf_alloc_doorbell_embedded(struct pci_epf *epf, u16 num_db)
>  			.msg.address_hi = (u32)(addr >> 32),
>  			.msg.data = doorbell->u.db_mmio.data,
>  			.virq = doorbell->u.db_mmio.irq,
> -			.irq_flags = IRQF_SHARED,
>  			.type = PCI_EPF_DOORBELL_EMBEDDED,
>  			.bar = doorbell->bar,
>  			.offset = (doorbell->bar == NO_BAR) ? 0 :
> diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
> index 8a6c64a35890..e5b9a79beb93 100644
> --- a/include/linux/pci-epf.h
> +++ b/include/linux/pci-epf.h
> @@ -163,7 +163,8 @@ enum pci_epf_doorbell_type {
>   *       For MSI-backed doorbells this is the MSI message, while for
>   *       "embedded" doorbells this represents an MMIO write that asserts
>   *       an interrupt on the EP side.
> - * @virq: IRQ number of this doorbell message
> + * @virq: IRQ number of this doorbell message. Multiple messages may use the
> + *        same IRQ; consumers must request each distinct IRQ only once.
>   * @irq_flags: Required flags for request_irq()/request_threaded_irq().
>   *             Callers may OR-in additional flags (e.g. IRQF_ONESHOT).
>   * @type: Doorbell type.
> --
> 2.51.0
>

Re: [PATCH 3/3] PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells

[Frank Li]

On Wed, Jul 29, 2026 at 02:23:06AM +0900, Koichiro Den wrote:
> pci_epf_alloc_doorbell() rejects every EPF that is not the first one
> attached to the EPC because the MSI-backed path can populate doorbell
> messages only for that EPF. This also prevents non-first EPFs from using
> the embedded doorbell backend.
>
> Keep the MSI-backed path limited to the first attached EPF. Let
> non-first EPFs skip it and try the embedded doorbell directly. The
> embedded IRQ is exclusive, so a second embedded user fails to request
> the IRQ instead of receiving another EPF's notifications.
>
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---

Reviewed-by: Frank Li <Frank.Li@nxp.com>

>  drivers/pci/endpoint/pci-ep-msi.c | 31 +++++++++++++++++--------------
>  1 file changed, 17 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c
> index a123a189b4e6..5ea7eef9794f 100644
> --- a/drivers/pci/endpoint/pci-ep-msi.c
> +++ b/drivers/pci/endpoint/pci-ep-msi.c
> @@ -199,28 +199,31 @@ static int pci_epf_alloc_doorbell_msi(struct pci_epf *epf, u16 num_db)
>  int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 num_db)
>  {
>  	struct pci_epc *epc = epf->epc;
> +	struct pci_epf *first_epf;
>  	struct device *dev = &epf->dev;
>  	int ret;
>
> -	/* TODO: Multi-EPF support */
> -	if (list_first_entry_or_null(&epc->pci_epf, struct pci_epf, list) != epf) {
> -		dev_err(dev, "Doorbell doesn't support multiple EPF\n");
> -		return -EINVAL;
> -	}
> -
>  	if (epf->db_msg)
>  		return -EBUSY;
>
> -	ret = pci_epf_alloc_doorbell_msi(epf, num_db);
> -	if (!ret)
> -		return 0;
> -
>  	/*
> -	 * Fall back to embedded doorbell only when platform MSI is unavailable
> -	 * for this EPC.
> +	 * The MSI-backed doorbell path currently targets the first EPF attached
> +	 * to the EPC. Let non-first EPFs try the embedded doorbell instead.
>  	 */
> -	if (ret != -ENODEV)
> -		return ret;
> +	first_epf = list_first_entry_or_null(&epc->pci_epf, struct pci_epf,
> +					     list);
> +	if (first_epf == epf) {
> +		ret = pci_epf_alloc_doorbell_msi(epf, num_db);
> +		if (!ret)
> +			return 0;
> +
> +		/*
> +		 * Fall back to embedded doorbell only when platform MSI is
> +		 * unavailable for this EPC.
> +		 */
> +		if (ret != -ENODEV)
> +			return ret;
> +	}
>
>  	ret = pci_epf_alloc_doorbell_embedded(epf, num_db);
>  	if (ret) {
> --
> 2.51.0
>

Re: [PATCH 0/3] PCI: endpoint: Support vNTB as a non-first EPF

[Manivannan Sadhasivam]

On Wed, 29 Jul 2026 02:23:03 +0900, Koichiro Den wrote:
> This series allows a vNTB endpoint function to be instantiated as PF1+.
> 
> vntb_epf_mw_set_trans() programs the memory-window BAR with hardcoded
> function numbers (0, 0), so binding the EPF to any other function
> programs the wrong one. Patch 1 resolves it.
> 
> The platform-MSI doorbell path can currently populate messages only for
> the first EPF attached to the EPC. Supporting multiple EPFs in that path
> requires separate work. This series leaves it unchanged and lets a
> non-first EPF use the embedded doorbell instead.
> 
> [...]

Applied, thanks!

[1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming
      commit: afe2aa9dc93d904730bab63f5ccb514a318e6464
[2/3] PCI: endpoint: pci-ep-msi: Make embedded doorbell IRQ exclusive
      commit: 52c73e3cecd0603346e176da1d04a0c4d3c98f1b
[3/3] PCI: endpoint: pci-ep-msi: Let non-first EPFs use embedded doorbells
      commit: 68fcdf30a1582574fe67e0b02488f8ba0d4d1242

Best regards,
-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Bjorn Helgaas]

On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> vntb_epf_mw_set_trans() programs the memory-window BAR through
> pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> lands on the wrong function whenever the vNTB EPF is bound to anything
> but PF0. The other BAR programming sites in the vNTB driver already pass
> the EPF's own numbers.
> 
> Pass the EPF's own func_no/vfunc_no here as well.

We're referring to these as "PF" and "VF" in the subject and "physical
endpoint function" and "virtual endpoint function" in the
pci_epc_set_bar() kernel-doc, but I don't think these have anything to
do with the SR-IOV PF and VF concepts, do they?

I don't have a better naming suggestion, but this is slightly
confusing.

> Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---
>  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> index c3caec927d74..fba65abfb6b2 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
>  	epf_bar->barno = barno;
>  	epf_bar->size = size;
>  
> -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> +			      ntb->epf->vfunc_no, epf_bar);
>  	if (ret) {
>  		dev_err(dev, "failure set mw trans\n");
>  		return ret;
> -- 
> 2.51.0
> 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Bjorn Helgaas]

On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> vntb_epf_mw_set_trans() programs the memory-window BAR through
> pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> lands on the wrong function whenever the vNTB EPF is bound to anything
> but PF0. The other BAR programming sites in the vNTB driver already pass
> the EPF's own numbers.
> 
> Pass the EPF's own func_no/vfunc_no here as well.
> 
> Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---
>  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> index c3caec927d74..fba65abfb6b2 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
>  	epf_bar->barno = barno;
>  	epf_bar->size = size;
>  
> -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> +			      ntb->epf->vfunc_no, epf_bar);

Thanks for this fix!

If anybody is bored, I think this file could be somewhat improved by
adding local "epf = ntb->epf" variables to reduce the repetition of
"ntb->epf", as epf_ntb_config_spad_bar_alloc() does.

Even then there's a lot of repetition, but I think much of it is there
to make room for the PRIMARY_INTERFACE/SECONDARY_INTERFACE stuff for
NTBs.

>  	if (ret) {
>  		dev_err(dev, "failure set mw trans\n");
>  		return ret;
> -- 
> 2.51.0
> 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Koichiro Den]

On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > lands on the wrong function whenever the vNTB EPF is bound to anything
> > but PF0. The other BAR programming sites in the vNTB driver already pass
> > the EPF's own numbers.
> > 
> > Pass the EPF's own func_no/vfunc_no here as well.
> 
> We're referring to these as "PF" and "VF" in the subject and "physical
> endpoint function" and "virtual endpoint function" in the
> pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> do with the SR-IOV PF and VF concepts, do they?

Not in the specific case that motivated this series. But AFAICT, the EPC API
uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
associated with the PF selected by func_no.

> 
> I don't have a better naming suggestion, but this is slightly
> confusing.

Perhaps a better subject might be:

  PCI: endpoint: pci-epf-vntb: Pass (func_no, vfunc_no) when programming BARs

Best regards,
Koichiro

> 
> > Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > ---
> >  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > index c3caec927d74..fba65abfb6b2 100644
> > --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
> >  	epf_bar->barno = barno;
> >  	epf_bar->size = size;
> >  
> > -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> > +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> > +			      ntb->epf->vfunc_no, epf_bar);
> >  	if (ret) {
> >  		dev_err(dev, "failure set mw trans\n");
> >  		return ret;
> > -- 
> > 2.51.0
> > 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Bjorn Helgaas]

On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > the EPF's own numbers.
> > > 
> > > Pass the EPF's own func_no/vfunc_no here as well.
> > 
> > We're referring to these as "PF" and "VF" in the subject and "physical
> > endpoint function" and "virtual endpoint function" in the
> > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > do with the SR-IOV PF and VF concepts, do they?
> 
> Not in the specific case that motivated this series. But AFAICT, the EPC API
> uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> associated with the PF selected by func_no.

Now I'm even more confused :)

Are you saying that a non-zero vfunc_no always identifies an SR-IOV
VF?  And there's some dependency on that?  I don't any mention of
"iov" in drivers/pci/endpoint/.

> > I don't have a better naming suggestion, but this is slightly
> > confusing.
> 
> Perhaps a better subject might be:
> 
>   PCI: endpoint: pci-epf-vntb: Pass (func_no, vfunc_no) when programming BARs
> 
> Best regards,
> Koichiro
> 
> > 
> > > Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > ---
> > >  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > index c3caec927d74..fba65abfb6b2 100644
> > > --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
> > >  	epf_bar->barno = barno;
> > >  	epf_bar->size = size;
> > >  
> > > -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> > > +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> > > +			      ntb->epf->vfunc_no, epf_bar);
> > >  	if (ret) {
> > >  		dev_err(dev, "failure set mw trans\n");
> > >  		return ret;
> > > -- 
> > > 2.51.0
> > > 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Koichiro Den]

On Tue, Sep 08, 2026 at 09:23:11PM -0500, Bjorn Helgaas wrote:
> On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> > On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > > the EPF's own numbers.
> > > > 
> > > > Pass the EPF's own func_no/vfunc_no here as well.
> > > 
> > > We're referring to these as "PF" and "VF" in the subject and "physical
> > > endpoint function" and "virtual endpoint function" in the
> > > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > > do with the SR-IOV PF and VF concepts, do they?
> > 
> > Not in the specific case that motivated this series. But AFAICT, the EPC API
> > uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> > SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> > ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> > associated with the PF selected by func_no.
> 
> Now I'm even more confused :)
> 
> Are you saying that a non-zero vfunc_no always identifies an SR-IOV
> VF?  And there's some dependency on that?  I don't any mention of
> "iov" in drivers/pci/endpoint/.

Yes, that is my understanding of the current in-tree implementation. You're
right that drivers/pci/endpoint/ itself contains no explicit reference to
SR-IOV. E.g. pci_epf_add_vepf() just calls it a "virtual EP function".
So my saying was kind of assumptive, but I still think the same because:

- The support was introduced for SR-IOV:
  https://lore.kernel.org/r/20210819123343.1951-1-kishon@ti.com/

- The core rejects a non-zero vfunc_no unless the EPC provides max_vfs, and
  Cadence is the only in-tree EPC driver I found that does so. For example,
  cdns_pcie_ep_set_bar() calls cdns_pcie_get_fn_from_vfn(), which uses the
  SR-IOV First VF Offset and VF Stride for a non-zero vfn.

Best regards,
Koichiro

> 
> > > I don't have a better naming suggestion, but this is slightly
> > > confusing.
> > 
> > Perhaps a better subject might be:
> > 
> >   PCI: endpoint: pci-epf-vntb: Pass (func_no, vfunc_no) when programming BARs
> > 
> > Best regards,
> > Koichiro
> > 
> > > 
> > > > Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> > > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > > ---
> > > >  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
> > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > index c3caec927d74..fba65abfb6b2 100644
> > > > --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
> > > >  	epf_bar->barno = barno;
> > > >  	epf_bar->size = size;
> > > >  
> > > > -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> > > > +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> > > > +			      ntb->epf->vfunc_no, epf_bar);
> > > >  	if (ret) {
> > > >  		dev_err(dev, "failure set mw trans\n");
> > > >  		return ret;
> > > > -- 
> > > > 2.51.0
> > > > 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Bjorn Helgaas]

On Wed, Sep 09, 2026 at 01:53:53PM +0900, Koichiro Den wrote:
> On Tue, Sep 08, 2026 at 09:23:11PM -0500, Bjorn Helgaas wrote:
> > On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> > > On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > > > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > > > the EPF's own numbers.
> > > > > 
> > > > > Pass the EPF's own func_no/vfunc_no here as well.
> > > > 
> > > > We're referring to these as "PF" and "VF" in the subject and "physical
> > > > endpoint function" and "virtual endpoint function" in the
> > > > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > > > do with the SR-IOV PF and VF concepts, do they?
> > > 
> > > Not in the specific case that motivated this series. But AFAICT, the EPC API
> > > uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> > > SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> > > ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> > > associated with the PF selected by func_no.
> > 
> > Now I'm even more confused :)
> > 
> > Are you saying that a non-zero vfunc_no always identifies an SR-IOV
> > VF?  And there's some dependency on that?  I don't any mention of
> > "iov" in drivers/pci/endpoint/.
> 
> Yes, that is my understanding of the current in-tree implementation. You're
> right that drivers/pci/endpoint/ itself contains no explicit reference to
> SR-IOV. E.g. pci_epf_add_vepf() just calls it a "virtual EP function".
> So my saying was kind of assumptive, but I still think the same because:
> 
> - The support was introduced for SR-IOV:
>   https://lore.kernel.org/r/20210819123343.1951-1-kishon@ti.com/
> 
> - The core rejects a non-zero vfunc_no unless the EPC provides max_vfs, and
>   Cadence is the only in-tree EPC driver I found that does so. For example,
>   cdns_pcie_ep_set_bar() calls cdns_pcie_get_fn_from_vfn(), which uses the
>   SR-IOV First VF Offset and VF Stride for a non-zero vfn.

Thanks, that's helpful.  I still have to work hard to change my point
of view from host-side drivers to endpoint drivers operating on the
other end of the link.  The fact that there are several interfaces
that need (func_no, vfunc_no) suggests that callers really do need to
understand what's going on, and maybe we should try to connect the
kernel-doc and abbreviations more closely with PCIe spec terms.

E.g., if "physical EP function" and "virtual EP function" refer to
SR-IOV PF and VF, maybe we should word them as "endpoint PF" or
"endpoint VF" (or "EP PF", "EP VF" for short).  If
"pci_epf_add_vepf()" adds an SR-IOV VF, maybe "pci_epf_add_vf()" would
be descriptive enough.  We already know we're on the endpoint because
of "epf", so we probably don't need another hint in "vepf", which
includes a "pf" that doesn't mean SR-IOV PF.

> > > > I don't have a better naming suggestion, but this is slightly
> > > > confusing.
> > > 
> > > Perhaps a better subject might be:
> > > 
> > >   PCI: endpoint: pci-epf-vntb: Pass (func_no, vfunc_no) when programming BARs
> > > 
> > > Best regards,
> > > Koichiro
> > > 
> > > > 
> > > > > Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> > > > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > > > ---
> > > > >  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
> > > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > index c3caec927d74..fba65abfb6b2 100644
> > > > > --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
> > > > >  	epf_bar->barno = barno;
> > > > >  	epf_bar->size = size;
> > > > >  
> > > > > -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> > > > > +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> > > > > +			      ntb->epf->vfunc_no, epf_bar);
> > > > >  	if (ret) {
> > > > >  		dev_err(dev, "failure set mw trans\n");
> > > > >  		return ret;
> > > > > -- 
> > > > > 2.51.0
> > > > > 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Koichiro Den]

On Wed, Sep 09, 2026 at 11:09:49AM -0500, Bjorn Helgaas wrote:
> On Wed, Sep 09, 2026 at 01:53:53PM +0900, Koichiro Den wrote:
> > On Tue, Sep 08, 2026 at 09:23:11PM -0500, Bjorn Helgaas wrote:
> > > On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> > > > On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > > > > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > > > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > > > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > > > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > > > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > > > > the EPF's own numbers.
> > > > > > 
> > > > > > Pass the EPF's own func_no/vfunc_no here as well.
> > > > > 
> > > > > We're referring to these as "PF" and "VF" in the subject and "physical
> > > > > endpoint function" and "virtual endpoint function" in the
> > > > > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > > > > do with the SR-IOV PF and VF concepts, do they?
> > > > 
> > > > Not in the specific case that motivated this series. But AFAICT, the EPC API
> > > > uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> > > > SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> > > > ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> > > > associated with the PF selected by func_no.
> > > 
> > > Now I'm even more confused :)
> > > 
> > > Are you saying that a non-zero vfunc_no always identifies an SR-IOV
> > > VF?  And there's some dependency on that?  I don't any mention of
> > > "iov" in drivers/pci/endpoint/.
> > 
> > Yes, that is my understanding of the current in-tree implementation. You're
> > right that drivers/pci/endpoint/ itself contains no explicit reference to
> > SR-IOV. E.g. pci_epf_add_vepf() just calls it a "virtual EP function".
> > So my saying was kind of assumptive, but I still think the same because:
> > 
> > - The support was introduced for SR-IOV:
> >   https://lore.kernel.org/r/20210819123343.1951-1-kishon@ti.com/
> > 
> > - The core rejects a non-zero vfunc_no unless the EPC provides max_vfs, and
> >   Cadence is the only in-tree EPC driver I found that does so. For example,
> >   cdns_pcie_ep_set_bar() calls cdns_pcie_get_fn_from_vfn(), which uses the
> >   SR-IOV First VF Offset and VF Stride for a non-zero vfn.
> 
> Thanks, that's helpful.  I still have to work hard to change my point
> of view from host-side drivers to endpoint drivers operating on the
> other end of the link.  The fact that there are several interfaces
> that need (func_no, vfunc_no) suggests that callers really do need to
> understand what's going on, and maybe we should try to connect the
> kernel-doc and abbreviations more closely with PCIe spec terms.
> 
> E.g., if "physical EP function" and "virtual EP function" refer to
> SR-IOV PF and VF, maybe we should word them as "endpoint PF" or
> "endpoint VF" (or "EP PF", "EP VF" for short).  If

I personally agree. That sounds reasonable and would help clarify things.

> "pci_epf_add_vepf()" adds an SR-IOV VF, maybe "pci_epf_add_vf()" would
> be descriptive enough.  We already know we're on the endpoint because
> of "epf", so we probably don't need another hint in "vepf", which
> includes a "pf" that doesn't mean SR-IOV PF.

True. But I'd also like to hear what the PCI EP maintainers think about this.

Best regards,
Koichiro

> 
> > > > > I don't have a better naming suggestion, but this is slightly
> > > > > confusing.
> > > > 
> > > > Perhaps a better subject might be:
> > > > 
> > > >   PCI: endpoint: pci-epf-vntb: Pass (func_no, vfunc_no) when programming BARs
> > > > 
> > > > Best regards,
> > > > Koichiro
> > > > 
> > > > > 
> > > > > > Fixes: e35f56bb0330 ("PCI: endpoint: Support NTB transfer between RC and EP")
> > > > > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > > > > ---
> > > > > >  drivers/pci/endpoint/functions/pci-epf-vntb.c | 3 ++-
> > > > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > > index c3caec927d74..fba65abfb6b2 100644
> > > > > > --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > > +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> > > > > > @@ -1427,7 +1427,8 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
> > > > > >  	epf_bar->barno = barno;
> > > > > >  	epf_bar->size = size;
> > > > > >  
> > > > > > -	ret = pci_epc_set_bar(ntb->epf->epc, 0, 0, epf_bar);
> > > > > > +	ret = pci_epc_set_bar(ntb->epf->epc, ntb->epf->func_no,
> > > > > > +			      ntb->epf->vfunc_no, epf_bar);
> > > > > >  	if (ret) {
> > > > > >  		dev_err(dev, "failure set mw trans\n");
> > > > > >  		return ret;
> > > > > > -- 
> > > > > > 2.51.0
> > > > > > 

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Manivannan Sadhasivam]

On Thu, Sep 10, 2026 at 01:56:53PM +0900, Koichiro Den wrote:
> On Wed, Sep 09, 2026 at 11:09:49AM -0500, Bjorn Helgaas wrote:
> > On Wed, Sep 09, 2026 at 01:53:53PM +0900, Koichiro Den wrote:
> > > On Tue, Sep 08, 2026 at 09:23:11PM -0500, Bjorn Helgaas wrote:
> > > > On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> > > > > On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > > > > > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > > > > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > > > > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > > > > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > > > > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > > > > > the EPF's own numbers.
> > > > > > > 
> > > > > > > Pass the EPF's own func_no/vfunc_no here as well.
> > > > > > 
> > > > > > We're referring to these as "PF" and "VF" in the subject and "physical
> > > > > > endpoint function" and "virtual endpoint function" in the
> > > > > > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > > > > > do with the SR-IOV PF and VF concepts, do they?
> > > > > 
> > > > > Not in the specific case that motivated this series. But AFAICT, the EPC API
> > > > > uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> > > > > SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> > > > > ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> > > > > associated with the PF selected by func_no.
> > > > 
> > > > Now I'm even more confused :)
> > > > 
> > > > Are you saying that a non-zero vfunc_no always identifies an SR-IOV
> > > > VF?  And there's some dependency on that?  I don't any mention of
> > > > "iov" in drivers/pci/endpoint/.
> > > 
> > > Yes, that is my understanding of the current in-tree implementation. You're
> > > right that drivers/pci/endpoint/ itself contains no explicit reference to
> > > SR-IOV. E.g. pci_epf_add_vepf() just calls it a "virtual EP function".
> > > So my saying was kind of assumptive, but I still think the same because:
> > > 
> > > - The support was introduced for SR-IOV:
> > >   https://lore.kernel.org/r/20210819123343.1951-1-kishon@ti.com/
> > > 
> > > - The core rejects a non-zero vfunc_no unless the EPC provides max_vfs, and
> > >   Cadence is the only in-tree EPC driver I found that does so. For example,
> > >   cdns_pcie_ep_set_bar() calls cdns_pcie_get_fn_from_vfn(), which uses the
> > >   SR-IOV First VF Offset and VF Stride for a non-zero vfn.
> > 

Your understanding is correct.

> > Thanks, that's helpful.  I still have to work hard to change my point
> > of view from host-side drivers to endpoint drivers operating on the
> > other end of the link.  The fact that there are several interfaces
> > that need (func_no, vfunc_no) suggests that callers really do need to
> > understand what's going on, and maybe we should try to connect the
> > kernel-doc and abbreviations more closely with PCIe spec terms.
> > 
> > E.g., if "physical EP function" and "virtual EP function" refer to
> > SR-IOV PF and VF, maybe we should word them as "endpoint PF" or
> > "endpoint VF" (or "EP PF", "EP VF" for short).  If
> 
> I personally agree. That sounds reasonable and would help clarify things.
> 
> > "pci_epf_add_vepf()" adds an SR-IOV VF, maybe "pci_epf_add_vf()" would
> > be descriptive enough.  We already know we're on the endpoint because
> > of "epf", so we probably don't need another hint in "vepf", which
> > includes a "pf" that doesn't mean SR-IOV PF.
> 
> True. But I'd also like to hear what the PCI EP maintainers think about this.
> 

Fine with me.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH 1/3] PCI: endpoint: pci-epf-vntb: Pass PF/VF number when BAR programming

[Koichiro Den]

On Thu, Sep 10, 2026 at 08:08:10AM +0200, Manivannan Sadhasivam wrote:
> On Thu, Sep 10, 2026 at 01:56:53PM +0900, Koichiro Den wrote:
> > On Wed, Sep 09, 2026 at 11:09:49AM -0500, Bjorn Helgaas wrote:
> > > On Wed, Sep 09, 2026 at 01:53:53PM +0900, Koichiro Den wrote:
> > > > On Tue, Sep 08, 2026 at 09:23:11PM -0500, Bjorn Helgaas wrote:
> > > > > On Wed, Sep 09, 2026 at 11:01:48AM +0900, Koichiro Den wrote:
> > > > > > On Tue, Sep 08, 2026 at 11:53:15AM -0500, Bjorn Helgaas wrote:
> > > > > > > On Wed, Jul 29, 2026 at 02:23:04AM +0900, Koichiro Den wrote:
> > > > > > > > vntb_epf_mw_set_trans() programs the memory-window BAR through
> > > > > > > > pci_epc_set_bar() with hardcoded function numbers (0, 0), so the BAR
> > > > > > > > lands on the wrong function whenever the vNTB EPF is bound to anything
> > > > > > > > but PF0. The other BAR programming sites in the vNTB driver already pass
> > > > > > > > the EPF's own numbers.
> > > > > > > > 
> > > > > > > > Pass the EPF's own func_no/vfunc_no here as well.
> > > > > > > 
> > > > > > > We're referring to these as "PF" and "VF" in the subject and "physical
> > > > > > > endpoint function" and "virtual endpoint function" in the
> > > > > > > pci_epc_set_bar() kernel-doc, but I don't think these have anything to
> > > > > > > do with the SR-IOV PF and VF concepts, do they?
> > > > > > 
> > > > > > Not in the specific case that motivated this series. But AFAICT, the EPC API
> > > > > > uses the same (func_no, vfunc_no) pair to cover both ordinary functions and
> > > > > > SR-IOV PFs/VFs scenarios. With vfunc_no == 0, func_no can identify either an
> > > > > > ordinary function or an SR-IOV PF. A non-zero vfunc_no identifies a VF
> > > > > > associated with the PF selected by func_no.
> > > > > 
> > > > > Now I'm even more confused :)
> > > > > 
> > > > > Are you saying that a non-zero vfunc_no always identifies an SR-IOV
> > > > > VF?  And there's some dependency on that?  I don't any mention of
> > > > > "iov" in drivers/pci/endpoint/.
> > > > 
> > > > Yes, that is my understanding of the current in-tree implementation. You're
> > > > right that drivers/pci/endpoint/ itself contains no explicit reference to
> > > > SR-IOV. E.g. pci_epf_add_vepf() just calls it a "virtual EP function".
> > > > So my saying was kind of assumptive, but I still think the same because:
> > > > 
> > > > - The support was introduced for SR-IOV:
> > > >   https://lore.kernel.org/r/20210819123343.1951-1-kishon@ti.com/
> > > > 
> > > > - The core rejects a non-zero vfunc_no unless the EPC provides max_vfs, and
> > > >   Cadence is the only in-tree EPC driver I found that does so. For example,
> > > >   cdns_pcie_ep_set_bar() calls cdns_pcie_get_fn_from_vfn(), which uses the
> > > >   SR-IOV First VF Offset and VF Stride for a non-zero vfn.
> > > 
> 
> Your understanding is correct.
> 
> > > Thanks, that's helpful.  I still have to work hard to change my point
> > > of view from host-side drivers to endpoint drivers operating on the
> > > other end of the link.  The fact that there are several interfaces
> > > that need (func_no, vfunc_no) suggests that callers really do need to
> > > understand what's going on, and maybe we should try to connect the
> > > kernel-doc and abbreviations more closely with PCIe spec terms.
> > > 
> > > E.g., if "physical EP function" and "virtual EP function" refer to
> > > SR-IOV PF and VF, maybe we should word them as "endpoint PF" or
> > > "endpoint VF" (or "EP PF", "EP VF" for short).  If
> > 
> > I personally agree. That sounds reasonable and would help clarify things.
> > 
> > > "pci_epf_add_vepf()" adds an SR-IOV VF, maybe "pci_epf_add_vf()" would
> > > be descriptive enough.  We already know we're on the endpoint because
> > > of "epf", so we probably don't need another hint in "vepf", which
> > > includes a "pf" that doesn't mean SR-IOV PF.
> > 
> > True. But I'd also like to hear what the PCI EP maintainers think about this.
> > 
> 
> Fine with me.

All right, thanks for the comment. Sounds like a small refactoring task then.
I'll try to pick it up when I have got some spare cycles, unless someone gets to
it first.

Best regards,
Koichiro

> 
> - Mani
> 
> -- 
> மணிவண்ணன் சதாசிவம்