Re: [PATCH v6 0/8] PCI: endpoint: pci-ep-msi: Add embedded doorbell fallback

[Niklas Cassel <cassel@kernel.org>]

On Tue, Feb 10, 2026 at 11:07:16PM +0900, Koichiro Den wrote:
> On Tue, Feb 10, 2026 at 01:24:29PM +0100, Niklas Cassel wrote:
> > On Mon, Feb 09, 2026 at 09:53:08PM +0900, Koichiro Den wrote:
> > > Tested on
> > > =========
> > > 
> > > I tested the embedded (DMA) doorbell fallback path (via pci-epf-test) on
> > > R-Car Spider boards:
> > > 
> > >   $ ./pci_endpoint_test -t DOORBELL_TEST
> > >   TAP version 13
> > >   1..1
> > >   # Starting 1 tests from 1 test cases.
> > >   #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
> > >   #            OK  pcie_ep_doorbell.DOORBELL_TEST
> > >   ok 1 pcie_ep_doorbell.DOORBELL_TEST
> > >   # PASSED: 1 / 1 tests passed.
> > >   # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
> > > 
> > > with the following message observed on the EP side:
> > > 
> > >   [   80.464653] pci_epf_test pci_epf_test.0: Using embedded (DMA) doorbell fallback
> > > 
> > > (Note: for the test to pass on R-Car Spider, one of the following was required:
> > >  - echo 1048576 > functions/pci_epf_test/func1/pci_epf_test.0/bar2_size
> > >  - apply https://lore.kernel.org/all/20251023072217.901888-1-den@valinux.co.jp/)
> > 
> > I applied this series on top of branch pci/controller/dwc
> > on Rock 5B (pcie-dw-rockchip.c).
> > 
> > On EP side:
> > [   39.218533] pci_epf_test pci_epf_test.0: Can't find MSI domain for EPC
> > [   39.219125] pci_epf_test pci_epf_test.0: Using embedded (DMA) doorbell fallback
> > 
> > On RC side:
> > #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
> > [   40.297892] pci-endpoint-test 0000:01:00.0: Failed to trigger doorbell in endpoint
> > # pci_endpoint_test.c:279:DOORBELL_TEST:Expected 0 (0) == ret (-22)
> > # pci_endpoint_test.c:279:DOORBELL_TEST:Test failed for Doorbell
> > 
> > # DOORBELL_TEST: Test failed
> > #          FAIL  pcie_ep_doorbell.DOORBELL_TEST
> > not ok 23 pcie_ep_doorbell.DOORBELL_TEST
> > 
> > Any suggestions?
> > 
> > (All BARs in pcie-dw-rockchip.c is marked as BAR_RESIZABLE.)
> 
> Thank you for testing.
> 
> If the failure was observed in a scenario other than a plain
> `./pci_endpoint_test -t DOORBELL_TEST`, could you please try again with [1]
> applied as well?
> 
> [1] https://lore.kernel.org/linux-pci/20260202145407.503348-1-den@valinux.co.jp/

I applied that series, but I got the same problem.

I added debug, and the EP side does use the correct address for the eDMA:
[   26.279457] msg_addr: 0xa403800a0
[   26.279898] phys_addr: 0xa40300000 offset: 0x800a0


If I write to the msg_addr directly on the EP using devmem, I do see the print
that I added in the IRQ handler:
# devmem 0xa403800a0 32 0
[  155.861989] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  158.809160] dw_edma_interrupt_emulated:696
[  158.809543] pci_epf_test_doorbell_primary:729
# [  158.809986] pci_epf_test_doorbell_handler:703
# devmem 0xa403800a0 32 0
[  161.241326] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  163.466054] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  167.378662] dw_edma_interrupt_emulated:696
[  167.379045] pci_epf_test_doorbell_primary:729
# [  167.379512] pci_epf_test_doorbell_handler:703
# devmem 0xa403800a0 32 0
[  168.880179] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  170.492176] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  171.729154] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  173.481271] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  174.985787] dw_edma_interrupt_emulated:696
# devmem 0xa403800a0 32 0
[  176.517131] dw_edma_interrupt_emulated:696
[  176.517511] pci_epf_test_doorbell_primary:729
# [  176.517963] pci_epf_test_doorbell_handler:703

But not on every write....

I'm not sure, but could this perhaps be because we are missing this patch:
https://lore.kernel.org/dmaengine/20260105075904.1254012-1-den@valinux.co.jp/

# dmesg | grep eDMA
[    1.243339] rockchip-dw-pcie a40000000.pcie-ep: eDMA: unroll T, 2 wr, 2 rd

# cat /proc/interrupts | grep edma
 53:          8          0          0          0          0          0          0          0    GICv3 303 Level     dw-edma-core:a40000000.pcie-ep, pci-ep-test-doorbell
 54:          7          0          0          0          0          0          0          0    GICv3 304 Level     dw-edma-core:a40000000.pcie-ep
 55:         15          0          0          0          0          0          0          0    GICv3 301 Level     dw-edma-core:a40000000.pcie-ep
 56:          7          0          0          0          0          0          0          0    GICv3 302 Level     dw-edma-core:a40000000.pcie-ep



Anyway, I was still curious why this did never worked when writing from the
host side, even when running the test case many, many times.
AFAICT, the inbound translation looks correct.

RK3588 (pcie-dw-rockchip.c) exposes the DMA registers in BAR4 by default.
If I hack pci-epf-test on top of your patch to unconditionally return BAR4 with
offset 0xa0, it works. So my best guess is that the fixed inbound translation
in BAR4 (to the eDMA registers) somehow messes with the inbound translation if
another BAR tries to use an inbound translation to the eDMA registers as well.


Kind regards,
Niklas