Re: [PATCH] PCI: dwc: ep: Cache MSI outbound iATU mapping

[Shawn Lin <shawn.lin@rock-chips.com>]

在 2025/12/23 星期二 12:35, Krishna Chaitanya Chundru 写道:
> 
> 
> On 12/23/2025 6:42 AM, Shawn Lin wrote:
>> 在 2025/12/22 星期一 21:00, Niklas Cassel 写道:
>>> + Shawn
>>>
>>> On Mon, Dec 22, 2025 at 05:53:27PM +0530, Krishna Chaitanya Chundru 
>>> wrote:
>>>>
>>>>
>>>> On 12/22/2025 4:41 PM, Niklas Cassel wrote:
>>>>> On Mon, Dec 22, 2025 at 03:28:30PM +0530, Manivannan Sadhasivam wrote:
>>>>>>> Use the MSIX doorbell method which will not use iATU at all,
>>>>>>> dw_pcie_ep_raise_msix_irq_doorbell().
>>>>>>>
>>>>>> I think this is the safe bet since this feature doesn't seem like 
>>>>>> an optional
>>>>>> one.
>>>>>>
>>>>>> Niklas, if you can just fix MSI in this patch and leave out MSI-X 
>>>>>> for the vendor
>>>>>> drivers to transition to doorbell, I'm OK to merge it. Otherwise, 
>>>>>> I don't know
>>>>>> how you can reliably fix MSI-X generation with AXI slave interface.
>>>>> FWIW, I did try to simply change:
>>>>>
>>>>> diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/ 
>>>>> drivers/pci/controller/dwc/pcie-dw-rockchip.c
>>>>> index 8f2cc1ef25e3..00770f9786e3 100644
>>>>> --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c
>>>>> +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c
>>>>> @@ -319,7 +319,8 @@ static int rockchip_pcie_raise_irq(struct 
>>>>> dw_pcie_ep *ep, u8 func_no,
>>>>>           case PCI_IRQ_MSI:
>>>>>                   return dw_pcie_ep_raise_msi_irq(ep, func_no, 
>>>>> interrupt_num);
>>>>>           case PCI_IRQ_MSIX:
>>>>> -               return dw_pcie_ep_raise_msix_irq(ep, func_no, 
>>>>> interrupt_num);
>>>>> +               return dw_pcie_ep_raise_msix_irq_doorbell(ep, func_no,
>>>>> + interrupt_num);
>>>>>           default:
>>>>>                   dev_err(pci->dev, "UNKNOWN IRQ type\n");
>>>>>           }
>>>>>
>>>>>
>>>>> For the pcie-dw-rockchip driver, but it is not working:
>>>>> [  130.042849] nvme nvme0: I/O tag 0 (1000) QID 0 timeout, 
>>>>> completion polled
>>>>>
>>>>> Without this change, things work.
>>>>>
>>>>> Perhaps this feature is not an optional one, but at least we will 
>>>>> require
>>>>> more changes than a simple one liner.
>>>> Hi Niklas,
>>>>
>>>> It should be automatic only, no extra configurations should be
>>>> required, I believe your
>>>> HW doesn't support this feature, from spec 6..0a, sec 3.9.1.3
>>>> iMSIX-TX: Integrated MSI-X Transmit (USP)
>>>> I believe your HW is not generated with MSIX_TABLE_EN =1. In that
>>>> case you can't use this feature.
>>>
>>> Looking at the RK3588 TRM, it does have register:
>>> USP_PCIE_PL_MSIX_DOORBELL_OFF
>>> Address: Operational Base + offset (0x0248)
>>>
>>> Port Logic registers start at offset 0x700 on this SoC,
>>> so 0x700 + 0x248 == 0x948, which matches:
>>> drivers/pci/controller/dwc/pcie-designware.h:#define 
>>> PCIE_MSIX_DOORBELL         0x948
>>>
>>> I don't think the TRM would include this register if the
>>> DWC coere was not generated with MSIX_TABLE_EN=1.
>>>
>>
>> I checked the IP configurtion parameters for RK3588 DM controller for
>> sure, it sets MSIX_TABLE_EN=1.
>>
>> Looking into dw_pcie_ep_raise_msix_irq_doorbell(), it doesn't seem to
>> match the dwc databook. No matter for non-AXI mode or AXI access mode,
>> shouldn't we need to generate a MSI-X table RAM with data/address/ 
>> vector/TC in advanced? Am I missing anything because I didn't look
> The MSI-X table will updated automatically when host updates the MSI-X 
> table, when MSI-X is enabled
> by host.

Thanks for these details.
I re-read the databook, especially "Figure 3-49 iMSIX-TX: MSIX
Transmit ". Yes, it's updated automatically when host write access
it, which either comes from RX or local DBI(for debug purpose).


> 
> - Krishna Chaitanya.
>> too much regarding to the EPC side?
>>
>>> Shawn, any suggestions?
>>> For full thread, see:
>>> https://lore.kernel.org/linux-pci/3b34aa66- 
>>> a418-4f6b-930a-0728d87d79b6@oss.qualcomm.com/T/#t
>>>
>>>
>>> FWIW, Krishna Chaitanya, I did try the 
>>> dw_pcie_ep_raise_msix_irq_doorbell()
>>> change above also with the pci-epf-test EPF driver, and it also 
>>> caused the
>>> pci-epf-test driver to stop working.
>>>
>>>
>>> Kind regards,
>>> Niklas
>>>
>>
> 
>