Re: [PATCH v7 1/3] PCI: AtomicOps: Do not enable requests by RCiEPs

["Kuehling, Felix" <felix.kuehling@amd.com>]

On 2026-03-30 17:42, Bjorn Helgaas wrote:
> [+to amdgpu, bnxe_re, mlx5 IB, qedr, mlx5 maintainers]
>
> On Mon, Mar 30, 2026 at 03:09:44PM +0200, Gerd Bayer wrote:
>> Since root complex integrated end points (RCiEPs) attach to a bus that
>> has no bridge device describing the root port, the capability to
>> complete AtomicOps requests cannot be determined with PCIe methods.
>>
>> Change default of pci_enable_atomic_ops_to_root() to not enable
>> AtomicOps requests on RCiEPs.
> I know I suggested this because there's nothing explicit that tells us
> whether the RC supports atomic ops from RCiEPs [1].  But I'm concerned
> that GPUs, infiniband HCAs, and NICs that use atomic ops may be
> implemented as RCiEPs and would be broken by this.

FWIW, on AMD APUs our driver doesn't call pci_enable_atomic_ops_to_root. 
It just assumes that the GPU can do atomic accesses because it doesn't 
actually go through PCIe: 
https://elixir.bootlin.com/linux/v6.19.10/source/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c#L4785

Regards,
   Felix


>
> These drivers use pci_enable_atomic_ops_to_root():
>
>    amdgpu
>    bnxt_re (infiniband)
>    mlx5 (infinband)
>    qedr (infiniband)
>    mlx5 (ethernet)
>
> Maybe we should assume that because RCiEPs are directly integrated
> into the RC, the RCiEP would only allow AtomicOp Requester Enable to
> be set if the RC supports atomic ops?
>
> I don't like making assumptions like that, but it'd be worse to break
> these devices.
>
> [1] https://lore.kernel.org/all/20260326164002.GA1325368@bhelgaas
>
>> Signed-off-by: Gerd Bayer <gbayer@linux.ibm.com>
>> ---
>>   drivers/pci/pci.c | 5 ++---
>>   1 file changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>> index 8479c2e1f74f1044416281aba11bf071ea89488a..135e5b591df405e87e7f520a618d7e2ccba55ce1 100644
>> --- a/drivers/pci/pci.c
>> +++ b/drivers/pci/pci.c
>> @@ -3692,15 +3692,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
>>   
>>   	/*
>>   	 * Per PCIe r4.0, sec 6.15, endpoints and root ports may be
>> -	 * AtomicOp requesters.  For now, we only support endpoints as
>> -	 * requesters and root ports as completers.  No endpoints as
>> +	 * AtomicOp requesters.  For now, we only support (legacy) endpoints
>> +	 * as requesters and root ports as completers.  No endpoints as
>>   	 * completers, and no peer-to-peer.
>>   	 */
>>   
>>   	switch (pci_pcie_type(dev)) {
>>   	case PCI_EXP_TYPE_ENDPOINT:
>>   	case PCI_EXP_TYPE_LEG_END:
>> -	case PCI_EXP_TYPE_RC_END:
>>   		break;
>>   	default:
>>   		return -EINVAL;
>>
>> -- 
>> 2.51.0
>>