From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 668513A4532; Mon, 30 Mar 2026 21:42:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774906975; cv=none; b=jL4fxhLa6rw6AoC4mtqN9SUnfrk9hsnV6e5KwtRIYCXcQ56E9S0aQS2yd4Zix8IlXa/6QMesyAaH5yZYRbQZjnUnRDLNnZdKRZbQ3wOIKfA8YkaUO8AQz+rPHNd2irXsshRbncomx/EmHe89rW0UtqR9Y5eSUaB12vaaePBwrG0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774906975; c=relaxed/simple; bh=WDmKX5hyCYCsZw9yvr0oU4+GmiqTh75fiyz8/DAsRWA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=CBiPcCPOcfLAzs23fdj388cufd3RMZoIl30C3BMYbhw7X+eWm4DsviUSZfW3Q1hb5gvkJTfoZLk5ubaSDhejdSEB1p+oN89fiNmSd5+PFXJ236dl6YS4/tdvvW0puLFOMlnc3QWA2wOpc35JjUiZNxtgcgZyS3sQ9YPloyskrCU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=n4IC9t/9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="n4IC9t/9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F285C4CEF7; Mon, 30 Mar 2026 21:42:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1774906975; bh=WDmKX5hyCYCsZw9yvr0oU4+GmiqTh75fiyz8/DAsRWA=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=n4IC9t/9kMlDfn3o72V3jZPI6iJYHmJHV+A1YEz4cLBcZrXHDboxAGObjtIy9KecY xcqaUGg/W40l3Zqlidf83ecSn0Mg/+nmI9mIa7G1JHexHwfEfn1HbSY1Y/r9U5JPUK tH7e6O9QOa4xznvpZVisd8QxwheStdjdng4i4piQ2+Ym+gkcY8fYLkB4G4SgBIP/L2 0Hp9oS+L8oH9RZi02qe37K5dPaRrMnHmC1GFXKnUfU89SrgLqhwOXDBcBhWdJoEpHZ n9tNyv6c4Y0WSpqc+7ESC+cWocrV0MFGJm+3fIQ/jd7XAzMzvs73MjYWjU0Ine+9g3 pwp+v9byuUMxQ== Date: Mon, 30 Mar 2026 16:42:53 -0500 From: Bjorn Helgaas To: Gerd Bayer , Alex Deucher , Christian =?utf-8?B?S8O2bmln?= , Selvin Xavier , Kalesh AP , Jason Gunthorpe , Leon Romanovsky , Michal Kalderon , Saeed Mahameed , Tariq Toukan , Mark Bloch Cc: Bjorn Helgaas , Jay Cornwall , Felix Kuehling , Ilpo =?utf-8?B?SsOkcnZpbmVu?= , Christian Borntraeger , Niklas Schnelle , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Sven Schnelle , Alexander Schmidt , linux-s390@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Subject: Re: [PATCH v7 1/3] PCI: AtomicOps: Do not enable requests by RCiEPs Message-ID: <20260330214253.GA92498@bhelgaas> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260330-fix_pciatops-v7-1-f601818417e8@linux.ibm.com> [+to amdgpu, bnxe_re, mlx5 IB, qedr, mlx5 maintainers] On Mon, Mar 30, 2026 at 03:09:44PM +0200, Gerd Bayer wrote: > Since root complex integrated end points (RCiEPs) attach to a bus that > has no bridge device describing the root port, the capability to > complete AtomicOps requests cannot be determined with PCIe methods. > > Change default of pci_enable_atomic_ops_to_root() to not enable > AtomicOps requests on RCiEPs. I know I suggested this because there's nothing explicit that tells us whether the RC supports atomic ops from RCiEPs [1]. But I'm concerned that GPUs, infiniband HCAs, and NICs that use atomic ops may be implemented as RCiEPs and would be broken by this. These drivers use pci_enable_atomic_ops_to_root(): amdgpu bnxt_re (infiniband) mlx5 (infinband) qedr (infiniband) mlx5 (ethernet) Maybe we should assume that because RCiEPs are directly integrated into the RC, the RCiEP would only allow AtomicOp Requester Enable to be set if the RC supports atomic ops? I don't like making assumptions like that, but it'd be worse to break these devices. [1] https://lore.kernel.org/all/20260326164002.GA1325368@bhelgaas > Signed-off-by: Gerd Bayer > --- > drivers/pci/pci.c | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 8479c2e1f74f1044416281aba11bf071ea89488a..135e5b591df405e87e7f520a618d7e2ccba55ce1 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -3692,15 +3692,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask) > > /* > * Per PCIe r4.0, sec 6.15, endpoints and root ports may be > - * AtomicOp requesters. For now, we only support endpoints as > - * requesters and root ports as completers. No endpoints as > + * AtomicOp requesters. For now, we only support (legacy) endpoints > + * as requesters and root ports as completers. No endpoints as > * completers, and no peer-to-peer. > */ > > switch (pci_pcie_type(dev)) { > case PCI_EXP_TYPE_ENDPOINT: > case PCI_EXP_TYPE_LEG_END: > - case PCI_EXP_TYPE_RC_END: > break; > default: > return -EINVAL; > > -- > 2.51.0 >