From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD58CCA9EBE for ; Fri, 9 Oct 2026 17:43:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E1FBD10EA90; Fri, 9 Oct 2026 17:43:07 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="dCpHTy2k"; dkim-atps=neutral Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2A56C10EA90 for ; Fri, 9 Oct 2026 17:43:07 +0000 (UTC) Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by sea.source.kernel.org (Postfix) with ESMTP id DD8D943E14; Fri, 9 Oct 2026 17:43:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB21D1F000FF; Fri, 9 Oct 2026 17:43:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1791567786; bh=U0s03iNrq790So+pbxwkgjCu207XK/E0NIw4uR3r3S4=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=dCpHTy2kRayrBM7jKK+ewjW+/1nlHlYFvCNzUWHvIu4BhDEB40Ckb1z6CuG6HTcHP k0R5edj57wikOJaKKbUpz/U6Uj2TkOq3otUC/mhhMHhIX1qtO+Abkqvj1HGZ1Suwsb 9gv70Q5NfRUrjEy02QyerM0P1NLHA9bxKt0wf6Gwfz+6DPhCjog6Gzf+m6XKDFMWVz 4V63bvdWNQqHPqcGFyl3Cja06HwpnTIU2Ppi+uVZvAm12V1aM7HOlI1P/OJDDrZFuK vza8KWGlZrphMiubCqZL0OEZzyaXsLrJGdfDm4YSvNuV+YQn7ap18HyVVtuVFDiDku ptjFns74hSqpw== Date: Fri, 9 Oct 2026 20:43:01 +0300 From: Leon Romanovsky To: Bjorn Helgaas Cc: Bjorn Helgaas , Logan Gunthorpe , Jason Gunthorpe , "Joerg Roedel (AMD)" , Will Deacon , Robin Murphy , Christian =?iso-8859-1?Q?K=F6nig?= , Thomas =?iso-8859-1?Q?Hellstr=F6m?= , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, iommu@lists.linux.dev, Tushar Dave , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-rdma@vger.kernel.org, kvm@vger.kernel.org, Chaitanya Kulkarni , Greg Kroah-Hartman , Jens Axboe , Alex Williamson , Ankit Agrawal , Jonathan Corbet , Shuah Khan , Randy Dunlap , Sumit Semwal Subject: Re: [PATCH v9 02/18] PCI/P2PDMA: Derive routing from directional ACS controls Message-ID: <20261009174301.GC11438@unreal> References: <20261007063227.GC7822@unreal> <20261007204045.GA783967@bhelgaas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20261007204045.GA783967@bhelgaas> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, Oct 07, 2026 at 03:40:45PM -0500, Bjorn Helgaas wrote: > On Wed, Oct 07, 2026 at 09:32:27AM +0300, Leon Romanovsky wrote: > > On Tue, Oct 06, 2026 at 03:49:47PM -0500, Bjorn Helgaas wrote: > > > On Thu, Oct 01, 2026 at 02:55:10PM +0300, Leon Romanovsky wrote: > > > > From: Leon Romanovsky > > > > > > > > pci_bridge_has_acs_redir() treats Request and Completion Redirect as > > > > interchangeable. On asymmetric fabrics, a control for only the reverse TLP > > > > direction can unnecessarily force P2PDMA through the host bridge. > > > > > > Does "the reverse TLP direction" refer to Completions? > > > > In general, the P2P code treats TLPs flowing from device A to device B the > > same as TLPs flowing from device B to device A. > > > > However, in the context of this commit message, yes: completions flow in the > > opposite direction from the device's perspective. > > And I guess asymmetric fabrics must mean fabrics where Request > Redirect and Completion Redirect are not set the same way? Yes, in some of our systems, deviceA to deviceB is going through different route than deviceB to deviceA. > > > > > Evaluate Request Redirect for client Requests and Completion Redirect for > > > > provider read Completions. Continue treating enabled Egress Control > > > > conservatively as a Request redirect. > > > > > > Completion Redirect is intended to avoid ordering rule violations > > > between Completions and Requests when Requests are redirected (PCIe > > > r7.0, sec 6.12.1.1). I assume this patch preserves the ordering rule, > > > but does the commit log need to say something about that? I don't > > > know enough about P2P DMA for it to be obvious to me. > > > > I don't think so, i didn't change anything related to ordering. > > I don't think there's anything in this whole series that changes any > ACS settings, so I shouldn't have wondered about *preserving* the > ordering rule. > > But I asked about ordering because it sounds like this patch expects > to encounter asymmetric fabrics where Request Redirect and Completion > Redirect may not be set the same way, and the spec implies that > asymmetry may result in ordering violations. My guess is that the hardware and system architects have ensured this doesn't happen. At least, I haven't received any complaints about this series during internal testing. > > > > Not really a question for this series, but p2pdma.c and p2pdma.rst > > > refer to "clients" and "providers", neither of which are mentioned in > > > the PCIe spec. In this case it sounds like a client is a Requester > > > and a provider is a Completer in spec terms. Is that always the case? > > > If so, "client" and "provider" in this paragraph are not adding any > > > information. > > > > > > If "client" is not the same concept as "Requester" and "provider" not > > > the same as "Completer", maybe p2pdma.rst could explain the > > > difference? > > > > Client vs. provider are actual target vs. initiator. They express the > > device role in the flow. > > I'd rather use "Requester" and "Completer" when possible because they > have specific meanings in the PCI spec and they correspond to the ACS > control bits. Client, provider, target, initiator are all from the > outer world that makes use of PCIe constructs, but they don't mean > anything inside the PCIe world. These names come from p2p users such as dma-buf and NVMe. I didn't want to rename them as they already exists and is in use inside p2p code. Thanks > > Bjorn