From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B7ADC5472F for ; Tue, 27 Aug 2024 17:35:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hoiU2zUmE2JHmeB+vqZBzekkmF9nQsKx/TgnjtaRqvo=; b=e5DHrWuzKm4kKFrG1mBOQkFesd 3chdlxBBsL3RrpY9ufhvwxLQDNq0LW0ncTIqnugXp66xFcFbRUbJpNP9qhiMpgtjEY+yiTRSvBFvx 3hyPb1L0H66dS3Nbxw2UYgNbRSDH4ZgSuAgZoE0zzk0yaFpS6QRgrDgKhb1A6KiFpKRLKjzE7RLFH PgyW3y6W9LrOxNop7i0PplSizJ8GIU3Sz3ZsRFgLcOvlnQxJ/nUN30xU0P1v6kKPEW0mSv5yk7t0z /Ma85EaZFAdLd4Bu4Q33mfW/7Z8s+K66rR+pkOwDwD6sxjPU3XE1o+Kaxurss5ipgT15jT5xql7Jb nZ/vHa3g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sj06Q-0000000CJuB-2I3I; Tue, 27 Aug 2024 17:35:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sj02M-0000000CIjU-02tZ for linux-arm-kernel@lists.infradead.org; Tue, 27 Aug 2024 17:31:23 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9FD8ADA7; Tue, 27 Aug 2024 10:31:46 -0700 (PDT) Received: from [10.1.196.40] (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F3DD83F66E; Tue, 27 Aug 2024 10:31:18 -0700 (PDT) Message-ID: Date: Tue, 27 Aug 2024 18:31:17 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 0/4] vfio/iommu: Flag to allow userspace to set DMA buffers system cacheable To: Jason Gunthorpe , Alex Williamson Cc: Manoj Vishwanathan , Will Deacon , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, David Dillow References: <20240826071641.2691374-1-manojvishy@google.com> <20240826110447.6522e0a7.alex.williamson@redhat.com> <20240826231749.GM3773488@nvidia.com> From: Robin Murphy Content-Language: en-GB In-Reply-To: <20240826231749.GM3773488@nvidia.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240827_103122_144238_33DDC2B4 X-CRM114-Status: GOOD ( 17.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 27/08/2024 12:17 am, Jason Gunthorpe wrote: > On Mon, Aug 26, 2024 at 11:04:47AM -0600, Alex Williamson wrote: >> On Mon, 26 Aug 2024 07:16:37 +0000 >> Manoj Vishwanathan wrote: >> >>> Hi maintainers, >>> >>> This RFC patch introduces the ability for userspace to control whether >>> device (DMA) buffers are marked as cacheable, enabling them to utilize >>> the system-level cache. >>> >>> The specific changes made in this patch are: >>> >>> * Introduce a new flag in `include/linux/iommu.h`: >>> * `IOMMU_SYS_CACHE` - Indicates if the associated page should be cached in the system's cache hierarchy. >>> * Add `VFIO_DMA_MAP_FLAG_SYS_CACHE` to `include/uapi/linux/vfio.h`: > > You'll need a much better description of what this is supposed to do > when you resend it. > > IOMMU_CACHE already largely means that pages should be cached. > > So I don't know what ARM's "INC_OCACHE" actually is doing. Causing > writes to land in a cache somewhere in hierarchy? Something platform > specific? Kinda both - the Inner Non-Cacheable attribute means it's still fundamentally non-snooping and non-coherent with CPU caches, but the Outer Write-back Write-allocate attribute can still control allocation in a system cache downstream of the point of coherency if the platform is built with such a thing (it's not overly common). However, as you point out, this is in direct conflict with the Inner Write-back Write-allocate attribute implied by the IOMMU_CACHE which VFIO adds in further down in vfio_iommu_map(). Plus the way it's actually implemented in patch #2, IOMMU_CACHE still takes precedence and would lead to this new value being completely ignored, so there's a lot which smells suspicious here... Thanks, Robin. > I have no idea. By your description it sounds similar to the > x86 data placement stuff, whatever that was called, and the more > modern TPH approach. > > Jason