From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E492DC433E0 for ; Fri, 8 Jan 2021 18:20:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D0CA23A69 for ; Fri, 8 Jan 2021 18:20:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D0CA23A69 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=xHksPIuIogVvu7U2QQ5eQSOc5o0sm9Uh7TItC9nBpT4=; b=N7PZosz480U4iXu0gVUWlUvhb WiSDA1rRvZWyz17zHBnfGgFDUOfc63Lq2jtmsDUnFLU94fiOXAkRDGRGP4nZC1OfY7ldHhYNCtNkx SWgAOr2F3LGmU02zDwfWObNpN8TC/RO9KHgzKbmzVdwvfukOLFm9ufzNi8ztWjy2z8aOD1ph4FPGS AM0ufRKAuwwWTShngLx0lzMy7tgkzKzXsakZLuRb9FTpOh1HloxEyRNM7ccxHnBj5Mp8uRtbHfnC9 dNY0D2JUjN85lls7bd14HOpgAjzqLnD28HJKKz3StT6VLCDuS99jsrtfIz8i2NIwxyC4R3FF8l+CO AiXGPwXog==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxwLR-0004IP-1s; Fri, 08 Jan 2021 18:18:41 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxwLO-0004Hp-QR for linux-arm-kernel@lists.infradead.org; Fri, 08 Jan 2021 18:18:39 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1BB9723A7A; Fri, 8 Jan 2021 18:18:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1610129916; bh=1EJDjUMPNL/dY0XEEkmAoh+OfzkB5yqH0QpcKX27KdQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=gOAaZKMBGevPezWNeCeeRPpwnW3ki8JMZ3ojjuJnY5SMO0nydVHO/xXY0yA6k6eMc ruSWe9plVJ0Zo3oSx5nQvnje1e0kI3FdZX3Q/dEf1lExhqP7XIq74eEnikkOOKavco duc/mpql1uOwc/kZucSz5iG+biQbSXdzPhSuI/k5ZNagSsk8YmI7PObnEnJpakgRje Aw/uLHiRUU0A3VHsVUPnqKqPOC0plw2zNgYfVDEr1sriYbuyMku128jorI5DcaHf7j /9Cnj8jfwv+6xqPDZBioVAfeT4kfCrcS66RFA22SqEGYaJSdqC+E0nWaV8k3bnE6Hx 4LHFeRNRBeeFQ== Date: Fri, 8 Jan 2021 18:18:30 +0000 From: Will Deacon To: Sai Prakash Ranjan Subject: Re: [PATCH] iommu/io-pgtable-arm: Allow non-coherent masters to use system cache Message-ID: <20210108181830.GA5457@willie-the-truck> References: <20201224064007.2339-1-saiprakash.ranjan@codeaurora.org> <20210106115615.GA1763@willie-the-truck> <8cfefbff135a5287d177b6ab2ccc3304@codeaurora.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <8cfefbff135a5287d177b6ab2ccc3304@codeaurora.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210108_131839_026030_AE2FAF8F X-CRM114-Status: GOOD ( 33.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: isaacm@codeaurora.org, linux-arm-msm@vger.kernel.org, Joerg Roedel , Jordan Crouse , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Rob Clark , Akhil P Oommen , Robin Murphy , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jan 08, 2021 at 11:17:25AM +0530, Sai Prakash Ranjan wrote: > On 2021-01-07 22:27, isaacm@codeaurora.org wrote: > > On 2021-01-06 03:56, Will Deacon wrote: > > > On Thu, Dec 24, 2020 at 12:10:07PM +0530, Sai Prakash Ranjan wrote: > > > > commit ecd7274fb4cd ("iommu: Remove unused IOMMU_SYS_CACHE_ONLY > > > > flag") > > > > removed unused IOMMU_SYS_CACHE_ONLY prot flag and along with it went > > > > the memory type setting required for the non-coherent masters to use > > > > system cache. Now that system cache support for GPU is added, we will > > > > need to mark the memory as normal sys-cached for GPU to use > > > > system cache. > > > > Without this, the system cache lines are not allocated for GPU. > > > > We use > > > > the IO_PGTABLE_QUIRK_ARM_OUTER_WBWA quirk instead of a page > > > > protection > > > > flag as the flag cannot be exposed via DMA api because of no in-tree > > > > users. > > > > > > > > Signed-off-by: Sai Prakash Ranjan > > > > --- > > > > drivers/iommu/io-pgtable-arm.c | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git a/drivers/iommu/io-pgtable-arm.c > > > > b/drivers/iommu/io-pgtable-arm.c > > > > index 7c9ea9d7874a..3fb7de8304a2 100644 > > > > --- a/drivers/iommu/io-pgtable-arm.c > > > > +++ b/drivers/iommu/io-pgtable-arm.c > > > > @@ -415,6 +415,9 @@ static arm_lpae_iopte > > > > arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, > > > > else if (prot & IOMMU_CACHE) > > > > pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE > > > > << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > + else if (data->iop.cfg.quirks & IO_PGTABLE_QUIRK_ARM_OUTER_WBWA) > > > > + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE > > > > + << ARM_LPAE_PTE_ATTRINDX_SHIFT); > > > > } > > > > > While this approach of enabling system cache globally for both page > > tables and other buffers > > works for the GPU usecase, this isn't ideal for other clients that use > > system cache. For example, > > video clients only want to cache a subset of their buffers in the > > system cache, due to the sizing constraint > > imposed by how much of the system cache they can use. So, it would be > > ideal to have > > a way of expressing the desire to use the system cache on a per-buffer > > basis. Additionally, > > our video clients use the DMA layer, and since the requirement is for > > caching in the system cache > > to be a per buffer attribute, it seems like we would have to have a > > DMA attribute to express > > this on a per-buffer basis. > > > > I did bring this up initially [1], also where is this video client > in upstream? AFAIK, only system cache user in upstream is GPU. > We cannot add any DMA attribute unless there is any user upstream > as per [2], so when the support for such a client is added, wouldn't > ((data->iop.cfg.quirks & IO_PGTABLE_QUIRK_ARM_OUTER_WBWA) || PROT_FLAG) > work? Hmm, I think this is another case where we need to separate out the page-table walker attributes from the access attributes. Currently, IO_PGTABLE_QUIRK_ARM_OUTER_WBWA applies _only_ to the page-table walker and I don't think it makes any sense for that to be per-buffer (how would you even manage that?). However, if we want to extend this to data accesses and we know that there are valid use-cases where this should be per-buffer, then shoe-horning it in with the walker quirk does not feel like the best thing to do. As a starting point, we could: 1. Rename IO_PGTABLE_QUIRK_ARM_OUTER_WBWA to IO_PGTABLE_QUIRK_PTW_LLC 2. Add a new prot flag IOMMU_LLC 3. Have the GPU pass the new prot for its buffer mappings Does that work? One thing I'm not sure about is whether IOMMU_CACHE should imply IOMMU_LLC, or whether there is a use-case for inner-cacheable, outer non-cacheable mappings for a coherent device. Have you ever seen that sort of thing before? Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel