From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39248CE7B02 for ; Fri, 6 Sep 2024 12:26:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Qs9HZPeTpbKekrPS+Zvj5ZqabwcxRRgenzbAsW1JQzk=; b=1Xu6IEomdwl1k9SFgV1XqVB4Os ZJJx7RGlr2WFrr7M9VGVLNOYadOfT2ZV7FMOKB1LEwNtnddKb29KjyBfJmW0cfgYcnxg7j4f1OpoV dHxiAeCCuCgtQWclMnW+ydF7G8LMvicGbAGliLsFSuOektjeUgJjThRjAHlwjP+JoiqzIvvvPYYl0 S32HXDBj9ro7DW4PTv8sWpgYzl7K/7k/9QrWW176sZtKn2gwbidKAheeqcr6PnXcUtjh6dVfHII8n hItdD/Vzpz/xG478xQIdYxtW2S5b3Ht+N6GAaM9qf/NdLOQBaIIofm6fufZcD2AlEeMtxMfDAdAmw a3bbzIlw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1smY2B-0000000C9n9-2lr4; Fri, 06 Sep 2024 12:25:51 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1smY19-0000000C9Wm-0c6U for linux-arm-kernel@lists.infradead.org; Fri, 06 Sep 2024 12:24:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 84D70113E; Fri, 6 Sep 2024 05:25:11 -0700 (PDT) Received: from [10.1.36.41] (010265703453.arm.com [10.1.36.41]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EFE513F73B; Fri, 6 Sep 2024 05:24:42 -0700 (PDT) Message-ID: Date: Fri, 6 Sep 2024 13:24:37 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] Revert "iommu/io-pgtable-arm: Optimise non-coherent unmap" To: Rob Clark Cc: Will Deacon , iommu@lists.linux.dev, linux-arm-msm@vger.kernel.org, freedreno@lists.freedesktop.org, Ashish Mhetre , Rob Clark , Joerg Roedel , "moderated list:ARM SMMU DRIVERS" , open list References: <20240905124956.84932-1-robdclark@gmail.com> <20240905155330.GA15246@willie-the-truck> <53f13813-a515-475a-836d-0b6017a117eb@arm.com> From: Robin Murphy Content-Language: en-GB In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240906_052447_411677_18E735B7 X-CRM114-Status: GOOD ( 18.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2024-09-05 6:10 pm, Rob Clark wrote: > On Thu, Sep 5, 2024 at 10:00 AM Rob Clark wrote: >> >> On Thu, Sep 5, 2024 at 9:27 AM Robin Murphy wrote: >>> >>> On 05/09/2024 4:53 pm, Will Deacon wrote: >>>> Hi Rob, >>>> >>>> On Thu, Sep 05, 2024 at 05:49:56AM -0700, Rob Clark wrote: >>>>> From: Rob Clark >>>>> >>>>> This reverts commit 85b715a334583488ad7fbd3001fe6fd617b7d4c0. >>>>> >>>>> It was causing gpu smmu faults on x1e80100. >>>>> >>>>> I _think_ what is causing this is the change in ordering of >>>>> __arm_lpae_clear_pte() (dma_sync_single_for_device() on the pgtable >>>>> memory) and io_pgtable_tlb_flush_walk(). I'm not entirely sure how >>>>> this patch is supposed to work correctly in the face of other >>>>> concurrent translations (to buffers unrelated to the one being >>>>> unmapped(), because after the io_pgtable_tlb_flush_walk() we can have >>>>> stale data read back into the tlb. >>>>> >>>>> Signed-off-by: Rob Clark >>>>> --- >>>>> drivers/iommu/io-pgtable-arm.c | 31 ++++++++++++++----------------- >>>>> 1 file changed, 14 insertions(+), 17 deletions(-) >>>> >>>> Please can you try the diff below, instead? >>> >>> Given that the GPU driver's .tlb_add_page is a no-op, I can't see this >>> making a difference. In fact, given that msm_iommu_pagetable_unmap() >>> still does a brute-force iommu_flush_iotlb_all() after io-pgtable >>> returns, and in fact only recently made .tlb_flush_walk start doing >>> anything either for the sake of the map path, I'm now really wondering >>> how this patch has had any effect at all... :/ >> >> Yeah.. and unfortunately the TBU code only supports two devices so >> far, so I can't easily repro with TBU enabled atm. Hmm.. >> __arm_lpae_unmap() is also called in the ->map() path, although not >> sure how that changes things. > > Ok, an update.. after a reboot, still with this patch reverted, I once > again see faults. So I guess that vindicates the original patch, and > leaves me still searching.. > > fwiw, fault info from the gpu devcore: > > ------------- > fault-info: > - ttbr0=0000000919306000 > - iova=0000000100c17000 > - dir=WRITE > - type=UNKNOWN > - source=CP > pgtable-fault-info: > - ttbr0: 000000090ca40000 > - asid: 0 > - ptes: 000000095db47003 000000095db48003 0000000914c8f003 00000008fd7f0f47 > ------------- > > the 'ptes' part shows the table walk, which looks ok to me.. But is it the right pagetable at all, given that the "ttbr0" values appear to be indicating different places? Robin.