From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00029C10F14 for ; Tue, 8 Oct 2019 08:24:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C1F1C206B6 for ; Tue, 8 Oct 2019 08:24:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="KldPZKDq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C1F1C206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rc7bXBkZXRbqHBE+zfipZr7MEvmLDnSWkHqq043PVQo=; b=KldPZKDqYelZZ1 TCUdup+VluO/lMy0kc98oAEplRK+s1LD3/iaX2LwQ4TurrPyAyfnuFgsBZHLynaHTHPwNHiY7h9My RobKe9K169PWMW4sj/KnE8P6Z0PbWYCRLE3pjOt4KdAUaQxPfOa15gcDbewIG8qjx/BVPQCXmR0OG HCl4QOgE3cS/PToXTq7W7x13H/IqS874NZrxP7tokAfIRvV0EN6L6bHEANEIS6ipg13Ai8Bfv1dci Jx0g8SRHAKxg2nay38hlClOHBd5mYShx0AjZTslv8zJaTPZCqqHJuWalrAPUSFL+yi731VuDt25yp 4s5+FWrEyEztLe/ITDuQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1iHknT-0001PS-L2; Tue, 08 Oct 2019 08:24:43 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1iHknP-0001Ox-P2; Tue, 08 Oct 2019 08:24:42 +0000 X-UUID: d75ebf5fa54b444d8c195b78977668c1-20191008 X-UUID: d75ebf5fa54b444d8c195b78977668c1-20191008 Received: from mtkcas67.mediatek.inc [(172.29.193.45)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLS) with ESMTP id 905765810; Tue, 08 Oct 2019 00:24:36 -0800 Received: from MTKMBS32DR.mediatek.inc (172.27.6.104) by MTKMBS62N1.mediatek.inc (172.29.193.41) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 8 Oct 2019 01:24:34 -0700 Received: from MTKCAS32.mediatek.inc (172.27.4.184) by MTKMBS32DR.mediatek.inc (172.27.6.104) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 8 Oct 2019 16:09:28 +0800 Received: from [10.17.3.153] (172.27.4.253) by MTKCAS32.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1395.4 via Frontend Transport; Tue, 8 Oct 2019 16:09:27 +0800 Message-ID: <1570522168.19130.39.camel@mhfsdcap03> Subject: Re: [PATCH] iommu/mediatek: Move the tlb_sync into tlb_flush From: Yong Wu To: Robin Murphy Date: Tue, 8 Oct 2019 16:09:28 +0800 In-Reply-To: <366b0bda-d874-9109-5c83-ff27301f3486@arm.com> References: <1569822142-14303-1-git-send-email-yong.wu@mediatek.com> <366b0bda-d874-9109-5c83-ff27301f3486@arm.com> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 3E9F2367AB2AA294B0B502AA0BA1AABDAC92BE66BC543487A09D89709BFE290B2000:8 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191008_012439_820799_E6E8953A X-CRM114-Status: GOOD ( 30.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: youlin.pei@mediatek.com, anan.sun@mediatek.com, Nicolas Boichat , cui.zhang@mediatek.com, srv_heupstream , Joerg Roedel , Will Deacon , Linux Kernel Mailing List , Evan Green , Tomasz Figa , chao.hao@mediatek.com, "list@263.net:IOMMU DRIVERS" , "moderated list:ARM/Mediatek SoC support" , Matthias Brugger , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 2019-10-02 at 11:35 +0100, Robin Murphy wrote: > On 02/10/2019 06:18, Tomasz Figa wrote: > > Hi Yong, > > > > On Mon, Sep 30, 2019 at 2:42 PM Yong Wu wrote: > >> > >> The commit 4d689b619445 ("iommu/io-pgtable-arm-v7s: Convert to IOMMU API > >> TLB sync") help move the tlb_sync of unmap from v7s into the iommu > >> framework. It helps add a new function "mtk_iommu_iotlb_sync", But it > >> lacked the dom->pgtlock, then it will cause the variable > >> "tlb_flush_active" may be changed unexpectedly, we could see this warning > >> log randomly: > >> > > > > Thanks for the patch! Please see my comments inline. > > > >> mtk-iommu 10205000.iommu: Partial TLB flush timed out, falling back to > >> full flush > >> > >> To fix this issue, we can add dom->pgtlock in the "mtk_iommu_iotlb_sync". > >> And when checking this issue, we find that __arm_v7s_unmap call > >> io_pgtable_tlb_add_flush consecutively when it is supersection/largepage, > >> this also is potential unsafe for us. There is no tlb flush queue in the > >> MediaTek M4U HW. The HW always expect the tlb_flush/tlb_sync one by one. > >> If v7s don't always gurarantee the sequence, Thus, In this patch I move > >> the tlb_sync into tlb_flush(also rename the function deleting "_nosync"). > >> and we don't care if it is leaf, rearrange the callback functions. Also, > >> the tlb flush/sync was already finished in v7s, then iotlb_sync and > >> iotlb_sync_all is unnecessary. > > > > Performance-wise, we could do much better. Instead of synchronously > > syncing at the end of mtk_iommu_tlb_add_flush(), we could sync at the > > beginning, if there was any previous flush still pending. We would > > also have to keep the .iotlb_sync() callback, to take care of waiting > > for the last flush. That would allow better pipelining with CPU in > > cases like this: > > > > for (all pages in range) { > > change page table(); > > flush(); > > } > > > > "change page table()" could execute while the IOMMU is flushing the > > previous change. > > FWIW, given that the underlying invalidation mechanism is range-based, > this driver would be an ideal candidate for making use of the new > iommu_gather mechanism. As a fix for stable, though, simply ensuring > that add_flush syncs any pending invalidation before issuing a new one > sounds like a good idea (and probably a simpler patch too). Thanks very much for the confirmation. > > [...] > >> @@ -574,8 +539,7 @@ static int mtk_iommu_of_xlate(struct device *dev, struct of_phandle_args *args) > >> .detach_dev = mtk_iommu_detach_device, > >> .map = mtk_iommu_map, > >> .unmap = mtk_iommu_unmap, > >> - .flush_iotlb_all = mtk_iommu_flush_iotlb_all, > > > > Don't we still want .flush_iotlb_all()? I think it should be more > > efficient in some cases than doing a big number of single flushes. > > (That said, the previous implementation didn't do any flush at all. It > > just waited for previously queued flushes to happen. Was that > > expected?) > > Commit 07fdef34d2be ("iommu/arm-smmu-v3: Implement flush_iotlb_all > hook") has an explanation of what the deal was there - similarly, it's > probably worth this driver implementing it properly as well now (but > that's really a separate patch). Thanks the hint, At the beginning, I noticed that we don't have DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, thus I thought flush_iotlb_all is unnecessary. After grep, iommu_flush_tlb_all also is called in the x_direct_mapping, then we still need this. If putting it in a new patch(switch flush_iotlb_all to mtk_iommu_tlb_flush_all), then the Fixes tag(commit id: 4d689b619445) is needed? > > Robin. > > _______________________________________________ > Linux-mediatek mailing list > Linux-mediatek@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-mediatek _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel