From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.0 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED, USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82618C5517A for ; Fri, 23 Oct 2020 06:11:09 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D3E762192A for ; Fri, 23 Oct 2020 06:11:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="hsuw3fVt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3E762192A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 71AAB876D2; Fri, 23 Oct 2020 06:11:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ELfLt+dhrcnM; Fri, 23 Oct 2020 06:11:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 831F0872A4; Fri, 23 Oct 2020 06:11:07 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 42F5DC0893; Fri, 23 Oct 2020 06:11:07 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id AD48AC0052 for ; Fri, 23 Oct 2020 06:11:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 810B3203C7 for ; Fri, 23 Oct 2020 06:11:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nmupAlD-pVpV for ; Fri, 23 Oct 2020 06:11:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by silver.osuosl.org (Postfix) with ESMTP id 9D1BC2036E for ; Fri, 23 Oct 2020 06:11:03 +0000 (UTC) X-UUID: be932c73ffe54b3d8d46c5418962f501-20201023 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=lE1AaRbp7IlJWDDgqTmejQzZzwuR5fT51G6JynHVeJ4=; b=hsuw3fVt1w54pKwXaTtt9H2NhjLMJjDfNEgAHjjQzhryqu0AmYgsTkOpnTKXHBAWW2AzP3ICyse0Qivy2TN30tzqZ9v4XRyLpblyzruEY1EBPf2I6h2T0Vto7tXLrTBgjjjzUKdCMMIcOwAeXns3ZiTCvhr3yoHiMjjbIz7uYFA=; X-UUID: be932c73ffe54b3d8d46c5418962f501-20201023 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Cellopoint E-mail Firewall v4.1.14 Build 0819 with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 275060813; Fri, 23 Oct 2020 14:11:01 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs01n1.mediatek.inc (172.21.101.68) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 23 Oct 2020 14:11:00 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 23 Oct 2020 14:10:59 +0800 Message-ID: <1603433064.2024.6.camel@mbjsdccf07> Subject: Re: [PATCH 2/4] iommu/mediatek: Add iotlb_sync_range() support From: chao hao To: Robin Murphy Date: Fri, 23 Oct 2020 14:04:24 +0800 In-Reply-To: <1603432677.2024.3.camel@mbjsdccf07> References: <20201019113100.23661-1-chao.hao@mediatek.com> <20201019113100.23661-3-chao.hao@mediatek.com> <7fbe0305-91e4-949e-7d84-bf91e81d6b27@arm.com> <1603432677.2024.3.camel@mbjsdccf07> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N Cc: Jun Wen , FY Yang , wsd_upstream@mediatek.com, linux-kernel@vger.kernel.org, Chao Hao , iommu@lists.linux-foundation.org, linux-mediatek@lists.infradead.org, Matthias Brugger , Mingyuan Ma , linux-arm-kernel@lists.infradead.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Fri, 2020-10-23 at 13:57 +0800, chao hao wrote: > On Wed, 2020-10-21 at 17:55 +0100, Robin Murphy wrote: > > On 2020-10-19 12:30, Chao Hao wrote: > > > MTK_IOMMU driver writes one page entry and does tlb flush at a time > > > currently. More optimal would be to aggregate the writes and flush > > > BUS buffer in the end. > > > > That's exactly what iommu_iotlb_gather_add_page() is meant to achieve. > > Rather than jumping straight into hacking up a new API to go round the > > back of the existing API design, it would be far better to ask the > > question of why that's not behaving as expected. > > Thanks for you review! > > iommu_iotlb_gather_add_page is put in io_pgtable_tlb_add_page(). > io_pgtable_tlb_add_page() only be called in > unmapping and mapping flow doesn't have it in linux iommu driver, but > mtk iommu needs to do tlb sync in mapping > and unmapping to avoid old data being in the iommu tlb. > > In addtion, we hope to do tlb sync once when all the pages mapping done. > iommu_iotlb_gather_add_page maybe do > tlb sync more than once. because one whole buffer consists of different > page size(1MB/64K/4K). > > Based on the previous considerations, don't find more appropriate the > way of tlb sync for mtk iommu, so we add a new API. > > > > > > For 50MB buffer mapping, if mtk_iommu driver use iotlb_sync_range() > > > instead of tlb_add_range() and tlb_flush_walk/leaf(), it can increase > > > 50% performance or more(depending on size of every page size) in > > > comparison to flushing after each page entry update. So we prefer to > > > use iotlb_sync_range() to replace iotlb_sync(), tlb_add_range() and > > > tlb_flush_walk/leaf() for MTK platforms. > > > > In the case of mapping, it sounds like what you actually want to do is > > hook up .iotlb_sync_map and generally make IO_PGTABLE_QUIRK_TLBI_ON_MAP > > cleverer, because the current implementation is as dumb as it could > > possibly be. > > iotlb_sync_map only has one parameter(iommu_domain), but mtk > iommu_domain maybe include the whole iova space, if mtk_iommu to do tlb > sync based on iommu_domain, it is equivalent to do tlb flush all in > fact. > iommu driver will do tlb sync in every mapping page when mtk iommu sets > IO_PGTABLE_QUIRK_TLBI_ON_MAP(io_pgtable_tlb_flush_walk), > as is the commit message mentioned, it will drop mapping performance in > mtk platform. > > > > In fact if we simply passed an address range to > > .iotlb_sync_map, io-pgtable probably wouldn't need to be involved at all > > any more. Sorry, I forget to reply the question in previous mail. Do you mean we need to modify iotlb_sync_map() input parameter(ex: add start/end iova)? > > I know it is not a good idea probably by adding a new api, but I found > out that tlb sync only to be done after mapping one page, so if > mtk_iommu hope to do tlb sync once after all the pages map done, could > you give me some advices? thanks! > > > > > Robin. > > > > > Signed-off-by: Chao Hao > > > --- > > > drivers/iommu/mtk_iommu.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > index 785b228d39a6..d3400c15ff7b 100644 > > > --- a/drivers/iommu/mtk_iommu.c > > > +++ b/drivers/iommu/mtk_iommu.c > > > @@ -224,6 +224,11 @@ static void mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size, > > > } > > > } > > > > > > +static void __mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size) > > > +{ > > > + mtk_iommu_tlb_flush_range_sync(iova, size, 0, NULL) > > > +} > > > + > > > static void mtk_iommu_tlb_flush_page_nosync(struct iommu_iotlb_gather *gather, > > > unsigned long iova, size_t granule, > > > void *cookie) > > > @@ -536,6 +541,7 @@ static const struct iommu_ops mtk_iommu_ops = { > > > .map = mtk_iommu_map, > > > .unmap = mtk_iommu_unmap, > > > .flush_iotlb_all = mtk_iommu_flush_iotlb_all, > > > + .iotlb_sync_range = __mtk_iommu_tlb_flush_range_sync, > > > .iotlb_sync = mtk_iommu_iotlb_sync, > > > .iova_to_phys = mtk_iommu_iova_to_phys, > > > .probe_device = mtk_iommu_probe_device, > > > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C224C388F9 for ; Fri, 23 Oct 2020 06:11:19 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 180622177B for ; Fri, 23 Oct 2020 06:11:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="apCK5XHb"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="hsuw3fVt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 180622177B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pPuO6Tv+kSm047E0uc51YpuZoGZ2iWvfPhPdGjTxFys=; b=apCK5XHbdLUyH7+hxlu4YyVBk Zoy16jq8Vi3BKUqSMQgSvBvYREAgMwpZzv2vrqF0vxY+jlV4l/MmllPQQT8ZiJYBSvBjRHGaawt2E ykS6w8Y3HdlEeAqTa/f8fDDBZ6exu1iolf1dHm12QDxjloeJBDWPhc8RJMY/ov40HLJ1k9ElS1bls w/WPtLWSMG0lzXEgce/c0j07JnJRAoBWce1uGL+AJyjDA87w//DTWXKXxNjj08oGUA1N3iewGCfxU xsw8hjSdFZPrWKbiDnYcwV0uQCEx5GADfGPRE1KjDR+s9lmtYxU6iRZn0+w3AUHD/hSWQ2s/Rh8GN 8AiEHqmNg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kVqIA-00055Q-EM; Fri, 23 Oct 2020 06:11:10 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kVqI6-00054M-OD; Fri, 23 Oct 2020 06:11:07 +0000 X-UUID: 037d6450b31e4844bf75ffffb81aa03c-20201022 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=lE1AaRbp7IlJWDDgqTmejQzZzwuR5fT51G6JynHVeJ4=; b=hsuw3fVt1w54pKwXaTtt9H2NhjLMJjDfNEgAHjjQzhryqu0AmYgsTkOpnTKXHBAWW2AzP3ICyse0Qivy2TN30tzqZ9v4XRyLpblyzruEY1EBPf2I6h2T0Vto7tXLrTBgjjjzUKdCMMIcOwAeXns3ZiTCvhr3yoHiMjjbIz7uYFA=; X-UUID: 037d6450b31e4844bf75ffffb81aa03c-20201022 Received: from mtkcas67.mediatek.inc [(172.29.193.45)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 2043454342; Thu, 22 Oct 2020 22:11:02 -0800 Received: from MTKMBS01N1.mediatek.inc (172.21.101.68) by MTKMBS62N2.mediatek.inc (172.29.193.42) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 22 Oct 2020 23:11:00 -0700 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs01n1.mediatek.inc (172.21.101.68) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 23 Oct 2020 14:11:00 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 23 Oct 2020 14:10:59 +0800 Message-ID: <1603433064.2024.6.camel@mbjsdccf07> Subject: Re: [PATCH 2/4] iommu/mediatek: Add iotlb_sync_range() support From: chao hao To: Robin Murphy Date: Fri, 23 Oct 2020 14:04:24 +0800 In-Reply-To: <1603432677.2024.3.camel@mbjsdccf07> References: <20201019113100.23661-1-chao.hao@mediatek.com> <20201019113100.23661-3-chao.hao@mediatek.com> <7fbe0305-91e4-949e-7d84-bf91e81d6b27@arm.com> <1603432677.2024.3.camel@mbjsdccf07> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201023_021106_999387_E2F99F91 X-CRM114-Status: GOOD ( 37.17 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jun Wen , FY Yang , wsd_upstream@mediatek.com, Joerg Roedel , linux-kernel@vger.kernel.org, Chao Hao , iommu@lists.linux-foundation.org, linux-mediatek@lists.infradead.org, Matthias Brugger , Mingyuan Ma , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On Fri, 2020-10-23 at 13:57 +0800, chao hao wrote: > On Wed, 2020-10-21 at 17:55 +0100, Robin Murphy wrote: > > On 2020-10-19 12:30, Chao Hao wrote: > > > MTK_IOMMU driver writes one page entry and does tlb flush at a time > > > currently. More optimal would be to aggregate the writes and flush > > > BUS buffer in the end. > > > > That's exactly what iommu_iotlb_gather_add_page() is meant to achieve. > > Rather than jumping straight into hacking up a new API to go round the > > back of the existing API design, it would be far better to ask the > > question of why that's not behaving as expected. > > Thanks for you review! > > iommu_iotlb_gather_add_page is put in io_pgtable_tlb_add_page(). > io_pgtable_tlb_add_page() only be called in > unmapping and mapping flow doesn't have it in linux iommu driver, but > mtk iommu needs to do tlb sync in mapping > and unmapping to avoid old data being in the iommu tlb. > > In addtion, we hope to do tlb sync once when all the pages mapping done. > iommu_iotlb_gather_add_page maybe do > tlb sync more than once. because one whole buffer consists of different > page size(1MB/64K/4K). > > Based on the previous considerations, don't find more appropriate the > way of tlb sync for mtk iommu, so we add a new API. > > > > > > For 50MB buffer mapping, if mtk_iommu driver use iotlb_sync_range() > > > instead of tlb_add_range() and tlb_flush_walk/leaf(), it can increase > > > 50% performance or more(depending on size of every page size) in > > > comparison to flushing after each page entry update. So we prefer to > > > use iotlb_sync_range() to replace iotlb_sync(), tlb_add_range() and > > > tlb_flush_walk/leaf() for MTK platforms. > > > > In the case of mapping, it sounds like what you actually want to do is > > hook up .iotlb_sync_map and generally make IO_PGTABLE_QUIRK_TLBI_ON_MAP > > cleverer, because the current implementation is as dumb as it could > > possibly be. > > iotlb_sync_map only has one parameter(iommu_domain), but mtk > iommu_domain maybe include the whole iova space, if mtk_iommu to do tlb > sync based on iommu_domain, it is equivalent to do tlb flush all in > fact. > iommu driver will do tlb sync in every mapping page when mtk iommu sets > IO_PGTABLE_QUIRK_TLBI_ON_MAP(io_pgtable_tlb_flush_walk), > as is the commit message mentioned, it will drop mapping performance in > mtk platform. > > > > In fact if we simply passed an address range to > > .iotlb_sync_map, io-pgtable probably wouldn't need to be involved at all > > any more. Sorry, I forget to reply the question in previous mail. Do you mean we need to modify iotlb_sync_map() input parameter(ex: add start/end iova)? > > I know it is not a good idea probably by adding a new api, but I found > out that tlb sync only to be done after mapping one page, so if > mtk_iommu hope to do tlb sync once after all the pages map done, could > you give me some advices? thanks! > > > > > Robin. > > > > > Signed-off-by: Chao Hao > > > --- > > > drivers/iommu/mtk_iommu.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > index 785b228d39a6..d3400c15ff7b 100644 > > > --- a/drivers/iommu/mtk_iommu.c > > > +++ b/drivers/iommu/mtk_iommu.c > > > @@ -224,6 +224,11 @@ static void mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size, > > > } > > > } > > > > > > +static void __mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size) > > > +{ > > > + mtk_iommu_tlb_flush_range_sync(iova, size, 0, NULL) > > > +} > > > + > > > static void mtk_iommu_tlb_flush_page_nosync(struct iommu_iotlb_gather *gather, > > > unsigned long iova, size_t granule, > > > void *cookie) > > > @@ -536,6 +541,7 @@ static const struct iommu_ops mtk_iommu_ops = { > > > .map = mtk_iommu_map, > > > .unmap = mtk_iommu_unmap, > > > .flush_iotlb_all = mtk_iommu_flush_iotlb_all, > > > + .iotlb_sync_range = __mtk_iommu_tlb_flush_range_sync, > > > .iotlb_sync = mtk_iommu_iotlb_sync, > > > .iova_to_phys = mtk_iommu_iova_to_phys, > > > .probe_device = mtk_iommu_probe_device, > > > > _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EA4DC55178 for ; Fri, 23 Oct 2020 06:12:29 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1727F2074A for ; Fri, 23 Oct 2020 06:12:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="lm4s4Nb1"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="hsuw3fVt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1727F2074A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gNfW6EX9NGW9ATBHsd6HumDtOpWrHNLvD1Tr4cTDMdo=; b=lm4s4Nb1nb6Tf0VAZJ5DKHV4G DSWQyxRSzeohrQSnsVc9r33iVJSoCv+6xtDT5JByja5zwa0aoTE5uEeSdn/50TVGDVdq/kXjxIsq5 v6332wBTRx3vtOUWqZdpS4rWw8PekJqQNZMqpJ8TqKWGVD3pRxl7Br2Vl72vUmiupNXSdDYJNDgjh eQC/RrRd/CnflX5R0iScxrQWtZz3YDNaAv6xNRyr64bPELui29KKcUwkii1r3xlcVF3edX1N7B/Us P7M+99l32QlSC4FqBF6K7lHkjCp3e6CtW1XLOX4/sPKBCrENrN42fp0Ps0ioaLuV7JOFjW2LO1WO2 rYjFiG3Qw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kVqI8-00054z-Sn; Fri, 23 Oct 2020 06:11:08 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kVqI6-00054M-OD; Fri, 23 Oct 2020 06:11:07 +0000 X-UUID: 037d6450b31e4844bf75ffffb81aa03c-20201022 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=lE1AaRbp7IlJWDDgqTmejQzZzwuR5fT51G6JynHVeJ4=; b=hsuw3fVt1w54pKwXaTtt9H2NhjLMJjDfNEgAHjjQzhryqu0AmYgsTkOpnTKXHBAWW2AzP3ICyse0Qivy2TN30tzqZ9v4XRyLpblyzruEY1EBPf2I6h2T0Vto7tXLrTBgjjjzUKdCMMIcOwAeXns3ZiTCvhr3yoHiMjjbIz7uYFA=; X-UUID: 037d6450b31e4844bf75ffffb81aa03c-20201022 Received: from mtkcas67.mediatek.inc [(172.29.193.45)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 2043454342; Thu, 22 Oct 2020 22:11:02 -0800 Received: from MTKMBS01N1.mediatek.inc (172.21.101.68) by MTKMBS62N2.mediatek.inc (172.29.193.42) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 22 Oct 2020 23:11:00 -0700 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs01n1.mediatek.inc (172.21.101.68) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 23 Oct 2020 14:11:00 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 23 Oct 2020 14:10:59 +0800 Message-ID: <1603433064.2024.6.camel@mbjsdccf07> Subject: Re: [PATCH 2/4] iommu/mediatek: Add iotlb_sync_range() support From: chao hao To: Robin Murphy Date: Fri, 23 Oct 2020 14:04:24 +0800 In-Reply-To: <1603432677.2024.3.camel@mbjsdccf07> References: <20201019113100.23661-1-chao.hao@mediatek.com> <20201019113100.23661-3-chao.hao@mediatek.com> <7fbe0305-91e4-949e-7d84-bf91e81d6b27@arm.com> <1603432677.2024.3.camel@mbjsdccf07> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201023_021106_999387_E2F99F91 X-CRM114-Status: GOOD ( 37.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jun Wen , FY Yang , wsd_upstream@mediatek.com, Joerg Roedel , linux-kernel@vger.kernel.org, Chao Hao , iommu@lists.linux-foundation.org, linux-mediatek@lists.infradead.org, Matthias Brugger , Mingyuan Ma , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 2020-10-23 at 13:57 +0800, chao hao wrote: > On Wed, 2020-10-21 at 17:55 +0100, Robin Murphy wrote: > > On 2020-10-19 12:30, Chao Hao wrote: > > > MTK_IOMMU driver writes one page entry and does tlb flush at a time > > > currently. More optimal would be to aggregate the writes and flush > > > BUS buffer in the end. > > > > That's exactly what iommu_iotlb_gather_add_page() is meant to achieve. > > Rather than jumping straight into hacking up a new API to go round the > > back of the existing API design, it would be far better to ask the > > question of why that's not behaving as expected. > > Thanks for you review! > > iommu_iotlb_gather_add_page is put in io_pgtable_tlb_add_page(). > io_pgtable_tlb_add_page() only be called in > unmapping and mapping flow doesn't have it in linux iommu driver, but > mtk iommu needs to do tlb sync in mapping > and unmapping to avoid old data being in the iommu tlb. > > In addtion, we hope to do tlb sync once when all the pages mapping done. > iommu_iotlb_gather_add_page maybe do > tlb sync more than once. because one whole buffer consists of different > page size(1MB/64K/4K). > > Based on the previous considerations, don't find more appropriate the > way of tlb sync for mtk iommu, so we add a new API. > > > > > > For 50MB buffer mapping, if mtk_iommu driver use iotlb_sync_range() > > > instead of tlb_add_range() and tlb_flush_walk/leaf(), it can increase > > > 50% performance or more(depending on size of every page size) in > > > comparison to flushing after each page entry update. So we prefer to > > > use iotlb_sync_range() to replace iotlb_sync(), tlb_add_range() and > > > tlb_flush_walk/leaf() for MTK platforms. > > > > In the case of mapping, it sounds like what you actually want to do is > > hook up .iotlb_sync_map and generally make IO_PGTABLE_QUIRK_TLBI_ON_MAP > > cleverer, because the current implementation is as dumb as it could > > possibly be. > > iotlb_sync_map only has one parameter(iommu_domain), but mtk > iommu_domain maybe include the whole iova space, if mtk_iommu to do tlb > sync based on iommu_domain, it is equivalent to do tlb flush all in > fact. > iommu driver will do tlb sync in every mapping page when mtk iommu sets > IO_PGTABLE_QUIRK_TLBI_ON_MAP(io_pgtable_tlb_flush_walk), > as is the commit message mentioned, it will drop mapping performance in > mtk platform. > > > > In fact if we simply passed an address range to > > .iotlb_sync_map, io-pgtable probably wouldn't need to be involved at all > > any more. Sorry, I forget to reply the question in previous mail. Do you mean we need to modify iotlb_sync_map() input parameter(ex: add start/end iova)? > > I know it is not a good idea probably by adding a new api, but I found > out that tlb sync only to be done after mapping one page, so if > mtk_iommu hope to do tlb sync once after all the pages map done, could > you give me some advices? thanks! > > > > > Robin. > > > > > Signed-off-by: Chao Hao > > > --- > > > drivers/iommu/mtk_iommu.c | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > index 785b228d39a6..d3400c15ff7b 100644 > > > --- a/drivers/iommu/mtk_iommu.c > > > +++ b/drivers/iommu/mtk_iommu.c > > > @@ -224,6 +224,11 @@ static void mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size, > > > } > > > } > > > > > > +static void __mtk_iommu_tlb_flush_range_sync(unsigned long iova, size_t size) > > > +{ > > > + mtk_iommu_tlb_flush_range_sync(iova, size, 0, NULL) > > > +} > > > + > > > static void mtk_iommu_tlb_flush_page_nosync(struct iommu_iotlb_gather *gather, > > > unsigned long iova, size_t granule, > > > void *cookie) > > > @@ -536,6 +541,7 @@ static const struct iommu_ops mtk_iommu_ops = { > > > .map = mtk_iommu_map, > > > .unmap = mtk_iommu_unmap, > > > .flush_iotlb_all = mtk_iommu_flush_iotlb_all, > > > + .iotlb_sync_range = __mtk_iommu_tlb_flush_range_sync, > > > .iotlb_sync = mtk_iommu_iotlb_sync, > > > .iova_to_phys = mtk_iommu_iova_to_phys, > > > .probe_device = mtk_iommu_probe_device, > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C28FC388F9 for ; Fri, 23 Oct 2020 06:11:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9EF3F2177B for ; Fri, 23 Oct 2020 06:11:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="hsuw3fVt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S372209AbgJWGLG (ORCPT ); Fri, 23 Oct 2020 02:11:06 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:44914 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2898027AbgJWGLG (ORCPT ); Fri, 23 Oct 2020 02:11:06 -0400 X-UUID: be932c73ffe54b3d8d46c5418962f501-20201023 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=lE1AaRbp7IlJWDDgqTmejQzZzwuR5fT51G6JynHVeJ4=; b=hsuw3fVt1w54pKwXaTtt9H2NhjLMJjDfNEgAHjjQzhryqu0AmYgsTkOpnTKXHBAWW2AzP3ICyse0Qivy2TN30tzqZ9v4XRyLpblyzruEY1EBPf2I6h2T0Vto7tXLrTBgjjjzUKdCMMIcOwAeXns3ZiTCvhr3yoHiMjjbIz7uYFA=; X-UUID: be932c73ffe54b3d8d46c5418962f501-20201023 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Cellopoint E-mail Firewall v4.1.14 Build 0819 with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 275060813; Fri, 23 Oct 2020 14:11:01 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs01n1.mediatek.inc (172.21.101.68) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 23 Oct 2020 14:11:00 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 23 Oct 2020 14:10:59 +0800 Message-ID: <1603433064.2024.6.camel@mbjsdccf07> Subject: Re: [PATCH 2/4] iommu/mediatek: Add iotlb_sync_range() support From: chao hao To: Robin Murphy CC: Joerg Roedel , Matthias Brugger , Jun Wen , FY Yang , , , , , , Mingyuan Ma , Chao Hao Date: Fri, 23 Oct 2020 14:04:24 +0800 In-Reply-To: <1603432677.2024.3.camel@mbjsdccf07> References: <20201019113100.23661-1-chao.hao@mediatek.com> <20201019113100.23661-3-chao.hao@mediatek.com> <7fbe0305-91e4-949e-7d84-bf91e81d6b27@arm.com> <1603432677.2024.3.camel@mbjsdccf07> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N Content-Transfer-Encoding: base64 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org T24gRnJpLCAyMDIwLTEwLTIzIGF0IDEzOjU3ICswODAwLCBjaGFvIGhhbyB3cm90ZToNCj4gT24g V2VkLCAyMDIwLTEwLTIxIGF0IDE3OjU1ICswMTAwLCBSb2JpbiBNdXJwaHkgd3JvdGU6DQo+ID4g T24gMjAyMC0xMC0xOSAxMjozMCwgQ2hhbyBIYW8gd3JvdGU6DQo+ID4gPiBNVEtfSU9NTVUgZHJp dmVyIHdyaXRlcyBvbmUgcGFnZSBlbnRyeSBhbmQgZG9lcyB0bGIgZmx1c2ggYXQgYSB0aW1lDQo+ ID4gPiBjdXJyZW50bHkuIE1vcmUgb3B0aW1hbCB3b3VsZCBiZSB0byBhZ2dyZWdhdGUgdGhlIHdy aXRlcyBhbmQgZmx1c2gNCj4gPiA+IEJVUyBidWZmZXIgaW4gdGhlIGVuZC4NCj4gPiANCj4gPiBU aGF0J3MgZXhhY3RseSB3aGF0IGlvbW11X2lvdGxiX2dhdGhlcl9hZGRfcGFnZSgpIGlzIG1lYW50 IHRvIGFjaGlldmUuIA0KPiA+IFJhdGhlciB0aGFuIGp1bXBpbmcgc3RyYWlnaHQgaW50byBoYWNr aW5nIHVwIGEgbmV3IEFQSSB0byBnbyByb3VuZCB0aGUgDQo+ID4gYmFjayBvZiB0aGUgZXhpc3Rp bmcgQVBJIGRlc2lnbiwgaXQgd291bGQgYmUgZmFyIGJldHRlciB0byBhc2sgdGhlIA0KPiA+IHF1 ZXN0aW9uIG9mIHdoeSB0aGF0J3Mgbm90IGJlaGF2aW5nIGFzIGV4cGVjdGVkLg0KPiANCj4gVGhh bmtzIGZvciB5b3UgcmV2aWV3IQ0KPiANCj4gaW9tbXVfaW90bGJfZ2F0aGVyX2FkZF9wYWdlIGlz IHB1dCBpbiBpb19wZ3RhYmxlX3RsYl9hZGRfcGFnZSgpLg0KPiBpb19wZ3RhYmxlX3RsYl9hZGRf cGFnZSgpIG9ubHkgYmUgY2FsbGVkIGluDQo+IHVubWFwcGluZyBhbmQgbWFwcGluZyBmbG93IGRv ZXNuJ3QgaGF2ZSBpdCBpbiBsaW51eCBpb21tdSBkcml2ZXIsIGJ1dA0KPiBtdGsgaW9tbXUgbmVl ZHMgdG8gZG8gdGxiIHN5bmMgaW4gbWFwcGluZw0KPiBhbmQgdW5tYXBwaW5nIHRvIGF2b2lkIG9s ZCBkYXRhIGJlaW5nIGluIHRoZSBpb21tdSB0bGIuDQo+IA0KPiBJbiBhZGR0aW9uLCB3ZSBob3Bl IHRvIGRvIHRsYiBzeW5jIG9uY2Ugd2hlbiBhbGwgdGhlIHBhZ2VzIG1hcHBpbmcgZG9uZS4NCj4g aW9tbXVfaW90bGJfZ2F0aGVyX2FkZF9wYWdlIG1heWJlIGRvDQo+IHRsYiBzeW5jIG1vcmUgdGhh biBvbmNlLiBiZWNhdXNlIG9uZSB3aG9sZSBidWZmZXIgY29uc2lzdHMgb2YgZGlmZmVyZW50DQo+ IHBhZ2Ugc2l6ZSgxTUIvNjRLLzRLKS4NCj4gDQo+IEJhc2VkIG9uIHRoZSBwcmV2aW91cyBjb25z aWRlcmF0aW9ucywgIGRvbid0IGZpbmQgbW9yZSBhcHByb3ByaWF0ZSB0aGUNCj4gd2F5IG9mIHRs YiBzeW5jIGZvciBtdGsgaW9tbXUsIHNvIHdlIGFkZCBhIG5ldyBBUEkuDQo+IA0KPiA+IA0KPiA+ ID4gRm9yIDUwTUIgYnVmZmVyIG1hcHBpbmcsIGlmIG10a19pb21tdSBkcml2ZXIgdXNlIGlvdGxi X3N5bmNfcmFuZ2UoKQ0KPiA+ID4gaW5zdGVhZCBvZiB0bGJfYWRkX3JhbmdlKCkgYW5kIHRsYl9m bHVzaF93YWxrL2xlYWYoKSwgaXQgY2FuIGluY3JlYXNlDQo+ID4gPiA1MCUgcGVyZm9ybWFuY2Ug b3IgbW9yZShkZXBlbmRpbmcgb24gc2l6ZSBvZiBldmVyeSBwYWdlIHNpemUpIGluDQo+ID4gPiBj b21wYXJpc29uIHRvIGZsdXNoaW5nIGFmdGVyIGVhY2ggcGFnZSBlbnRyeSB1cGRhdGUuIFNvIHdl IHByZWZlciB0bw0KPiA+ID4gdXNlIGlvdGxiX3N5bmNfcmFuZ2UoKSB0byByZXBsYWNlIGlvdGxi X3N5bmMoKSwgdGxiX2FkZF9yYW5nZSgpIGFuZA0KPiA+ID4gdGxiX2ZsdXNoX3dhbGsvbGVhZigp IGZvciBNVEsgcGxhdGZvcm1zLg0KPiA+IA0KPiA+IEluIHRoZSBjYXNlIG9mIG1hcHBpbmcsIGl0 IHNvdW5kcyBsaWtlIHdoYXQgeW91IGFjdHVhbGx5IHdhbnQgdG8gZG8gaXMgDQo+ID4gaG9vayB1 cCAuaW90bGJfc3luY19tYXAgYW5kIGdlbmVyYWxseSBtYWtlIElPX1BHVEFCTEVfUVVJUktfVExC SV9PTl9NQVAgDQo+ID4gY2xldmVyZXIsIGJlY2F1c2UgdGhlIGN1cnJlbnQgaW1wbGVtZW50YXRp b24gaXMgYXMgZHVtYiBhcyBpdCBjb3VsZCANCj4gPiBwb3NzaWJseSBiZS4gDQo+IA0KPiBpb3Rs Yl9zeW5jX21hcCBvbmx5IGhhcyBvbmUgcGFyYW1ldGVyKGlvbW11X2RvbWFpbiksIGJ1dCBtdGsN Cj4gaW9tbXVfZG9tYWluIG1heWJlIGluY2x1ZGUgdGhlIHdob2xlIGlvdmEgc3BhY2UsIGlmIG10 a19pb21tdSB0byBkbyB0bGINCj4gc3luYyBiYXNlZCBvbiBpb21tdV9kb21haW4sIGl0IGlzIGVx dWl2YWxlbnQgdG8gZG8gdGxiIGZsdXNoIGFsbCBpbg0KPiBmYWN0Lg0KPiBpb21tdSBkcml2ZXIg d2lsbCBkbyB0bGIgc3luYyBpbiBldmVyeSBtYXBwaW5nIHBhZ2Ugd2hlbiBtdGsgaW9tbXUgc2V0 cw0KPiBJT19QR1RBQkxFX1FVSVJLX1RMQklfT05fTUFQKGlvX3BndGFibGVfdGxiX2ZsdXNoX3dh bGspLA0KPiBhcyBpcyB0aGUgY29tbWl0IG1lc3NhZ2UgbWVudGlvbmVkLCBpdCB3aWxsIGRyb3Ag bWFwcGluZyBwZXJmb3JtYW5jZSBpbg0KPiBtdGsgcGxhdGZvcm0uDQo+IA0KPiANCj4gPiBJbiBm YWN0IGlmIHdlIHNpbXBseSBwYXNzZWQgYW4gYWRkcmVzcyByYW5nZSB0byANCj4gPiAuaW90bGJf c3luY19tYXAsIGlvLXBndGFibGUgcHJvYmFibHkgd291bGRuJ3QgbmVlZCB0byBiZSBpbnZvbHZl ZCBhdCBhbGwgDQo+ID4gYW55IG1vcmUuDQpTb3JyeSwgSSBmb3JnZXQgdG8gcmVwbHkgdGhlIHF1 ZXN0aW9uIGluIHByZXZpb3VzIG1haWwuDQpEbyB5b3UgbWVhbiB3ZSBuZWVkIHRvIG1vZGlmeSBp b3RsYl9zeW5jX21hcCgpIGlucHV0IHBhcmFtZXRlcihleDogYWRkDQpzdGFydC9lbmQgaW92YSk/ DQoNCj4gDQo+IEkga25vdyBpdCBpcyBub3QgYSBnb29kIGlkZWEgcHJvYmFibHkgYnkgYWRkaW5n IGEgbmV3IGFwaSwgYnV0IEkgZm91bmQNCj4gb3V0IHRoYXQgdGxiIHN5bmMgb25seSB0byBiZSBk b25lIGFmdGVyIG1hcHBpbmcgb25lIHBhZ2UsIHNvIGlmDQo+IG10a19pb21tdSBob3BlIHRvIGRv IHRsYiBzeW5jIG9uY2UgYWZ0ZXIgYWxsIHRoZSBwYWdlcyBtYXAgZG9uZSwgY291bGQNCj4geW91 IGdpdmUgbWUgc29tZSBhZHZpY2VzPyB0aGFua3MhDQo+IA0KPiA+IA0KPiA+IFJvYmluLg0KPiA+ IA0KPiA+ID4gU2lnbmVkLW9mZi1ieTogQ2hhbyBIYW8gPGNoYW8uaGFvQG1lZGlhdGVrLmNvbT4N Cj4gPiA+IC0tLQ0KPiA+ID4gICBkcml2ZXJzL2lvbW11L210a19pb21tdS5jIHwgNiArKysrKysN Cj4gPiA+ICAgMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQ0KPiA+ID4gDQo+ID4gPiBk aWZmIC0tZ2l0IGEvZHJpdmVycy9pb21tdS9tdGtfaW9tbXUuYyBiL2RyaXZlcnMvaW9tbXUvbXRr X2lvbW11LmMNCj4gPiA+IGluZGV4IDc4NWIyMjhkMzlhNi4uZDM0MDBjMTVmZjdiIDEwMDY0NA0K PiA+ID4gLS0tIGEvZHJpdmVycy9pb21tdS9tdGtfaW9tbXUuYw0KPiA+ID4gKysrIGIvZHJpdmVy cy9pb21tdS9tdGtfaW9tbXUuYw0KPiA+ID4gQEAgLTIyNCw2ICsyMjQsMTEgQEAgc3RhdGljIHZv aWQgbXRrX2lvbW11X3RsYl9mbHVzaF9yYW5nZV9zeW5jKHVuc2lnbmVkIGxvbmcgaW92YSwgc2l6 ZV90IHNpemUsDQo+ID4gPiAgIAl9DQo+ID4gPiAgIH0NCj4gPiA+ICAgDQo+ID4gPiArc3RhdGlj IHZvaWQgX19tdGtfaW9tbXVfdGxiX2ZsdXNoX3JhbmdlX3N5bmModW5zaWduZWQgbG9uZyBpb3Zh LCBzaXplX3Qgc2l6ZSkNCj4gPiA+ICt7DQo+ID4gPiArCW10a19pb21tdV90bGJfZmx1c2hfcmFu Z2Vfc3luYyhpb3ZhLCBzaXplLCAwLCBOVUxMKQ0KPiA+ID4gK30NCj4gPiA+ICsNCj4gPiA+ICAg c3RhdGljIHZvaWQgbXRrX2lvbW11X3RsYl9mbHVzaF9wYWdlX25vc3luYyhzdHJ1Y3QgaW9tbXVf aW90bGJfZ2F0aGVyICpnYXRoZXIsDQo+ID4gPiAgIAkJCQkJICAgIHVuc2lnbmVkIGxvbmcgaW92 YSwgc2l6ZV90IGdyYW51bGUsDQo+ID4gPiAgIAkJCQkJICAgIHZvaWQgKmNvb2tpZSkNCj4gPiA+ IEBAIC01MzYsNiArNTQxLDcgQEAgc3RhdGljIGNvbnN0IHN0cnVjdCBpb21tdV9vcHMgbXRrX2lv bW11X29wcyA9IHsNCj4gPiA+ICAgCS5tYXAJCT0gbXRrX2lvbW11X21hcCwNCj4gPiA+ICAgCS51 bm1hcAkJPSBtdGtfaW9tbXVfdW5tYXAsDQo+ID4gPiAgIAkuZmx1c2hfaW90bGJfYWxsID0gbXRr X2lvbW11X2ZsdXNoX2lvdGxiX2FsbCwNCj4gPiA+ICsJLmlvdGxiX3N5bmNfcmFuZ2UgPSBfX210 a19pb21tdV90bGJfZmx1c2hfcmFuZ2Vfc3luYywNCj4gPiA+ICAgCS5pb3RsYl9zeW5jCT0gbXRr X2lvbW11X2lvdGxiX3N5bmMsDQo+ID4gPiAgIAkuaW92YV90b19waHlzCT0gbXRrX2lvbW11X2lv dmFfdG9fcGh5cywNCj4gPiA+ICAgCS5wcm9iZV9kZXZpY2UJPSBtdGtfaW9tbXVfcHJvYmVfZGV2 aWNlLA0KPiA+ID4gDQo+IA0KDQo=