From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9673C4BA0B for ; Wed, 26 Feb 2020 06:37:37 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A0778206E2 for ; Wed, 26 Feb 2020 06:37:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="W0MYxIP0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A0778206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 340BA86AB1; Wed, 26 Feb 2020 06:37:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MkA5StpzDp7U; Wed, 26 Feb 2020 06:37:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 3391D86108; Wed, 26 Feb 2020 06:37:35 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03255C18DA; Wed, 26 Feb 2020 06:37:35 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 34348C0177 for ; Wed, 26 Feb 2020 06:37:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 1C3968550D for ; Wed, 26 Feb 2020 06:37:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2lQahvQss4C6 for ; Wed, 26 Feb 2020 06:37:31 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by whitealder.osuosl.org (Postfix) with ESMTP id EDE9F810F5 for ; Wed, 26 Feb 2020 06:37:30 +0000 (UTC) X-UUID: 1b097331b3a745f1a9e4250c5c2a374c-20200226 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=sNVknrlIb9UubW2Xtv3IENlncX0Wt6DkRbVInod5eos=; b=W0MYxIP0ipAW6V+u8HfRqk82USGGiHzavuRoGjY+/dmwCGiq0SNtjhHCavFsH1ueTASym9q4iEC6suGMiurWzqMRAk9Ujmzz/MhlJYFEZ5dtWTAWSgQpa70xZLBBI3su1An5TIwte/Sye8xN8d+TXIdOS1jRz+ewLzph7ZMQNVc=; X-UUID: 1b097331b3a745f1a9e4250c5c2a374c-20200226 Received: from mtkexhb01.mediatek.inc [(172.21.101.102)] by mailgw02.mediatek.com (envelope-from ) (Cellopoint E-mail Firewall v4.1.10 Build 0809 with TLS) with ESMTP id 1468911021; Wed, 26 Feb 2020 14:37:27 +0800 Received: from mtkcas08.mediatek.inc (172.21.101.126) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 26 Feb 2020 14:36:33 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas08.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1395.4 via Frontend Transport; Wed, 26 Feb 2020 14:37:33 +0800 Message-ID: <1582698979.27773.17.camel@mbjsdccf07> Subject: Re: [RESEND,PATCH 03/13] iommu/mediatek: Add mtk_iommu_pgtable structure From: chao hao To: Matthias Brugger , Joerg Roedel , Yong Wu Date: Wed, 26 Feb 2020 14:36:19 +0800 In-Reply-To: <1581769046.32039.27.camel@mhfsdcap03> References: <20191104115238.2394-1-chao.hao@mediatek.com> <20191104115238.2394-4-chao.hao@mediatek.com> <1576498396.28043.78.camel@mhfsdcap03> <1577785148.30177.5.camel@mbjsdccf07> <1581769046.32039.27.camel@mhfsdcap03> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N Cc: Anan Sun =?UTF-8?Q?=28=E5=AD=99=E5=AE=89=E5=AE=89=29?= , "devicetree@vger.kernel.org" , Cui Zhang =?UTF-8?Q?=28=E5=BC=A0=E7=BF=A0=29?= , Jun Yan =?UTF-8?Q?=28=E9=A2=9C=E5=86=9B=29?= , wsd_upstream , youlin.pei@mediatek.com, "linux-kernel@vger.kernel.org" , chao.hao@mediatek.com, "iommu@lists.linux-foundation.org" , Rob Herring , "linux-mediatek@lists.infradead.org" , Miles Chen =?UTF-8?Q?=28=E9=99=B3=E6=B0=91=E6=A8=BA=29?= , Matthias Brugger , fy.yang@mediatek.com, "linux-arm-kernel@lists.infradead.org" , Guangming Cao =?UTF-8?Q?=28=E6=9B=B9=E5=85=89=E6=98=8E=29?= X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Sat, 2020-02-15 at 20:17 +0800, Yong Wu wrote: > On Tue, 2019-12-31 at 17:39 +0800, chao hao wrote: > > On Mon, 2019-12-16 at 20:13 +0800, Yong Wu wrote: > > > On Mon, 2019-11-04 at 19:52 +0800, Chao Hao wrote: > > > > Start with this patch, we will change the SW architecture > > > > to support multiple domains. SW architecture will has a big change, > > > > so we need to modify a little bit by more than one patch. > > > > The new SW overall architecture is as below: > > > > > > > > iommu0 iommu1 > > > > | | > > > > ----------- > > > > | > > > > mtk_iommu_pgtable > > > > | > > > > ------------------------------------------ > > > > | | | > > > > mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 > > > > | | | > > > > iommu_group1 iommu_group2 iommu_group3 > > > > | | | > > > > iommu_domain1 iommu_domain2 iommu_domain3 > > > > | | | > > > > iova region1(normal) iova region2(CCU) iova region3(VPU) > > > > > > > > For current structure, no matter how many iommus there are, > > > > they use the same page table to simplify the usage of module. > > > > In order to make the software architecture more explicit, this > > > > patch will create a global mtk_iommu_pgtable structure to describe > > > > page table and all the iommus use it. > > > > > > Thanks for the hard work of this file. Actually this patch and the later > > > ones confuse me. Why do you make this flow change? > > > for making the code "more explicit" or for adding multi-domain support > > > in 13/13. > > > > > > IMHO, the change is unnecessary. > > > a) For me, this change has no improvement. currently we use a global > > > mtk_iommu_get_m4u_data to get the M4U data. I will be very glad if you > > > could get rid of it. But in this patchset, You use a another global > > > mtk_iommu_pgtable to instead. For me. It has no improvement. > > > > Thanks for you advice! > > > > For current SW arch, all the IOMMU HW use the same page table, we can > > use a global mtk_iommu_pgtable to discribe the information of page table > > What's your plan if the 4GB iova range is not enough for us in future? > Do you plan to add a new global mtk_iommu_pgtable again? > > > and all the IOMMU attach it, I think that it is more clear and > > unambiguous. For beginners, it maybe more easily explicable? > > I still don't get the necessity of this change. it is only for making > code clear from your point for view, right? > > This code has been reviewed for many years, I don't know why you think > it is ambiguous. it is clear for me at lease. and I will complain that > you add a new global variable in this change. > > > > > > > b) This patchset break the original flow. device_group give you a > > > software chance for initializing, then you move pagetable allocating > > > code into it. But it isn't device_group job. > > > > > > > As is shown above diagram, mtk_iommu_pgtable includes iommu_group and > > iommu_domain,so we need to allocate mtk_iommu_pgtable and initialize it > > in device_group firstly,and then execute the original flow, it only > > changes place for creating mtk_iommu_pgtable and don't break original > > device_group flow. > > I understand you have to do this change after you adjust the structure. > I mean that it may be not proper since allocating pagetable should not > be done in device_group logically. From here, Could we get this change > looks not good?. > gentle ping ... Dear Matthias and Joerg, >From mt6779 platform, mtk_iommu.c needs to support multiple domains for different iova regions.About the change, there are some disagreements among our internal. We hope to get your helps and advices: Based on current SW architecture to support multiple domain, diagram is as below: iommu0 iommu1 | | ---------- | ------------------------------------------ | | | iommu_group1 iommu_group2 iommu_group3 | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) PS: the information of page table is included struct mtk_iommu_domain In my opinion, if all the iommus share the same page table(include all iova regions) and different iova regions are created based on the page table, we can put the information of page table to a global structure(mtk_iommu_pgtable) and all the iommus attach it. It maybe make the SW architecture look clearly. New SW architecture diagram is as below(This patchset is based on it): iommu0 iommu1 | | ---------- | mtk_iommu_pgtable | ------------------------------------------ | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iommu_group1 iommu_group2 iommu_group3 | | | iommu_domain1 iommu_domain2 iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) >From above new SW architecture, we will create a global page table firstly(mtk_iommu_pgtable), and all the iommus can use it. The page table can include 4GB iova space, different iova regions are created based on it, so the information of pgtable needs to be created in device_group. I have two problems to consult you, can you provide some advices? Thanks for Matthias and Joerg so much! (1) I don't understand if the pgtable can be created in device_group ? (2) In addition, can you help to review which SW architecture are more reasonable or whether it is necessary to change current SW architecture to support multiple domain ? Dear Yong, If I described ambiguously, please help to add extra explanation. Thanks a lot. > > > I can not decide if your flow is right. But if you only want to add > > > support multi-domain, I guess you could extend the current "m4u_group" > > > to a array "m4u_group[N]". It may be more simple. To make mt6779 > > > progress easily, I suggest you can use this way to support multi-domain > > > firstly. Then you could send this new mtk_iommu_pgtable patchset for the > > > code "more explicit" if you insist. > > Could you help try this way if it could meet your requirement? Then > let's compare which one is better. > > > BTW, your patches(including v2) cause hangup as below since > "data->m4u_dom" was uninitialized. > > > Unable to handle kernel NULL pointer dereference at virtual address > 0000000000000010 > ... > pc : mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > lr : __arm_v7s_unmap+0x174/0x598 > ... > Call trace: > mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > __arm_v7s_unmap+0x174/0x598 > arm_v7s_unmap+0x30/0x48 > mtk_iommu_unmap+0x20/0x28 > __iommu_unmap+0xa4/0xf8 > iommu_unmap+0x44/0x90 > > > > > > > > The diagram is as below: > > > > > > > > mtk_iommu_data1(MM) mtk_iommu_data2(APU) > > > > | | > > > > | | > > > > ------mtk_iommu_pgtable----- > > > > > > > > We need to create global mtk_iommu_pgtable to include all the iova > > > > regions firstly and special iova regions by divided based on it, > > > > so the information of pgtable needs to be created in device_group. > > > > > > > > Signed-off-by: Chao Hao > > > > --- > > > > drivers/iommu/mtk_iommu.c | 84 +++++++++++++++++++++++++++++++++++++++ > > > > drivers/iommu/mtk_iommu.h | 1 + > > > > 2 files changed, 85 insertions(+) > > > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > > index f2847e661137..fcbde6b0f58d 100644 > > > > --- a/drivers/iommu/mtk_iommu.c > > > > +++ b/drivers/iommu/mtk_iommu.c > > > > @@ -123,6 +123,12 @@ struct mtk_iommu_domain { > > > > struct iommu_domain domain; > > > > }; > > > > > > > > +struct mtk_iommu_pgtable { > > > > + struct io_pgtable_cfg cfg; > > > > + struct io_pgtable_ops *iop; > > > > +}; > > > > + > > > > +static struct mtk_iommu_pgtable *share_pgtable; > > > > static const struct iommu_ops mtk_iommu_ops; > > > > > > > > /* > > > > @@ -170,6 +176,11 @@ static struct mtk_iommu_data *mtk_iommu_get_m4u_data(void) > > > > return NULL; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *mtk_iommu_get_pgtable(void) > > > > +{ > > > > + return share_pgtable; > > > > +} > > > > + > > > > static struct mtk_iommu_domain *to_mtk_domain(struct iommu_domain *dom) > > > > { > > > > return container_of(dom, struct mtk_iommu_domain, domain); > > > > @@ -322,6 +333,13 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > > > > > + if (data->pgtable) { > > > > + dom->cfg = data->pgtable->cfg; > > > > + dom->iop = data->pgtable->iop; > > > > + dom->domain.pgsize_bitmap = data->pgtable->cfg.pgsize_bitmap; > > > > + return 0; > > > > + } > > > > + > > > > dom->cfg = (struct io_pgtable_cfg) { > > > > .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > IO_PGTABLE_QUIRK_NO_PERMS | > > > > @@ -345,6 +363,61 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > return 0; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *create_pgtable(struct mtk_iommu_data *data) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + > > > > + pgtable = kzalloc(sizeof(*pgtable), GFP_KERNEL); > > > > + if (!pgtable) > > > > + return ERR_PTR(-ENOMEM); > > > > + > > > > + pgtable->cfg = (struct io_pgtable_cfg) { > > > > + .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > + IO_PGTABLE_QUIRK_NO_PERMS | > > > > + IO_PGTABLE_QUIRK_TLBI_ON_MAP | > > > > + IO_PGTABLE_QUIRK_ARM_MTK_EXT, > > > > + .pgsize_bitmap = mtk_iommu_ops.pgsize_bitmap, > > > > + .ias = 32, > > > > + .oas = 34, > > > > + .tlb = &mtk_iommu_flush_ops, > > > > + .iommu_dev = data->dev, > > > > + }; > > > > + > > > > + pgtable->iop = alloc_io_pgtable_ops(ARM_V7S, &pgtable->cfg, data); > > > > + if (!pgtable->iop) { > > > > + dev_err(data->dev, "Failed to alloc io pgtable\n"); > > > > + return ERR_PTR(-EINVAL); > > > > + } > > > > + > > > > + dev_info(data->dev, "%s create pgtable done\n", __func__); > > > > + > > > > + return pgtable; > > > > +} > > > > + > > > > +static int mtk_iommu_attach_pgtable(struct mtk_iommu_data *data, > > > > + struct device *dev) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable = mtk_iommu_get_pgtable(); > > > > + > > > > + /* create share pgtable */ > > > > + if (!pgtable) { > > > > + pgtable = create_pgtable(data); > > > > + if (IS_ERR(pgtable)) { > > > > + dev_err(data->dev, "Failed to create pgtable\n"); > > > > + return -ENOMEM; > > > > + } > > > > + > > > > + share_pgtable = pgtable; > > > > + } > > > > + > > > > + /* binding to pgtable */ > > > > + data->pgtable = pgtable; > > > > + > > > > + dev_info(data->dev, "m4u%d attach_pgtable done!\n", data->m4u_id); > > > > + > > > > + return 0; > > > > +} > > > > + > > > > static struct iommu_domain *mtk_iommu_domain_alloc(unsigned type) > > > > { > > > > struct mtk_iommu_domain *dom; > > > > @@ -508,10 +581,21 @@ static void mtk_iommu_remove_device(struct device *dev) > > > > static struct iommu_group *mtk_iommu_device_group(struct device *dev) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + int ret = 0; > > > > > > > > if (!data) > > > > return ERR_PTR(-ENODEV); > > > > > > > > + pgtable = data->pgtable; > > > > + if (!pgtable) { > > > > + ret = mtk_iommu_attach_pgtable(data, dev); > > > > + if (ret) { > > > > + dev_err(data->dev, "Failed to device_group\n"); > > > > + return NULL; > > > > + } > > > > + } > > > > + > > > > /* All the client devices are in the same m4u iommu-group */ > > > > if (!data->m4u_group) { > > > > data->m4u_group = iommu_group_alloc(); > > > > diff --git a/drivers/iommu/mtk_iommu.h b/drivers/iommu/mtk_iommu.h > > > > index 132dc765a40b..dd5f19f78b62 100644 > > > > --- a/drivers/iommu/mtk_iommu.h > > > > +++ b/drivers/iommu/mtk_iommu.h > > > > @@ -61,6 +61,7 @@ struct mtk_iommu_data { > > > > struct clk *bclk; > > > > phys_addr_t protect_base; /* protect memory base */ > > > > struct mtk_iommu_suspend_reg reg; > > > > + struct mtk_iommu_pgtable *pgtable; > > > > struct mtk_iommu_domain *m4u_dom; > > > > struct iommu_group *m4u_group; > > > > bool enable_4GB; > > > > > > > > > > > > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC076C4BA0B for ; Wed, 26 Feb 2020 06:48:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7B1D12084E for ; Wed, 26 Feb 2020 06:48:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="AR0RkOOR"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="OGCs3kP2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B1D12084E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Nu0hf8BauWnoqNViH8i8P5HHeu/OdyOMqEnMquy4has=; b=AR0RkOOR6D2UYu FxsRQuC3dAuYz1ckzltBWwejHvYSO2p+PIANu2tNHJVFwBEeTKny4dzKQwADY99Lh8JHm6QlAXv7G oe1NjoOYXb8puz3RJ+uC3gfdqI2HI9jcbMN3FSXxgsICMnR81hTBwUuIx1g0iKOejv/qTvFq3Wwpi F+mvt+BGt+LC9DS4S2RIvV+zRYbAbl3Ag9mmZE8D0ekugENlcxX7Q5UE65vDP8jL5ZzwHke6Zj6Mv 3TWvKARADPnExuw+BYIcXI6lxdgcE6zlUN8rt9h26Nxd7FfAvoL0UPCGDp1ys61I4RCMsZoc54NcE jlqT1FOGfy0M3+CS9jlQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6qU2-00039K-33; Wed, 26 Feb 2020 06:47:50 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6qTo-00030o-TJ; Wed, 26 Feb 2020 06:47:38 +0000 X-UUID: 5f3b22708ee74800967b7cd00c3f5c8a-20200225 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=sNVknrlIb9UubW2Xtv3IENlncX0Wt6DkRbVInod5eos=; b=OGCs3kP2+/1CiLo9t89UXcoCLu3c5XZZ/2rMM09u8vdDOxWV6kLTKMmcrjtPDj2rlMS+vclVJjmCKZeiUdraShkOcoGkAQDXVUs6Q23RFyk/Ol+CuEunlI+nDaPXxvPSfqS2wsbuB8+KsSjtRFZyAf8/WoKB3LV7MWigEc2EC3A=; X-UUID: 5f3b22708ee74800967b7cd00c3f5c8a-20200225 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLS) with ESMTP id 1285811432; Tue, 25 Feb 2020 22:47:29 -0800 Received: from mtkmbs07n1.mediatek.inc (172.21.101.16) by MTKMBS62DR.mediatek.inc (172.29.94.18) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 25 Feb 2020 22:37:26 -0800 Received: from mtkcas08.mediatek.inc (172.21.101.126) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 26 Feb 2020 14:36:33 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas08.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1395.4 via Frontend Transport; Wed, 26 Feb 2020 14:37:33 +0800 Message-ID: <1582698979.27773.17.camel@mbjsdccf07> Subject: Re: [RESEND,PATCH 03/13] iommu/mediatek: Add mtk_iommu_pgtable structure From: chao hao To: Matthias Brugger , Joerg Roedel , Yong Wu Date: Wed, 26 Feb 2020 14:36:19 +0800 In-Reply-To: <1581769046.32039.27.camel@mhfsdcap03> References: <20191104115238.2394-1-chao.hao@mediatek.com> <20191104115238.2394-4-chao.hao@mediatek.com> <1576498396.28043.78.camel@mhfsdcap03> <1577785148.30177.5.camel@mbjsdccf07> <1581769046.32039.27.camel@mhfsdcap03> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200225_224736_957880_34261D4D X-CRM114-Status: GOOD ( 40.65 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anan Sun =?UTF-8?Q?=28=E5=AD=99=E5=AE=89=E5=AE=89=29?= , "devicetree@vger.kernel.org" , Cui Zhang =?UTF-8?Q?=28=E5=BC=A0=E7=BF=A0=29?= , Jun Yan =?UTF-8?Q?=28=E9=A2=9C=E5=86=9B=29?= , wsd_upstream , Joerg Roedel , youlin.pei@mediatek.com, "linux-kernel@vger.kernel.org" , chao.hao@mediatek.com, "iommu@lists.linux-foundation.org" , Rob Herring , "linux-mediatek@lists.infradead.org" , Miles Chen =?UTF-8?Q?=28=E9=99=B3=E6=B0=91=E6=A8=BA=29?= , Matthias Brugger , fy.yang@mediatek.com, "linux-arm-kernel@lists.infradead.org" , Guangming Cao =?UTF-8?Q?=28=E6=9B=B9=E5=85=89=E6=98=8E=29?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On Sat, 2020-02-15 at 20:17 +0800, Yong Wu wrote: > On Tue, 2019-12-31 at 17:39 +0800, chao hao wrote: > > On Mon, 2019-12-16 at 20:13 +0800, Yong Wu wrote: > > > On Mon, 2019-11-04 at 19:52 +0800, Chao Hao wrote: > > > > Start with this patch, we will change the SW architecture > > > > to support multiple domains. SW architecture will has a big change, > > > > so we need to modify a little bit by more than one patch. > > > > The new SW overall architecture is as below: > > > > > > > > iommu0 iommu1 > > > > | | > > > > ----------- > > > > | > > > > mtk_iommu_pgtable > > > > | > > > > ------------------------------------------ > > > > | | | > > > > mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 > > > > | | | > > > > iommu_group1 iommu_group2 iommu_group3 > > > > | | | > > > > iommu_domain1 iommu_domain2 iommu_domain3 > > > > | | | > > > > iova region1(normal) iova region2(CCU) iova region3(VPU) > > > > > > > > For current structure, no matter how many iommus there are, > > > > they use the same page table to simplify the usage of module. > > > > In order to make the software architecture more explicit, this > > > > patch will create a global mtk_iommu_pgtable structure to describe > > > > page table and all the iommus use it. > > > > > > Thanks for the hard work of this file. Actually this patch and the later > > > ones confuse me. Why do you make this flow change? > > > for making the code "more explicit" or for adding multi-domain support > > > in 13/13. > > > > > > IMHO, the change is unnecessary. > > > a) For me, this change has no improvement. currently we use a global > > > mtk_iommu_get_m4u_data to get the M4U data. I will be very glad if you > > > could get rid of it. But in this patchset, You use a another global > > > mtk_iommu_pgtable to instead. For me. It has no improvement. > > > > Thanks for you advice! > > > > For current SW arch, all the IOMMU HW use the same page table, we can > > use a global mtk_iommu_pgtable to discribe the information of page table > > What's your plan if the 4GB iova range is not enough for us in future? > Do you plan to add a new global mtk_iommu_pgtable again? > > > and all the IOMMU attach it, I think that it is more clear and > > unambiguous. For beginners, it maybe more easily explicable? > > I still don't get the necessity of this change. it is only for making > code clear from your point for view, right? > > This code has been reviewed for many years, I don't know why you think > it is ambiguous. it is clear for me at lease. and I will complain that > you add a new global variable in this change. > > > > > > > b) This patchset break the original flow. device_group give you a > > > software chance for initializing, then you move pagetable allocating > > > code into it. But it isn't device_group job. > > > > > > > As is shown above diagram, mtk_iommu_pgtable includes iommu_group and > > iommu_domain,so we need to allocate mtk_iommu_pgtable and initialize it > > in device_group firstly,and then execute the original flow, it only > > changes place for creating mtk_iommu_pgtable and don't break original > > device_group flow. > > I understand you have to do this change after you adjust the structure. > I mean that it may be not proper since allocating pagetable should not > be done in device_group logically. From here, Could we get this change > looks not good?. > gentle ping ... Dear Matthias and Joerg, >From mt6779 platform, mtk_iommu.c needs to support multiple domains for different iova regions.About the change, there are some disagreements among our internal. We hope to get your helps and advices: Based on current SW architecture to support multiple domain, diagram is as below: iommu0 iommu1 | | ---------- | ------------------------------------------ | | | iommu_group1 iommu_group2 iommu_group3 | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) PS: the information of page table is included struct mtk_iommu_domain In my opinion, if all the iommus share the same page table(include all iova regions) and different iova regions are created based on the page table, we can put the information of page table to a global structure(mtk_iommu_pgtable) and all the iommus attach it. It maybe make the SW architecture look clearly. New SW architecture diagram is as below(This patchset is based on it): iommu0 iommu1 | | ---------- | mtk_iommu_pgtable | ------------------------------------------ | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iommu_group1 iommu_group2 iommu_group3 | | | iommu_domain1 iommu_domain2 iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) >From above new SW architecture, we will create a global page table firstly(mtk_iommu_pgtable), and all the iommus can use it. The page table can include 4GB iova space, different iova regions are created based on it, so the information of pgtable needs to be created in device_group. I have two problems to consult you, can you provide some advices? Thanks for Matthias and Joerg so much! (1) I don't understand if the pgtable can be created in device_group ? (2) In addition, can you help to review which SW architecture are more reasonable or whether it is necessary to change current SW architecture to support multiple domain ? Dear Yong, If I described ambiguously, please help to add extra explanation. Thanks a lot. > > > I can not decide if your flow is right. But if you only want to add > > > support multi-domain, I guess you could extend the current "m4u_group" > > > to a array "m4u_group[N]". It may be more simple. To make mt6779 > > > progress easily, I suggest you can use this way to support multi-domain > > > firstly. Then you could send this new mtk_iommu_pgtable patchset for the > > > code "more explicit" if you insist. > > Could you help try this way if it could meet your requirement? Then > let's compare which one is better. > > > BTW, your patches(including v2) cause hangup as below since > "data->m4u_dom" was uninitialized. > > > Unable to handle kernel NULL pointer dereference at virtual address > 0000000000000010 > ... > pc : mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > lr : __arm_v7s_unmap+0x174/0x598 > ... > Call trace: > mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > __arm_v7s_unmap+0x174/0x598 > arm_v7s_unmap+0x30/0x48 > mtk_iommu_unmap+0x20/0x28 > __iommu_unmap+0xa4/0xf8 > iommu_unmap+0x44/0x90 > > > > > > > > The diagram is as below: > > > > > > > > mtk_iommu_data1(MM) mtk_iommu_data2(APU) > > > > | | > > > > | | > > > > ------mtk_iommu_pgtable----- > > > > > > > > We need to create global mtk_iommu_pgtable to include all the iova > > > > regions firstly and special iova regions by divided based on it, > > > > so the information of pgtable needs to be created in device_group. > > > > > > > > Signed-off-by: Chao Hao > > > > --- > > > > drivers/iommu/mtk_iommu.c | 84 +++++++++++++++++++++++++++++++++++++++ > > > > drivers/iommu/mtk_iommu.h | 1 + > > > > 2 files changed, 85 insertions(+) > > > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > > index f2847e661137..fcbde6b0f58d 100644 > > > > --- a/drivers/iommu/mtk_iommu.c > > > > +++ b/drivers/iommu/mtk_iommu.c > > > > @@ -123,6 +123,12 @@ struct mtk_iommu_domain { > > > > struct iommu_domain domain; > > > > }; > > > > > > > > +struct mtk_iommu_pgtable { > > > > + struct io_pgtable_cfg cfg; > > > > + struct io_pgtable_ops *iop; > > > > +}; > > > > + > > > > +static struct mtk_iommu_pgtable *share_pgtable; > > > > static const struct iommu_ops mtk_iommu_ops; > > > > > > > > /* > > > > @@ -170,6 +176,11 @@ static struct mtk_iommu_data *mtk_iommu_get_m4u_data(void) > > > > return NULL; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *mtk_iommu_get_pgtable(void) > > > > +{ > > > > + return share_pgtable; > > > > +} > > > > + > > > > static struct mtk_iommu_domain *to_mtk_domain(struct iommu_domain *dom) > > > > { > > > > return container_of(dom, struct mtk_iommu_domain, domain); > > > > @@ -322,6 +333,13 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > > > > > + if (data->pgtable) { > > > > + dom->cfg = data->pgtable->cfg; > > > > + dom->iop = data->pgtable->iop; > > > > + dom->domain.pgsize_bitmap = data->pgtable->cfg.pgsize_bitmap; > > > > + return 0; > > > > + } > > > > + > > > > dom->cfg = (struct io_pgtable_cfg) { > > > > .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > IO_PGTABLE_QUIRK_NO_PERMS | > > > > @@ -345,6 +363,61 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > return 0; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *create_pgtable(struct mtk_iommu_data *data) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + > > > > + pgtable = kzalloc(sizeof(*pgtable), GFP_KERNEL); > > > > + if (!pgtable) > > > > + return ERR_PTR(-ENOMEM); > > > > + > > > > + pgtable->cfg = (struct io_pgtable_cfg) { > > > > + .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > + IO_PGTABLE_QUIRK_NO_PERMS | > > > > + IO_PGTABLE_QUIRK_TLBI_ON_MAP | > > > > + IO_PGTABLE_QUIRK_ARM_MTK_EXT, > > > > + .pgsize_bitmap = mtk_iommu_ops.pgsize_bitmap, > > > > + .ias = 32, > > > > + .oas = 34, > > > > + .tlb = &mtk_iommu_flush_ops, > > > > + .iommu_dev = data->dev, > > > > + }; > > > > + > > > > + pgtable->iop = alloc_io_pgtable_ops(ARM_V7S, &pgtable->cfg, data); > > > > + if (!pgtable->iop) { > > > > + dev_err(data->dev, "Failed to alloc io pgtable\n"); > > > > + return ERR_PTR(-EINVAL); > > > > + } > > > > + > > > > + dev_info(data->dev, "%s create pgtable done\n", __func__); > > > > + > > > > + return pgtable; > > > > +} > > > > + > > > > +static int mtk_iommu_attach_pgtable(struct mtk_iommu_data *data, > > > > + struct device *dev) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable = mtk_iommu_get_pgtable(); > > > > + > > > > + /* create share pgtable */ > > > > + if (!pgtable) { > > > > + pgtable = create_pgtable(data); > > > > + if (IS_ERR(pgtable)) { > > > > + dev_err(data->dev, "Failed to create pgtable\n"); > > > > + return -ENOMEM; > > > > + } > > > > + > > > > + share_pgtable = pgtable; > > > > + } > > > > + > > > > + /* binding to pgtable */ > > > > + data->pgtable = pgtable; > > > > + > > > > + dev_info(data->dev, "m4u%d attach_pgtable done!\n", data->m4u_id); > > > > + > > > > + return 0; > > > > +} > > > > + > > > > static struct iommu_domain *mtk_iommu_domain_alloc(unsigned type) > > > > { > > > > struct mtk_iommu_domain *dom; > > > > @@ -508,10 +581,21 @@ static void mtk_iommu_remove_device(struct device *dev) > > > > static struct iommu_group *mtk_iommu_device_group(struct device *dev) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + int ret = 0; > > > > > > > > if (!data) > > > > return ERR_PTR(-ENODEV); > > > > > > > > + pgtable = data->pgtable; > > > > + if (!pgtable) { > > > > + ret = mtk_iommu_attach_pgtable(data, dev); > > > > + if (ret) { > > > > + dev_err(data->dev, "Failed to device_group\n"); > > > > + return NULL; > > > > + } > > > > + } > > > > + > > > > /* All the client devices are in the same m4u iommu-group */ > > > > if (!data->m4u_group) { > > > > data->m4u_group = iommu_group_alloc(); > > > > diff --git a/drivers/iommu/mtk_iommu.h b/drivers/iommu/mtk_iommu.h > > > > index 132dc765a40b..dd5f19f78b62 100644 > > > > --- a/drivers/iommu/mtk_iommu.h > > > > +++ b/drivers/iommu/mtk_iommu.h > > > > @@ -61,6 +61,7 @@ struct mtk_iommu_data { > > > > struct clk *bclk; > > > > phys_addr_t protect_base; /* protect memory base */ > > > > struct mtk_iommu_suspend_reg reg; > > > > + struct mtk_iommu_pgtable *pgtable; > > > > struct mtk_iommu_domain *m4u_dom; > > > > struct iommu_group *m4u_group; > > > > bool enable_4GB; > > > > > > > > > > > > _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E456C4BA0A for ; Wed, 26 Feb 2020 06:47:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 22F5A2084E for ; Wed, 26 Feb 2020 06:47:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XTvHDJz/"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="OGCs3kP2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 22F5A2084E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7GQOJ1rgS4vamc6qH93bNweveTR1SxQuCJX8cc473wU=; b=XTvHDJz/mAzuHj 54zBUYcRf8qoRyE/e8fLI3BfvkFrqlJjFCG5q5l5WKVy401SGrbnkaRfuzGaTJw2yKJ1VkfVksrnQ XpSzI7REISiiWZQ1LkXR5/qBMTNu13TkqMtjvZhr9JElZl01jI6fYNlkWBm8mineng9k2MJvCAdq7 ThcyM8dCiFFPYh171pBaiDQFlu6DvPudGj6aWCfgYxGbKN9QmdmPdYvB7iZHxF0zeD7K6gI6colF0 2ZlTm/6eFj6VeT+rWFB4TfGWou0X7n2Gieg4vtWUYL0mZ091SAYH3q/S5T/2Ihn74q/KigOQUZcG+ nVEeZfD4wyrE44vvCPtg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6qTs-00031F-EM; Wed, 26 Feb 2020 06:47:40 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6qTo-00030o-TJ; Wed, 26 Feb 2020 06:47:38 +0000 X-UUID: 5f3b22708ee74800967b7cd00c3f5c8a-20200225 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=sNVknrlIb9UubW2Xtv3IENlncX0Wt6DkRbVInod5eos=; b=OGCs3kP2+/1CiLo9t89UXcoCLu3c5XZZ/2rMM09u8vdDOxWV6kLTKMmcrjtPDj2rlMS+vclVJjmCKZeiUdraShkOcoGkAQDXVUs6Q23RFyk/Ol+CuEunlI+nDaPXxvPSfqS2wsbuB8+KsSjtRFZyAf8/WoKB3LV7MWigEc2EC3A=; X-UUID: 5f3b22708ee74800967b7cd00c3f5c8a-20200225 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLS) with ESMTP id 1285811432; Tue, 25 Feb 2020 22:47:29 -0800 Received: from mtkmbs07n1.mediatek.inc (172.21.101.16) by MTKMBS62DR.mediatek.inc (172.29.94.18) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 25 Feb 2020 22:37:26 -0800 Received: from mtkcas08.mediatek.inc (172.21.101.126) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 26 Feb 2020 14:36:33 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas08.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1395.4 via Frontend Transport; Wed, 26 Feb 2020 14:37:33 +0800 Message-ID: <1582698979.27773.17.camel@mbjsdccf07> Subject: Re: [RESEND,PATCH 03/13] iommu/mediatek: Add mtk_iommu_pgtable structure From: chao hao To: Matthias Brugger , Joerg Roedel , Yong Wu Date: Wed, 26 Feb 2020 14:36:19 +0800 In-Reply-To: <1581769046.32039.27.camel@mhfsdcap03> References: <20191104115238.2394-1-chao.hao@mediatek.com> <20191104115238.2394-4-chao.hao@mediatek.com> <1576498396.28043.78.camel@mhfsdcap03> <1577785148.30177.5.camel@mbjsdccf07> <1581769046.32039.27.camel@mhfsdcap03> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200225_224736_957880_34261D4D X-CRM114-Status: GOOD ( 40.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anan Sun =?UTF-8?Q?=28=E5=AD=99=E5=AE=89=E5=AE=89=29?= , "devicetree@vger.kernel.org" , Cui Zhang =?UTF-8?Q?=28=E5=BC=A0=E7=BF=A0=29?= , Jun Yan =?UTF-8?Q?=28=E9=A2=9C=E5=86=9B=29?= , wsd_upstream , Joerg Roedel , youlin.pei@mediatek.com, "linux-kernel@vger.kernel.org" , chao.hao@mediatek.com, "iommu@lists.linux-foundation.org" , Rob Herring , "linux-mediatek@lists.infradead.org" , Miles Chen =?UTF-8?Q?=28=E9=99=B3=E6=B0=91=E6=A8=BA=29?= , Matthias Brugger , fy.yang@mediatek.com, "linux-arm-kernel@lists.infradead.org" , Guangming Cao =?UTF-8?Q?=28=E6=9B=B9=E5=85=89=E6=98=8E=29?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sat, 2020-02-15 at 20:17 +0800, Yong Wu wrote: > On Tue, 2019-12-31 at 17:39 +0800, chao hao wrote: > > On Mon, 2019-12-16 at 20:13 +0800, Yong Wu wrote: > > > On Mon, 2019-11-04 at 19:52 +0800, Chao Hao wrote: > > > > Start with this patch, we will change the SW architecture > > > > to support multiple domains. SW architecture will has a big change, > > > > so we need to modify a little bit by more than one patch. > > > > The new SW overall architecture is as below: > > > > > > > > iommu0 iommu1 > > > > | | > > > > ----------- > > > > | > > > > mtk_iommu_pgtable > > > > | > > > > ------------------------------------------ > > > > | | | > > > > mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 > > > > | | | > > > > iommu_group1 iommu_group2 iommu_group3 > > > > | | | > > > > iommu_domain1 iommu_domain2 iommu_domain3 > > > > | | | > > > > iova region1(normal) iova region2(CCU) iova region3(VPU) > > > > > > > > For current structure, no matter how many iommus there are, > > > > they use the same page table to simplify the usage of module. > > > > In order to make the software architecture more explicit, this > > > > patch will create a global mtk_iommu_pgtable structure to describe > > > > page table and all the iommus use it. > > > > > > Thanks for the hard work of this file. Actually this patch and the later > > > ones confuse me. Why do you make this flow change? > > > for making the code "more explicit" or for adding multi-domain support > > > in 13/13. > > > > > > IMHO, the change is unnecessary. > > > a) For me, this change has no improvement. currently we use a global > > > mtk_iommu_get_m4u_data to get the M4U data. I will be very glad if you > > > could get rid of it. But in this patchset, You use a another global > > > mtk_iommu_pgtable to instead. For me. It has no improvement. > > > > Thanks for you advice! > > > > For current SW arch, all the IOMMU HW use the same page table, we can > > use a global mtk_iommu_pgtable to discribe the information of page table > > What's your plan if the 4GB iova range is not enough for us in future? > Do you plan to add a new global mtk_iommu_pgtable again? > > > and all the IOMMU attach it, I think that it is more clear and > > unambiguous. For beginners, it maybe more easily explicable? > > I still don't get the necessity of this change. it is only for making > code clear from your point for view, right? > > This code has been reviewed for many years, I don't know why you think > it is ambiguous. it is clear for me at lease. and I will complain that > you add a new global variable in this change. > > > > > > > b) This patchset break the original flow. device_group give you a > > > software chance for initializing, then you move pagetable allocating > > > code into it. But it isn't device_group job. > > > > > > > As is shown above diagram, mtk_iommu_pgtable includes iommu_group and > > iommu_domain,so we need to allocate mtk_iommu_pgtable and initialize it > > in device_group firstly,and then execute the original flow, it only > > changes place for creating mtk_iommu_pgtable and don't break original > > device_group flow. > > I understand you have to do this change after you adjust the structure. > I mean that it may be not proper since allocating pagetable should not > be done in device_group logically. From here, Could we get this change > looks not good?. > gentle ping ... Dear Matthias and Joerg, >From mt6779 platform, mtk_iommu.c needs to support multiple domains for different iova regions.About the change, there are some disagreements among our internal. We hope to get your helps and advices: Based on current SW architecture to support multiple domain, diagram is as below: iommu0 iommu1 | | ---------- | ------------------------------------------ | | | iommu_group1 iommu_group2 iommu_group3 | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) PS: the information of page table is included struct mtk_iommu_domain In my opinion, if all the iommus share the same page table(include all iova regions) and different iova regions are created based on the page table, we can put the information of page table to a global structure(mtk_iommu_pgtable) and all the iommus attach it. It maybe make the SW architecture look clearly. New SW architecture diagram is as below(This patchset is based on it): iommu0 iommu1 | | ---------- | mtk_iommu_pgtable | ------------------------------------------ | | | mtk_iommu_domain1 mtk_iommu_domain2 mtk_iommu_domain3 | | | iommu_group1 iommu_group2 iommu_group3 | | | iommu_domain1 iommu_domain2 iommu_domain3 | | | iova region1(normal) iova region2(CCU) iova region3(VPU) >From above new SW architecture, we will create a global page table firstly(mtk_iommu_pgtable), and all the iommus can use it. The page table can include 4GB iova space, different iova regions are created based on it, so the information of pgtable needs to be created in device_group. I have two problems to consult you, can you provide some advices? Thanks for Matthias and Joerg so much! (1) I don't understand if the pgtable can be created in device_group ? (2) In addition, can you help to review which SW architecture are more reasonable or whether it is necessary to change current SW architecture to support multiple domain ? Dear Yong, If I described ambiguously, please help to add extra explanation. Thanks a lot. > > > I can not decide if your flow is right. But if you only want to add > > > support multi-domain, I guess you could extend the current "m4u_group" > > > to a array "m4u_group[N]". It may be more simple. To make mt6779 > > > progress easily, I suggest you can use this way to support multi-domain > > > firstly. Then you could send this new mtk_iommu_pgtable patchset for the > > > code "more explicit" if you insist. > > Could you help try this way if it could meet your requirement? Then > let's compare which one is better. > > > BTW, your patches(including v2) cause hangup as below since > "data->m4u_dom" was uninitialized. > > > Unable to handle kernel NULL pointer dereference at virtual address > 0000000000000010 > ... > pc : mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > lr : __arm_v7s_unmap+0x174/0x598 > ... > Call trace: > mtk_iommu_tlb_flush_page_nosync+0x38/0xb8 > __arm_v7s_unmap+0x174/0x598 > arm_v7s_unmap+0x30/0x48 > mtk_iommu_unmap+0x20/0x28 > __iommu_unmap+0xa4/0xf8 > iommu_unmap+0x44/0x90 > > > > > > > > The diagram is as below: > > > > > > > > mtk_iommu_data1(MM) mtk_iommu_data2(APU) > > > > | | > > > > | | > > > > ------mtk_iommu_pgtable----- > > > > > > > > We need to create global mtk_iommu_pgtable to include all the iova > > > > regions firstly and special iova regions by divided based on it, > > > > so the information of pgtable needs to be created in device_group. > > > > > > > > Signed-off-by: Chao Hao > > > > --- > > > > drivers/iommu/mtk_iommu.c | 84 +++++++++++++++++++++++++++++++++++++++ > > > > drivers/iommu/mtk_iommu.h | 1 + > > > > 2 files changed, 85 insertions(+) > > > > > > > > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c > > > > index f2847e661137..fcbde6b0f58d 100644 > > > > --- a/drivers/iommu/mtk_iommu.c > > > > +++ b/drivers/iommu/mtk_iommu.c > > > > @@ -123,6 +123,12 @@ struct mtk_iommu_domain { > > > > struct iommu_domain domain; > > > > }; > > > > > > > > +struct mtk_iommu_pgtable { > > > > + struct io_pgtable_cfg cfg; > > > > + struct io_pgtable_ops *iop; > > > > +}; > > > > + > > > > +static struct mtk_iommu_pgtable *share_pgtable; > > > > static const struct iommu_ops mtk_iommu_ops; > > > > > > > > /* > > > > @@ -170,6 +176,11 @@ static struct mtk_iommu_data *mtk_iommu_get_m4u_data(void) > > > > return NULL; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *mtk_iommu_get_pgtable(void) > > > > +{ > > > > + return share_pgtable; > > > > +} > > > > + > > > > static struct mtk_iommu_domain *to_mtk_domain(struct iommu_domain *dom) > > > > { > > > > return container_of(dom, struct mtk_iommu_domain, domain); > > > > @@ -322,6 +333,13 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > > > > > + if (data->pgtable) { > > > > + dom->cfg = data->pgtable->cfg; > > > > + dom->iop = data->pgtable->iop; > > > > + dom->domain.pgsize_bitmap = data->pgtable->cfg.pgsize_bitmap; > > > > + return 0; > > > > + } > > > > + > > > > dom->cfg = (struct io_pgtable_cfg) { > > > > .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > IO_PGTABLE_QUIRK_NO_PERMS | > > > > @@ -345,6 +363,61 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) > > > > return 0; > > > > } > > > > > > > > +static struct mtk_iommu_pgtable *create_pgtable(struct mtk_iommu_data *data) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + > > > > + pgtable = kzalloc(sizeof(*pgtable), GFP_KERNEL); > > > > + if (!pgtable) > > > > + return ERR_PTR(-ENOMEM); > > > > + > > > > + pgtable->cfg = (struct io_pgtable_cfg) { > > > > + .quirks = IO_PGTABLE_QUIRK_ARM_NS | > > > > + IO_PGTABLE_QUIRK_NO_PERMS | > > > > + IO_PGTABLE_QUIRK_TLBI_ON_MAP | > > > > + IO_PGTABLE_QUIRK_ARM_MTK_EXT, > > > > + .pgsize_bitmap = mtk_iommu_ops.pgsize_bitmap, > > > > + .ias = 32, > > > > + .oas = 34, > > > > + .tlb = &mtk_iommu_flush_ops, > > > > + .iommu_dev = data->dev, > > > > + }; > > > > + > > > > + pgtable->iop = alloc_io_pgtable_ops(ARM_V7S, &pgtable->cfg, data); > > > > + if (!pgtable->iop) { > > > > + dev_err(data->dev, "Failed to alloc io pgtable\n"); > > > > + return ERR_PTR(-EINVAL); > > > > + } > > > > + > > > > + dev_info(data->dev, "%s create pgtable done\n", __func__); > > > > + > > > > + return pgtable; > > > > +} > > > > + > > > > +static int mtk_iommu_attach_pgtable(struct mtk_iommu_data *data, > > > > + struct device *dev) > > > > +{ > > > > + struct mtk_iommu_pgtable *pgtable = mtk_iommu_get_pgtable(); > > > > + > > > > + /* create share pgtable */ > > > > + if (!pgtable) { > > > > + pgtable = create_pgtable(data); > > > > + if (IS_ERR(pgtable)) { > > > > + dev_err(data->dev, "Failed to create pgtable\n"); > > > > + return -ENOMEM; > > > > + } > > > > + > > > > + share_pgtable = pgtable; > > > > + } > > > > + > > > > + /* binding to pgtable */ > > > > + data->pgtable = pgtable; > > > > + > > > > + dev_info(data->dev, "m4u%d attach_pgtable done!\n", data->m4u_id); > > > > + > > > > + return 0; > > > > +} > > > > + > > > > static struct iommu_domain *mtk_iommu_domain_alloc(unsigned type) > > > > { > > > > struct mtk_iommu_domain *dom; > > > > @@ -508,10 +581,21 @@ static void mtk_iommu_remove_device(struct device *dev) > > > > static struct iommu_group *mtk_iommu_device_group(struct device *dev) > > > > { > > > > struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); > > > > + struct mtk_iommu_pgtable *pgtable; > > > > + int ret = 0; > > > > > > > > if (!data) > > > > return ERR_PTR(-ENODEV); > > > > > > > > + pgtable = data->pgtable; > > > > + if (!pgtable) { > > > > + ret = mtk_iommu_attach_pgtable(data, dev); > > > > + if (ret) { > > > > + dev_err(data->dev, "Failed to device_group\n"); > > > > + return NULL; > > > > + } > > > > + } > > > > + > > > > /* All the client devices are in the same m4u iommu-group */ > > > > if (!data->m4u_group) { > > > > data->m4u_group = iommu_group_alloc(); > > > > diff --git a/drivers/iommu/mtk_iommu.h b/drivers/iommu/mtk_iommu.h > > > > index 132dc765a40b..dd5f19f78b62 100644 > > > > --- a/drivers/iommu/mtk_iommu.h > > > > +++ b/drivers/iommu/mtk_iommu.h > > > > @@ -61,6 +61,7 @@ struct mtk_iommu_data { > > > > struct clk *bclk; > > > > phys_addr_t protect_base; /* protect memory base */ > > > > struct mtk_iommu_suspend_reg reg; > > > > + struct mtk_iommu_pgtable *pgtable; > > > > struct mtk_iommu_domain *m4u_dom; > > > > struct iommu_group *m4u_group; > > > > bool enable_4GB; > > > > > > > > > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D64DCC4BA0A for ; Wed, 26 Feb 2020 06:37:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 92DC5206E2 for ; Wed, 26 Feb 2020 06:37:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=mediatek.com header.i=@mediatek.com header.b="W0MYxIP0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726089AbgBZGhg (ORCPT ); Wed, 26 Feb 2020 01:37:36 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:37823 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1725890AbgBZGhf (ORCPT ); Wed, 26 Feb 2020 01:37:35 -0500 X-UUID: 1b097331b3a745f1a9e4250c5c2a374c-20200226 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=sNVknrlIb9UubW2Xtv3IENlncX0Wt6DkRbVInod5eos=; b=W0MYxIP0ipAW6V+u8HfRqk82USGGiHzavuRoGjY+/dmwCGiq0SNtjhHCavFsH1ueTASym9q4iEC6suGMiurWzqMRAk9Ujmzz/MhlJYFEZ5dtWTAWSgQpa70xZLBBI3su1An5TIwte/Sye8xN8d+TXIdOS1jRz+ewLzph7ZMQNVc=; X-UUID: 1b097331b3a745f1a9e4250c5c2a374c-20200226 Received: from mtkexhb01.mediatek.inc [(172.21.101.102)] by mailgw02.mediatek.com (envelope-from ) (Cellopoint E-mail Firewall v4.1.10 Build 0809 with TLS) with ESMTP id 1468911021; Wed, 26 Feb 2020 14:37:27 +0800 Received: from mtkcas08.mediatek.inc (172.21.101.126) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 26 Feb 2020 14:36:33 +0800 Received: from [10.15.20.246] (10.15.20.246) by mtkcas08.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1395.4 via Frontend Transport; Wed, 26 Feb 2020 14:37:33 +0800 Message-ID: <1582698979.27773.17.camel@mbjsdccf07> Subject: Re: [RESEND,PATCH 03/13] iommu/mediatek: Add mtk_iommu_pgtable structure From: chao hao To: Matthias Brugger , Joerg Roedel , Yong Wu CC: Joerg Roedel , Rob Herring , Matthias Brugger , "iommu@lists.linux-foundation.org" , "devicetree@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mediatek@lists.infradead.org" , wsd_upstream , Jun Yan =?UTF-8?Q?=28=E9=A2=9C=E5=86=9B=29?= , Cui Zhang =?UTF-8?Q?=28=E5=BC=A0=E7=BF=A0=29?= , "Guangming Cao =?UTF-8?Q?=28=E6=9B=B9=E5=85=89=E6=98=8E=29?=" , Anan Sun =?UTF-8?Q?=28=E5=AD=99=E5=AE=89=E5=AE=89=29?= , Miles Chen =?UTF-8?Q?=28=E9=99=B3=E6=B0=91=E6=A8=BA=29?= , , , Date: Wed, 26 Feb 2020 14:36:19 +0800 In-Reply-To: <1581769046.32039.27.camel@mhfsdcap03> References: <20191104115238.2394-1-chao.hao@mediatek.com> <20191104115238.2394-4-chao.hao@mediatek.com> <1576498396.28043.78.camel@mhfsdcap03> <1577785148.30177.5.camel@mbjsdccf07> <1581769046.32039.27.camel@mhfsdcap03> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-MTK: N Content-Transfer-Encoding: base64 Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org T24gU2F0LCAyMDIwLTAyLTE1IGF0IDIwOjE3ICswODAwLCBZb25nIFd1IHdyb3RlOg0KPiBPbiBU dWUsIDIwMTktMTItMzEgYXQgMTc6MzkgKzA4MDAsIGNoYW8gaGFvIHdyb3RlOg0KPiA+IE9uIE1v biwgMjAxOS0xMi0xNiBhdCAyMDoxMyArMDgwMCwgWW9uZyBXdSB3cm90ZToNCj4gPiA+IE9uIE1v biwgMjAxOS0xMS0wNCBhdCAxOTo1MiArMDgwMCwgQ2hhbyBIYW8gd3JvdGU6DQo+ID4gPiA+IFN0 YXJ0IHdpdGggdGhpcyBwYXRjaCwgd2Ugd2lsbCBjaGFuZ2UgdGhlIFNXIGFyY2hpdGVjdHVyZQ0K PiA+ID4gPiB0byBzdXBwb3J0IG11bHRpcGxlIGRvbWFpbnMuIFNXIGFyY2hpdGVjdHVyZSB3aWxs IGhhcyBhIGJpZyBjaGFuZ2UsDQo+ID4gPiA+IHNvIHdlIG5lZWQgdG8gbW9kaWZ5IGEgbGl0dGxl IGJpdCBieSBtb3JlIHRoYW4gb25lIHBhdGNoLg0KPiA+ID4gPiBUaGUgbmV3IFNXIG92ZXJhbGwg YXJjaGl0ZWN0dXJlIGlzIGFzIGJlbG93Og0KPiA+ID4gPiANCj4gPiA+ID4gCQkJCWlvbW11MCAg IGlvbW11MQ0KPiA+ID4gPiAJCQkJICB8CSAgICB8DQo+ID4gPiA+IAkJCQkgIC0tLS0tLS0tLS0t DQo+ID4gPiA+IAkJCQkJfA0KPiA+ID4gPiAJCQkJbXRrX2lvbW11X3BndGFibGUNCj4gPiA+ID4g CQkJCQl8DQo+ID4gPiA+IAkJCS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQ0KPiA+ID4gPiAJCQl8CQkgICAgIHwJCQkgfA0KPiA+ID4gPiAJCW10a19pb21tdV9kb21h aW4xICAgbXRrX2lvbW11X2RvbWFpbjIgIG10a19pb21tdV9kb21haW4zDQo+ID4gPiA+IAkJCXwg ICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgfA0KPiA+ID4gPiAJCWlvbW11 X2dyb3VwMSAgICAgICAgIGlvbW11X2dyb3VwMiAgICAgICAgICAgaW9tbXVfZ3JvdXAzDQo+ID4g PiA+IAkJCXwgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgfA0KPiA+ID4g PiAJCWlvbW11X2RvbWFpbjEgICAgICAgaW9tbXVfZG9tYWluMgkgICAgaW9tbXVfZG9tYWluMw0K PiA+ID4gPiAJCQl8ICAgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgIHwNCj4g PiA+ID4gCQlpb3ZhIHJlZ2lvbjEobm9ybWFsKSAgaW92YSByZWdpb24yKENDVSkgICAgaW92YSBy ZWdpb24zKFZQVSkNCj4gPiA+ID4gDQo+ID4gPiA+IEZvciBjdXJyZW50IHN0cnVjdHVyZSwgbm8g bWF0dGVyIGhvdyBtYW55IGlvbW11cyB0aGVyZSBhcmUsDQo+ID4gPiA+IHRoZXkgdXNlIHRoZSBz YW1lIHBhZ2UgdGFibGUgdG8gc2ltcGxpZnkgdGhlIHVzYWdlIG9mIG1vZHVsZS4NCj4gPiA+ID4g SW4gb3JkZXIgdG8gbWFrZSB0aGUgc29mdHdhcmUgYXJjaGl0ZWN0dXJlIG1vcmUgZXhwbGljaXQs IHRoaXMNCj4gPiA+ID4gcGF0Y2ggd2lsbCBjcmVhdGUgYSBnbG9iYWwgbXRrX2lvbW11X3BndGFi bGUgc3RydWN0dXJlIHRvIGRlc2NyaWJlDQo+ID4gPiA+IHBhZ2UgdGFibGUgYW5kIGFsbCB0aGUg aW9tbXVzIHVzZSBpdC4NCj4gPiA+IA0KPiA+ID4gVGhhbmtzIGZvciB0aGUgaGFyZCB3b3JrIG9m IHRoaXMgZmlsZS4gQWN0dWFsbHkgdGhpcyBwYXRjaCBhbmQgdGhlIGxhdGVyDQo+ID4gPiBvbmVz IGNvbmZ1c2UgbWUuIFdoeSBkbyB5b3UgbWFrZSB0aGlzIGZsb3cgY2hhbmdlPyANCj4gPiA+IGZv ciBtYWtpbmcgdGhlIGNvZGUgIm1vcmUgZXhwbGljaXQiIG9yIGZvciBhZGRpbmcgbXVsdGktZG9t YWluIHN1cHBvcnQNCj4gPiA+IGluIDEzLzEzLg0KPiA+ID4gDQo+ID4gPiBJTUhPLCB0aGUgY2hh bmdlIGlzIHVubmVjZXNzYXJ5Lg0KPiA+ID4gYSkgRm9yIG1lLCB0aGlzIGNoYW5nZSBoYXMgbm8g aW1wcm92ZW1lbnQuIGN1cnJlbnRseSB3ZSB1c2UgYSBnbG9iYWwNCj4gPiA+IG10a19pb21tdV9n ZXRfbTR1X2RhdGEgdG8gZ2V0IHRoZSBNNFUgZGF0YS4gSSB3aWxsIGJlIHZlcnkgZ2xhZCBpZiB5 b3UNCj4gPiA+IGNvdWxkIGdldCByaWQgb2YgaXQuIEJ1dCBpbiB0aGlzIHBhdGNoc2V0LCBZb3Ug dXNlIGEgYW5vdGhlciBnbG9iYWwNCj4gPiA+IG10a19pb21tdV9wZ3RhYmxlIHRvIGluc3RlYWQu IEZvciBtZS4gSXQgaGFzIG5vIGltcHJvdmVtZW50Lg0KPiA+IA0KPiA+IFRoYW5rcyBmb3IgeW91 IGFkdmljZSENCj4gPiANCj4gPiBGb3IgY3VycmVudCBTVyBhcmNoLCBhbGwgdGhlIElPTU1VIEhX IHVzZSB0aGUgc2FtZSBwYWdlIHRhYmxlLCB3ZSBjYW4NCj4gPiB1c2UgYSBnbG9iYWwgbXRrX2lv bW11X3BndGFibGUgdG8gZGlzY3JpYmUgdGhlIGluZm9ybWF0aW9uIG9mIHBhZ2UgdGFibGUNCj4g DQo+IFdoYXQncyB5b3VyIHBsYW4gaWYgdGhlIDRHQiBpb3ZhIHJhbmdlIGlzIG5vdCBlbm91Z2gg Zm9yIHVzIGluIGZ1dHVyZT8NCj4gRG8geW91IHBsYW4gdG8gYWRkIGEgbmV3IGdsb2JhbCBtdGtf aW9tbXVfcGd0YWJsZSBhZ2Fpbj8NCj4gDQo+ID4gYW5kIGFsbCB0aGUgSU9NTVUgYXR0YWNoIGl0 LCBJIHRoaW5rIHRoYXQgaXQgaXMgbW9yZSBjbGVhciBhbmQNCj4gPiB1bmFtYmlndW91cy4gRm9y IGJlZ2lubmVycywgaXQgbWF5YmUgbW9yZSBlYXNpbHkgZXhwbGljYWJsZT8gDQo+IA0KPiBJIHN0 aWxsIGRvbid0IGdldCB0aGUgbmVjZXNzaXR5IG9mIHRoaXMgY2hhbmdlLiBpdCBpcyBvbmx5IGZv ciBtYWtpbmcNCj4gY29kZSBjbGVhciBmcm9tIHlvdXIgcG9pbnQgZm9yIHZpZXcsIHJpZ2h0Pw0K PiANCj4gVGhpcyBjb2RlIGhhcyBiZWVuIHJldmlld2VkIGZvciBtYW55IHllYXJzLCBJIGRvbid0 IGtub3cgd2h5IHlvdSB0aGluaw0KPiBpdCBpcyBhbWJpZ3VvdXMuIGl0IGlzIGNsZWFyIGZvciBt ZSBhdCBsZWFzZS4gYW5kIEkgd2lsbCBjb21wbGFpbiB0aGF0DQo+IHlvdSBhZGQgYSBuZXcgZ2xv YmFsIHZhcmlhYmxlIGluIHRoaXMgY2hhbmdlLg0KPiANCj4gPiA+IA0KPiA+ID4gYikgVGhpcyBw YXRjaHNldCBicmVhayB0aGUgb3JpZ2luYWwgZmxvdy4gZGV2aWNlX2dyb3VwIGdpdmUgeW91IGEN Cj4gPiA+IHNvZnR3YXJlIGNoYW5jZSBmb3IgaW5pdGlhbGl6aW5nLCB0aGVuIHlvdSBtb3ZlIHBh Z2V0YWJsZSBhbGxvY2F0aW5nDQo+ID4gPiBjb2RlIGludG8gaXQuIEJ1dCBpdCBpc24ndCBkZXZp Y2VfZ3JvdXAgam9iLg0KPiA+ID4gDQo+ID4gDQo+ID4gQXMgaXMgc2hvd24gYWJvdmUgZGlhZ3Jh bSwgbXRrX2lvbW11X3BndGFibGUgaW5jbHVkZXMgaW9tbXVfZ3JvdXAgYW5kDQo+ID4gaW9tbXVf ZG9tYWluLHNvIHdlIG5lZWQgdG8gYWxsb2NhdGUgbXRrX2lvbW11X3BndGFibGUgYW5kIGluaXRp YWxpemUgaXQNCj4gPiBpbiBkZXZpY2VfZ3JvdXAgZmlyc3RseSxhbmQgdGhlbiBleGVjdXRlIHRo ZSBvcmlnaW5hbCBmbG93LCBpdCBvbmx5DQo+ID4gY2hhbmdlcyBwbGFjZSBmb3IgY3JlYXRpbmcg bXRrX2lvbW11X3BndGFibGUgYW5kIGRvbid0IGJyZWFrIG9yaWdpbmFsDQo+ID4gZGV2aWNlX2dy b3VwIGZsb3cuDQo+IA0KPiBJIHVuZGVyc3RhbmQgeW91IGhhdmUgdG8gZG8gdGhpcyBjaGFuZ2Ug YWZ0ZXIgeW91IGFkanVzdCB0aGUgc3RydWN0dXJlLg0KPiBJIG1lYW4gdGhhdCBpdCBtYXkgYmUg bm90IHByb3BlciBzaW5jZSBhbGxvY2F0aW5nIHBhZ2V0YWJsZSBzaG91bGQgbm90DQo+IGJlIGRv bmUgaW4gZGV2aWNlX2dyb3VwIGxvZ2ljYWxseS4gRnJvbSBoZXJlLCBDb3VsZCB3ZSBnZXQgdGhp cyBjaGFuZ2UNCj4gbG9va3Mgbm90IGdvb2Q/Lg0KPiANCmdlbnRsZSBwaW5nIC4uLg0KDQpEZWFy IE1hdHRoaWFzIGFuZCBKb2VyZywNCkZyb20gbXQ2Nzc5IHBsYXRmb3JtLCBtdGtfaW9tbXUuYyBu ZWVkcyB0byBzdXBwb3J0IG11bHRpcGxlIGRvbWFpbnMgZm9yDQpkaWZmZXJlbnQgaW92YSByZWdp b25zLkFib3V0IHRoZSBjaGFuZ2UsIHRoZXJlIGFyZSBzb21lIGRpc2FncmVlbWVudHMNCmFtb25n IG91ciBpbnRlcm5hbC4gV2UgaG9wZSB0byBnZXQgeW91ciBoZWxwcyBhbmQgYWR2aWNlczoNCg0K QmFzZWQgb24gY3VycmVudCBTVyBhcmNoaXRlY3R1cmUgdG8gc3VwcG9ydCBtdWx0aXBsZSBkb21h aW4sIGRpYWdyYW0gaXMNCmFzIGJlbG93Og0KICAgICAgICAgICAgICAgICAgICAgICAgICAgaW9t bXUwICAgaW9tbXUxDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAgICAgICB8DQog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtLS0tLS0tLS0tDQogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHwNCiAgICAgICAgICAgICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg IHwgICAgICAgICAgICAgICAgICAgfA0KICAgICAgICAgaW9tbXVfZ3JvdXAxICAgICAgICAgaW9t bXVfZ3JvdXAyICAgICAgICBpb21tdV9ncm91cDMNCiAgICAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgICB8DQogICAgICAgbXRrX2lvbW11X2RvbWFpbjEg ICAgIG10a19pb21tdV9kb21haW4yICAgbXRrX2lvbW11X2RvbWFpbjMNCiAgICAgICAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICB8DQogICAgICAgaW92YSBy ZWdpb24xKG5vcm1hbCkgIGlvdmEgcmVnaW9uMihDQ1UpICAgaW92YSByZWdpb24zKFZQVSkNCiAN CiAgUFM6IHRoZSBpbmZvcm1hdGlvbiBvZiBwYWdlIHRhYmxlIGlzIGluY2x1ZGVkIHN0cnVjdCBt dGtfaW9tbXVfZG9tYWluDQoNCkluIG15IG9waW5pb24sIGlmIGFsbCB0aGUgaW9tbXVzIHNoYXJl IHRoZSBzYW1lIHBhZ2UgdGFibGUoaW5jbHVkZSBhbGwNCmlvdmEgcmVnaW9ucykgYW5kIGRpZmZl cmVudCBpb3ZhIHJlZ2lvbnMgYXJlIGNyZWF0ZWQgYmFzZWQgb24gdGhlIHBhZ2UNCnRhYmxlLCB3 ZSBjYW4gcHV0IHRoZSBpbmZvcm1hdGlvbiBvZiBwYWdlIHRhYmxlIHRvIGEgZ2xvYmFsDQpzdHJ1 Y3R1cmUobXRrX2lvbW11X3BndGFibGUpIGFuZCBhbGwgdGhlIGlvbW11cyBhdHRhY2ggaXQuIEl0 IG1heWJlIG1ha2UNCnRoZSBTVyBhcmNoaXRlY3R1cmUgbG9vayBjbGVhcmx5Lg0KTmV3IFNXIGFy Y2hpdGVjdHVyZSBkaWFncmFtIGlzIGFzIGJlbG93KFRoaXMgcGF0Y2hzZXQgaXMgYmFzZWQgb24g aXQpOg0KDQogICAgICAgICAgICAgICAgICAgICAgICAgICBpb21tdTAgICBpb21tdTENCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwNCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIC0tLS0tLS0tLS0NCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg fA0KICAgICAgICAgICAgICAgICAgICAgICAgICBtdGtfaW9tbXVfcGd0YWJsZQ0KICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8DQogICAgICAgICAgICAgIC0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAgICAgICAgICAgICB8ICAgICAgICAgICAg ICAgICAgICB8ICAgICAgICAgICAgICAgICAgIHwNCiAgICAgICAgbXRrX2lvbW11X2RvbWFpbjEg ICBtdGtfaW9tbXVfZG9tYWluMiAgIG10a19pb21tdV9kb21haW4zDQogICAgICAgICAgICAgIHwg ICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgfA0KICAgICAgICAgaW9tbXVf Z3JvdXAxICAgICAgICAgaW9tbXVfZ3JvdXAyICAgICAgICBpb21tdV9ncm91cDMNCiAgICAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICB8DQogICAgICAg ICBpb21tdV9kb21haW4xICAgICAgIGlvbW11X2RvbWFpbjIgICAgICAgIGlvbW11X2RvbWFpbjMN CiAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICB8 DQogICAgICAgaW92YSByZWdpb24xKG5vcm1hbCkgIGlvdmEgcmVnaW9uMihDQ1UpICAgaW92YSBy ZWdpb24zKFZQVSkNCg0KRnJvbSBhYm92ZSBuZXcgU1cgYXJjaGl0ZWN0dXJlLCB3ZSB3aWxsIGNy ZWF0ZSBhIGdsb2JhbCBwYWdlIHRhYmxlDQpmaXJzdGx5KG10a19pb21tdV9wZ3RhYmxlKSwgYW5k IGFsbCB0aGUgaW9tbXVzIGNhbiB1c2UgaXQuIFRoZSBwYWdlDQp0YWJsZSBjYW4gaW5jbHVkZSA0 R0IgaW92YSBzcGFjZSwgZGlmZmVyZW50IGlvdmEgcmVnaW9ucyBhcmUgY3JlYXRlZA0KYmFzZWQg b24gaXQsIHNvIHRoZSBpbmZvcm1hdGlvbiBvZiBwZ3RhYmxlIG5lZWRzIHRvIGJlIGNyZWF0ZWQg aW4NCmRldmljZV9ncm91cC4NCg0KSSBoYXZlIHR3byBwcm9ibGVtcyB0byBjb25zdWx0IHlvdSwg Y2FuIHlvdSBwcm92aWRlIHNvbWUgYWR2aWNlcz8gVGhhbmtzDQpmb3IgTWF0dGhpYXMgYW5kIEpv ZXJnIHNvIG11Y2ghDQooMSkgSSBkb24ndCB1bmRlcnN0YW5kIGlmIHRoZSBwZ3RhYmxlIGNhbiBi ZSBjcmVhdGVkIGluIGRldmljZV9ncm91cCA/DQooMikgSW4gYWRkaXRpb24sIGNhbiB5b3UgaGVs cCB0byByZXZpZXcgd2hpY2ggU1cgYXJjaGl0ZWN0dXJlIGFyZSBtb3JlDQpyZWFzb25hYmxlIG9y IHdoZXRoZXIgaXQgaXMgbmVjZXNzYXJ5IHRvIGNoYW5nZSBjdXJyZW50IFNXIGFyY2hpdGVjdHVy ZQ0KdG8gc3VwcG9ydCBtdWx0aXBsZSBkb21haW4gPw0KDQoNCkRlYXIgWW9uZywNCklmIEkgZGVz Y3JpYmVkIGFtYmlndW91c2x5LCBwbGVhc2UgaGVscCB0byBhZGQgZXh0cmEgZXhwbGFuYXRpb24u DQpUaGFua3MgYSBsb3QuDQoNCg0KPiA+ID4gSSBjYW4gbm90IGRlY2lkZSBpZiB5b3VyIGZsb3cg aXMgcmlnaHQuIEJ1dCBpZiB5b3Ugb25seSB3YW50IHRvIGFkZA0KPiA+ID4gc3VwcG9ydCBtdWx0 aS1kb21haW4sIEkgZ3Vlc3MgeW91IGNvdWxkIGV4dGVuZCB0aGUgY3VycmVudCAibTR1X2dyb3Vw Ig0KPiA+ID4gdG8gYSBhcnJheSAibTR1X2dyb3VwW05dIi4gSXQgbWF5IGJlIG1vcmUgc2ltcGxl LiBUbyBtYWtlIG10Njc3OQ0KPiA+ID4gcHJvZ3Jlc3MgZWFzaWx5LCBJIHN1Z2dlc3QgeW91IGNh biB1c2UgdGhpcyB3YXkgdG8gc3VwcG9ydCBtdWx0aS1kb21haW4NCj4gPiA+IGZpcnN0bHkuIFRo ZW4geW91IGNvdWxkIHNlbmQgdGhpcyBuZXcgbXRrX2lvbW11X3BndGFibGUgcGF0Y2hzZXQgZm9y IHRoZQ0KPiA+ID4gY29kZSAibW9yZSBleHBsaWNpdCIgaWYgeW91IGluc2lzdC4NCj4gDQo+IENv dWxkIHlvdSBoZWxwIHRyeSB0aGlzIHdheSBpZiBpdCBjb3VsZCBtZWV0IHlvdXIgcmVxdWlyZW1l bnQ/IFRoZW4NCj4gbGV0J3MgY29tcGFyZSB3aGljaCBvbmUgaXMgYmV0dGVyLg0KPiANCj4gDQo+ IEJUVywgeW91ciBwYXRjaGVzKGluY2x1ZGluZyB2MikgY2F1c2UgaGFuZ3VwIGFzIGJlbG93IHNp bmNlDQo+ICJkYXRhLT5tNHVfZG9tIiB3YXMgdW5pbml0aWFsaXplZC4NCj4gDQo+IA0KPiBVbmFi bGUgdG8gaGFuZGxlIGtlcm5lbCBOVUxMIHBvaW50ZXIgZGVyZWZlcmVuY2UgYXQgdmlydHVhbCBh ZGRyZXNzDQo+IDAwMDAwMDAwMDAwMDAwMTANCj4gLi4uDQo+IHBjIDogbXRrX2lvbW11X3RsYl9m bHVzaF9wYWdlX25vc3luYysweDM4LzB4YjgNCj4gbHIgOiBfX2FybV92N3NfdW5tYXArMHgxNzQv MHg1OTgNCj4gLi4uDQo+IENhbGwgdHJhY2U6DQo+ICBtdGtfaW9tbXVfdGxiX2ZsdXNoX3BhZ2Vf bm9zeW5jKzB4MzgvMHhiOA0KPiAgX19hcm1fdjdzX3VubWFwKzB4MTc0LzB4NTk4DQo+ICBhcm1f djdzX3VubWFwKzB4MzAvMHg0OA0KPiAgbXRrX2lvbW11X3VubWFwKzB4MjAvMHgyOA0KPiAgX19p b21tdV91bm1hcCsweGE0LzB4ZjgNCj4gIGlvbW11X3VubWFwKzB4NDQvMHg5MA0KPiANCj4gPiA+ IA0KPiA+ID4gPiBUaGUgZGlhZ3JhbSBpcyBhcyBiZWxvdzoNCj4gPiA+ID4gDQo+ID4gPiA+IAlt dGtfaW9tbXVfZGF0YTEoTU0pICAgICAgIG10a19pb21tdV9kYXRhMihBUFUpDQo+ID4gPiA+IAkJ fAkJCSAgIHwNCj4gPiA+ID4gCQl8CQkJICAgfA0KPiA+ID4gPiAJCS0tLS0tLW10a19pb21tdV9w Z3RhYmxlLS0tLS0NCj4gPiA+ID4gDQo+ID4gPiA+IFdlIG5lZWQgdG8gY3JlYXRlIGdsb2JhbCBt dGtfaW9tbXVfcGd0YWJsZSB0byBpbmNsdWRlIGFsbCB0aGUgaW92YQ0KPiA+ID4gPiByZWdpb25z IGZpcnN0bHkgYW5kIHNwZWNpYWwgaW92YSByZWdpb25zIGJ5IGRpdmlkZWQgYmFzZWQgb24gaXQs DQo+ID4gPiA+IHNvIHRoZSBpbmZvcm1hdGlvbiBvZiBwZ3RhYmxlIG5lZWRzIHRvIGJlIGNyZWF0 ZWQgaW4gZGV2aWNlX2dyb3VwLg0KPiA+ID4gPiANCj4gPiA+ID4gU2lnbmVkLW9mZi1ieTogQ2hh byBIYW8gPGNoYW8uaGFvQG1lZGlhdGVrLmNvbT4NCj4gPiA+ID4gLS0tDQo+ID4gPiA+ICBkcml2 ZXJzL2lvbW11L210a19pb21tdS5jIHwgODQgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKysrDQo+ID4gPiA+ICBkcml2ZXJzL2lvbW11L210a19pb21tdS5oIHwgIDEgKw0KPiA+ ID4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA4NSBpbnNlcnRpb25zKCspDQo+ID4gPiA+IA0KPiA+ID4g PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9pb21tdS9tdGtfaW9tbXUuYyBiL2RyaXZlcnMvaW9tbXUv bXRrX2lvbW11LmMNCj4gPiA+ID4gaW5kZXggZjI4NDdlNjYxMTM3Li5mY2JkZTZiMGY1OGQgMTAw NjQ0DQo+ID4gPiA+IC0tLSBhL2RyaXZlcnMvaW9tbXUvbXRrX2lvbW11LmMNCj4gPiA+ID4gKysr IGIvZHJpdmVycy9pb21tdS9tdGtfaW9tbXUuYw0KPiA+ID4gPiBAQCAtMTIzLDYgKzEyMywxMiBA QCBzdHJ1Y3QgbXRrX2lvbW11X2RvbWFpbiB7DQo+ID4gPiA+ICAJc3RydWN0IGlvbW11X2RvbWFp bgkJZG9tYWluOw0KPiA+ID4gPiAgfTsNCj4gPiA+ID4gIA0KPiA+ID4gPiArc3RydWN0IG10a19p b21tdV9wZ3RhYmxlIHsNCj4gPiA+ID4gKwlzdHJ1Y3QgaW9fcGd0YWJsZV9jZmcJY2ZnOw0KPiA+ ID4gPiArCXN0cnVjdCBpb19wZ3RhYmxlX29wcwkqaW9wOw0KPiA+ID4gPiArfTsNCj4gPiA+ID4g Kw0KPiA+ID4gPiArc3RhdGljIHN0cnVjdCBtdGtfaW9tbXVfcGd0YWJsZSAqc2hhcmVfcGd0YWJs ZTsNCj4gPiA+ID4gIHN0YXRpYyBjb25zdCBzdHJ1Y3QgaW9tbXVfb3BzIG10a19pb21tdV9vcHM7 DQo+ID4gPiA+ICANCj4gPiA+ID4gIC8qDQo+ID4gPiA+IEBAIC0xNzAsNiArMTc2LDExIEBAIHN0 YXRpYyBzdHJ1Y3QgbXRrX2lvbW11X2RhdGEgKm10a19pb21tdV9nZXRfbTR1X2RhdGEodm9pZCkN Cj4gPiA+ID4gIAlyZXR1cm4gTlVMTDsNCj4gPiA+ID4gIH0NCj4gPiA+ID4gIA0KPiA+ID4gPiAr c3RhdGljIHN0cnVjdCBtdGtfaW9tbXVfcGd0YWJsZSAqbXRrX2lvbW11X2dldF9wZ3RhYmxlKHZv aWQpDQo+ID4gPiA+ICt7DQo+ID4gPiA+ICsJcmV0dXJuIHNoYXJlX3BndGFibGU7DQo+ID4gPiA+ ICt9DQo+ID4gPiA+ICsNCj4gPiA+ID4gIHN0YXRpYyBzdHJ1Y3QgbXRrX2lvbW11X2RvbWFpbiAq dG9fbXRrX2RvbWFpbihzdHJ1Y3QgaW9tbXVfZG9tYWluICpkb20pDQo+ID4gPiA+ICB7DQo+ID4g PiA+ICAJcmV0dXJuIGNvbnRhaW5lcl9vZihkb20sIHN0cnVjdCBtdGtfaW9tbXVfZG9tYWluLCBk b21haW4pOw0KPiA+ID4gPiBAQCAtMzIyLDYgKzMzMywxMyBAQCBzdGF0aWMgaW50IG10a19pb21t dV9kb21haW5fZmluYWxpc2Uoc3RydWN0IG10a19pb21tdV9kb21haW4gKmRvbSkNCj4gPiA+ID4g IHsNCj4gPiA+ID4gIAlzdHJ1Y3QgbXRrX2lvbW11X2RhdGEgKmRhdGEgPSBtdGtfaW9tbXVfZ2V0 X200dV9kYXRhKCk7DQo+ID4gPiA+ICANCj4gPiA+ID4gKwlpZiAoZGF0YS0+cGd0YWJsZSkgew0K PiA+ID4gPiArCQlkb20tPmNmZyA9IGRhdGEtPnBndGFibGUtPmNmZzsNCj4gPiA+ID4gKwkJZG9t LT5pb3AgPSBkYXRhLT5wZ3RhYmxlLT5pb3A7DQo+ID4gPiA+ICsJCWRvbS0+ZG9tYWluLnBnc2l6 ZV9iaXRtYXAgPSBkYXRhLT5wZ3RhYmxlLT5jZmcucGdzaXplX2JpdG1hcDsNCj4gPiA+ID4gKwkJ cmV0dXJuIDA7DQo+ID4gPiA+ICsJfQ0KPiA+ID4gPiArDQo+ID4gPiA+ICAJZG9tLT5jZmcgPSAo c3RydWN0IGlvX3BndGFibGVfY2ZnKSB7DQo+ID4gPiA+ICAJCS5xdWlya3MgPSBJT19QR1RBQkxF X1FVSVJLX0FSTV9OUyB8DQo+ID4gPiA+ICAJCQlJT19QR1RBQkxFX1FVSVJLX05PX1BFUk1TIHwN Cj4gPiA+ID4gQEAgLTM0NSw2ICszNjMsNjEgQEAgc3RhdGljIGludCBtdGtfaW9tbXVfZG9tYWlu X2ZpbmFsaXNlKHN0cnVjdCBtdGtfaW9tbXVfZG9tYWluICpkb20pDQo+ID4gPiA+ICAJcmV0dXJu IDA7DQo+ID4gPiA+ICB9DQo+ID4gPiA+ICANCj4gPiA+ID4gK3N0YXRpYyBzdHJ1Y3QgbXRrX2lv bW11X3BndGFibGUgKmNyZWF0ZV9wZ3RhYmxlKHN0cnVjdCBtdGtfaW9tbXVfZGF0YSAqZGF0YSkN Cj4gPiA+ID4gK3sNCj4gPiA+ID4gKwlzdHJ1Y3QgbXRrX2lvbW11X3BndGFibGUgKnBndGFibGU7 DQo+ID4gPiA+ICsNCj4gPiA+ID4gKwlwZ3RhYmxlID0ga3phbGxvYyhzaXplb2YoKnBndGFibGUp LCBHRlBfS0VSTkVMKTsNCj4gPiA+ID4gKwlpZiAoIXBndGFibGUpDQo+ID4gPiA+ICsJCXJldHVy biBFUlJfUFRSKC1FTk9NRU0pOw0KPiA+ID4gPiArDQo+ID4gPiA+ICsJcGd0YWJsZS0+Y2ZnID0g KHN0cnVjdCBpb19wZ3RhYmxlX2NmZykgew0KPiA+ID4gPiArCQkucXVpcmtzID0gSU9fUEdUQUJM RV9RVUlSS19BUk1fTlMgfA0KPiA+ID4gPiArCQkJSU9fUEdUQUJMRV9RVUlSS19OT19QRVJNUyB8 DQo+ID4gPiA+ICsJCQlJT19QR1RBQkxFX1FVSVJLX1RMQklfT05fTUFQIHwNCj4gPiA+ID4gKwkJ CUlPX1BHVEFCTEVfUVVJUktfQVJNX01US19FWFQsDQo+ID4gPiA+ICsJCS5wZ3NpemVfYml0bWFw ID0gbXRrX2lvbW11X29wcy5wZ3NpemVfYml0bWFwLA0KPiA+ID4gPiArCQkuaWFzID0gMzIsDQo+ ID4gPiA+ICsJCS5vYXMgPSAzNCwNCj4gPiA+ID4gKwkJLnRsYiA9ICZtdGtfaW9tbXVfZmx1c2hf b3BzLA0KPiA+ID4gPiArCQkuaW9tbXVfZGV2ID0gZGF0YS0+ZGV2LA0KPiA+ID4gPiArCX07DQo+ ID4gPiA+ICsNCj4gPiA+ID4gKwlwZ3RhYmxlLT5pb3AgPSBhbGxvY19pb19wZ3RhYmxlX29wcyhB Uk1fVjdTLCAmcGd0YWJsZS0+Y2ZnLCBkYXRhKTsNCj4gPiA+ID4gKwlpZiAoIXBndGFibGUtPmlv cCkgew0KPiA+ID4gPiArCQlkZXZfZXJyKGRhdGEtPmRldiwgIkZhaWxlZCB0byBhbGxvYyBpbyBw Z3RhYmxlXG4iKTsNCj4gPiA+ID4gKwkJcmV0dXJuIEVSUl9QVFIoLUVJTlZBTCk7DQo+ID4gPiA+ ICsJfQ0KPiA+ID4gPiArDQo+ID4gPiA+ICsJZGV2X2luZm8oZGF0YS0+ZGV2LCAiJXMgY3JlYXRl IHBndGFibGUgZG9uZVxuIiwgX19mdW5jX18pOw0KPiA+ID4gPiArDQo+ID4gPiA+ICsJcmV0dXJu IHBndGFibGU7DQo+ID4gPiA+ICt9DQo+ID4gPiA+ICsNCj4gPiA+ID4gK3N0YXRpYyBpbnQgbXRr X2lvbW11X2F0dGFjaF9wZ3RhYmxlKHN0cnVjdCBtdGtfaW9tbXVfZGF0YSAqZGF0YSwNCj4gPiA+ ID4gKwkJCQkgICAgc3RydWN0IGRldmljZSAqZGV2KQ0KPiA+ID4gPiArew0KPiA+ID4gPiArCXN0 cnVjdCBtdGtfaW9tbXVfcGd0YWJsZSAqcGd0YWJsZSA9IG10a19pb21tdV9nZXRfcGd0YWJsZSgp Ow0KPiA+ID4gPiArDQo+ID4gPiA+ICsJLyogY3JlYXRlIHNoYXJlIHBndGFibGUgKi8NCj4gPiA+ ID4gKwlpZiAoIXBndGFibGUpIHsNCj4gPiA+ID4gKwkJcGd0YWJsZSA9IGNyZWF0ZV9wZ3RhYmxl KGRhdGEpOw0KPiA+ID4gPiArCQlpZiAoSVNfRVJSKHBndGFibGUpKSB7DQo+ID4gPiA+ICsJCQlk ZXZfZXJyKGRhdGEtPmRldiwgIkZhaWxlZCB0byBjcmVhdGUgcGd0YWJsZVxuIik7DQo+ID4gPiA+ ICsJCQlyZXR1cm4gLUVOT01FTTsNCj4gPiA+ID4gKwkJfQ0KPiA+ID4gPiArDQo+ID4gPiA+ICsJ CXNoYXJlX3BndGFibGUgPSBwZ3RhYmxlOw0KPiA+ID4gPiArCX0NCj4gPiA+ID4gKw0KPiA+ID4g PiArCS8qIGJpbmRpbmcgdG8gcGd0YWJsZSAqLw0KPiA+ID4gPiArCWRhdGEtPnBndGFibGUgPSBw Z3RhYmxlOw0KPiA+ID4gPiArDQo+ID4gPiA+ICsJZGV2X2luZm8oZGF0YS0+ZGV2LCAibTR1JWQg YXR0YWNoX3BndGFibGUgZG9uZSFcbiIsIGRhdGEtPm00dV9pZCk7DQo+ID4gPiA+ICsNCj4gPiA+ ID4gKwlyZXR1cm4gMDsNCj4gPiA+ID4gK30NCj4gPiA+ID4gKw0KPiA+ID4gPiAgc3RhdGljIHN0 cnVjdCBpb21tdV9kb21haW4gKm10a19pb21tdV9kb21haW5fYWxsb2ModW5zaWduZWQgdHlwZSkN Cj4gPiA+ID4gIHsNCj4gPiA+ID4gIAlzdHJ1Y3QgbXRrX2lvbW11X2RvbWFpbiAqZG9tOw0KPiA+ ID4gPiBAQCAtNTA4LDEwICs1ODEsMjEgQEAgc3RhdGljIHZvaWQgbXRrX2lvbW11X3JlbW92ZV9k ZXZpY2Uoc3RydWN0IGRldmljZSAqZGV2KQ0KPiA+ID4gPiAgc3RhdGljIHN0cnVjdCBpb21tdV9n cm91cCAqbXRrX2lvbW11X2RldmljZV9ncm91cChzdHJ1Y3QgZGV2aWNlICpkZXYpDQo+ID4gPiA+ ICB7DQo+ID4gPiA+ICAJc3RydWN0IG10a19pb21tdV9kYXRhICpkYXRhID0gbXRrX2lvbW11X2dl dF9tNHVfZGF0YSgpOw0KPiA+ID4gPiArCXN0cnVjdCBtdGtfaW9tbXVfcGd0YWJsZSAqcGd0YWJs ZTsNCj4gPiA+ID4gKwlpbnQgcmV0ID0gMDsNCj4gPiA+ID4gIA0KPiA+ID4gPiAgCWlmICghZGF0 YSkNCj4gPiA+ID4gIAkJcmV0dXJuIEVSUl9QVFIoLUVOT0RFVik7DQo+ID4gPiA+ICANCj4gPiA+ ID4gKwlwZ3RhYmxlID0gZGF0YS0+cGd0YWJsZTsNCj4gPiA+ID4gKwlpZiAoIXBndGFibGUpIHsN Cj4gPiA+ID4gKwkJcmV0ID0gbXRrX2lvbW11X2F0dGFjaF9wZ3RhYmxlKGRhdGEsIGRldik7DQo+ ID4gPiA+ICsJCWlmIChyZXQpIHsNCj4gPiA+ID4gKwkJCWRldl9lcnIoZGF0YS0+ZGV2LCAiRmFp bGVkIHRvIGRldmljZV9ncm91cFxuIik7DQo+ID4gPiA+ICsJCQlyZXR1cm4gTlVMTDsNCj4gPiA+ ID4gKwkJfQ0KPiA+ID4gPiArCX0NCj4gPiA+ID4gKw0KPiA+ID4gPiAgCS8qIEFsbCB0aGUgY2xp ZW50IGRldmljZXMgYXJlIGluIHRoZSBzYW1lIG00dSBpb21tdS1ncm91cCAqLw0KPiA+ID4gPiAg CWlmICghZGF0YS0+bTR1X2dyb3VwKSB7DQo+ID4gPiA+ICAJCWRhdGEtPm00dV9ncm91cCA9IGlv bW11X2dyb3VwX2FsbG9jKCk7DQo+ID4gPiA+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2lvbW11L210 a19pb21tdS5oIGIvZHJpdmVycy9pb21tdS9tdGtfaW9tbXUuaA0KPiA+ID4gPiBpbmRleCAxMzJk Yzc2NWE0MGIuLmRkNWYxOWY3OGI2MiAxMDA2NDQNCj4gPiA+ID4gLS0tIGEvZHJpdmVycy9pb21t dS9tdGtfaW9tbXUuaA0KPiA+ID4gPiArKysgYi9kcml2ZXJzL2lvbW11L210a19pb21tdS5oDQo+ ID4gPiA+IEBAIC02MSw2ICs2MSw3IEBAIHN0cnVjdCBtdGtfaW9tbXVfZGF0YSB7DQo+ID4gPiA+ ICAJc3RydWN0IGNsawkJCSpiY2xrOw0KPiA+ID4gPiAgCXBoeXNfYWRkcl90CQkJcHJvdGVjdF9i YXNlOyAvKiBwcm90ZWN0IG1lbW9yeSBiYXNlICovDQo+ID4gPiA+ICAJc3RydWN0IG10a19pb21t dV9zdXNwZW5kX3JlZwlyZWc7DQo+ID4gPiA+ICsJc3RydWN0IG10a19pb21tdV9wZ3RhYmxlCSpw Z3RhYmxlOw0KPiA+ID4gPiAgCXN0cnVjdCBtdGtfaW9tbXVfZG9tYWluCQkqbTR1X2RvbTsNCj4g PiA+ID4gIAlzdHJ1Y3QgaW9tbXVfZ3JvdXAJCSptNHVfZ3JvdXA7DQo+ID4gPiA+ICAJYm9vbCAg ICAgICAgICAgICAgICAgICAgICAgICAgICBlbmFibGVfNEdCOw0KPiA+ID4gDQo+ID4gPiANCj4g PiANCj4gPiANCj4gDQo+IA0KDQo=