From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D650AC433E1 for ; Fri, 21 Aug 2020 08:31:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A481C207DA for ; Fri, 21 Aug 2020 08:31:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UkyhUnPn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A481C207DA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References:Message-ID:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QfQu31Iv/E6DQkkVo2jWs1XsYH5vUKy9hhmlwFH11q0=; b=UkyhUnPn5vxr9TJzMNYX3FxhJ JoSzflR2ygZM1joCnG33jQI75nABF/R7TilRJDCem2DRUqAHR/hqJ9o6Rv+cckel4XkyMrUxApFX6 +nLN0LPqTYjQN+xvjNhoHMKa+YIXJv4QXDbHfqmlA1pFvQtZrgVL6Wo0bX+tcy4PQ9YE1GTqavR8V z4Ab29XRWNtuoO6i2wBhldqQ99Hk8EhKPPuMCF3Q6PdHCzK0cJJ6f8VcKcNlYUO2h8ksp/JpzxBrc yuzemyod4RJj8MB0WqQNwn5YTIGRcOWTYxxlsSIYEwbVtoYInLUS5uFxCNIx+qoAf+NsvgOfO7QTO S2k+UGdmw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k92Qp-0005qd-Or; Fri, 21 Aug 2020 08:29:51 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187] helo=huawei.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k92Qm-0005oE-Kb for linux-arm-kernel@lists.infradead.org; Fri, 21 Aug 2020 08:29:50 +0000 Received: from DGGEMM406-HUB.china.huawei.com (unknown [172.30.72.53]) by Forcepoint Email with ESMTP id B0E2AF97C94A8C457749; Fri, 21 Aug 2020 16:29:37 +0800 (CST) Received: from dggema771-chm.china.huawei.com (10.1.198.213) by DGGEMM406-HUB.china.huawei.com (10.3.20.214) with Microsoft SMTP Server (TLS) id 14.3.487.0; Fri, 21 Aug 2020 16:29:37 +0800 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by dggema771-chm.china.huawei.com (10.1.198.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Fri, 21 Aug 2020 16:29:36 +0800 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.1913.007; Fri, 21 Aug 2020 16:29:37 +0800 From: "Song Bao Hua (Barry Song)" To: Randy Dunlap , "hch@lst.de" , "m.szyprowski@samsung.com" , "robin.murphy@arm.com" , "will@kernel.org" , "ganapatrao.kulkarni@cavium.com" , "catalin.marinas@arm.com" Subject: RE: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA Thread-Topic: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA Thread-Index: AQHWd2LcxiMeqDASCk2fUz72+ALjbKlBVpwAgADf/CA= Date: Fri, 21 Aug 2020 08:29:37 +0000 Message-ID: <81b215005152449ca4392862d0676ded@hisilicon.com> References: <20200821022615.28596-1-song.bao.hua@hisilicon.com> <20200821022615.28596-2-song.bao.hua@hisilicon.com> <5dbe456d-d408-11ef-788e-63d996435fa8@infradead.org> In-Reply-To: <5dbe456d-d408-11ef-788e-63d996435fa8@infradead.org> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.202.192] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200821_042949_216903_3508D156 X-CRM114-Status: GOOD ( 34.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Steve Capper , "linux-kernel@vger.kernel.org" , Linuxarm , huangdaode , "iommu@lists.linux-foundation.org" , Nicolas Saenz Julienne , Jonathan Cameron , Andrew Morton , Mike Rapoport , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: linux-kernel-owner@vger.kernel.org > [mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Randy Dunlap > Sent: Friday, August 21, 2020 2:50 PM > To: Song Bao Hua (Barry Song) ; hch@lst.de; > m.szyprowski@samsung.com; robin.murphy@arm.com; will@kernel.org; > ganapatrao.kulkarni@cavium.com; catalin.marinas@arm.com > Cc: iommu@lists.linux-foundation.org; Linuxarm ; > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > huangdaode ; Jonathan Cameron > ; Nicolas Saenz Julienne > ; Steve Capper ; Andrew > Morton ; Mike Rapoport > Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve > per-numa CMA > > On 8/20/20 7:26 PM, Barry Song wrote: > > > > > > Cc: Jonathan Cameron > > Cc: Christoph Hellwig > > Cc: Marek Szyprowski > > Cc: Will Deacon > > Cc: Robin Murphy > > Cc: Ganapatrao Kulkarni > > Cc: Catalin Marinas > > Cc: Nicolas Saenz Julienne > > Cc: Steve Capper > > Cc: Andrew Morton > > Cc: Mike Rapoport > > Signed-off-by: Barry Song > > --- > > v6: rebase on top of 5.9-rc1; > > doc cleanup > > > > .../admin-guide/kernel-parameters.txt | 9 ++ > > include/linux/dma-contiguous.h | 6 ++ > > kernel/dma/Kconfig | 10 ++ > > kernel/dma/contiguous.c | 100 > ++++++++++++++++-- > > 4 files changed, 115 insertions(+), 10 deletions(-) > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt > b/Documentation/admin-guide/kernel-parameters.txt > > index bdc1f33fd3d1..3f33b89aeab5 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -599,6 +599,15 @@ > > altogether. For more information, see > > include/linux/dma-contiguous.h > > > > + pernuma_cma=nn[MG] > > memparse() allows any one of these suffixes: K, M, G, T, P, E > and nothing in the option parsing function cares what suffix is used... Hello Randy, Thanks for your comments. Actually I am following the suffix of default cma: cma=nn[MG]@[start[MG][-end[MG]]] [ARM,X86,KNL] Sets the size of kernel global memory area for contiguous memory allocations and optionally the placement constraint by the physical address range of memory allocations. A value of 0 disables CMA altogether. For more information, see include/linux/dma-contiguous.h I suggest users should set the size in either MB or GB as they set cma. > > > + [ARM64,KNL] > > + Sets the size of kernel per-numa memory area for > > + contiguous memory allocations. A value of 0 disables > > + per-numa CMA altogether. DMA users on node nid will > > + first try to allocate buffer from the pernuma area > > + which is located in node nid, if the allocation fails, > > + they will fallback to the global default memory area. > > + > > cmo_free_hint= [PPC] Format: { yes | no } > > Specify whether pages are marked as being inactive > > when they are freed. This is used in CMO environments > > > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > > index cff7e60968b9..89b95f10e56d 100644 > > --- a/kernel/dma/contiguous.c > > +++ b/kernel/dma/contiguous.c > > @@ -69,6 +69,19 @@ static int __init early_cma(char *p) > > } > > early_param("cma", early_cma); > > > > +#ifdef CONFIG_DMA_PERNUMA_CMA > > + > > +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES]; > > +static phys_addr_t pernuma_size_bytes __initdata; > > why phys_addr_t? couldn't it just be unsigned long long? > Mainly because of following the programming habit in kernel/dma/contiguous.c: I think the original code probably meant the size should not be larger than the MAXIMUM value of phys_addr_t: /* * Default global CMA area size can be defined in kernel's .config. * This is useful mainly for distro maintainers to create a kernel * that works correctly for most supported systems. * The size can be set in bytes or as a percentage of the total memory * in the system. * * Users, who want to set the size of global CMA area for their system * should use cma= kernel parameter. */ static const phys_addr_t size_bytes __initconst = (phys_addr_t)CMA_SIZE_MBYTES * SZ_1M; static phys_addr_t size_cmdline __initdata = -1; static phys_addr_t base_cmdline __initdata; static phys_addr_t limit_cmdline __initdata; void __init dma_contiguous_reserve(phys_addr_t limit) { phys_addr_t selected_size = 0; phys_addr_t selected_base = 0; phys_addr_t selected_limit = limit; bool fixed = false; pr_debug("%s(limit %08lx)\n", __func__, (unsigned long)limit); if (size_cmdline != -1) { selected_size = size_cmdline; selected_base = base_cmdline; selected_limit = min_not_zero(limit_cmdline, limit); if (base_cmdline + size_cmdline == limit_cmdline) fixed = true; if the whole file is using phys_addr_t for size, I don't want to make the new code weird. > OK, so cma_declare_contiguous_nid() uses phys_addr_t. Fine. > > > + > > +static int __init early_pernuma_cma(char *p) > > +{ > > + pernuma_size_bytes = memparse(p, &p); > > + return 0; > > +} > > +early_param("pernuma_cma", early_pernuma_cma); > > +#endif > > + > > #ifdef CONFIG_CMA_SIZE_PERCENTAGE > > > > static phys_addr_t __init __maybe_unused > cma_early_percent_memory(void) > > @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t > cma_early_percent_memory(void) > > > > #endif > > > > +#ifdef CONFIG_DMA_PERNUMA_CMA > > +void __init dma_pernuma_cma_reserve(void) > > +{ > > + int nid; > > + > > + if (!pernuma_size_bytes) > > + return; > > + > > + for_each_node_state(nid, N_ONLINE) { > > + int ret; > > + char name[20]; > > + struct cma **cma = &dma_contiguous_pernuma_area[nid]; > > + > > + snprintf(name, sizeof(name), "pernuma%d", nid); > > + ret = cma_declare_contiguous_nid(0, pernuma_size_bytes, 0, 0, > > + 0, false, name, cma, nid); > > + if (ret) { > > + pr_warn("%s: reservation failed: err %d, node %d", __func__, > > + ret, nid); > > + continue; > > + } > > + > > + pr_debug("%s: reserved %llu MiB on node %d\n", __func__, > > + (unsigned long long)pernuma_size_bytes / SZ_1M, nid); > > Conversely, if you want to leave pernuma_size_bytes as phys_addr_t, > you should use %pa (or %pap) to print it. Here I think it is working as "size" in integer. > > > + } > > +} > > +#endif Thanks Barry _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel