From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CAB6C433E3 for ; Fri, 21 Aug 2020 08:50:27 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5EB7E208DB for ; Fri, 21 Aug 2020 08:50:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="cg8s5Y5N"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="s3m8yO83" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EB7E208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GwOxdkGmIq0MOOLhAINcGl0o9yr0DYgqyBC4baLwH+E=; b=cg8s5Y5Nzw1O3mH6qS2UUbYAF 6/A7KIAad2S6+iWBRxdXTW5PMnZpKnAsGnSwC2AWC5wDYnAsGaVHJpd0CQjTxnI1U+DzlbRJO0IMi QleTtjk8rqUVSiycldBT71ItTfxdfKzUj3RGcA3w1xQ8qWAdHkdwK+sgUz1MHJsWWZy95OILjDh12 Pv+Z9203UDtQa598YTDW7DCJh785zLV/CDhxqPnn3iza3/umYSKa9Q8+S21hL+fqtrrLkogY5Qj6i ec+WKl6snhYPJo8xgKGWs9o4wSnzbVuKDBSsq5HwMEBcQBtsFGX8rQLgc1plNgAOR1tWLC2iGcJYC jpRFlz+pw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k92hw-0007V9-PZ; Fri, 21 Aug 2020 08:47:32 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k92ht-0007Ud-MN for linux-arm-kernel@lists.infradead.org; Fri, 21 Aug 2020 08:47:30 +0000 Received: from willie-the-truck (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 61B5420720; Fri, 21 Aug 2020 08:47:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597999648; bh=swmC7UAXvWT1qPRy+nmc9swuccmpZt4xQ+XxVpcsw4o=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=s3m8yO83/JP46GlOofMZy8mYtCgS84V8pMCg79cDORut0LvOof7cy6AH5z6O2Xxaw I/WYJNMHujM65sbfFB3EPI1hcMkpSHIxz8O87NXIRciOLPxm1rKhYaHDjFhofKhh7d RNhKOvrWVZDPB9J2eaJ+DgiQXbx9IrqSP2FjESOU= Date: Fri, 21 Aug 2020 09:47:17 +0100 From: Will Deacon To: Barry Song Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA Message-ID: <20200821084717.GA20255@willie-the-truck> References: <20200821022615.28596-1-song.bao.hua@hisilicon.com> <20200821022615.28596-2-song.bao.hua@hisilicon.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200821022615.28596-2-song.bao.hua@hisilicon.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200821_044729_978493_CD188374 X-CRM114-Status: GOOD ( 24.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mike Rapoport , Steve Capper , catalin.marinas@arm.com, Jonathan Cameron , linuxarm@huawei.com, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Nicolas Saenz Julienne , ganapatrao.kulkarni@cavium.com, Andrew Morton , huangdaode@huawei.com, robin.murphy@arm.com, hch@lst.de, linux-arm-kernel@lists.infradead.org, m.szyprowski@samsung.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Aug 21, 2020 at 02:26:14PM +1200, Barry Song wrote: > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index bdc1f33fd3d1..3f33b89aeab5 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -599,6 +599,15 @@ > altogether. For more information, see > include/linux/dma-contiguous.h > > + pernuma_cma=nn[MG] > + [ARM64,KNL] > + Sets the size of kernel per-numa memory area for > + contiguous memory allocations. A value of 0 disables > + per-numa CMA altogether. DMA users on node nid will > + first try to allocate buffer from the pernuma area > + which is located in node nid, if the allocation fails, > + they will fallback to the global default memory area. What is the default behaviour if this option is not specified? Seems like that should be mentioned here. > diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig > index 847a9d1fa634..db7a37ed35eb 100644 > --- a/kernel/dma/Kconfig > +++ b/kernel/dma/Kconfig > @@ -118,6 +118,16 @@ config DMA_CMA > If unsure, say "n". > > if DMA_CMA > + > +config DMA_PERNUMA_CMA > + bool "Enable separate DMA Contiguous Memory Area for each NUMA Node" I don't understand the need for this config option. If you have DMA_DMA and you have NUMA, why wouldn't you want this enabled? > + help > + Enable this option to get pernuma CMA areas so that devices like > + ARM64 SMMU can get local memory by DMA coherent APIs. > + > + You can set the size of pernuma CMA by specifying "pernuma_cma=size" > + on the kernel's command line. > + > comment "Default contiguous memory area size:" > > config CMA_SIZE_MBYTES > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index cff7e60968b9..89b95f10e56d 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -69,6 +69,19 @@ static int __init early_cma(char *p) > } > early_param("cma", early_cma); > > +#ifdef CONFIG_DMA_PERNUMA_CMA > + > +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES]; > +static phys_addr_t pernuma_size_bytes __initdata; > + > +static int __init early_pernuma_cma(char *p) > +{ > + pernuma_size_bytes = memparse(p, &p); > + return 0; > +} > +early_param("pernuma_cma", early_pernuma_cma); > +#endif > + > #ifdef CONFIG_CMA_SIZE_PERCENTAGE > > static phys_addr_t __init __maybe_unused cma_early_percent_memory(void) > @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t cma_early_percent_memory(void) > > #endif > > +#ifdef CONFIG_DMA_PERNUMA_CMA > +void __init dma_pernuma_cma_reserve(void) > +{ > + int nid; > + > + if (!pernuma_size_bytes) > + return; If this is useful (I assume it is), then I think we should have a non-zero default value, a bit like normal CMA does via CMA_SIZE_MBYTES. > + for_each_node_state(nid, N_ONLINE) { for_each_online_node() { > + int ret; > + char name[20]; 20? Ah, wait, this is copy-pasta from hugetlb_cma_reserve(). Can you factor out the common parts at all? Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel