From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C04BEC61D97 for ; Fri, 24 Nov 2023 19:44:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52D288D00A8; Fri, 24 Nov 2023 14:44:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DD008D0096; Fri, 24 Nov 2023 14:44:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A5668D00A8; Fri, 24 Nov 2023 14:44:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2A8958D0096 for ; Fri, 24 Nov 2023 14:44:48 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 03AACA05F5 for ; Fri, 24 Nov 2023 19:44:47 +0000 (UTC) X-FDA: 81493875456.06.CC53F40 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf07.hostedemail.com (Postfix) with ESMTP id 3454440004 for ; Fri, 24 Nov 2023 19:44:45 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=GdheEjr3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of rientjes@google.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700855086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZcJe1cTUwmVFg6ikVC/lcaSHTAPPpULPQy7PDbLjWEk=; b=Hztxr/2WR+AN4FYV6Z0KmOLFta1XNkTanQxFnIVHdBY3FKhzie8bqxYyQPsx4UR7OXaxL+ xigl2f7+w5YlxwbwfSpboNRDploREpxVA5QbICVEC2U3dXQ4nsOR0X6qqFWDgy8iiycaIv 2n0+hgDDhzf3MFjkrIBpAn6EO2tM/U0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=GdheEjr3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of rientjes@google.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700855086; a=rsa-sha256; cv=none; b=c4VsrYUb/PJEIkozfchKBZwk7Q7feQUZL4O2Ep7YFmHCLR26jvezOqOrCAopWFbW+2/vgv MDUoTmAE6g+WbKrsGsgEmEv8VV5DBkh0JegitRxmRtVCGLm1lVCCSuIC3kuCP4JJ49l1te NV05D9IMaE8uxGkMUeRzCf3NqXaZJbo= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1cfaf05db73so53365ad.0 for ; Fri, 24 Nov 2023 11:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1700855085; x=1701459885; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=ZcJe1cTUwmVFg6ikVC/lcaSHTAPPpULPQy7PDbLjWEk=; b=GdheEjr3tKHq2SKBiidmf43MWVMtYbDc/bJY+Y+JnZ7ZFtKdGK/LXsI4XrARs178MS u6LGZ12Cr06pR07g6+59wTOkpn3R9G9yr+VZ/tgs/hlfwbCvOWLLXLx2FCC+f5RbMkl0 0ak19j8oGNck/roe3GShDpBKe4yHEMQgEGE/8uiicbt/agymhX1NX6seKbumBgFNPO/Z xE3SYjn+62T4or5sbGtzEdQQzgRQiiuVqczPygfR42E3ES/tk2Lnskzr/OxzhMi4gp93 3+/ovy3T6tPw+yxSzgJxVsPCmR+ScRkaEFmmulkGqzjU7lb57qUkAU3EnoUrhZIWcrz4 luUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700855085; x=1701459885; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZcJe1cTUwmVFg6ikVC/lcaSHTAPPpULPQy7PDbLjWEk=; b=L3/UFDJUE8VL7sduz3CZLAE17XA6pcmAkL63kPnmFt9Y2FCu9nWJrXIjsKHpk1CTIB rbNlwtvEXBT8Y8TkgWZi+Ty1igXzjV/QK0u9pexl+G2U0yEp6/40vMwa8uZUMgnO5Vgm tKY5iYhtuXSWmo4XC5ZKX2hvdGvObdZdhVNeP2+dDtps4zxqZD8z6dtG8hiwNVPfUVoY NILqyk9FfAMmJoDRWpSfnSIZ26sHWyK7gOG/SW+azmPMThzh0E6KfbigF4jnxIh2SsQl O1rJACXfyJY4zu2KY/7hpis6vevba90s5m6KWaC/K6f6MJWThufIp6SUnWRDwSTqL3HH oBig== X-Gm-Message-State: AOJu0YzvRt/t4r7IjbE9o7UCyujeeq4gkjxBu6vKiJQfnxPzZzoDJnLr zJETMi0WkiXSemsLCrUOLgdNRqDJblbChhbA+WAZAg== X-Google-Smtp-Source: AGHT+IH0k42WBbKtozlNqbG7QJC35OPgY972kv9kDmDxplkj0CdVXeD1A4NZmU0Q7tEfZ0OWOTcqHQ== X-Received: by 2002:a17:903:4d1:b0:1cf:a032:aeff with SMTP id jm17-20020a17090304d100b001cfa032aeffmr198568plb.11.1700855084806; Fri, 24 Nov 2023 11:44:44 -0800 (PST) Received: from [2620:0:1008:15:d807:a0b3:20ea:f28f] ([2620:0:1008:15:d807:a0b3:20ea:f28f]) by smtp.gmail.com with ESMTPSA id hq19-20020a056a00681300b006cbb80e4577sm3190013pfb.210.2023.11.24.11.44.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Nov 2023 11:44:44 -0800 (PST) Date: Fri, 24 Nov 2023 11:44:43 -0800 (PST) From: David Rientjes To: David Hildenbrand cc: Gang Li , Mike Kravetz , Muchun Song , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Gang Li Subject: Re: [RFC PATCH v1 0/4] hugetlb: parallelize hugetlb page allocation on boot In-Reply-To: <5c0e27f2-5826-4537-a1ab-1debfab65b9a@redhat.com> Message-ID: <28e28c2a-e72d-a181-e87a-39cecc8c3c76@google.com> References: <20231123133036.68540-1-gang.li@linux.dev> <5c0e27f2-5826-4537-a1ab-1debfab65b9a@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 3454440004 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: ocqzwww5i5ja4nmpdxh9p8foeq48sic1 X-HE-Tag: 1700855085-290805 X-HE-Meta: U2FsdGVkX19zQKmlseeviQs6a69yo6UpqKHJ6K9d7PA/YjK854FUmEXdSfG4eNX+3yYHK/COhxGyUu59G6UVwfpbcpssq/YVAv4RRYAd8nTUP+0p5dxRVKxrtJ9WYh8byCnIlDV0MCdakwY6Qssl66UrT5Dd2c/9xI2zjzxNYyXK5MhdzUEDEdArBwq4udlbT/z4vJjdpvDdlEUNBTpp7slomAab5TSpThiKGtAXEv85eYWu3yTYO+o0O0+FE9/qFLYzBZuwgDp1z+6jjUNRGAQCsOqM2ttIiMCZXzy7sKvT1q606GbmUpvzRIhSQxvWpyjYcOyCs8iz6wbjZNrUcHwfUSjboFE0Gx3RDS7hPH5cHv2X89dAr+SdgbNQaAFkE2X6sz3yV/uVckq/UofL66oC9mJ4ePY3EzoR3NVbQiVdUBAx00VOLDmeNwXbFH8EOE+eVyTr7vsrypsPqXAo/vamdF1okN6drCCTljYbls4cE4RDfN2xE0WsAIrgLpdx0x7BfvYpZQ39aFvYqpQYXDaSDq+iWIN8VifX8sP9pDTurrt3qo43Ob1rksr9rUaa7+LCNHD9IDVlpCJJawIeoJnGLEC71vMfDtag9BsGgZgoNC4QmgGB5pmLmwVnGRimjZV8HMqXvK7g6fctHeK96A5b8zLlGYU4hc8ZccDXBkfFILxfclxv59mQOs/rENZ9pBVGW5a1h9engAJ+R30xUl1pNaLPJaRudbltjGwDATIs8fqmCOTK7KdS2Gm+g1CMxQWUi35MNynu2EKk/Fct2HG1kxX4r/P2bB8cNFW86WytHeDDA8+EjCR5MpSDDzgqFNrtrijM1Dzjo/sHLF+JXiWhm7oxE7tkGVCDm45GnFJ6K55i83+2ndYWxl834UycZ5ReZxOuf62hpsvr3XTFbqfHN2Zo2qIELmHKvJ7sMNm9qpFCn24K/JIqsphtZdbq8QZaab6g6ECTfZp46qK XTAlJE+a HtDfizctI4yAG9hdgmzQzh2OEhG3l3WB0gaa587fMyb8u4H9VfbvhZtV6PUaGT7ivGY2izk+0rnwgSnflFJ43+9UNdK8uL+v36Zz6kcLIVAaNj5oQvYDoMhfTmMJRVcx/thpp+MHDncCjpdYmhrwXbCG7485ZIWlKnnYacGoaDgRWpQMf6vIae4Ie0Sx+AmhIvenb8DEby6aEC5Uei6xJpNWDNnyYyRJJ3lzC5FsHEEWIE0N5yyf3Mb7iOnMi+kXh6GSCXHWTGDretM70edIHtQqOJwzQnJPBlhnAeaFNZpd1DHsQf/Y7wZrpqEphDhOtt1W2VY7oanZst9n8aXrLUCNIGNFf3mMAHGJalAPHdShnTH2HWH5TE17zDIFmLjFqGH8MqtzcLDuhC2fUYjTXpYzaIu+FnrxNiL6G6HJukGVoSXowp1I80LTSQmeWw2KlnAMuUSOtlZtd2djCZ1bc1X5eEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000093, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 23 Nov 2023, David Hildenbrand wrote: > On 23.11.23 14:30, Gang Li wrote: > > From: Gang Li > > > > Inspired by these patches [1][2], this series aims to speed up the > > initialization of hugetlb during the boot process through > > parallelization. > > > > It is particularly effective in large systems. On a machine equipped > > with 1TB of memory and two NUMA nodes, the time for hugetlb > > initialization was reduced from 2 seconds to 1 second. > > Sorry to say, but why is that a scenario worth adding complexity for / > optimizing for? You don't cover that, so there is a clear lack in the > motivation. > > 2 vs. 1 second on a 1 TiB system is usually really just noise. > The cost will continue to grow over time, so I presume that Gang is trying to get out in front of the issue even though it may not be a large savings today. Running single boot tests, with the latest upstream kernel, allocating 1,440 1GB hugetlb pages on a 1.5TB AMD host appears to take 1.47s. But allocating 11,776 1GB hugetlb pages on a 12TB Intel host takes 65.2s today with the current implementation. So it's likely something worth optimizing. Gang, I'm curious about this in the cover letter: """ This series currently focuses on optimizing 2MB hugetlb. Since gigantic pages are few in number, their optimization effects are not as pronounced. We may explore optimizations for gigantic pages in the future. """ For >1TB hosts, why the emphasis on 2MB hugetlb? :) I would have expected 1GB pages. Are you really allocating ~500k 2MB hugetlb pages? So if the patchset optimizes for the more likely scenario on these large hosts, which would be 1GB pages, that would be great.