From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A9F2C36017 for ; Wed, 2 Apr 2025 12:17:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A9EE280003; Wed, 2 Apr 2025 08:17:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 75918280001; Wed, 2 Apr 2025 08:17:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62045280003; Wed, 2 Apr 2025 08:17:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 46890280001 for ; Wed, 2 Apr 2025 08:17:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 676BC1A18AF for ; Wed, 2 Apr 2025 12:17:25 +0000 (UTC) X-FDA: 83289004050.12.610639D Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf12.hostedemail.com (Postfix) with ESMTP id 6CDDF40010 for ; Wed, 2 Apr 2025 12:17:23 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=eaxRae+3; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf12.hostedemail.com: domain of mhocko@suse.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743596243; a=rsa-sha256; cv=none; b=XvPPP4t1LEgXbwMoh0XkN72OkAjKQGwd5aiLAGwqsC0vl6mjHr7A4cSJOtntV0KYdk3W/u YA39BlscPNlNO0Ho7G5oUCk++z5MFttGTK8kqoFK8l1fBZxgWmaNJ7bkHxcmkRfmXU3sxx l7GVeBmgrsDH3ZGAARgniNsKiXMIifg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=eaxRae+3; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf12.hostedemail.com: domain of mhocko@suse.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743596243; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u+KkYGiRVTeKTKTukwG7gqG5k92Sb1BtHpbYVZQpWnQ=; b=TdmmJzUGZ4bj8Ux5iTEzLX6zEmA6Woeo3ZfbH7Aagt9Our4iY85AXUEaS9jEiHSKKZIsyL uBt9NypfVDbDKuK5h0965SRpoxAPCw9xvR5lsrnWwGnXnl1VpgJKdSnPh1q+Gr4CSv+Ftb SDtuaxHMl/38835F8CdBfXD0dzXAIYw= Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-43ce71582e9so48681125e9.1 for ; Wed, 02 Apr 2025 05:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1743596242; x=1744201042; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=u+KkYGiRVTeKTKTukwG7gqG5k92Sb1BtHpbYVZQpWnQ=; b=eaxRae+3x0zTE5Bb2XCj4jmRXy85mDIvBHsj8MYeMI5rA08lPHegLhvU21ts57daVy EkX/6Rxsh/6fL72gpP8u2oFew/LSgLT0g5bQ4HYby/kveMJ7QE7d1V6DvKQmvKJ8kyZ3 /fX8MLXLunufSEIFBj1R/HWpVVtsAtb23TBq7jEAgrk1URnsr94Metx4N/vtrT2CBYwT +5NWqjVaQfqp4t/yw01tOr71fsRetGbE6g54bzJA0PiM1NKck5idZ6nuCRKnewpLxog9 N0mIe7gGueIUqcgQBtOkgZkVvqgPSKqNVvEMvmbl1ruohf05YngVxP7h+wVls9bRnsCS fTmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743596242; x=1744201042; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=u+KkYGiRVTeKTKTukwG7gqG5k92Sb1BtHpbYVZQpWnQ=; b=ukcA1EuHBpR3hJUMBy5M/XYlZRD+uI6DzkrZEr6GuLgw+WcE8PNV3jQtvZR2gTBJfW XotBf4yMklQVBh/chZ4gpeTDpsAtPS+IHJIu2sPJx3+GtyIX8NeYj0mlPves90KXuz4Q I4vJG658P5TvLyPLOdoWwCO0H5P2rzZiTzlcU3NYPwy50mqPgmT8+0IYXU0iDVBaQilJ S30v/Gj91LA1wLnzBOA6V9kC7u+fakBowEz1zpsY+NHF3A7psrLraL1wrNRX1hFxIPUU Y7rl/oCS+R+mNLrgI10uFfA6zEY/8NzBK9kxNdAzHGvLKQ66K/kdJ4rvhgP8/HYtCSx0 9tYw== X-Forwarded-Encrypted: i=1; AJvYcCU8amtRlrbmgyCA51UhAKoOveweJADWrigfurIqu+oxvuEeq7QmmzJtqSDvqsD5OUL0i+GR5WUfAw==@kvack.org X-Gm-Message-State: AOJu0YwRy9vBQ3q9d24tMeOFzPc0zApcvH6e04nUIk2WF/Qd9khFLI0v 0rH8GzTk5ALqJdsgPgknVhwhcxgThxyuxYXaBtHdpA9Uaf1bCTwgfJr2VGn5LtI= X-Gm-Gg: ASbGncvyiOH7veSdfMASUKOhDOCUbBOuBsklpH8w8hdbsY1a6zK2651zv/04VXao8ZL R13IXi1vh+NWCG004T5MSgDs9UhNZufCaLnP3Tj6JTh3FnNG1zaGMw6+w7J5O5mpWcKB1RBlh/+ ycXpTvrK+ahk4c7jY+cbQiuPyYSZtHif0Wycub7KLFpKOuLu2fU7TXy8QCdJJ+PFJDvMS7Jfzht 1cLWJ09uGYg8o9YW/ZSlV2qNjsYjVAc+J3UYIEagdVQY2CuuHQ143j+jep3XthIceIqH+19wI1h IbsExsByrtu3i6GLhCSYQ0ZvMWkeLXfnSfCisF6rj56ys/XN5gGh66jhJw== X-Google-Smtp-Source: AGHT+IEzL1dLp6w4/vrpTpg8YH1leTJhRx3ESY+EtSrwaEQ5UAajY5nV9dBkST09FFg5gDba1jeZjA== X-Received: by 2002:a05:600c:3d0c:b0:43d:8ea:8d7a with SMTP id 5b1f17b1804b1-43e8eeb6696mr111382655e9.28.1743596241822; Wed, 02 Apr 2025 05:17:21 -0700 (PDT) Received: from localhost (109-81-92-185.rct.o2.cz. [109.81.92.185]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-43eb5fd138esm18831065e9.13.2025.04.02.05.17.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Apr 2025 05:17:21 -0700 (PDT) Date: Wed, 2 Apr 2025 14:17:20 +0200 From: Michal Hocko To: Vlastimil Babka Cc: Yafang Shao , Harry Yoo , Kees Cook , joel.granados@kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Josef Bacik , linux-mm@kvack.org Subject: Re: [PATCH] proc: Avoid costly high-order page allocations when reading proc files Message-ID: References: <20250401073046.51121-1-laoar.shao@gmail.com> <3315D21B-0772-4312-BCFB-402F408B0EF6@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6CDDF40010 X-Stat-Signature: hnaqpqiatmbccje4bwwggosw1g99n8c3 X-Rspam-User: X-HE-Tag: 1743596243-494202 X-HE-Meta: U2FsdGVkX1+SXrjC4ebKD7EesTAKz/TzogrwraMW3SbUbKx0Dd/S8QqHSWqI2/7S24w03PHrGJiyJxK2tIigCK2E8hPIgVU91LiaSoJpWsnXxv69FtliP3D3K1Y6BETYucqwn06QnyKfMc9X1ER0xYBEiOFlyoJMsdJSnbiFqaQ57vmmMY/Xy81M09WLx0Ls4Dq/eXfXa6FbQxD100Xy6uptz71Z7qw569HUS9b5yzo9mgMDBkp8kQiP5ooJY2gRRBLqcmSshQiHeL3vynLDGPPHvdNZI7VvebU/sHWwnj7QMBPQHhvxHTS+MWa8lYRG1KqIAuYFuG1+jobkHmT5fLFlMgluyygTtzSqIsSKJdoQ6+Xuqo402kSralArnTi7tOAfE3AQqHszrhQyILRRcEKk+G1dTSwhp2BpTR5gg5DmfqgK6cb4Sx8x9+lrE95DqZm3Zn4Ft6pHX4VmccrINhfiJTD5Qrp0/Q5ig0AH6bLCBSImW8vKkMDJP4JSiXb0pfZkQtRCu9rshVBeuj2k8WltrL/oUT3rYdp2Q7oEeug6cota/ZZ0Y6BpQWrtCm2K0BUdLltpTyQAYp0CvffR4R2MBU5W/mZkZW2JaIXbG/GY7lo2XlYZTa2TbNqBEEx1Da4z24QoQ0r9ff9/AS9iDnHA52ImjT7nKeuP//nKycw5zWy18pebtUoIo5eWdpJrgkfIeTYfmjRIT+3QfamyIRu//+wuFP/NGjjW5B/cSNuRZw4Ak+RnPhKYF17bPbEsgLWG9vjFvo3YcboJzuWztwK3LbbGnMaZTRD2fr0iOODpmJ2hJ24ZgjZwgirgJnfOeV3hPFddsS6UNEtpkKI9aFiAcPvv5Jnjpz3h9ReRSYMUIpLjxUPX+Q9pVffZ9hwgI2t65NtiDUEX5Fbin/PL2j4wjywPJW2S5HT3+J5pTuH5OiA3B3r7gEkiIP7TaXGLa2C9wCYjuNzKxhIwPNY yeaH1WzY idehMT7ZFSgMu3LrfQYvZ1sOH8c5x3N4KgtMwXWRxODTQndbr0mBOvwTWhvEjks5HgVyBrBY2VF28wGIiegOQ2dwgkQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed 02-04-25 11:25:12, Vlastimil Babka wrote: > On 4/2/25 10:42, Yafang Shao wrote: > > On Wed, Apr 2, 2025 at 12:15 PM Harry Yoo wrote: > >> > >> On Tue, Apr 01, 2025 at 07:01:04AM -0700, Kees Cook wrote: > >> > > >> > > >> > On April 1, 2025 12:30:46 AM PDT, Yafang Shao wrote: > >> > >While investigating a kcompactd 100% CPU utilization issue in production, I > >> > >observed frequent costly high-order (order-6) page allocations triggered by > >> > >proc file reads from monitoring tools. This can be reproduced with a simple > >> > >test case: > >> > > > >> > > fd = open(PROC_FILE, O_RDONLY); > >> > > size = read(fd, buff, 256KB); > >> > > close(fd); > >> > > > >> > >Although we should modify the monitoring tools to use smaller buffer sizes, > >> > >we should also enhance the kernel to prevent these expensive high-order > >> > >allocations. > >> > > > >> > >Signed-off-by: Yafang Shao > >> > >Cc: Josef Bacik > >> > >--- > >> > > fs/proc/proc_sysctl.c | 10 +++++++++- > >> > > 1 file changed, 9 insertions(+), 1 deletion(-) > >> > > > >> > >diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c > >> > >index cc9d74a06ff0..c53ba733bda5 100644 > >> > >--- a/fs/proc/proc_sysctl.c > >> > >+++ b/fs/proc/proc_sysctl.c > >> > >@@ -581,7 +581,15 @@ static ssize_t proc_sys_call_handler(struct kiocb *iocb, struct iov_iter *iter, > >> > > error = -ENOMEM; > >> > > if (count >= KMALLOC_MAX_SIZE) > >> > > goto out; > >> > >- kbuf = kvzalloc(count + 1, GFP_KERNEL); > >> > >+ > >> > >+ /* > >> > >+ * Use vmalloc if the count is too large to avoid costly high-order page > >> > >+ * allocations. > >> > >+ */ > >> > >+ if (count < (PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER)) > >> > >+ kbuf = kvzalloc(count + 1, GFP_KERNEL); > >> > > >> > Why not move this check into kvmalloc family? > >> > >> Hmm should this check really be in kvmalloc family? > > > > Modifying the existing kvmalloc functions risks performance regressions. > > Could we instead introduce a new variant like vkmalloc() (favoring > > vmalloc over kmalloc) or kvmalloc_costless()? > > We have gfp flags and kmalloc_gfp_adjust() to moderate how aggressive > kmalloc() is before the vmalloc() fallback. It does e.g.: > > if (!(flags & __GFP_RETRY_MAYFAIL)) > flags |= __GFP_NORETRY; > > However if your problem is kcompactd utilization then the kmalloc() attempt > would have to avoid ___GFP_KSWAPD_RECLAIM to avoid waking up kswapd and then > kcompactd. Should we remove the flag for costly orders? Dunno. Ideally the > deferred compaction mechanism would limit the issue in the first place. Yes, triggering heavy compation for costly allocations seems to be quite bad. We have GFP_RETRY_MAYFAIL for that purpose if the caller really needs the allocation to try really hard. > The ad-hoc fixing up of a particular place (/proc files reading) or creating > a new vkmalloc() and then spreading its use as you see other places > triggering the issue seems quite suboptimal to me. Yes I absolutely agree. -- Michal Hocko SUSE Labs