From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-187.mta0.migadu.com (out-187.mta0.migadu.com [91.218.175.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32B451E88C for ; Tue, 16 Jan 2024 22:14:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.187 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705443264; cv=none; b=sKP5ulw7ZkFerNcgVXQUzZGZyE11ZucxdxAQGR/zXrho9Dt2kwvcMhZHdthi+ZNhhScb59h1VErKVx5yqFjz94/J4KpAkzS/8ReJawiqJUFI+p9r2xXol6tETDkYlfOHQN/W7abCUBklDuJgpuFXsbtGP1eqfNO8BquqDVuafLs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705443264; c=relaxed/simple; bh=BHxOvbPe0gRsEgsVgptRc7fQ2mie7HRH2j1MIAtD0bU=; h=Date:DKIM-Signature:X-Report-Abuse:From:To:Cc:Subject:Message-ID: References:MIME-Version:Content-Type:Content-Disposition: In-Reply-To:X-Migadu-Flow; b=IvImsw+Z23dEHobQ9JL3HQdjO/OgAWDQg5srd9rc2seuE5o+JhdAM7noelHxHcCnkGvNEeLP7tJiOPFsDPrkFmEs7Oze7WA+pigFA8njnCpTOJ/5F0mrWJ+SxMwFupxul6p/5rFqxlDRp+Dsq2qCtfcV73nZGJA5XAkhFOXRpcc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Rgz/59S1; arc=none smtp.client-ip=91.218.175.187 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Rgz/59S1" Date: Tue, 16 Jan 2024 14:14:13 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1705443259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=GnrZ1bB7nI/fgK6fIh23djJJYB59aTDjzqfjJ+Q7M6s=; b=Rgz/59S1gAi5Ri4GCK5iixPff47rRLyjQTBSh8eZnkkgNKoVLsDYuVQmTE6hP4Mu2yXBp3 jWnnzd76vOza2jY44qfSLO2Mp6+Om2TcIUWIPeGG13poMGoyLf7yx5sIgu+QV1U6JUvGpz ZceQUv5ZGgqzWuXiOaLeQ3qPBpGEGqQ= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Andrew Morton Cc: mm-commits@vger.kernel.org, yosryahmed@google.com, tj@kernel.org, shakeelb@google.com, schatzberg.dan@gmail.com, muchun.song@linux.dev, mhocko@kernel.org, hannes@cmpxchg.org Subject: Re: + mm-memcontrol-dont-throttle-dying-tasks-on-memoryhigh.patch added to mm-hotfixes-unstable branch Message-ID: References: <20240116214549.6E4EAC433C7@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240116214549.6E4EAC433C7@smtp.kernel.org> X-Migadu-Flow: FLOW_OUT On Tue, Jan 16, 2024 at 01:45:47PM -0800, Andrew Morton wrote: > > The patch titled > Subject: mm: memcontrol: don't throttle dying tasks on memory.high > has been added to the -mm mm-hotfixes-unstable branch. Its filename is > mm-memcontrol-dont-throttle-dying-tasks-on-memoryhigh.patch > > This patch will shortly appear at > https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-memcontrol-dont-throttle-dying-tasks-on-memoryhigh.patch Hi Andrew, there is an updated version from Johannes in the same thread. It seems like you've picked the original version. Please, pick the new one instead. Thank you! > > This patch will later appear in the mm-hotfixes-unstable branch at > git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm > > Before you just go and hit "reply", please: > a) Consider who else should be cc'ed > b) Prefer to cc a suitable mailing list as well > c) Ideally: find the original patch on the mailing list and do a > reply-to-all to that, adding suitable additional cc's > > *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** > > The -mm tree is included into linux-next via the mm-everything > branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm > and is updated there every 2-3 working days > > ------------------------------------------------------ > From: Johannes Weiner > Subject: mm: memcontrol: don't throttle dying tasks on memory.high > Date: Thu, 11 Jan 2024 08:29:02 -0500 > > While investigating hosts with high cgroup memory pressures, Tejun > found culprit zombie tasks that had were holding on to a lot of > memory, had SIGKILL pending, but were stuck in memory.high reclaim. > > In the past, we used to always force-charge allocations from tasks > that were exiting in order to accelerate them dying and freeing up > their rss. This changed for memory.max in a4ebf1b6ca1e ("memcg: > prohibit unconditional exceeding the limit of dying tasks"); it noted > that this can cause (userspace inducable) containment failures, so it > added a mandatory reclaim and OOM kill cycle before forcing charges. > At the time, memory.high enforcement was handled in the userspace > return path, which isn't reached by dying tasks, and so memory.high > was still never enforced by dying tasks. > > When c9afe31ec443 ("memcg: synchronously enforce memory.high for large > overcharges") added synchronous reclaim for memory.high, it added > unconditional memory.high enforcement for dying tasks as well. The > callstack shows that this path is where the zombie is stuck in. > > We need to accelerate dying tasks getting past memory.high, but we > cannot do it quite the same way as we do for memory.max: memory.max is > enforced strictly, and tasks aren't allowed to move past it without > FIRST reclaiming and OOM killing if necessary. This ensures very small > levels of excess. With memory.high, though, enforcement happens lazily > after the charge, and OOM killing is never triggered. A lot of > concurrent threads could have pushed, or could actively be pushing, > the cgroup into excess. The dying task will enter reclaim on every > allocation attempt, with little hope of restoring balance. > > To fix this, skip synchronous memory.high enforcement on dying tasks > altogether again. Update memory.high path documentation while at it. > > Link: https://lkml.kernel.org/r/20240111132902.389862-1-hannes@cmpxchg.org > Fixes: c9afe31ec443 ("memcg: synchronously enforce memory.high for large overcharges") > Signed-off-by: Johannes Weiner > Reported-by: Tejun Heo > Reviewed-by: Yosry Ahmed > Acked-by: Shakeel Butt > Acked-by: Roman Gushchin > Cc: Dan Schatzberg > Cc: Michal Hocko > Cc: Muchun Song > Signed-off-by: Andrew Morton > --- > > mm/memcontrol.c | 24 +++++++++++++++++++++--- > 1 file changed, 21 insertions(+), 3 deletions(-) > > --- a/mm/memcontrol.c~mm-memcontrol-dont-throttle-dying-tasks-on-memoryhigh > +++ a/mm/memcontrol.c > @@ -2623,8 +2623,9 @@ static unsigned long calculate_high_dela > } > > /* > - * Scheduled by try_charge() to be executed from the userland return path > - * and reclaims memory over the high limit. > + * Reclaims memory over the high limit. Called directly from > + * try_charge() when possible, but also scheduled to be called from > + * the userland return path where reclaim is always able to block. > */ > void mem_cgroup_handle_over_high(gfp_t gfp_mask) > { > @@ -2693,6 +2694,9 @@ retry_reclaim: > } > > /* > + * Reclaim didn't manage to push usage below the limit, slow > + * this allocating task down. > + * > * If we exit early, we're guaranteed to die (since > * schedule_timeout_killable sets TASK_KILLABLE). This means we don't > * need to account for any ill-begotten jiffies to pay them off later. > @@ -2887,8 +2891,22 @@ done_restock: > } > } while ((memcg = parent_mem_cgroup(memcg))); > > + /* > + * Reclaim is scheduled for the userland return path already, > + * but also attempt synchronous reclaim to avoid excessive > + * overrun while the task is still inside the kernel. If this > + * is successful, the return path will see it when it rechecks > + * the overage, and simply bail out. > + * > + * Skip if the task is already dying, though. Unlike > + * memory.max, memory.high enforcement isn't as strict, and > + * there is no OOM killer involved, which means the excess > + * could already be much bigger (and still growing) than it > + * could for memory.max; the dying task could get stuck in > + * fruitless reclaim for a long time, which isn't desirable. > + */ > if (current->memcg_nr_pages_over_high > MEMCG_CHARGE_BATCH && > - !(current->flags & PF_MEMALLOC) && > + !(current->flags & PF_MEMALLOC) && !task_is_dying() && > gfpflags_allow_blocking(gfp_mask)) { > mem_cgroup_handle_over_high(gfp_mask); > } > _ > > Patches currently in -mm which might be from hannes@cmpxchg.org are > > mm-memcontrol-dont-throttle-dying-tasks-on-memoryhigh.patch >