From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 465E2C433DF for ; Thu, 21 May 2020 14:22:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0972220748 for ; Thu, 21 May 2020 14:22:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="WUx7EMZX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0972220748 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9415C80008; Thu, 21 May 2020 10:22:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F1EB80007; Thu, 21 May 2020 10:22:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8079180008; Thu, 21 May 2020 10:22:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0241.hostedemail.com [216.40.44.241]) by kanga.kvack.org (Postfix) with ESMTP id 69D8880007 for ; Thu, 21 May 2020 10:22:29 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2ADA0824805A for ; Thu, 21 May 2020 14:22:29 +0000 (UTC) X-FDA: 76840941618.30.bomb97_8a0f80a609a35 X-HE-Tag: bomb97_8a0f80a609a35 X-Filterd-Recvd-Size: 5844 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 May 2020 14:22:28 +0000 (UTC) Received: by mail-ed1-f44.google.com with SMTP id bs4so6764020edb.6 for ; Thu, 21 May 2020 07:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=HXVdafa9z18urmVF3KS9w5IeKlLTrJcs4nowtAYCuh4=; b=WUx7EMZXoTwxert//L8QB2qHfWrZiacoi76OCdMTNyuay8JGMq6fWNmQj0SQnbRFEh WRh1UcI6g53+15lQHxMx26j8V4Gxuc5gDqobeFXzjZDMWfdl3fkdcoRgn0v8vArqtsrt 1Vv1WER8yCs5NkdKv+KCn0M/GlQDXzUtCKOAE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=HXVdafa9z18urmVF3KS9w5IeKlLTrJcs4nowtAYCuh4=; b=J5UTviFFqwIjUMBh1sY9iUGUIQv34uyx18hCnzAuGhPmOjUzPfToIUONrsY697YZTF 1GGAlqFYfiUDzrZbSj9/rKVvwjWcPTlYoKHVQ3tRcagFF6Y6LvjaFOfIjGu1Vr6UfZ74 FEs89OiBIyslMHfuNBhCyhrvxeUF3Etr4ayjLdaBvOQw1gYmx8KbZ72D8RHfWMN3IpKO qSb+/8xyoxGLiGNBmo59tPyzyevQnBMnd5pCOWQo4VufOHIw5b7PuEYIcBisCQl+41r+ ZJN9hAQ/5N8txp4X+huc9FoJ0N4DmSyZMZXBCO1wJy++ZR5Ozmskqnk0CgBFya4Q50mc uGkA== X-Gm-Message-State: AOAM531NIjATqLsmYIoV81N3dUCt0M47Re3gNFOgfkQhizhGoPi420rO /HzqLe7yLjpe99T557p23e8WoA== X-Google-Smtp-Source: ABdhPJy8mfT4IP25yMwlh6TstUIzlUkg7uNP8ZDxoGoGuZc54HVQpdqG/mu217uXfwgIu/hzemCzXA== X-Received: by 2002:a05:6402:1c0e:: with SMTP id ck14mr8165813edb.356.1590070947093; Thu, 21 May 2020 07:22:27 -0700 (PDT) Received: from localhost ([2620:10d:c093:400::5:4262]) by smtp.gmail.com with ESMTPSA id i2sm4588656edy.30.2020.05.21.07.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 May 2020 07:22:26 -0700 (PDT) Date: Thu, 21 May 2020 15:22:23 +0100 From: Chris Down To: Michal Hocko Cc: Andrew Morton , Johannes Weiner , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling Message-ID: <20200521142223.GG990580@chrisdown.name> References: <20200520202650.GB558281@chrisdown.name> <20200521071929.GH6462@dhcp22.suse.cz> <20200521112711.GA990580@chrisdown.name> <20200521120455.GM6462@dhcp22.suse.cz> <20200521122327.GB990580@chrisdown.name> <20200521123742.GO6462@dhcp22.suse.cz> <20200521125759.GD990580@chrisdown.name> <20200521132120.GR6462@dhcp22.suse.cz> <20200521133324.GF990580@chrisdown.name> <20200521135838.GT6462@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20200521135838.GT6462@dhcp22.suse.cz> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Michal Hocko writes: >On Thu 21-05-20 14:41:47, Chris Down wrote: >> Michal Hocko writes: >> > On Thu 21-05-20 13:57:59, Chris Down wrote: >[...] >> > > If you're talking about reclaim, trying to reason about whether the overage >> > > is the result of some other task in this cgroup or the task that's >> > > allocating right now is something that we already know doesn't work well >> > > (eg. global OOM). >> > >> > I am not sure I follow you here. >> >> Let me rephrase: your statement is that it's not desirable "that some task >> would be throttled unexpectedly too long because of [the activity of another >> task also within that cgroup]" (let me know if that's not what you meant). >> But trying to avoid that requires knowing which activity abstractly >> instigates this entire mess in the first place, which we have nowhere near >> enough context to determine. > >Yeah, if we want to be really precise then you are right, nothing like >that is really feasible for the reclaim. Reclaiming 1 page might be much >more expensive than 100 pages because LRU order doesn't reflect the >cost of the reclaim at all. What, I believe, we want is a best effort, >really. If the reclaim target is somehow bound to the requested amount >of memory then we can at least say that more memory hungry consumers are >reclaiming more. Which is something people can wrap their head around >much easier than a free competition on the reclaim with some hard to >predict losers who do all the work and some lucky ones which just happen >to avoid throttling by a better timing. Really think of the direct >reclaim and how the unfairness suck there. I really don't follow this logic. You're talking about reclaim-induced latency, but the alternative isn't freedom from latency, it's scheduler-induced latency from allocator throttling (and probably of a significantly higher magnitude). And again, that's totally justified if you are part of a cgroup which is significantly above its memory.high -- that's the kind of grouping you sign up for when you put multiple tasks in the same cgroup. The premise of being over memory.high is that everyone in the affected cgroup must do their utmost to reclaim where possible, and if they fail to get below it again, we're going to deschedule them. *That's* what's best-effort about it. The losers aren't hard to predict. It's *all* the tasks in this cgroup if they don't each make their utmost attempt to get the cgroup's memory back under control. Doing more reclaim isn't even in the same magnitude of sucking as getting allocator throttled.