From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A4DC433DF for ; Thu, 21 May 2020 13:05:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE5C02078B for ; Thu, 21 May 2020 13:05:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="lm15zqD7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE5C02078B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 455C480008; Thu, 21 May 2020 09:05:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4052A80007; Thu, 21 May 2020 09:05:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CC7580008; Thu, 21 May 2020 09:05:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 12B8580007 for ; Thu, 21 May 2020 09:05:34 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C569A584D for ; Thu, 21 May 2020 13:05:33 +0000 (UTC) X-FDA: 76840747746.08.mist69_308080ef9f73d X-HE-Tag: mist69_308080ef9f73d X-Filterd-Recvd-Size: 4578 Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 May 2020 13:05:33 +0000 (UTC) Received: by mail-ed1-f68.google.com with SMTP id bs4so6607208edb.6 for ; Thu, 21 May 2020 06:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=lIVlACF4LDO220gEGSvhc/aauhEeT3bDk/ER46APYDY=; b=lm15zqD7bQsCw4wiTNv6NzVw1jCgcLOsWTU1x6wfuW5NkSlsXD99ZPhpzoFHTLPOU8 b29eUZlH28mu05WYQ2aUGv31+3wJp8CLc5ANp+8VlL/Yd5xKTarIVPKSxWu3uJWSpvsU I9xvM12xty15SeSsxedimd/k5ql9zjCbFIxvA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=lIVlACF4LDO220gEGSvhc/aauhEeT3bDk/ER46APYDY=; b=sThfkoJjGo5v78c0T6DzouZxJB7RRtdDvaOcEBD5nRwCSYtMp6Jj8KtYjEDbdTmQBc Jgo2D0JPDAZrJHeZtXqHiPswz2bFfh6j4+UUzTzq8J11SpbFYwlFrks+c4vNyfgzlMwx gCJpt7mJAp0bbrN2MpXngVAxuT4gbbHPf5wGlTlyfd+sKif4LjTdIAlO+NuBNsb0VdI6 TrVGTsQP1DthpVQIjLlTgG/DQ/fATqtbf+Z67puDBWS0tRTUb+QdiYHQKTMk9LS/1NeY wXl5YGqZ5n5P7AQMVEGoQmU+54H/QGjaBgMZWLmWzOttRzR3RFMzvlPYJPUi0g8QvqvL PGZA== X-Gm-Message-State: AOAM530wCsM33D8q81CVIspPtdXU7a+mvEif5oRO9eS5sUhgmFGJNHZ6 4hqlHP0YP/WORVpdTlRfBcNeeg== X-Google-Smtp-Source: ABdhPJw3rAmdCOdWp+3NYJhOme20reqKS4bSnIvR84PMg8X2oYRibYSG/X+k07nw/mVqMFTIJed2HQ== X-Received: by 2002:aa7:d1c6:: with SMTP id g6mr7728524edp.303.1590066331942; Thu, 21 May 2020 06:05:31 -0700 (PDT) Received: from localhost ([2620:10d:c093:400::5:4262]) by smtp.gmail.com with ESMTPSA id r1sm4850187ejz.59.2020.05.21.06.05.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 May 2020 06:05:31 -0700 (PDT) Date: Thu, 21 May 2020 14:05:30 +0100 From: Chris Down To: Michal Hocko Cc: Andrew Morton , Johannes Weiner , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling Message-ID: <20200521130530.GE990580@chrisdown.name> References: <20200520143712.GA749486@chrisdown.name> <20200520160756.GE6462@dhcp22.suse.cz> <20200520202650.GB558281@chrisdown.name> <20200521071929.GH6462@dhcp22.suse.cz> <20200521112711.GA990580@chrisdown.name> <20200521120455.GM6462@dhcp22.suse.cz> <20200521122327.GB990580@chrisdown.name> <20200521123742.GO6462@dhcp22.suse.cz> <20200521125759.GD990580@chrisdown.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20200521125759.GD990580@chrisdown.name> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Chris Down writes: >>I believe I have asked in other email in this thread. Could you explain >>why enforcint the requested target (memcg_nr_pages_over_high) is >>insufficient for the problem you are dealing with? Because that would >>make sense for large targets to me while it would keep relatively >>reasonable semantic of the throttling - aka proportional to the memory >>demand rather than the excess. > >memcg_nr_pages_over_high is related to the charge size. As such, if >you're way over memory.high as a result of transient reclaim failures, >but the majority of your charges are small, it's going to hard to make >meaningful progress: > >1. Most nr_pages will be MEMCG_CHARGE_BATCH, which is not enough to help; >2. Large allocations will only get a single reclaim attempt to succeed. > >As such, in many cases we're either doomed to successfully reclaim a >paltry amount of pages, or fail to reclaim a lot of pages. Asking >try_to_free_pages() to deal with those huge allocations is generally >not reasonable, regardless of the specifics of why it doesn't work in >this case. Oh, I somehow elided the "enforcing" part of your proposal. Still, there's no guarantee even if large allocations are reclaimed fully that we will end up going back below memory.high, because even a single other large allocation which fails to reclaim can knock us out of whack again.