From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BF1C00140 for ; Fri, 5 Aug 2022 19:24:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 554516B0071; Fri, 5 Aug 2022 15:24:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5035F6B0072; Fri, 5 Aug 2022 15:24:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CB018E0001; Fri, 5 Aug 2022 15:24:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2AE9D6B0071 for ; Fri, 5 Aug 2022 15:24:35 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ECA0C8063C for ; Fri, 5 Aug 2022 19:24:32 +0000 (UTC) X-FDA: 79766515584.12.0B4C9ED Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id 363A640068 for ; Fri, 5 Aug 2022 19:24:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=RKUUIUL1iHAOgJ7936O2ZK1JZMqC/oeNpBXr/pO1c2M=; b=XJonR3uwtW+WFiDJvcdgEOl5Kg L664nZw2YIp6hE1O6Qa+MmAJJ4n0YqnRjJBba1/WsTLd9MgOcskgoMdSRVSB2JbQm1LVodaeCXU+q 7iHdF1eiEah4flMBuDP7zr50XS1sGdOl1hrQD/cGRypY011bkx1UCICnW09y2C+Pppfp9+5F3+iYY jUy6BGqiw16FGVBSikLW2qXTeGZJ14+ksYukXsbdKqTPnMHt829lbOqZhcHf8GwbO3n8ZlOFAGRqv weBjofOa+m4GgZLBgWKRsxY8yWv4FOzwuIyso3BHJO1mhGCUmzJVz6/Nfb8+FHUMLGb7lKjKsrnJR jUP1fpVg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oK2vp-00BVNL-VE; Fri, 05 Aug 2022 19:24:26 +0000 Date: Fri, 5 Aug 2022 20:24:25 +0100 From: Matthew Wilcox To: Rik van Riel Cc: "Alex Zhu (Kernel)" , Kernel Team , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "akpm@linux-foundation.org" Subject: Re: [PATCH v3] mm: add thp_utilization metrics to /proc/thp_utilization Message-ID: References: <20220805184016.2926168-1-alexlzhu@fb.com> <0b16dbac6444bfcdfbeb4df4280354839bfe1a8f.camel@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0b16dbac6444bfcdfbeb4df4280354839bfe1a8f.camel@fb.com> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659727472; a=rsa-sha256; cv=none; b=1SQUwOGv4HGCLKkuuV70VBaUm/1jbBlHr4KgEhGb2cB1amOtEHjTwWJiWVBj4lTRrOUZDi 4oijPgYfI+g/+4Vr0LMNJIK+Y/JnM2TTO8L3AWNbxPmUqUOrU0y6gEQpvKFWMMFyIRuGWK iFuJwVTTzSEmOAnjNrgUeVoHxyF6UIQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XJonR3uw; dmarc=none; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659727472; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RKUUIUL1iHAOgJ7936O2ZK1JZMqC/oeNpBXr/pO1c2M=; b=Frlo1SW/Ja1KO8oXMvy3xwHavCArZwc7QbAWqXSJKgiYvYq7WxGAI0BFimwLCB5pseiXMJ ZkY0HUsLb4r/vQ0+8FBne+hh75bEN7cqkv5rbBTJCD6ZHt6S43R3q5GIwttVPe6UdHwzIv 6SBlsKR/W+s7NzVGodspFmUmP9qwSAg= Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XJonR3uw; dmarc=none; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 363A640068 X-Rspam-User: X-Stat-Signature: 1zxuu5q9n8dt9ni67miqkizbf9reft5f X-HE-Tag: 1659727472-503278 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 05, 2022 at 07:04:30PM +0000, Rik van Riel wrote: > On Fri, 2022-08-05 at 19:50 +0100, Matthew Wilcox wrote: > > > > > > This change introduces a tool that scans through all of physical > > > memory for anonymous THPs and groups them into buckets based > > > on utilization. It also includes an interface under > > > /proc/thp_utilization. > > > > OK, so I understand why we want to do the scanning, but why do we > > want to > > report it to userspace at all?  And if we do, why do we want to do it > > in > > this format?  AFAIK, there's nothing userspace can do with the > > information > > "93% of your THPs are underutilised".  If there was something > > userspace > > could do about it, wouldn't it need to know which ones? > > > > Isn't the real solution here entirely in-kernel?  This scanning > > thread > > you've created should be the one splitting the THPs.  And anyway, > > isn't > > this exactly the kind of thing that DAMON was invented for? > > Alex does have an (in kernel) shrinker that can reclaim > underutilized THPs in order to free memory. Ah! So when that exists, this interface tells us "how well" we're doing. > This is a regular shrinker called from kswapd. I am not > sure a shrinker going through the DAMON infrastructure > would be any smaller, faster, or better. OTOH, DAMON > does allow for more flexible policy... > > Getting some info on the effectiveness of the shrinker > seems useful, though. Could debugfs be a better place? > Should this be resubmitted together with the shrinker > code that makes this more useful? Yeah, debugfs seems like a better place. And I'd love to see the shrinker code. Before you mentioned that I was having all kinds of peculiar feelings about this code. For example, suppose you have incredibly hot 256kB of data, but the other 1792kB of data are never touched ... that could cause us to do entirely the wrong thing and break up this THP. Having it as a shrinker makes sense because the hot 256kB will keep the THP from reaching the end of the list and being reclaimed.