From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A318C25B78 for ; Thu, 23 May 2024 02:05:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 067556B0082; Wed, 22 May 2024 22:05:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 017226B0088; Wed, 22 May 2024 22:05:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E20CD6B0089; Wed, 22 May 2024 22:05:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C32FF6B0082 for ; Wed, 22 May 2024 22:05:52 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5F78E120175 for ; Thu, 23 May 2024 02:05:52 +0000 (UTC) X-FDA: 82148019744.18.2021E1F Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf18.hostedemail.com (Postfix) with ESMTP id 8CFB91C0013 for ; Thu, 23 May 2024 02:05:49 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf18.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716429950; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BRcO18RaLkSeMLfBwB3n4QD/reMnMZUORX9m0hUnfwc=; b=OSxe7pLdZI7pUatzwvvaxHdmHOsgf40UBmfqSlGSqlwiMtCXyBMN6XrYolUBQaZjfvNsEG PlGr97tWF2R0BiBFyQzktandCjXyY7H/1mWIGZrxrxE+zeejOawXDR7WYeq6Eh7Wz/XU6l rVbHDE8JJJakf8TCDnr0Q7WiJub4l8Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716429950; a=rsa-sha256; cv=none; b=SLqrGZ/WEvWGZ05S3HadDk4UHTyNWtIiiWPepJGC8NRYM1KN6Run6mnIrETR8ze0Cf5tEY zH9wJ8UvjAyIVGhuHsLZ3rttkJOLQXQc5Syoe7xicU5hWmhFjhql2S5kVRoYsIpd/pzPzM F3yL53GzMlOXHZh8aPn6tZ5JB8basjw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf18.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com X-AuditID: a67dfc5b-d6dff70000001748-71-664ea4793e2c Date: Thu, 23 May 2024 11:05:39 +0900 From: Byungchul Park To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: Re: [RESEND PATCH v10 00/12] LUF(Lazy Unmap Flush) reducing tlb numbers over 90% Message-ID: <20240523020539.GA21061@system.software.com> References: <20240520021734.21527-1-byungchul@sk.com> <20240522155641.a726c5cd3b25aa23e861045d@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240522155641.a726c5cd3b25aa23e861045d@linux-foundation.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNIsWRmVeSWpSXmKPExsXC9ZZnkW7lEr80g9/PhS3mrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAlfG j+tHWAu6+Sp6l2Q0ME7i7mLk5JAQMJHYcOo3M4z9recySxcjBweLgKrElquBIGE2AXWJGzd+ gpWICOhKrHq+C8jm4mAWeMQksfzjKjaQhLBAtMTD689ZQWxeAQuJrSvugsWFBMokHsw4ywIR F5Q4OfMJmM0soCVx499LJpBdzALSEsv/cYCEOQW8JV7Ma2cEsUUFlCUObDvOBLJLQmAVu8SH Dbeg7pSUOLjiBssERoFZSMbOQjJ2FsLYBYzMqxiFMvPKchMzc0z0MirzMiv0kvNzNzECo3BZ 7Z/oHYyfLgQfYhTgYFTi4e2Q9UsTYk0sK67MPcQowcGsJMIbvdI3TYg3JbGyKrUoP76oNCe1 +BCjNAeLkjiv0bfyFCGB9MSS1OzU1ILUIpgsEwenVAPj+vyZRfpVOQFM01NmFimulm8/E3FL 1tNiofG/5uMcvjcyt/Lmvt3yujHxd9ktMabteeJ/XvyI/Dy7oEh5k4pyySwpma6eL/ZRgWeu /67nT3H9IrjwhZrl5EP+U157hXRE9AgF3/KNOCzA2Pxc7g+HxVvzCd/DTX8+aH+d/XNL36yD NRnPxKYrsRRnJBpqMRcVJwIAR89vRL4CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprEIsWRmVeSWpSXmKPExsXC5WfdrFu5xC/N4OJbVos569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfHeA0wW 8+99ZrPYvGkqs8XxKVMZLX7/ACo+OWsyi4Ogx/fWPhaPnbPusnss2FTqsXmFlsfiPS+ZPDat 6mTz2PRpErvHu3Pn2D1OzPjN4jHvZKDH+31X2TwWv/jA5LH1l51H49RrbB6fN8kF8Edx2aSk 5mSWpRbp2yVwZfy4foS1oJuvondJRgPjJO4uRk4OCQETiW89l1m6GDk4WARUJbZcDQQJswmo S9y48ZMZxBYR0JVY9XwXkM3FwSzwiEli+cdVbCAJYYFoiYfXn7OC2LwCFhJbV9wFiwsJlEk8 mHGWBSIuKHFy5hMwm1lAS+LGv5dMILuYBaQllv/jAAlzCnhLvJjXzghiiwooSxzYdpxpAiPv LCTds5B0z0LoXsDIvIpRJDOvLDcxM8dUrzg7ozIvs0IvOT93EyMwppbV/pm4g/HLZfdDjAIc jEo8vAm/fNOEWBPLiitzDzFKcDArifBGrwQK8aYkVlalFuXHF5XmpBYfYpTmYFES5/UKT00Q EkhPLEnNTk0tSC2CyTJxcEo1MJp/DVIM+/81U2q92knPRetnnz10Py323pKOP1zeC5XNZdgj m+Jnhr/8r3ts110nZ2PBk77x//0Y/Ru/f3zeX6Bic1mORZV/STOHwHTeUqeoG0XBjc2pK08t 6fo69Z+m5oqzb28ur1eN6U891Grr9PSDpatbdXbMrD8WK2e/zlFbcJTn3P2tXUosxRmJhlrM RcWJAFm7ha6lAgAA X-CFilter-Loop: Reflected X-Stat-Signature: 9ee5q3pn6in8s8bcih44xpjiamtrq59b X-Rspamd-Queue-Id: 8CFB91C0013 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1716429949-535733 X-HE-Meta: U2FsdGVkX1+4WLCbT/JjrBHZ8h+T7YSfKeM1NUtC112IpZcI3M2RF0U8HaDtc+E7gFqieMceuo98VRa0kiCSrsklQeJwrcksWipkyGcMOYYycZykzXJ3xeM/1K0F0HLTmkaoDge6/+CxUcgY5hnB06sqiK/OXRoggc/AAwtSOP88kKw9DYdL8oPEPC0Rr8TBNgpaThuGuUiNoivBb5HzBXOA/kR/HPCgVhwV+dKVS15OjuUz6FIBo9BMQdTfzyF6/xvPm3bLZ5eZflJqVloAoM9SZKFRvffhwypduhtu/3tbRS3/stpdIWFPwifmTgLVnIAzkq6EHmgo+DRkjQPZVAiF4lTOUq67zAdA84yeaE/xAK9K/ZD8klTqwz7x+RXH9QuyXMWDF2yIHGgoANHCfmampE0O3W3fBDGR/Vzi7f3PDJBfamy77bPNjyx8JI0DawyURBqYk4PyGsPTLfBA0UgmWWv2OxfospT4n0IRGWvy8rx9d4me+okChFPOH7SMotFT6vRQZqKpBhDcATG99S/4KCwtPkC3N6mtTB+Z7PKm9KXZiiKe22zZyb4r8UsI9j/UL8Fd6TTT+BmRYpMYb1Ly7H7Kliksc04t+lsckm0ndHnuES10y9Y7En1l14XX6B1zExLKRIl4UIaN/UccTBNU1YTUk3yZ9lA9OSSK0bM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, May 22, 2024 at 03:56:41PM -0700, Andrew Morton wrote: > On Mon, 20 May 2024 11:17:22 +0900 Byungchul Park wrote: > > > While I'm working with a tiered memory system e.g. CXL memory, I have > > been facing migration overhead esp. tlb shootdown on promotion or > > demotion between different tiers. Yeah.. most tlb shootdowns on > > migration through hinting fault can be avoided thanks to Huang Ying's > > work, commit 4d4b6d66db ("mm,unmap: avoid flushing tlb in batch if PTE > > is inaccessible"). See the following link for more information: > > > > https://lore.kernel.org/lkml/20231115025755.GA29979@system.software.com/ > > > > However, it's only for migration through hinting fault. I thought it'd > > be much better if we have a general mechanism to reduce all the tlb > > numbers that we can apply to any unmap code, that we normally believe > > tlb flush should be followed. > > > > I'm suggesting a new mechanism, LUF(Lazy Unmap Flush), defers tlb flush > > until folios that have been unmapped and freed, eventually get allocated > > again. It's safe for folios that had been mapped read-only and were > > unmapped, since the contents of the folios don't change while staying in > > pcp or buddy so we can still read the data through the stale tlb entries. > > Version 10 and no reviewed-by's or acked-by's. Reviewing the review > history isn't helped by the change in the naming of the patch series. > > Seems that you're measuring a ~5% overall speedup in a realistic > workload? That's nice. > > I'll defer this for a week or so to see what reviewers have to say. If > "nothing", please poke me and I guess I'll merge it up to see what I will poke you and will be ready for that ;) Byungchul > happens ;)