From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BBD3C25B75 for ; Mon, 27 May 2024 01:57:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF5816B0082; Sun, 26 May 2024 21:57:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA5B56B0083; Sun, 26 May 2024 21:57:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6E6F6B0085; Sun, 26 May 2024 21:57:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BA5586B0082 for ; Sun, 26 May 2024 21:57:45 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 32C9940A98 for ; Mon, 27 May 2024 01:57:45 +0000 (UTC) X-FDA: 82162514490.10.EFDB32B Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf27.hostedemail.com (Postfix) with ESMTP id 915424000C for ; Mon, 27 May 2024 01:57:42 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716775063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WN8V4GH+LLlLQ/iSwFC6ocKlwCqxVWBbqiJ6Vx9iHzU=; b=ep9SbJz/fxEJo+B6hwfwRqNKfNKYgdpHR4Kdp+595q6TS4Pma/qS8q8PYY2gNu1ZHAZR4K GGdsD/4QIuvXo8D55oLCL69I0Ot5MhKhABz6GCi7Qb7O3FsxYwVYvDlAcRWsniC10km97+ YBHUJ6hWNIU6cqfhPtVtKuBrFUeoRDA= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716775063; a=rsa-sha256; cv=none; b=3ZF9MzbH2kyiDVdqxjRc5Yyia7GxMRYf43PtzNMvFP/lkzvsO73vlqeSTqyJpRevGMa13D kGhImZlW0jwfIAkRsPN8Dd35M2BahPBJGPF4nelPENudFAhVyjfu6WNgO14cSi5q4IM3Bc HVhBeGjvATIwPsr8I0qlLUZfGIWpVTg= X-AuditID: a67dfc5b-d85ff70000001748-f4-6653e89109d3 Date: Mon, 27 May 2024 10:57:32 +0900 From: Byungchul Park To: Dave Hansen Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: Re: [PATCH v10 00/12] LUF(Lazy Unmap Flush) reducing tlb numbers over 90% Message-ID: <20240527015732.GA61604@system.software.com> References: <20240510065206.76078-1-byungchul@sk.com> <982317c0-7faa-45f0-82a1-29978c3c9f4d@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <982317c0-7faa-45f0-82a1-29978c3c9f4d@intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrEIsWRmVeSWpSXmKPExsXC9ZZnoe7EF8FpBh83m1nMWb+GzeLzhn9s Fp9ePmC0eLGhndHi6/pfzBZPP/WxWFzeNYfN4t6a/6wW53etZbXYsXQfk8WlAwuYLI73HmCy mH/vM5vF5k1TmS2OT5nKaPH7B1DxyVmTWRwEPb639rF47Jx1l91jwaZSj80rtDwW73nJ5LFp VSebx6ZPk9g93p07x+5xYsZvFo95JwM93u+7yuax9ZedR+PUa2wenzfJBfBFcdmkpOZklqUW 6dslcGV83LGDvWCXSMXSP9/YGxgv8HcxcnJICJhInPh+kB3GnjJ3AwuIzSKgKtHU1c8MYrMJ qEvcuPETzBYBsk+tXA5Uz8XBLPCfSeL+w1awhLBAiMS0D2uYQGxeAQuJFed3sIHYQgIZEjNu XGSHiAtKnJz5BGwBs4CWxI1/L4HqOYBsaYnl/zhAwpwCthLLmnaCjREVUJY4sO04E8guCYFN 7BLzLz5ngzhUUuLgihssExgFZiEZOwvJ2FkIYxcwMq9iFMrMK8tNzMwx0cuozMus0EvOz93E CIzJZbV/oncwfroQfIhRgINRiYc3wz04TYg1say4MvcQowQHs5IIr8i8wDQh3pTEyqrUovz4 otKc1OJDjNIcLErivEbfylOEBNITS1KzU1MLUotgskwcnFINjPE1S//+lcrOtw9za/m1V21u ifNVTbkZmzYYpun9jzF4Vv1VfbHStcM1P1U3/JHhz+yOWtdQVnDno95FiVnaU73SMoyYbq6N +bpu1xWpWYbf03TqeHY9fP1wqn326fMKJtfFPryTrOIOylzt895m9wQlTfHHcy4wqqu2ZGwN iPiWf0lwedjM90osxRmJhlrMRcWJAIzPxWrFAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJIsWRmVeSWpSXmKPExsXC5WfdrDvxRXCawa25ahZz1q9hs/i84R+b xaeXDxgtXmxoZ7T4uv4Xs8XTT30sFofnnmS1uLxrDpvFvTX/WS3O71rLarFj6T4mi0sHFjBZ HO89wGQx/95nNovNm6YyWxyfMpXR4vcPoOKTsyazOAh5fG/tY/HYOesuu8eCTaUem1doeSze 85LJY9OqTjaPTZ8msXu8O3eO3ePEjN8sHvNOBnq833eVzWPxiw9MHlt/2Xk0Tr3G5vF5k1wA fxSXTUpqTmZZapG+XQJXxscdO9gLdolULP3zjb2B8QJ/FyMnh4SAicSUuRtYQGwWAVWJpq5+ ZhCbTUBd4saNn2C2CJB9auVy9i5GLg5mgf9MEvcftoIlhAVCJKZ9WMMEYvMKWEisOL+DDcQW EsiQmHHjIjtEXFDi5MwnYAuYBbQkbvx7CVTPAWRLSyz/xwES5hSwlVjWtBNsjKiAssSBbceZ JjDyzkLSPQtJ9yyE7gWMzKsYRTLzynITM3NM9YqzMyrzMiv0kvNzNzECI2xZ7Z+JOxi/XHY/ xCjAwajEw5vhHpwmxJpYVlyZe4hRgoNZSYRXZF5gmhBvSmJlVWpRfnxRaU5q8SFGaQ4WJXFe r/DUBCGB9MSS1OzU1ILUIpgsEwenVANjjPK/Bl4XLqVLqjmZa/dPObNwldCji599L+x+HloQ POfuCsW73RnFStHZLa8/G7/5nSnheNiuqDeiZMOX06/fnha5NNG16ZjQ8jhWWb81xQLVXb/N Jpjuqb9dd52xTyr/VwyjCFOjtmVrV1Xonj/Mkw3MXlmWrf22KlUt+vL2is9xzmpCD5crsRRn JBpqMRcVJwIAdZB/vawCAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 915424000C X-Stat-Signature: e5xpa7m5z8afy654xwzhk3j7k1ir3fo5 X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1716775062-778287 X-HE-Meta: U2FsdGVkX1+o9vc0SYA4idAuXK7sYyiCQJ1eY4ctopFQL+brAn2TYZBPKvEtU5F4Q03NAkFWGp4++VPX+vKPSOTidToGefxerI3bMo55Kc8Uf4rQdPC73Aod79bcXW5o2r84B2JiP7vZeE/7iAh5zqW/WWux+KpVFJ/oy6sVtfPVGsz6bj+RaXCHvCefrldiss5SGLcT2q8Zhj/wRwDTRDC/RYqMQHV81iiM2tb6e95U4BUjkv2fmRtx6iDhWt937AkihHG/tQ8Z1Bp+cGE9sw13Sqgfg5wGBtosWamsJlljRXSTWsqtOdr6uAi7i013tL4qk87xsLd/35RyMGCE6xUyQ+WZHJ9iVSOmbfM4cI2kR4WiERg2mFTL5bzKp1d5EnEcSKDfiCz/JLbxP7rQfqu7YJz8Sy9hLQcl36UrWnoEmFkQgglkdbjHvcI3hFafX1RwFNfK/TMq4pJx3LQesZI+zFpkcmkzuUajvpQCowH1JzBxv3rZtZAxEEszS6g3q9iCPAecd1QghDdBYJ87ep/8pqI2+Uf99pZqtV0hIDM7Q8AaBjxQR0Kf5J8SIaYGJeMN53iOICSvJODQs+pUaNRS5Hnt1t0Z501v18aGEMexsWC3OA+MsmvnwU/BcQdaS+TQLOJuxRTjCGOrwTVeaTnaj0DIo0Ns7QTkezLq2TVIBDUNB6Io3Y8fypUZyy6dTzSOKQuudOY0meyk8q00KbtNH1Vq8nDaasday3l8yg7QeMvm83fRG84TvNcSOr20/G1d++w4Kb6PFYbUnH77xtVRm1FbhQ9J0/nSvjvJBzofhiLKEtyQTXl1zwaHYMTesg1wY15YOizBRX4nrKT3boeR9+pR8PLAiopR2069kINjAnFtZQ2f47HQF7uoL3YMrf432dGtyrzjqMJl67VatEVLVojq6jct X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 24, 2024 at 10:16:39AM -0700, Dave Hansen wrote: > On 5/9/24 23:51, Byungchul Park wrote: > > To achieve that: > > > > 1. For the folios that map only to non-writable tlb entries, prevent > > tlb flush during unmapping but perform it just before the folios > > actually become used, out of buddy or pcp. > > Is this just _pure_ unmapping (like MADV_DONTNEED), or does it apply to > changing the memory map, like munmap() itself? I think it can be applied to any unmapping of ro ones but LUF for now is working only with unmapping during folio migrion and reclaim. > > 2. When any non-writable ptes change to writable e.g. through fault > > handler, give up luf mechanism and perform tlb flush required > > right away. > > > > 3. When a writable mapping is created e.g. through mmap(), give up > > luf mechanism and perform tlb flush required right away. > > Let's say you do this: > > fd = open("/some/file", O_RDONLY); > ptr1 = mmap(-1, size, PROT_READ, ..., fd, ...); > foo1 = *ptr1; > > You now have a read-only PTE pointing to the first page of /some/file. > Let's say try_to_unmap() comes along and decides it can_luf_folio(). > The page gets pulled out of the page cache and freed, the PTE is zeroed. > But the TLB is never flushed. > > Now, someone does: > > fd2 = open("/some/other/file", O_RDONLY); > ptr2 = mmap(ptr1, size, PROT_READ, MAP_FIXED, fd, ...); > foo2 = *ptr2; > > and they overwrite the old VMA. Does foo2 have the contents of the new > "/some/other/file" or the old "/some/file"? How does the new mmap() Good point. It should've give up LUF at the 2nd mmap() in this case. I will fix it by introducing a new flag in task_struct indicating if LUF has left stale maps for the task so that LUF can give up and flush right away in mmap(). > know that there was something to flush? > > BTW, the same thing could happen without a new mmap(). Someone could > modify the file in the middle, maybe even from another process. Thank you for the pointing out. I will fix it too by introducing a new flag in inode or something to make LUF aware if updating the file has been tried so that LUF can give up and flush right away in the case. Plus, I will add another give-up at code changing the permission of vma to writable. Thank you very much. Byungchul > fd = open("/some/file", O_RDONLY); > ptr1 = mmap(-1, size, PROT_READ, ..., fd, ...); > foo1 = *ptr1; > // LUF happens here > // "/some/file" changes > foo2 = *ptr1; // Does this see the change?