From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1713C25B74 for ; Mon, 27 May 2024 04:03:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 679146B0089; Mon, 27 May 2024 00:03:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 627BB6B008A; Mon, 27 May 2024 00:03:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EFCC6B008C; Mon, 27 May 2024 00:03:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F2166B0089 for ; Mon, 27 May 2024 00:03:03 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CFE2A1411FD for ; Mon, 27 May 2024 04:03:02 +0000 (UTC) X-FDA: 82162830204.25.2581050 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf12.hostedemail.com (Postfix) with ESMTP id 358424000B for ; Mon, 27 May 2024 04:02:59 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf12.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716782581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DiJh0fnqP9xqDPFep0R/i/nsA3LwaQOXD8CfrRWdf9w=; b=BQGzoebdUOYPUXlX+o3TkjeRzQnOMOnXVZxCEz+FdV8ny/d11MpdtVJhoH3mGdJ7RCcb9I eTqHYMWXncTw8aiJ4eqO7mIsYpiw8+3mRL19uB+ednNJrrU67mK5NF6lVom45KfUbRC368 yJ8wcFZ1oAlY0Rs4P1Q8vazXKwgrVs8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf12.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716782581; a=rsa-sha256; cv=none; b=W5NzYxNU6TLjYww4YSzq/tdk1SndMl/vLtJGRTteVcyrSSBUYPLFYy335fJ4Uca34yiOWp PLrrv3y03XNqnkwqGTdgBdaC4F6H9+r+bKc7QvnD/MfBdP/xPUSr6Rg4lMinzdHjMoxPPN oEd/DKRhtfNBxhipyauo2//4X7U/M+A= X-AuditID: a67dfc5b-d6dff70000001748-eb-6654045d84c7 Date: Mon, 27 May 2024 12:56:08 +0900 From: Byungchul Park To: "Huang, Ying" Cc: Dave Hansen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: Re: [PATCH v10 00/12] LUF(Lazy Unmap Flush) reducing tlb numbers over 90% Message-ID: <20240527035608.GB12937@system.software.com> References: <20240510065206.76078-1-byungchul@sk.com> <982317c0-7faa-45f0-82a1-29978c3c9f4d@intel.com> <20240527015732.GA61604@system.software.com> <8734q46jc8.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8734q46jc8.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrAIsWRmVeSWpSXmKPExsXC9ZZnoe4H1pA0g7dfdSzmrF/DZvF5wz82 i08vHzBavNjQzmjxdf0vZounn/pYLC7vmsNmcW/Nf1aL87vWslrsWLqPyeLSgQVMFsd7DzBZ zL/3mc1i86apzBbHp0xltPj9A6j45KzJLA6CHt9b+1g8ds66y+6xYFOpx+YVWh6L97xk8ti0 qpPNY9OnSewe786dY/c4MeM3i8e8k4Ee7/ddZfPY+svOo3HqNTaPz5vkAviiuGxSUnMyy1KL 9O0SuDI+HbvGWvBGouL1ssUsDYyPhboYOTkkBEwkGietZu9i5ACzL7eLgYRZBFQlJr/8zApi swmoS9y48ZMZxBYR0JD4tHA5UDkXB7NAH7PEmsWHGEESwgIhEtM+rGECsXkFLCQuLN0PZgsJ nGaU2NrmChEXlDg58wkLiM0soCVx499LJpC9zALSEsv/cYCYnAJ2Eg93SIJUiAooSxzYdpwJ ZJWEwC52iU37z7FCnCwpcXDFDZYJjAKzkEydhWTqLISpCxiZVzEKZeaV5SZm5pjoZVTmZVbo JefnbmIExuOy2j/ROxg/XQg+xCjAwajEw5vhHpwmxJpYVlyZe4hRgoNZSYRXZF5gmhBvSmJl VWpRfnxRaU5q8SFGaQ4WJXFeo2/lKUIC6YklqdmpqQWpRTBZJg5OqQZGAQW3lAO/3WLNZB3u b5lx7H1wh+018V0q2m98bGUuKvn1s1/j2vx/j3VNS9/JiTP2Sd29ciUswPPU8kMdr1jEa431 rzHHqr+49VaCfdo3oa+le74dXCAeWMAgtYIzSi1cSlD0KnsIT2D71IYlNVqu35UXrPWe63un rbOUPaFuJ8OsG4+4t/xQYinOSDTUYi4qTgQAiJQRe8MCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprOIsWRmVeSWpSXmKPExsXC5WfdrBvLEpJmcGKFuMWc9WvYLD5v+Mdm 8enlA0aLFxvaGS2+rv/FbPH0Ux+LxeG5J1ktLu+aw2Zxb81/Vovzu9ayWuxYuo/J4tKBBUwW x3sPMFnMv/eZzWLzpqnMFsenTGW0+P0DqPjkrMksDkIe31v7WDx2zrrL7rFgU6nH5hVaHov3 vGTy2LSqk81j06dJ7B7vzp1j9zgx4zeLx7yTgR7v911l81j84gOTx9Zfdh6NU6+xeXzeJBfA H8Vlk5Kak1mWWqRvl8CV8enYNdaCNxIVr5ctZmlgfCzUxcjBISFgInG5XayLkZODRUBVYvLL z6wgNpuAusSNGz+ZQWwRAQ2JTwuXs3cxcnEwC/QxS6xZfIgRJCEsECIx7cMaJhCbV8BC4sLS /WC2kMBpRomtba4QcUGJkzOfsIDYzAJaEjf+vWQC2cssIC2x/B8HiMkpYCfxcIckSIWogLLE gW3HmSYw8s5C0jwLSfMshOYFjMyrGEUy88pyEzNzTPWKszMq8zIr9JLzczcxAqNrWe2fiTsY v1x2P8QowMGoxMOb4R6cJsSaWFZcmXuIUYKDWUmEV2ReYJoQb0piZVVqUX58UWlOavEhRmkO FiVxXq/w1AQhgfTEktTs1NSC1CKYLBMHp1QD4/zrMn9mpt5cXHXij9nO8hUH/L6cSeTd8XhJ yp8V3jyfPr3q+SLn/m/L9N3quYFm1xMu1P8Qn9fZdUH00MtLbSly9hMdU5wXPP8SqK/2XKom ZVP1pt06Xaf4/+rWpVYeVXeZem/imTdN09ZMn9Tss6XcZqd/2q46c3f7KyY/ll2qq/mscvxN vasSS3FGoqEWc1FxIgDJUBi2qgIAAA== X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 358424000B X-Stat-Signature: yqgcakij3abk51skaxhwttzw8di8eirj X-HE-Tag: 1716782579-748936 X-HE-Meta: U2FsdGVkX1+gYXxxpSNv8u58vqTf9Uu8mcHysroskcGuMO6K8asuA0el9V80QdHFfpmWMKdyR3gYWPCI+U4NBL+PnSFrS36mz1XRotJM+EfMP4aE1Zpj3+iI68D1rOPB7DGipEY4/iU6dN8RHe/379dvSmatx1e1GE6Cexe/rP8Pmm5cZVkTIqUVg+mhKjvP6Y1AifOUXUnRH+3C3/wr2SskhImh8nZ7/spnNEbaQ39d5+fGOS2dv/07oLApFQd2hCpEBP8dMOvYPiI55CTHmskYGHHJxoLBtaDzLPYccUU0pdumIBhdc2N2iRiurOy9Wo4dvN543uxXnbyE+j0PadzEsrisg8acn5naqLXOdVwxX04itraToM1YEpt9X9H9Sr5kPnB5KcfhmMC99JgXKfrFr69twzob+nK0qSox5JynOAU5hYWjpEk4qMk36IX1yM6tW9HFvX5lGLD8zm1f6mJhc3q+piYfLCGmxNjVX9mphZzbrulAXzNNYVcIHddcsikehmw6XOQTn48hbcZaMhJ0+ICbQBxfN2g5Ecy14Hr8Kxfj/hkCwWgmKM2rK6Xhm0U9dnjJLzYZv08VghkI9lDtqYb4tEnYzDE3s4dN76eWuORIWWz+ATvse/zE3osYHiXlqMKSRzdyOl/jDmwqfBQoHbIlrEyP1j/9BLkhFEaJXfqt0tsf1qscyzklrZdy09NMlzbP7ibnTDm4UyVRFQ/C9Qn8ZUoJHF10i0IVJeQW30mvjnXFHVK2SibEhqLCQGSMN5Svo9z6Qt6lXUL/zmTCmixEfcA7ZsDzPUeSL/5h9LGJVwNVILHT2AOJH9KMk9jXkefIa8Ka2x1E0+h1Yom9Spa2OobgRIMMRhe9OlqgUp1xToc7zWaPzghBpOTmNIwL7ANHdLr8tdxW+HMPoCdWdcQpHu5yR8/p1V6MbvcY5GcfYk8MAvnM5Oa3uUsc2broIQvE11lVsVF1Wyv Mckvhx7/ rc4MoxhOp8AcNZ1zqo8U73gvT53pJaokVaMPP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, May 27, 2024 at 11:10:15AM +0800, Huang, Ying wrote: > Byungchul Park writes: > > > On Fri, May 24, 2024 at 10:16:39AM -0700, Dave Hansen wrote: > >> On 5/9/24 23:51, Byungchul Park wrote: > >> > To achieve that: > >> > > >> > 1. For the folios that map only to non-writable tlb entries, prevent > >> > tlb flush during unmapping but perform it just before the folios > >> > actually become used, out of buddy or pcp. > >> > >> Is this just _pure_ unmapping (like MADV_DONTNEED), or does it apply to > >> changing the memory map, like munmap() itself? > > > > I think it can be applied to any unmapping of ro ones but LUF for now is > > working only with unmapping during folio migrion and reclaim. > > > >> > 2. When any non-writable ptes change to writable e.g. through fault > >> > handler, give up luf mechanism and perform tlb flush required > >> > right away. > >> > > >> > 3. When a writable mapping is created e.g. through mmap(), give up > >> > luf mechanism and perform tlb flush required right away. > >> > >> Let's say you do this: > >> > >> fd = open("/some/file", O_RDONLY); > >> ptr1 = mmap(-1, size, PROT_READ, ..., fd, ...); > >> foo1 = *ptr1; > >> > >> You now have a read-only PTE pointing to the first page of /some/file. > >> Let's say try_to_unmap() comes along and decides it can_luf_folio(). > >> The page gets pulled out of the page cache and freed, the PTE is zeroed. > >> But the TLB is never flushed. > >> > >> Now, someone does: > >> > >> fd2 = open("/some/other/file", O_RDONLY); > >> ptr2 = mmap(ptr1, size, PROT_READ, MAP_FIXED, fd, ...); > >> foo2 = *ptr2; > >> > >> and they overwrite the old VMA. Does foo2 have the contents of the new > >> "/some/other/file" or the old "/some/file"? How does the new mmap() > > > > Good point. It should've give up LUF at the 2nd mmap() in this case. > > I will fix it by introducing a new flag in task_struct indicating if LUF > > has left stale maps for the task so that LUF can give up and flush right > > away in mmap(). > > > >> know that there was something to flush? > >> > >> BTW, the same thing could happen without a new mmap(). Someone could > >> modify the file in the middle, maybe even from another process. > > > > Thank you for the pointing out. I will fix it too by introducing a new > > flag in inode or something to make LUF aware if updating the file has > > been tried so that LUF can give up and flush right away in the case. > > > > Plus, I will add another give-up at code changing the permission of vma > > to writable. > > I guess that you need a framework similar as > "flush_tlb_batched_pending()" to deal with interaction with other TLB > related operations. Thank you. I will check it. Byungchul > -- > Best Regards, > Huang, Ying > > > Thank you very much. > > > > Byungchul > > > >> fd = open("/some/file", O_RDONLY); > >> ptr1 = mmap(-1, size, PROT_READ, ..., fd, ...); > >> foo1 = *ptr1; > >> // LUF happens here > >> // "/some/file" changes > >> foo2 = *ptr1; // Does this see the change?