From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98A48C36010 for ; Tue, 8 Apr 2025 20:23:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E9C06B0099; Tue, 8 Apr 2025 16:23:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09B666B009E; Tue, 8 Apr 2025 16:23:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA26B6B00A2; Tue, 8 Apr 2025 16:23:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CDAB76B0099 for ; Tue, 8 Apr 2025 16:23:38 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 97C93BAB1E for ; Tue, 8 Apr 2025 20:23:40 +0000 (UTC) X-FDA: 83312002200.09.0F77929 Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by imf15.hostedemail.com (Postfix) with ESMTP id DAC39A0011 for ; Tue, 8 Apr 2025 20:23:38 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=u00QKcuc; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of sj@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744143819; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uDbhXh8Jkq2e5fy5ZyNfKpdmGru9W2M9l6yA+82K9mo=; b=haPX9qCNSTDwoYJ6yIq9e4P4Gb+ZzwVGB7oMAmMLcQ447XAJDngngFKfhS8Zdsxo7xzzDw XSKSefDA2aZ699sgOuvSsFCmYG8PE2n4L7PUjYLHfzGuFIiHT4S18OzbuBWQ9HxakLQjUr CzCJ1nIu+EUMmF7W2lnhUjwINyK9iss= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=u00QKcuc; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of sj@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744143819; a=rsa-sha256; cv=none; b=ko8FbuouPVv6pymM9Np+ntRcBFA8wtjiGFXBzN2RZSxb7dD6EZBEklOKJj4uxGO3ot0pjj nPNDAS63MperfEVq5ZPh8mR4yC7hCH4wq4TviuKvkra4nHhBi6KEti8FrRdoBwrNnw3/H6 +T+kpof4CfjobBEem/wKlEEltS81Qcc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id D398E43A93; Tue, 8 Apr 2025 20:23:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 48AB7C4CEE5; Tue, 8 Apr 2025 20:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1744143817; bh=t4dpsxhoSJicILQ+ZU+F72OUh+2j3XDJDE9iB0faTV0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u00QKcucJGpBXB3S05SDZ5ju4wOe8gmgglIm6qzGf+1mieBEz9ZCNrdb5nchtdWkV MWk+P+3KB7HDCtvi96OWsm2rLP/MuItrwpysYu5UByPPee3zyOcVPDdnJuBot5qv/s 1ogUpy3rRhcMPpJ7lrQtcU65yGk/HV/JHRCsx4ZiAoxguBoLbGjFWxxJlkDFuaQ08G SWfsKi3tRsRha7RpudPzb1kJY8E3QcbBAzDf2BWGVGBze8XKINxdyOfrE2N8Wnb/wA n3Wmk+xkydv4WZOo7vtq0f1bW7+TPZ2VKacjQLvDJW1YU+B3yQRMZh71wi90RUk84A rZ9SR2c2Trf5g== From: SeongJae Park To: Lorenzo Stoakes Cc: SeongJae Park , Andrew Morton , "Liam R.Howlett" , David Hildenbrand , Rik van Riel , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 0/4] mm/madvise: batch tlb flushes for MADV_DONTNEED and MADV_FREE Date: Tue, 8 Apr 2025 13:23:35 -0700 Message-Id: <20250408202335.63434-1-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: DAC39A0011 X-Stat-Signature: qu6qhp1gmbhzay8xe3rjf8r6xitut878 X-Rspam-User: X-HE-Tag: 1744143818-609628 X-HE-Meta: U2FsdGVkX1/h5T3+8rzQkQBQkuFhnIjD2tIe0KvD3ar28+kW5YKpR64P+ojnUhnFpcukx5cliGL93ghiddJmWCl52ayheJ0RZ7upEgUzXMM3B7jXQpsDEiR1cp4ZMbEREA9KkXEtRzgs0iR4wpavywgKfsDR8aX/ErZsFC9soGE+LjFsoVoJ/Qu0XZZvETr5scENXUN61F5KMgk3LnBI88kWdL4Z3P/xDa2L0UOwivIb/S/8qH/MWEccrkqfo6khwSn3LWZ1NAujvf8jnJ+aU7SnY0xXhx9jocRnHHm3d0whkMn1lkMYn5lGZa1esPB/Pt7AbmoDppabM5aU6XxdwdcGLhtlB9kimdJM/jY0Qypq8ul4t1MF8wcXz6862pBbJfBUQ2C6BU+r62kd0L2Lj32ystjDKPzMGh4bx3Bo5DW7o4wNjWOtEXH1af/OjwlDlLPZcE+Y9GhsHQXeSq0aDs6SXdK98a+MeeZBT6bII7+0xG2Zg8chItuxf+KjE3fCYFAU5B/i7fY1mbZgfQv7OR3rbRY/i6r2Fg9UhIHbUlpJrjaMQtqDuTwPjwkdAWSR9BM5I6Vkp42AsrErHB/4w6DPhBbDT5zr7nbxSSbnnWJwRjFlUL5PvWCKtTQNzYi8aObqDXulD4/eGLqJrvLivivqdhsx2SsjNIZspUqXKFiw+Z/SbnpQKegjUVVq2G141XdE8KTvQl5eJBrr4IJM6eAGcpgJZ5qcDVBMAEOP+oBwlCWbgIzJSrGJ7ZlI2xrltS33EJffwopvCzdZ/0WY1EWtF5Zilt0vr2Ocph7ZU76zxXGPfK1IEl8c+HdxC+NvZjnxESs1FMJ22GYlzl95o+Lf3D0IZjkrH34gAgK8Z3mDv0mh+PVGJNx1pdFVlwAEOpMuJ2gQBtg5eDD/VnR6KMBli27aBcHsPrje5CRBPk1BAwDVrBGXGNTM5wGkQpbJLotNKHiBmQYu+LfTVrW +X5s4u3X WRbkd77lcA0vY7FL8sng54xSlVptnmXf159jAvcjlhJl4g88cpbvStL7hih/nb+fRxg4fzfTjr8jxHWfCMkGaG2Ti8AqLGaX8QGwoATCR74ssEKKGWsyFNB/x65ytIXZQpdzzgjCTyPwnOSEslN7xmioxIoTJrtwQXRqSOWti0H66IJFr1StalE8V22shFe5jpqyBPeYm0Pghz7GR2gyQ7RChY+g1OjwTepmqOvF7+oqppqk01SEvsP/0fwVynw4xxNrUN9h3wAv8sToUfKjCa8GO2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 8 Apr 2025 14:44:40 +0100 Lorenzo Stoakes wrote: > On Fri, Apr 04, 2025 at 02:06:56PM -0700, SeongJae Park wrote: > > When process_madvise() is called to do MADV_DONTNEED[_LOCKED] or > > MADV_FREE with multiple address ranges, tlb flushes happen for each of > > the given address ranges. Because such tlb flushes are for same > > Nit: for _the_ same. Thank you for kindly finding and suggesting fixes for these mistakes. I will update following your suggestions here and below. [...] > > Similar optimizations might be applicable to other madvise behaviros > > Typo: behaviros -> behavior (or 'behaviors', but since behavior is already plural > probably unnecessary). > > > such as MADV_COLD and MADV_PAGEOUT. Those are simply out of the scope > > of this patch series, though. > > Well well, for now :) Yes. Hopefully we will have another chance to further improve the cases. [...] > > Test Results > > ============ > > > > I measured the latency to apply MADV_DONTNEED advice to 256 MiB memory > > using multiple process_madvise() calls. I apply the advice in 4 KiB > > sized regions granularity, but with varying batch size per > > process_madvise() call (vlen) from 1 to 1024. The source code for the > > measurement is available at GitHub[1]. To reduce measurement errors, I > > did the measurement five times. > > Be interesting to see how this behaves with mTHP sizing too! But probably a bit > out of scope perhaps. Obviously we have many more rooms to explore and get fun :) > > > > > The measurement results are as below. 'sz_batch' column shows the batch > > size of process_madvise() calls. 'Before' and 'After' columns show the > > average of latencies in nanoseconds that measured five times on kernels > > that built without and with the tlb flushes batching of this series > > (patches 3 and 4), respectively. For the baseline, mm-new tree of > > 2025-04-04[2] has been used. 'B-stdev' and 'A-stdev' columns show > > ratios of latency measurements standard deviation to average in percent > > for 'Before' and 'After', respectively. 'Latency_reduction' shows the > > reduction of the latency that the 'After' has achieved compared to > > 'Before', in percent. Higher 'Latency_reduction' values mean more > > efficiency improvements. > > > > sz_batch Before B-stdev After A-stdev Latency_reduction > > 1 110948138.2 5.55 109476402.8 4.28 1.33 > > 2 75678535.6 1.67 70470722.2 3.55 6.88 > > 4 59530647.6 4.77 51735606.6 3.44 13.09 > > 8 50013051.6 4.39 44377029.8 5.20 11.27 > > 16 48657878.2 9.32 37291600.4 3.39 23.36 > > 32 43614180.2 6.06 34127428 3.75 21.75 > > 64 42466694.2 5.70 26737935.2 2.54 37.04 > > 128 42977970 6.99 25050444.2 4.06 41.71 > > 256 41549546 1.88 24644375.8 3.77 40.69 > > 512 42162698.6 6.17 24068224.8 2.87 42.92 > > 1024 40978574 5.44 23872024.2 3.65 41.75 > > Very nice! Great work. > > > > > As expected, tlb flushes batching provides latency reduction that > > proportional to the batch size. The efficiency gain ranges from about > > 6.88 percent with batch size 2, to about 40 percent with batch size 128. > > > > Please note that this is a very simple microbenchmark, so real > > efficiency gain on real workload could be very different. > > Indeed, accepted, but it makes a great deal of sense to batch these operations, > especially when we get to the point of actually increasing the process_madvise() > iov size. Cannot agree more. Thank you for your kind review with great suggestions for this patchset. I will post the next spin with the suggested changes, soon. Thanks, SJ [...]