From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8EA4CC6B0E for ; Tue, 13 Oct 2026 07:54:18 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2C47F10F5CD; Tue, 13 Oct 2026 07:53:42 +0000 (UTC) Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 98E9510F2E5 for ; Mon, 12 Oct 2026 18:38:57 +0000 (UTC) Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by sea.source.kernel.org (Postfix) with ESMTP id 4AF0F4378B; Mon, 12 Oct 2026 18:38:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B16F1F0089B; Mon, 12 Oct 2026 18:38:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1791830337; bh=fP03qOjNgvJcNFxilOwiajFRX7c9NaLWJPxXiXEaFwM=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=FpTrSmMagijQ4s8+E1TqfKdKHrTHU0BFRjA/iCJzTavYCoIGrP+M/jA/kByrAy4dF hg5+n9OJT0wTIiPuLZk3nQ4HlvQEC1Pt7lSKlbbnQhSLrANVy6PsCVdXT+ZqRFf+WV GBi1IdoRtPBhhxaaK0OLjDeY5tcu18IyA0h0uTRG50/+GGfE+jKtZkG0HLfBGJiUWU Hu8DdnHwjFwwYku6PhhIIj1fCcErlCDKnAGYjbWKOUSId3Gm+w+Lucn03/P2JAkYqI yQmNHzkW+VZUHiTptwbi5AP53a4fnEFmQ+CHOiXo+2e7qaN83k4kAuhfxRivDIuAWs d/avRkBSQydRA== Date: Mon, 12 Oct 2026 19:38:21 +0100 From: "Lorenzo Stoakes (ARM)" To: "David Hildenbrand (Arm)" Cc: Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Jann Horn , Pedro Falcato , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Greg Kroah-Hartman , Dennis Dalessandro , Jason Gunthorpe , Leon Romanovsky , Paul Moore , Stephen Smalley , Jaroslav Kysela , Takashi Iwai , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Doug Gilbert , "James E.J. Bottomley" , "Martin K. Petersen" , Simona Vetter , Helge Deller , Sebastian Reichel , John Hubbard , Peter Xu , Masami Hiramatsu , Oleg Nesterov , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Rik van Riel , Harry Yoo , Juri Lelli , Vincent Guittot , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Arnd Bergmann , Muchun Song , Oscar Salvador , "Matthew Wilcox (Oracle)" , Jan Kara , Marc Zyngier , Oliver Upton , Catalin Marinas , Madhavan Srinivasan , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , "David S. Miller" , Andreas Larsson , Alexander Viro , Christian Brauner , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Youngjun Park , Johannes Weiner , Qi Zheng , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chengming Zhou , Michal Hocko , Miklos Szeredi , Xu Xin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-usb@vger.kernel.org, linux-rdma@vger.kernel.org, selinux@vger.kernel.org, linux-sound@vger.kernel.org, bpf@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fbdev@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, fuse-devel@lists.linux.dev, Ihor Solodrai , Bert Karwatzki Subject: Re: [PATCH v5 23/38] mm/mlock: eliminate weird VMA_IO_BIT abuse and simplify Message-ID: References: <20261009-b4-mmap-prepare-vma-flag-sanify-v5-0-923b70125b1e@kernel.org> <20261009-b4-mmap-prepare-vma-flag-sanify-v5-23-923b70125b1e@kernel.org> <50c967fb-b6de-4189-9430-5455c26a5385@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50c967fb-b6de-4189-9430-5455c26a5385@kernel.org> X-Mailman-Approved-At: Tue, 13 Oct 2026 07:52:58 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Mon, Oct 12, 2026 at 08:23:15PM +0200, David Hildenbrand (Arm) wrote: > On 10/9/26 20:04, Lorenzo Stoakes (ARM) wrote: > > When performing mlock() or munlock() otherwise normal VMAs have VMA_IO_BIT > > solely to fix a race with migration which might otherwise double-count > > mlock VMAs. > > > > This is unnecessary - at the point of applying folio mlock state, whether > > setting or clearing PG_mlocked, we know whether or not we are locking. > > > > Solve this in two ways - thread a boolean through the page table walk > > indicating whether a lock or unlock is being performed, and run a locking > > walk with VMA_LOCKONFAULT_BIT set and VMA_LOCKED_BIT cleared. > > > > This state never occurs otherwise, as VMA_LOCKONFAULT_BIT always implies > > VMA_LOCKED_BIT. These are also always cleared together. > > > > Then, update folio_add_lru_vma() and mlock_folio() to check only for > > VMA_LOCKED_BIT, and update try_to_unmap_one() to check for VMA_LOCKED_MASK > > instead. > > > > Also remove the useless invocation of allow_mlock_munlock() which simply > > returns true if unlocking and instead rename it to allow_mlock() and only > > call it when locking. > > > > With the other mlock abuse of VMA_IO_BIT addressed, update > > mlock_vma_folio() and folio_add_lru_vma() to simply test for > > VMA_LOCKED_BIT. > > > > The munlock_vma_folio() function tests VMA_LOCKED_MASK instead, as an > > unmap racing with the locking walk must still munlock folios the walk has > > already counted. > > > > Additionally, in order to ensure a racing unmap/migration releases the > > reference munlock_folio() took (whose put was batched), check > > VMA_LOCKED_MASK before calling mlock_drain_local() in try_to_unmap_one() > > and try_to_migrate_one(). > > > > While here, also replace some deprecated VMA flag predicates. > > > > Reviewed-by: Zi Yan > > Signed-off-by: Lorenzo Stoakes (ARM) > > --- > > mm/folio.c | 2 +- > > mm/internal.h | 10 +++++++--- > > mm/mlock.c | 51 +++++++++++++++++++-------------------------------- > > mm/rmap.c | 8 +++++--- > > 4 files changed, 32 insertions(+), 39 deletions(-) > > > > diff --git a/mm/folio.c b/mm/folio.c > > index 47a437e0f7fd..35e242b48870 100644 > > --- a/mm/folio.c > > +++ b/mm/folio.c > > @@ -505,7 +505,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma) > > { > > VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); > > > > - if (unlikely((vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) == VM_LOCKED)) > > + if (vma_test(vma, VMA_LOCKED_BIT)) > > mlock_new_folio(folio); > > else > > folio_add_lru(folio); > > diff --git a/mm/internal.h b/mm/internal.h > > index 4ae08fedffff..ea8fa94ebe1b 100644 > > --- a/mm/internal.h > > +++ b/mm/internal.h > > @@ -970,8 +970,7 @@ void mlock_folio(struct folio *folio); > > static inline void mlock_vma_folio(struct folio *folio, > > struct vm_area_struct *vma) > > { > > - /* The VM_IO check prevents migration from double-counting during mlock. */ > > - if (unlikely((vma->vm_flags & (VM_LOCKED|VM_SPECIAL)) == VM_LOCKED)) > > + if (vma_test(vma, VMA_LOCKED_BIT)) > > mlock_folio(folio); > > } > > > > @@ -988,7 +987,12 @@ static inline void munlock_vma_folio(struct folio *folio, > > * always munlock the folio and page reclaim will correct it > > * if it's wrong. > > */ > > - if (unlikely(vma->vm_flags & VM_LOCKED)) > > + /* > > + * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see > > + * mlock_vma_pages_range(). An unmap racing with the walk must still > > + * munlock folios the walk has already counted. > > + */ > > + if (unlikely(vma_test_any_mask(vma, VMA_LOCKED_MASK))) > > munlock_folio(folio); > > } > > > > diff --git a/mm/mlock.c b/mm/mlock.c > > index 39215a3eab1f..4235a1518fc9 100644 > > --- a/mm/mlock.c > > +++ b/mm/mlock.c > > @@ -316,22 +316,10 @@ static inline unsigned int folio_mlock_step(struct folio *folio, > > return folio_pte_batch(folio, pte, ptent, count); > > } > > > > -static inline bool allow_mlock_munlock(struct folio *folio, > > +static inline bool allow_mlock(struct folio *folio, > > struct vm_area_struct *vma, unsigned long start, > > unsigned long end, unsigned int step) > > { > > - /* > > - * For unlock, allow munlock large folio which is partially > > - * mapped to VMA. As it's possible that large folio is > > - * mlocked and VMA is split later. > > - * > > - * During memory pressure, such kind of large folio can > > - * be split. And the pages are not in VM_LOCKed VMA > > - * can be reclaimed. > > - */ > > - if (!vma_test(vma, VMA_LOCKED_BIT)) > > - return true; > > - > > /* folio_within_range() cannot take KSM, but any small folio is OK */ > > if (!folio_test_large(folio)) > > return true; > > @@ -352,6 +340,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, > > > > { > > struct vm_area_struct *vma = walk->vma; > > + const bool lock = walk->private; > > spinlock_t *ptl; > > pte_t *start_pte, *pte; > > pte_t ptent; > > @@ -368,7 +357,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, > > folio = pmd_folio(*pmd); > > if (folio_is_zone_device(folio)) > > goto out; > > - if (vma_test(vma, VMA_LOCKED_BIT)) > > + if (lock) > > mlock_folio(folio); > > else > > munlock_folio(folio); > > @@ -390,10 +379,10 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, > > continue; > > > > step = folio_mlock_step(folio, pte, addr, end); > > - if (!allow_mlock_munlock(folio, vma, start, end, step)) > > + if (lock && !allow_mlock(folio, vma, start, end, step)) > > goto next_entry; > > > > - if (vma_test(vma, VMA_LOCKED_BIT)) > > + if (lock) > > mlock_folio(folio); > > else > > munlock_folio(folio); > > @@ -428,31 +417,29 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma, > > .pmd_entry = mlock_pte_range, > > .walk_lock = PGWALK_WRLOCK_VERIFY, > > }; > > + const bool lock = vma_flags_test(new_vma_flags, VMA_LOCKED_BIT); > > + vma_flags_t walk_flags = *new_vma_flags; > > > > /* > > - * There is a slight chance that concurrent page migration, > > - * or page reclaim finding a page of this now-VMA_LOCKED_BIT vma, > > - * will call mlock_vma_folio() and raise page's mlock_count: > > - * double counting, leaving the page unevictable indefinitely. > > - * Communicate this danger to mlock_vma_folio() with VMA_IO_BIT, > > - * which is a VMA_SPECIAL_FLAGS flag not allowed on VMA_LOCKED_BIT vmas. > > - * mmap_lock is held in write mode here, so this weird > > - * combination should not be visible to other mmap_lock users; > > - * but WRITE_ONCE so rmap walkers must see VMA_IO_BIT if VMA_LOCKED_BIT. > > + * LOCKONFAULT without LOCKED never otherwise occurs: it marks a walk in > > + * progress so that rmap-side callers, which test VMA_LOCKED_BIT, do not > > + * count folios, while try_to_unmap_one(), which tests VMA_LOCKED_MASK, > > + * still refuses to unmap them. > > */ > > - if (vma_flags_test(new_vma_flags, VMA_LOCKED_BIT)) > > - vma_flags_set(new_vma_flags, VMA_IO_BIT); > > + if (lock) { > > + vma_flags_clear(&walk_flags, VMA_LOCKED_BIT); > > + vma_flags_set(&walk_flags, VMA_LOCKONFAULT_BIT); > > + } > > + > > vma_start_write(vma); > > - vma_flags_reset_once(vma, new_vma_flags); > > + vma_flags_reset_once(vma, &walk_flags); > > > > lru_add_drain(); > > - walk_page_range_vma(vma, start, end, &mlock_walk_ops, NULL); > > + walk_page_range_vma(vma, start, end, &mlock_walk_ops, (void *)lock); > > lru_add_drain(); > > > > - if (vma_flags_test(new_vma_flags, VMA_IO_BIT)) { > > - vma_flags_clear(new_vma_flags, VMA_IO_BIT); > > + if (lock) > > vma_flags_reset_once(vma, new_vma_flags); > > - } > > } > > > > /* > > diff --git a/mm/rmap.c b/mm/rmap.c > > index 5332c52909be..b4a4b3929044 100644 > > --- a/mm/rmap.c > > +++ b/mm/rmap.c > > @@ -2239,9 +2239,11 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, > > > > /* > > * If the folio is in an mlock()d vma, we must not swap it out. > > + * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see > > + * mlock_vma_pages_range(). > > */ > > if (!(flags & TTU_IGNORE_MLOCK) && > > - (vma->vm_flags & VM_LOCKED)) { > > + vma_test_any_mask(vma, VMA_LOCKED_MASK)) { > > ptes++; > > > > /* > > @@ -2395,7 +2397,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, > > } > > finish_unmap: > > folio_remove_rmap_ptes(folio, page, nr_pages, vma); > > - if (vma->vm_flags & VM_LOCKED) > > + if (vma_test_any_mask(vma, VMA_LOCKED_MASK)) > > mlock_drain_local(); > > folio_put_refs(folio, nr_pages); > > > > @@ -2770,7 +2772,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, > > hugetlb_remove_rmap(folio); > > else > > folio_remove_rmap_pte(folio, subpage, vma); > > - if (vma->vm_flags & VM_LOCKED) > > + if (vma_test_any_mask(vma, VMA_LOCKED_MASK)) > > mlock_drain_local(); > > folio_put(folio); > > } > > > > The mlock.c changes are a bit over my head (mostly the old VM_IO handling ...), > but it looks good from what I can tell (hoping Hugh could review ...) If it helps, Lance tested thoroughly and found 1 place I forgot to update, now addressed, and Zi has carefully gone through too. Essentially it's doing the same thing VMA_IO_BIT did only with VMA_LOCKONFAULT_BIT, which is still pretty gross but much much less so and makes it all make a lot more sense (esp. with the locked mask). (VMA_LOCKONFAULT_BIT when used for lock-on-fault is only ever set with VMA_LOCKED_BIT set too, we track the folio operation by it being set alone so it doesn't trigger lock-on-fault behaviour). > > Acked-by: David Hildenbrand (Arm) Thanks! > > -- > Cheers, > > David -- Cheers, Lorenzo