[PATCH v4 00/25] mm/vma: convert vm_flags_t to vma_flags_t in vma code

Lorenzo Stoakes (Oracle)
2026-03-20T19:38:52Z

This series converts a lot of the existing use of the legacy vm_flags_t
data type to the new vma_flags_t type which replaces it.

In order to do so it adds a number of additional helpers:

* vma_flags_empty() - Determines whether a vma_flags_t value has no bits
set.

* vma_flags_and() - Performs a bitwise AND between two vma_flags_t values.

* vma_flags_diff_pair() - Determines which flags are not shared between a
pair of VMA flags (typically non-constant values)

* append_vma_flags() - Similar to mk_vma_flags(), but allows a vma_flags_t
value to be specified (typically a constant value) which will be copied
and appended to to create a new vma_flags_t value, with additional flags
specified to append to it.

* vma_flags_same() - Determines if a vma_flags_t value is exactly equal to
a set of VMA flags.

* vma_flags_same_mask() - Determines if a vma_flags_t value is eactly equal
to another vma_flags_t value (typically constant).

* vma_flags_same_pair() - Determines if a pair of vma_flags_t values are
exactly equal to one another (typically both non-constant).

* vma_flags_to_legacy() - Converts a vma_flags_t value to a vm_flags_t
value, used to enable more iterative introduction of the use of
vma_flags_t.

* legacy_to_vma_flags() - Converts a vm_flags_t value to a vma_flags-t
value, for the same purpose.

* vma_flags_test_single_mask() - Tests whether a vma_flags_t value contain
the single flag specified in an input vma_flags_t flag mask, or if that
flag mask is empty, is defined to return false. Useful for
config-predicated VMA flag mask defines.

* vma_test() - Tests whether a VMA's flags contain a specific singular VMA
flag.

* vma_test_any() - Tests whether a VMA's flags contain any of a set of VMA
flags.

* vma_test_any_mask() - Tests whether a VMA's flags contain any of the
flags specified in another, typically constant, vma_flags_t value.

* vma_test_single_mask() - Tests whether a VMA's flags contain the single
flag specified in an input vma_flags_t flag mask, or if that flag mask is
empty, is defined to return false. Useful for config-predicated VMA flag
mask defines.

* vma_clear_flags() - Clears a specific set of VMA flags from a vma_flags_t
value.

* vma_clear_flags_mask() - Clears those flag set in a vma_flags_t value
(typically constant) from a (typically not constant) vma_flags_t value.

The series mostly focuses on the the VMA specific code, especially that
contained in mm/vma.c and mm/vma.h.

It updates both brk() and mmap() logic to utils vma_flags_t values as much
as is practiaclly possible at this point, changing surrounding logic to be
able to do so.

It also updates the vma_modify_xxx() functions where they interact with VMA
flags directly to use vm_flags_t values where possible.

There is extensive testing added in the VMA userland tests to assert that
all of these new VMA flag functions work correctly.

v4:
* Propagated tags, thanks Vlasta!
* Removed superfluous parens around vma_test_any_mask() as per Vlasta.
* Converted masked functions into more understandable equivalent form as
per Vlasta in 24/25.
* Redefined VM_SPECIAL using vma_flags_to_legacy() as per Vlasta.
* Fixed whitespace as per Vlasta.
* Added vma_flags_reset_once() as per Vlasta.
* Expanded 22/23 commit message to describe why I'm replacing things as
per Vlasta.
* Added bitmap_copy() to test headers in order to implement
vma_flags_reset_once().

v3:
* Folded in tags, thanks Paul, Vlastimil!
* Respun to apply the correct suggestions/reports from
https://sashiko.dev/#/patchset/cover.1773665966.git.ljs%40kernel.org
(note there is also quite a lot of noise, that is ignored).
* Const-ified vma_flags_t * param for vma_flags_empty() as per Sashiko.
* Obtained sticky flag masks after VMA write lock acquired on merge and
update vma_expand() similarly. This is meaningful, as the maybe guard
flag is set atomically which might race outside of a VMA write lock, as
per Sashiko.
* Dropped comment about 'VM_NONE convenience' from 5/23 commit message as
the VMA_xxx form flags can now provide that.
* Updated legacy_to_vma_flags() to ensure upper bits are cleared for
NUM_VMA_FLAG_BITS > 64 as per Sashiko.
* Updated legacy_to_vma_flags() to use vma_flags_overwrite_word() for
consistency.
* Refreshed vma_flags_overwrite_word(), vma_flag_overwrite_word_once(),
vma_flags_set_word() and vma_flags_clear_word() in the VMA test dup.h
header to keep them consistent with the kernel.
* Updated VMA_DATA_DEFAULT_FLAGS declaration on arm64 to be predicated on
CONFIG_ARM64_MTE as to whether VMA_MTE_ALLOWED_BIT is set, as per
Sashiko.
* Fixed bug where the VMA did not have VMA_SOFTDIRTY_BIT set if
pgtable_supports_soft_dirty(), but rather the stack variable vma_flags,
as per Sashiko.
* Corrected vmag -> vma typo in VMA test code as per Sashiko.
* Fixed typo in 20/23 commit message 'correctly' -> 'correct' as per
Sashiko.
* Fixed VMA flag clear tests to consistently do vma_clear_flags_mask(&vma,
mask) rather than vma_flags_clear_mask(&vma.flags, mask) as per Sashiko.
* Added missing vma_start_write() in mseal_apply() as per Sahiko.
https://lore.kernel.org/all/cover.1773846935.git.ljs@kernel.org/

v2:
* Rebased on mm-unstable.
* Added vma_flags_count() and vma[_flags]_test_single_mask() for testing
whether flags have a single flag set depending on an input flag mask,
returning false if the flag mask is empty.
* Added tests for vma_flags_count() and vma[_flags]_test_single_mask().
* Updated the KSM VMA_DROPPABLE test to use vma_flags_test_single_mask().
* Updated the newly-introduced-since-rebase vma_supports_mlock() to use
vma_flags_t.
https://lore.kernel.org/linux-mm/cover.1773665966.git.ljs@kernel.org/

v1:
https://lore.kernel.org/linux-mm/cover.1773342102.git.ljs@kernel.org/

Lorenzo Stoakes (Oracle) (25):
mm/vma: add vma_flags_empty(), vma_flags_and(), vma_flags_diff_pair()
tools/testing/vma: add unit tests flag empty, diff_pair, and[_mask]
mm/vma: add further vma_flags_t unions
tools/testing/vma: convert bulk of test code to vma_flags_t
mm/vma: use new VMA flags for sticky flags logic
tools/testing/vma: fix VMA flag tests
mm/vma: add append_vma_flags() helper
tools/testing/vma: add simple test for append_vma_flags()
mm: unexport vm_brk_flags() and eliminate vm_flags parameter
mm/vma: introduce vma_flags_same[_mask/_pair]()
mm/vma: introduce [vma_flags,legacy]_to_[legacy,vma_flags]() helpers
tools/testing/vma: test that legacy flag helpers work correctly
mm/vma: introduce vma_test[_any[_mask]](), and make inlining
consistent
tools/testing/vma: update VMA flag tests to test vma_test[_any_mask]()
mm: introduce vma_flags_count() and vma[_flags]_test_single_mask()
tools/testing/vma: test vma_flags_count,vma[_flags]_test_single_mask
mm: convert do_brk_flags() to use vma_flags_t
mm: update vma_supports_mlock() to use new VMA flags
mm/vma: introduce vma_clear_flags[_mask]()
tools/testing/vma: update VMA tests to test vma_clear_flags[_mask]()
mm/vma: convert as much as we can in mm/vma.c to vma_flags_t
tools: bitmap: add missing bitmap_copy() implementation
mm/vma: convert vma_modify_flags[_uffd]() to use vma_flags_t
mm/vma: convert __mmap_region() to use vma_flags_t
mm: simplify VMA flag tests of excluded flags

arch/arc/include/asm/page.h | 2 +-
arch/arm/include/asm/page.h | 2 +-
arch/arm64/include/asm/page.h | 7 +-
arch/hexagon/include/asm/page.h | 2 +-
arch/loongarch/include/asm/page.h | 2 +-
arch/mips/include/asm/page.h | 2 +-
arch/nios2/include/asm/page.h | 2 +-
arch/powerpc/include/asm/page.h | 4 +-
arch/powerpc/include/asm/page_32.h | 2 +-
arch/powerpc/include/asm/page_64.h | 12 +-
arch/riscv/include/asm/page.h | 2 +-
arch/s390/include/asm/page.h | 2 +-
arch/x86/include/asm/page_types.h | 2 +-
arch/x86/um/asm/vm-flags.h | 4 +-
fs/binfmt_elf.c | 3 +-
include/linux/ksm.h | 10 +-
include/linux/mm.h | 352 +++++++++++++++++++++-------
include/linux/mm_types.h | 52 ++++-
include/linux/mman.h | 49 ----
include/linux/userfaultfd_k.h | 3 +
mm/internal.h | 7 +-
mm/ksm.c | 43 ++--
mm/madvise.c | 10 +-
mm/mlock.c | 38 +--
mm/mmap.c | 19 +-
mm/mprotect.c | 49 ++--
mm/mremap.c | 6 +-
mm/mseal.c | 11 +-
mm/userfaultfd.c | 21 +-
mm/vma.c | 210 ++++++++++-------
mm/vma.h | 84 ++++++-
mm/vma_exec.c | 5 +-
security/selinux/hooks.c | 4 +-
tools/include/linux/bitmap.h | 24 ++
tools/lib/bitmap.c | 10 +
tools/testing/vma/include/custom.h | 25 --
tools/testing/vma/include/dup.h | 309 ++++++++++++++++++-------
tools/testing/vma/include/stubs.h | 11 +-
tools/testing/vma/shared.c | 8 +-
tools/testing/vma/shared.h | 22 +-
tools/testing/vma/tests/merge.c | 311 +++++++++++++------------
tools/testing/vma/tests/mmap.c | 18 +-
tools/testing/vma/tests/vma.c | 359 +++++++++++++++++++++++++----
tools/testing/vma/vma_internal.h | 6 +
44 files changed, 1463 insertions(+), 663 deletions(-)

--
2.53.0