[PATCH stable 6.1, 5.15, 5.10 0/3] mm: backport mmap_min_addr handling fix to vm_unmapped_area()

[Florian Fainelli <florian.fainelli@broadcom.com>]

To: stable@vger.kernel.org
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>, Linus Torvalds <torvalds@linux-foundation.org>, Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org

This series backports

  6b008640db73 ("mm: move 'mmap_min_addr' logic from callers into vm_unmapped_area()")

to the 6.1, 5.15 and 5.10 stable trees. It is already part of v6.4-rc1
and therefore present in 6.6.y and later. The three patches are
independent of each other, one per tree, and are meant to be applied to
the respective stable branch only.

Problem
=======

An unprivileged process can get EPERM from mmap(MAP_ANONYMOUS) although
plenty of address space is still available. This happens when
vm.mmap_min_addr is larger than PAGE_SIZE (the usual case, distributions
and embedded systems set 32 KiB or 64 KiB):

  - with the top-down layout, the search starts at mm->mmap_base and
    walks down towards low_limit;
  - several architectures pass a low_limit below mmap_min_addr to
    vm_unmapped_area() (PAGE_SIZE, FIRST_USER_ADDRESS, ...), so the
    search can return an address in [low_limit, mmap_min_addr);
  - get_unmapped_area() gets that address back from the architecture
    code, but security_mmap_addr() then refuses it, and the caller sees
    -EPERM.

The right outcome is to treat the range below mmap_min_addr as not
available, so that the top-down search fails with -ENOMEM and the caller
falls back to the bottom-up search above mmap_base, as the generic code
already does.

Commit 2afc745f3e30 ("mm: ensure get_unmapped_area() returns higher
address than mmap_min_addr") fixed this only for the generic
arch_get_unmapped_area_topdown(). Architectures with their own
implementation were left alone. 6b008640db73 moves the clamp into
vm_unmapped_area() itself, which fixes all callers at once.

Affected code, determined by reading the 6.1, 5.15 and 5.10 trees and
confirmed by running the kernels below:

  - x86-64 (arch/x86/kernel/sys_x86_64.c), including 32-bit processes
    on a 64-bit kernel, and the x86 hugetlb search
    (arch/x86/mm/hugetlbpage.c)
  - arm (arch/arm/mm/mmap.c)
  - mips (arch/mips/mm/mmap.c)
  - sparc64 (arch/sparc/kernel/sys_sparc_64.c and
    arch/sparc/mm/hugetlbpage.c); only 32-bit tasks use the top-down
    layout there
  - by code inspection only, not run: parisc, sh and, in 6.1, loongarch

arm64, s390 and powerpc already clamp to mmap_min_addr in their own or in
the generic code, so their behaviour does not change; they were run as
regression checks (see below). 32-bit x86 kernels use the generic code,
which already clamps; that was checked by reading the code only.

Backport notes
==============

  6.1.y   The patch applies cleanly.

  5.15.y  Same as 5.10.y, see below.
  5.10.y  These trees predate the maple tree, so the mm/mmap.c hunks do
          not apply. unmapped_area() and unmapped_area_topdown() are the
          rbtree based versions; the lower search bound is clamped
          there with a local min_addr = max(info->low_limit,
          mmap_min_addr), which is then used in place of
          info->low_limit (including the early "low_limit > high_limit"
          check, the final clipping of the gap and the VM_BUG_ON()). The
          arch/s390 and fs/hugetlbfs hunks apply as in the original
          commit. This is noted in the commit message of each patch.

Reproducer
==========

Run it as an unprivileged user, with vm.mmap_min_addr set to something
larger than PAGE_SIZE (for example 65536). It exits with status 1 and
prints "Operation not permitted" on an affected kernel, and exits with
status 0 and prints "Cannot allocate memory" on a fixed one.

  - 32-bit process:                      ./repro
  - 64-bit process:                      ./repro 1099511627776
    (starting with 1 TiB chunks makes the 47/48 bit address space run out
    after a few hundred mappings instead of billions)

--- mmap-minaddr-repro.c ---
// SPDX-License-Identifier: GPL-2.0
/*
 * Map anonymous memory until mmap() fails and report why.
 *
 * Run as an unprivileged user (root has CAP_SYS_RAWIO, which bypasses
 * vm.mmap_min_addr) with vm.mmap_min_addr larger than PAGE_SIZE.
 *
 *   32-bit process:  ./repro                 (64 KiB first, halved on ENOMEM)
 *   64-bit process:  ./repro 1099511627776   (start with 1 TiB so that the
 *                                             address space is used up quickly)
 *
 * A kernel that handles vm.mmap_min_addr correctly ends with ENOMEM once the
 * address space is exhausted.  An affected kernel ends with EPERM while there
 * is still free address space above mmap_base.
 */
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>

int main(int argc, char **argv)
{
	size_t size = argc > 1 ? strtoull(argv[1], NULL, 0) : 65536;
	unsigned long count = 0;

	for (;;) {
		void *p = mmap(NULL, size, PROT_READ | PROT_WRITE,
			       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

		if (p != MAP_FAILED) {
			count++;
			continue;
		}
		if (errno == ENOMEM && size > 4096) {
			size /= 2;
			continue;
		}
		printf("mmap(%zu) failed after %lu mappings: %s (errno %d)\n",
		       size, count, strerror(errno), errno);
		return errno == EPERM;	/* exit status 1: bug present */
	}
}
--- end ---

Example output with vm.mmap_min_addr=65536, 6.1.y, x86-64 kernel:

  without the patch, 32-bit process:
    mmap(65536) failed after 63458 mappings: Operation not permitted (errno 1)
  without the patch, 64-bit process (./repro 1099511627776):
    mmap(32768) failed after 172 mappings: Operation not permitted (errno 1)
  with the patch, 32-bit process:
    mmap(4096) failed after 65513 mappings: Cannot allocate memory (errno 12)
  with the patch, 64-bit process:
    mmap(4096) failed after 184 mappings: Cannot allocate memory (errno 12)

The hugetlb paths can be exercised with a variant of the same loop: use
MAP_HUGETLB | MAP_NORESERVE (no huge pages need to be reserved), keep the
size a multiple of the huge page size and stop halving at the huge page
size, and set vm.mmap_min_addr (as root, before dropping privileges)
above the huge page size, for example to 256 MiB. Huge pages are
aligned, so with a smaller mmap_min_addr the search can never land below
it and the problem is not reachable.

Testing
=======

Every tree was built twice per architecture, without and with its patch,
and booted under QEMU (TCG) with a minimal initramfs. A process running
as uid 1000 was run three times per configuration (once on s390, where
a run takes about 17 million mmap() calls) with vm.mmap_min_addr=65536,
with 1 MiB on powerpc64 (64 KiB pages) and with 256 MiB for the hugetlb
runs. The results were identical for 6.1.y, 5.15.y and 5.10.y.

  kernel / process                           without patch   with patch
  ------------------------------------------------------------------------
  x86_64, 32-bit and 64-bit process          EPERM           ENOMEM
  x86_64, hugetlb, 64-bit process            EPERM           ENOMEM
  arm (multi_v7), 32-bit process             EPERM           ENOMEM
  mips (malta, big endian), 32-bit process   EPERM           ENOMEM
  sparc64, 32-bit process                    EPERM           ENOMEM
  sparc64, hugetlb, 32-bit process           EPERM           ENOMEM

  arm64, 64-bit and 32-bit (compat) process  ENOMEM          ENOMEM
  arm64, hugetlb (generic hugetlbfs path)    ENOMEM          ENOMEM
  s390                                       ENOMEM          ENOMEM
  powerpc64 hash (POWER8) and radix (POWER9) ENOMEM          ENOMEM
  powerpc64 radix, hugetlb                   ENOMEM          ENOMEM
  powerpc32                                  ENOMEM          ENOMEM

The arch/s390 files touched by the patch were compiled and linked on all
three trees.

Not covered: parisc, sh and loongarch were not run at all. hugetlb on
s390 and on powerpc64 with the hash MMU could not be run, because QEMU
does not provide a usable default huge page size for them. The
generic hugetlbfs hunk was exercised through arm64 only (where the
behaviour is unchanged, as that path already clamped locally).

 6.1.y:   4 files changed, 16 insertions(+), 9 deletions(-)
 5.15.y:  4 files changed, 13 insertions(+), 11 deletions(-)
 5.10.y:  4 files changed, 13 insertions(+), 11 deletions(-)