[PATCH v1 0/9] RAM_NORESERVE, MAP_NORESERVE and hostmem "reserve" property

[David Hildenbrand <david@redhat.com>]

Some cleanups previously sent in other context (resizeable allocations),
followed by RAM_NORESERVE, implementing it under POSIX using MAP_NORESERVE,
and letting users configure it for memory backens using the "reserve"
property (default: true).

MAP_NORESERVE under Linux has in the context of QEMU an effect on
1) Private anonymous memory
-> memory-backend-ram,id=mem0,size=10G
2) Private file-based mappings
-> memory-backend-file,id=mem0,size=10G,mem-path=/dev/shm/0
3) Private/shared hugetlbfs memory
-> memory-backend-memfd,id=mem0,size=10G,hugetlb=on,hugetlbsize=2M

With MAP_NORESERVE/"reserve=off", we won't be reserving swap space (1/2) or
huge pages (3) for the whole memory region.

The target use case is virtio-mem, which dynamically exposes memory
inside a large, sparse memory area to the VM. MAP_NORESERVE tells the OS
"this mapping might be very sparse". This essentially allows
avoiding to set "/proc/sys/vm/overcommit_memory == 0") when using
virtio-mem and also supporting hugetlbfs in the future.

virtio-mem currently only supports anonymous memory, in the future we want
to also support shared file-based and shared hugetlbfs mappings. We most
probably won't be supporting private mappings as they can end up behaving
very weird when it comes to memory consumption.

Future work for virtio-mem I am currently working on includes
1. Introducing a prealloc option for virtio-mem (e.g., using fallocate()
   when plugging blocks) to fail nicely when running out of
   backing storage like huge pages.
2. Supporting resizable RAM block/memoryr egions, such that we won't always
   expose a large, sparse memory region to the VM.
3. Protecting unplugged memory e.g., using userfaultfd.
4. (resizeable allocations / optimized mmap handling when resizing RAM
    blocks)

Cc: Peter Xu <peterx@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Liam Merwick <liam.merwick@oracle.com>

David Hildenbrand (9):
  softmmu/physmem: drop "shared" parameter from ram_block_add()
  util/mmap-alloc: factor out calculation of the pagesize for the guard
    page
  util/mmap-alloc: factor out reserving of a memory region to
    mmap_reserve()
  util/mmap-alloc: factor out activating of memory to mmap_activate()
  softmmu/memory: pass ram_flags into qemu_ram_alloc_from_fd()
  softmmu/memory: pass ram_flags into
    memory_region_init_ram_shared_nomigrate()
  memory: introduce RAM_NORESERVE and wire it up in qemu_ram_mmap()
  util/mmap-alloc: support RAM_NORESERVE via MAP_NORESERVE
  hostmem: wire up RAM_NORESERVE via "reserve" property

 backends/hostmem-file.c                       |  11 +-
 backends/hostmem-memfd.c                      |   8 +-
 backends/hostmem-ram.c                        |   7 +-
 backends/hostmem.c                            |  33 +++
 hw/m68k/next-cube.c                           |   4 +-
 hw/misc/ivshmem.c                             |   4 +-
 include/exec/cpu-common.h                     |   1 +
 include/exec/memory.h                         |  43 ++--
 include/exec/ram_addr.h                       |   9 +-
 include/qemu/mmap-alloc.h                     |   4 +-
 include/qemu/osdep.h                          |   3 +-
 include/sysemu/hostmem.h                      |   2 +-
 include/sysemu/kvm.h                          |   3 +-
 migration/ram.c                               |   3 +-
 .../memory-region-housekeeping.cocci          |   8 +-
 softmmu/memory.c                              |  27 ++-
 softmmu/physmem.c                             |  48 +++--
 target/s390x/kvm.c                            |   6 +-
 util/mmap-alloc.c                             | 188 ++++++++++++------
 util/oslib-posix.c                            |   5 +-
 util/oslib-win32.c                            |  13 +-
 21 files changed, 281 insertions(+), 149 deletions(-)

-- 
2.29.2