[LSF/MM/BFP TOPIC] Deprecate SPARSEMEM and have only SPARSEMEM_VMEMMAP

[Oscar Salvador <osalvador@suse.de>]

Hi all,

I would like to discuss the following topic in the LSFMM.

We have SPARSEMEM memory model and SPARSEMEM_VMEMMAP, where the only difference
between the two of them is that the latter allocates a virtual chunk to represent
the memmap array, which speeds operations like
page_to_pfn/pfn_to_page/nth_page/folio_page_idx.

page->flags layout would also be changed.

Currently we have (extracted from include/linux/page-flags-layout.h):

/*
 * page->flags layout:
 * No sparsemem or sparsemem vmemmap: |       NODE     | ZONE |
 * ... | FLAGS |
 *      " plus space for last_cpupid: |       NODE     | ZONE |
 *      LAST_CPUPID ... | FLAGS |
 * classic sparse with space for node:| SECTION | NODE | ZONE |
 * ... | FLAGS |
 *      " plus space for last_cpupid: | SECTION | NODE | ZONE |
 *      LAST_CPUPID ... | FLAGS |
 * classic sparse no space for node:  | SECTION |     ZONE    | ...
 * | FLAGS |
 */

The last three could be gone.

Also, by getting rid of SPARSEMEM we would also get rid of a non-trivial amount
of code.

I did some research on which arches use CONFIG_SPARSE_MEMMAP/VMEMMAP or
none(using flatmem?).
                   
 SPARSE_MEMMAP       SPARSE_VMEMMAP
 arc
 arm              X
 arm64            X                     X
 csky
 hexagon 
 loongarch        X                     X
 m68k
 microblaze
 mips             X
 nios2
 openrisc
 parisc
 powerpc          X                     X
 riscv            X                     X
 s390             X                     X
 sh               X
 sparc            X                     X
 um
 x86              X                     X
 xtensa

arm, mips, parisc and sh operate with SPARSE_MEMMAP but are lacking code for
SPARSE_VMEMMAP.
I think these archs should be the first thing to focus on, to see if we can
make them work on SPARSE_VMEMMAP.
If we can, and when all arches can run on SPARSE_VMEMMAP, we can think of killing
SPARSE_MEMMAP.

This is not for free, of course.
There is a certain memory overhead because we have to allocate the virtual
memmap for each section.

Taking MIPS as an example:

- SECTION_SIZE_BITS 28 (256MB)
  PAGE_SIZE_4KB:  4MB per section
  PAGE_SIZE_16KB: 1MB per section
  PAGE_SIZE_64KB: 256KB per section
- SECTION_SIZE_BITS 29 (512MB)
  PAGE_SIZE_64KB: 512KB per section

e.g: When the section size is 256MB and we operate on 4KB page size, we spend
     4MB for the virtual memmap array per section.
    (65536 pages per section * sizeof(struct page)) = 4MB

Ideally, we can discuss:

1) whether it makes sense to definitely switch over to SPARSE_VMEMMAP
2) if yes, how can this be arranged. Implementing all the code and giving 
   a grace period?
3) if not, why? and what can be done so we can proceed.

Thanks

-- 
Oscar Salvador
SUSE Labs