Re: [PATCH v2 10/13] arch, mm: set high_memory in free_area_init()

[Mike Rapoport <rppt@kernel.org>]

Hi Pratyush,

On Fri, May 16, 2025 at 05:28:17PM +0200, Pratyush Yadav wrote:
> Hi Mike, Andrew,
> 
> On Thu, Mar 13 2025, Mike Rapoport wrote:
> 
> > From: "Mike Rapoport (Microsoft)" <rppt@kernel.org>
> >
> > high_memory defines upper bound on the directly mapped memory.
> > This bound is defined by the beginning of ZONE_HIGHMEM when a system has
> > high memory and by the end of memory otherwise.
> >
> > All this is known to generic memory management initialization code that
> > can set high_memory while initializing core mm structures.
> >
> > Add a generic calculation of high_memory to free_area_init() and remove
> > per-architecture calculation except for the architectures that set and
> > use high_memory earlier than that.
> >
> > Acked-by: Dave Hansen <dave.hansen@linux.intel.com>	# x86
> > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> > ---
> >  arch/alpha/mm/init.c         |  1 -
> >  arch/arc/mm/init.c           |  2 --
> >  arch/arm64/mm/init.c         |  2 --
> >  arch/csky/mm/init.c          |  1 -
> >  arch/hexagon/mm/init.c       |  6 ------
> >  arch/loongarch/kernel/numa.c |  1 -
> >  arch/loongarch/mm/init.c     |  2 --
> >  arch/microblaze/mm/init.c    |  2 --
> >  arch/mips/mm/init.c          |  2 --
> >  arch/nios2/mm/init.c         |  6 ------
> >  arch/openrisc/mm/init.c      |  2 --
> >  arch/parisc/mm/init.c        |  1 -
> >  arch/riscv/mm/init.c         |  1 -
> >  arch/s390/mm/init.c          |  2 --
> >  arch/sh/mm/init.c            |  7 -------
> >  arch/sparc/mm/init_32.c      |  1 -
> >  arch/sparc/mm/init_64.c      |  2 --
> >  arch/um/kernel/um_arch.c     |  1 -
> >  arch/x86/kernel/setup.c      |  2 --
> >  arch/x86/mm/init_32.c        |  3 ---
> >  arch/x86/mm/numa_32.c        |  3 ---
> >  arch/xtensa/mm/init.c        |  2 --
> >  mm/memory.c                  |  8 --------
> >  mm/mm_init.c                 | 30 ++++++++++++++++++++++++++++++
> >  mm/nommu.c                   |  2 --
> >  25 files changed, 30 insertions(+), 62 deletions(-)
> 
> This patch causes a BUG() when built with CONFIG_DEBUG_VIRTUAL and
> passing in the cma= commandline parameter:
> 
>     ------------[ cut here ]------------
>     kernel BUG at arch/x86/mm/physaddr.c:23!
>     ception 0x06 IP 10:ffffffff812ebbf8 error 0 cr2 0xffff88903ffff000
>     CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc6+ #231 PREEMPT(undef)
>     Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014
>     RIP: 0010:__phys_addr+0x58/0x60
>     Code: 01 48 89 c2 48 d3 ea 48 85 d2 75 05 e9 91 52 cf 00 0f 0b 48 3d ff ff ff 1f 77 0f 48 8b 05 20 54 55 01 48 01 d0 e9 78 52 cf 00 <0f> 0b 90 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
>     RSP: 0000:ffffffff82803dd8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000
>     RAX: 000000007fffffff RBX: 00000000ffffffff RCX: 0000000000000000
>     RDX: 000000007fffffff RSI: 0000000280000000 RDI: ffffffffffffffff
>     RBP: ffffffff82803e68 R08: 0000000000000000 R09: 0000000000000000
>     R10: ffffffff83153180 R11: ffffffff82803e48 R12: ffffffff83c9aed0
>     R13: 0000000000000000 R14: 0000001040000000 R15: 0000000000000000
>     FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
>     CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>     CR2: ffff88903ffff000 CR3: 0000000002838000 CR4: 00000000000000b0
>     Call Trace:
>      <TASK>
>      ? __cma_declare_contiguous_nid+0x6e/0x340
>      ? cma_declare_contiguous_nid+0x33/0x70
>      ? dma_contiguous_reserve_area+0x2f/0x70
>      ? setup_arch+0x6f1/0x870
>      ? start_kernel+0x52/0x4b0
>      ? x86_64_start_reservations+0x29/0x30
>      ? x86_64_start_kernel+0x7c/0x80
>      ? common_startup_64+0x13e/0x141
> 
> The reason is that __cma_declare_contiguous_nid() does:
> 
>     	highmem_start = __pa(high_memory - 1) + 1;
> 
> If dma_contiguous_reserve_area() (or any other CMA declaration) is
> called before free_area_init(), high_memory is uninitialized. Without
> CONFIG_DEBUG_VIRTUAL, it will likely work but use the wrong value for
> highmem_start.
> 
> Among the architectures this patch touches, the below call
> dma_contiguous_reserve_area() _before_ free_area_init():
> 
> - x86
> - s390
> - mips
> - riscv
> - xtensa
> - loongarch
> - csky

For most of those this patch didn't really change anything because they
initialized high_memory in mem_init() which is a part of free_area_init().
In those cases cma just did

	highmem_start = __pa(-1) + 1;

and everyone was happy :)
 
> The below call it _after_ free_area_init():
> - arm64
> 
> And the below don't call it at all:
> - sparc
> - nios2
> - openrisc
> - hexagon
> - sh
> - um
> - alpha
> 
> One possible fix would be to move the calls to
> dma_contiguous_reserve_area() after free_area_init(). On x86, it would
> look like the diff below. The obvious downside is that moving the call
> later increases the chances of allocation failure. I'm not sure how much
> that actually matters, but at least on x86, that means crash kernel and
> hugetlb reservations go before DMA reservation. Also, adding a patch
> like that at rc7 is a bit risky.

I don't think there's a risk of allocation failure, but moving things
around in setup_arch() is always risky :)
 
> The other option would be to revert this. I tried a revert, but it isn't
> trivial. It runs into merge conflicts in pretty much all of the arch
> files. Maybe reverting patches 11, 12, and 13 as well would make it
> easier but I didn't try that.

What I think we can do is to add this to mm/cma.c (not even compile tested)

diff --git a/mm/cma.c b/mm/cma.c
index 15632939f20a..c04be488b099 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -608,7 +608,10 @@ static int __init __cma_declare_contiguous_nid(phys_addr_t *basep,
 	 * complain. Find the boundary by adding one to the last valid
 	 * address.
 	 */
-	highmem_start = __pa(high_memory - 1) + 1;
+	if (IS_ENABLED(CONFIG_HIGHMEM))
+		highmem_start = __pa(high_memory - 1) + 1;
+	else
+		highmem_start = memblock_end_of_DRAM();
 	pr_debug("%s(size %pa, base %pa, limit %pa alignment %pa)\n",
 		__func__, &size, &base, &limit, &alignment);
 
so that highmem_start in __cma_declare_contiguous_nid() will be always
correct for !HIGHMEM configs and then restore setting of highmem_start in
mips::paging_init() as mips is the only architecture that actually set
high_memory before free_area_init() before this patch.

(for 32 bit configs of x86 there alrady a fixup d893aca973c3 ("x86/mm: restore
early initialization of high_memory for 32-bits"))

-- 
Sincerely yours,
Mike.