Re: [PATCH v3 05/10] x86: remove HIGHMEM64G support

Re: [PATCH v3 05/10] x86: remove HIGHMEM64G support

[Arnd Bergmann]

On Thu, Feb 27, 2025, at 17:51, Linus Torvalds wrote:
> On Thu, 27 Feb 2025 at 07:41, H. Peter Anvin <hpa@zytor.com> wrote:
>>
>> One of the generations of kernel.org ran on a dual socket system with
>> 6 GiB RAM. It was a mess; basically it achieved less than 50% memory
>> utilization because of highmem.
>
> I'll be very very happy when HIGHMEM is gone completely, but yes,
> HIGHMEM64G was particularly painful.

I'm optimistic about being able to remove highmem in the future
as well, hopefully this series was a good step in that direction.

This is a rough list of what I think the remaining problems are:

- anyone running 32-bit kernels on 64-bit hardware or virtual
  machines should move to 64-bit kernels. This is mostly about
  educating users (on x86) and embedded system builders (on arm
  and possibly powerpc/booke64).
  One idea we had discussed in the past was to have the
  kernel refuse to run on 64-bit hardware if highmem is
  detected (with a command line override to allow regression
  testing).

- arm32 needs a way to have sparse physical memory linearized
  into lowmem, to deal with the case where the total memory is
  small enough to fit, but half of it sits at a high physical
  address. We are working on this.

- Ideally the same change should support a boot-time vmsplit
  selection on arm32 instead of compile-time CONFIG_VMSPLIT_*.
  This would also help x86-32 when a distro kernel needs to run
  on both 1GB and 2GB RAM machines but still allow
  3GB of user addresses on smaller machines.

- Machines with 3GB or 4GB of RAM (pre-2005 Intel PC/server
  systems, 2005-2007 era Intel Laptops, 2007-2009 era
  Netbooks, 2013-2015 era Arm Chromebooks, rare Arm/x86/PowerPC
  embedded systems) can hopefully get away with a combination
  of VMSPLIT_2G and zram/zswap.
  Someone needs to rework the highmem code to separate the
  bits needed for zsmalloc from the rest so we can disable
  and eventually remove normal highmem while keeping
  zsmalloc-in-highmem. I have not looked in detail at how
  that would work in practice.

- There is still an open question about implementing a
  4GB-split on ARM-LPAE (flipping a 3.75GB ttbr0 between
  lowmem and user). Linus Walleij has done some work on
  this, but we may find that VMSPLIT_2G plus zsmalloc-only
  highmem is good enough that we don't need both. Both
  approaches have important downsides.

- arm32 machines with more than 4GB of total memory need to
  get rare enough that we can throw them under the bus.
  There are still a few of them in use, but the chips are
  all more than 10 years old by now and eventually they
  will stop needing kernel updates.

- sparc32 kernels should start supporting more than 192MB of
  lowmem (or stop existing). Apparently there are Leon3
  machines with up to 2GB of RAM.

- mips32 has a hardware lowmem limit of 512MB, and I think
  we can live with that because they tend to have less actual
  RAM. There is one important chip (mt7621) that supports
  512MB RAM in machines with real users, and apparently that
  number goes down to 448MB without highmem.

- arc/csky/microblaze/xtensa all support highmem in theory,
  but I am not aware of anyone actually needing it any more.

- Lastly, there has to be some executive decision to actually
  do it as there will always be /some/ systems that are
  affected in the end. I don't think we can do enough of
  the mitigation work done for arm32 for this year's longterm
  kernel, but probably enough that we can agree on a
  deadline at the Tokyo LPC kernel summit, and maybe remove
  highmem after the 2026 longterm kernel.

I'll try to collect more data from affected users about what
will really break for them, and what I may have missed.

     Arnd