Re: [LSF/MM/BPF TOPIC] Restricting or migrating unmovable kernel allocations from slow tier

Byungchul Park
2025-02-07T10:51:13Z

On Fri, Feb 07, 2025 at 04:54:10AM -0500, Gregory Price wrote:
> On Fri, Feb 07, 2025 at 06:34:43PM +0900, Honggyu Kim wrote:
> > On 2/7/2025 5:57 PM, Gregory Price wrote:
> >
> > > The default kernel stack size is like 16kb. You'd need like 100,000
> > > threads to eat up 1.5GB, and 2048 threads only eats like 32MB.
> > >
> > > It's not an interesting amount of memory if you have a 20TB system.
> >
> > The amount might be small, but having those data in slow tier can
> > make performance degradation if it is heavily accessed.
> >
> > The number of accesses isn't linearly corelated to the size of the
> > memory region.
> >
>
> Right, I started by saying:
>
> [CXL is] "generally not fit for kernel use"
>
> I have the opinion that CXL memory should be defaulted to ZONE_MOVABLE,
> but I understand the pressure on ZONE_NORMAL means this may not be
> possible for large capacities.

Just to clarify, for moderate capacities where ZONE_NORMAL in DRAM
covers the whole memory, it's not a big issue since the easiest solution
would be to place kernel objects in DRAM's ZONE_NORMAL and not allow
ZONE_NORMAL in non-DRAM. No objection on it.

For large capacities, kernel object migratability might be a must.

For capacities between moderate and large, kernel object migratibility
or allowing ZONE_NORMAL in non-DRAM, or something better idea, would
help to reduce ZONE_NORMAL cost.

I'm adding my opinion with the last two cases in mind.

Byungchul

> I don't think the solution is to make kernel memory migratable and allow
> kernel allocations on CXL.
>
> There's a reason most kernel allocations are not swappable.
>
> > Thanks,
> > Honggyu