Re: [RFC PATCH v1 57/57] arm64: Enable boot-time page size selection

Zi Yan
2024-10-16T14:23:08Z

[-- Attachment #1: Type: text/plain, Size: 3722 bytes --]

On 16 Oct 2024, at 4:14, Ryan Roberts wrote:

> On 15/10/2024 18:42, Zi Yan wrote:
>> On 14 Oct 2024, at 6:59, Ryan Roberts wrote:
>>
>>> Introduce a new Kconfig, ARM64_BOOT_TIME_PAGE_SIZE, which can be
>>> selected instead of a page size. When selected, the resulting kernel's
>>> page size can be configured at boot via the command line.
>>>
>>> For now, boot-time page size kernels are limited to 48-bit VA, since
>>> more work is required to support LPA2. Additionally MMAP_RND_BITS and
>>> SECTION_SIZE_BITS are configured for the worst case (64K pages). Future
>>> work could be implemented to be able to configure these at boot time for
>>> optimial page size-specific values.
>>>
>>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
>>> ---
>>
>> <snip>
>>
>>>
>>> @@ -1588,9 +1601,10 @@ config XEN
>>> # 4K | 27 | 12 | 15 | 10 |
>>> # 16K | 27 | 14 | 13 | 11 |
>>> # 64K | 29 | 16 | 13 | 13 |
>>> +# BOOT| 29 | 16 (max) | 13 | 13 |
>>> config ARCH_FORCE_MAX_ORDER
>>> int
>>> - default "13" if ARM64_64K_PAGES
>>> + default "13" if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE
>>> default "11" if ARM64_16K_PAGES
>>> default "10"
>>> help
>>
>> So boot-time page size kernel always has the highest MAX_PAGE_ORDER, which
>> means the section size increases for 4KB and 16KB page sizes. Any downside
>> for this?
>
> I guess there is some cost to the buddy when MAX_PAGE_ORDER is larger than it
> needs to be - I expect you can explain those details much better than I can. I'm
> just setting it to the worst case for now as it was the easiest solution for the
> initial series.

From my past experience (around 5.19), the perf impact (using vm-scalability)
seems very small due to MAX_PAGE_ORDER increases [1] (I made MAX_PAGE_ORDER
a boot time variable and increased it to 20 for my 1GB THP experiments).

Larger MAX_PAGE_ORDER means larger section size and larger mem_block size,
so the granularity of memory hotplug also increases. In this case:
1. ARM64 4KB: mem_block size increases from 4MB to 32MB,
2. ARM64 16KB: mem_block size increases from 32MB to 128MB,
3. ARM64 64KB: mem_block size keeps the same, 512MB.

DavidH was concerned about large mem_block size before. He might have some
opinion on this.

>
>>
>> Is there any plan (not in this patchset) to support boot-time MAX_PAGE_ORDER
>> to keep section size the same?
>
> Yes absolutely. I should have documented MAX_PAGE_ORDER in the commit log along
> with the comments for MMAP_RND_BITS and SECTION_SIZE_BITS - that was an
> oversight and I'll fix it in the next version. I plan to look at making all 3
> values boot-time configurable in future (although I have no idea at this point
> how involved that will be).

In [1], I tried to make MAX_PAGE_ORDER a boot time variable,
but for a different purpose, allocating 1GB THP. I needed some additional
changes in my patchset, since I assumed MAX_PAGE_ORDER can go beyond
section size, which makes things a little bit complicated. For your case,
I assume you are not planning to make MAX_PAGE_ORDER bigger than section
size, then I should be able to revive my patchset with fewer changes.

In terms of SECTION_SIZE_BITS, why do you want to make it a boot time variable?
Since it decides the minimum memory hotplug size, I assume we should keep
it unchanged or as small as possible to make virtual machine memory usage
efficient.

[1] https://lore.kernel.org/linux-mm/20220811231643.1012912-1-zi.yan@sent.com/

Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]