Re: [RFC PATCH v1 57/57] arm64: Enable boot-time page size selection

Zi Yan
2024-10-16T14:37:52Z

[-- Attachment #1: Type: text/plain, Size: 5026 bytes --]

On 16 Oct 2024, at 10:31, Ryan Roberts wrote:

> On 16/10/2024 15:21, Zi Yan wrote:
>> On 16 Oct 2024, at 4:14, Ryan Roberts wrote:
>>
>>> On 15/10/2024 18:42, Zi Yan wrote:
>>>> On 14 Oct 2024, at 6:59, Ryan Roberts wrote:
>>>>
>>>>> Introduce a new Kconfig, ARM64_BOOT_TIME_PAGE_SIZE, which can be
>>>>> selected instead of a page size. When selected, the resulting kernel's
>>>>> page size can be configured at boot via the command line.
>>>>>
>>>>> For now, boot-time page size kernels are limited to 48-bit VA, since
>>>>> more work is required to support LPA2. Additionally MMAP_RND_BITS and
>>>>> SECTION_SIZE_BITS are configured for the worst case (64K pages). Future
>>>>> work could be implemented to be able to configure these at boot time for
>>>>> optimial page size-specific values.
>>>>>
>>>>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
>>>>> ---
>>>>
>>>> <snip>
>>>>
>>>>>
>>>>> @@ -1588,9 +1601,10 @@ config XEN
>>>>> # 4K | 27 | 12 | 15 | 10 |
>>>>> # 16K | 27 | 14 | 13 | 11 |
>>>>> # 64K | 29 | 16 | 13 | 13 |
>>>>> +# BOOT| 29 | 16 (max) | 13 | 13 |
>>>>> config ARCH_FORCE_MAX_ORDER
>>>>> int
>>>>> - default "13" if ARM64_64K_PAGES
>>>>> + default "13" if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE
>>>>> default "11" if ARM64_16K_PAGES
>>>>> default "10"
>>>>> help
>>>>
>>>> So boot-time page size kernel always has the highest MAX_PAGE_ORDER, which
>>>> means the section size increases for 4KB and 16KB page sizes. Any downside
>>>> for this?
>>>
>>> I guess there is some cost to the buddy when MAX_PAGE_ORDER is larger than it
>>> needs to be - I expect you can explain those details much better than I can. I'm
>>> just setting it to the worst case for now as it was the easiest solution for the
>>> initial series.
>>
>> From my past experience (around 5.19), the perf impact (using vm-scalability)
>> seems very small due to MAX_PAGE_ORDER increases [1] (I made MAX_PAGE_ORDER
>> a boot time variable and increased it to 20 for my 1GB THP experiments).
>>
>> Larger MAX_PAGE_ORDER means larger section size and larger mem_block size,
>> so the granularity of memory hotplug also increases. In this case:
>> 1. ARM64 4KB: mem_block size increases from 4MB to 32MB,
>> 2. ARM64 16KB: mem_block size increases from 32MB to 128MB,
>> 3. ARM64 64KB: mem_block size keeps the same, 512MB.
>>
>> DavidH was concerned about large mem_block size before. He might have some
>> opinion on this.
>>
>>
>>>
>>>>
>>>> Is there any plan (not in this patchset) to support boot-time MAX_PAGE_ORDER
>>>> to keep section size the same?
>>>
>>> Yes absolutely. I should have documented MAX_PAGE_ORDER in the commit log along
>>> with the comments for MMAP_RND_BITS and SECTION_SIZE_BITS - that was an
>>> oversight and I'll fix it in the next version. I plan to look at making all 3
>>> values boot-time configurable in future (although I have no idea at this point
>>> how involved that will be).
>>
>> In [1], I tried to make MAX_PAGE_ORDER a boot time variable,
>> but for a different purpose, allocating 1GB THP. I needed some additional
>> changes in my patchset, since I assumed MAX_PAGE_ORDER can go beyond
>> section size, which makes things a little bit complicated. For your case,
>> I assume you are not planning to make MAX_PAGE_ORDER bigger than section
>> size, then I should be able to revive my patchset with fewer changes.
>
> Yes correct; no need to make it bigger than section size. Thanks for the patch,
> I'll certainly use it as a base when I get there or if you're interested in
> doing it then even better ;-)
>
> But I don't think this is urgent. For now, boot-time page size is a new Kconfig
> for arm64. It still supports the compile-time page size options. So having a
> larger MAX_PAGE_ORDER than strictly necessary doesn't represent a regression,
> just a limitation of boot-time page size config - something we can optimize later.

Sure. I will revisit my boot time MAX_PAGE_ORDER patchset when this patchset
settles. Glad to help. :)

>
>>
>> In terms of SECTION_SIZE_BITS, why do you want to make it a boot time variable?
>> Since it decides the minimum memory hotplug size, I assume we should keep
>> it unchanged or as small as possible to make virtual machine memory usage
>> efficient.
>
> When I say "boot-time variable" I just mean something that the arch can
> configure at boot based on the selected page size. I'm not proposing to allow
> the user to set it via the command line. That means we need to rid the code of
> any assumptions that it is compile time constant (e.g. c preprocessor usage of
> the value, etc). The same goes for MAX_PAGE_ORDER and the MMAP_RND_BITS stuff.
>

Got it.

>> [1] https://lore.kernel.org/linux-mm/20220811231643.1012912-1-zi.yan@sent.com/

Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]