Re: [PATCH v2] arm64/cpufeature: annotate arm64_use_ng_mappings with ro_after_init to prevent wrong idmap generation

[Ryan Roberts <ryan.roberts@arm.com>]

On 06/05/2025 09:53, Yeoreum Yun wrote:
> Hi Ryan,
> 
>> On 06/05/2025 09:09, Yeoreum Yun wrote:
>>> Hi Catalin,
>>>
>>>> On Sat, May 03, 2025 at 09:23:27PM +0100, Yeoreum Yun wrote:
>>>>> Hi Catalin,
>>>>>
>>>>>> On Sat, May 03, 2025 at 11:16:12AM +0100, Catalin Marinas wrote:
>>>>>>> On Fri, 02 May 2025 19:04:12 +0100, Yeoreum Yun wrote:
>>>>>>>> create_init_idmap() could be called before .bss section initialization
>>>>>>>> which is done in early_map_kernel().
>>>>>>>> Therefore, data/test_prot could be set incorrectly by PTE_MAYBE_NG macro.
>>>>>>>>
>>>>>>>> PTE_MAYBE_NG macro set NG bit according to value of "arm64_use_ng_mappings".
>>>>>>>> and this variable places in .bss section.
>>>>>>>>
>>>>>>>> [...]
>>>>>>>
>>>>>>> Applied to arm64 (for-next/fixes), with some slight tweaking of the
>>>>>>> comment, thanks!
>>>>>>>
>>>>>>> [1/1] arm64/cpufeature: annotate arm64_use_ng_mappings with ro_after_init to prevent wrong idmap generation
>>>>>>>       https://git.kernel.org/arm64/c/12657bcd1835
>>>>>>
>>>>>> I'm going to drop this for now. The kernel compiled with a clang 19.1.5
>>>>>> version I have around (Debian sid) fails to boot, gets stuck early on:
>>>>>>
>>>>>> $ clang --version
>>>>>> Debian clang version 19.1.5 (1)
>>>>>> Target: aarch64-unknown-linux-gnu
>>>>>> Thread model: posix
>>>>>> InstalledDir: /usr/lib/llvm-19/bin
>>>>>>
>>>>>> I didn't have time to investigate, disassemble etc. I'll have a look
>>>>>> next week.
>>>>>
>>>>> Just for your information.
>>>>> When I see the debian package, clang 19.1.5-1 doesn't supply anymore:
>>>>>  - https://ftp.debian.org/debian/pool/main/l/llvm-toolchain-19/
>>>>>
>>>>> and the default version for sid is below:
>>>>>
>>>>> $ clang-19 --version
>>>>> Debian clang version 19.1.7 (3)
>>>>> Target: aarch64-unknown-linux-gnu
>>>>> Thread model: posix
>>>>> InstalledDir: /usr/lib/llvm-19/bin
>>>>>
>>>>> When I tested with above version with arm64-linux's for-next/fixes
>>>>> including this patch. it works well.
>>>>
>>>> It doesn't seem to be toolchain related. It fails with gcc as well from
>>>> Debian stable but you'd need some older CPU (even if emulated, e.g.
>>>> qemu). It fails with Cortex-A72 (guest on Raspberry Pi 4) but not
>>>> Neoverse-N2. Also changing the annotation from __ro_after_init to
>>>> __read_mostly also works.
>>
>> I think this is likely because __ro_after_init is also "ro before init" - i.e.
>> if you try to write to it in the PI code an exception is generated due to it
>> being mapped RO. Looks like early_map_kernel() is writiing to it.
>>
>> I've noticed a similar problem in the past and it would be nice to fix it so
>> that PI code maps __ro_after_init RW.
>>
> 
> Personally, I don't believe this because the create_init_idmap()
> maps the the .rodata section with PAGE_KERNEL pgprot
> from __initdata_begin to _end.

But __ro_after_init is in the ".data..ro_after_init" section, which is in the
.rodata section. That's mapped PAGE_KERNEL_ROX as Ard says.

> 
> and at the mark_readonly() the pgprot is changed to PAGE_KERNEL_RO
> But, arm64_use_ng_mappings is accessed with write before mark_readonly()
> only via smp_cpus_done().
> 
> JFYI here is map information:
> 
> // mark_readlonly() changes to ro perm below ranges:
> ffff800081b30000 g       .rodata	0000000000000000 __start_rodata
> ffff800082560000 g       .rodata.text	0000000000000000 __init_begin
> 
> // create_init_idmap() maps below range with PAGE_KERNEL.
> ffff8000826d0000 g       .altinstructions	0000000000000000 __initdata_begin
> ffff800082eb0000 g       .bss	0000000000000000 _end
> 
> ffff8000824596d0 g     O .rodata	0000000000000001 arm64_use_ng_mappings
> 
> Thanks.
> 
> 
>> Thanks,
>> Ryan
>>
>>>
>>> Thanks to let me know. But still I've failed to reproduce this
>>> on Cortex-a72 and any older cpu on qeum.
>>> If you don't mind, would you share your Kconfig?
>>>
>>>> I haven't debugged it yet but I wonder whether something wants to write
>>>> this variable after it was made read-only (well, I couldn't find any by
>>>> grep'ing the code, so it needs some step-by-step debugging).
>>>>
>>> [...]
>>>
>>> Thanks!
>>>
>>> --
>>> Sincerely,
>>> Yeoreum Yun
>>
> 
> --
> Sincerely,
> Yeoreum Yun