Panic when loading modules with CONFIG_ARM64_BTI_KERNEL

Panic when loading modules with CONFIG_ARM64_BTI_KERNEL

[D Scott Phillips]

Hi all,

I'm occasionally seeing panics due to branch target exceptions when
loading kernel modules when bti is enabled. It seems to happen when
__init code calls into non __init code which happens to have been loaded
a long way away, and so a module plt entry has been created.  When the
code for the module was generated, the call from init to non-init was
through a direct branch, so no `bti c` was injected in the callee.

I believe gcc needs to inject bti instructions due to those
cross-section calls regardless of direct/indirect, so I've filed:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671

for the issue, along with a little snippet that makes the same type of
issue happen in user space.

In the meantime, should we mark BTI_KERNEL as broken? or any other ideas
on how to work around code generated without bti instructions like this?
Enable the guard page attribute later? or try to keep init and non-init
allocations close together?

And then finally, here are the exact instructions that are causing me
problems most frequently, from ipmi_si.ko:

> Disassembly of section .init.text:
> 
> 0000000000000000 <init_module>:
>  ... snip ...
>  158:   94000000        bl      0 <init_module>
>                         158: R_AARCH64_CALL26   .text+0x13b4
> 
> Disassembly of section .text:
> 
> 00000000000013b4 <cleanup_ipmi_si.part.0>:
>     13b4:       d503201f        nop
>     13b8:       d503201f        nop
>     13bc:       d503233f        paciasp

Where the plt entry is injected at the call at init_module+0x158, and
then the branch target exception happens at cleanup_ipmi_si.part.0+0x0.

-- 
Scott

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Panic when loading modules with CONFIG_ARM64_BTI_KERNEL

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1167 bytes --]

On Wed, Aug 17, 2022 at 04:42:43PM -0700, D Scott Phillips wrote:

> In the meantime, should we mark BTI_KERNEL as broken? or any other ideas

The clang versions I have to hand appear fine with your userspace test
program, it emits a BTI C at the start of func with -O1 so I guess
that'd only be for GCC.  Ideally we'd be able to detect particular
configurations that would trigger this but I don't think we can.

> on how to work around code generated without bti instructions like this?
> Enable the guard page attribute later? or try to keep init and non-init
> allocations close together?

Keeping the allocations closer so we don't need veneers would mitigate
the issue, though that's not going to be something that we can rely on
so it's awkward - we can *try* to keep them closer together but I'm sure
someone will always be able to craft a case where we get pushed to using
a veneer again.

Not enabling BTI for the module until after init has run would mean that
we'd have to remap the code while things like interrupt handlers might
be running which smells like there'd be problems but I'd need to go and
refresh my memory as to what the actual issues are.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Panic when loading modules with CONFIG_ARM64_BTI_KERNEL

[D Scott Phillips]

Mark Brown <broonie@kernel.org> writes:

> On Wed, Aug 17, 2022 at 04:42:43PM -0700, D Scott Phillips wrote:
>
>> In the meantime, should we mark BTI_KERNEL as broken? or any other ideas
>
> The clang versions I have to hand appear fine with your userspace test
> program, it emits a BTI C at the start of func with -O1 so I guess
> that'd only be for GCC.  Ideally we'd be able to detect particular
> configurations that would trigger this but I don't think we can.

Good point, it looks like LLVM is specifically aware that veneers might
get added and so inserts `bti c` at the start of *all* functions IIUC:

https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/AArch64/AArch64BranchTargets.cpp#L80

> // Even in cases where a function has internal linkage and is only
> // called directly in its translation unit, it can still be called
> // indirectly if the linker decides to add a thunk to it for whatever
> // reason (say, for example, if it is finally placed far from its call
> // site and a BL is not long-range enough).

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel