Re: ppc64le STRICT_MODULE_RWX and livepatch apply_relocate_add() crashes

[Joe Lawrence <joe.lawrence@redhat.com>]

On 12/13/21 11:36 AM, Christophe Leroy wrote:
> 
> 
> Le 13/12/2021 à 15:47, Joe Lawrence a écrit :
>> On 12/13/21 2:42 AM, Christophe Leroy wrote:
>>>
>>> Hello Joe,
>>>
>>> I'm implementing LIVEPATCH on PPC32 and I wanted to test with
>>> STRICT_MODULE_RWX enabled so I took your branch as suggested, but I'm
>>> getting the following errors on build. What shall I do ?
>>>
>>>     CALL    scripts/checksyscalls.sh
>>>     CALL    scripts/atomic/check-atomics.sh
>>>     CHK     include/generated/compile.h
>>>     KLP     lib/livepatch/test_klp_convert1.ko
>>> klp-convert: section .rela.klp.module_relocs.test_klp_convert_mod length
>>> beyond nr_entries
>>>
>>> klp-convert: Unable to load user-provided sympos
>>> make[2]: *** [scripts/Makefile.modfinal:79:
>>> lib/livepatch/test_klp_convert1.ko] Error 255
>>>     KLP     lib/livepatch/test_klp_convert2.ko
>>> klp-convert: section .rela.klp.module_relocs.test_klp_convert_mod length
>>> beyond nr_entries
>>>
>>> klp-convert: Unable to load user-provided sympos
>>> make[2]: *** [scripts/Makefile.modfinal:79:
>>> lib/livepatch/test_klp_convert2.ko] Error 255
>>>     KLP     lib/livepatch/test_klp_convert_sections.ko
>>> klp-convert: section .rela.klp.module_relocs.test_klp_convert_mod length
>>> beyond nr_entries
>>>
>>> klp-convert: Unable to load user-provided sympos
>>> make[2]: *** [scripts/Makefile.modfinal:79:
>>> lib/livepatch/test_klp_convert_sections.ko] Error 255
>>> make[2]: Target '__modfinal' not remade because of errors.
>>> make[1]: *** [scripts/Makefile.modpost:145: __modpost] Error 2
>>> make: *** [Makefile:1770: modules] Error 2
>>>
>>
>> Hi Christophe,
>>
>> Interesting failure mode.  That's klp-convert complaining that it found
>> more relocations in a .klp.module_relocs.<objname> section than
>> expected, i.e. nr_entries = sec->size / sizeof(struct klp_module_reloc).
>>
>> A few possibilities: the ELF sec->size was incorrectly set/read by
>> build/libelf (I doubt that).  Or maybe the layout/size of struct
>> klp_module_reloc is not consistent between kernel and userspace (I'm
>> more suspicious of this).
>>
>> Can you post a copy of the build's symbols.klp and
>> lib/livepatch/test_klp_convert1.tmp.ko somewhere?  I should be able to
>> start debug with those files.
>>
> 
> I sent you both files off list.
> 
> It looks like klp-convert doesn't use the correct size. It finds a 
> struct of size 12 hence 3 entries for a section of size 40.
> 
> On PPC32 the struct has size 8 (void * is 4 and int is 4).
> 
> But I'm cross-building from x86_64 where the struct is 8 + 4 = 12.
> 
> Can it be the reason ?
> 

I'm pretty sure that is it.  I haven't had much runtime with klp-convert
and cross-building (I've only found one big/little endian bug with
x86_64->s390x) and was going to ask you how you were testing :)

Do you know if there are other kernel build tools that deal with similar
situations?  This seems like a tricky job for the userspace build tool
to determine non-native target struct layout.

In the meantime, hacking in:

 struct klp_module_reloc {
-       void *sym;
+       uint32_t sym;
        unsigned int sympos;
 } __packed;

gets me generating an output .ko file, but the readelf output doesn't
look right.

I'll add this to the patchset TODO list, but may not get to it for a
while -- is there any chance the above hack works or could you test a
local non-cross build?

Thanks,

-- 
Joe