Re: KUEP broken on FSP2?

[Eddie James <eajames@linux.ibm.com>]

On 10/6/23 10:55, Christophe Leroy wrote:
> Hi,
>
> Le 06/10/2023 à 17:43, Eddie James a écrit :
>> On 10/6/23 00:21, Christophe Leroy wrote:
>>> Hi,
>>>
>>> Le 05/10/2023 à 21:06, Eddie James a écrit :
>>>> Hi,
>>>>
>>>> I'm attempting to run linux 6.1 on my FSP2, but my kernel crashes
>>>> attempting to get into userspace. The init script works, but the first
>>>> binary (mount) I run results in oops. Can anyone help me to debug this
>>>> further or suggest anything?
>>> I can't see anything in your dump suggesting that KUEP is broken, can
>>> you ?
>>>
>>> What I see is that kernel tries to execute user memory, which is wrong.
>>> And KUEP perfectly works by blocking that access. There is no call
>>> trace, suggesting that the kernel has jumped in the weed.
>>
>> Right, the function works as intended, but the fact remains that I can't
>> call anything in userspace (except init) without the kernel trying to
>> execute that memory. I saw KUEP in the commit history and it seemed
>> relevant, but I could certainly be mistaken. Can anyone think of
>> anything else that might cause this? Or how I can debug further?
>>
>>
>> I went ahead and removed the couple of lines of assembly that enabled
>> KUEP on 44x and tried again. Now I get a crash in load_elf_binary. NIP
>> is the kfree(elf_phdata) and LR is garbage, so not entirely sure where
>> it actually crashed...
> Which confirms that KUEP is not the culprit.


Right.


>
> By the way when booting a bamboo defconfig on QEMU I have to problem.


Yes FSP2 is a bit "special"...


>
> Apparently KUEP for 4xx appears in Kernel 5.14.
>
> Do you know of a kernel version that works ?
>
> Can you check 5.14 (you have to explicitely select KUEP in that version,
> it is not forced yet) ?
>
> Once you have a good version, then what about a bisect ?


Yea 5.10 works. I'll try 5.14. I was hoping to avoid a bisect as my 
build and test process for this platform is quite time consuming.


Thanks,

Eddie


>
> Christophe
>
>>
>> Thanks,
>>
>> Eddie
>>
>>
>>> Christophe
>>>
>>>> Thanks,
>>>>
>>>> Eddie
>>>>
>>>>
>>>> [    1.042743] kernel tried to execute user page (b7ee2000) - exploit
>>>> attempt? (
>>>> uid: 0)
>>>> [    1.042846] BUG: Unable to handle kernel instruction fetch
>>>> [    1.042919] Faulting instruction address: 0xb7ee2000
>>>> [    1.042986] Oops: Kernel access of bad area, sig: 11 [#1]
>>>> [    1.043059] BE PAGE_SIZE=4K FSP-2
>>>> [    1.043106] Modules linked in:
>>>> [    1.043149] CPU: 0 PID: 61 Comm: mount Not tainted
>>>> 6.1.55-d23900f.ppcnf-fsp2
>>>> #1
>>>> [    1.043249] Hardware name: ibm,fsp2 476fpe 0x7ff520c0 FSP-2
>>>> [    1.043323] NIP:  b7ee2000 LR: 8c008000 CTR: 00000000
>>>> [    1.043392] REGS: bffebd83 TRAP: 0400   Not tainted
>>>> (6.1.55-d23900f.ppcnf-fs
>>>> p2)
>>>> [    1.043491] MSR:  00000030 <IR,DR>  CR: 00001000  XER: 20000000
>>>> [    1.043579]
>>>> [    1.043579] GPR00: c00110ac bffebe63 bffebe7e bffebe88 8c008000
>>>> 00001000 0000
>>>> 0d12 b7ee2000
>>>> [    1.043579] GPR08: 00000033 00000000 00000000 c139df10 48224824
>>>> 1016c314 1016
>>>> 0000 00000000
>>>> [    1.043579] GPR16: 10160000 10160000 00000008 00000000 10160000
>>>> 00000000 1016
>>>> 0000 1017f5b0
>>>> [    1.043579] GPR24: 1017fa50 1017f4f0 1017fa50 1017f740 1017f630
>>>> 00000000 0000
>>>> 0000 1017f4f0
>>>> [    1.044101] NIP [b7ee2000] 0xb7ee2000
>>>> [    1.044153] LR [8c008000] 0x8c008000
>>>> [    1.044204] Call Trace:
>>>> [    1.044238] Instruction dump:
>>>> [    1.044279] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
>>>> XXXXXXXX XX
>>>> XXXXXX
>>>> [    1.044392] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
>>>> XXXXXXXX XX
>>>> XXXXXX
>>>> [    1.044506] ---[ end trace 0000000000000000 ]---
>>>> [    1.044568]
>>>> [    1.044590] note: mount[61] exited with irqs disabled
>>>>