Re: [PATCH qemu v19] spapr: Implement Open Firmware client interface

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 5/17/21 09:34, BALATON Zoltan wrote:
> On Sat, 15 May 2021, BALATON Zoltan wrote:
>> On Sat, 15 May 2021, BALATON Zoltan wrote:
>>> On Thu, 22 Apr 2021, Alexey Kardashevskiy wrote:

[snip]

>>>> +/* Defined as Big Endian */
>>>> +struct prom_args {
>>>> +    uint32_t service;
>>>> +    uint32_t nargs;
>>>> +    uint32_t nret;
>>>> +    uint32_t args[10];
>>>> +} QEMU_PACKED;
>>>
>>> This #define and struct definition should probably be moved to 
>>> include/hw/ppc/vof.h as I had to copy these when trying to get VOF 
>>> running with pegasos2 and these seem to be VOF specific not spapr.
>>>
>>> I was trying to wire up VOF on pegasos2 as proof of concept but I did 
>>> not get very far as it crashed at the first move due to using mtmsrd 
>>> which does not exist on the 32 bit CPUs (G4 or G3) used by pegasos2:
>>>
>>> vof_claim virt=0x0 size=0xc38 align=0x0 => 0x0
>>> vof_claim virt=0x0 size=0x8000 align=0x8000 => 0x8000
>>> vof_claim virt=0xc00000 size=0x18fd62 align=0x0 => 0xc00000
>>> vof_claimed 0x0..0xc38 size=0xc38
>>> vof_claimed 0x8000..0x10000 size=0x8000
>>> vof_claimed 0xc00000..0xd8fd62 size=0x18fd62
>>> vof_avail 0xc38..0x8000 size=0x73c8
>>> vof_avail 0x10000..0xc00000 size=0xbf0000
>>> vof_avail 0xd8fd62..0x20000000 size=0x1f27029e
>>> via_superio_cfg: unimplemented register 0xf2
>>> via_superio_cfg: unimplemented register 0xf4
>>> via_superio_cfg: unimplemented register 0xf6
>>> via_superio_cfg: unimplemented register 0xf7
>>> invalid/unsupported opcode: 1f - 12 - 05 - 00 (7fe00164) fff00108 0
>>> ----------------
>>> IN:
>>> 0xfff00100:  3fe00000  lis      r31, 0
>>> 0xfff00104:  63ff0000  ori      r31, r31, 0
>>> 0xfff00108:  7fe00164  mtmsrd   r31
>>>
>>> ----------------
>>> IN:
>>> 0xfff00700:  807e8020  lwz      r3, -0x7fe0(r30)
>>> 0xfff00704:  4cc63182  crclr    6
>>> 0xfff00708:  4bfffd1d  bl       0xfff00424
>>>
>>> Invalid access at addr 0xFFFF8020, size 4, region '(null)', reason: 
>>> rejected
>>>
>>> Is this mtmsrd really needed? Do 64-bit Power CPUs start in 64 bit 
>>> mode? I'd expect them to be in compatibility mode unless 64 bit is 
>>> enabled but I did not check the docs. If it's needed maybe a dummy 
>>> handler has to be at 0x700 to ignore this exception if it's running 
>>> on a 32-bit CPU.
>>>
>>> By the way does vof need to be loaded at addr 0 or it could work at 
>>> the default ROM address as well? That would simplify usage if it 
>>> could run position independent.
>>
>> Just for the sake of experimenting I've patched out the mtmsrd for now 
>> from vof.bin and got a bit further to the point that it's trying to do 
>> hypercalls now but there must be something wrong with decoding the 
>> parameters as I'm getting:
>>
>> IN:
>> 0xfff00590:  393f000c  addi     r9, r31, 0xc
>> 0xfff00594:  7d234b78  mr       r3, r9
>> 0xfff00598:  4bfffbad  bl       0xfff00144
>>
>> ----------------
>> IN:
>> 0xfff00144:  7c641b78  mr       r4, r3
>> 0xfff00148:  3c600000  lis      r3, 0
>> 0xfff0014c:  6063f005  ori      r3, r3, 0xf005
>> 0xfff00150:  44000022  sc       1
>>
>> Raise exception at fff00150 => 00000008 (01)
>> hypercall r3=000000000000f005 r4=000000000000fe7c r5=0000000000000001 
>> r6=0000000000000be8 r7=0000000000000000 r8=000000000000fe78 
>> r9=000000000000fe7c r10=0000000000000001 r11=0000000000000000 
>> r12=0000000000000000 nip=fff00150
>> vof_error_unknown_service "" args=1 rets=1
>>
>> (This is the first hypercall vof does, did not check what this should 
>> be.) I've basically blindly copied spapr_h_vof_client() from 
>> spapr_vof.c but I only vagely understand what it tries to do. Since I 
>> did not have ppc64_phys_to_real() on the 32-bit PPC CPU pegasos2 is 
>> using I've just dropped that and using _args[0] as args_real but maybe 
>> that's wrong and it needs some conversion? For other hypercalls later 
>> I get same result with service being empty while args and rets change. 
>> Any idea what I could be missing or how to debug it?
>>
>> One thing I don't get is how it will find the kernel entry point to 
>> start executing? Does it have to be in the device tree somewhere or it 
>> expects a fixed address? (I could probably find out from the source 
>> but it's easier to ask.)
> 
> OK, I've found that vof.bin needs to be at address 0 then hypercalls 
> work and it tries to query /chosen/qemu.boot-kernel but I get len = -1 
> for some reason. I'm adding the kernel address and size like this:
> 
>     uint64_t cells[2];
> 
>     cells[0] = cpu_to_be64(pm->kernel_addr);
>     cells[1] = cpu_to_be64(pm->kernel_size);
>     qemu_fdt_setprop(fdt, "/chosen", "qemu,boot-kernel",
>                      cells, sizeof(cells));
> 
> which is very much like what spapr does but when vof tries to query it I 
> get:


spapr_vof_reset() also claims the kernel/initrd/VOF memory and allocates 
the stack,  vof_claim() barfs if there are overlaps.


> Raise exception at 00000150 => 00000008 (01)
> hypercall r3=000000000000f005 r4=000000000000fe6c r5=0000000000000001 
> r6=0000000000000005 r7=0000000000000bf0 r8=000000000000fe68 
> r9=000000000000fe6c r10=0000000000000001 r11=000000000000ff60 
> r12=0000000000000000 nip=00000150
> vof_getprop ph=0x5 "qemu,boot-kernel" => len=-1 []
> 
> then it calls exit and the VM stops. Any idea what could be wrong with 
> the above or what to check?

Memory allocation. Another thing I saw was clang/llvm incorrectly 
initializing bss start/end for prom (very early boot) so the prim init 
code in the kernel would memset(0) wrong page and break things. Dunno.




-- 
Alexey