Re: PowerPC KVM-PR issue

[Nicholas Piggin <npiggin@gmail.com>]

Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>> Hello,
>>>>>
>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out 
>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>
>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at 
>>>>> 700 failed (00000000)
>>>>>
>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the 
>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>
>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>
>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>
>>>>> Thanks,
>>>>> Christian
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>
>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for 
>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and 
>>>> check the PowerPC updates 5.7-1 [1].
>>>>
>>>> Thanks
>>>>
>>>> [1] 
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>
>>>>
>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today. 
>>> Unfortunately I can't use KVM-PR with MoL anymore because of this 
>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>
>>> Thanks
>>>
>>> [1]
>>> - 
>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>> - 
>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>
>> Hi All,
>>
>> I bisected today because of the KVM-PR issue.
>>
>> Result:
>>
>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>> Author: Nicholas Piggin <npiggin@gmail.com>
>> Date:   Wed Feb 26 03:35:21 2020 +1000
>>
>>     powerpc/64s/exception: Move KVM test to common code
>>
>>     This allows more code to be moved out of unrelocated regions. The
>>     system call KVMTEST is changed to be open-coded and remain in the
>>     tramp area to avoid having to move it to entry_64.S. The custom 
>> nature
>>     of the system call entry code means the hcall case can be made more
>>     streamlined than regular interrupt handlers.
>>
>>     mpe: Incorporate fix from Nick:
>>
>>     Moving KVM test to the common entry code missed the case of HMI and
>>     MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>>     switch to virt mode).
>>
>>     This means a MCE or HMI exception that is taken while KVM is 
>> running a
>>     guest context will not be switched out of that context, and KVM won't
>>     be notified. Found by running sigfuz in guest with patched host on
>>     POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>>     expected and supposed to be handled by KVM).
>>
>>     This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>>     the KVM test. This makes them look a little more like other handlers
>>     that all use __GEN_COMMON_ENTRY.
>>
>>     Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>     Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>     Link: 
>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>
>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0 
>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M    arch
>>
>> -----
>>
>> The following commit is the problem: powerpc/64s/exception: Move KVM 
>> test to common code [1]
>>
>> These changes were included in the PowerPC updates 5.7-1. [2]
>>
>> Another test:
>>
>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates 
>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>
>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1 
>> -> KVM-PR works.
>>
>> Could you please check the first bad commit? [1]
>>
>> Thanks,
>> Christian
>>
>>
>> [1] 
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>> [2] 
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
> 
> Hi All,
> 
> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git 
> kernel and for the stable kernel 5.7.2 but without any success. There 
> was lot of restructuring work during the kernel 5.7 development time in 
> the PowerPC area so it isn't possible reactivate the old code. That 
> means we have lost the whole KVM-PR support. I also reported this issue 
> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty 
> broken. Have you ever made a bisect to see where the problem comes from?"
> 
> Please check the KVM-PR code.

Hey, thanks for debugging it and reporting. I'm looking into it, will 
try to get a fix soon.

Thanks,
Nick