Re: PowerPC KVM-PR issue

[Christian Zigotzky <chzigotzky@xenosoft.de>]

On 14 June 2020 at 02:53 pm, Nicholas Piggin wrote:
> Excerpts from Christian Zigotzky's message of June 12, 2020 11:01 pm:
>> On 11 June 2020 at 04:47 pm, Christian Zigotzky wrote:
>>> On 10 June 2020 at 01:23 pm, Christian Zigotzky wrote:
>>>> On 10 June 2020 at 11:06 am, Christian Zigotzky wrote:
>>>>> On 10 June 2020 at 00:18 am, Christian Zigotzky wrote:
>>>>>> Hello,
>>>>>>
>>>>>> KVM-PR doesn't work anymore on my Nemo board [1]. I figured out
>>>>>> that the Git kernels and the kernel 5.7 are affected.
>>>>>>
>>>>>> Error message: Fienix kernel: kvmppc_exit_pr_progint: emulation at
>>>>>> 700 failed (00000000)
>>>>>>
>>>>>> I can boot virtual QEMU PowerPC machines with KVM-PR with the
>>>>>> kernel 5.6 without any problems on my Nemo board.
>>>>>>
>>>>>> I tested it with QEMU 2.5.0 and QEMU 5.0.0 today.
>>>>>>
>>>>>> Could you please check KVM-PR on your PowerPC machine?
>>>>>>
>>>>>> Thanks,
>>>>>> Christian
>>>>>>
>>>>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000
>>>>> I figured out that the PowerPC updates 5.7-1 [1] are responsible for
>>>>> the KVM-PR issue. Please test KVM-PR on your PowerPC machines and
>>>>> check the PowerPC updates 5.7-1 [1].
>>>>>
>>>>> Thanks
>>>>>
>>>>> [1]
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>>>>>
>>>>>
>>>> I tested the latest Git kernel with Mac-on-Linux/KVM-PR today.
>>>> Unfortunately I can't use KVM-PR with MoL anymore because of this
>>>> issue (see screenshots [1]). Please check the PowerPC updates 5.7-1.
>>>>
>>>> Thanks
>>>>
>>>> [1]
>>>> -
>>>> https://i.pinimg.com/originals/0c/b3/64/0cb364a40241fa2b7f297d4272bbb8b7.png
>>>> -
>>>> https://i.pinimg.com/originals/9a/61/d1/9a61d170b1c9f514f7a78a3014ffd18f.png
>>>>
>>> Hi All,
>>>
>>> I bisected today because of the KVM-PR issue.
>>>
>>> Result:
>>>
>>> 9600f261acaaabd476d7833cec2dd20f2919f1a0 is the first bad commit
>>> commit 9600f261acaaabd476d7833cec2dd20f2919f1a0
>>> Author: Nicholas Piggin <npiggin@gmail.com>
>>> Date:   Wed Feb 26 03:35:21 2020 +1000
>>>
>>>      powerpc/64s/exception: Move KVM test to common code
>>>
>>>      This allows more code to be moved out of unrelocated regions. The
>>>      system call KVMTEST is changed to be open-coded and remain in the
>>>      tramp area to avoid having to move it to entry_64.S. The custom
>>> nature
>>>      of the system call entry code means the hcall case can be made more
>>>      streamlined than regular interrupt handlers.
>>>
>>>      mpe: Incorporate fix from Nick:
>>>
>>>      Moving KVM test to the common entry code missed the case of HMI and
>>>      MCE, which do not do __GEN_COMMON_ENTRY (because they don't want to
>>>      switch to virt mode).
>>>
>>>      This means a MCE or HMI exception that is taken while KVM is
>>> running a
>>>      guest context will not be switched out of that context, and KVM won't
>>>      be notified. Found by running sigfuz in guest with patched host on
>>>      POWER9 DD2.3, which causes some TM related HMI interrupts (which are
>>>      expected and supposed to be handled by KVM).
>>>
>>>      This fix adds a __GEN_REALMODE_COMMON_ENTRY for those handlers to add
>>>      the KVM test. This makes them look a little more like other handlers
>>>      that all use __GEN_COMMON_ENTRY.
>>>
>>>      Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>      Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
>>>      Link:
>>> https://lore.kernel.org/r/20200225173541.1549955-13-npiggin@gmail.com
>>>
>>> :040000 040000 ec21cec22d165f8696d69532734cb2985d532cb0
>>> 87dd49a9cd7202ec79350e8ca26cea01f1dbd93d M    arch
>>>
>>> -----
>>>
>>> The following commit is the problem: powerpc/64s/exception: Move KVM
>>> test to common code [1]
>>>
>>> These changes were included in the PowerPC updates 5.7-1. [2]
>>>
>>> Another test:
>>>
>>> git checkout d38c07afc356ddebaa3ed8ecb3f553340e05c969 (PowerPC updates
>>> 5.7-1 [2] ) -> KVM-PR doesn't work.
>>>
>>> After that: git revert d38c07afc356ddebaa3ed8ecb3f553340e05c969 -m 1
>>> -> KVM-PR works.
>>>
>>> Could you please check the first bad commit? [1]
>>>
>>> Thanks,
>>> Christian
>>>
>>>
>>> [1]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9600f261acaaabd476d7833cec2dd20f2919f1a0
>>> [2]
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d38c07afc356ddebaa3ed8ecb3f553340e05c969
>> Hi All,
>>
>> I tried to revert the __GEN_REALMODE_COMMON_ENTRY fix for the latest Git
>> kernel and for the stable kernel 5.7.2 but without any success. There
>> was lot of restructuring work during the kernel 5.7 development time in
>> the PowerPC area so it isn't possible reactivate the old code. That
>> means we have lost the whole KVM-PR support. I also reported this issue
>> to Alexander Graf two days ago. He wrote: "Howdy :). It looks pretty
>> broken. Have you ever made a bisect to see where the problem comes from?"
>>
>> Please check the KVM-PR code.
> Does this patch fix it for you?
>
> The CTR register reload in the KVM interrupt path used the wrong save
> area for SLB (and NMI) interrupts.
>
> Fixes: 9600f261acaaa ("powerpc/64s/exception: Move KVM test to common code")
> Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   arch/powerpc/kernel/exceptions-64s.S | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
> index e70ebb5c318c..fa080694e581 100644
> --- a/arch/powerpc/kernel/exceptions-64s.S
> +++ b/arch/powerpc/kernel/exceptions-64s.S
> @@ -270,7 +270,7 @@ BEGIN_FTR_SECTION
>   END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
>   	.endif
>   
> -	ld	r10,PACA_EXGEN+EX_CTR(r13)
> +	ld	r10,IAREA+EX_CTR(r13)
>   	mtctr	r10
>   BEGIN_FTR_SECTION
>   	ld	r10,IAREA+EX_PPR(r13)
> @@ -298,7 +298,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
>   
>   	.if IKVM_SKIP
>   89:	mtocrf	0x80,r9
> -	ld	r10,PACA_EXGEN+EX_CTR(r13)
> +	ld	r10,IAREA+EX_CTR(r13)
>   	mtctr	r10
>   	ld	r9,IAREA+EX_R9(r13)
>   	ld	r10,IAREA+EX_R10(r13)
Many thanks for the fix! I will test it with the RC1 tomorrow.

-- Christian