[PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Thomas Huth]

If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
returned EMULATE_FAIL, so the guest gets an program interrupt for the
illegal opcode.
However, the kvmppc_emulate_instruction() also tried to inject a
program exception for this already, so the program interrupt gets
injected twice and the return address in srr0 gets destroyed.
All other callers of kvmppc_emulate_instruction() are also injecting
a program interrupt, and since the callers have the right knowledge
about the srr1 flags that should be used, it is the function
kvmppc_emulate_instruction() that should _not_ inject program
interrupts, so remove the kvmppc_core_queue_program() here.

This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
where the logs are filled with these messages when the test tries
to execute an illegal instruction:

     Couldn't emulate instruction 0x00000000 (op 0 xop 0)
     kvmppc_handle_exit_pr: emulation at 700 failed (00000000)

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 arch/powerpc/kvm/emulate.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
index 5cc2e7a..b379146 100644
--- a/arch/powerpc/kvm/emulate.c
+++ b/arch/powerpc/kvm/emulate.c
@@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
 			advance = 0;
 			printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
 			       "(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
-			kvmppc_core_queue_program(vcpu, 0);
 		}
 	}
 
-- 
1.8.3.1

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Laurent Vivier]

On 18/05/2016 21:01, Thomas Huth wrote:
> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
> returned EMULATE_FAIL, so the guest gets an program interrupt for the
> illegal opcode.
> However, the kvmppc_emulate_instruction() also tried to inject a
> program exception for this already, so the program interrupt gets
> injected twice and the return address in srr0 gets destroyed.
> All other callers of kvmppc_emulate_instruction() are also injecting
> a program interrupt, and since the callers have the right knowledge
> about the srr1 flags that should be used, it is the function
> kvmppc_emulate_instruction() that should _not_ inject program
> interrupts, so remove the kvmppc_core_queue_program() here.
> 
> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
> where the logs are filled with these messages when the test tries
> to execute an illegal instruction:
> 
>      Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>      kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  arch/powerpc/kvm/emulate.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
> index 5cc2e7a..b379146 100644
> --- a/arch/powerpc/kvm/emulate.c
> +++ b/arch/powerpc/kvm/emulate.c
> @@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
>  			advance = 0;
>  			printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
>  			       "(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
> -			kvmppc_core_queue_program(vcpu, 0);
>  		}
>  	}
>  
> 

I've tested this patch with kvm-unit-tests: it solves the multiple
illegal instruction exceptions, but the test fails because SRR1 is not
updated correctly. It should contains the bit for "Illegal Instruction"
whereas it is 0.
[But I think it's what you explain in your last email]

Thanks,
Laurent

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Alexander Graf]

On 05/18/2016 09:01 PM, Thomas Huth wrote:
> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
> returned EMULATE_FAIL, so the guest gets an program interrupt for the
> illegal opcode.
> However, the kvmppc_emulate_instruction() also tried to inject a
> program exception for this already, so the program interrupt gets
> injected twice and the return address in srr0 gets destroyed.
> All other callers of kvmppc_emulate_instruction() are also injecting
> a program interrupt, and since the callers have the right knowledge
> about the srr1 flags that should be used, it is the function
> kvmppc_emulate_instruction() that should _not_ inject program
> interrupts, so remove the kvmppc_core_queue_program() here.
>
> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
> where the logs are filled with these messages when the test tries
> to execute an illegal instruction:
>
>       Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>       kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

I'm surprised you're the first one to encounter this :).

Reviewed-by: Alexander Graf <agraf@suse.de>


Alex

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Alexander Graf]

On 05/19/2016 09:58 AM, Laurent Vivier wrote:
>
> On 18/05/2016 21:01, Thomas Huth wrote:
>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>> illegal opcode.
>> However, the kvmppc_emulate_instruction() also tried to inject a
>> program exception for this already, so the program interrupt gets
>> injected twice and the return address in srr0 gets destroyed.
>> All other callers of kvmppc_emulate_instruction() are also injecting
>> a program interrupt, and since the callers have the right knowledge
>> about the srr1 flags that should be used, it is the function
>> kvmppc_emulate_instruction() that should _not_ inject program
>> interrupts, so remove the kvmppc_core_queue_program() here.
>>
>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>> where the logs are filled with these messages when the test tries
>> to execute an illegal instruction:
>>
>>       Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>       kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>   arch/powerpc/kvm/emulate.c | 1 -
>>   1 file changed, 1 deletion(-)
>>
>> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
>> index 5cc2e7a..b379146 100644
>> --- a/arch/powerpc/kvm/emulate.c
>> +++ b/arch/powerpc/kvm/emulate.c
>> @@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
>>   			advance = 0;
>>   			printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
>>   			       "(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
>> -			kvmppc_core_queue_program(vcpu, 0);
>>   		}
>>   	}
>>   
>>
> I've tested this patch with kvm-unit-tests: it solves the multiple
> illegal instruction exceptions, but the test fails because SRR1 is not
> updated correctly. It should contains the bit for "Illegal Instruction"
> whereas it is 0.
> [But I think it's what you explain in your last email]

So if the illegal instruction flag is missing, that's probably because 
the host CPU didn't pass that in via SRR1. That's probably a subtle 
difference between EMUL_ASSIST and PROGRAM.

Please send a follow-up patch that sets the illegal instruction bit in 
flags on EMULATE_FAIL as well.


Alex

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Thomas Huth]

On 19.05.2016 09:58, Laurent Vivier wrote:
> 
> 
> On 18/05/2016 21:01, Thomas Huth wrote:
>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>> illegal opcode.
>> However, the kvmppc_emulate_instruction() also tried to inject a
>> program exception for this already, so the program interrupt gets
>> injected twice and the return address in srr0 gets destroyed.
>> All other callers of kvmppc_emulate_instruction() are also injecting
>> a program interrupt, and since the callers have the right knowledge
>> about the srr1 flags that should be used, it is the function
>> kvmppc_emulate_instruction() that should _not_ inject program
>> interrupts, so remove the kvmppc_core_queue_program() here.
>>
>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>> where the logs are filled with these messages when the test tries
>> to execute an illegal instruction:
>>
>>      Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>      kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  arch/powerpc/kvm/emulate.c | 1 -
>>  1 file changed, 1 deletion(-)
>>
>> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
>> index 5cc2e7a..b379146 100644
>> --- a/arch/powerpc/kvm/emulate.c
>> +++ b/arch/powerpc/kvm/emulate.c
>> @@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
>>  			advance = 0;
>>  			printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
>>  			       "(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
>> -			kvmppc_core_queue_program(vcpu, 0);
>>  		}
>>  	}
>>  
>>
> 
> I've tested this patch with kvm-unit-tests: it solves the multiple
> illegal instruction exceptions, but the test fails because SRR1 is not
> updated correctly. It should contains the bit for "Illegal Instruction"
> whereas it is 0.
> [But I think it's what you explain in your last email]

Yes, but that's a separate problem, so I did not include it in this
patch here yet. (Also, I'm not sure yet how to exactly fix that other
problem with SRR1)

 Thomas

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Thomas Huth]

On 19.05.2016 10:05, Alexander Graf wrote:
> On 05/19/2016 09:58 AM, Laurent Vivier wrote:
>>
>> On 18/05/2016 21:01, Thomas Huth wrote:
>>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>>> illegal opcode.
>>> However, the kvmppc_emulate_instruction() also tried to inject a
>>> program exception for this already, so the program interrupt gets
>>> injected twice and the return address in srr0 gets destroyed.
>>> All other callers of kvmppc_emulate_instruction() are also injecting
>>> a program interrupt, and since the callers have the right knowledge
>>> about the srr1 flags that should be used, it is the function
>>> kvmppc_emulate_instruction() that should _not_ inject program
>>> interrupts, so remove the kvmppc_core_queue_program() here.
>>>
>>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>>> where the logs are filled with these messages when the test tries
>>> to execute an illegal instruction:
>>>
>>>       Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>>       kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>   arch/powerpc/kvm/emulate.c | 1 -
>>>   1 file changed, 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
>>> index 5cc2e7a..b379146 100644
>>> --- a/arch/powerpc/kvm/emulate.c
>>> +++ b/arch/powerpc/kvm/emulate.c
>>> @@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run
>>> *run, struct kvm_vcpu *vcpu)
>>>               advance = 0;
>>>               printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
>>>                      "(op %d xop %d)\n", inst, get_op(inst),
>>> get_xop(inst));
>>> -            kvmppc_core_queue_program(vcpu, 0);
>>>           }
>>>       }
>>>  
>> I've tested this patch with kvm-unit-tests: it solves the multiple
>> illegal instruction exceptions, but the test fails because SRR1 is not
>> updated correctly. It should contains the bit for "Illegal Instruction"
>> whereas it is 0.
>> [But I think it's what you explain in your last email]
> 
> So if the illegal instruction flag is missing, that's probably because
> the host CPU didn't pass that in via SRR1. That's probably a subtle
> difference between EMUL_ASSIST and PROGRAM.

Thanks for that hint! You're right, flags is only 0 if exit_nr is
BOOK3S_INTERRUPT_H_EMUL_ASSIST. It seems to contain proper values for
BOOK3S_INTERRUPT_PROGRAM, e.g. for privileged-instruction program
interrupts.

Looking at the PowerISA, this also supports this theory: For emulation
assist interrupts, the bits in SRR1 are 0.

So the right fix really seems to be to set flags = SRR1_PROGILL in case
the function has been called with EMUL_ASSIST.

> Please send a follow-up patch that sets the illegal instruction bit in
> flags on EMULATE_FAIL as well.

I think I'll best send a separate patch, since it is a separate issue.

 Thomas

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Laurent Vivier]

On 19/05/2016 10:07, Thomas Huth wrote:
> On 19.05.2016 09:58, Laurent Vivier wrote:
>>
>>
>> On 18/05/2016 21:01, Thomas Huth wrote:
>>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>>> illegal opcode.
>>> However, the kvmppc_emulate_instruction() also tried to inject a
>>> program exception for this already, so the program interrupt gets
>>> injected twice and the return address in srr0 gets destroyed.
>>> All other callers of kvmppc_emulate_instruction() are also injecting
>>> a program interrupt, and since the callers have the right knowledge
>>> about the srr1 flags that should be used, it is the function
>>> kvmppc_emulate_instruction() that should _not_ inject program
>>> interrupts, so remove the kvmppc_core_queue_program() here.
>>>
>>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>>> where the logs are filled with these messages when the test tries
>>> to execute an illegal instruction:
>>>
>>>      Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>>      kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  arch/powerpc/kvm/emulate.c | 1 -
>>>  1 file changed, 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c
>>> index 5cc2e7a..b379146 100644
>>> --- a/arch/powerpc/kvm/emulate.c
>>> +++ b/arch/powerpc/kvm/emulate.c
>>> @@ -302,7 +302,6 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)
>>>  			advance = 0;
>>>  			printk(KERN_ERR "Couldn't emulate instruction 0x%08x "
>>>  			       "(op %d xop %d)\n", inst, get_op(inst), get_xop(inst));
>>> -			kvmppc_core_queue_program(vcpu, 0);
>>>  		}
>>>  	}
>>>  
>>>
>>
>> I've tested this patch with kvm-unit-tests: it solves the multiple
>> illegal instruction exceptions, but the test fails because SRR1 is not
>> updated correctly. It should contains the bit for "Illegal Instruction"
>> whereas it is 0.
>> [But I think it's what you explain in your last email]
> 
> Yes, but that's a separate problem, so I did not include it in this
> patch here yet. (Also, I'm not sure yet how to exactly fix that other
> problem with SRR1)

OK, so for this one I can say:

Tested-by: Laurent Vivier <lvivier@redhat.com>

Laurent

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Thomas Huth]

On 19.05.2016 10:04, Alexander Graf wrote:
> On 05/18/2016 09:01 PM, Thomas Huth wrote:
>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>> illegal opcode.
>> However, the kvmppc_emulate_instruction() also tried to inject a
>> program exception for this already, so the program interrupt gets
>> injected twice and the return address in srr0 gets destroyed.
>> All other callers of kvmppc_emulate_instruction() are also injecting
>> a program interrupt, and since the callers have the right knowledge
>> about the srr1 flags that should be used, it is the function
>> kvmppc_emulate_instruction() that should _not_ inject program
>> interrupts, so remove the kvmppc_core_queue_program() here.
>>
>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>> where the logs are filled with these messages when the test tries
>> to execute an illegal instruction:
>>
>>       Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>       kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> 
> I'm surprised you're the first one to encounter this :).
> 
> Reviewed-by: Alexander Graf <agraf@suse.de>

*ping*

Paul, could you maybe pick up this patch?

Thanks,
 Thomas

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Thomas Huth]

On 31.05.2016 12:29, Thomas Huth wrote:
> On 19.05.2016 10:04, Alexander Graf wrote:
>> On 05/18/2016 09:01 PM, Thomas Huth wrote:
>>> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
>>> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
>>> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
>>> returned EMULATE_FAIL, so the guest gets an program interrupt for the
>>> illegal opcode.
>>> However, the kvmppc_emulate_instruction() also tried to inject a
>>> program exception for this already, so the program interrupt gets
>>> injected twice and the return address in srr0 gets destroyed.
>>> All other callers of kvmppc_emulate_instruction() are also injecting
>>> a program interrupt, and since the callers have the right knowledge
>>> about the srr1 flags that should be used, it is the function
>>> kvmppc_emulate_instruction() that should _not_ inject program
>>> interrupts, so remove the kvmppc_core_queue_program() here.
>>>
>>> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
>>> where the logs are filled with these messages when the test tries
>>> to execute an illegal instruction:
>>>
>>>       Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>>       kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>
>> I'm surprised you're the first one to encounter this :).
>>
>> Reviewed-by: Alexander Graf <agraf@suse.de>
> 
> *ping*
> 
> Paul, could you maybe pick up this patch?

ping^2

 Thomas

Re: [PATCH] KVM: PPC: Fix illegal opcode emulation in kvm-pr

[Paul Mackerras]

On Wed, May 18, 2016 at 09:01:20PM +0200, Thomas Huth wrote:
> If kvmppc_handle_exit_pr() calls kvmppc_emulate_instruction() to emulate
> one instruction (in the BOOK3S_INTERRUPT_H_EMUL_ASSIST case), it calls
> kvmppc_core_queue_program() afterwards if kvmppc_emulate_instruction()
> returned EMULATE_FAIL, so the guest gets an program interrupt for the
> illegal opcode.
> However, the kvmppc_emulate_instruction() also tried to inject a
> program exception for this already, so the program interrupt gets
> injected twice and the return address in srr0 gets destroyed.
> All other callers of kvmppc_emulate_instruction() are also injecting
> a program interrupt, and since the callers have the right knowledge
> about the srr1 flags that should be used, it is the function
> kvmppc_emulate_instruction() that should _not_ inject program
> interrupts, so remove the kvmppc_core_queue_program() here.
> 
> This fixes the issue discovered by Laurent Vivier with kvm-unit-tests
> where the logs are filled with these messages when the test tries
> to execute an illegal instruction:
> 
>      Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>      kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Thanks, applied to my kvm-ppc-next branch (with adjusted subject).

Paul.