[PATCH 0/3] KVM: x86: Various emulator fixes

[PATCH 0/3] KVM: x86: Various emulator fixes

[Liran Alon]

Hi,

The series includes various independent fixes in x86 emulator.

The first patch fix #UD intercept handlers (both in VMX and SVM)
to actually exit to user-mode when required by the x86 emulator.

The second patch fixes a bug in the return-value of
handle_emulation_failure() which cause callers to not exit user-mode
even though the x86 emulator intended to do so.

The third patch fixes bad emulation_type passed to the x86 emulator
from a wrapper utility function (emulate_instruction()).

Thanks.
-Liran Alon

[PATCH 1/3] KVM: x86: Exit to user-mode on #UD intercept when emulator requires

[Liran Alon]

From: Liran Alon <liran.alon@ravellosystems.com>

Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/kvm/svm.c | 2 ++
 arch/x86/kvm/vmx.c | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 0e68f0b3cbf7..e0162b20e3c9 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -2189,6 +2189,8 @@ static int ud_interception(struct vcpu_svm *svm)
 	int er;
 
 	er = emulate_instruction(&svm->vcpu, EMULTYPE_TRAP_UD);
+	if (er == EMULATE_USER_EXIT)
+		return 0;
 	if (er != EMULATE_DONE)
 		kvm_queue_exception(&svm->vcpu, UD_VECTOR);
 	return 1;
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 95a01609d7ee..2b63d9edc207 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5886,6 +5886,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
 			return 1;
 		}
 		er = emulate_instruction(vcpu, EMULTYPE_TRAP_UD);
+		if (er == EMULATE_USER_EXIT)
+			return 0;
 		if (er != EMULATE_DONE)
 			kvm_queue_exception(vcpu, UD_VECTOR);
 		return 1;
-- 
1.9.1

[PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Liran Alon]

From: Liran Alon <liran.alon@ravellosystems.com>

On this case, handle_emulation_failure() fills kvm_run with
internal-error information which it expects to be delivered
to user-mode for further processing.
However, the code reports a wrong return-value which makes KVM to never
return to user-mode on this scenario.

Fixes: 6d77dbfc88e3 ("KVM: inject #UD if instruction emulation fails and exit to
userspace")

Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/kvm/x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 03869eb7fcd6..f4edb4baf441 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5413,7 +5413,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu)
 		vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
 		vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION;
 		vcpu->run->internal.ndata = 0;
-		r = EMULATE_FAIL;
+		r = EMULATE_USER_EXIT;
 	}
 	kvm_queue_exception(vcpu, UD_VECTOR);
 
-- 
1.9.1

[PATCH 3/3] KVM: x86: Don't re-execute instruction when not passing CR2 value

[Liran Alon]

From: Liran Alon <liran.alon@ravellosystems.com>

In case of instruction-decode failure or emulation failure,
x86_emulate_instruction() will call reexecute_instruction() which will
attempt to use the cr2 value passed to x86_emulate_instruction().
However, when x86_emulate_instruction() is called from
emulate_instruction(), cr2 is not passed (passed as 0) and therefore
it doesn't make sense to execute reexecute_instruction() logic at all.

Fixes: 51d8b66199e9 ("KVM: cleanup emulate_instruction")

Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/kvm_host.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c73e493adf07..bc1347949cef 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1156,7 +1156,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2,
 static inline int emulate_instruction(struct kvm_vcpu *vcpu,
 			int emulation_type)
 {
-	return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0);
+	return x86_emulate_instruction(vcpu, 0,
+			emulation_type | EMULTYPE_NO_REEXECUTE, NULL, 0);
 }
 
 void kvm_enable_efer_bits(u64);
-- 
1.9.1

[PATCH 3/3] KVM: x86: Don't re-execute instruction when not passing CR2 value

[Liran Alon]

In case of instruction-decode failure or emulation failure,
x86_emulate_instruction() will call reexecute_instruction() which will
attempt to use the cr2 value passed to x86_emulate_instruction().
However, when x86_emulate_instruction() is called from
emulate_instruction(), cr2 is not passed (passed as 0) and therefore
it doesn't make sense to execute reexecute_instruction() logic at all.

Fixes: 51d8b66199e9 ("KVM: cleanup emulate_instruction")

Signed-off-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/kvm_host.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c73e493adf07..bc1347949cef 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1156,7 +1156,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2,
 static inline int emulate_instruction(struct kvm_vcpu *vcpu,
 			int emulation_type)
 {
-	return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0);
+	return x86_emulate_instruction(vcpu, 0,
+			emulation_type | EMULTYPE_NO_REEXECUTE, NULL, 0);
 }
 
 void kvm_enable_efer_bits(u64);
-- 
1.9.1

Re: [PATCH 1/3] KVM: x86: Exit to user-mode on #UD intercept when emulator requires

[Paolo Bonzini]

On 05/11/2017 15:21, Liran Alon wrote:
> From: Liran Alon <liran.alon@ravellosystems.com>
> 
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  arch/x86/kvm/svm.c | 2 ++
>  arch/x86/kvm/vmx.c | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
> index 0e68f0b3cbf7..e0162b20e3c9 100644
> --- a/arch/x86/kvm/svm.c
> +++ b/arch/x86/kvm/svm.c
> @@ -2189,6 +2189,8 @@ static int ud_interception(struct vcpu_svm *svm)
>  	int er;
>  
>  	er = emulate_instruction(&svm->vcpu, EMULTYPE_TRAP_UD);
> +	if (er == EMULATE_USER_EXIT)
> +		return 0;
>  	if (er != EMULATE_DONE)
>  		kvm_queue_exception(&svm->vcpu, UD_VECTOR);
>  	return 1;
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index 95a01609d7ee..2b63d9edc207 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -5886,6 +5886,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
>  			return 1;
>  		}
>  		er = emulate_instruction(vcpu, EMULTYPE_TRAP_UD);
> +		if (er == EMULATE_USER_EXIT)
> +			return 0;
>  		if (er != EMULATE_DONE)
>  			kvm_queue_exception(vcpu, UD_VECTOR);
>  		return 1;
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Cc: stable@vger.kernel.org

Re: [PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Paolo Bonzini]

On 05/11/2017 15:21, Liran Alon wrote:
> From: Liran Alon <liran.alon@ravellosystems.com>
> 
> On this case, handle_emulation_failure() fills kvm_run with
> internal-error information which it expects to be delivered
> to user-mode for further processing.
> However, the code reports a wrong return-value which makes KVM to never
> return to user-mode on this scenario.
> 
> Fixes: 6d77dbfc88e3 ("KVM: inject #UD if instruction emulation fails and exit to
> userspace")
> 
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  arch/x86/kvm/x86.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 03869eb7fcd6..f4edb4baf441 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -5413,7 +5413,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu)
>  		vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
>  		vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION;
>  		vcpu->run->internal.ndata = 0;
> -		r = EMULATE_FAIL;
> +		r = EMULATE_USER_EXIT;
>  	}
>  	kvm_queue_exception(vcpu, UD_VECTOR);
>  
> 

You should still get an emulation failure when the emulator is invoked
by mmu.c:

        case EMULATE_USER_EXIT:
                ++vcpu->stat.mmio_exits;
                /* fall through */
        case EMULATE_FAIL:
                return 0;

What is the caller of x86_emulate_instruction that you are interested
in?  #UD is not affected, because emulation_type has EMULTYPE_TRAP_UD
set and therefore x86_emulate_instruction exits before invoking
handle_emulation_failure.

Thanks,

Paolo

Re: [PATCH 3/3] KVM: x86: Don't re-execute instruction when not passing CR2 value

[Paolo Bonzini]

On 05/11/2017 15:21, Liran Alon wrote:
> From: Liran Alon <liran.alon@ravellosystems.com>
> 
> In case of instruction-decode failure or emulation failure,
> x86_emulate_instruction() will call reexecute_instruction() which will
> attempt to use the cr2 value passed to x86_emulate_instruction().
> However, when x86_emulate_instruction() is called from
> emulate_instruction(), cr2 is not passed (passed as 0) and therefore
> it doesn't make sense to execute reexecute_instruction() logic at all.
> 
> Fixes: 51d8b66199e9 ("KVM: cleanup emulate_instruction")
> 
> Signed-off-by: Liran Alon <liran.alon@oracle.com>
> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  arch/x86/include/asm/kvm_host.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index c73e493adf07..bc1347949cef 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1156,7 +1156,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2,
>  static inline int emulate_instruction(struct kvm_vcpu *vcpu,
>  			int emulation_type)
>  {
> -	return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0);
> +	return x86_emulate_instruction(vcpu, 0,
> +			emulation_type | EMULTYPE_NO_REEXECUTE, NULL, 0);
>  }
>  
>  void kvm_enable_efer_bits(u64);
> 

You can remove EMULTYPE_NO_REEXECUTE from handle_invalid_guest_state now.

Thanks,

Paolo

Re: [PATCH 3/3] KVM: x86: Don't re-execute instruction when not passing CR2 value

[Liran Alon]

On 06/11/17 11:21, Paolo Bonzini wrote:
> On 05/11/2017 15:21, Liran Alon wrote:
>> From: Liran Alon <liran.alon@ravellosystems.com>
>>
>> In case of instruction-decode failure or emulation failure,
>> x86_emulate_instruction() will call reexecute_instruction() which will
>> attempt to use the cr2 value passed to x86_emulate_instruction().
>> However, when x86_emulate_instruction() is called from
>> emulate_instruction(), cr2 is not passed (passed as 0) and therefore
>> it doesn't make sense to execute reexecute_instruction() logic at all.
>>
>> Fixes: 51d8b66199e9 ("KVM: cleanup emulate_instruction")
>>
>> Signed-off-by: Liran Alon <liran.alon@oracle.com>
>> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
>> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> ---
>>   arch/x86/include/asm/kvm_host.h | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>> index c73e493adf07..bc1347949cef 100644
>> --- a/arch/x86/include/asm/kvm_host.h
>> +++ b/arch/x86/include/asm/kvm_host.h
>> @@ -1156,7 +1156,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2,
>>   static inline int emulate_instruction(struct kvm_vcpu *vcpu,
>>   			int emulation_type)
>>   {
>> -	return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0);
>> +	return x86_emulate_instruction(vcpu, 0,
>> +			emulation_type | EMULTYPE_NO_REEXECUTE, NULL, 0);
>>   }
>>
>>   void kvm_enable_efer_bits(u64);
>>
>
> You can remove EMULTYPE_NO_REEXECUTE from handle_invalid_guest_state now.
Nice catch. Will remove it in next version of this commit. Thanks.
>
> Thanks,
>
> Paolo
>

Re: [PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Liran Alon]

On 06/11/17 11:19, Paolo Bonzini wrote:
> On 05/11/2017 15:21, Liran Alon wrote:
>> From: Liran Alon <liran.alon@ravellosystems.com>
>>
>> On this case, handle_emulation_failure() fills kvm_run with
>> internal-error information which it expects to be delivered
>> to user-mode for further processing.
>> However, the code reports a wrong return-value which makes KVM to never
>> return to user-mode on this scenario.
>>
>> Fixes: 6d77dbfc88e3 ("KVM: inject #UD if instruction emulation fails and exit to
>> userspace")
>>
>> Signed-off-by: Liran Alon <liran.alon@oracle.com>
>> Reviewed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
>> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> ---
>>   arch/x86/kvm/x86.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 03869eb7fcd6..f4edb4baf441 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -5413,7 +5413,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu)
>>   		vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
>>   		vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION;
>>   		vcpu->run->internal.ndata = 0;
>> -		r = EMULATE_FAIL;
>> +		r = EMULATE_USER_EXIT;
>>   	}
>>   	kvm_queue_exception(vcpu, UD_VECTOR);
>>
>>
>
> You should still get an emulation failure when the emulator is invoked
> by mmu.c:
>
>          case EMULATE_USER_EXIT:
>                  ++vcpu->stat.mmio_exits;
>                  /* fall through */
>          case EMULATE_FAIL:
>                  return 0;
>
> What is the caller of x86_emulate_instruction that you are interested
> in?  #UD is not affected, because emulation_type has EMULTYPE_TRAP_UD
> set and therefore x86_emulate_instruction exits before invoking
> handle_emulation_failure.
I think this patch is still correct from a number of reasons:

1) If I understand the code correctly, semantically it doesn't make 
sense to fill kvm_run struct without exiting to user-mode. Therefore, if 
emulator filled kvm_run, it makes sense that it needs to return 
EMULATE_USER_EXIT.

2) EMULTYPE_TRAP_UD only causes the emulator to return EMULATE_FAIL in 
case emulation fails on instruction decoding. However, consider a case 
where #UD intercept happens on a valid instruction (such as VMMCALL AMD 
opcode on physical Intel CPU). In that case, instruction decoding 
doesn't fail but we could still fail on instruction emulation at 
x86_emulate_insn(). In this case, EMULTYPE_TRAP_UD flag is not 
considered anymore and failure will reach handle_emulation_failure().

3) We have another KVM commits series (not upstream yet) that adds 
intercept on #GP which calls the x86 emulator. This is done to allow 
access to I/O ports even though they aren't allowed via guest's TSS I/O 
permissions bitmap.
>
> Thanks,
>
> Paolo
>

Re: [PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Paolo Bonzini]

On 06/11/2017 14:25, Liran Alon wrote:
>> What is the caller of x86_emulate_instruction that you are interested
>> in?  #UD is not affected, because emulation_type has EMULTYPE_TRAP_UD
>> set and therefore x86_emulate_instruction exits before invoking
>> handle_emulation_failure. 
>>
> I think this patch is still correct from a number of reasons:
> 
> 1) If I understand the code correctly, semantically it doesn't make
> sense to fill kvm_run struct without exiting to user-mode. Therefore, if
> emulator filled kvm_run, it makes sense that it needs to return
> EMULATE_USER_EXIT.
> 
> 2) EMULTYPE_TRAP_UD only causes the emulator to return EMULATE_FAIL in
> case emulation fails on instruction decoding. However, consider a case
> where #UD intercept happens on a valid instruction (such as VMMCALL AMD
> opcode on physical Intel CPU). In that case, instruction decoding
> doesn't fail but we could still fail on instruction emulation at
> x86_emulate_insn(). In this case, EMULTYPE_TRAP_UD flag is not
> considered anymore and failure will reach handle_emulation_failure().

(1) is true, but (2) more or less answers my question.  So

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

> 3) We have another KVM commits series (not upstream yet) that adds
> intercept on #GP which calls the x86 emulator. This is done to allow
> access to I/O ports even though they aren't allowed via guest's TSS I/O
> permissions bitmap.

vmport?... :)

Paolo

Re: [PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Liran Alon]

On 06/11/17 15:50, Paolo Bonzini wrote:
> On 06/11/2017 14:25, Liran Alon wrote:
>>> What is the caller of x86_emulate_instruction that you are interested
>>> in?  #UD is not affected, because emulation_type has EMULTYPE_TRAP_UD
>>> set and therefore x86_emulate_instruction exits before invoking
>>> handle_emulation_failure.
>>>
>> I think this patch is still correct from a number of reasons:
>>
>> 1) If I understand the code correctly, semantically it doesn't make
>> sense to fill kvm_run struct without exiting to user-mode. Therefore, if
>> emulator filled kvm_run, it makes sense that it needs to return
>> EMULATE_USER_EXIT.
>>
>> 2) EMULTYPE_TRAP_UD only causes the emulator to return EMULATE_FAIL in
>> case emulation fails on instruction decoding. However, consider a case
>> where #UD intercept happens on a valid instruction (such as VMMCALL AMD
>> opcode on physical Intel CPU). In that case, instruction decoding
>> doesn't fail but we could still fail on instruction emulation at
>> x86_emulate_insn(). In this case, EMULTYPE_TRAP_UD flag is not
>> considered anymore and failure will reach handle_emulation_failure().
>
> (1) is true, but (2) more or less answers my question.  So
>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
>
>> 3) We have another KVM commits series (not upstream yet) that adds
>> intercept on #GP which calls the x86 emulator. This is done to allow
>> access to I/O ports even though they aren't allowed via guest's TSS I/O
>> permissions bitmap.
>
> vmport?... :)
Yep. Stay tuned. :)
>
> Paolo
>

Re: [PATCH 2/3] KVM: x86: emulator: Return to user-mode on L1 CPL=0 emulation failure

[Paolo Bonzini]

On 06/11/2017 15:08, Liran Alon wrote:
>>> 3) We have another KVM commits series (not upstream yet) that adds
>>> intercept on #GP which calls the x86 emulator. This is done to allow
>>> access to I/O ports even though they aren't allowed via guest's TSS I/O
>>> permissions bitmap.
>>
>> vmport?... :)
>
> Yep. Stay tuned. :)

Is that a warning? ;)

Paolo