[PATCH 1/2] KVM: Exception during emulation decode should propagate

[PATCH 1/2] KVM: Exception during emulation decode should propagate

[Nadav Amit]

An exception might occur during decode (e.g., #PF during fetch).
Currently, the exception is ignored and emulation is performed.
Instead, emulation should be skipped and the fault should be injected.
Skipping instruction should report a failure in this case.

Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
---
 arch/x86/kvm/emulate.c |    3 +++
 arch/x86/kvm/x86.c     |    8 ++++++++
 2 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 05a562b..e06dc98 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -3869,6 +3869,9 @@ done:
 	if (ctxt->memopp && ctxt->memopp->type == OP_MEM && ctxt->rip_relative)
 		ctxt->memopp->addr.mem.ea += ctxt->_eip;
 
+	if (rc == X86EMUL_PROPAGATE_FAULT)
+		ctxt->have_exception = true;
+
 	return (rc != X86EMUL_CONTINUE) ? EMULATION_FAILED : EMULATION_OK;
 }
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1171def..05fd3d7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4443,10 +4443,17 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,
 	}
 
 	if (emulation_type & EMULTYPE_SKIP) {
+		if (ctxt->have_exception)
+			return EMULATE_FAIL;
 		kvm_rip_write(vcpu, ctxt->_eip);
 		return EMULATE_DONE;
 	}
 
+	if (ctxt->have_exception) {
+		writeback = false;
+		goto post;
+	}
+
 	if (retry_instruction(ctxt, cr2, emulation_type))
 		return EMULATE_DONE;
 
@@ -4470,6 +4477,7 @@ restart:
 		return handle_emulation_failure(vcpu);
 	}
 
+post:
 	if (ctxt->have_exception) {
 		inject_emulated_exception(vcpu);
 		r = EMULATE_DONE;
-- 
1.7.4.1

[PATCH 2/2] KVM: Fix writeback on page boundary that propagate changes in spite of #PF

[Nadav Amit]

Consider the case in which an instruction emulation writeback is performed on a page boundary.
In such case, if a #PF occurs on the second page, the write to the first page already occurred and cannot be retracted.
Therefore, validation of the second page access must be performed prior to writeback.

Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
---
 arch/x86/kvm/x86.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 05fd3d7..7af3d67 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3626,6 +3626,8 @@ struct read_write_emulator_ops {
 			       int bytes, void *val);
 	int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
 				    void *val, int bytes);
+	gpa_t (*read_write_validate)(struct kvm_vcpu *vcpu, gva_t gva,
+				     struct x86_exception *exception);
 	bool write;
 };
 
@@ -3686,6 +3688,7 @@ static struct read_write_emulator_ops write_emultor = {
 	.read_write_emulate = write_emulate,
 	.read_write_mmio = write_mmio,
 	.read_write_exit_mmio = write_exit_mmio,
+	.read_write_validate = kvm_mmu_gva_to_gpa_write,
 	.write = true,
 };
 
@@ -3750,6 +3753,16 @@ int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
 		int rc, now;
 
 		now = -addr & ~PAGE_MASK;
+
+		/* First check there is no page-fault on the next page */
+		if (ops->read_write_validate &&
+			ops->read_write_validate(vcpu, addr+now, exception) ==
+			UNMAPPED_GVA) {
+			/* #PF on the first page should be reported first */
+			ops->read_write_validate(vcpu, addr, exception);
+			return X86EMUL_PROPAGATE_FAULT;
+		}
+
 		rc = emulator_read_write_onepage(addr, val, now, exception,
 						 vcpu, ops);
 
-- 
1.7.4.1

Re: [PATCH 1/2] KVM: Exception during emulation decode should propagate

[Takuya Yoshikawa]

On Wed, 11 Jan 2012 18:53:30 +0200
Nadav Amit <namit@cs.technion.ac.il> wrote:

> An exception might occur during decode (e.g., #PF during fetch).
> Currently, the exception is ignored and emulation is performed.

When I cleaned up insn_fetch(), I thought that fetching the instruction
which is being executed by the guest cannot cause #PF.

The possibility that a meaningless userspace might similtaneously unmap
the page, noted by Avi IIRC, was ignored intentionally, so we just fail
in such a case.

Did you see any real problem?

	Takuya


> Instead, emulation should be skipped and the fault should be injected.
> Skipping instruction should report a failure in this case.

Re: [PATCH 1/2] KVM: Exception during emulation decode should propagate

[Takuya Yoshikawa]

(2012/01/12 7:11), Takuya Yoshikawa wrote:
> On Wed, 11 Jan 2012 18:53:30 +0200
> Nadav Amit<namit@cs.technion.ac.il>  wrote:
>
>> An exception might occur during decode (e.g., #PF during fetch).
>> Currently, the exception is ignored and emulation is performed.

Note that the decode/emulation will not be continued in such a case.

insn_fetch() is a bit tricky macro and it contains "goto done" to outside.
So if an error happens during fetching the instruction, x86_decode_insn()
will handle the X86EMUL_* fault value and returns FAIL immediately.

	Takuya

>
> When I cleaned up insn_fetch(), I thought that fetching the instruction
> which is being executed by the guest cannot cause #PF.
>
> The possibility that a meaningless userspace might similtaneously unmap
> the page, noted by Avi IIRC, was ignored intentionally, so we just fail
> in such a case.
>
> Did you see any real problem?
>
> 	Takuya
>
>
>> Instead, emulation should be skipped and the fault should be injected.
>> Skipping instruction should report a failure in this case.

Re: [PATCH 1/2] KVM: Exception during emulation decode should propagate

[Nadav Amit]

On Jan 12, 2012, at 2:26 AM, Takuya Yoshikawa wrote:

> (2012/01/12 7:11), Takuya Yoshikawa wrote:
>> On Wed, 11 Jan 2012 18:53:30 +0200
>> Nadav Amit<namit@cs.technion.ac.il>  wrote:
>> 
>>> An exception might occur during decode (e.g., #PF during fetch).
>>> Currently, the exception is ignored and emulation is performed.
> 
> Note that the decode/emulation will not be continued in such a case.
> 
> insn_fetch() is a bit tricky macro and it contains "goto done" to outside.
> So if an error happens during fetching the instruction, x86_decode_insn()
> will handle the X86EMUL_* fault value and returns FAIL immediately.

You got a point. Yet, a problem still exists.
I now notice I was originally working on previous version (3.0.0)
where the return-code of x86_decode_insn is handled differently.
Nonetheless, I think the current implementation might report emulation
error in such a scenario (instead of triggering #PF/#GP in the guest).

>> 
>> When I cleaned up insn_fetch(), I thought that fetching the instruction
>> which is being executed by the guest cannot cause #PF.
>> 
>> The possibility that a meaningless userspace might similtaneously unmap
>> the page, noted by Avi IIRC, was ignored intentionally, so we just fail
>> in such a case.
>> 
>> Did you see any real problem?

Well, I run some research project for which I emulate instructions quite
often. I do see a real problem with Linux 3.0.0. Please note AFAIK #GP
might occur as well during instruction fetch. I don't think failing is the
right behavior in such case - there is no real reason to fail.

Please tell me whether you are OK with KVM failing in such a scenario.
If not - I'll send an updated patch (in which x86_decode_insn returns
EMULATION_OK when rc == X86EMUL_PROPAGATE_FAULT).

Regards,
Nadav 

Re: [PATCH 1/2] KVM: Exception during emulation decode should propagate

[Takuya Yoshikawa]

(2012/01/12 18:07), Nadav Amit wrote:
>
> On Jan 12, 2012, at 2:26 AM, Takuya Yoshikawa wrote:
>
>> (2012/01/12 7:11), Takuya Yoshikawa wrote:
>>> On Wed, 11 Jan 2012 18:53:30 +0200
>>> Nadav Amit<namit@cs.technion.ac.il>   wrote:
>>>
>>>> An exception might occur during decode (e.g., #PF during fetch).
>>>> Currently, the exception is ignored and emulation is performed.
>>
>> Note that the decode/emulation will not be continued in such a case.
>>
>> insn_fetch() is a bit tricky macro and it contains "goto done" to outside.
>> So if an error happens during fetching the instruction, x86_decode_insn()
>> will handle the X86EMUL_* fault value and returns FAIL immediately.
>
> You got a point. Yet, a problem still exists.
> I now notice I was originally working on previous version (3.0.0)
> where the return-code of x86_decode_insn is handled differently.
> Nonetheless, I think the current implementation might report emulation
> error in such a scenario (instead of triggering #PF/#GP in the guest).

It's me who did that fix.

>
>>>
>>> When I cleaned up insn_fetch(), I thought that fetching the instruction
>>> which is being executed by the guest cannot cause #PF.
>>>
>>> The possibility that a meaningless userspace might similtaneously unmap
>>> the page, noted by Avi IIRC, was ignored intentionally, so we just fail
>>> in such a case.
>>>
>>> Did you see any real problem?
>
> Well, I run some research project for which I emulate instructions quite
> often. I do see a real problem with Linux 3.0.0. Please note AFAIK #GP
> might occur as well during instruction fetch. I don't think failing is the
> right behavior in such case - there is no real reason to fail.
>
> Please tell me whether you are OK with KVM failing in such a scenario.
> If not - I'll send an updated patch (in which x86_decode_insn returns
> EMULATION_OK when rc == X86EMUL_PROPAGATE_FAULT).

You need comments from maintainers.

	Takuya

Re: [PATCH 2/2] KVM: Fix writeback on page boundary that propagate changes in spite of #PF

[Gleb Natapov]

On Wed, Jan 11, 2012 at 06:53:31PM +0200, Nadav Amit wrote:
> Consider the case in which an instruction emulation writeback is performed on a page boundary.
> In such case, if a #PF occurs on the second page, the write to the first page already occurred and cannot be retracted.
> Therefore, validation of the second page access must be performed prior to writeback.
> 
> Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
> ---
>  arch/x86/kvm/x86.c |   13 +++++++++++++
>  1 files changed, 13 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 05fd3d7..7af3d67 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -3626,6 +3626,8 @@ struct read_write_emulator_ops {
>  			       int bytes, void *val);
>  	int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
>  				    void *val, int bytes);
> +	gpa_t (*read_write_validate)(struct kvm_vcpu *vcpu, gva_t gva,
> +				     struct x86_exception *exception);
>  	bool write;
>  };
>  
> @@ -3686,6 +3688,7 @@ static struct read_write_emulator_ops write_emultor = {
>  	.read_write_emulate = write_emulate,
>  	.read_write_mmio = write_mmio,
>  	.read_write_exit_mmio = write_exit_mmio,
> +	.read_write_validate = kvm_mmu_gva_to_gpa_write,
>  	.write = true,
>  };
>  
> @@ -3750,6 +3753,16 @@ int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
>  		int rc, now;
>  
>  		now = -addr & ~PAGE_MASK;
> +
> +		/* First check there is no page-fault on the next page */
> +		if (ops->read_write_validate &&
> +			ops->read_write_validate(vcpu, addr+now, exception) ==
> +			UNMAPPED_GVA) {
> +			/* #PF on the first page should be reported first */
> +			ops->read_write_validate(vcpu, addr, exception);
> +			return X86EMUL_PROPAGATE_FAULT;
> +		}
> +
This undoes optimization that vcpu_mmio_gva_to_gpa() has for handling
mmio. Furthermore for common (non faulting) case we will check page
tables twice on each write that crosses page boundary, first time here
and second time in emulator_read_write_onepage().

>  		rc = emulator_read_write_onepage(addr, val, now, exception,
>  						 vcpu, ops);
>  
> -- 
> 1.7.4.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

--
			Gleb.

Re: [PATCH 1/2] KVM: Exception during emulation decode should propagate

[Avi Kivity]

On 01/12/2012 11:07 AM, Nadav Amit wrote:
> >> 
> >> When I cleaned up insn_fetch(), I thought that fetching the instruction
> >> which is being executed by the guest cannot cause #PF.
> >> 
> >> The possibility that a meaningless userspace might similtaneously unmap
> >> the page, noted by Avi IIRC, was ignored intentionally, so we just fail
> >> in such a case.
> >> 
> >> Did you see any real problem?
>
> Well, I run some research project for which I emulate instructions quite
> often. I do see a real problem with Linux 3.0.0. Please note AFAIK #GP
> might occur as well during instruction fetch. I don't think failing is the
> right behavior in such case - there is no real reason to fail.
>
> Please tell me whether you are OK with KVM failing in such a scenario.

So long as it's just the guest who is affected (at the same privilege
level; we don't want guest userspace to cause a host failure).

We might have issues with userspace causing such a failure, or a nested
guest.  I see we already check for that in handle_emulation_failure()
(but not userspace).

> If not - I'll send an updated patch (in which x86_decode_insn returns
> EMULATION_OK when rc == X86EMUL_PROPAGATE_FAULT).

It guess it's better to be correct in the emulator than rely on a
failure allowing for guest-internal DoS.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 2/2] KVM: Fix writeback on page boundary that propagate changes in spite of #PF

[Avi Kivity]

On 01/12/2012 12:12 PM, Gleb Natapov wrote:
> On Wed, Jan 11, 2012 at 06:53:31PM +0200, Nadav Amit wrote:
> > Consider the case in which an instruction emulation writeback is performed on a page boundary.
> > In such case, if a #PF occurs on the second page, the write to the first page already occurred and cannot be retracted.
> > Therefore, validation of the second page access must be performed prior to writeback.
> > 
> > Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
> > ---
> >  arch/x86/kvm/x86.c |   13 +++++++++++++
> >  1 files changed, 13 insertions(+), 0 deletions(-)
> > 
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index 05fd3d7..7af3d67 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -3626,6 +3626,8 @@ struct read_write_emulator_ops {
> >  			       int bytes, void *val);
> >  	int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
> >  				    void *val, int bytes);
> > +	gpa_t (*read_write_validate)(struct kvm_vcpu *vcpu, gva_t gva,
> > +				     struct x86_exception *exception);
> >  	bool write;
> >  };
> >  
> > @@ -3686,6 +3688,7 @@ static struct read_write_emulator_ops write_emultor = {
> >  	.read_write_emulate = write_emulate,
> >  	.read_write_mmio = write_mmio,
> >  	.read_write_exit_mmio = write_exit_mmio,
> > +	.read_write_validate = kvm_mmu_gva_to_gpa_write,
> >  	.write = true,
> >  };
> >  
> > @@ -3750,6 +3753,16 @@ int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
> >  		int rc, now;
> >  
> >  		now = -addr & ~PAGE_MASK;
> > +
> > +		/* First check there is no page-fault on the next page */
> > +		if (ops->read_write_validate &&
> > +			ops->read_write_validate(vcpu, addr+now, exception) ==
> > +			UNMAPPED_GVA) {
> > +			/* #PF on the first page should be reported first */
> > +			ops->read_write_validate(vcpu, addr, exception);
> > +			return X86EMUL_PROPAGATE_FAULT;
> > +		}
> > +
> This undoes optimization that vcpu_mmio_gva_to_gpa() has for handling
> mmio. 

Right.  I suggest changing I/O to have two phases: first, translate the
virtual address into an array of two physical addresses; check
exceptions and report.  Then do the actual writes.

> Furthermore for common (non faulting) case we will check page
> tables twice on each write that crosses page boundary, first time here
> and second time in emulator_read_write_onepage().

Those should be very uncommon.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 2/2] KVM: Fix writeback on page boundary that propagate changes in spite of #PF

[Gleb Natapov]

On Thu, Jan 12, 2012 at 12:21:00PM +0200, Avi Kivity wrote:
> On 01/12/2012 12:12 PM, Gleb Natapov wrote:
> > On Wed, Jan 11, 2012 at 06:53:31PM +0200, Nadav Amit wrote:
> > > Consider the case in which an instruction emulation writeback is performed on a page boundary.
> > > In such case, if a #PF occurs on the second page, the write to the first page already occurred and cannot be retracted.
> > > Therefore, validation of the second page access must be performed prior to writeback.
> > > 
> > > Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
> > > ---
> > >  arch/x86/kvm/x86.c |   13 +++++++++++++
> > >  1 files changed, 13 insertions(+), 0 deletions(-)
> > > 
> > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > > index 05fd3d7..7af3d67 100644
> > > --- a/arch/x86/kvm/x86.c
> > > +++ b/arch/x86/kvm/x86.c
> > > @@ -3626,6 +3626,8 @@ struct read_write_emulator_ops {
> > >  			       int bytes, void *val);
> > >  	int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
> > >  				    void *val, int bytes);
> > > +	gpa_t (*read_write_validate)(struct kvm_vcpu *vcpu, gva_t gva,
> > > +				     struct x86_exception *exception);
> > >  	bool write;
> > >  };
> > >  
> > > @@ -3686,6 +3688,7 @@ static struct read_write_emulator_ops write_emultor = {
> > >  	.read_write_emulate = write_emulate,
> > >  	.read_write_mmio = write_mmio,
> > >  	.read_write_exit_mmio = write_exit_mmio,
> > > +	.read_write_validate = kvm_mmu_gva_to_gpa_write,
> > >  	.write = true,
> > >  };
> > >  
> > > @@ -3750,6 +3753,16 @@ int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
> > >  		int rc, now;
> > >  
> > >  		now = -addr & ~PAGE_MASK;
> > > +
> > > +		/* First check there is no page-fault on the next page */
> > > +		if (ops->read_write_validate &&
> > > +			ops->read_write_validate(vcpu, addr+now, exception) ==
> > > +			UNMAPPED_GVA) {
> > > +			/* #PF on the first page should be reported first */
> > > +			ops->read_write_validate(vcpu, addr, exception);
> > > +			return X86EMUL_PROPAGATE_FAULT;
> > > +		}
> > > +
> > This undoes optimization that vcpu_mmio_gva_to_gpa() has for handling
> > mmio. 
> 
> Right.  I suggest changing I/O to have two phases: first, translate the
> virtual address into an array of two physical addresses; check
> exceptions and report.  Then do the actual writes.
> 
> > Furthermore for common (non faulting) case we will check page
> > tables twice on each write that crosses page boundary, first time here
> > and second time in emulator_read_write_onepage().
> 
> Those should be very uncommon.
> 
Still it is better to have all the checks in one place like you suggest
above.

--
			Gleb.

Re: [PATCH 2/2] KVM: Fix writeback on page boundary that propagate changes in spite of #PF

[Nadav Amit]

On Jan 12, 2012, at 12:27 PM, Gleb Natapov wrote:

> On Thu, Jan 12, 2012 at 12:21:00PM +0200, Avi Kivity wrote:
>> On 01/12/2012 12:12 PM, Gleb Natapov wrote:
>>> On Wed, Jan 11, 2012 at 06:53:31PM +0200, Nadav Amit wrote:
>>>> Consider the case in which an instruction emulation writeback is performed on a page boundary.
>>>> In such case, if a #PF occurs on the second page, the write to the first page already occurred and cannot be retracted.
>>>> Therefore, validation of the second page access must be performed prior to writeback.
>>>> 
>>>> Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
>>>> ---
>>>> arch/x86/kvm/x86.c |   13 +++++++++++++
>>>> 1 files changed, 13 insertions(+), 0 deletions(-)
>>>> 
>>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>>> index 05fd3d7..7af3d67 100644
>>>> --- a/arch/x86/kvm/x86.c
>>>> +++ b/arch/x86/kvm/x86.c
>>>> @@ -3626,6 +3626,8 @@ struct read_write_emulator_ops {
>>>> 			       int bytes, void *val);
>>>> 	int (*read_write_exit_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa,
>>>> 				    void *val, int bytes);
>>>> +	gpa_t (*read_write_validate)(struct kvm_vcpu *vcpu, gva_t gva,
>>>> +				     struct x86_exception *exception);
>>>> 	bool write;
>>>> };
>>>> 
>>>> @@ -3686,6 +3688,7 @@ static struct read_write_emulator_ops write_emultor = {
>>>> 	.read_write_emulate = write_emulate,
>>>> 	.read_write_mmio = write_mmio,
>>>> 	.read_write_exit_mmio = write_exit_mmio,
>>>> +	.read_write_validate = kvm_mmu_gva_to_gpa_write,
>>>> 	.write = true,
>>>> };
>>>> 
>>>> @@ -3750,6 +3753,16 @@ int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
>>>> 		int rc, now;
>>>> 
>>>> 		now = -addr & ~PAGE_MASK;
>>>> +
>>>> +		/* First check there is no page-fault on the next page */
>>>> +		if (ops->read_write_validate &&
>>>> +			ops->read_write_validate(vcpu, addr+now, exception) ==
>>>> +			UNMAPPED_GVA) {
>>>> +			/* #PF on the first page should be reported first */
>>>> +			ops->read_write_validate(vcpu, addr, exception);
>>>> +			return X86EMUL_PROPAGATE_FAULT;
>>>> +		}
>>>> +
>>> This undoes optimization that vcpu_mmio_gva_to_gpa() has for handling
>>> mmio. 
>> 
>> Right.  I suggest changing I/O to have two phases: first, translate the
>> virtual address into an array of two physical addresses; check
>> exceptions and report.  Then do the actual writes.
>> 
>>> Furthermore for common (non faulting) case we will check page
>>> tables twice on each write that crosses page boundary, first time here
>>> and second time in emulator_read_write_onepage().
>> 
>> Those should be very uncommon.
>> 
> Still it is better to have all the checks in one place like you suggest
> above.

I agree. I'll submit a revised version soon.
I'm just struggling with this emulation code as more stuff is broken.

Regards,
Nadav