[PATCH 0/3] kvm: fix cr0 initialization on SIPI reset

[PATCH 0/3] kvm: fix cr0 initialization on SIPI reset

[Eduardo Habkost]

Hi,

The following patches fix a bug on the SIPI reset code for SVM. cr0 was not
being reset properly, making KVM keep the vcpu on paging mode, thus not
being able to run the real-mode boostrap code. This bug was reported at:
https://bugzilla.redhat.com/show_bug.cgi?id=525699

The first patch is cosmetic, and it just changes the vmx code to use the same
macros when initializing cr0, instead of a hardcoded hex value.

The patches were tested by running a RHEL-5.4 guest and and triggering the
SIPI reset by offlining and onlining cpus on the guest. They were tested on
both NPT-enabled and NPT-disabled cases.

-- 
Eduardo

[PATCH 1/3] kvm: vmx: use macros instead of hex value on cr0 initialization

[Eduardo Habkost]

This should have no effect, it is just to make the code clearer.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 arch/x86/kvm/vmx.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 364263a..42409cc 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -2538,7 +2538,7 @@ static int vmx_vcpu_reset(struct kvm_vcpu *vcpu)
 	if (vmx->vpid != 0)
 		vmcs_write16(VIRTUAL_PROCESSOR_ID, vmx->vpid);
 
-	vmx->vcpu.arch.cr0 = 0x60000010;
+	vmx->vcpu.arch.cr0 = X86_CR0_NW | X86_CR0_CD | X86_CR0_ET;
 	vmx_set_cr0(&vmx->vcpu, vmx->vcpu.arch.cr0); /* enter rmode */
 	vmx_set_cr4(&vmx->vcpu, 0);
 	vmx_set_efer(&vmx->vcpu, 0);
-- 
1.6.3.rc4.29.g8146

[PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Eduardo Habkost]

svm_vcpu_reset() was not properly resetting the contents of the guest-visible
cr0 register, causing the following issue:
https://bugzilla.redhat.com/show_bug.cgi?id=525699

Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
with paging enabled, making the vcpu get a pagefault exception while trying to
run it.

Instead of setting vmcb->save.cr0 directly, the new code just resets
kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().

kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 arch/x86/kvm/svm.c |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 170b2d9..6b86a11 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -626,11 +626,12 @@ static void init_vmcb(struct vcpu_svm *svm)
 	save->rip = 0x0000fff0;
 	svm->vcpu.arch.regs[VCPU_REGS_RIP] = save->rip;
 
-	/*
-	 * cr0 val on cpu init should be 0x60000010, we enable cpu
-	 * cache by default. the orderly way is to enable cache in bios.
+	/* This is the guest-visible cr0 value.
+	 * svm_set_cr0() sets PG and WP and clears NW and CD on save->cr0.
 	 */
-	save->cr0 = 0x00000010 | X86_CR0_PG | X86_CR0_WP;
+	svm->vcpu.arch.cr0 = X86_CR0_NW | X86_CR0_CD | X86_CR0_ET;
+	kvm_set_cr0(&svm->vcpu, svm->vcpu.arch.cr0);
+
 	save->cr4 = X86_CR4_PAE;
 	/* rdx = ?? */
 
-- 
1.6.3.rc4.29.g8146

[PATCH 3/3] kvm: svm: init_vmcb(): remove redundant save->cr0 initialization

[Eduardo Habkost]

The svm_set_cr0() call will initialize save->cr0 properly even when npt is
enabled, clearing the NW and CD bits as expected, so we don't need to
initialize it manually for npt_enabled anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 arch/x86/kvm/svm.c |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 6b86a11..cd32ea7 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -646,8 +646,6 @@ static void init_vmcb(struct vcpu_svm *svm)
 		control->intercept_cr_write &= ~(INTERCEPT_CR0_MASK|
 						 INTERCEPT_CR3_MASK);
 		save->g_pat = 0x0007040600070406ULL;
-		/* enable caching because the QEMU Bios doesn't enable it */
-		save->cr0 = X86_CR0_ET;
 		save->cr3 = 0;
 		save->cr4 = 0;
 	}
-- 
1.6.3.rc4.29.g8146

Re: [PATCH 0/3] kvm: fix cr0 initialization on SIPI reset

[Avi Kivity]

On 10/24/2009 06:49 AM, Eduardo Habkost wrote:
> Hi,
>
> The following patches fix a bug on the SIPI reset code for SVM. cr0 was not
> being reset properly, making KVM keep the vcpu on paging mode, thus not
> being able to run the real-mode boostrap code. This bug was reported at:
> https://bugzilla.redhat.com/show_bug.cgi?id=525699
>
> The first patch is cosmetic, and it just changes the vmx code to use the same
> macros when initializing cr0, instead of a hardcoded hex value.
>
> The patches were tested by running a RHEL-5.4 guest and and triggering the
> SIPI reset by offlining and onlining cpus on the guest. They were tested on
> both NPT-enabled and NPT-disabled cases.
>    

Applied, thanks.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Alexander Graf]

Eduardo Habkost wrote:
> svm_vcpu_reset() was not properly resetting the contents of the guest-visible
> cr0 register, causing the following issue:
> https://bugzilla.redhat.com/show_bug.cgi?id=525699
>
> Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
> with paging enabled, making the vcpu get a pagefault exception while trying to
> run it.
>
> Instead of setting vmcb->save.cr0 directly, the new code just resets
> kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
> vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().
>
> kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
> kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.
>
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>   

Should this go into -stable?


Alex

Re: [PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Eduardo Habkost]

On Wed, Mar 17, 2010 at 07:17:32PM +0100, Alexander Graf wrote:
> Eduardo Habkost wrote:
> > svm_vcpu_reset() was not properly resetting the contents of the guest-visible
> > cr0 register, causing the following issue:
> > https://bugzilla.redhat.com/show_bug.cgi?id=525699
> >
> > Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
> > with paging enabled, making the vcpu get a pagefault exception while trying to
> > run it.
> >
> > Instead of setting vmcb->save.cr0 directly, the new code just resets
> > kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
> > vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().
> >
> > kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
> > kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.
> >
> > Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> >   
> 
> Should this go into -stable?

I think so. The patch is from October, was -stable branched before that?

-- 
Eduardo

Re: [PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Alexander Graf]

On 17.03.2010, at 22:42, Eduardo Habkost wrote:

> On Wed, Mar 17, 2010 at 07:17:32PM +0100, Alexander Graf wrote:
>> Eduardo Habkost wrote:
>>> svm_vcpu_reset() was not properly resetting the contents of the guest-visible
>>> cr0 register, causing the following issue:
>>> https://bugzilla.redhat.com/show_bug.cgi?id=525699
>>> 
>>> Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
>>> with paging enabled, making the vcpu get a pagefault exception while trying to
>>> run it.
>>> 
>>> Instead of setting vmcb->save.cr0 directly, the new code just resets
>>> kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
>>> vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().
>>> 
>>> kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
>>> kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.
>>> 
>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>> 
>> 
>> Should this go into -stable?
> 
> I think so. The patch is from October, was -stable branched before that?

If I read the diff log correctly 2.6.32 kvm development was branched off end of July 2009. The important question is if this patch fixes a regression introduced by some speedup magic.


Alex

Re: [PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Eduardo Habkost]

On Wed, Mar 17, 2010 at 10:48:23PM +0100, Alexander Graf wrote:
> 
> On 17.03.2010, at 22:42, Eduardo Habkost wrote:
> 
> > On Wed, Mar 17, 2010 at 07:17:32PM +0100, Alexander Graf wrote:
> >> Eduardo Habkost wrote:
> >>> svm_vcpu_reset() was not properly resetting the contents of the guest-visible
> >>> cr0 register, causing the following issue:
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=525699
> >>> 
> >>> Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
> >>> with paging enabled, making the vcpu get a pagefault exception while trying to
> >>> run it.
> >>> 
> >>> Instead of setting vmcb->save.cr0 directly, the new code just resets
> >>> kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
> >>> vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().
> >>> 
> >>> kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
> >>> kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.
> >>> 
> >>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> >>> 
> >> 
> >> Should this go into -stable?
> > 
> > I think so. The patch is from October, was -stable branched before that?
> 
> If I read the diff log correctly 2.6.32 kvm development was branched
> off end of July 2009. The important question is if this patch fixes a
> regression introduced by some speedup magic.

I have just checked git history, and it looks like this is not a
regression. Before this patch, vcpu->cr0 (the guest-visible cr0 value)
was never reset on vcpu reset, but only vcpu->svm->vmcb->save.cr0 (the
actual cr0 value used by the CPU).

-- 
Eduardo

Re: [PATCH 2/3] kvm: svm: reset cr0 properly on vcpu reset

[Alexander Graf]

Eduardo Habkost wrote:
> On Wed, Mar 17, 2010 at 10:48:23PM +0100, Alexander Graf wrote:
>   
>> On 17.03.2010, at 22:42, Eduardo Habkost wrote:
>>
>>     
>>> On Wed, Mar 17, 2010 at 07:17:32PM +0100, Alexander Graf wrote:
>>>       
>>>> Eduardo Habkost wrote:
>>>>         
>>>>> svm_vcpu_reset() was not properly resetting the contents of the guest-visible
>>>>> cr0 register, causing the following issue:
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=525699
>>>>>
>>>>> Without resetting cr0 properly, the vcpu was running the SIPI bootstrap routine
>>>>> with paging enabled, making the vcpu get a pagefault exception while trying to
>>>>> run it.
>>>>>
>>>>> Instead of setting vmcb->save.cr0 directly, the new code just resets
>>>>> kvm->arch.cr0 and calls kvm_set_cr0(). The bits that were set/cleared on
>>>>> vmcb->save.cr0 (PG, WP, !CD, !NW) will be set properly by svm_set_cr0().
>>>>>
>>>>> kvm_set_cr0() is used instead of calling svm_set_cr0() directly to make sure
>>>>> kvm_mmu_reset_context() is called to reset the mmu to nonpaging mode.
>>>>>
>>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>>>>
>>>>>           
>>>> Should this go into -stable?
>>>>         
>>> I think so. The patch is from October, was -stable branched before that?
>>>       
>> If I read the diff log correctly 2.6.32 kvm development was branched
>> off end of July 2009. The important question is if this patch fixes a
>> regression introduced by some speedup magic.
>>     
>
> I have just checked git history, and it looks like this is not a
> regression. Before this patch, vcpu->cr0 (the guest-visible cr0 value)
> was never reset on vcpu reset, but only vcpu->svm->vmcb->save.cr0 (the
> actual cr0 value used by the CPU).
>   

Good to know. Thanks for looking into this!


Alex