%fs PDA & KVM

%fs PDA & KVM

[Ingo Molnar]

hm, i just noticed that in vmx_vcpu_run() KVM restores %fs quite late. 

This is a bit awkward under v2.6.21 x86 because there we use %fs for the 
KERNEL_PDA area. Things like 'current' rely on it. So maybe we should 
just do this unconditionally in assembly:

        movl $(__KERNEL_PDA), %edx;
        movl %edx, %fs

it's also a problem with -rt too, which has that whole codepath up to 
the load_fs() call preemptible.

i'm also wondering about this bit:

        kvm_run->exit_type = 0;
        if (fail) {
                kvm_run->exit_type = KVM_EXIT_TYPE_FAIL_ENTRY;
                kvm_run->exit_reason = vmcs_read32(VM_INSTRUCTION_ERROR);
                kvm_cr3_cache_sync(vcpu);
                r = 0;
        } else {
                if (fs_gs_ldt_reload_needed) {
                        load_ldt(ldt_sel);
                        load_fs(fs_sel);

are you sure we dont need an FS reload in the 'fail' case?

	Ingo

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: %fs PDA & KVM

[Avi Kivity]

Ingo Molnar wrote:
> hm, i just noticed that in vmx_vcpu_run() KVM restores %fs quite late. 
>
> This is a bit awkward under v2.6.21 x86 because there we use %fs for the 
> KERNEL_PDA area. Things like 'current' rely on it. So maybe we should 
> just do this unconditionally in assembly:
>
>         movl $(__KERNEL_PDA), %edx;
>         movl %edx, %fs
>   

Sure, but with %dx instead of %edx, please.

> it's also a problem with -rt too, which has that whole codepath up to 
> the load_fs() call preemptible.
>
> i'm also wondering about this bit:
>
>         kvm_run->exit_type = 0;
>         if (fail) {
>                 kvm_run->exit_type = KVM_EXIT_TYPE_FAIL_ENTRY;
>                 kvm_run->exit_reason = vmcs_read32(VM_INSTRUCTION_ERROR);
>                 kvm_cr3_cache_sync(vcpu);
>                 r = 0;
>         } else {
>                 if (fs_gs_ldt_reload_needed) {
>                         load_ldt(ldt_sel);
>                         load_fs(fs_sel);
>
> are you sure we dont need an FS reload in the 'fail' case?
>   

The manual is not explicit about it, but I think that an entry failure 
during loading of guest state _can_ cause fs to be loaded, so yes, that 
path is missing the segment reloads.

Non -rt kvm cannot preempt there even with CONFIG_PREEMPT=y.  Can 
interrupts reference current?  If so, that fix is needed badly for .21.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: %fs PDA & KVM

[Ingo Molnar]

* Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:

> >i'm also wondering about this bit:
> >
> >        kvm_run->exit_type = 0;
> >        if (fail) {
> >                kvm_run->exit_type = KVM_EXIT_TYPE_FAIL_ENTRY;
> >                kvm_run->exit_reason = vmcs_read32(VM_INSTRUCTION_ERROR);
> >                kvm_cr3_cache_sync(vcpu);
> >                r = 0;
> >        } else {
> >                if (fs_gs_ldt_reload_needed) {
> >                        load_ldt(ldt_sel);
> >                        load_fs(fs_sel);
> >
> >are you sure we dont need an FS reload in the 'fail' case?
> >  
> 
> The manual is not explicit about it, but I think that an entry failure 
> during loading of guest state _can_ cause fs to be loaded, so yes, 
> that path is missing the segment reloads.

ok, i'll send a patch.

> Non -rt kvm cannot preempt there even with CONFIG_PREEMPT=y.  Can 
> interrupts reference current?  If so, that fix is needed badly for 
> .21.

interrupts can reference 'current' (although it's generally not 
encouraged) - for example scheduler_tick(), but the IRQ entry code loads 
%fs anyway via SAVE_ALL, so it's not an issue.

	Ingo

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[patch] KVM: always reload segment selectors

[Ingo Molnar]

Subject: [patch] KVM: always reload segment selectors
From: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>

failed VM entry on VMX might still change %fs or %gs, thus make sure 
that KVM always reloads the segment selectors. This is crutial on both 
x86 and x86_64: x86 has __KERNEL_PDA in %fs on which things like 
'current' depends and x86_64 has 0 there and needs MSR_GS_BASE to work.

Signed-off-by: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>
---
 drivers/kvm/vmx.c |   37 +++++++++++++++++++++----------------
 1 file changed, 21 insertions(+), 16 deletions(-)

Index: linux/drivers/kvm/vmx.c
===================================================================
--- linux.orig/drivers/kvm/vmx.c
+++ linux/drivers/kvm/vmx.c
@@ -1896,6 +1896,27 @@ again:
 		[cr2]"i"(offsetof(struct kvm_vcpu, cr2))
 	      : "cc", "memory" );
 
+	/*
+	 * Reload segment selectors ASAP. (it's needed for a functional
+	 * kernel: x86 relies on having __KERNEL_PDA in %fs and x86_64
+	 * relies on having 0 in %gs for the CPU PDA to work.)
+	 */
+	if (fs_gs_ldt_reload_needed) {
+		load_ldt(ldt_sel);
+		load_fs(fs_sel);
+		/*
+		 * If we have to reload gs, we must take care to
+		 * preserve our gs base.
+		 */
+		local_irq_disable();
+		load_gs(gs_sel);
+#ifdef CONFIG_X86_64
+		wrmsrl(MSR_GS_BASE, vmcs_readl(HOST_GS_BASE));
+#endif
+		local_irq_enable();
+
+		reload_tss();
+	}
 	++kvm_stat.exits;
 
 	save_msrs(vcpu->guest_msrs, NR_BAD_MSRS);
@@ -1913,22 +1934,6 @@ again:
 		kvm_run->exit_reason = vmcs_read32(VM_INSTRUCTION_ERROR);
 		r = 0;
 	} else {
-		if (fs_gs_ldt_reload_needed) {
-			load_ldt(ldt_sel);
-			load_fs(fs_sel);
-			/*
-			 * If we have to reload gs, we must take care to
-			 * preserve our gs base.
-			 */
-			local_irq_disable();
-			load_gs(gs_sel);
-#ifdef CONFIG_X86_64
-			wrmsrl(MSR_GS_BASE, vmcs_readl(HOST_GS_BASE));
-#endif
-			local_irq_enable();
-
-			reload_tss();
-		}
 		/*
 		 * Profile KVM exit RIPs:
 		 */

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [patch] KVM: always reload segment selectors

[Avi Kivity]

Ingo Molnar wrote:
> Subject: [patch] KVM: always reload segment selectors
> From: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>
>
> failed VM entry on VMX might still change %fs or %gs, thus make sure 
> that KVM always reloads the segment selectors. This is crutial on both 
> x86 and x86_64: x86 has __KERNEL_PDA in %fs on which things like 
> 'current' depends and x86_64 has 0 there and needs MSR_GS_BASE to work.
>
> Signed-off-by: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>
> ---
>  drivers/kvm/vmx.c |   37 +++++++++++++++++++++----------------
>  1 file changed, 21 insertions(+), 16 deletions(-)
>
> Index: linux/drivers/kvm/vmx.c
> ===================================================================
> --- linux.orig/drivers/kvm/vmx.c
> +++ linux/drivers/kvm/vmx.c
> @@ -1896,6 +1896,27 @@ again:
>  		[cr2]"i"(offsetof(struct kvm_vcpu, cr2))
>  	      : "cc", "memory" );
>  
> +	/*
> +	 * Reload segment selectors ASAP. (it's needed for a functional
> +	 * kernel: x86 relies on having __KERNEL_PDA in %fs and x86_64
> +	 * relies on having 0 in %gs for the CPU PDA to work.)
> +	 */
> +	if (fs_gs_ldt_reload_needed) {
> +		load_ldt(ldt_sel);
> +		load_fs(fs_sel);
> +		/*
> +		 * If we have to reload gs, we must take care to
> +		 * preserve our gs base.
> +		 */
> +		local_irq_disable();
> +		load_gs(gs_sel);
> +#ifdef CONFIG_X86_64
> +		wrmsrl(MSR_GS_BASE, vmcs_readl(HOST_GS_BASE));
> +#endif
> +		local_irq_enable();
> +
> +		reload_tss();
> +	}
>  	++kvm_stat.exits;
>  
>  	save_msrs(vcpu->guest_msrs, NR_BAD_MSRS);

btw, looking at the code, we could just remove fs from the 
fs_gs_reload_needed and make in unconditional.  VT knows how to reload 
segments, except if they're user segments (groan).  In the case of fs, 
if it's used for the pda, it's obviously a kernel segment.

gs is different: since only the segment base is loaded (via swapgs), the 
selector part could well be a userspace selector, and thus the 
irq-protected reload is needed.

Anyway, I'm applying the patch as the above discourse is irrelevant to 
the fix.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: %fs PDA & KVM

[Rusty Russell]

On Sun, 2007-03-11 at 13:24 +0100, Ingo Molnar wrote:
> hm, i just noticed that in vmx_vcpu_run() KVM restores %fs quite late. 
> 
> This is a bit awkward under v2.6.21 x86 because there we use %fs for the 
> KERNEL_PDA area. Things like 'current' rely on it. So maybe we should 
> just do this unconditionally in assembly:
> 
>         movl $(__KERNEL_PDA), %edx;
>         movl %edx, %fs

Note that at some stage as Andi merges the pda->percpu conversion, this
will need to be replaced by __KERNEL_PERCPU.  Trivial change.

Cheers,
Rusty.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV