question on #UD emulation

question on #UD emulation

[Dong, Eddie]

Avi/Anthony:
	Following commit added #UD trap in KVM and do emulation in KVM.
It is good to avoid the mis hypercall instruction issue, but I am not
sure if it will have any serious side effect. I am blind to AMD
instruction, I assume, for example, AMD 3D now instruction may use
different binary and thus invalid_ops in Intel platform. If an
application with 3D now instructions is executed in Intel platform, now
KVM get #UD and may do emulation inside, will that cause problem rather
then we inject #UD to guest? If this suspicion is true, then probably we
need to do something more to this commitment, i.e. to emulate
cross-architecturally VMCALL instruction only for this purpose.

BTW, do we need to support live migrating an AMD VM to Intel platform or
vice versa?

thx,eddie



commit 8db6e3d54971e76019b02a6ad860c9df35218dfc
Date:   Mon Sep 17 14:57:50 2007 -0500

    KVM: Refactor hypercall infrastructure (v3)

    This patch refactors the current hypercall infrastructure to better
    support live migration and SMP.  It eliminates the hypercall page by
    trapping the UD exception that would occur if you used the wrong
hypercall
    instruction for the underlying architecture and replacing it with
the right
    one lazily.

    A fall-out of this patch is that the unhandled hypercalls no longer
trap to
    userspace.  There is very little reason though to use a hypercall to
    communicate with userspace as PIO or MMIO can be used.  There is no
code
    in tree that uses userspace hypercalls.

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: question on #UD emulation

[Avi Kivity]

Dong, Eddie wrote:
> Avi/Anthony:
> 	Following commit added #UD trap in KVM and do emulation in KVM.
> It is good to avoid the mis hypercall instruction issue, but I am not
> sure if it will have any serious side effect. I am blind to AMD
> instruction, I assume, for example, AMD 3D now instruction may use
> different binary and thus invalid_ops in Intel platform. If an
> application with 3D now instructions is executed in Intel platform, now
> KVM get #UD and may do emulation inside, will that cause problem rather
> then we inject #UD to guest? If this suspicion is true, then probably we
> need to do something more to this commitment, i.e. to emulate
> cross-architecturally VMCALL instruction only for this purpose.
>
>   

Since we don't emulate 3Dnow instructions, a #UD will be injected instead.

The plan for having mixed vendor migration is to disable such extensions 
in cpuid, so the guest won't even try to execute them.  In other words, 
the cpuid in a virtualization farm will be set to the greatest common 
subset of cpu features.


> BTW, do we need to support live migrating an AMD VM to Intel platform or
> vice versa

Yes, it allows users to buy the machines with the best cost/performance 
ratio, rather than continue to buy machines from the same vendor all the 
time.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: question on #UD emulation

[Joerg Roedel]

On Sun, Dec 16, 2007 at 01:32:40PM +0200, Avi Kivity wrote:
> Dong, Eddie wrote:
> 
> > BTW, do we need to support live migrating an AMD VM to Intel platform or
> > vice versa
> 
> Yes, it allows users to buy the machines with the best cost/performance 
> ratio, rather than continue to buy machines from the same vendor all the 
> time.

I don't agree with this. A user can still buy machines from both vendors
and migrate offline it he wants to. In general I think that allowing
live migration between Intel and AMD hosts is not a good idea because
you limit the guest to the subset of the CPU features available on both
platforms. This disallows many optimizations and costs performance for
the guest.

Take the problem Dor mentioned this week about the performance impact of
the gettimeofday() syscall causing many cpuid guest exits. If we had
not to support migration between AMD and Intel we could simply propagate
FEATURE_RDTSCP to an guest on AMD an SYNC_RDTSC on Intel platforms. The
Linux get_cycles_sync() function would then no longer execute CPUID.
This is only one example, I found some more during my work on KVM.

Joerg

-- 
           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy



-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: question on #UD emulation

[Dor Laor]

[-- Attachment #1.1: Type: text/plain, Size: 2677 bytes --]

Joerg Roedel wrote:
>
> On Sun, Dec 16, 2007 at 01:32:40PM +0200, Avi Kivity wrote:
> > Dong, Eddie wrote:
> >
> > > BTW, do we need to support live migrating an AMD VM to Intel 
> platform or
> > > vice versa
> >
> > Yes, it allows users to buy the machines with the best cost/performance
> > ratio, rather than continue to buy machines from the same vendor all the
> > time.
>
> I don't agree with this. A user can still buy machines from both vendors
> and migrate offline it he wants to. In general I think that allowing
> live migration between Intel and AMD hosts is not a good idea because
> you limit the guest to the subset of the CPU features available on both
> platforms. This disallows many optimizations and costs performance for
> the guest.
>
Having the option to live migrate between various hosts is a powerful 
feature.
Of course it costs some performance penalty but it might not be relevant 
for the user
work load.
By having cmd line parameters to control the cpuid and cpu type one can 
dynamically calculate
the lowest common denominator and to run his virtual machines using it.
One can always pass -cpu=host to have maximum performance.
For users who like to live migrate between Amd-Intel we'll use one of 
cpu types
for both.
Regards,
Dor
>
> Take the problem Dor mentioned this week about the performance impact of
> the gettimeofday() syscall causing many cpuid guest exits. If we had
> not to support migration between AMD and Intel we could simply propagate
> FEATURE_RDTSCP to an guest on AMD an SYNC_RDTSC on Intel platforms. The
> Linux get_cycles_sync() function would then no longer execute CPUID.
> This is only one example, I found some more during my work on KVM.
>
> Joerg
>
> --
>            |           AMD Saxony Limited Liability Company & Co. KG
>  Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
>  System    |                  Register Court Dresden: HRA 4896
>  Research  |              General Partner authorized to represent:
>  Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
>            | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, 
> Thomas McCoy
>
>
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>


[-- Attachment #1.2: Type: text/html, Size: 4358 bytes --]

[-- Attachment #2: Type: text/plain, Size: 308 bytes --]

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: question on #UD emulation

[Dong, Eddie]

[-- Attachment #1: Type: text/plain, Size: 3564 bytes --]

A little bit suspicion for the usage model too :(
The CPUID feature list can't be changed dynamically, so I doutb
if a real user will give up performance at beginning when he/she
starts the guest for "maybe" future migration cross host platform.

A VT or SVM enabled machine usually has many features 
which doesn't exist in the common feature list for both Intel & AMD,
such as MMX/SSE2/SSE3. So the performance penalty could be big.

Eddie
 


________________________________

	From: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org [mailto:kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org] On Behalf Of Dor Laor
	Sent: 2007年12月16日 20:50
	To: Joerg Roedel
	Cc: kvm-devel; Avi Kivity
	Subject: Re: [kvm-devel] question on #UD emulation
	
	
	Joerg Roedel wrote: 

		On Sun, Dec 16, 2007 at 01:32:40PM +0200, Avi Kivity wrote:
		> Dong, Eddie wrote:
		>
		> > BTW, do we need to support live migrating an AMD VM to Intel platform or
		> > vice versa
		>
		> Yes, it allows users to buy the machines with the best cost/performance
		> ratio, rather than continue to buy machines from the same vendor all the
		> time.
		
		I don't agree with this. A user can still buy machines from both vendors
		and migrate offline it he wants to. In general I think that allowing
		live migration between Intel and AMD hosts is not a good idea because
		you limit the guest to the subset of the CPU features available on both
		platforms. This disallows many optimizations and costs performance for
		the guest.
		
		

	Having the option to live migrate between various hosts is a powerful feature.
	Of course it costs some performance penalty but it might not be relevant for the user
	work load.
	By having cmd line parameters to control the cpuid and cpu type one can dynamically calculate
	the lowest common denominator and to run his virtual machines using it.
	One can always pass -cpu=host to have maximum performance.
	For users who like to live migrate between Amd-Intel we'll use one of cpu types
	for both.
	Regards,
	Dor
	

		Take the problem Dor mentioned this week about the performance impact of
		the gettimeofday() syscall causing many cpuid guest exits. If we had
		not to support migration between AMD and Intel we could simply propagate
		FEATURE_RDTSCP to an guest on AMD an SYNC_RDTSC on Intel platforms. The
		Linux get_cycles_sync() function would then no longer execute CPUID.
		This is only one example, I found some more during my work on KVM.
		
		Joerg
		
		--
		           |           AMD Saxony Limited Liability Company & Co. KG
		 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
		 System    |                  Register Court Dresden: HRA 4896
		 Research  |              General Partner authorized to represent:
		 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
		           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
		
		
		
		-------------------------------------------------------------------------
		SF.Net email is sponsored by:
		Check out the new SourceForge.net Marketplace.
		It's the best place to buy or sell services
		for just about anything Open Source.
		http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
		_______________________________________________
		kvm-devel mailing list
		kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
		https://lists.sourceforge.net/lists/listinfo/kvm-devel
		




[-- Attachment #2: Type: text/plain, Size: 308 bytes --]

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: question on #UD emulation

[Dong, Eddie]

   
>
>Since we don't emulate 3Dnow instructions, a #UD will be 
>injected instead.
>
OK, how many architecture specific instructiosn are emulated now?
Or what is planed to be implemented? If no, then adding those 
emulation in KVM just means dead code.
thx,eddie

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: question on #UD emulation

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 672 bytes --]

Dong, Eddie wrote:
> A little bit suspicion for the usage model too :(
> The CPUID feature list can't be changed dynamically, so I doutb
> if a real user will give up performance at beginning when he/she
> starts the guest for "maybe" future migration cross host platform.
>
> A VT or SVM enabled machine usually has many features 
> which doesn't exist in the common feature list for both Intel & AMD,
> such as MMX/SSE2/SSE3. So the performance penalty could be big.
>   

mmx/sse2/sse3 are actually common across intel/amd; the performance
difference should be minor (well, depending on the workload).

-- 
error compiling committee.c: too many arguments to function



[-- Attachment #2: Type: text/plain, Size: 308 bytes --]

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: question on #UD emulation

[Avi Kivity]

Dong, Eddie wrote:
>    
>   
>> Since we don't emulate 3Dnow instructions, a #UD will be 
>> injected instead.
>>
>>     
> OK, how many architecture specific instructiosn are emulated now?
>   

Just vmcall/vmmcall (which is emulated once, then patched).

> Or what is planed to be implemented? If no, then adding those 
> emulation in KVM just means dead code.

sysenter/sysexit (which aren't implemented in amd compatibility mode) 
and syscall/sysret (which aren't implemented in intel compatibility mode).

Both are only used for 32-bit userspace on 64-bit guests, so they aren't 
too critical.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace