[Qemu-devel] cpuid problem in upstream qemu with kvm

[Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

Hi!
I have a linux guest where init crashes during boot with invalid opcode error
with kernel 2.6.31, but works fine with kernel 2.6.32.
It also works fine even with 2.6.31 with qemu-kvm.

With Gleb's help, we figured out that the opcode in question is syscall, which
is unsupported by hardware on kvm with intel hosts, and is emulated in software
with intel hosts in 2.6.32 and up. AMD hosts support this natively.

The root cause: by default qemu with kvm enabled reports the following CPU ID:
               mst@robin:~$ less /proc/cpuinfo
               processor       : 0
               vendor_id       : AuthenticAMD
               cpu family      : 6
               model           : 2
               model name      : QEMU Virtual CPU version 0.11.50
               stepping        : 3
               cpu MHz         : 2493.679
               cache size      : 512 KB
               fdiv_bug        : no
               hlt_bug         : no
               f00f_bug        : no
               coma_bug        : no
               fpu             : yes
               fpu_exception   : yes
               cpuid level     : 4
               wp              : yes
               flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
               pse36 clflush mmx fxsr sse sse2 syscall lm up pni
               bogomips        : 4987.35
               clflush size    : 64
               power management:

Problem is, guest sees syscall+AMD vendor and concludes that it is ok to use
syscall opcode, which results in a crash (or at least, extra VMExits).
I understand from Avi that qemu-kvm solves this problem by reporting
the actual CPU vendor from host, to guest.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> Hi!
> I have a linux guest where init crashes during boot with invalid opcode error
> with kernel 2.6.31, but works fine with kernel 2.6.32.
> It also works fine even with 2.6.31 with qemu-kvm.
>
> With Gleb's help, we figured out that the opcode in question is syscall, which
> is unsupported by hardware on kvm with intel hosts, and is emulated in software
> with intel hosts in 2.6.32 and up. AMD hosts support this natively.
>
> The root cause: by default qemu with kvm enabled reports the following CPU ID:
>                mst@robin:~$ less /proc/cpuinfo
>                processor       : 0
>                vendor_id       : AuthenticAMD
>                cpu family      : 6
>                model           : 2
>                model name      : QEMU Virtual CPU version 0.11.50
>                stepping        : 3
>                cpu MHz         : 2493.679
>                cache size      : 512 KB
>                fdiv_bug        : no
>                hlt_bug         : no
>                f00f_bug        : no
>                coma_bug        : no
>                fpu             : yes
>                fpu_exception   : yes
>                cpuid level     : 4
>                wp              : yes
>                flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
>                pse36 clflush mmx fxsr sse sse2 syscall lm up pni
>                bogomips        : 4987.35
>                clflush size    : 64
>                power management:
>
> Problem is, guest sees syscall+AMD vendor and concludes that it is ok to use
> syscall opcode, which results in a crash (or at least, extra VMExits).
> I understand from Avi that qemu-kvm solves this problem by reporting
> the actual CPU vendor from host, to guest.
>   

The bit that's ugly about that is that we do not save/restore 
cpuid.vendor_id so that would break live migration.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 01:44:22PM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> Hi!
>> I have a linux guest where init crashes during boot with invalid opcode error
>> with kernel 2.6.31, but works fine with kernel 2.6.32.
>> It also works fine even with 2.6.31 with qemu-kvm.
>>
>> With Gleb's help, we figured out that the opcode in question is syscall, which
>> is unsupported by hardware on kvm with intel hosts, and is emulated in software
>> with intel hosts in 2.6.32 and up. AMD hosts support this natively.
>>
>> The root cause: by default qemu with kvm enabled reports the following CPU ID:
>>                mst@robin:~$ less /proc/cpuinfo
>>                processor       : 0
>>                vendor_id       : AuthenticAMD
>>                cpu family      : 6
>>                model           : 2
>>                model name      : QEMU Virtual CPU version 0.11.50
>>                stepping        : 3
>>                cpu MHz         : 2493.679
>>                cache size      : 512 KB
>>                fdiv_bug        : no
>>                hlt_bug         : no
>>                f00f_bug        : no
>>                coma_bug        : no
>>                fpu             : yes
>>                fpu_exception   : yes
>>                cpuid level     : 4
>>                wp              : yes
>>                flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
>>                pse36 clflush mmx fxsr sse sse2 syscall lm up pni
>>                bogomips        : 4987.35
>>                clflush size    : 64
>>                power management:
>>
>> Problem is, guest sees syscall+AMD vendor and concludes that it is ok to use
>> syscall opcode, which results in a crash (or at least, extra VMExits).
>> I understand from Avi that qemu-kvm solves this problem by reporting
>> the actual CPU vendor from host, to guest.
>>   
>
> The bit that's ugly about that is that we do not save/restore  
> cpuid.vendor_id so that would break live migration.
>
> Regards,
>
> Anthony Liguori

This is a feature.  It would be very wrong to save/restore cpuid IMO.
It must be part of machine config.

For example, if you want kvm to work with live migration you must either
use least common denominator cpu id, or use hardware from the same
vendor.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> On Mon, Dec 14, 2009 at 01:44:22PM -0600, Anthony Liguori wrote:
>   
>> Michael S. Tsirkin wrote:
>>     
>>> Hi!
>>> I have a linux guest where init crashes during boot with invalid opcode error
>>> with kernel 2.6.31, but works fine with kernel 2.6.32.
>>> It also works fine even with 2.6.31 with qemu-kvm.
>>>
>>> With Gleb's help, we figured out that the opcode in question is syscall, which
>>> is unsupported by hardware on kvm with intel hosts, and is emulated in software
>>> with intel hosts in 2.6.32 and up. AMD hosts support this natively.
>>>
>>> The root cause: by default qemu with kvm enabled reports the following CPU ID:
>>>                mst@robin:~$ less /proc/cpuinfo
>>>                processor       : 0
>>>                vendor_id       : AuthenticAMD
>>>                cpu family      : 6
>>>                model           : 2
>>>                model name      : QEMU Virtual CPU version 0.11.50
>>>                stepping        : 3
>>>                cpu MHz         : 2493.679
>>>                cache size      : 512 KB
>>>                fdiv_bug        : no
>>>                hlt_bug         : no
>>>                f00f_bug        : no
>>>                coma_bug        : no
>>>                fpu             : yes
>>>                fpu_exception   : yes
>>>                cpuid level     : 4
>>>                wp              : yes
>>>                flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
>>>                pse36 clflush mmx fxsr sse sse2 syscall lm up pni
>>>                bogomips        : 4987.35
>>>                clflush size    : 64
>>>                power management:
>>>
>>> Problem is, guest sees syscall+AMD vendor and concludes that it is ok to use
>>> syscall opcode, which results in a crash (or at least, extra VMExits).
>>> I understand from Avi that qemu-kvm solves this problem by reporting
>>> the actual CPU vendor from host, to guest.
>>>   
>>>       
>> The bit that's ugly about that is that we do not save/restore  
>> cpuid.vendor_id so that would break live migration.
>>
>> Regards,
>>
>> Anthony Liguori
>>     
>
> This is a feature.  It would be very wrong to save/restore cpuid IMO.
> It must be part of machine config.
>   

You can already do -cpu qemu64,vendor_id=AuthenticAMD or 
,vendor_id=GenuineIntel.

If you don't specify anything explicitly, we use AuthenticAMD as a 
default.  Detecting the host by default is problematic because then the 
same invocation on a different host would change the visible vendor_id.

I thought KVM emulates the syscall instruction?  I swear I've seen 
patches for that.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Mon, Dec 14, 2009 at 01:57:29PM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
> >On Mon, Dec 14, 2009 at 01:44:22PM -0600, Anthony Liguori wrote:
> >>Michael S. Tsirkin wrote:
> >>>Hi!
> >>>I have a linux guest where init crashes during boot with invalid opcode error
> >>>with kernel 2.6.31, but works fine with kernel 2.6.32.
> >>>It also works fine even with 2.6.31 with qemu-kvm.
> >>>
> >>>With Gleb's help, we figured out that the opcode in question is syscall, which
> >>>is unsupported by hardware on kvm with intel hosts, and is emulated in software
> >>>with intel hosts in 2.6.32 and up. AMD hosts support this natively.
> >>>
> >>>The root cause: by default qemu with kvm enabled reports the following CPU ID:
> >>>               mst@robin:~$ less /proc/cpuinfo
> >>>               processor       : 0
> >>>               vendor_id       : AuthenticAMD
> >>>               cpu family      : 6
> >>>               model           : 2
> >>>               model name      : QEMU Virtual CPU version 0.11.50
> >>>               stepping        : 3
> >>>               cpu MHz         : 2493.679
> >>>               cache size      : 512 KB
> >>>               fdiv_bug        : no
> >>>               hlt_bug         : no
> >>>               f00f_bug        : no
> >>>               coma_bug        : no
> >>>               fpu             : yes
> >>>               fpu_exception   : yes
> >>>               cpuid level     : 4
> >>>               wp              : yes
> >>>               flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> >>>               pse36 clflush mmx fxsr sse sse2 syscall lm up pni
> >>>               bogomips        : 4987.35
> >>>               clflush size    : 64
> >>>               power management:
> >>>
> >>>Problem is, guest sees syscall+AMD vendor and concludes that it is ok to use
> >>>syscall opcode, which results in a crash (or at least, extra VMExits).
> >>>I understand from Avi that qemu-kvm solves this problem by reporting
> >>>the actual CPU vendor from host, to guest.
> >>The bit that's ugly about that is that we do not save/restore
> >>cpuid.vendor_id so that would break live migration.
> >>
> >>Regards,
> >>
> >>Anthony Liguori
> >
> >This is a feature.  It would be very wrong to save/restore cpuid IMO.
> >It must be part of machine config.
> 
> You can already do -cpu qemu64,vendor_id=AuthenticAMD or
> ,vendor_id=GenuineIntel.
> 
> If you don't specify anything explicitly, we use AuthenticAMD as a
> default.  Detecting the host by default is problematic because then
> the same invocation on a different host would change the visible
> vendor_id.
> 
> I thought KVM emulates the syscall instruction?  I swear I've seen
> patches for that.
> 
It is. Starting from 2.6.32.

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Gleb Natapov wrote:
>> I thought KVM emulates the syscall instruction?  I swear I've seen
>> patches for that.
>>
>>     
> It is. Starting from 2.6.32.
>   

Okay, so this is a performance vs. migration compatibility issue then?

BTW, couldn't we just not advertise syscall in cpuid?  That should fix 
it too without sacrificing migration compatibility.  We get a slight 
slowdown on AMD hosts but that's probably minor compared to the cost of 
using emulated syscall on Intel hosts.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Mon, Dec 14, 2009 at 02:02:03PM -0600, Anthony Liguori wrote:
> Gleb Natapov wrote:
> >>I thought KVM emulates the syscall instruction?  I swear I've seen
> >>patches for that.
> >>
> >It is. Starting from 2.6.32.
> 
> Okay, so this is a performance vs. migration compatibility issue then?
> 
> BTW, couldn't we just not advertise syscall in cpuid?  That should
> fix it too without sacrificing migration compatibility.  We get a
> slight slowdown on AMD hosts but that's probably minor compared to
> the cost of using emulated syscall on Intel hosts.
> 
As far as I understand not advertising syscall for 64bit CPU is not a
valid option.

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 10:08:38PM +0200, Gleb Natapov wrote:
> On Mon, Dec 14, 2009 at 02:02:03PM -0600, Anthony Liguori wrote:
> > Gleb Natapov wrote:
> > >>I thought KVM emulates the syscall instruction?  I swear I've seen
> > >>patches for that.
> > >>
> > >It is. Starting from 2.6.32.
> > 
> > Okay, so this is a performance vs. migration compatibility issue then?
> > 
> > BTW, couldn't we just not advertise syscall in cpuid?  That should
> > fix it too without sacrificing migration compatibility.  We get a
> > slight slowdown on AMD hosts but that's probably minor compared to
> > the cost of using emulated syscall on Intel hosts.
> > 
> As far as I understand not advertising syscall for 64bit CPU is not a
> valid option.

Yes. 64 bit kernel 32 bit userpace crashes with -cpu qemu64,-syscall.

> --
> 			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 02:02:03PM -0600, Anthony Liguori wrote:
> Gleb Natapov wrote:
>>> I thought KVM emulates the syscall instruction?  I swear I've seen
>>> patches for that.
>>>
>>>     
>> It is. Starting from 2.6.32.
>>   
>
> Okay, so this is a performance vs. migration compatibility issue then?
>
> BTW, couldn't we just not advertise syscall in cpuid?  That should fix  
> it too without sacrificing migration compatibility.  We get a slight  
> slowdown on AMD hosts but that's probably minor compared to the cost of  
> using emulated syscall on Intel hosts.
>
> Regards,
>
> Anthony Liguori

This might help 32 bit guests, but not guests with 64 bit
kernel and 32 bit userspace (my case) because all 64 bit
CPUs advertise syscall bit in cpuid. Thus 64 bit guests
do not seem to even bother checking this bit:
AMD + 64 bit -> syscall.


-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> This might help 32 bit guests, but not guests with 64 bit
> kernel and 32 bit userspace (my case) because all 64 bit
> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
> do not seem to even bother checking this bit:
> AMD + 64 bit -> syscall.
>   

Okay, I don't see a great option other than migrating the vendor_id string.

Otherwise, cross vendor migration will not work by default.  Maybe 
that's not a problem but if so, we really should change the default cpu 
model to be much more aggressive about exposing host features.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> This might help 32 bit guests, but not guests with 64 bit
>> kernel and 32 bit userspace (my case) because all 64 bit
>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>> do not seem to even bother checking this bit:
>> AMD + 64 bit -> syscall.
>>   
>
> Okay, I don't see a great option other than migrating the vendor_id string.

This won't help with kernels <2.6.32 though.  I guess we can switch
default vendor to Intel, this likely has some other side effects.

> Otherwise, cross vendor migration will not work by default.  Maybe  
> that's not a problem but if so, we really should change the default cpu  
> model to be much more aggressive about exposing host features.
> 
> Regards,
>
> Anthony Liguori

It's a tradeoff, but yes.  We also need more sanity checks and
management commands giving management tools to understand whether
emulating guest CPU X on host CPU Y is safe/possible, and which guests
this might break.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote:
>   
>> Michael S. Tsirkin wrote:
>>     
>>> This might help 32 bit guests, but not guests with 64 bit
>>> kernel and 32 bit userspace (my case) because all 64 bit
>>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>>> do not seem to even bother checking this bit:
>>> AMD + 64 bit -> syscall.
>>>   
>>>       
>> Okay, I don't see a great option other than migrating the vendor_id string.
>>     
>
> This won't help with kernels <2.6.32 though.  I guess we can switch
> default vendor to Intel, this likely has some other side effects.
>   

That's a kernel bug.  If we think it effects a lot of users, we should 
introduce a CAP such that we can detect this in userspace and fail 
gracefully.

>> Otherwise, cross vendor migration will not work by default.  Maybe  
>> that's not a problem but if so, we really should change the default cpu  
>> model to be much more aggressive about exposing host features.
>>     
> It's a tradeoff, but yes.  We also need more sanity checks and
> management commands giving management tools to understand whether
> emulating guest CPU X on host CPU Y is safe/possible, and which guests
> this might break.
>   

I'd like to see Avi weigh in as historically, he's been one of the 
strongest advocates of a default migration policy of maximum 
compatibility.  Personally, I think cross vendor migration is extremely 
uncommon in production and I would strongly recommend against it.

My own experience has been that hardware homogeneity is pretty common in 
deployments, particularly in the processor space.  There's such a 
different between Nehalem and non-Nehalem class processors that you 
really can't run the same workloads across the two without seeing a 
significant impact.

So I'd actually be in favor of changing the default cpu to something 
that exposes a lot more of the underlying processor features.  I think 
increased performance would be better for most consumers vs. increased 
migration flexibility.

Then again, I'm certainly bias based on being employed by a hardware 
vendor :-)

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote:
>>   
>>> Michael S. Tsirkin wrote:
>>>     
>>>> This might help 32 bit guests, but not guests with 64 bit
>>>> kernel and 32 bit userspace (my case) because all 64 bit
>>>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>>>> do not seem to even bother checking this bit:
>>>> AMD + 64 bit -> syscall.
>>>>         
>>> Okay, I don't see a great option other than migrating the vendor_id string.
>>>     
>>
>> This won't help with kernels <2.6.32 though.  I guess we can switch
>> default vendor to Intel, this likely has some other side effects.
>>   
>
> That's a kernel bug.  If we think it effects a lot of users, we should  
> introduce a CAP such that we can detect this in userspace and fail  
> gracefully.

Not emulating feature host CPU does not have is a kernel bug?
Okay ...
Yes, almost no one runs 2.6.32 yet.

>>> Otherwise, cross vendor migration will not work by default.  Maybe   
>>> that's not a problem but if so, we really should change the default 
>>> cpu  model to be much more aggressive about exposing host features.
>>>     
>> It's a tradeoff, but yes.  We also need more sanity checks and
>> management commands giving management tools to understand whether
>> emulating guest CPU X on host CPU Y is safe/possible, and which guests
>> this might break.
>>   
>
> I'd like to see Avi weigh in as historically, he's been one of the  
> strongest advocates of a default migration policy of maximum  
> compatibility.  Personally, I think cross vendor migration is extremely  
> uncommon in production and I would strongly recommend against it.
>
> My own experience has been that hardware homogeneity is pretty common in  
> deployments, particularly in the processor space.  There's such a  
> different between Nehalem and non-Nehalem class processors that you  
> really can't run the same workloads across the two without seeing a  
> significant impact.
>
> So I'd actually be in favor of changing the default cpu to something  
> that exposes a lot more of the underlying processor features.  I think  
> increased performance would be better for most consumers vs. increased  
> migration flexibility.
>
> Then again, I'm certainly bias based on being employed by a hardware  
> vendor :-)
>
> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote:
>   
>> Michael S. Tsirkin wrote:
>>     
>>> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote:
>>>   
>>>       
>>>> Michael S. Tsirkin wrote:
>>>>     
>>>>         
>>>>> This might help 32 bit guests, but not guests with 64 bit
>>>>> kernel and 32 bit userspace (my case) because all 64 bit
>>>>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>>>>> do not seem to even bother checking this bit:
>>>>> AMD + 64 bit -> syscall.
>>>>>         
>>>>>           
>>>> Okay, I don't see a great option other than migrating the vendor_id string.
>>>>     
>>>>         
>>> This won't help with kernels <2.6.32 though.  I guess we can switch
>>> default vendor to Intel, this likely has some other side effects.
>>>   
>>>       
>> That's a kernel bug.  If we think it effects a lot of users, we should  
>> introduce a CAP such that we can detect this in userspace and fail  
>> gracefully.
>>     
>
> Not emulating feature host CPU does not have is a kernel bug?
> Okay ...
> Yes, almost no one runs 2.6.32 yet.
>   

The kernel has the ability to filter feature bits from cpuid.  We assume 
it's going to filter out things it doesn't support.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 14, 2009 at 03:49:39PM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote:
>>   
>>> Michael S. Tsirkin wrote:
>>>     
>>>> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote:
>>>>         
>>>>> Michael S. Tsirkin wrote:
>>>>>             
>>>>>> This might help 32 bit guests, but not guests with 64 bit
>>>>>> kernel and 32 bit userspace (my case) because all 64 bit
>>>>>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>>>>>> do not seem to even bother checking this bit:
>>>>>> AMD + 64 bit -> syscall.
>>>>>>                   
>>>>> Okay, I don't see a great option other than migrating the vendor_id string.
>>>>>             
>>>> This won't help with kernels <2.6.32 though.  I guess we can switch
>>>> default vendor to Intel, this likely has some other side effects.
>>>>         
>>> That's a kernel bug.  If we think it effects a lot of users, we 
>>> should  introduce a CAP such that we can detect this in userspace and 
>>> fail  gracefully.
>>>     
>>
>> Not emulating feature host CPU does not have is a kernel bug?
>> Okay ...
>> Yes, almost no one runs 2.6.32 yet.
>>   
>
> The kernel has the ability to filter feature bits from cpuid.

I don't think it does: cpuid is an unpriveledged operation,
is it not?

>  We assume  
> it's going to filter out things it doesn't support.

It's tricky: this bit has multiple meanings.
This is a cpu architecture bug, not a kernel bug.

> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Michael S. Tsirkin wrote:
> I don't think it does: cpuid is an unpriveledged operation,
> is it not?
>   

We (qemu) ask the kernel (kvm.ko) to filter out the features bits from 
the cpuid we expose to the guest in order to remove feature bits it 
(kvm.ko) does not support.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Tue, Dec 15, 2009 at 11:37:34AM -0600, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> I don't think it does: cpuid is an unpriveledged operation,
>> is it not?
>>   
>
> We (qemu) ask the kernel (kvm.ko) to filter out the features bits from  
> the cpuid we expose to the guest in order to remove feature bits it  
> (kvm.ko) does not support.
>
> Regards,
>
> Anthony Liguori

I see.
But unfortunately this bit has multiple meanings
for 64/32 bit, kvm does not know whether you will
run a 32 bit or a 64 bit program.
This is a cpu architecture bug.



-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/15/2009 07:56 PM, Michael S. Tsirkin wrote:
>
> I see.
> But unfortunately this bit has multiple meanings
> for 64/32 bit, kvm does not know whether you will
> run a 32 bit or a 64 bit program.
> This is a cpu architecture bug.
>
>    

Correct.  One bit is used for two separate features (syscall-32 and 
syscall-64).

syscall-32 rarely matters because i386 guests use sysenter.  It's only 
32-on-64 guests that attempt to use syscall.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/14/2009 10:18 PM, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> This might help 32 bit guests, but not guests with 64 bit
>> kernel and 32 bit userspace (my case) because all 64 bit
>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>> do not seem to even bother checking this bit:
>> AMD + 64 bit -> syscall.
>
> Okay, I don't see a great option other than migrating the vendor_id 
> string.
>

That's not strictly necessary since the guest cannot change the vendor 
string; all the user has to do is to launch both guests with explicit 
vendor ids.  Of course that imposes more on the user (or the management 
application).

> Otherwise, cross vendor migration will not work by default.  Maybe 
> that's not a problem but if so, we really should change the default 
> cpu model to be much more aggressive about exposing host features.

Maybe we should make -cpu host the default.  That will give the best 
performance for casual users, more testing for newer features, and will 
force management apps to treat migration much more seriously.  The 
downside is that casual users upgrading their machines might experience 
issues with Windows.  Feature compatibility is not just about migration.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Avi Kivity wrote:
>
> Maybe we should make -cpu host the default.  That will give the best 
> performance for casual users, more testing for newer features, and 
> will force management apps to treat migration much more seriously.  
> The downside is that casual users upgrading their machines might 
> experience issues with Windows.  Feature compatibility is not just 
> about migration.

Yes, I'd much rather do this than mucking with vendor_id.  If we're 
going to give up on cross vendor migration by default, I think we should 
go for best performance.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 04:48 PM, Anthony Liguori wrote:
> Avi Kivity wrote:
>>
>> Maybe we should make -cpu host the default.  That will give the best 
>> performance for casual users, more testing for newer features, and 
>> will force management apps to treat migration much more seriously.  
>> The downside is that casual users upgrading their machines might 
>> experience issues with Windows.  Feature compatibility is not just 
>> about migration.
>
> Yes, I'd much rather do this than mucking with vendor_id.  If we're 
> going to give up on cross vendor migration by default, I think we 
> should go for best performance.

If we do this, we need to advertise it clearly:

- casual (non-management-app-using) users will start seeing problems 
with Windows guests unless they change their command lines
- management apps really have to take cpuid into account now 
(previously, it was just a good idea)

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Avi Kivity wrote:
> On 12/20/2009 04:48 PM, Anthony Liguori wrote:
>> Avi Kivity wrote:
>>>
>>> Maybe we should make -cpu host the default.  That will give the best 
>>> performance for casual users, more testing for newer features, and 
>>> will force management apps to treat migration much more seriously.  
>>> The downside is that casual users upgrading their machines might 
>>> experience issues with Windows.  Feature compatibility is not just 
>>> about migration.
>>
>> Yes, I'd much rather do this than mucking with vendor_id.  If we're 
>> going to give up on cross vendor migration by default, I think we 
>> should go for best performance.
>
> If we do this, we need to advertise it clearly:
>
> - casual (non-management-app-using) users will start seeing problems 
> with Windows guests unless they change their command lines

Assuming their migrating across different CPU types.

> - management apps really have to take cpuid into account now 
> (previously, it was just a good idea)

Yes.  I think we probably should have a management app writers guide 
that spells out all of the quirks and best practices that we often 
discuss here and assume that management app writers are going to follow.

Sounds like a good thing to live in the proposes docs directory :-)

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 05:33 PM, Anthony Liguori wrote:
>> - casual (non-management-app-using) users will start seeing problems 
>> with Windows guests unless they change their command lines
>
>
> Assuming their migrating across different CPU types.
>

I was thinking about upgrading their host cpu.  I doubt you'd live 
migrate without a management app.

>> - management apps really have to take cpuid into account now 
>> (previously, it was just a good idea)
>
> Yes.  I think we probably should have a management app writers guide 
> that spells out all of the quirks and best practices that we often 
> discuss here and assume that management app writers are going to follow.

Definitely.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Sun, Dec 20, 2009 at 05:36:57PM +0200, Avi Kivity wrote:
> On 12/20/2009 05:33 PM, Anthony Liguori wrote:
> >>- casual (non-management-app-using) users will start seeing
> >>problems with Windows guests unless they change their command
> >>lines
> >
> >
> >Assuming their migrating across different CPU types.
> >
> 
> I was thinking about upgrading their host cpu.  I doubt you'd live
> migrate without a management app.
> 
And what about VM on disk-on-key or VM image on NFS that can be started
from different locations?

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 05:38 PM, Gleb Natapov wrote:
>
>> I was thinking about upgrading their host cpu.  I doubt you'd live
>> migrate without a management app.
>>
>>      
> And what about VM on disk-on-key or VM image on NFS that can be started
> from different locations?
>    

That needs a config file anyway to let the host qemu know which machine 
level (e.g. -M pc-0.11) to use.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Sun, Dec 20, 2009 at 05:40:53PM +0200, Avi Kivity wrote:
> On 12/20/2009 05:38 PM, Gleb Natapov wrote:
>>
>>> I was thinking about upgrading their host cpu.  I doubt you'd live
>>> migrate without a management app.
>>>
>>>      
>> And what about VM on disk-on-key or VM image on NFS that can be started
>> from different locations?
>>    
>
> That needs a config file anyway to let the host qemu know which machine  
> level (e.g. -M pc-0.11) to use.

Hmm, yes, but will qemu ship config files which configure host cpu
as well?

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 05:49 PM, Michael S. Tsirkin wrote:
>
>> That needs a config file anyway to let the host qemu know which machine
>> level (e.g. -M pc-0.11) to use.
>>      
> Hmm, yes, but will qemu ship config files which configure host cpu
> as well?
>    

If it doesn't, the management app will have to to this itself.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Sun, Dec 20, 2009 at 11:49:40AM +0200, Avi Kivity wrote:
> On 12/14/2009 10:18 PM, Anthony Liguori wrote:
>> Michael S. Tsirkin wrote:
>>> This might help 32 bit guests, but not guests with 64 bit
>>> kernel and 32 bit userspace (my case) because all 64 bit
>>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests
>>> do not seem to even bother checking this bit:
>>> AMD + 64 bit -> syscall.
>>
>> Okay, I don't see a great option other than migrating the vendor_id  
>> string.
>>
>
> That's not strictly necessary since the guest cannot change the vendor  
> string; all the user has to do is to launch both guests with explicit  
> vendor ids.  Of course that imposes more on the user (or the management  
> application).
>
>> Otherwise, cross vendor migration will not work by default.  Maybe  
>> that's not a problem but if so, we really should change the default  
>> cpu model to be much more aggressive about exposing host features.
>
> Maybe we should make -cpu host the default.  That will give the best  
> performance for casual users, more testing for newer features, and will  
> force management apps to treat migration much more seriously.  The  
> downside is that casual users upgrading their machines might experience  
> issues with Windows.  Feature compatibility is not just about migration.

This seems very aggressive.  Can't we whitelist features that we know
about?  Further, doesn't KVM already do this?

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
>
>> Maybe we should make -cpu host the default.  That will give the best
>> performance for casual users, more testing for newer features, and will
>> force management apps to treat migration much more seriously.  The
>> downside is that casual users upgrading their machines might experience
>> issues with Windows.  Feature compatibility is not just about migration.
>>      
> This seems very aggressive.  Can't we whitelist features that we know
> about?  Further, doesn't KVM already do this?
>
>    

It does, but without -cpuid host you're stuck with qemu64 (kvm.ko 
doesn't add features userspace didn't request).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
>>
>>> Maybe we should make -cpu host the default.  That will give the best
>>> performance for casual users, more testing for newer features, and will
>>> force management apps to treat migration much more seriously.  The
>>> downside is that casual users upgrading their machines might experience
>>> issues with Windows.  Feature compatibility is not just about migration.
>>>      
>> This seems very aggressive.  Can't we whitelist features that we know
>> about?  Further, doesn't KVM already do this?
>>
>>    
>
> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
> doesn't add features userspace didn't request).

Hmm, then, shouldn't either kvm or qemu mask features that we do not
emulate well enough to make windows not crash?


> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:

> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
>> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
>>> 
>>>> Maybe we should make -cpu host the default.  That will give the best
>>>> performance for casual users, more testing for newer features, and will
>>>> force management apps to treat migration much more seriously.  The
>>>> downside is that casual users upgrading their machines might experience
>>>> issues with Windows.  Feature compatibility is not just about migration.
>>>> 
>>> This seems very aggressive.  Can't we whitelist features that we know
>>> about?  Further, doesn't KVM already do this?
>>> 
>>> 
>> 
>> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
>> doesn't add features userspace didn't request).
> 
> Hmm, then, shouldn't either kvm or qemu mask features that we do not
> emulate well enough to make windows not crash?

-cpu host does that already, no?

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote:
> 
> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:
> 
> > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
> >> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
> >>> 
> >>>> Maybe we should make -cpu host the default.  That will give the best
> >>>> performance for casual users, more testing for newer features, and will
> >>>> force management apps to treat migration much more seriously.  The
> >>>> downside is that casual users upgrading their machines might experience
> >>>> issues with Windows.  Feature compatibility is not just about migration.
> >>>> 
> >>> This seems very aggressive.  Can't we whitelist features that we know
> >>> about?  Further, doesn't KVM already do this?
> >>> 
> >>> 
> >> 
> >> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
> >> doesn't add features userspace didn't request).
> > 
> > Hmm, then, shouldn't either kvm or qemu mask features that we do not
> > emulate well enough to make windows not crash?
> 
> -cpu host does that already, no?
> 
> Alex

I expected so, but Avi here seems to say windows will crash if you
use a new CPU with it ...

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 20.12.2009, at 18:18, Michael S. Tsirkin wrote:

> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote:
>> 
>> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:
>> 
>>> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
>>>> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
>>>>> 
>>>>>> Maybe we should make -cpu host the default.  That will give the best
>>>>>> performance for casual users, more testing for newer features, and will
>>>>>> force management apps to treat migration much more seriously.  The
>>>>>> downside is that casual users upgrading their machines might experience
>>>>>> issues with Windows.  Feature compatibility is not just about migration.
>>>>>> 
>>>>> This seems very aggressive.  Can't we whitelist features that we know
>>>>> about?  Further, doesn't KVM already do this?
>>>>> 
>>>>> 
>>>> 
>>>> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
>>>> doesn't add features userspace didn't request).
>>> 
>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>> emulate well enough to make windows not crash?
>> 
>> -cpu host does that already, no?
>> 
>> Alex
> 
> I expected so, but Avi here seems to say windows will crash if you
> use a new CPU with it ...

Avi, have you narrowed it down to a specific feature bit?

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote:
> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote:
> > 
> > On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:
> > 
> > > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
> > >> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
> > >>> 
> > >>>> Maybe we should make -cpu host the default.  That will give the best
> > >>>> performance for casual users, more testing for newer features, and will
> > >>>> force management apps to treat migration much more seriously.  The
> > >>>> downside is that casual users upgrading their machines might experience
> > >>>> issues with Windows.  Feature compatibility is not just about migration.
> > >>>> 
> > >>> This seems very aggressive.  Can't we whitelist features that we know
> > >>> about?  Further, doesn't KVM already do this?
> > >>> 
> > >>> 
> > >> 
> > >> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
> > >> doesn't add features userspace didn't request).
> > > 
> > > Hmm, then, shouldn't either kvm or qemu mask features that we do not
> > > emulate well enough to make windows not crash?
> > 
> > -cpu host does that already, no?
> > 
> > Alex
> 
> I expected so, but Avi here seems to say windows will crash if you
> use a new CPU with it ...
> 
Windows _may_ crash if you'll _upgrade_ your _host_ CPU.

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 20.12.2009, at 18:23, Gleb Natapov wrote:

> On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote:
>> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote:
>>> 
>>> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:
>>> 
>>>> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
>>>>> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
>>>>>> 
>>>>>>> Maybe we should make -cpu host the default.  That will give the best
>>>>>>> performance for casual users, more testing for newer features, and will
>>>>>>> force management apps to treat migration much more seriously.  The
>>>>>>> downside is that casual users upgrading their machines might experience
>>>>>>> issues with Windows.  Feature compatibility is not just about migration.
>>>>>>> 
>>>>>> This seems very aggressive.  Can't we whitelist features that we know
>>>>>> about?  Further, doesn't KVM already do this?
>>>>>> 
>>>>>> 
>>>>> 
>>>>> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
>>>>> doesn't add features userspace didn't request).
>>>> 
>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>>> emulate well enough to make windows not crash?
>>> 
>>> -cpu host does that already, no?
>>> 
>>> Alex
>> 
>> I expected so, but Avi here seems to say windows will crash if you
>> use a new CPU with it ...
>> 
> Windows _may_ crash if you'll _upgrade_ your _host_ CPU.

Uh. It may lose activation I guess. But apart from that I can't see how it'd break as long as you don't expect loadvm to work.

Anybody mind to go into a bit more detail here? :-)

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Sun, Dec 20, 2009 at 06:29:11PM +0100, Alexander Graf wrote:
> 
> On 20.12.2009, at 18:23, Gleb Natapov wrote:
> 
> > On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote:
> >> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote:
> >>> 
> >>> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote:
> >>> 
> >>>> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote:
> >>>>> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote:
> >>>>>> 
> >>>>>>> Maybe we should make -cpu host the default.  That will give the best
> >>>>>>> performance for casual users, more testing for newer features, and will
> >>>>>>> force management apps to treat migration much more seriously.  The
> >>>>>>> downside is that casual users upgrading their machines might experience
> >>>>>>> issues with Windows.  Feature compatibility is not just about migration.
> >>>>>>> 
> >>>>>> This seems very aggressive.  Can't we whitelist features that we know
> >>>>>> about?  Further, doesn't KVM already do this?
> >>>>>> 
> >>>>>> 
> >>>>> 
> >>>>> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko  
> >>>>> doesn't add features userspace didn't request).
> >>>> 
> >>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
> >>>> emulate well enough to make windows not crash?
> >>> 
> >>> -cpu host does that already, no?
> >>> 
> >>> Alex
> >> 
> >> I expected so, but Avi here seems to say windows will crash if you
> >> use a new CPU with it ...
> >> 
> > Windows _may_ crash if you'll _upgrade_ your _host_ CPU.
> 
> Uh. It may lose activation I guess. But apart from that I can't see how it'd break as long as you don't expect loadvm to work.
> 
> Anybody mind to go into a bit more detail here? :-)
> 
Windows is a mystery box, so we can speculate as much as we want about it.
If you don't like something just say "it may break Windows" :) Losing
activation does sound like an issue too.

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

Gleb Natapov wrote:
> Windows is a mystery box, so we can speculate as much as we want about it.
> If you don't like something just say "it may break Windows" :) Losing
> activation does sound like an issue too.
>   

Downgrading seems more likely to cause problems.  Considering running 
mplayer in a guest.  If it detects SSE3 in one host and migrate to 
another host that doesn't have SSE3, you'll be running an instruction 
stream that uses instructions the processor doesn't support resulting in 
the application faulting due to an illegal operation.

KVM's cpuid filter doesn't help here because it won't attempt to mask 
things like sse3.  It would be insane to emulate sse3 too.

This is a classic problem with live migration.  You can detect this 
scenario and potentially block the migration, but honestly, how likely 
is it that any given guest is using an obscure feature?  This is a 
classic management tool problem and the solution usually is a set of 
heuristics depending on how conservative the target audience is.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 20.12.2009, at 18:59, Anthony Liguori wrote:

> Gleb Natapov wrote:
>> Windows is a mystery box, so we can speculate as much as we want about it.
>> If you don't like something just say "it may break Windows" :) Losing
>> activation does sound like an issue too.
>>  
> 
> Downgrading seems more likely to cause problems.  Considering running mplayer in a guest.  If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation.

Yeah, migration with -cpu host doesn't sound like a sane thing to do.

> KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3.  It would be insane to emulate sse3 too.

Well, I wouldn't be too sure on that one. Software may use SSE3 instructions to access MMIO in which case we do have to emulate it.

> This is a classic problem with live migration.  You can detect this scenario and potentially block the migration, but honestly, how likely is it that any given guest is using an obscure feature?  This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is.

I agree. I think the discussion was about setting -cpu host to the default? I was merely wondering what'd keep us from doing that, as management software would just -cpu qemu64 and be good.

Btw: I'd actually prefer if it'd be called -cpu safe or so. That way we could differenciate between "tcg could run the code" and "doesn't break migration". The former one is -cpu qemu64 and coincidently it matches with the latter in most cases, but I'm not sure it'd keep that way. Think of default enablement of nested SVM...

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Sun, Dec 20, 2009 at 07:06:03PM +0100, Alexander Graf wrote:
> 
> On 20.12.2009, at 18:59, Anthony Liguori wrote:
> 
> > Gleb Natapov wrote:
> >> Windows is a mystery box, so we can speculate as much as we want about it.
> >> If you don't like something just say "it may break Windows" :) Losing
> >> activation does sound like an issue too.
> >>  
> > 
> > Downgrading seems more likely to cause problems.  Considering running mplayer in a guest.  If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation.
> 
> Yeah, migration with -cpu host doesn't sound like a sane thing to do.
> 
> > KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3.  It would be insane to emulate sse3 too.
> 
> Well, I wouldn't be too sure on that one. Software may use SSE3 instructions to access MMIO in which case we do have to emulate it.
Unfortunately this is already a reality. Look here:
http://groups.google.com/group/linux.kernel/browse_thread/thread/ada9fa2d2574a8af


--
			Gleb.

[Qemu-devel] Re: cpuid problem in upstream qemu with kvm

[Paolo Bonzini]

On 12/21/2009 08:48 AM, Gleb Natapov wrote:
>> >  Well, I wouldn't be too sure on that one. Software may use SSE3 instructions to access MMIO in which case we do have to emulate it.
> Unfortunately this is already a reality. Look here:
> http://groups.google.com/group/linux.kernel/browse_thread/thread/ada9fa2d2574a8af

Emulating 16-byte moves is a tad simpler than emulating the whole of 
SSE3 though. :-)

Paolo

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote:
> Gleb Natapov wrote:
>> Windows is a mystery box, so we can speculate as much as we want about it.
>> If you don't like something just say "it may break Windows" :) Losing
>> activation does sound like an issue too.
>>   
>
> Downgrading seems more likely to cause problems.  Considering running  
> mplayer in a guest.  If it detects SSE3 in one host and migrate to  
> another host that doesn't have SSE3, you'll be running an instruction  
> stream that uses instructions the processor doesn't support resulting in  
> the application faulting due to an illegal operation.
>
> KVM's cpuid filter doesn't help here because it won't attempt to mask  
> things like sse3.  It would be insane to emulate sse3 too.
>
> This is a classic problem with live migration.  You can detect this  
> scenario and potentially block the migration, but honestly, how likely  
> is it that any given guest is using an obscure feature?  This is a  
> classic management tool problem and the solution usually is a set of  
> heuristics depending on how conservative the target audience is.

Yes, but I suspect it is we that need to implement these heuristics,
so that management tools only have a small set of knobs to turn.
Duplicating knowledge of CPU features in management tools
does not make much sense to me.
Does not have to be linked into qemu though, we could have
a library or a set of utilities for management to use.


> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Gleb Natapov]

On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote:
> Gleb Natapov wrote:
> >Windows is a mystery box, so we can speculate as much as we want about it.
> >If you don't like something just say "it may break Windows" :) Losing
> >activation does sound like an issue too.
> 
> Downgrading seems more likely to cause problems.  Considering
> running mplayer in a guest.  If it detects SSE3 in one host and
> migrate to another host that doesn't have SSE3, you'll be running an
> instruction stream that uses instructions the processor doesn't
> support resulting in the application faulting due to an illegal
> operation.
> 
> KVM's cpuid filter doesn't help here because it won't attempt to
> mask things like sse3.  It would be insane to emulate sse3 too.
> 
> This is a classic problem with live migration.  You can detect this
> scenario and potentially block the migration, but honestly, how
> likely is it that any given guest is using an obscure feature?  This
> is a classic management tool problem and the solution usually is a
> set of heuristics depending on how conservative the target audience
> is.
> 
I am confused. We explicitly not talking about migration, why bring it
here? We are talking about casual user who can't care less about
migration, but he upgrades his home computer and Windows VM stops
working for him.

--
			Gleb.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 12/21/2009 09:43 AM, Gleb Natapov wrote:
> On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote:
>> Gleb Natapov wrote:
>>> Windows is a mystery box, so we can speculate as much as we want about it.
>>> If you don't like something just say "it may break Windows" :) Losing
>>> activation does sound like an issue too.
>>
>> Downgrading seems more likely to cause problems.  Considering
>> running mplayer in a guest.  If it detects SSE3 in one host and
>> migrate to another host that doesn't have SSE3, you'll be running an
>> instruction stream that uses instructions the processor doesn't
>> support resulting in the application faulting due to an illegal
>> operation.
>>
>> KVM's cpuid filter doesn't help here because it won't attempt to
>> mask things like sse3.  It would be insane to emulate sse3 too.
>>
>> This is a classic problem with live migration.  You can detect this
>> scenario and potentially block the migration, but honestly, how
>> likely is it that any given guest is using an obscure feature?  This
>> is a classic management tool problem and the solution usually is a
>> set of heuristics depending on how conservative the target audience
>> is.
>>
> I am confused. We explicitly not talking about migration, why bring it
> here? We are talking about casual user who can't care less about
> migration, but he upgrades his home computer and Windows VM stops
> working for him.

John's new cpu definitions are the exact solution for this issue - all 
users, whether using mgmt app or direct qemu (this is no user, this is a 
developer/hacker/other, let's do not optimize this case) should use the 
various 'real' cpu definitions like -cpu Merom | Nehalem | Penry | 
Opteron G1, ....

Qemu will check the required cpuid of the cpu model on the host and 
refuse to load otherwise. When moving to this model, migration can be 
simplified too since there are fewer combination, and one can choose 
performance over migration flexibility and wise versa.
Due to the above check, the destination qemu won't load if the host does 
not support its cpu model.


>
> --
> 			Gleb.
>
>

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[john cooper]

Dor Laor wrote:
 
> Qemu will check the required cpuid of the cpu model on the host and 
> refuse to load otherwise. When moving to this model, migration can be 
> simplified too since there are fewer combination, and one can choose 
> performance over migration flexibility and wise versa.
> Due to the above check, the destination qemu won't load if the host does 
> not support its cpu model.

If you're referring to the check in my patch, that's
currently advisory only.

The existing cpu model encoding of CPUID tosses flags
in to the soup on speculation they may be available
on the host.  If not it assumes they will be quietly
disabled on their way to the guest along with whatever
flags were enabled via +<flag> on the command line. 

This mixed treatment for model implied vs. user
specified flags could be partly an artifact of trying
to adapt the legacy qemu64 model to whatever host
silicon it finds itself upon.

-john


-- 
john.cooper@third-harmonic.com

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 12/22/2009 12:51 AM, john cooper wrote:
> Dor Laor wrote:
>
>> Qemu will check the required cpuid of the cpu model on the host and
>> refuse to load otherwise. When moving to this model, migration can be
>> simplified too since there are fewer combination, and one can choose
>> performance over migration flexibility and wise versa.
>> Due to the above check, the destination qemu won't load if the host
>> does not support its cpu model.
>
> If you're referring to the check in my patch, that's
> currently advisory only.
>
> The existing cpu model encoding of CPUID tosses flags
> in to the soup on speculation they may be available
> on the host. If not it assumes they will be quietly
> disabled on their way to the guest along with whatever

why not shout loudly and abort? Do you want windows to reactivate itself?

> flags were enabled via +<flag> on the command line.
> This mixed treatment for model implied vs. user
> specified flags could be partly an artifact of trying
> to adapt the legacy qemu64 model to whatever host
> silicon it finds itself upon.
>
> -john
>
>

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[john cooper]

Dor Laor wrote:
> On 12/22/2009 12:51 AM, john cooper wrote:
>> Dor Laor wrote:
>>
>>> Qemu will check the required cpuid of the cpu model on the host and
>>> refuse to load otherwise. When moving to this model, migration can be
>>> simplified too since there are fewer combination, and one can choose
>>> performance over migration flexibility and wise versa.
>>> Due to the above check, the destination qemu won't load if the host
>>> does not support its cpu model.
>>
>> If you're referring to the check in my patch, that's
>> currently advisory only.
>>
>> The existing cpu model encoding of CPUID tosses flags
>> in to the soup on speculation they may be available
>> on the host. If not it assumes they will be quietly
>> disabled on their way to the guest along with whatever
> 
> why not shout loudly and abort? Do you want windows to reactivate itself?

This was the existing behavior of that code which
expected certain requested flags to be casually
dropped.  But bailing after the warning is probably
fine as well given the warning itself is optional
and bailing wouldn't happen by default.

That code could probably use a cleanup going forward
to iron out such issues.  Andre mentioned he had
pending patches which in part addressed this so I
thought to let those resolve beforehand.

Thanks,

-john

-- 
john.cooper@redhat.com

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 12/21/2009 02:28 AM, Dor Laor wrote:
> John's new cpu definitions are the exact solution for this issue - all 
> users, whether using mgmt app or direct qemu (this is no user, this is 
> a developer/hacker/other, let's do not optimize this case) should use 
> the various 'real' cpu definitions like -cpu Merom | Nehalem | Penry | 
> Opteron G1, ....

Of course, the tricky part is at what level do you define these names.  
For instance, do you do just Nehalem, or do you also do Nehalem, 
Nehalem-EP, Nehalem-EX?

Nehalem is really just a code name.  Would it be better to use core-i7?

I think the only two Fully Correct approachs are to support a very 
specific CPU (e.g. Xeon-X5270) or provide the ability to individually 
tweak cpu flags.

The notion of compatibility classes should probably be left to 
management tools.  We can make it a lot easier for them though by 
supporting turning point CPU models.

For instance, Xeon-X5570 should be a least common denominator for 
Nehalem processors.  It's probably better for users too.  It's easier 
for them to answer "do I have anything older than a Xeon-X5570" than to 
ask "do I have any Woodcrest class processors".

I encounter this confusion a lot.  I usually ask people whether they 
have a Nehalem processor when debugging something and their response is 
always, I have a Xeon-XYZ, is that Nehalem?

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[john cooper]

Anthony Liguori wrote:
> On 12/21/2009 02:28 AM, Dor Laor wrote:
>> John's new cpu definitions are the exact solution for this issue - all
>> users, whether using mgmt app or direct qemu (this is no user, this is
>> a developer/hacker/other, let's do not optimize this case) should use
>> the various 'real' cpu definitions like -cpu Merom | Nehalem | Penry |
>> Opteron G1, ....
> 
> Of course, the tricky part is at what level do you define these names. 
> For instance, do you do just Nehalem, or do you also do Nehalem,
> Nehalem-EP, Nehalem-EX?

>From a strict technical perspective I'd agree.  But this is
more an effort to define simplified discrete cpu models which
reasonably reflect deployed silicon.  An equal (if not greater)
motivation is to intentionally dumb-down portions of the
spectrum to simplify migration, albeit at the possible loss
of features provided by an individual cpu.

> Nehalem is really just a code name.  Would it be better to use core-i7?

I'd inquired whether marketing sanctioned names existed but
didn't receive anything conclusive.  The resulting monikers
at least are in common usage.
 
> I think the only two Fully Correct approachs are to support a very
> specific CPU (e.g. Xeon-X5270) or provide the ability to individually
> tweak cpu flags.

We have the ability today to pass an exhaustive list of
feature flags.  Offering a sprawling list of fine tuned
models is possible although somewhat cumbersome, and
doesn't address the need to collapse ranges into a least
common denominator for the benefit of migration.

> For instance, Xeon-X5570 should be a least common denominator for
> Nehalem processors.  It's probably better for users too.  It's easier
> for them to answer "do I have anything older than a Xeon-X5570" than to
> ask "do I have any Woodcrest class processors".

I'm not particularly biased to a naming scheme.  Although the
generic labels avoid suggesting a specific instance of a cpu
class which is helpful in this context.

Engaging the vendors with this scheme was also somewhat
of a sanity check to assure it reflects commercially deployed
vs. development silicon.

-john 


-- 
john.cooper@redhat.com

[Qemu-devel] Re: cpuid problem in upstream qemu with kvm

[Paolo Bonzini]

On 01/05/2010 07:06 AM, john cooper wrote:
>> >  Nehalem is really just a code name.  Would it be better to use core-i7?
> I'd inquired whether marketing sanctioned names existed but
> didn't receive anything conclusive.  The resulting monikers
> at least are in common usage.

Unfortunately it goes on a case-by-case basis.  Nobody is talking about 
Willamette anymore, but the Prescott moniker is still with us.

Paolo

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 12/23/2009 04:32 AM, Avi Kivity wrote:
> On 12/22/2009 06:12 PM, Anthony Liguori wrote:
>>
>> I think the only two Fully Correct approachs are to support a very 
>> specific CPU (e.g. Xeon-X5270) or provide the ability to individually 
>> tweak cpu flags.
>
> Yes.  By a curious coincidence these are what the hardware vendors 
> define (unlike compat classes etc.).

Whenever possible, steal from what the hardware vendors do :-)

>> The notion of compatibility classes should probably be left to 
>> management tools.  We can make it a lot easier for them though by 
>> supporting turning point CPU models.
>
> Agreed.
>
>>
>> For instance, Xeon-X5570 should be a least common denominator for 
>> Nehalem processors.  It's probably better for users too.  It's easier 
>> for them to answer "do I have anything older than a Xeon-X5570" than 
>> to ask "do I have any Woodcrest class processors".
>>
>> I encounter this confusion a lot.  I usually ask people whether they 
>> have a Nehalem processor when debugging something and their response 
>> is always, I have a Xeon-XYZ, is that Nehalem?
>
> This is complicated by the fact that processors don't have 
> straight-line development, and that marketing names don't correspond 
> to anything.  For example Xeon can be any of the Pentium 4, Core, Core 
> 2, and Nehalem (and perhaps other) microachitectures.  A newer Xeon is 
> often introduced with an older core (usually for larger machines) than 
> previously existing Xeons.

Typically, there is at least a little sanity naming for these cases.  
For instance, any Xeon W35xx should have the same features.  A Xeon 
W55xx may be different.

It's not going to be easy to include every possible model.  It's a hard 
problem for management tools too.  The thing is, I imagine most 
management tools are going to cat /proc/cpuinfo to get what the 
processor is and that's going to be a Xeon YYXXXX type name so I really 
believe that's the thing that makes sense to expose in QEMU.

Maybe we could name models like IntelXeonW35xx.

Clever use of the preprocessor will make this effort much, much saner.  
Also, we really need some sort of human readable document that explains 
the differences between processor classes and makes recommendations 
about what management tools should take into consideration.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 02:10 AM, Anthony Liguori wrote:
> On 12/23/2009 04:32 AM, Avi Kivity wrote:
>> On 12/22/2009 06:12 PM, Anthony Liguori wrote:
>>>
>>> I think the only two Fully Correct approachs are to support a very 
>>> specific CPU (e.g. Xeon-X5270) or provide the ability to 
>>> individually tweak cpu flags.
>>
>> Yes.  By a curious coincidence these are what the hardware vendors 
>> define (unlike compat classes etc.).
>
> Whenever possible, steal from what the hardware vendors do :-)
>

[...]

> Typically, there is at least a little sanity naming for these cases.  
> For instance, any Xeon W35xx should have the same features.  A Xeon 
> W55xx may be different.
>
> It's not going to be easy to include every possible model.  It's a 
> hard problem for management tools too.  The thing is, I imagine most 
> management tools are going to cat /proc/cpuinfo to get what the 
> processor is and that's going to be a Xeon YYXXXX type name so I 
> really believe that's the thing that makes sense to expose in QEMU.
>
> Maybe we could name models like IntelXeonW35xx.
>

While a W3501 should be similar to a W3599, we don't know if it actually 
will be.  You are no longer on a Fully Correct path and instead you are 
wandering in Marketing Land.

Note that the processor type is just part of what determines which 
features are exposed to the guest.  Qemu version, kvm version, host 
kernel version, and even kernel command-line parameters all play a part, 
so to really determine migratability the management tool should talk to 
qemu, not /proc/cpuinfo.

> Clever use of the preprocessor will make this effort much, much saner.

I cringe whenever I read something like this.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/05/2010 09:25 PM, Avi Kivity wrote:
>> Typically, there is at least a little sanity naming for these cases.  
>> For instance, any Xeon W35xx should have the same features.  A Xeon 
>> W55xx may be different.
>>
>> It's not going to be easy to include every possible model.  It's a 
>> hard problem for management tools too.  The thing is, I imagine most 
>> management tools are going to cat /proc/cpuinfo to get what the 
>> processor is and that's going to be a Xeon YYXXXX type name so I 
>> really believe that's the thing that makes sense to expose in QEMU.
>>
>> Maybe we could name models like IntelXeonW35xx.
>>
>
> While a W3501 should be similar to a W3599, we don't know if it 
> actually will be.  You are no longer on a Fully Correct path and 
> instead you are wandering in Marketing Land.
>
> Note that the processor type is just part of what determines which 
> features are exposed to the guest.  Qemu version, kvm version, host 
> kernel version, and even kernel command-line parameters all play a 
> part, so to really determine migratability the management tool should 
> talk to qemu, not /proc/cpuinfo.

So if I understand correctly, you're advocating to drop the idea of 
common model names, and provide a mechanism for a management tool to 
query which cpu features are supported both by the processor, but also 
filtered by qemu, kvm, etc?

I think that's workable but I think there may be some subtle issues 
especially across qemu versions.  Can you give an example of what you 
would expect the output to be?

>> Clever use of the preprocessor will make this effort much, much saner.
>
> I cringe whenever I read something like this.

:-)

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Wed, Jan 06, 2010 at 07:25:10AM -0600, Anthony Liguori wrote:
> On 01/05/2010 09:25 PM, Avi Kivity wrote:
>>> Typically, there is at least a little sanity naming for these cases.  
>>> For instance, any Xeon W35xx should have the same features.  A Xeon  
>>> W55xx may be different.
>>>
>>> It's not going to be easy to include every possible model.  It's a  
>>> hard problem for management tools too.  The thing is, I imagine most  
>>> management tools are going to cat /proc/cpuinfo to get what the  
>>> processor is and that's going to be a Xeon YYXXXX type name so I  
>>> really believe that's the thing that makes sense to expose in QEMU.
>>>
>>> Maybe we could name models like IntelXeonW35xx.
>>>
>>
>> While a W3501 should be similar to a W3599, we don't know if it  
>> actually will be.  You are no longer on a Fully Correct path and  
>> instead you are wandering in Marketing Land.
>>
>> Note that the processor type is just part of what determines which  
>> features are exposed to the guest.  Qemu version, kvm version, host  
>> kernel version, and even kernel command-line parameters all play a  
>> part, so to really determine migratability the management tool should  
>> talk to qemu, not /proc/cpuinfo.
>
> So if I understand correctly, you're advocating to drop the idea of  
> common model names, and provide a mechanism for a management tool to  
> query which cpu features are supported both by the processor, but also  
> filtered by qemu, kvm, etc?

The syscall issue shows there are several other things necessary IMO:
- features need to be scoped with Vendor ID as some feature bits might
  not be 100% compatible.
- Some features might be emulated. So it's not yes/no but
  yes/no/slow and management needs to know IMO.

> I think that's workable but I think there may be some subtle issues  
> especially across qemu versions.  Can you give an example of what you  
> would expect the output to be?
>
>>> Clever use of the preprocessor will make this effort much, much saner.
>>
>> I cringe whenever I read something like this.
>
> :-)
>
> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 03:25 PM, Anthony Liguori wrote:
> On 01/05/2010 09:25 PM, Avi Kivity wrote:
>>> Typically, there is at least a little sanity naming for these 
>>> cases.  For instance, any Xeon W35xx should have the same features.  
>>> A Xeon W55xx may be different.
>>>
>>> It's not going to be easy to include every possible model.  It's a 
>>> hard problem for management tools too.  The thing is, I imagine most 
>>> management tools are going to cat /proc/cpuinfo to get what the 
>>> processor is and that's going to be a Xeon YYXXXX type name so I 
>>> really believe that's the thing that makes sense to expose in QEMU.
>>>
>>> Maybe we could name models like IntelXeonW35xx.
>>>
>>
>> While a W3501 should be similar to a W3599, we don't know if it 
>> actually will be.  You are no longer on a Fully Correct path and 
>> instead you are wandering in Marketing Land.
>>
>> Note that the processor type is just part of what determines which 
>> features are exposed to the guest.  Qemu version, kvm version, host 
>> kernel version, and even kernel command-line parameters all play a 
>> part, so to really determine migratability the management tool should 
>> talk to qemu, not /proc/cpuinfo.
>
> So if I understand correctly, you're advocating to drop the idea of 
> common model names, and provide a mechanism for a management tool to 
> query which cpu features are supported both by the processor, but also 
> filtered by qemu, kvm, etc?

Well, it's nice to have a -cpu X1234, so I wouldn't recommend dropping 
it, but certainly a migration pool that doesn't assume anything about 
the host qemu and kernel version needs more fine-grained information.

>
> I think that's workable but I think there may be some subtle issues 
> especially across qemu versions.  Can you give an example of what you 
> would expect the output to be?

-> { command: query-cpu-capabalities }
<- { result: { features: [vm, fpu, lm, sse2, sse3, sssse3, ssssssse3, 
sse3.14 ], cache: { ... }, vendor: { .... }, etc. } }

Or something.  We'd need similar queries for the number of PCI slots, 
for example, so the GUI can tell the user when adding hardware is no 
longer an option instead of trying it blindly and returning an error.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/06/2010 07:47 AM, Avi Kivity wrote:
> On 01/06/2010 03:25 PM, Anthony Liguori wrote:
>> On 01/05/2010 09:25 PM, Avi Kivity wrote:
>>>> Typically, there is at least a little sanity naming for these 
>>>> cases.  For instance, any Xeon W35xx should have the same 
>>>> features.  A Xeon W55xx may be different.
>>>>
>>>> It's not going to be easy to include every possible model.  It's a 
>>>> hard problem for management tools too.  The thing is, I imagine 
>>>> most management tools are going to cat /proc/cpuinfo to get what 
>>>> the processor is and that's going to be a Xeon YYXXXX type name so 
>>>> I really believe that's the thing that makes sense to expose in QEMU.
>>>>
>>>> Maybe we could name models like IntelXeonW35xx.
>>>>
>>>
>>> While a W3501 should be similar to a W3599, we don't know if it 
>>> actually will be.  You are no longer on a Fully Correct path and 
>>> instead you are wandering in Marketing Land.
>>>
>>> Note that the processor type is just part of what determines which 
>>> features are exposed to the guest.  Qemu version, kvm version, host 
>>> kernel version, and even kernel command-line parameters all play a 
>>> part, so to really determine migratability the management tool 
>>> should talk to qemu, not /proc/cpuinfo.
>>
>> So if I understand correctly, you're advocating to drop the idea of 
>> common model names, and provide a mechanism for a management tool to 
>> query which cpu features are supported both by the processor, but 
>> also filtered by qemu, kvm, etc?
>
> Well, it's nice to have a -cpu X1234, so I wouldn't recommend dropping 
> it, but certainly a migration pool that doesn't assume anything about 
> the host qemu and kernel version needs more fine-grained information.
>
>>
>> I think that's workable but I think there may be some subtle issues 
>> especially across qemu versions.  Can you give an example of what you 
>> would expect the output to be?
>
> -> { command: query-cpu-capabalities }
> <- { result: { features: [vm, fpu, lm, sse2, sse3, sssse3, ssssssse3, 
> sse3.14 ], cache: { ... }, vendor: { .... }, etc. } }
>
> Or something.  We'd need similar queries for the number of PCI slots, 
> for example, so the GUI can tell the user when adding hardware is no 
> longer an option instead of trying it blindly and returning an error.

Yeah, the trick with this is that we don't have such a thing as a bare 
bones CPU in qemu right now.

So you can't really say -cpu qemu64+vm+fpu+lm....

Because some extra features might sneak in.  What I'm getting at is that 
we need a way to make the results of this command translate into a -cpu 
invocation that works reliably across multiple versions of qemu.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 03:49 PM, Anthony Liguori wrote:
>>
>>>
>>> I think that's workable but I think there may be some subtle issues 
>>> especially across qemu versions.  Can you give an example of what 
>>> you would expect the output to be?
>>
>> -> { command: query-cpu-capabalities }
>> <- { result: { features: [vm, fpu, lm, sse2, sse3, sssse3, ssssssse3, 
>> sse3.14 ], cache: { ... }, vendor: { .... }, etc. } }
>>
>> Or something.  We'd need similar queries for the number of PCI slots, 
>> for example, so the GUI can tell the user when adding hardware is no 
>> longer an option instead of trying it blindly and returning an error.
>
>
> Yeah, the trick with this is that we don't have such a thing as a bare 
> bones CPU in qemu right now.
>
> So you can't really say -cpu qemu64+vm+fpu+lm....
>
> Because some extra features might sneak in.  What I'm getting at is 
> that we need a way to make the results of this command translate into 
> a -cpu invocation that works reliably across multiple versions of qemu.

Well, we can freeze qemu64 if we wish.  That's still not 100% accurate 
since kvm can remove features from qemu64.

-cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 06.01.2010, at 14:54, Avi Kivity wrote:

> On 01/06/2010 03:49 PM, Anthony Liguori wrote:
>>> 
>>>> 
>>>> I think that's workable but I think there may be some subtle issues especially across qemu versions.  Can you give an example of what you would expect the output to be?
>>> 
>>> -> { command: query-cpu-capabalities }
>>> <- { result: { features: [vm, fpu, lm, sse2, sse3, sssse3, ssssssse3, sse3.14 ], cache: { ... }, vendor: { .... }, etc. } }
>>> 
>>> Or something.  We'd need similar queries for the number of PCI slots, for example, so the GUI can tell the user when adding hardware is no longer an option instead of trying it blindly and returning an error.
>> 
>> 
>> Yeah, the trick with this is that we don't have such a thing as a bare bones CPU in qemu right now.
>> 
>> So you can't really say -cpu qemu64+vm+fpu+lm....
>> 
>> Because some extra features might sneak in.  What I'm getting at is that we need a way to make the results of this command translate into a -cpu invocation that works reliably across multiple versions of qemu.
> 
> Well, we can freeze qemu64 if we wish.  That's still not 100% accurate since kvm can remove features from qemu64.
> 
> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?

I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features.

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 03:55 PM, Alexander Graf wrote:
>
>> Well, we can freeze qemu64 if we wish.  That's still not 100% accurate since kvm can remove features from qemu64.
>>
>> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?
>>      
> I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features.
>    

We can probably default -enable-kvm to -cpu host, as long as we explain 
very carefully that if users wish to preserve cpu features across 
upgrades, they can't depend on the default.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Wed, Jan 06, 2010 at 03:58:01PM +0200, Avi Kivity wrote:
> On 01/06/2010 03:55 PM, Alexander Graf wrote:
>>
>>> Well, we can freeze qemu64 if we wish.  That's still not 100% accurate since kvm can remove features from qemu64.
>>>
>>> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?
>>>      
>> I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features.
>>    
>
> We can probably default -enable-kvm to -cpu host, as long as we explain  
> very carefully that if users wish to preserve cpu features across  
> upgrades, they can't depend on the default.

Hardware upgrades or software upgrades?

> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>> We can probably default -enable-kvm to -cpu host, as long as we explain
>> very carefully that if users wish to preserve cpu features across
>> upgrades, they can't depend on the default.
>>      
> Hardware upgrades or software upgrades?
>    

Yes.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/06/2010 04:32 PM, Avi Kivity wrote:
> On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>>> We can probably default -enable-kvm to -cpu host, as long as we explain
>>> very carefully that if users wish to preserve cpu features across
>>> upgrades, they can't depend on the default.
>> Hardware upgrades or software upgrades?
>
> Yes.
>

I just want to remind all the the main motivation for using -cpu 
realModelThatWasOnceShiped is to provide correct cpu emulation for the 
guest. Using a random qemu|kvm64+flag1-flag2 might really cause trouble 
for the guest OS or guest apps.

On top of -cpu nehalem we can always add fancy features like x2apic, etc.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/06/2010 08:48 AM, Dor Laor wrote:
> On 01/06/2010 04:32 PM, Avi Kivity wrote:
>> On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>>>> We can probably default -enable-kvm to -cpu host, as long as we 
>>>> explain
>>>> very carefully that if users wish to preserve cpu features across
>>>> upgrades, they can't depend on the default.
>>> Hardware upgrades or software upgrades?
>>
>> Yes.
>>
>
> I just want to remind all the the main motivation for using -cpu 
> realModelThatWasOnceShiped is to provide correct cpu emulation for the 
> guest. Using a random qemu|kvm64+flag1-flag2 might really cause 
> trouble for the guest OS or guest apps.
>
> On top of -cpu nehalem we can always add fancy features like x2apic, etc.

I think it boils down to, how are people going to use this.

For individuals, code names like Nehalem are too obscure.  From my own 
personal experience, even power users often have no clue whether there 
processor is a Nehalem or not.

For management tools, Nehalem is a somewhat imprecise target because it 
covers a wide range of potential processors.  In general, I think what 
we really need to do is simplify the process of going from, here's the 
output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu 
so that migration always works for these systems.

I don't think -cpu nehalem really helps with that problem.  -cpu none 
helps a bit, but I hope we can find something nicer.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/06/2010 05:16 PM, Anthony Liguori wrote:
> On 01/06/2010 08:48 AM, Dor Laor wrote:
>> On 01/06/2010 04:32 PM, Avi Kivity wrote:
>>> On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>>>>> We can probably default -enable-kvm to -cpu host, as long as we
>>>>> explain
>>>>> very carefully that if users wish to preserve cpu features across
>>>>> upgrades, they can't depend on the default.
>>>> Hardware upgrades or software upgrades?
>>>
>>> Yes.
>>>
>>
>> I just want to remind all the the main motivation for using -cpu
>> realModelThatWasOnceShiped is to provide correct cpu emulation for the
>> guest. Using a random qemu|kvm64+flag1-flag2 might really cause
>> trouble for the guest OS or guest apps.
>>
>> On top of -cpu nehalem we can always add fancy features like x2apic, etc.
>
> I think it boils down to, how are people going to use this.
>
> For individuals, code names like Nehalem are too obscure. From my own
> personal experience, even power users often have no clue whether there
> processor is a Nehalem or not.
>
> For management tools, Nehalem is a somewhat imprecise target because it
> covers a wide range of potential processors. In general, I think what we
> really need to do is simplify the process of going from, here's the
> output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu
> so that migration always works for these systems.
>
> I don't think -cpu nehalem really helps with that problem. -cpu none
> helps a bit, but I hope we can find something nicer.

We can debate about the exact name/model to represent the Nehalem 
family, I don't have an issue with that and actually Intel and Amd 
should define it.

There are two main motivations behind the above approach:
1. Sound guest cpu definition.
    Using a predefined model should automatically set all the relevant
    vendor/stepping/cpuid flags/cache sizes/etc.
    We just can let every management application deal with it. It breaks
    guest OS/apps. For instance there are MSI support in windows guest
    relay on the stepping.

2. Simplifying end user and mgmt tools.
    qemu/kvm have the best knowledge about these low levels. If we push
    it up in the stack, eventually it reaches the user. The end user,
    not a 'qemu-devel user' which is actually far better from the
    average user.

    This means that such users will have to know what is popcount and
    whether or not to limit migration on one host by adding sse4.2 or
    not.

This is exactly what vmware are doing:
  - Intel CPUs : 
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991
  - AMD CPUs : 
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992

Why should we invent the wheel (qemu64..)? Let's learn from their 
experience.

This is the test description of the original patch by John:


     # Intel
     # -----

     # Management layers remove pentium3 by default.
     # It primarily remains here for testing of 32-bit migration.
     #
     [0:Pentium 3 Intel
             :vmx
             :pentium3;]

     # Core 2, 65nm
     # possible option sets: (+nx,+cx16), (+nx,+cx16,+ssse3)
     #
     1:Merom
             :vmx,sse2
             :qemu64,-nx,+sse2;

     # Core2 45nm
     #
     2:Penryn
             :vmx,sse2,nx,cx16,ssse3,sse4_1
             :qemu64,+sse2,+cx16,+ssse3,+sse4_1;

     # Core i7 45/32nm
     #
     3:Nehalem
             :vmx,sse2,nx,cx16,ssse3,sse4_1,sse4_2,popcnt
             :qemu64,+sse2,+cx16,+ssse3,+sse4_1,+sse4_2,+popcnt;


     # AMD
     # ---

     # Management layers remove pentium3 by default.
     # It primarily remains here for testing of 32-bit migration.
     #
     [0:Pentium 3 AMD
             :svm
             :pentium3;]

     # Opteron 90nm stepping E1/E4/E6
     # possible option sets: (-nx) for 130nm
     #
     1:Opteron G1
             :svm,sse2,nx
             :qemu64,+sse2;

     # Opteron 90nm stepping F2/F3
     #
     2:Opteron G2
             :svm,sse2,nx,cx16,rdtscp
             :qemu64,+sse2,+cx16,+rdtscp;

     # Opteron 65/45nm
     #
     3:Opteron G3
             :svm,sse2,nx,cx16,sse4a,misalignsse,popcnt,abm
             :qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm;



>
> Regards,
>
> Anthony Liguori
>
>

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 10:03 AM, Dor Laor wrote:
>
> We can debate about the exact name/model to represent the Nehalem 
> family, I don't have an issue with that and actually Intel and Amd 
> should define it.

AMD and Intel already defined their names (in cat /proc/cpuinfo).  They 
don't define families, the whole idea is to segment the market.

>
> There are two main motivations behind the above approach:
> 1. Sound guest cpu definition.
>    Using a predefined model should automatically set all the relevant
>    vendor/stepping/cpuid flags/cache sizes/etc.
>    We just can let every management application deal with it. It breaks
>    guest OS/apps. For instance there are MSI support in windows guest
>    relay on the stepping.
>
> 2. Simplifying end user and mgmt tools.
>    qemu/kvm have the best knowledge about these low levels. If we push
>    it up in the stack, eventually it reaches the user. The end user,
>    not a 'qemu-devel user' which is actually far better from the
>    average user.
>
>    This means that such users will have to know what is popcount and
>    whether or not to limit migration on one host by adding sse4.2 or
>    not.
>
> This is exactly what vmware are doing:
>  - Intel CPUs : 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 
>
>  - AMD CPUs : 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 
>

They don't have to deal with different qemu and kvm versions.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 10:18 AM, Avi Kivity wrote:
> On 01/07/2010 10:03 AM, Dor Laor wrote:
>>
>> We can debate about the exact name/model to represent the Nehalem
>> family, I don't have an issue with that and actually Intel and Amd
>> should define it.
>
> AMD and Intel already defined their names (in cat /proc/cpuinfo). They
> don't define families, the whole idea is to segment the market.

The idea here is to minimize the number of models we should have the 
following range for Intel for example:
   pentium3 - merom -  penry - Nehalem - host - kvm/qemu64
So we're supplying wide range of cpus, p3 for maximum flexibility and 
migration, nehalem for performance and migration, host for maximum 
performance and qemu/kvm64 for custom maid.

>
>>
>> There are two main motivations behind the above approach:
>> 1. Sound guest cpu definition.
>> Using a predefined model should automatically set all the relevant
>> vendor/stepping/cpuid flags/cache sizes/etc.
>> We just can let every management application deal with it. It breaks
>> guest OS/apps. For instance there are MSI support in windows guest
>> relay on the stepping.
>>
>> 2. Simplifying end user and mgmt tools.
>> qemu/kvm have the best knowledge about these low levels. If we push
>> it up in the stack, eventually it reaches the user. The end user,
>> not a 'qemu-devel user' which is actually far better from the
>> average user.
>>
>> This means that such users will have to know what is popcount and
>> whether or not to limit migration on one host by adding sse4.2 or
>> not.
>>
>> This is exactly what vmware are doing:
>> - Intel CPUs :
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991
>>
>> - AMD CPUs :
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992
>>
>
> They don't have to deal with different qemu and kvm versions.
>

Both our customers - the end users. It's not their problem.
IMO what's missing today is a safe and sound cpu emulation that is 
simply and friendly to represent. qemu64,+popcount is not simple for the 
end user. There is no reason to through it on higher level mgmt.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 11:11 AM, Dor Laor wrote:
> On 01/07/2010 10:18 AM, Avi Kivity wrote:
>> On 01/07/2010 10:03 AM, Dor Laor wrote:
>>>
>>> We can debate about the exact name/model to represent the Nehalem
>>> family, I don't have an issue with that and actually Intel and Amd
>>> should define it.
>>
>> AMD and Intel already defined their names (in cat /proc/cpuinfo). They
>> don't define families, the whole idea is to segment the market.
>
> The idea here is to minimize the number of models we should have the 
> following range for Intel for example:
>   pentium3 - merom -  penry - Nehalem - host - kvm/qemu64
> So we're supplying wide range of cpus, p3 for maximum flexibility and 
> migration, nehalem for performance and migration, host for maximum 
> performance and qemu/kvm64 for custom maid.

There's no such thing as Nehalem.

>>>
>>> This is exactly what vmware are doing:
>>> - Intel CPUs :
>>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 
>>>
>>>
>>> - AMD CPUs :
>>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 
>>>
>>>
>>
>> They don't have to deal with different qemu and kvm versions.
>>
>
> Both our customers - the end users. It's not their problem.
> IMO what's missing today is a safe and sound cpu emulation that is 
> simply and friendly to represent. qemu64,+popcount is not simple for 
> the end user. There is no reason to through it on higher level mgmt.

There's no simple solution except to restrict features to what was 
available on the first processors.



-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 11:24 AM, Avi Kivity wrote:
> On 01/07/2010 11:11 AM, Dor Laor wrote:
>> On 01/07/2010 10:18 AM, Avi Kivity wrote:
>>> On 01/07/2010 10:03 AM, Dor Laor wrote:
>>>>
>>>> We can debate about the exact name/model to represent the Nehalem
>>>> family, I don't have an issue with that and actually Intel and Amd
>>>> should define it.
>>>
>>> AMD and Intel already defined their names (in cat /proc/cpuinfo). They
>>> don't define families, the whole idea is to segment the market.
>>
>> The idea here is to minimize the number of models we should have the
>> following range for Intel for example:
>> pentium3 - merom - penry - Nehalem - host - kvm/qemu64
>> So we're supplying wide range of cpus, p3 for maximum flexibility and
>> migration, nehalem for performance and migration, host for maximum
>> performance and qemu/kvm64 for custom maid.
>
> There's no such thing as Nehalem.

Intel were ok with it. Again, you can name is corei7 or xeon34234234234, 
I don't care, the principle remains the same.

>
>>>>
>>>> This is exactly what vmware are doing:
>>>> - Intel CPUs :
>>>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991
>>>>
>>>>
>>>> - AMD CPUs :
>>>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992
>>>>
>>>>
>>>
>>> They don't have to deal with different qemu and kvm versions.
>>>
>>
>> Both our customers - the end users. It's not their problem.
>> IMO what's missing today is a safe and sound cpu emulation that is
>> simply and friendly to represent. qemu64,+popcount is not simple for
>> the end user. There is no reason to through it on higher level mgmt.
>
> There's no simple solution except to restrict features to what was
> available on the first processors.

What's not simple about the above 4 options?
What's a better alternative (that insures users understand it and use it 
and guest msi and even skype application is happy about it)?

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/07/2010 03:40 AM, Dor Laor wrote:
>> There's no simple solution except to restrict features to what was
>> available on the first processors.
>
> What's not simple about the above 4 options?
> What's a better alternative (that insures users understand it and use 
> it and guest msi and even skype application is happy about it)?

Even if you have -cpu Nehalem, different versions of the KVM kernel 
module may additionally filter cpuid flags.

So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary 
to say:

(2.6.33) qemu -cpu Nehalem,-syscall
(2.6.18) qemu -cpu Nehalem

In order to be compatible.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 01:39 PM, Anthony Liguori wrote:
> On 01/07/2010 03:40 AM, Dor Laor wrote:
>>> There's no simple solution except to restrict features to what was
>>> available on the first processors.
>>
>> What's not simple about the above 4 options?
>> What's a better alternative (that insures users understand it and use
>> it and guest msi and even skype application is happy about it)?
>
> Even if you have -cpu Nehalem, different versions of the KVM kernel
> module may additionally filter cpuid flags.
>
> So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary
> to say:
>
> (2.6.33) qemu -cpu Nehalem,-syscall
> (2.6.18) qemu -cpu Nehalem

Or let qemu do it automatically for you.

>
> In order to be compatible.
>
> Regards,
>
> Anthony Liguori
>

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 01:44 PM, Dor Laor wrote:
>> So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary
>> to say:
>>
>> (2.6.33) qemu -cpu Nehalem,-syscall
>> (2.6.18) qemu -cpu Nehalem
>
>
> Or let qemu do it automatically for you.

qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on 
another node.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 02:00 PM, Avi Kivity wrote:
> On 01/07/2010 01:44 PM, Dor Laor wrote:
>>> So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary
>>> to say:
>>>
>>> (2.6.33) qemu -cpu Nehalem,-syscall
>>> (2.6.18) qemu -cpu Nehalem
>>
>>
>> Or let qemu do it automatically for you.
>
> qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on
> another node.
>

We can live with it, either have qemu realize the kernel version out of 
another existing feature or query uname.

Alternatively, the matching libvirt package can be the one adding or 
removing it in the right distribution.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/07/2010 06:20 AM, Dor Laor wrote:
> On 01/07/2010 02:00 PM, Avi Kivity wrote:
>> On 01/07/2010 01:44 PM, Dor Laor wrote:
>>>> So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary
>>>> to say:
>>>>
>>>> (2.6.33) qemu -cpu Nehalem,-syscall
>>>> (2.6.18) qemu -cpu Nehalem
>>>
>>>
>>> Or let qemu do it automatically for you.
>>
>> qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on
>> another node.
>>
>
> We can live with it, either have qemu realize the kernel version out 
> of another existing feature or query uname.
>
> Alternatively, the matching libvirt package can be the one adding or 
> removing it in the right distribution.

There's another option.

Make cpuid information part of live migration protocol, and then support 
something like -cpu Xeon-3550.  We would remember the exact cpuid mask 
we present to the guest and then we could validate that we can obtain 
the same mask on the destination.

Regards,

Anthony Liguori

> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 02:33 PM, Anthony Liguori wrote:
>
> There's another option.
>
> Make cpuid information part of live migration protocol, and then 
> support something like -cpu Xeon-3550.  We would remember the exact 
> cpuid mask we present to the guest and then we could validate that we 
> can obtain the same mask on the destination.

Currently, our policy is to only migrate dynamic (from the guest's point 
of view) state, and specify static state on the command line [1].

I think your suggestion makes a lot of sense, but I'd like to expand it 
to move all guest state, whether dynamic or static.  So '-m 1G' would be 
migrated as well (but not -mem-path).  Similarly, in -drive 
file=...,if=ide,index=1, everything but file=... would be migrated.

This has an advantage wrt hotplug: since qemu is responsible for 
migrating all guest visible information, the migrator is no longer 
responsible for replaying hotplug events in the exact sequence they 
happened.

In short, I think we should apply your suggestion as broadly as possible.

[1] cpuid state is actually dynamic; repeated cpuid instruction 
execution with the same operands can return different results.  kvm 
supports querying and setting this state.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Daniel P. Berrange]

On Thu, Jan 07, 2010 at 02:40:34PM +0200, Avi Kivity wrote:
> On 01/07/2010 02:33 PM, Anthony Liguori wrote:
> >
> >There's another option.
> >
> >Make cpuid information part of live migration protocol, and then 
> >support something like -cpu Xeon-3550.  We would remember the exact 
> >cpuid mask we present to the guest and then we could validate that we 
> >can obtain the same mask on the destination.
> 
> Currently, our policy is to only migrate dynamic (from the guest's point 
> of view) state, and specify static state on the command line [1].
> 
> I think your suggestion makes a lot of sense, but I'd like to expand it 
> to move all guest state, whether dynamic or static.  So '-m 1G' would be 
> migrated as well (but not -mem-path).  Similarly, in -drive 
> file=...,if=ide,index=1, everything but file=... would be migrated.
> 
> This has an advantage wrt hotplug: since qemu is responsible for 
> migrating all guest visible information, the migrator is no longer 
> responsible for replaying hotplug events in the exact sequence they 
> happened.

With the introduction of the new -device spport, there's no need to
replay hotplug events in order any more. Instead just use static
PCI addresses when starting the guest, and the same addresses after
migration. You could argue that QEMU should preserve the addressing
automatically during migration, but apps need to do it manually
already to keep addreses stable across power-offs, so doing it manually
across migration too is no extra burden.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 02:47 PM, Daniel P. Berrange wrote:
>
> With the introduction of the new -device spport, there's no need to
> replay hotplug events in order any more. Instead just use static
> PCI addresses when starting the guest, and the same addresses after
> migration. You could argue that QEMU should preserve the addressing
> automatically during migration, but apps need to do it manually
> already to keep addreses stable across power-offs, so doing it manually
> across migration too is no extra burden.
>
>    

That's true - shutdown and startup are an equivalent problem to live 
migration from that point of view.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/07/2010 06:40 AM, Avi Kivity wrote:
> On 01/07/2010 02:33 PM, Anthony Liguori wrote:
>>
>> There's another option.
>>
>> Make cpuid information part of live migration protocol, and then 
>> support something like -cpu Xeon-3550.  We would remember the exact 
>> cpuid mask we present to the guest and then we could validate that we 
>> can obtain the same mask on the destination.
>
> Currently, our policy is to only migrate dynamic (from the guest's 
> point of view) state, and specify static state on the command line [1].
>
> I think your suggestion makes a lot of sense, but I'd like to expand 
> it to move all guest state, whether dynamic or static.  So '-m 1G' 
> would be migrated as well (but not -mem-path).  Similarly, in -drive 
> file=...,if=ide,index=1, everything but file=... would be migrated.

Yes, I agree with this and it should be in the form of an fdt.  This 
means we need full qdev conversion.

But I think cpuid is somewhere in the middle with respect to static vs. 
dynamic.  For instance, -cpu host is very dynamic in that you get very 
difficult results on different systems.  Likewise, because of kvm 
filtering, even -cpu qemu64 can be dynamic.

So if we didn't have filtering and -cpu host, I'd agree that it's 
totally static but I think in the current state, it's dynamic.

> This has an advantage wrt hotplug: since qemu is responsible for 
> migrating all guest visible information, the migrator is no longer 
> responsible for replaying hotplug events in the exact sequence they 
> happened.

Yup, 100% in agreement as a long term goal.

> In short, I think we should apply your suggestion as broadly as possible.
>
> [1] cpuid state is actually dynamic; repeated cpuid instruction 
> execution with the same operands can return different results.  kvm 
> supports querying and setting this state.

Yes, and we save some cpuid state in cpu.  We just don't save all of it.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 03:14 PM, Anthony Liguori wrote:
> On 01/07/2010 06:40 AM, Avi Kivity wrote:
>> On 01/07/2010 02:33 PM, Anthony Liguori wrote:
>>>
>>> There's another option.
>>>
>>> Make cpuid information part of live migration protocol, and then
>>> support something like -cpu Xeon-3550. We would remember the exact
>>> cpuid mask we present to the guest and then we could validate that we
>>> can obtain the same mask on the destination.

It solves controlling the destination qemu execution all right but does 
not change the initial spawning of the original guest - to know whether 
,-syscall is needed or not.

Anyway, I'm in favor of it too.

>>
>> Currently, our policy is to only migrate dynamic (from the guest's
>> point of view) state, and specify static state on the command line [1].
>>
>> I think your suggestion makes a lot of sense, but I'd like to expand
>> it to move all guest state, whether dynamic or static. So '-m 1G'
>> would be migrated as well (but not -mem-path). Similarly, in -drive
>> file=...,if=ide,index=1, everything but file=... would be migrated.
>
> Yes, I agree with this and it should be in the form of an fdt. This
> means we need full qdev conversion.
>
> But I think cpuid is somewhere in the middle with respect to static vs.
> dynamic. For instance, -cpu host is very dynamic in that you get very
> difficult results on different systems. Likewise, because of kvm
> filtering, even -cpu qemu64 can be dynamic.
>
> So if we didn't have filtering and -cpu host, I'd agree that it's
> totally static but I think in the current state, it's dynamic.
>
>> This has an advantage wrt hotplug: since qemu is responsible for
>> migrating all guest visible information, the migrator is no longer
>> responsible for replaying hotplug events in the exact sequence they
>> happened.
>
> Yup, 100% in agreement as a long term goal.
>
>> In short, I think we should apply your suggestion as broadly as possible.
>>
>> [1] cpuid state is actually dynamic; repeated cpuid instruction
>> execution with the same operands can return different results. kvm
>> supports querying and setting this state.
>
> Yes, and we save some cpuid state in cpu. We just don't save all of it.
>
> Regards,
>
> Anthony Liguori
>

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Markus Armbruster]

Avi Kivity <avi@redhat.com> writes:

> On 01/07/2010 02:33 PM, Anthony Liguori wrote:
>>
>> There's another option.
>>
>> Make cpuid information part of live migration protocol, and then
>> support something like -cpu Xeon-3550.  We would remember the exact
>> cpuid mask we present to the guest and then we could validate that
>> we can obtain the same mask on the destination.
>
> Currently, our policy is to only migrate dynamic (from the guest's
> point of view) state, and specify static state on the command line
> [1].
>
> I think your suggestion makes a lot of sense, but I'd like to expand
> it to move all guest state, whether dynamic or static.  So '-m 1G'
> would be migrated as well (but not -mem-path).  Similarly, in -drive
> file=...,if=ide,index=1, everything but file=... would be migrated.

Becomes a bit clearer with the new way to configure stuff:

  -drive if=none,id=DRIVE-ID,...        <--- host, don't migrate
  -device ide=drive,drive=DRIVE-ID,...  <--- guest, do migrate

[...]

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/07/2010 11:40 AM, Dor Laor wrote:
>> There's no such thing as Nehalem.
>
>
> Intel were ok with it. Again, you can name is corei7 or 
> xeon34234234234, I don't care, the principle remains the same.
>

There are several processors belonging to the Nehalem family and each 
have different features.

>
> What's not simple about the above 4 options?

If a qemu/kvm/processor combo doesn't support a feature (say, nx) we 
have to remove it from the migration pool even if the Nehalem processor 
class says it's included.  Or else not admit that combination into the 
migration pool in the first place.

> What's a better alternative (that insures users understand it and use 
> it and guest msi and even skype application is happy about it)?
>

Have management scan new nodes and classify them.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 01:59 PM, Avi Kivity wrote:
> On 01/07/2010 11:40 AM, Dor Laor wrote:
>>> There's no such thing as Nehalem.
>>
>>
>> Intel were ok with it. Again, you can name is corei7 or
>> xeon34234234234, I don't care, the principle remains the same.
>>
>
> There are several processors belonging to the Nehalem family and each
> have different features.

We can start with the older one, and once it get's that important add 
the newer ones.
Until that happens users can either use -host or have 
nehalem,+sse4_2,+newFeature

>
>>
>> What's not simple about the above 4 options?
>
> If a qemu/kvm/processor combo doesn't support a feature (say, nx) we
> have to remove it from the migration pool even if the Nehalem processor
> class says it's included. Or else not admit that combination into the
> migration pool in the first place.

It still management role to compute least common denominator for live 
migration sets.

btw: for nx disabled bios, we should just ignore it from product. Qemu 
direct users should add ,-nx

>
>> What's a better alternative (that insures users understand it and use
>> it and guest msi and even skype application is happy about it)?
>>
>
> Have management scan new nodes and classify them.
>

Of course. Just don't let management control stepping automatically, it 
should only be made for very advanced users and only manually.

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Daniel P. Berrange]

On Thu, Jan 07, 2010 at 10:03:28AM +0200, Dor Laor wrote:
> On 01/06/2010 05:16 PM, Anthony Liguori wrote:
> >On 01/06/2010 08:48 AM, Dor Laor wrote:
> >>On 01/06/2010 04:32 PM, Avi Kivity wrote:
> >>>On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
> >>>>>We can probably default -enable-kvm to -cpu host, as long as we
> >>>>>explain
> >>>>>very carefully that if users wish to preserve cpu features across
> >>>>>upgrades, they can't depend on the default.
> >>>>Hardware upgrades or software upgrades?
> >>>
> >>>Yes.
> >>>
> >>
> >>I just want to remind all the the main motivation for using -cpu
> >>realModelThatWasOnceShiped is to provide correct cpu emulation for the
> >>guest. Using a random qemu|kvm64+flag1-flag2 might really cause
> >>trouble for the guest OS or guest apps.
> >>
> >>On top of -cpu nehalem we can always add fancy features like x2apic, etc.
> >
> >I think it boils down to, how are people going to use this.
> >
> >For individuals, code names like Nehalem are too obscure. From my own
> >personal experience, even power users often have no clue whether there
> >processor is a Nehalem or not.
> >
> >For management tools, Nehalem is a somewhat imprecise target because it
> >covers a wide range of potential processors. In general, I think what we
> >really need to do is simplify the process of going from, here's the
> >output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu
> >so that migration always works for these systems.
> >
> >I don't think -cpu nehalem really helps with that problem. -cpu none
> >helps a bit, but I hope we can find something nicer.
> 
> We can debate about the exact name/model to represent the Nehalem 
> family, I don't have an issue with that and actually Intel and Amd 
> should define it.
> 
> There are two main motivations behind the above approach:
> 1. Sound guest cpu definition.
>    Using a predefined model should automatically set all the relevant
>    vendor/stepping/cpuid flags/cache sizes/etc.
>    We just can let every management application deal with it. It breaks
>    guest OS/apps. For instance there are MSI support in windows guest
>    relay on the stepping.
> 
> 2. Simplifying end user and mgmt tools.
>    qemu/kvm have the best knowledge about these low levels. If we push
>    it up in the stack, eventually it reaches the user. The end user,
>    not a 'qemu-devel user' which is actually far better from the
>    average user.
> 
>    This means that such users will have to know what is popcount and
>    whether or not to limit migration on one host by adding sse4.2 or
>    not.
> 
> This is exactly what vmware are doing:
>  - Intel CPUs : 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991
>  - AMD CPUs : 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992
> 
> Why should we invent the wheel (qemu64..)? Let's learn from their 
> experience.

NB, be careful to distinguish the different levels of VMwares mgmt stack. In
terms of guest configuration, VMWare ESX APIs require the management app to
specify the raw CPUID masks. With VirtualCenter VMotion they defined this 
handful of common Intel/AMD CPU sets, and will automatically classify hosts
into one  of these sets and use that to specify a default CPUID mask, in the
case that the guest does not have an explicit one in its config. This gives
them good default, out-of-the-box behaviour, while also allowing mgmt apps
100% control over each guest's CPUID should they want it.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Dor Laor]

On 01/07/2010 10:24 AM, Daniel P. Berrange wrote:
> On Thu, Jan 07, 2010 at 10:03:28AM +0200, Dor Laor wrote:
>> On 01/06/2010 05:16 PM, Anthony Liguori wrote:
>>> On 01/06/2010 08:48 AM, Dor Laor wrote:
>>>> On 01/06/2010 04:32 PM, Avi Kivity wrote:
>>>>> On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>>>>>>> We can probably default -enable-kvm to -cpu host, as long as we
>>>>>>> explain
>>>>>>> very carefully that if users wish to preserve cpu features across
>>>>>>> upgrades, they can't depend on the default.
>>>>>> Hardware upgrades or software upgrades?
>>>>>
>>>>> Yes.
>>>>>
>>>>
>>>> I just want to remind all the the main motivation for using -cpu
>>>> realModelThatWasOnceShiped is to provide correct cpu emulation for the
>>>> guest. Using a random qemu|kvm64+flag1-flag2 might really cause
>>>> trouble for the guest OS or guest apps.
>>>>
>>>> On top of -cpu nehalem we can always add fancy features like x2apic, etc.
>>>
>>> I think it boils down to, how are people going to use this.
>>>
>>> For individuals, code names like Nehalem are too obscure. From my own
>>> personal experience, even power users often have no clue whether there
>>> processor is a Nehalem or not.
>>>
>>> For management tools, Nehalem is a somewhat imprecise target because it
>>> covers a wide range of potential processors. In general, I think what we
>>> really need to do is simplify the process of going from, here's the
>>> output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu
>>> so that migration always works for these systems.
>>>
>>> I don't think -cpu nehalem really helps with that problem. -cpu none
>>> helps a bit, but I hope we can find something nicer.
>>
>> We can debate about the exact name/model to represent the Nehalem
>> family, I don't have an issue with that and actually Intel and Amd
>> should define it.
>>
>> There are two main motivations behind the above approach:
>> 1. Sound guest cpu definition.
>>     Using a predefined model should automatically set all the relevant
>>     vendor/stepping/cpuid flags/cache sizes/etc.
>>     We just can let every management application deal with it. It breaks
>>     guest OS/apps. For instance there are MSI support in windows guest
>>     relay on the stepping.
>>
>> 2. Simplifying end user and mgmt tools.
>>     qemu/kvm have the best knowledge about these low levels. If we push
>>     it up in the stack, eventually it reaches the user. The end user,
>>     not a 'qemu-devel user' which is actually far better from the
>>     average user.
>>
>>     This means that such users will have to know what is popcount and
>>     whether or not to limit migration on one host by adding sse4.2 or
>>     not.
>>
>> This is exactly what vmware are doing:
>>   - Intel CPUs :
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991
>>   - AMD CPUs :
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992
>>
>> Why should we invent the wheel (qemu64..)? Let's learn from their
>> experience.
>
> NB, be careful to distinguish the different levels of VMwares mgmt stack. In
> terms of guest configuration, VMWare ESX APIs require the management app to
> specify the raw CPUID masks. With VirtualCenter VMotion they defined this
> handful of common Intel/AMD CPU sets, and will automatically classify hosts

Live migration is the prime motivation for it.
In addition, as we all know windows guest do not like to find a new cpu 
every time they boot.

> into one  of these sets and use that to specify a default CPUID mask, in the
> case that the guest does not have an explicit one in its config. This gives
> them good default, out-of-the-box behaviour, while also allowing mgmt apps
> 100% control over each guest's CPUID should they want it.

That's exactly what we need.

>
> Regards,
> Daniel

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Wed, Jan 06, 2010 at 04:32:07PM +0200, Avi Kivity wrote:
> On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote:
>>> We can probably default -enable-kvm to -cpu host, as long as we explain
>>> very carefully that if users wish to preserve cpu features across
>>> upgrades, they can't depend on the default.
>>>      
>> Hardware upgrades or software upgrades?
>>    
>
> Yes.

IMO software upgrades should not change anything as long as user uses -M
<machine description>.  CPU flags should be whitelisted in machine
description.

> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Anthony Liguori]

On 01/06/2010 07:54 AM, Avi Kivity wrote:
> On 01/06/2010 03:49 PM, Anthony Liguori wrote:
>>>
>>>>
>>>> I think that's workable but I think there may be some subtle issues 
>>>> especially across qemu versions.  Can you give an example of what 
>>>> you would expect the output to be?
>>>
>>> -> { command: query-cpu-capabalities }
>>> <- { result: { features: [vm, fpu, lm, sse2, sse3, sssse3, 
>>> ssssssse3, sse3.14 ], cache: { ... }, vendor: { .... }, etc. } }
>>>
>>> Or something.  We'd need similar queries for the number of PCI 
>>> slots, for example, so the GUI can tell the user when adding 
>>> hardware is no longer an option instead of trying it blindly and 
>>> returning an error.
>>
>>
>> Yeah, the trick with this is that we don't have such a thing as a 
>> bare bones CPU in qemu right now.
>>
>> So you can't really say -cpu qemu64+vm+fpu+lm....
>>
>> Because some extra features might sneak in.  What I'm getting at is 
>> that we need a way to make the results of this command translate into 
>> a -cpu invocation that works reliably across multiple versions of qemu.
>
> Well, we can freeze qemu64 if we wish.  That's still not 100% accurate 
> since kvm can remove features from qemu64.
>
> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?

Yeah, I'm not sure I can think of a better way.

Regards,

Anthony Liguori

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Daniel P. Berrange]

On Tue, Jan 05, 2010 at 06:10:10PM -0600, Anthony Liguori wrote:
> >>For instance, Xeon-X5570 should be a least common denominator for 
> >>Nehalem processors.  It's probably better for users too.  It's easier 
> >>for them to answer "do I have anything older than a Xeon-X5570" than 
> >>to ask "do I have any Woodcrest class processors".
> >>
> >>I encounter this confusion a lot.  I usually ask people whether they 
> >>have a Nehalem processor when debugging something and their response 
> >>is always, I have a Xeon-XYZ, is that Nehalem?
> >
> >This is complicated by the fact that processors don't have 
> >straight-line development, and that marketing names don't correspond 
> >to anything.  For example Xeon can be any of the Pentium 4, Core, Core 
> >2, and Nehalem (and perhaps other) microachitectures.  A newer Xeon is 
> >often introduced with an older core (usually for larger machines) than 
> >previously existing Xeons.
> 
> Typically, there is at least a little sanity naming for these cases.  
> For instance, any Xeon W35xx should have the same features.  A Xeon 
> W55xx may be different.
> 
> It's not going to be easy to include every possible model.  It's a hard 
> problem for management tools too.  The thing is, I imagine most 
> management tools are going to cat /proc/cpuinfo to get what the 
> processor is and that's going to be a Xeon YYXXXX type name so I really 
> believe that's the thing that makes sense to expose in QEMU.
> 
> Maybe we could name models like IntelXeonW35xx.

For libvirt, we have an XML database that contains a list of CPU model
names, and (for x86) associated CPU ID flags. The guest XML can specify
a model name, and we'll map that through to the matching QEMU model
name if there's an exact match. Otherwise we'll map it to the closest
QEMU model name + a set of named flags. The guest XML can also contain
a list of named flags ontop of the basic model, and again we'll map that
through to whatever QEMU provides. So apps using libvirt don't worry 
about what QEMU's exact naming scheme is for processors, but rather use
libvirt's database of defined names which are mapped to QEMUs.  Now for
convenience our database happens to match the QEMU database exactly at
this time :-) Finally we also use the same CPU model databse to expose
what the host CPU is, and provide APIs to allow apps to query whether a
desired guest model is compatible with the current host CPU model

The reason for libvirt doing it in this way, is that we wanted to have
apps use CPU model names primarily, but with option to override flags if
they needed. For the Xen and VMWare drivers the CPU model names are in
fact changed into CPUID masks, since that's all those two HVs accept.

This is all a very long way of saying that mgmt apps based on libvirt
won't care about model names exposed in /proc/cpuinfo so there's no
particular need to have a direct mapping from them to QEMU for libvirt's
needs.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 11:44 AM, Daniel P. Berrange wrote:
>
> This is all a very long way of saying that mgmt apps based on libvirt
> won't care about model names exposed in /proc/cpuinfo so there's no
> particular need to have a direct mapping from them to QEMU for libvirt's
> needs.
>    

There is still a need to query qemu, since it will filter out bits that 
kvm.ko or the hardware don't support.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Daniel P. Berrange]

On Wed, Jan 06, 2010 at 11:54:16AM +0200, Avi Kivity wrote:
> On 01/06/2010 11:44 AM, Daniel P. Berrange wrote:
> >
> >This is all a very long way of saying that mgmt apps based on libvirt
> >won't care about model names exposed in /proc/cpuinfo so there's no
> >particular need to have a direct mapping from them to QEMU for libvirt's
> >needs.
> >   
> 
> There is still a need to query qemu, since it will filter out bits that
> kvm.ko or the hardware don't support.

If there was a more verbose version of '-cpu ?' which also gave the CPUID
masks associated with each supported CPU model, that would be useful info
avoiding need for libvirt to hardcode the mapping to QEMU

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 01/06/2010 12:21 PM, Daniel P. Berrange wrote:
> On Wed, Jan 06, 2010 at 11:54:16AM +0200, Avi Kivity wrote:
>    
>> On 01/06/2010 11:44 AM, Daniel P. Berrange wrote:
>>      
>>> This is all a very long way of saying that mgmt apps based on libvirt
>>> won't care about model names exposed in /proc/cpuinfo so there's no
>>> particular need to have a direct mapping from them to QEMU for libvirt's
>>> needs.
>>>
>>>        
>> There is still a need to query qemu, since it will filter out bits that
>> kvm.ko or the hardware don't support.
>>      
> If there was a more verbose version of '-cpu ?' which also gave the CPUID
> masks associated with each supported CPU model, that would be useful info
> avoiding need for libvirt to hardcode the mapping to QEMU
>    

Yes, that's what I'm thinking of.  Along with enumerating all the other 
qemu capabilities.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Lennart Sorensen]

On Tue, Jan 05, 2010 at 06:10:10PM -0600, Anthony Liguori wrote:
> Typically, there is at least a little sanity naming for these cases.   
> For instance, any Xeon W35xx should have the same features.  A Xeon  
> W55xx may be different.

It doesn't work that way for intel.  For example:

Core 2 Duo E7400 AT80571PH0723M does not have VT support.
Core 2 Duo E7400 AT80571PH0723ML does have VT support.

Good luck buying the variant you want.  So model names do not define
the feature set.

> It's not going to be easy to include every possible model.  It's a hard  
> problem for management tools too.  The thing is, I imagine most  
> management tools are going to cat /proc/cpuinfo to get what the  
> processor is and that's going to be a Xeon YYXXXX type name so I really  
> believe that's the thing that makes sense to expose in QEMU.
>
> Maybe we could name models like IntelXeonW35xx.
>
> Clever use of the preprocessor will make this effort much, much saner.   
> Also, we really need some sort of human readable document that explains  
> the differences between processor classes and makes recommendations  
> about what management tools should take into consideration.

-- 
Len Sorensen

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 07:59 PM, Anthony Liguori wrote:
> Gleb Natapov wrote:
>> Windows is a mystery box, so we can speculate as much as we want 
>> about it.
>> If you don't like something just say "it may break Windows" :) Losing
>> activation does sound like an issue too.
>
> Downgrading seems more likely to cause problems.  Considering running 
> mplayer in a guest.  If it detects SSE3 in one host and migrate to 
> another host that doesn't have SSE3, you'll be running an instruction 
> stream that uses instructions the processor doesn't support resulting 
> in the application faulting due to an illegal operation.
>

Migration needs preparation beforehand.  You can't take two random hosts 
with two random qemu flag sets and expect it to work.

> KVM's cpuid filter doesn't help here because it won't attempt to mask 
> things like sse3.  It would be insane to emulate sse3 too.

It does expose sse3 support (called pni in Linux IIRC).  Not sure if 
qemu masks it, but the information is there.

>
> This is a classic problem with live migration.  You can detect this 
> scenario and potentially block the migration, but honestly, how likely 
> is it that any given guest is using an obscure feature? 

100% likely.

> This is a classic management tool problem and the solution usually is 
> a set of heuristics depending on how conservative the target audience is.

Right.  My concern is with casual users upgrading their machine, not 
enterprise users who should be protected by their tools.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Andre Przywara]

Avi Kivity wrote:
> On 12/20/2009 07:59 PM, Anthony Liguori wrote:
>> Gleb Natapov wrote:
>>> Windows is a mystery box, so we can speculate as much as we want 
>>> about it.
>>> If you don't like something just say "it may break Windows" :) Losing
>>> activation does sound like an issue too.
>>
>> Downgrading seems more likely to cause problems.  Considering running 
>> mplayer in a guest.  If it detects SSE3 in one host and migrate to 
>> another host that doesn't have SSE3, you'll be running an instruction 
>> stream that uses instructions the processor doesn't support resulting 
>> in the application faulting due to an illegal operation.
>>
> 
> Migration needs preparation beforehand.  You can't take two random hosts 
> with two random qemu flag sets and expect it to work.
> 
>> KVM's cpuid filter doesn't help here because it won't attempt to mask 
>> things like sse3.  It would be insane to emulate sse3 too.
> 
> It does expose sse3 support (called pni in Linux IIRC).  Not sure if 
> qemu masks it, but the information is there.
What about using -cpu kvm64?
This has SSE3 as well as CMPXCHG16, both are available in all VMX/SVM 
capable machines (as far as I could find out) and in TCG.
This gives a pretty descent base without loosing many relevant 
performance features.

>... 
>> This is a classic management tool problem and the solution usually is 
>> a set of heuristics depending on how conservative the target audience is.
> 
> Right.  My concern is with casual users upgrading their machine, not 
> enterprise users who should be protected by their tools.
Then what about fixing the CPUID bits to those returned by the KVM 
module the first time the guest is started? Later you would only use 
those bits (which may have been cropped) for later restarts of the same 
guest. If you only upgrade your machine, then there should be no problems.

Regards,
Andre.

P.S. What feature bits do we talk about?
Maybe we should group them and use aliases for those.
SS_S_E3 and SSE4.x come to mind, are any other bits really relevant? (Or 
do they justify the pain we have when propagating them?)
Then we would only need: Athlon64, Core2, Nehalem (and maybe Phenom).

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/21/2009 02:59 PM, Andre Przywara wrote:
>>
>>> KVM's cpuid filter doesn't help here because it won't attempt to 
>>> mask things like sse3.  It would be insane to emulate sse3 too.
>>
>> It does expose sse3 support (called pni in Linux IIRC).  Not sure if 
>> qemu masks it, but the information is there.
>
> What about using -cpu kvm64?
> This has SSE3 as well as CMPXCHG16, both are available in all VMX/SVM 
> capable machines (as far as I could find out) and in TCG.
> This gives a pretty descent base without loosing many relevant 
> performance features.

Sure.  This gives more performance to users and preserves compatibility 
pretty well.  The disadvantage is that the baseline will recede as more 
processor features are introduced.

>> ...
>>> This is a classic management tool problem and the solution usually 
>>> is a set of heuristics depending on how conservative the target 
>>> audience is.
>>
>> Right.  My concern is with casual users upgrading their machine, not 
>> enterprise users who should be protected by their tools.
> Then what about fixing the CPUID bits to those returned by the KVM 
> module the first time the guest is started? Later you would only use 
> those bits (which may have been cropped) for later restarts of the 
> same guest. If you only upgrade your machine, then there should be no 
> problems.

First, even an upgrade may cause loss of cpuid bits (moving to a 
different vendor).  Second, where do you store those bits?

>
> P.S. What feature bits do we talk about?
> Maybe we should group them and use aliases for those.
> SS_S_E3 and SSE4.x come to mind, are any other bits really relevant? 
> (Or do they justify the pain we have when propagating them?)
> Then we would only need: Athlon64, Core2, Nehalem (and maybe Phenom).

Future bits too, for example AVX.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Jamie Lokier]

Anthony Liguori wrote:
> It would be insane to emulate sse3 too.

It doesn't sound too ridiculous if TCG is involved, provided the
switching between TCG and KVM isn't too rapid.  TCG is slower, but
it's not ridiculously slow.

Though, I don't expect anyone to volunteer to implement it :-)

> how likely is it that any given guest is using an obscure feature?

Could KVM detect when a guest starts using each feature?  For example,
by disabling access to those features, enabling each one when the KVM
trap occurs and recording that fact?  Is it possible to enable them in
the guest-visible cpuid, but force a trap on access to them?

That might be quite useful information when you want to migrate
something that wasn't planned for X years ago.

-- Jamie

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
>
>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>> emulate well enough to make windows not crash?
>>>        
>> -cpu host does that already, no?
>>
>> Alex
>>      
> I expected so, but Avi here seems to say windows will crash if you
> use a new CPU with it ...
>    

No, Windows tries to detect changes in your hardware and assumes that if 
too many things change, you might be a pirate and requires you to phone 
their offices to re-authenticate.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote:
> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
>>
>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>>> emulate well enough to make windows not crash?
>>>>        
>>> -cpu host does that already, no?
>>>
>>> Alex
>>>      
>> I expected so, but Avi here seems to say windows will crash if you
>> use a new CPU with it ...
>>    
>
> No, Windows tries to detect changes in your hardware and assumes that if  
> too many things change, you might be a pirate and requires you to phone  
> their offices to re-authenticate.

How often does a casual user upgrade his host CPU?

> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 21.12.2009, at 12:18, Michael S. Tsirkin wrote:

> On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote:
>> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
>>> 
>>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>>>> emulate well enough to make windows not crash?
>>>>> 
>>>> -cpu host does that already, no?
>>>> 
>>>> Alex
>>>> 
>>> I expected so, but Avi here seems to say windows will crash if you
>>> use a new CPU with it ...
>>> 
>> 
>> No, Windows tries to detect changes in your hardware and assumes that if  
>> too many things change, you might be a pirate and requires you to phone  
>> their offices to re-authenticate.
> 
> How often does a casual user upgrade his host CPU?

I tend to have my VM images on an NFS share and expect them to work properly on all PCs I work on. So I guess the answer is "often"?

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote:
> 
> On 21.12.2009, at 12:18, Michael S. Tsirkin wrote:
> 
> > On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote:
> >> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
> >>> 
> >>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
> >>>>> emulate well enough to make windows not crash?
> >>>>> 
> >>>> -cpu host does that already, no?
> >>>> 
> >>>> Alex
> >>>> 
> >>> I expected so, but Avi here seems to say windows will crash if you
> >>> use a new CPU with it ...
> >>> 
> >> 
> >> No, Windows tries to detect changes in your hardware and assumes that if  
> >> too many things change, you might be a pirate and requires you to phone  
> >> their offices to re-authenticate.
> > 
> > How often does a casual user upgrade his host CPU?
> 
> I tend to have my VM images on an NFS share and expect them to work properly on all PCs I work on.
> So I guess the answer is "often"?
> 
> Alex

Yes but you are not a casual user, are you?  Consider a 64 bit guest to
see why moving a VM across machines needs some planning.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Alexander Graf]

On 21.12.2009, at 12:38, Michael S. Tsirkin wrote:

> On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote:
>> 
>> On 21.12.2009, at 12:18, Michael S. Tsirkin wrote:
>> 
>>> On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote:
>>>> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
>>>>> 
>>>>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>>>>>> emulate well enough to make windows not crash?
>>>>>>> 
>>>>>> -cpu host does that already, no?
>>>>>> 
>>>>>> Alex
>>>>>> 
>>>>> I expected so, but Avi here seems to say windows will crash if you
>>>>> use a new CPU with it ...
>>>>> 
>>>> 
>>>> No, Windows tries to detect changes in your hardware and assumes that if  
>>>> too many things change, you might be a pirate and requires you to phone  
>>>> their offices to re-authenticate.
>>> 
>>> How often does a casual user upgrade his host CPU?
>> 
>> I tend to have my VM images on an NFS share and expect them to work properly on all PCs I work on.
>> So I guess the answer is "often"?
>> 
>> Alex
> 
> Yes but you are not a casual user, are you?  

Well, we have two groups:

1) Casual user w/o management app
2) Enterprise user w/ management app

So I clearly belong to the first group.

> Consider a 64 bit guest to
> see why moving a VM across machines needs some planning.

-ENOPARSE

We're still talking about bootup on different machines, not migration / loadvm.

Alex

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 12:45:26PM +0100, Alexander Graf wrote:
> 
> On 21.12.2009, at 12:38, Michael S. Tsirkin wrote:
> 
> > On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote:
> >> 
> >> On 21.12.2009, at 12:18, Michael S. Tsirkin wrote:
> >> 
> >>> On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote:
> >>>> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
> >>>>> 
> >>>>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
> >>>>>>> emulate well enough to make windows not crash?
> >>>>>>> 
> >>>>>> -cpu host does that already, no?
> >>>>>> 
> >>>>>> Alex
> >>>>>> 
> >>>>> I expected so, but Avi here seems to say windows will crash if you
> >>>>> use a new CPU with it ...
> >>>>> 
> >>>> 
> >>>> No, Windows tries to detect changes in your hardware and assumes that if  
> >>>> too many things change, you might be a pirate and requires you to phone  
> >>>> their offices to re-authenticate.
> >>> 
> >>> How often does a casual user upgrade his host CPU?
> >> 
> >> I tend to have my VM images on an NFS share and expect them to work properly on all PCs I work on.
> >> So I guess the answer is "often"?
> >> 
> >> Alex
> > 
> > Yes but you are not a casual user, are you?  
> 
> Well, we have two groups:
> 
> 1) Casual user w/o management app
> 2) Enterprise user w/ management app
> 
> So I clearly belong to the first group.

No, you are in
3) virtualization hacker without management app
:)

> > Consider a 64 bit guest to
> > see why moving a VM across machines needs some planning.
> 
> -ENOPARSE
> 
> We're still talking about bootup on different machines, not migration / loadvm.
> 
> Alex

Translation: your requirement "work on all PCs I work on" might only be satisfied
by specifying the set of PCs you work on.

Bootup on different machines has some of the same issues as migration.
Consider a 64 bit guest as an example, I think it can not boot on a 32
bit host OS with kvm. I think you can use tcg, but it will be slower.
Same thing applies to other CPU features.

Many guests reconfigure themselves when you swap harware under them.
This makes it easier to move such guests between machines, or change
qemu configuration and still reuse guest image.  Others might record
hardware state on setup (64 bit above is one such example) making such
changes more painful.

I do think it would be good for us to supply management with utilities
that analyse system hardware features relevant to qemu in a portable
way. Could be some qemu monitor command, even.  Management would be able
to decide between using least common denominator and not supporting some
hosts.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Jamie Lokier]

Michael S. Tsirkin wrote:
> Bootup on different machines has some of the same issues as migration.
> Consider a 64 bit guest as an example, I think it can not boot on a 32
> bit host OS with kvm. I think you can use tcg, but it will be slower.
> Same thing applies to other CPU features.

Alas, perhaps tcg will not work.

The impression I got from irqchip threads was if you have a 64 bit KVM
guest and only access to a 32 bit host, you are screwed because
KVM/QEMU irqchip models are not interchangable.  Could be wrong
though, it was a bit too in depth for my knowledge.

> Many guests reconfigure themselves when you swap harware under them.
> This makes it easier to move such guests between machines, or change
> qemu configuration and still reuse guest image.  Others might record
> hardware state on setup (64 bit above is one such example) making such
> changes more painful.

Windows is notorious for breaking when the hardware changes.  Ignoring
the activation issue, which doesn't apply to corporate licenses or
time-limited free issues, for example replacing IDE on PIIX4 with IDE
on PIIX3 was enough to prevent both Windows 2000 and Windows Server
2003 from booting at all.  Same when switching between SCSI and IDE,
or vice versa.

That's because they load the drivers needed for booting from a
preassembled list of drivers that it needed list time, basically just
the specific IDE or SCSI driver and a few other things, and they look
for exact PCI ID matches to load them.

I think the behaviour goes back to NT days, and is much worse for
their "server" class OSes; their consumer ones like XP try harder to
detect and cope with changes.

In a nutshell, I have had to take a disk image from a real Windows
machine to a working KVM image several times, and I've never been able
to do it without either a technical blue screen on boot problem or an
activation problem.

-- Jamie

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/21/2009 01:45 PM, Alexander Graf wrote:
>
> Well, we have two groups:
>
> 1) Casual user w/o management app
> 2) Enterprise user w/ management app
>
> So I clearly belong to the first group.
>    

3) Developer/power user who knows what he's about.

You could simply add -cpu qemu64 for those guests that care about it.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[David S. Ahern]

On 12/21/2009 05:05 AM, Avi Kivity wrote:
> On 12/21/2009 01:45 PM, Alexander Graf wrote:
>>
>> Well, we have two groups:
>>
>> 1) Casual user w/o management app
>> 2) Enterprise user w/ management app
>>
>> So I clearly belong to the first group.
>>    
> 
> 3) Developer/power user who knows what he's about.
> 
> You could simply add -cpu qemu64 for those guests that care about it.
> 

4) embedded virtualization where the use of a management app provides
little to no added benefit and everything has to be
"automated" (ie., no user).

My point is there are other use cases besides data center deployments
(aka enterprise) and workstation (casual/power user). There are use
cases where virtualization is just yet another tool to achieve a product.

David

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 06:45:22AM -0700, David S. Ahern wrote:
> 
> On 12/21/2009 05:05 AM, Avi Kivity wrote:
> > On 12/21/2009 01:45 PM, Alexander Graf wrote:
> >>
> >> Well, we have two groups:
> >>
> >> 1) Casual user w/o management app
> >> 2) Enterprise user w/ management app
> >>
> >> So I clearly belong to the first group.
> >>    
> > 
> > 3) Developer/power user who knows what he's about.
> > 
> > You could simply add -cpu qemu64 for those guests that care about it.
> > 
> 
> 4) embedded virtualization where the use of a management app provides
> little to no added benefit and everything has to be
> "automated" (ie., no user).
> 
> My point is there are other use cases besides data center deployments
> (aka enterprise) and workstation (casual/power user). There are use
> cases where virtualization is just yet another tool to achieve a product.
> 
> David

Yes, but unless someone runs qemu directly, default value for any flag
does not matter much.

-- 
MST

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[David S. Ahern]

On 12/21/2009 06:51 AM, Michael S. Tsirkin wrote:
> On Mon, Dec 21, 2009 at 06:45:22AM -0700, David S. Ahern wrote:
>>
>> On 12/21/2009 05:05 AM, Avi Kivity wrote:
>>> On 12/21/2009 01:45 PM, Alexander Graf wrote:
>>>>
>>>> Well, we have two groups:
>>>>
>>>> 1) Casual user w/o management app
>>>> 2) Enterprise user w/ management app
>>>>
>>>> So I clearly belong to the first group.
>>>>    
>>>
>>> 3) Developer/power user who knows what he's about.
>>>
>>> You could simply add -cpu qemu64 for those guests that care about it.
>>>
>>
>> 4) embedded virtualization where the use of a management app provides
>> little to no added benefit and everything has to be
>> "automated" (ie., no user).
>>
>> My point is there are other use cases besides data center deployments
>> (aka enterprise) and workstation (casual/power user). There are use
>> cases where virtualization is just yet another tool to achieve a product.
>>
>> David
> 
> Yes, but unless someone runs qemu directly, default value for any flag
> does not matter much.
> 

That's what I was getting at - "direct" invocation of qemu in an
automated sense without a libvirt layer. Defaults that do the right
thing on the host would be nice, with tweaking from the defaults only if
something extra is wanted.

David

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/21/2009 03:45 PM, David S. Ahern wrote:
> On 12/21/2009 05:05 AM, Avi Kivity wrote:
>    
>> On 12/21/2009 01:45 PM, Alexander Graf wrote:
>>      
>>> Well, we have two groups:
>>>
>>> 1) Casual user w/o management app
>>> 2) Enterprise user w/ management app
>>>
>>> So I clearly belong to the first group.
>>>
>>>        
>> 3) Developer/power user who knows what he's about.
>>
>> You could simply add -cpu qemu64 for those guests that care about it.
>>
>>      
> 4) embedded virtualization where the use of a management app provides
> little to no added benefit and everything has to be
> "automated" (ie., no user).
>
> My point is there are other use cases besides data center deployments
> (aka enterprise) and workstation (casual/power user). There are use
> cases where virtualization is just yet another tool to achieve a product.
>    

If there is no user, then a management app exists.  It may be as simple 
as execing qemu from initrd, but there is still something that is not a 
user that can set the correct parameters.  It is up to the author of 
this one-line management app to decide what the correct parameters are 
(-cpu host or some other cpu baseline that allows VM portability, if 
needed).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/21/2009 01:18 PM, Michael S. Tsirkin wrote:
>
>> No, Windows tries to detect changes in your hardware and assumes that if
>> too many things change, you might be a pirate and requires you to phone
>> their offices to re-authenticate.
>>      
> How often does a casual user upgrade his host CPU?
>    

I don't know.  Does it matter?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 02:04:47PM +0200, Avi Kivity wrote:
> On 12/21/2009 01:18 PM, Michael S. Tsirkin wrote:
>>
>>> No, Windows tries to detect changes in your hardware and assumes that if
>>> too many things change, you might be a pirate and requires you to phone
>>> their offices to re-authenticate.
>>>      
>> How often does a casual user upgrade his host CPU?
>>    
>
> I don't know.  Does it matter?

I think so - if this does not happen a lot, it's not a problem to phone home,
right?

> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Avi Kivity]

On 12/21/2009 02:04 PM, Michael S. Tsirkin wrote:
>
> I think so - if this does not happen a lot, it's not a problem to phone home,
> right?
>    

I'm sure it's very annoying when it happens.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Michael S. Tsirkin]

On Mon, Dec 21, 2009 at 02:09:31PM +0200, Avi Kivity wrote:
> On 12/21/2009 02:04 PM, Michael S. Tsirkin wrote:
>>
>> I think so - if this does not happen a lot, it's not a problem to phone home,
>> right?
>>    
>
> I'm sure it's very annoying when it happens.

It could well be less annoying than having your guest be slow every day
because qemu masked CPU features that guest needs to be fast.

At least with a CPU upgrade, casual user can easily disagnose the
problem: the system was changed. If qemu is slow by default, there's no
way to find out why without reading the manual.  And we don't have one
:)

> -- 
> error compiling committee.c: too many arguments to function

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Yaniv Kaul]

On 12/21/2009 1:12 PM, Avi Kivity wrote:
> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote:
>>
>>>> Hmm, then, shouldn't either kvm or qemu mask features that we do not
>>>> emulate well enough to make windows not crash?
>>> -cpu host does that already, no?
>>>
>>> Alex
>> I expected so, but Avi here seems to say windows will crash if you
>> use a new CPU with it ...
>
> No, Windows tries to detect changes in your hardware and assumes that 
> if too many things change, you might be a pirate and requires you to 
> phone their offices to re-authenticate.
>

'Just' the CPU is not big deal. Might hit you with two 'bad' points.
Network interface + CPU will require re-activation.
( http://www.aumha.org/win5/a/wpa.php )
Y.

[Qemu-devel] Re: cpuid problem in upstream qemu with kvm

[Paolo Bonzini]

>> No, Windows tries to detect changes in your hardware and assumes that
>> if too many things change, you might be a pirate and requires you to
>> phone their offices to re-authenticate.
>
> 'Just' the CPU is not big deal. Might hit you with two 'bad' points.
> Network interface + CPU will require re-activation.

I think the problem is only for networked images.  For normal images, it 
would be the same if you were running Windows natively---upgrade CPU, 
and you risk having to do reactivation.

For networked images, does changing CPU twice count as changing one 
thing?  If so that's not too bad, not many things are likely to change 
in a VM beyond the CPU.

Besides, if you have networked images, it's relatively likely that 
you're doing it for a company, hence that you have some kind of MSDN 
subscription which grants you unlimited activations for a given product key.

Paolo

Re: [Qemu-devel] Re: cpuid problem in upstream qemu with kvm

[Jamie Lokier]

Paolo Bonzini wrote:
> 
> >>No, Windows tries to detect changes in your hardware and assumes that
> >>if too many things change, you might be a pirate and requires you to
> >>phone their offices to re-authenticate.
> >
> >'Just' the CPU is not big deal. Might hit you with two 'bad' points.
> >Network interface + CPU will require re-activation.
> 
> I think the problem is only for networked images.  For normal images, it 
> would be the same if you were running Windows natively---upgrade CPU, 
> and you risk having to do reactivation.
> 
> For networked images, does changing CPU twice count as changing one 
> thing?  If so that's not too bad, not many things are likely to change 
> in a VM beyond the CPU.
> 
> Besides, if you have networked images, it's relatively likely that 
> you're doing it for a company, hence that you have some kind of MSDN 
> subscription which grants you unlimited activations for a given product key.

Are you joking?  Even as a casual users, when would you ever run
Windows without networking these days???!  I don't think you can even
buy a computer without networking any more :-)

Even just to transfer files from host to guest, you'll typically use a
network.

Or do you mean something else by "networked images" in a company context?

Fwiw, when I have done Windows migrations for companies, they never
have MSDN subscriptions (and neither do I :-) They have purchased
Windows with the machines, and later they want to migrate those
machines into VMs. No hint of MSDN anywhere, why would they?

-- Jamie

Re: [Qemu-devel] cpuid problem in upstream qemu with kvm

[Andre Przywara]

Anthony Liguori wrote:
> Gleb Natapov wrote:
>>> I thought KVM emulates the syscall instruction?  I swear I've seen
>>> patches for that.
>>>
>>>     
>> It is. Starting from 2.6.32.
>>   
> 
> Okay, so this is a performance vs. migration compatibility issue then?
> 
> BTW, couldn't we just not advertise syscall in cpuid?
Looks like a good idea, but at least Linux does not care ;-)
In 64-on-64 bit mode syscall is unconditionally used, which is OK, since 
it is architectural to AMD64. Intel integrated this, too.
In 32bit legacy mode AMD supports both syscall and sysenter, but mostly 
sysenter is used when the SEP CPUID bit is on. Intel does not support 
syscall, but this also is not advertised when CPUID is executed in 32bit.
In 32-on-64 compat mode the support is exclusive: AMD does not support 
sysenter, Intel does not support syscall. Since the AMD CPUID 
advertising is (at least) a bit "unfortunate" here ;-), Linux does not 
care about the bits, but unconditionally uses syscall on AMD and 
sysenter on Intel. See arch/x86/kernel/cpu/intel.c:early_init_intel():
#ifdef CONFIG_X86_64
     set_cpu_cap(c, X86_FEATURE_SYSENTER32);
#else
A similiar line is in cpu/amd.c:early_init_amd().
Then in arch/x86/vdso/vdso32-setup.c:sysenter_setup() these bit are used 
to populate the VDSO page. In the whole process CPUID feature bits are 
not honored, so you cannot get around it.
The only solution would be to inject neither AMD or Intel (I used 
VirtualLinux) in the past. This works fine for Linux (even better than 
the other two, since no vendor-specific workarounds are triggered), but 
64bit Windows refuses to boot on anything other than AuthenticAMD, 
GenuineIntel or CentaurHauls. Linux would then use int80h, which is 
slower than syscall or sysenter, but still much faster than emulation.

So given the whole mess I'd recommend to copy over Avi's "propagate 
local vendor" solution, as this solves the problem for 99.9+% of the 
cases. The remaining cases are cross-vendor migration, where we are fine 
with the 2.6.32 syscall/sysenter emulation. Although this is not as slow 
as one thinks (I measured less than 5% slowdown in kernbench), I'd not 
recommend it for common usage except cross-vendor migration.

BTW: Windows is not affected at all with this compat mode issue, because 
32bit applications switch to 64bit mode before doing the actual system 
call (which uses the 64bit syscall instruction).

> That should fix 
> it too without sacrificing migration compatibility.  We get a slight 
> slowdown on AMD hosts
I think there would be no slowdown. At least Linux only cares about this 
bit if in legacy 32bit mode, but here it uses sysenter anyway if this is 
present:
 From arch/x86/vdso/vdso32-setup.c:
#else  /* CONFIG_X86_32 */
#define vdso32_sysenter()   (boot_cpu_has(X86_FEATURE_SEP))
#define vdso32_syscall()    (0)

> but that's probably minor compared to the cost of 
> using emulated syscall on Intel hosts.


Regards,
Andre.


-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632