Re: vMCE vs migration

[George Dunlap <george.dunlap@citrix.com>]

On Tue, 2012-01-31 at 13:17 +0000, Jan Beulich wrote:
> >>> On 31.01.12 at 12:27, George Dunlap <george.dunlap@citrix.com> wrote:
> > On Mon, 2012-01-30 at 13:47 +0000, Jan Beulich wrote:
> >> >>> On 26.01.12 at 17:54, George Dunlap <george.dunlap@citrix.com> wrote:
> >> > On Tue, 2012-01-24 at 11:08 +0000, Jan Beulich wrote:
> >> >> >>> On 24.01.12 at 11:29, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> >> >> > On Mon, Jan 23, 2012 at 11:08 AM, Jan Beulich <JBeulich@suse.com> wrote:
> >> >> >> x86's vMCE implementation lets a guest know of as many MCE reporting
> >> >> >> banks as there are in the host. While a PV guest could be expected to
> >> >> >> deal with this number changing (particularly decreasing) during migration
> >> >> >> (not currently handled anywhere afaict), for HVM guests this is certainly
> >> >> >> wrong.
> >> >> >>
> >> >> >> At least to me it isn't, however, clear how to properly handle this. The
> >> >> >> easiest would appear to be to save and restore the number of banks
> >> >> >> the guest was made believe it can access, making vmce_{rd,wr}msr()
> >> >> >> silently tolerate accesses between the host and guest values.
> >> >> >
> >> >> > We ran into this in the XS 6.0 release as well.  I think that the
> >> >> > ideal thing to do would be to have a parameter that can be set at
> >> >> > boot, to say how many vMCE banks a guest has, defaulting to the number
> >> >> > of MCE banks on the host.  This parameter would be preserved across
> >> >> > migration.  Ideally, a pool-aware toolstack like xapi would then set
> >> >> > this value to be the value of the host in the pool with the largest
> >> >> > number of banks, allowing a guest to access all the banks on any host
> >> >> > to which it migrates.
> >> >> >
> >> >> > What do you think?
> >> >>
> >> >> That sounds like the way to go.
> >> >
> >> > So should we put this on IanC's to-do-be-done list?  Are you going to
> >> > put it on your to-do list? :-)
> >> 
> >> Below/attached a draft patch (compile tested only), handling save/
> >> restore of the bank count, but not allowing for a config setting to
> >> specify its initial value (yet).
> > 
> > Looks pretty good for a first blush.  Just one question: Why is the vmce
> > count made on a per-vcpu basis, rather than on a per-domain basis, like
> > the actual banks are?  Is the host MCE stuff per-vcpu?
> 
> The question should probably be the other way around - why is the
> vMCE implementation using global (fake) MSRs rather than per-vCPU
> ones (as they would be on hardware). If the change here was
> implemented as per-domain MSRs, a future move of the vMCE
> implementation to a more natural model would be impossible. Also,
> for the PV case the save/restore logic is much simpler this way.

Ah, that makes sense.  Good spot on the impossiblity of changing to
per-vcpu later.  

 -George