Xen 4.3 development update

Xen 4.3 development update

[George Dunlap]

There are basically two issues we're waiting to sort out (see below
for more info):
 * cpu hotplug in qemu-upsream
 * The MMIO hole issue

Anthony has patches for cpu hot-plug, and we have plans for how to
work around the MMIO hole issue, so we're getting really close.

Also note that we have slipped the release date a week, to the 26th,
to make sure we have enough time to catch any potential bugs in the
recent check-ins.

This information will be mirrored on the Xen 4.3 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.3

The key goals we're focusing on now, in order, are as follows:
 1. Have a bug-free 4.3 release
 2. Have an awesome 4.3 release
 3. Have a 4.3 release that happens on schedule (ready by June 15th)

The most important thing in making a case is to answer the question,
"If there are bugs in this patch, will they be discovered before the
June 17th release?"  The second most important thing is to consider the
cost/benefit analysis of bugs that are found: what is the risk of
introducing a bug which will delay the release, vs the benefit it will
have in making the release better?

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:
* Feature freeze: 25 March 2013
* Code freezing point: 15 April 2013
* First RC: 6 May 2013 <== WE ARE HERE
* Release: 26 June 2013

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  Each new feature will be
considered on a case-by-case basis.

The June 17th release is both an estimate and a goal.  At this point,
Xen 4.3 can be released whenever it's actually ready.  In fact, the
sooner we release, the sooner we can open the tree up for new
development and get on to 4.4 -- so keep fixing those bugs!

Last updated: 17 June 2013

== Completed ==

* Default to QEMU upstream (partial)
 - pci pass-thru (external)
 - enable dirtybit tracking during migration (external)
 - xl cd-{insert,eject} (external)

* openvswitch toostack integration
  To label "tech-preview" unless we get good testing (>10 individuals)

* NUMA scheduler affinity

* Install into /usr/local by default

* Allow config file to specify multiple usb devices for HVM domains

* Persistent grants for blk (external)
 - Linux
 - qemu

* Allow XSM to override IS_PRIV checks in the hypervisor

* vTPM updates

* Scalability: 16TiB of RAM

* CPUID-based idle (don't rely on ACPI info f/ dom0)

* Serial console improvements
  -EHCI debug port

== Bugs resolved since last update ==

* tmem map_domain_page issue
  resolution: fixed

* xl pci-detach broken for pv guests?
  resolution: Guest kernel bug

* XSA-55
  resolution: patches committed

* AMD IOMMU MSI interrupts missing?
  resolution: fixed

== Open bugs ==

* cpu hotplug broken in qemu-xen
  > Upstream qemu has qmp interface; either implement our own way or backport
  > Issue in SeaBIOS: can wake up "unplugged" cpus!
  owner: Anthony
  status: In progress
  priority: blocker

* qemu-upstream MMIO hole issue
 > http://lists.xen.org/archives/html/xen-devel/2013-03/msg00559.html
 > "You can reproduce it when a VM has a big pci hole size (such as
   512MB), e.g. create a VM with a virtual device which has a 512MB
   pci BAR."
 priority: high
 status: Original fix broken; investigating proper fix
 priority: blocker (?)

* Win2k8 fails install on many IvyBridge EP systems
 > Not reproducible by Intel
 > Reproducible on 4.2
 priority: Probably not a blocker

* Update 4.3-rc to 4.3 in README; add tag bragging about 4.3
  owner: George
  status: v1 sent, v2 pending

* perl 5.18 fails to compile qemu-traditional docs?
 > http://www.gossamer-threads.com/lists/xen/devel/284141
 status: discussion in progress
 priority: minor

* Scan through qemu-upstream changesets looking for important fixes
  (particularly for stubdoms) for qemu-traditional
  - cpu hotplug
  owner: Anthony

* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 priority: high, probably not a blocker
 status: patches posted

* AMD: IOMMU accidental clearing
  owner: Jan Beulich, Suravee Suthikulpanit
  status:

* __update_vcpu_system_time if system_time comes out negative
  status: Not for 4.3

Re: Xen 4.3 development update

[George Dunlap]

On Mon, Jun 17, 2013 at 11:58 AM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:

> * Win2k8 fails install on many IvyBridge EP systems
>  > Not reproducible by Intel
>  > Reproducible on 4.2
>  priority: Probably not a blocker

I've marked this "not a blocker" since it's not a regression and seems
to only affect a small number of systems in a particular
configuration.  Feel free to come back on that one if you disagree.

 -George

Re: Xen 4.3 development update

[Jan Beulich]

>>> On 17.06.13 at 12:58, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> * Win2k8 fails install on many IvyBridge EP systems
>  > Not reproducible by Intel
>  > Reproducible on 4.2

Reproducible only on 4.2 with APIC virtualization commits backported.
So this is a regression due to new functionality (which to date cannot
be worked around).

>  priority: Probably not a blocker

But nevertheless I agree that this is unlikely to be a blocker.

> * AMD: IOMMU accidental clearing
>   owner: Jan Beulich, Suravee Suthikulpanit
>   status:

I admit that from the title I don't recall what this is referring to,
and hence what the state of it is.

Jan

Re: Xen 4.3 development update

[George Dunlap]

On Mon, Jun 17, 2013 at 11:58 AM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> * perl 5.18 fails to compile qemu-traditional docs?
>  > http://www.gossamer-threads.com/lists/xen/devel/284141
>  status: discussion in progress
>  priority: minor

Is there an update to this?  Should I just mark it as "Not for 4.3"?

 -George

Re: Xen 4.3 development update

[Gordan Bobic]

 On Mon, 17 Jun 2013 11:58:23 +0100, George Dunlap 
 <George.Dunlap@eu.citrix.com> wrote:
> There are basically two issues we're waiting to sort out (see below
> for more info):
>  * cpu hotplug in qemu-upsream
>  * The MMIO hole issue

 Does that mean that the PCI hole mis-map issue is going to be fixed
 in 4.3? Or is this a different issue?

 I've been thinking about a domU workaround for this. I haven't yet
 had a chance to try it due to too many other things happening, but
 I was pondering something along the following lines.

 For Linux domU, something like memmap=2G$2G on the kernel command
 line. For Windows domU, use something like:

 bcdedit /set badmemorylist $page1 $page2 ...

 to mark the whole 2GB-4GB area as not usable.

 It's a gross hack, but it might just get things working for now.

 Gordan

Re: Xen 4.3 development update

[George Dunlap]

On 17/06/13 12:17, Gordan Bobic wrote:
> On Mon, 17 Jun 2013 11:58:23 +0100, George Dunlap 
> <George.Dunlap@eu.citrix.com> wrote:
>> There are basically two issues we're waiting to sort out (see below
>> for more info):
>>  * cpu hotplug in qemu-upsream
>>  * The MMIO hole issue
>
> Does that mean that the PCI hole mis-map issue is going to be fixed
> in 4.3? Or is this a different issue?

Well "fixed" in the sense that guests shouldn't mysteriously crash 
anymore.  See my proposed 4.3 fix here:

http://marc.info/?l=xen-devel&m=137121936531168&w=2

  -George

Re: Xen 4.3 development update

[Ian Campbell]

On Mon, 2013-06-17 at 12:16 +0100, George Dunlap wrote:
> On Mon, Jun 17, 2013 at 11:58 AM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
> > * perl 5.18 fails to compile qemu-traditional docs?
> >  > http://www.gossamer-threads.com/lists/xen/devel/284141
> >  status: discussion in progress
> >  priority: minor
> 
> Is there an update to this?  Should I just mark it as "Not for 4.3"?

Nothing since "<1369905414.13087.50.camel@zakaz.uk.xensource.com>":
8<------
Seems like we either need to backport:
        commit 3179d694a8dcaa091131e3db644d445c0130713e
        Author: Michael Tokarev <mjt@tls.msk.ru>
        Date:   Tue Mar 20 02:25:57 2012 +0400
        
            Support utf8 chars in pod docs
            
Or just disable the build of docs for qemu-xen-trad, it seems to me that
they are rather useless...
8<----------

The former seems like something we could still do at this stage (risk ==
obvious build failure, I think). I'm not sure how the latter can be
achieved so I've no idea about feasibility. Ian, what do you think?

Perl 5.18 was released this May, so it is pretty bleeding edge.  On the
other hand it appears that at least Arch has picked it up and people who
run bleeding edge stuff make good testers...

Ian.

Re: Xen 4.3 development update

[Fabio Fantoni]

Il 17/06/2013 12:58, George Dunlap ha scritto:
> There are basically two issues we're waiting to sort out (see below
> for more info):
>   * cpu hotplug in qemu-upsream
>   * The MMIO hole issue
>
> Anthony has patches for cpu hot-plug, and we have plans for how to
> work around the MMIO hole issue, so we're getting really close.
>
> Also note that we have slipped the release date a week, to the 26th,
> to make sure we have enough time to catch any potential bugs in the
> recent check-ins.
>
> This information will be mirrored on the Xen 4.3 Roadmap wiki page:
>   http://wiki.xen.org/wiki/Xen_Roadmap/4.3
>
> The key goals we're focusing on now, in order, are as follows:
>   1. Have a bug-free 4.3 release
>   2. Have an awesome 4.3 release
>   3. Have a 4.3 release that happens on schedule (ready by June 15th)
>
> The most important thing in making a case is to answer the question,
> "If there are bugs in this patch, will they be discovered before the
> June 17th release?"  The second most important thing is to consider the
> cost/benefit analysis of bugs that are found: what is the risk of
> introducing a bug which will delay the release, vs the benefit it will
> have in making the release better?
>
> = Timeline =
>
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature freeze: 25 March 2013
> * Code freezing point: 15 April 2013
> * First RC: 6 May 2013 <== WE ARE HERE
> * Release: 26 June 2013
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  Each new feature will be
> considered on a case-by-case basis.
>
> The June 17th release is both an estimate and a goal.  At this point,
> Xen 4.3 can be released whenever it's actually ready.  In fact, the
> sooner we release, the sooner we can open the tree up for new
> development and get on to 4.4 -- so keep fixing those bugs!
>
> Last updated: 17 June 2013
>
> == Completed ==
>
> * Default to QEMU upstream (partial)
>   - pci pass-thru (external)
>   - enable dirtybit tracking during migration (external)

>   - xl cd-{insert,eject}

Tested and working. No more bugs found.

>
> * openvswitch toostack integration
>    To label "tech-preview" unless we get good testing (>10 individuals)

Tested and working. It also works with hvm domUs that didn't work on 
initial version of vif script. No bugs found for now.

>
> * NUMA scheduler affinity
>
> * Install into /usr/local by default
>
> * Allow config file to specify multiple usb devices for HVM domains
>
> * Persistent grants for blk (external)
>   - Linux
>   - qemu
>
> * Allow XSM to override IS_PRIV checks in the hypervisor
>
> * vTPM updates
>
> * Scalability: 16TiB of RAM
>
> * CPUID-based idle (don't rely on ACPI info f/ dom0)
>
> * Serial console improvements
>    -EHCI debug port
>
> == Bugs resolved since last update ==
>
> * tmem map_domain_page issue
>    resolution: fixed
>
> * xl pci-detach broken for pv guests?
>    resolution: Guest kernel bug
>
> * XSA-55
>    resolution: patches committed
>
> * AMD IOMMU MSI interrupts missing?
>    resolution: fixed
>
> == Open bugs ==
>
> * cpu hotplug broken in qemu-xen
>    > Upstream qemu has qmp interface; either implement our own way or backport
>    > Issue in SeaBIOS: can wake up "unplugged" cpus!
>    owner: Anthony
>    status: In progress
>    priority: blocker
>
> * qemu-upstream MMIO hole issue
>   > http://lists.xen.org/archives/html/xen-devel/2013-03/msg00559.html
>   > "You can reproduce it when a VM has a big pci hole size (such as
>     512MB), e.g. create a VM with a virtual device which has a 512MB
>     pci BAR."
>   priority: high
>   status: Original fix broken; investigating proper fix
>   priority: blocker (?)
>
> * Win2k8 fails install on many IvyBridge EP systems
>   > Not reproducible by Intel
>   > Reproducible on 4.2
>   priority: Probably not a blocker
>
> * Update 4.3-rc to 4.3 in README; add tag bragging about 4.3
>    owner: George
>    status: v1 sent, v2 pending
>
> * perl 5.18 fails to compile qemu-traditional docs?
>   > http://www.gossamer-threads.com/lists/xen/devel/284141
>   status: discussion in progress
>   priority: minor
>
> * Scan through qemu-upstream changesets looking for important fixes
>    (particularly for stubdoms) for qemu-traditional
>    - cpu hotplug
>    owner: Anthony
>
> * qemu-upstream not freeing pirq
>   > http://www.gossamer-threads.com/lists/xen/devel/281498
>   priority: high, probably not a blocker
>   status: patches posted
>
> * AMD: IOMMU accidental clearing
>    owner: Jan Beulich, Suravee Suthikulpanit
>    status:
>
> * __update_vcpu_system_time if system_time comes out negative
>    status: Not for 4.3
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.3 development update

[George Dunlap]

On 17/06/13 13:35, Fabio Fantoni wrote:
> Il 17/06/2013 12:58, George Dunlap ha scritto:
>> There are basically two issues we're waiting to sort out (see below
>> for more info):
>>   * cpu hotplug in qemu-upsream
>>   * The MMIO hole issue
>>
>> Anthony has patches for cpu hot-plug, and we have plans for how to
>> work around the MMIO hole issue, so we're getting really close.
>>
>> Also note that we have slipped the release date a week, to the 26th,
>> to make sure we have enough time to catch any potential bugs in the
>> recent check-ins.
>>
>> This information will be mirrored on the Xen 4.3 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.3
>>
>> The key goals we're focusing on now, in order, are as follows:
>>   1. Have a bug-free 4.3 release
>>   2. Have an awesome 4.3 release
>>   3. Have a 4.3 release that happens on schedule (ready by June 15th)
>>
>> The most important thing in making a case is to answer the question,
>> "If there are bugs in this patch, will they be discovered before the
>> June 17th release?"  The second most important thing is to consider the
>> cost/benefit analysis of bugs that are found: what is the risk of
>> introducing a bug which will delay the release, vs the benefit it will
>> have in making the release better?
>>
>> = Timeline =
>>
>> We are planning on a 9-month release cycle.  Based on that, below are
>> our estimated dates:
>> * Feature freeze: 25 March 2013
>> * Code freezing point: 15 April 2013
>> * First RC: 6 May 2013 <== WE ARE HERE
>> * Release: 26 June 2013
>>
>> The RCs and release will of course depend on stability and bugs, and
>> will therefore be fairly unpredictable.  Each new feature will be
>> considered on a case-by-case basis.
>>
>> The June 17th release is both an estimate and a goal.  At this point,
>> Xen 4.3 can be released whenever it's actually ready.  In fact, the
>> sooner we release, the sooner we can open the tree up for new
>> development and get on to 4.4 -- so keep fixing those bugs!
>>
>> Last updated: 17 June 2013
>>
>> == Completed ==
>>
>> * Default to QEMU upstream (partial)
>>   - pci pass-thru (external)
>>   - enable dirtybit tracking during migration (external)
>
>>   - xl cd-{insert,eject}
>
> Tested and working. No more bugs found.
>
>>
>> * openvswitch toostack integration
>>    To label "tech-preview" unless we get good testing (>10 individuals)
>
> Tested and working. It also works with hvm domUs that didn't work on 
> initial version of vif script. No bugs found for now.

Great, thanks for testing.

  -George

Re: Xen 4.3 development update

[Konrad Rzeszutek Wilk]

On Mon, Jun 17, 2013 at 11:58:23AM +0100, George Dunlap wrote:
> There are basically two issues we're waiting to sort out (see below
> for more info):
>  * cpu hotplug in qemu-upsream
>  * The MMIO hole issue

There is also the issue of migration not working with 'xl' with
the PVHVM Linux guests. I am able to reproduce this (this is
the issue Vasiliy Tolstov experienced) when doing a simple
migration:

 xl migrate latest localhost

which hangs at 50% and stops. I hadn't been able yet to capture the
output from the guest so I don't know if I am hitting the same
_exact_ error Vasiliy is hitting.

If I do 'xm migrate latest localhost' it works nicely.

Note that I am using qemu-traditional for this (xl).

Re: Xen 4.3 development update

[George Dunlap]

On 17/06/13 14:24, Konrad Rzeszutek Wilk wrote:
> On Mon, Jun 17, 2013 at 11:58:23AM +0100, George Dunlap wrote:
>> There are basically two issues we're waiting to sort out (see below
>> for more info):
>>   * cpu hotplug in qemu-upsream
>>   * The MMIO hole issue
> There is also the issue of migration not working with 'xl' with
> the PVHVM Linux guests. I am able to reproduce this (this is
> the issue Vasiliy Tolstov experienced) when doing a simple
> migration:
>
>   xl migrate latest localhost
>
> which hangs at 50% and stops. I hadn't been able yet to capture the
> output from the guest so I don't know if I am hitting the same
> _exact_ error Vasiliy is hitting.
>
> If I do 'xm migrate latest localhost' it works nicely.
>
> Note that I am using qemu-traditional for this (xl).

You mentioned this on IRC, but you haven't actually given anything near 
a real bug report. :-)

xl localhost migrate is tested in osstest, and we just had that bug with 
the timing for PVHVM guests that we fixed a few weeks ago -- that 
involved doing 100+ localhost migrates.

Without some more specification, and/or someone else reporting the 
issue, I can only assume it's something  local to your configuration.

  -George

[PATCH] qemu-xen-traditional: disable docs

[Ian Jackson]

(Jacek: can you confirm that this fixes this other bug for you?
 George: can I have an ack wrt the release?)

Perl 5.18 is unhappy with the qemu-xen-traditional documents, giving
this error:
  qemu.pod around line 91: Non-ASCII character seen before =encoding in
  'Schuetz.'. Assuming UTF-8 POD document had syntax errors at /usr/bin/core_perl/pod2man line 71.
  make[3]: *** [qemu.1] Error 255

We do not want these docs.  They are not really relevant to Xen users.
So instead of backporting the utf-8 fix from qemu upstream
(3179d694a8dcaa091131e3db644d445c0130713e), we just disable the docs
build.

We do this in xen-hooks.mak because qemu-xen-traditional's configure
script lacks the ability to explicitly disable the docs build.

(The docs build for upstream-based qemu-xen was already disabled by
xen.git#a0d110801d701c43e7b8c73dbd6b2444a10a7cdb)

Reported-by: jacek burghardt <jaceksburghardt@gmail.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

diff --git a/xen-hooks.mak b/xen-hooks.mak
index b55f45b..58d61c9 100644
--- a/xen-hooks.mak
+++ b/xen-hooks.mak
@@ -83,4 +83,6 @@ datadir := $(subst qemu,xen/qemu,$(datadir))
 docdir :=  $(subst qemu,xen/qemu,$(docdir))
 mandir :=  $(subst share/man,share/xen/man,$(mandir))
 
+BUILD_DOCS=
+
 configdir := $(XEN_SCRIPT_DIR)

Re: Xen 4.3 development update

[Alex Bligh]

George,

--On 17 June 2013 14:29:22 +0100 George Dunlap 
<george.dunlap@eu.citrix.com> wrote:

>> There is also the issue of migration not working with 'xl' with
>> the PVHVM Linux guests. I am able to reproduce this (this is
>> the issue Vasiliy Tolstov experienced) when doing a simple
>> migration:
>>
>>   xl migrate latest localhost
>>
>> which hangs at 50% and stops. I hadn't been able yet to capture the
>> output from the guest so I don't know if I am hitting the same
>> _exact_ error Vasiliy is hitting.
>>
>> If I do 'xm migrate latest localhost' it works nicely.
>>
>> Note that I am using qemu-traditional for this (xl).
>
> You mentioned this on IRC, but you haven't actually given anything near a
> real bug report. :-)
>
> xl localhost migrate is tested in osstest, and we just had that bug with
> the timing for PVHVM guests that we fixed a few weeks ago -- that
> involved doing 100+ localhost migrates.

Was that on qemu-xen/qemu-upstream rather than qemu-traditional?

-- 
Alex Bligh

Re: [PATCH] qemu-xen-traditional: disable docs

[George Dunlap]

On 17/06/13 16:47, Ian Jackson wrote:
> (Jacek: can you confirm that this fixes this other bug for you?
>   George: can I have an ack wrt the release?)
>
> Perl 5.18 is unhappy with the qemu-xen-traditional documents, giving
> this error:
>    qemu.pod around line 91: Non-ASCII character seen before =encoding in
>    'Schuetz.'. Assuming UTF-8 POD document had syntax errors at /usr/bin/core_perl/pod2man line 71.
>    make[3]: *** [qemu.1] Error 255
>
> We do not want these docs.  They are not really relevant to Xen users.
> So instead of backporting the utf-8 fix from qemu upstream
> (3179d694a8dcaa091131e3db644d445c0130713e), we just disable the docs
> build.
>
> We do this in xen-hooks.mak because qemu-xen-traditional's configure
> script lacks the ability to explicitly disable the docs build.
>
> (The docs build for upstream-based qemu-xen was already disabled by
> xen.git#a0d110801d701c43e7b8c73dbd6b2444a10a7cdb)
>
> Reported-by: jacek burghardt <jaceksburghardt@gmail.com>
> Cc: George Dunlap <George.Dunlap@eu.citrix.com>
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Re the release:

Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Re: [PATCH] qemu-xen-traditional: disable docs

[Ian Jackson]

George Dunlap writes ("Re: [PATCH] qemu-xen-traditional: disable docs"):
> On 17/06/13 16:47, Ian Jackson wrote:
> > (Jacek: can you confirm that this fixes this other bug for you?
> >   George: can I have an ack wrt the release?)
> >
> > Perl 5.18 is unhappy with the qemu-xen-traditional documents, giving
...
> Re the release:
> 
> Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Thanks, I have applied this to qemu-xen-traditional and updated
xen.git's Config.mk.

Ian.

Re: Xen 4.3 development update

[Konrad Rzeszutek Wilk]

On Mon, Jun 17, 2013 at 05:16:38PM +0100, Alex Bligh wrote:
> George,
> 
> --On 17 June 2013 14:29:22 +0100 George Dunlap
> <george.dunlap@eu.citrix.com> wrote:
> 
> >>There is also the issue of migration not working with 'xl' with
> >>the PVHVM Linux guests. I am able to reproduce this (this is
> >>the issue Vasiliy Tolstov experienced) when doing a simple
> >>migration:
> >>
> >>  xl migrate latest localhost
> >>
> >>which hangs at 50% and stops. I hadn't been able yet to capture the
> >>output from the guest so I don't know if I am hitting the same
> >>_exact_ error Vasiliy is hitting.
> >>
> >>If I do 'xm migrate latest localhost' it works nicely.
> >>
> >>Note that I am using qemu-traditional for this (xl).
> >
> >You mentioned this on IRC, but you haven't actually given anything near a
> >real bug report. :-)

And to double check whether this is an issue with the machine I can't reproduce
this on another Intel box. Hmmm...

> >
> >xl localhost migrate is tested in osstest, and we just had that bug with
> >the timing for PVHVM guests that we fixed a few weeks ago -- that
> >involved doing 100+ localhost migrates.
> 
> Was that on qemu-xen/qemu-upstream rather than qemu-traditional?
> 
> -- 
> Alex Bligh
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: Xen 4.3 development update

[Jan Beulich]

>>> On 17.06.13 at 13:13, "Jan Beulich" <JBeulich@suse.com> wrote:
>>>> On 17.06.13 at 12:58, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>> * Win2k8 fails install on many IvyBridge EP systems
>>  > Not reproducible by Intel
>>  > Reproducible on 4.2
> 
> Reproducible only on 4.2 with APIC virtualization commits backported.
> So this is a regression due to new functionality (which to date cannot
> be worked around).

Update: This is a bad interaction between Viridian mode and APIC-V.
Disabling either makes things work again. Yang narrowed this down
to the EOI handling, and created a non-postable patch that deals
with the issue (non-postable because it does things in ways only
suitable for temporary testing). I do think, though, that this isn't the
way to go anyway - instead we likely will want to suppress Windows
using the APIC related MSRs when APIC register virtualization is in
use. Yang seems to agree with this, but this approach wasn't tested
yet (partly because for it to be complete we'd also want to
populate Viridian CPUID leaf 6, but the APIC related bit there is
insufficiently specified, so we first need to find out from MS when
exactly this bit is supposed to get set).

In any case, with the workaround of disabling Viridian support now
known, I guess we can indeed defer fixing this until 4.3.1.

Jan

Re: Xen 4.3 development update

[Zhang, Yang Z]

Jan Beulich wrote on 2013-06-20:
>>>> On 17.06.13 at 13:13, "Jan Beulich" <JBeulich@suse.com> wrote:
>>>>> On 17.06.13 at 12:58, George Dunlap <George.Dunlap@eu.citrix.com>
> wrote:
>>> * Win2k8 fails install on many IvyBridge EP systems
>>>> Not reproducible by Intel
>>>> Reproducible on 4.2
>> 
>> Reproducible only on 4.2 with APIC virtualization commits backported.
>> So this is a regression due to new functionality (which to date cannot
>> be worked around).
> 
> Update: This is a bad interaction between Viridian mode and APIC-V.
> Disabling either makes things work again. Yang narrowed this down
> to the EOI handling, and created a non-postable patch that deals
> with the issue (non-postable because it does things in ways only
> suitable for temporary testing). I do think, though, that this isn't the
> way to go anyway - instead we likely will want to suppress Windows
> using the APIC related MSRs when APIC register virtualization is in
> use. Yang seems to agree with this, but this approach wasn't tested
Just do a testing and it is working to not set CPUID3A_MSR_APIC_ACCESS and CPUID4A_MSR_BASED_APIC.

> yet (partly because for it to be complete we'd also want to
> populate Viridian CPUID leaf 6, but the APIC related bit there is
> insufficiently specified, so we first need to find out from MS when
> exactly this bit is supposed to get set).
I think CPUID leaf 6 just indicate which hardware feature is used by hypervisor. It should have nothing to do with this issue. We can consider it separately. 

> 
> In any case, with the workaround of disabling Viridian support now
> known, I guess we can indeed defer fixing this until 4.3.1.
> 
> Jan


Best regards,
Yang

Re: Xen 4.3 development update

[Jan Beulich]

>>> On 21.06.13 at 10:08, "Zhang, Yang Z" <yang.z.zhang@intel.com> wrote:
> Jan Beulich wrote on 2013-06-20:
>>>>> On 17.06.13 at 13:13, "Jan Beulich" <JBeulich@suse.com> wrote:
>>>>>> On 17.06.13 at 12:58, George Dunlap <George.Dunlap@eu.citrix.com>
>> wrote:
>>>> * Win2k8 fails install on many IvyBridge EP systems
>>>>> Not reproducible by Intel
>>>>> Reproducible on 4.2
>>> 
>>> Reproducible only on 4.2 with APIC virtualization commits backported.
>>> So this is a regression due to new functionality (which to date cannot
>>> be worked around).
>> 
>> Update: This is a bad interaction between Viridian mode and APIC-V.
>> Disabling either makes things work again. Yang narrowed this down
>> to the EOI handling, and created a non-postable patch that deals
>> with the issue (non-postable because it does things in ways only
>> suitable for temporary testing). I do think, though, that this isn't the
>> way to go anyway - instead we likely will want to suppress Windows
>> using the APIC related MSRs when APIC register virtualization is in
>> use. Yang seems to agree with this, but this approach wasn't tested
> Just do a testing and it is working to not set CPUID3A_MSR_APIC_ACCESS and 
> CPUID4A_MSR_BASED_APIC.

But after some more thought I think we ought to still use your
original patch (suitably converted to proper shape), and keep the
leaf 3 bit set (i.e. only turn off the respective leaf 4 bit). This
_should_ make Windows not use the MSRs, but would still cope
with it nevertheless doing so for whatever reason.

>> yet (partly because for it to be complete we'd also want to
>> populate Viridian CPUID leaf 6, but the APIC related bit there is
>> insufficiently specified, so we first need to find out from MS when
>> exactly this bit is supposed to get set).
> I think CPUID leaf 6 just indicate which hardware feature is used by 
> hypervisor. It should have nothing to do with this issue. We can consider it 
> separately. 

Sure, this would be a separate patch, but still belonging here as
potentially influencing Windows' decision on how to set up certain
aspects of its operation, including APIC handling.

I'm in the process of putting all these together, in case you haven't
already.

Jan