Xen 4.2 TODO / Release Plan

Xen 4.2 TODO / Release Plan

[Ian Campbell]

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
30 July         -- First release candidate
Weekly          -- RCN+1 until release          << WE ARE HERE


A handful of issues identified by the test day last week are included,
thanks to all who took part.

The updated TODO list follows.

hypervisor, blockers:

    * None

tools, blockers:

    * libxl stable API -- we would like 4.2 to define a stable API
      which downstream's can start to rely on not changing. Aspects of
      this are:

        * None known

    * xl compatibility with xm:

        * No known issues

    * [CHECK] More formally deprecate xm/xend. Manpage patches already
      in tree. Needs release noting and communication around -rc1 to
      remind people to test xl.

    * [CHECK] Confirm that migration from Xen 4.1 -> 4.2 works.

    * Bump library SONAMES as necessary.
      <20502.39440.969619.824976@mariner.uk.xensource.com>

    * [BUG] qemu-traditional has 50% cpu utilization on an idle
      Windows system if USB is enabled. Not 100% clear whether this is
      Xen or qemu.  George Dunlap is performing initial
      investigations.

hypervisor, nice to have:

    * [BUG(?)] Under certain conditions, the p2m_pod_sweep code will
      stop halfway through searching, causing a guest to crash even if
      there was zeroed memory available.  This is NOT a regression
      from 4.1, and is a very rare case, so probably shouldn't be a
      blocker.  (In fact, I'd be open to the idea that it should wait
      until after the release to get more testing.)
      	    (George Dunlap)

    * S3 regression(s?) reported by Ben Guthro (Ben & Jan Beulich)

    * fix high change rate to CMOS RTC periodic interrupt causing
      guest wall clock time to lag (possible fix outlined, needs to be
      put in patch form and thoroughly reviewed/tested for unwanted
      side effects, Jan Beulich)

tools, nice to have:

    * xl compatibility with xm:

        * the parameter io and irq in domU config files are not
          evaluated by xl.  So it is not possible to passthrough a
          parallel port for my printer to domU when I start the domU
          with xl command. (reported by Dieter Bloms,
          <20120814100704.GA19704@bloms.de>)

    * xl.cfg(5) documentation patch for qemu-upstream
      videoram/videomem support:
      http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
      qemu-upstream doesn't support specifying videomem size for the
      HVM guest cirrus/stdvga.  (but this works with
      qemu-xen-traditional). (Pasi Kärkkäinen)

    * [BUG] long stop during the guest boot process with qcow image,
      reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821

    * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
      http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822

    * Load blktap driver from xencommons initscript if available, thread at:
      <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
      properly in 4.3. (Patch posted, discussion, plan to take simple
      xencommons patch for 4.2 and revist for 4.3. Ping sent)

    * [BUG] xl allows same PCI device to be assigned to multiple
      guests. http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1826
      (<E4558C0C96688748837EB1B05BEED75A0FD5574A@SHSMSX102.ccr.corp.intel.com>)

    * address PoD problems with early host side accesses to guest
      address space (Jan Beulich, DONE)

    * fix ipxe build problems with gcc 4.7 (fedora 17).
      The following files fail to build:
        - ipxe/src/drivers/bus/isa.c
        - ipxe/src/drivers/net/myri10ge.c
        - ipxe/src/drivers/infiniband/qib7322.c
      Patches have been posted to ipxe-devel mailinglist, so we need
      to update our ipxe version or grab the patches. (DONE, Keir)

    * "xl list -l" does not produce proper json. Should be possible to
      make it into an array. Reported by Bastian Blank,
      <20120814121741.GA10214@wavehammer.waldi.eu.org>. (Ian Campbell,
      patch posted)

    * "xl cpupool-create" segfault on incorrect input. Reported by
      George Dunlap,
      <CAFLBxZaEci0mOcDCgFX9zk=wh3z4Nf1LD5E5Fcy7Y3=ioDAM=g@mail.gmail.com>
      (Ian Campbell, patch posted)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 TODO / Release Plan

[Jan Beulich]

>>> On 20.08.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>     * fix high change rate to CMOS RTC periodic interrupt causing
>       guest wall clock time to lag (possible fix outlined, needs to be
>       put in patch form and thoroughly reviewed/tested for unwanted
>       side effects, Jan Beulich)

Patch was posted, but no comments or approval to commit so far.
Also, reportedly the patch only improves the situation, it doesn't
completely eliminate the problem. For the moment I'm out of ideas,
though, and hence would hope some others could help here.

Jan

Re: RTC patch

[Jan Beulich]

>>> On 22.08.12 at 12:48, Christoph Egger <Christoph.Egger@amd.com> wrote:
> On 08/21/12 15:57, Jan Beulich wrote:
> 
>>>>> On 21.08.12 at 15:28, Christoph Egger <Christoph.Egger@amd.com> wrote:
>>> On 08/20/12 13:06, Jan Beulich wrote:
>>>
>>>>>>> On 20.08.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>>>>     * fix high change rate to CMOS RTC periodic interrupt causing
>>>>>       guest wall clock time to lag (possible fix outlined, needs to be
>>>>>       put in patch form and thoroughly reviewed/tested for unwanted
>>>>>       side effects, Jan Beulich)
>>>>
>>>> Patch was posted, but no comments or approval to commit so far.
>>>> Also, reportedly the patch only improves the situation, it doesn't
>>>> completely eliminate the problem. For the moment I'm out of ideas,
>>>> though, and hence would hope some others could help here.
>>>
>>>
>>> Can you point me to the patch (or resend it to me), please?
>>> I have some trouble with getting XP Mode in Windows 7 (nested
>>> virtualization) booting and figured out it uses the RTC.
>>> I want to give this patch a try.
>> 
>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg01303.html 
> 
> When booting Windows 7 I get a crash due to a NULL pointer dereference
> in xen/common/spinlock.c:45.
> It looks like the spin lock is not initialized.

I rather think NULL gets passed from pt_update_irq() to
rtc_periodic_interrupt(). Yet rtc.c's sole call to
create_periodic_time() clearly passes non-NULL. Oh,
hpet_set_timer() can pass a literal 8 (which I didn't spot
grepping for RTC_IRQ) - could you refine the check in
pt_update_irq()

    else if ( irq == RTC_IRQ )

to read

    else if ( irq == RTC_IRQ && pt_priv )

Jan

Re: RTC patch

[Christoph Egger]

On 08/22/12 16:47, Jan Beulich wrote:

>>>> On 22.08.12 at 12:48, Christoph Egger <Christoph.Egger@amd.com> wrote:
>> On 08/21/12 15:57, Jan Beulich wrote:
>>
>>>>>> On 21.08.12 at 15:28, Christoph Egger <Christoph.Egger@amd.com> wrote:
>>>> On 08/20/12 13:06, Jan Beulich wrote:
>>>>
>>>>>>>> On 20.08.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>>>>>     * fix high change rate to CMOS RTC periodic interrupt causing
>>>>>>       guest wall clock time to lag (possible fix outlined, needs to be
>>>>>>       put in patch form and thoroughly reviewed/tested for unwanted
>>>>>>       side effects, Jan Beulich)
>>>>>
>>>>> Patch was posted, but no comments or approval to commit so far.
>>>>> Also, reportedly the patch only improves the situation, it doesn't
>>>>> completely eliminate the problem. For the moment I'm out of ideas,
>>>>> though, and hence would hope some others could help here.
>>>>
>>>>
>>>> Can you point me to the patch (or resend it to me), please?
>>>> I have some trouble with getting XP Mode in Windows 7 (nested
>>>> virtualization) booting and figured out it uses the RTC.
>>>> I want to give this patch a try.
>>>
>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg01303.html 
>>
>> When booting Windows 7 I get a crash due to a NULL pointer dereference
>> in xen/common/spinlock.c:45.
>> It looks like the spin lock is not initialized.
> 
> I rather think NULL gets passed from pt_update_irq() to
> rtc_periodic_interrupt(). Yet rtc.c's sole call to
> create_periodic_time() clearly passes non-NULL. Oh,
> hpet_set_timer() can pass a literal 8 (which I didn't spot
> grepping for RTC_IRQ) - could you refine the check in
> pt_update_irq()
> 
>     else if ( irq == RTC_IRQ )
> 
> to read
> 
>     else if ( irq == RTC_IRQ && pt_priv )

Yes, Windows 7 boots with this change.

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: RTC patch

[Jan Beulich]

>>> On 22.08.12 at 17:01, Christoph Egger <Christoph.Egger@amd.com> wrote:
> On 08/22/12 16:47, Jan Beulich wrote:
> 
>>>>> On 22.08.12 at 12:48, Christoph Egger <Christoph.Egger@amd.com> wrote:
>>> On 08/21/12 15:57, Jan Beulich wrote:
>>>
>>>>>>> On 21.08.12 at 15:28, Christoph Egger <Christoph.Egger@amd.com> wrote:
>>>>> On 08/20/12 13:06, Jan Beulich wrote:
>>>>>
>>>>>>>>> On 20.08.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>>>>>>     * fix high change rate to CMOS RTC periodic interrupt causing
>>>>>>>       guest wall clock time to lag (possible fix outlined, needs to be
>>>>>>>       put in patch form and thoroughly reviewed/tested for unwanted
>>>>>>>       side effects, Jan Beulich)
>>>>>>
>>>>>> Patch was posted, but no comments or approval to commit so far.
>>>>>> Also, reportedly the patch only improves the situation, it doesn't
>>>>>> completely eliminate the problem. For the moment I'm out of ideas,
>>>>>> though, and hence would hope some others could help here.
>>>>>
>>>>>
>>>>> Can you point me to the patch (or resend it to me), please?
>>>>> I have some trouble with getting XP Mode in Windows 7 (nested
>>>>> virtualization) booting and figured out it uses the RTC.
>>>>> I want to give this patch a try.
>>>>
>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg01303.html 
>>>
>>> When booting Windows 7 I get a crash due to a NULL pointer dereference
>>> in xen/common/spinlock.c:45.
>>> It looks like the spin lock is not initialized.
>> 
>> I rather think NULL gets passed from pt_update_irq() to
>> rtc_periodic_interrupt(). Yet rtc.c's sole call to
>> create_periodic_time() clearly passes non-NULL. Oh,
>> hpet_set_timer() can pass a literal 8 (which I didn't spot
>> grepping for RTC_IRQ) - could you refine the check in
>> pt_update_irq()
>> 
>>     else if ( irq == RTC_IRQ )
>> 
>> to read
>> 
>>     else if ( irq == RTC_IRQ && pt_priv )
> 
> Yes, Windows 7 boots with this change.

Good, thanks. But I suppose it has no effect on the problem you
wanted to try this for?

Jan

Re: Xen 4.2 TODO / Release Plan

[Ren, Yongjie]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org
> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Ian Campbell
> Sent: Monday, August 20, 2012 5:17 PM
> To: xen-devel; xen-users
> Subject: [Xen-devel] Xen 4.2 TODO / Release Plan
> 
>     * [BUG] long stop during the guest boot process with qcow image,
>       reported by Intel:
> http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
> 
After reverting the "O_DIRECT for IDE" patch in qemu-xen tree, it works as fine as that before the bug.
But it still have some stop time (about 8~10s) after loading the Grub for a RHEL6.x guest.
I found even an old CS #23124 (about 1 year ago) also has the same phenomenon.
Currently, RHEL6.2 or RHEL6.3 guest can bootup in 30~40s (for either RAW or QCOW2 image).
And, RHEL5.5 guest (which has no stop time issue) can bootup in 50~60s.

I also commented in the bugzilla. You can also have a look for more details.

Will you want still track or fix this old issue ?  If not, I want to marked it as "fixed and verified".

Re: Xen 4.2 TODO / Release Plan

[Ian Campbell]

Users list to BCC.

On Tue, 2012-08-21 at 16:14 +0100, Ren, Yongjie wrote:
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xen.org
> > [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Ian Campbell
> > Sent: Monday, August 20, 2012 5:17 PM
> > To: xen-devel; xen-users
> > Subject: [Xen-devel] Xen 4.2 TODO / Release Plan
> > 
> >     * [BUG] long stop during the guest boot process with qcow image,
> >       reported by Intel:
> > http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
> > 
> After reverting the "O_DIRECT for IDE" patch in qemu-xen tree, it works as fine as that before the bug.
> But it still have some stop time (about 8~10s) after loading the Grub for a RHEL6.x guest.
> I found even an old CS #23124 (about 1 year ago) also has the same phenomenon.

23124 here is e3d4c34b14a3112481b5e86ff2406cd1bb5e1548 which is some
sort of tools build fix. What is the long hash of the changeset you are
referring to?

> Currently, RHEL6.2 or RHEL6.3 guest can bootup in 30~40s (for either RAW or QCOW2 image).
> And, RHEL5.5 guest (which has no stop time issue) can bootup in 50~60s.
> 
> I also commented in the bugzilla. You can also have a look for more details.
> 
> Will you want still track or fix this old issue ?  If not, I want to marked it as "fixed and verified".

If you think the issue is fixed then I will mark it done on the todo
list.

Ian.

Re: Xen 4.2 TODO / Release Plan

[Ren, Yongjie]

> -----Original Message-----
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> Sent: Tuesday, August 21, 2012 11:27 PM
> To: Ren, Yongjie
> Cc: xen-devel
> Subject: Re: [Xen-devel] Xen 4.2 TODO / Release Plan
> 
> Users list to BCC.
> 
> On Tue, 2012-08-21 at 16:14 +0100, Ren, Yongjie wrote:
> > > -----Original Message-----
> > > From: xen-devel-bounces@lists.xen.org
> > > [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Ian Campbell
> > > Sent: Monday, August 20, 2012 5:17 PM
> > > To: xen-devel; xen-users
> > > Subject: [Xen-devel] Xen 4.2 TODO / Release Plan
> > >
> > >     * [BUG] long stop during the guest boot process with qcow
> image,
> > >       reported by Intel:
> > > http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
> > >
> > After reverting the "O_DIRECT for IDE" patch in qemu-xen tree, it works
> as fine as that before the bug.
> > But it still have some stop time (about 8~10s) after loading the Grub for a
> RHEL6.x guest.
> > I found even an old CS #23124 (about 1 year ago) also has the same
> phenomenon.
> 
> 23124 here is e3d4c34b14a3112481b5e86ff2406cd1bb5e1548 which is
> some
> sort of tools build fix. What is the long hash of the changeset you are
> referring to?
> 
Yes, it's "changeset:  23124:e3d4c34b14a3".
That changeset is picked out randomly just to confirm the stop time (8~10s) should already exist for a long time.

> > Currently, RHEL6.2 or RHEL6.3 guest can bootup in 30~40s (for either
> RAW or QCOW2 image).
> > And, RHEL5.5 guest (which has no stop time issue) can bootup in 50~60s.
> >
> > I also commented in the bugzilla. You can also have a look for more
> details.
> >
> > Will you want still track or fix this old issue ?  If not, I want to marked it
> as "fixed and verified".
> 
> If you think the issue is fixed then I will mark it done on the todo
> list.
> 
I think yes. The regression for long stop time (about 50s) in bootup for qcow2 image has already been fixed.
Now, 30s for a guest booting up is acceptable from my viewpoint.
If the 8~10 stop time really matters, we can setup another thread to discuss it.

Re: Xen 4.2 TODO / Release Plan

[Ben Guthro]

On Mon, Aug 20, 2012 at 5:17 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>
>     * S3 regression(s?) reported by Ben Guthro (Ben & Jan Beulich)
>

No significant progress made on this since the last update.
More questions raised than answers found. I'm having trouble making
sense of the current test results.

More debugging is needed. Jan is working on a debug patch to give to me.

Re: Xen 4.2 TODO / Release Plan

[Jan Beulich]

>>> On 21.08.12 at 17:39, Ben Guthro <ben@guthro.net> wrote:
> On Mon, Aug 20, 2012 at 5:17 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>
>>     * S3 regression(s?) reported by Ben Guthro (Ben & Jan Beulich)
>>
> 
> No significant progress made on this since the last update.
> More questions raised than answers found. I'm having trouble making
> sense of the current test results.
> 
> More debugging is needed. Jan is working on a debug patch to give to me.

I haven't been able to get to this yesterday and today, and will
be working only half day tomorrow. After that I'll be traveling,
so it may well end up being only after the return from the summit
that I may be able to get you something.

Jan