Xen 4.2 TODO / Release

Xen 4.2 TODO / Release

[Ian Campbell]

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
                                                << WE ARE HERE
Mid/Late May    -- First release candidate
Weekly          -- RCN+1 until it is ready

It seems pretty obvious at this stage that we are not going to be
doing an RC in Mid/Late May. The big remaining items are being
actively worked on and are quite well understood. I've also done a
pass through the tools blockers list and have proposed moving some
items to the Nice To Have list.

The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.

If you are aware of any bugs which must/should be fixed for 4.2 then
please reply to this thread (otherwise I may not remember to pick them
up next week)

Other than that I think we should consider the freeze to be in full
effect and the bar to entry to 4.2 to be very high.

hypervisor, blockers:

    * None
 
tools, blockers:

    * libxl stable API -- we would like 4.2 to define a stable API
      which downstream's can start to rely on not changing. Aspects of
      this are:

        * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
          Interface needs an overhaul, related to
          locking/serialization over domain create (deferred to
          4.3). IanJ to add note about this interface being
          substandard but otherwise defer to 4.3. Will move to Nice To
          Have's next week.

        * libxl_*_path. Majority made internal, only configdir and
          lockdir remain public (used by xl). (IanC, to consider
          moving the two reamining into xl instead of libxl, will move
          to Nice To Have's next week)

        * Interfaces which may need to be async:

            * libxl_domain_suspend. Move xc_domain_save/restore into a
              separate process (IanJ, RFC posted, work ongoing).

            * libxl_device_{disk,nic,vkb,add,pci}_add (and
              remove?). Roger Pau Monné fixes disk as part of hotplug
              script series and adds infrastructure which should make
              the others trivial. (Roger, WIP)

            * libxl_cdrom_insert. Should be easy once
              disk_{add,remove} done. Nobody is looking at this? This
              is basically a helper function and its functionality can
              be implemented in terms of the libxl_disk_*
              interfaces. We could therefore consider marking this
              function as substandard and subject to change in the
              future. We could also consider simply moving it into xl
              as a helper function and revisting for 4.3.

    * xl compatibility with xm:

        * [BUG] cannot create an empty CD-ROM drive on HVM guest,
          reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>

        * does not automatically try to select a (set of) node(s) on
          which to create a VM and pin its vcpus there. (Dario
          Faggioli, patches posted)

        * `cpus=[2,3]` to select specific mapping vcpu0-->pcpu2,
          vcpu1-->pcpu3 as xm does. (Dario Faggioli, DONE)

    * [BUG] Spurious CPU params error message when starting a domain,
      e.g. "Cpu weight out of range, valid values are within range
      from 1 to 65535". (Ian C, patches posted, need final decision on
      "one struct" or "multiple struct" libxl API for sched params)

    * More formally deprecate xm/xend. Manpage patches already in
      tree. Needs release noting and communication around -rc1 to
      remind people to test xl.

    * Domain 0 block attach & general hotplug when using qdisk backend
      (need to start qemu as necessary etc) (Stefano S, patches
      posted, people seem happy bar the code motion)

    * Improved Hotplug script support (Roger Pau Monné, patches
      posted, nearing completion?)

    * Block script support -- follows on from hotplug script (Roger
      Pau Monné)

    * Adjustments needed for qdisk backend to work on non-pvops Linux.
      (Jan Beulich)

hypervisor, nice to have:

    * PoD performance improvements (George Dunlap)

    * IRQ problem for which debugging code got added by c/s
      24707:96987c324a4f. I have a tentative adjustment to this which
      I would hope to at least eliminate the bad consequences of
      crashing the hypervisor (converting the unsolicited
      PIC-originated IRQ into a spurious one). I'll submit as soon as
      I got to test this a little, particularly also on an old box
      without APIC. (Jan Beulich, DONE)

tools, nice to have:

    * xl compatibility with xm:

        * None remain

    * libxl: Add API to retrieve domain console tty (Bamvor Jian
      Zhang, patches posted)

    * xl.cfg(5) documentation patch for qemu-upstream
      videoram/videomem support:
      http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
      qemu-upstream doesn't support specifying videomem size for the
      HVM guest cirrus/stdvga.  (but this works with
      qemu-xen-traditional). (Pasi Kärkkäinen)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 TODO / Release

[Ian Campbell]

On Mon, 2012-05-28 at 10:23 +0100, Ian Campbell wrote:
> If you are aware of any bugs which must/should be fixed for 4.2 then
> please reply to this thread (otherwise I may not remember to pick them
> up next week)

Note to self -- pick up the bugs in the Intel VMX status reports.

:-)

Ian.

Re: Xen 4.2 TODO / Release

[Ian Campbell]

On Mon, 2012-05-28 at 10:23 +0100, Ian Campbell wrote:
> tools, blockers:

    * Confirm that migration from Xen 4.1 -> 4.2 works (Ian J)

Ian.

Re: Xen 4.2 TODO / Release

[Jan Beulich]

>>> On 28.05.12 at 11:23, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> hypervisor, nice to have:

* Grant table patches posted at the end of last week (no feedback
so far).

Jan

Re: Xen 4.2 TODO / Release

[Keir Fraser]

On 29/05/2012 09:15, "Jan Beulich" <JBeulich@suse.com> wrote:

>>>> On 28.05.12 at 11:23, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> hypervisor, nice to have:
> 
> * Grant table patches posted at the end of last week (no feedback
> so far).

Are they really for 4.2 at this point?

 -- Keir

> Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 TODO / Release

[Jan Beulich]

>>> On 29.05.12 at 17:03, Keir Fraser <keir.xen@gmail.com> wrote:
> On 29/05/2012 09:15, "Jan Beulich" <JBeulich@suse.com> wrote:
> 
>>>>> On 28.05.12 at 11:23, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>> hypervisor, nice to have:
>> 
>> * Grant table patches posted at the end of last week (no feedback
>> so far).
> 
> Are they really for 4.2 at this point?

I wanted to propose their inclusion, but I'm not intending to
talk you into accepting them (which partly is why they're under
"nice to have"). They're basically on the same page as the
massive p2m locking changes that went in pretty recently (used
as foundation for the changes here), so my perspective is that
they could reasonably go in for 4.2.

Jan

Re: Xen 4.2 TODO / Release

[Ian Campbell]

On Tue, 2012-05-29 at 09:15 +0100, Jan Beulich wrote:
> >>> On 28.05.12 at 11:23, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > hypervisor, nice to have:
> 
> * Grant table patches posted at the end of last week (no feedback
> so far).

I think these went in in the interim so I'm going to go straight to DONE
and garbage collect it from the list before posting. I like those kinds
of TODO items ;-)

Ian.

Re: Xen 4.2 TODO / Release

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 907 bytes --]

On Mon, 2012-05-28 at 10:23 +0100, Ian Campbell wrote: 
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> 
> ... 
> 
> tools, blockers:
> 
> ...
> 
> * xl compatibility with xm:
> 
>         * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>           reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>
> 
>         * does not automatically try to select a (set of) node(s) on
>           which to create a VM and pin its vcpus there. (Dario
>           Faggioli, patches posted)
            ^
            ^ Patches re-posted. Under review.

Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 TODO / Release

[W. Michael Petullo]

>> tools, blockers:
>> 
>> ...
>> 
>> * xl compatibility with xm:
>> 
>>         * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>>           reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>
>> 
>>         * does not automatically try to select a (set of) node(s) on
>>           which to create a VM and pin its vcpus there. (Dario
>>           Faggioli, patches posted)

I also noted another discrepancy (and provided a patch) between xm and
xl in:

	http://lists.xen.org/archives/html/xen-devel/2012-05/msg02285.html

-- 
Mike

:wq

Re: Xen 4.2 TODO / Release

[Ian Campbell]

On Fri, 2012-06-01 at 01:12 +0100, W. Michael Petullo wrote:
> >> tools, blockers:
> >> 
> >> ...
> >> 
> >> * xl compatibility with xm:
> >> 
> >>         * [BUG] cannot create an empty CD-ROM drive on HVM guest,
> >>           reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>
> >> 
> >>         * does not automatically try to select a (set of) node(s) on
> >>           which to create a VM and pin its vcpus there. (Dario
> >>           Faggioli, patches posted)
> 
> I also noted another discrepancy (and provided a patch) between xm and
> xl in:
> 
> 	http://lists.xen.org/archives/html/xen-devel/2012-05/msg02285.html

Thanks for the ping. I would have sworn this was fixed already in 4.2
but it seems I was thinking of 25236:7872b0918901 which relates to
cpupool-create and not create.

Ian.

Re: Xen 4.2 TODO / Release

[Roger Pau Monne]

Ian Campbell wrote:
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
> 2 April         -- Feature Freeze
>                                                  <<  WE ARE HERE
> Mid/Late May    -- First release candidate
> Weekly          -- RCN+1 until it is ready
>
> It seems pretty obvious at this stage that we are not going to be
> doing an RC in Mid/Late May. The big remaining items are being
> actively worked on and are quite well understood. I've also done a
> pass through the tools blockers list and have proposed moving some
> items to the Nice To Have list.
>
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.
>
> If you are aware of any bugs which must/should be fixed for 4.2 then
> please reply to this thread (otherwise I may not remember to pick them
> up next week)
>
> Other than that I think we should consider the freeze to be in full
> effect and the bar to entry to 4.2 to be very high.
>
> hypervisor, blockers:
>
>      * None
>
> tools, blockers:
>
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>
>          * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
>            Interface needs an overhaul, related to
>            locking/serialization over domain create (deferred to
>            4.3). IanJ to add note about this interface being
>            substandard but otherwise defer to 4.3. Will move to Nice To
>            Have's next week.
>
>          * libxl_*_path. Majority made internal, only configdir and
>            lockdir remain public (used by xl). (IanC, to consider
>            moving the two reamining into xl instead of libxl, will move
>            to Nice To Have's next week)
>
>          * Interfaces which may need to be async:
>
>              * libxl_domain_suspend. Move xc_domain_save/restore into a
>                separate process (IanJ, RFC posted, work ongoing).
>
>              * libxl_device_{disk,nic,vkb,add,pci}_add (and
>                remove?). Roger Pau Monné fixes disk as part of hotplug
>                script series and adds infrastructure which should make
>                the others trivial. (Roger, WIP)

Patches posted (in the hotplug series) to make disk and nic async (both 
addition and deletion). Once that is in, vfb and vkb are quite trivial. 
I'm not sure about PCI since I haven't looked at it.

You can also add:

* libxl_domain_destroy: convert to async. Mostly done since it will go 
in with my hotplug script series and the disk/nic add/remove.

>
>              * libxl_cdrom_insert. Should be easy once
>                disk_{add,remove} done. Nobody is looking at this? This
>                is basically a helper function and its functionality can
>                be implemented in terms of the libxl_disk_*
>                interfaces. We could therefore consider marking this
>                function as substandard and subject to change in the
>                future. We could also consider simply moving it into xl
>                as a helper function and revisting for 4.3.
>
>      * xl compatibility with xm:
>
>          * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>            reported by Fabio Fantoni in<4F9672DD.2080902@tiscali.it>
>
>          * does not automatically try to select a (set of) node(s) on
>            which to create a VM and pin its vcpus there. (Dario
>            Faggioli, patches posted)
>
>          * `cpus=[2,3]` to select specific mapping vcpu0-->pcpu2,
>            vcpu1-->pcpu3 as xm does. (Dario Faggioli, DONE)
>
>      * [BUG] Spurious CPU params error message when starting a domain,
>        e.g. "Cpu weight out of range, valid values are within range
>        from 1 to 65535". (Ian C, patches posted, need final decision on
>        "one struct" or "multiple struct" libxl API for sched params)
>
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>
>      * Domain 0 block attach&  general hotplug when using qdisk backend
>        (need to start qemu as necessary etc) (Stefano S, patches
>        posted, people seem happy bar the code motion)
>
>      * Improved Hotplug script support (Roger Pau Monné, patches
>        posted, nearing completion?)

I would call this "calling hotplug scripts from xl", "improved hotplug 
script support" seems quite vague. Yes, I think they are quite done, and 
people seems to be happy with that.

I also have the NetBSD support in my patch queue, waiting for the Linux 
support to go in (since it will be pointless to have to repost the 
NetBSD series every time the previous one changes).

>
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)

This will just be a matter of removing an "if" that's preventing libxl 
from using custom hotplug scripts for disks.

>      * Adjustments needed for qdisk backend to work on non-pvops Linux.
>        (Jan Beulich)

* qemu-upstream for NetBSD: I'm currently working on this. It compiles 
but VGA memory mapping seems to be wrong, and qemu gets a segfault when 
trying to write to it.

> hypervisor, nice to have:
>
>      * PoD performance improvements (George Dunlap)
>
>      * IRQ problem for which debugging code got added by c/s
>        24707:96987c324a4f. I have a tentative adjustment to this which
>        I would hope to at least eliminate the bad consequences of
>        crashing the hypervisor (converting the unsolicited
>        PIC-originated IRQ into a spurious one). I'll submit as soon as
>        I got to test this a little, particularly also on an old box
>        without APIC. (Jan Beulich, DONE)
>
> tools, nice to have:
>
>      * xl compatibility with xm:
>
>          * None remain
>
>      * libxl: Add API to retrieve domain console tty (Bamvor Jian
>        Zhang, patches posted)
>
>      * xl.cfg(5) documentation patch for qemu-upstream
>        videoram/videomem support:
>        http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
>        qemu-upstream doesn't support specifying videomem size for the
>        HVM guest cirrus/stdvga.  (but this works with
>        qemu-xen-traditional). (Pasi Kärkkäinen)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 TODO / Release

[Stefano Stabellini]

On Fri, 1 Jun 2012, Roger Pau Monne wrote:
> * qemu-upstream for NetBSD: I'm currently working on this. It compiles 
> but VGA memory mapping seems to be wrong, and qemu gets a segfault when 
> trying to write to it.

You might have to check the implementation of xc_domain_add_to_physmap
on NetBSD. On the QEMU side xen_add_to_physmap is the relevant function.
However I cannot see how it would be different from
qemu-xen-traditional.

Re: Xen 4.2 TODO / Release

[Roger Pau Monne]

Stefano Stabellini wrote:
> On Fri, 1 Jun 2012, Roger Pau Monne wrote:
>> * qemu-upstream for NetBSD: I'm currently working on this. It compiles
>> but VGA memory mapping seems to be wrong, and qemu gets a segfault when
>> trying to write to it.
>
> You might have to check the implementation of xc_domain_add_to_physmap
> on NetBSD. On the QEMU side xen_add_to_physmap is the relevant function.
> However I cannot see how it would be different from
> qemu-xen-traditional.

Well my description was not completely accurate, when the memory address 
is retrieved for the first time (vga.c:2225), qemu can write to it, but 
after some point (don't know exactly yet), the userspace process looses 
access to that memory region, and gets a segfault.