Xen 4.3 development update, 15 Oct

Xen 4.3 development update, 15 Oct

[George Dunlap]

This information will be mirrored on the Xen 4.3 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.3

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:
* Feature Freeze: 25 March 2013
* First RC: 6 May 2013
* Release: 17 June 2013

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Feature tracking =

Below is a list of features we're tracking for this release. Please
respond to this mail with any updates to the status.

There are a number of items whose owners are marked as "?".  If you
are working on this, or know who is working on it, please respond and
let me know.  Alternately, if you would *like* to work on it, please
let me know as well.

And if there is something you're working on you'd like tracked, please
respond, and I will add it to the list.

NB: Several of the items on this list are from external projects:
linux, qemu, and libvirt.  These are not part of the Xen tree, but are
directly related to our users' experience (e.g., work in Linux or
qemu) or to integration with other important projects (e.g., libvirt
bindings).  Since all of these are part of the Xen community work, and
comes from the same pool of labor, it makes sense to track the
progress here, even though they won't explicitly be released as part
of 4.3.

== Completed ==

* Linux console improvements
  -EHCI debug port

* Default to QEMU upstream (partial)
 - pci pass-thru (external)
 - enable dirtybit tracking during migration (external)

* CPUID-based idle (don't rely on ACPI info f/ dom0)

== Bugs ==

* xl, compat mode, and older kernels
  owner: ?
  Many older 32-bit PV kernels that can run on a 64-bit hypervisor with
  xend do not work when started with xl.  The following work-around seems to
  work:
    xl create -p lightning.cfg
    xenstore-write /local/domain/$(xl domid
lightning)/device/vbd/51713/protocol x86_32-abi
    xl unpause lightning
  This node is normally written by the guest kernel, but for older kernels
  seems not to be.  xend must have a work-around; port this work-around to xl.

== Not yet complete ==

* PVH mode (w/ Linux)
  owner: mukesh@oracle
  Linux: 3rd draft patches posted.
  Xen: Patches being cleaned up for submission

* Event channel scalability
  owner: attilio@citrix
  status: initial design proposed
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* ARM server port
  owner: ijc@citrix
  status: Core hypervisor patches accepted; Linux paches pending

* NUMA scheduler affinity
  critical
  owner: dario@citrix
  status: Patches posted

* NUMA Memory migration
  owner: dario@citrix
  status: ?

* blktap3
  owner: thanos@citrix
  status: in progress

* Default to QEMU upstream
 - qemu-based stubdom (Linux or BSD libc)
   owner: anthony@citrix
   status: in progress
   qemu-upstream needs a more fully-featured libc than exists in
   mini-os.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

 - xl cd-{insert,eject} (external)
   status: intilal implementation submitted

* Persistent grants (external)
  owner: roger.pau@citrix
  status: Initial implementation posted

* Multi-page blk rings (external)
 - blkback in kernel (konrad@oracle, ?@intel)
 - qemu blkback
  status: ?

* Multi-page net protocol (external)
  owner: ijc@citrix or annie.li@oracle
  status: Initial patches posted (by Wei Liu)
  expand the network ring protocol to allow multiple pages for
  increased throughput

* Scalability: 16TiB of RAM
  owner: jan@suse
  status: Not started

* vTPM updates
  owner: Matthew Fioravante @ Johns Hopkins
  status: v2 patches submitted
  - Allow all vTPM components to run in stub domains for increased security
  - Update vtpm to 0.7.1 from 0.5.x

* libvirt/libxl integration (external)
 - Migration
   owner: cyliu@suse (?)
   status: first draft implemented, not yet submitted
 - Itemize other things that need work
   To begin with, we need someone to go and make some lists:
   - Features available in libvirt/KVM not available in libvirt/libxl
     See http://libvirt.org/hvsupport.html
   - Features available in xl/Xen but not available in libvirt/Xen

* V4V: Inter-domain communication
  owner (Xen): jean.guyader@citrix.com
  status (Xen): patches submitted
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

* Wait queues for mm (NEW)
  owner: ?
  status: Draft posted Feb 2012; more work to do.

* xl vm-{export,import}
  owner: ?
  status: ?
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.

* xl PVUSB pass-through for both PV and HVM guests
  owner: ?
  status: ?
  xm/xend supports PVUSB pass-through to guests with PVUSB drivers
(both PV and HVM guests).
  - port the xm/xend functionality to xl.
  - this PVUSB feature does not require support or emulation from Qemu.
  - upstream the Linux frontend/backend drivers. Current
work-in-progress versions are in Konrad's git tree.
  - James Harper's GPLPV drivers for Windows include PVUSB frontend drivers.

* xl USB pass-through for HVM guests using Qemu USB emulation
  owner: ?
  status: ?
  xm/xend with qemu-traditional supports USB passthrough to HVM guests
using the Qemu emulated USB controller.
  The HVM guest does not need any special drivers for this feature.
  So basicly the qemu cmdline needs to have:
     -usb -usbdevice host:xxxx:yyyy
  - port the xm/xend functionality to xl.
  - make sure USB passthrough with xl works with both qemu-traditional
and qemu-upstream.

* xl QXL Spice support (NEW)
  owner: Zhou Peng
  status: Patches against 4.2-unstable posted, need refresh & resubmit

* openvswitch toostack integration
  owner: roger@citrix
  status: Sample script posted by Bastian ("[RFC] openvswitch support script")

* Rationalized backend scripts (incl. driver domains)
  owner: roger@citrix
  status: ?

* Linux console improvements
  owner: ?
  status: Stalled (see below)
  -xHCI debug port (Needs hardware)
  -Firewire (needs hardware)

* Remove hardcoded mobprobe's in xencommons
  owner: ?
  status: ?

* Make storage migration possible
  owner: ?
  status: ?
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: ?
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: May need review
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: May need review

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?

* Managed domains?

Re: Xen 4.3 development update, 15 Oct

[Jan Beulich]

>>> On 15.10.12 at 18:19, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> == Completed ==
> 
> * Linux console improvements

Here and below, please do s/Linux/Serial/ (functionality has nothing
to do with Linux).

Jan

>   -EHCI debug port

Re: Xen 4.3 development update, 15 Oct

[Konrad Rzeszutek Wilk]

> == Not yet complete ==
> 
> * PVH mode (w/ Linux)
>   owner: mukesh@oracle
>   Linux: 3rd draft patches posted.
>   Xen: Patches being cleaned up for submission
> 
> * Event channel scalability
>   owner: attilio@citrix
>   status: initial design proposed
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
> 
> * ARM server port
>   owner: ijc@citrix
>   status: Core hypervisor patches accepted; Linux paches pending

Ian,
Are these the ones that interact with the PVH ones?
> * blktap3
> * Multi-page blk rings (external)
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?

Nothing yet.
> 
> * Multi-page net protocol (external)
>   owner: ijc@citrix or annie.li@oracle
>   status: Initial patches posted (by Wei Liu)
>   expand the network ring protocol to allow multiple pages for
>   increased throughput

We are cleaning them up to get Ian's input. There are .. many things
here for this TODO - there is the feature persistent grant which gives a big
performance boost. Then there is the Wei Liu patches which need
some rework (they were RFC). The persistent grant (which is what
Annie is working on right now) nicely fit on top of Wei Liu's, but since
they (the RFC patches - which split the netback into per-vif threads)
are not yet ready - annie is looking to see if she can do some lookup
using the old netback mechanism.

Re: Xen 4.3 development update, 15 Oct

[Ian Campbell]

On Tue, 2012-10-16 at 15:45 +0100, Konrad Rzeszutek Wilk wrote:
> > * ARM server port
> >   owner: ijc@citrix
> >   status: Core hypervisor patches accepted; Linux paches pending
> 
> Ian,
> Are these the ones that interact with the PVH ones?

Some of the ARM functionality builds on the PVH stuff (e.g. the privcmd
stuff) but other bits are independent of PVH.

Ian.

Re: Xen 4.3 development update, 15 Oct

[George Dunlap]

On 16/10/12 15:45, Konrad Rzeszutek Wilk wrote:
>> * Multi-page net protocol (external)
>>    owner: ijc@citrix or annie.li@oracle
>>    status: Initial patches posted (by Wei Liu)
>>    expand the network ring protocol to allow multiple pages for
>>    increased throughput
> We are cleaning them up to get Ian's input. There are .. many things
> here for this TODO - there is the feature persistent grant which gives a big
> performance boost. Then there is the Wei Liu patches which need
> some rework (they were RFC). The persistent grant (which is what
> Annie is working on right now) nicely fit on top of Wei Liu's, but since
> they (the RFC patches - which split the netback into per-vif threads)
> are not yet ready - annie is looking to see if she can do some lookup
> using the old netback mechanism.
OK, so it sounds like you're actually giving an update on the persistent 
grants...?

  -George

Re: Xen 4.3 development update, 15 Oct

[Alex Bligh]

--On 15 October 2012 17:19:05 +0100 George Dunlap <George.Dunlap@eu.citrix.com> wrote:

> * Make storage migration possible
>   owner: ?
>   status: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of libxl
>   or xl.

We have this working with qemu-xen, qcow2 and snapshot rebase. At a libxl
level (but not an xl level), everything seems to be there.

-- 
Alex Bligh

Re: Xen 4.3 development update, 15 Oct

[George Dunlap]

[-- Attachment #1.1: Type: text/plain, Size: 641 bytes --]

On Tue, Dec 18, 2012 at 1:57 PM, Alex Bligh <alex@alex.org.uk> wrote:

>
>
> --On 15 October 2012 17:19:05 +0100 George Dunlap <
> George.Dunlap@eu.citrix.com> wrote:
>
>  * Make storage migration possible
>>   owner: ?
>>   status: ?
>>   There needs to be a way, either via command-line or via some hooks,
>>   that someone can build a "storage migration" feature on top of libxl
>>   or xl.
>>
>
> We have this working with qemu-xen, qcow2 and snapshot rebase. At a libxl
> level (but not an xl level), everything seems to be there.
>

Can you describe in more detail how you implement this?  Do you have a
script or something?

 -George

[-- Attachment #1.2: Type: text/html, Size: 1285 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.3 development update, 15 Oct

[Alex Bligh]

--On 18 December 2012 14:28:57 +0000 George Dunlap 
<George.Dunlap@eu.citrix.com> wrote:

>>> * Make storage migration possible
>>>   owner: ?
>>>   status: ?
>>>   There needs to be a way, either via command-line or via some hooks,
>>>   that someone can build a "storage migration" feature on top of libxl
>>>   or xl.
>>
>> We have this working with qemu-xen, qcow2 and snapshot rebase. At a libxl
>> level (but not an xl level), everything seems to be there.
>
> Can you describe in more detail how you implement this?  Do you have a
> script or something?

We have a pile of C code :-)

A script would do something like:

1. Ask qemu to do a live snapshot using snapshot_blkdev putting the
snapshot in a new file on the new storage device. This ensures that all new
writes go to the new storage device.

2. Rebase the snapshot to a null backing file (I think that's qemu-img
rebase with -b '' though we had to submit a couple of lines of patch to
qemu to make it work) which fills the non-written blocks from the new
snapshot from the old base image and breaks the link to the the old base
image.

In our implementation we have a separate hierarchy database from the parent
links stored within the qcow2 files so we get proper usage counting etc.
but that's an 'implementation detail'.

>From memory (and at risk of crossing threads) using this device model and
external snapshots you can use qemu-img resize to resize drives live at
least under KVM. Obviously the information that the drive has been resized
needs to get to the guest somehow.

-- 
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.3 development update, 15 Oct

[George Dunlap]

On 18/12/12 16:03, Alex Bligh wrote:
>
>
> --On 18 December 2012 14:28:57 +0000 George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
>
>>>> * Make storage migration possible
>>>>    owner: ?
>>>>    status: ?
>>>>    There needs to be a way, either via command-line or via some hooks,
>>>>    that someone can build a "storage migration" feature on top of libxl
>>>>    or xl.
>>>
>>> We have this working with qemu-xen, qcow2 and snapshot rebase. At a libxl
>>> level (but not an xl level), everything seems to be there.
>>
>> Can you describe in more detail how you implement this?  Do you have a
>> script or something?
>
> We have a pile of C code :-)
>
> A script would do something like:
>
> 1. Ask qemu to do a live snapshot using snapshot_blkdev putting the
> snapshot in a new file on the new storage device. This ensures that all new
> writes go to the new storage device.
>
> 2. Rebase the snapshot to a null backing file (I think that's qemu-img
> rebase with -b '' though we had to submit a couple of lines of patch to
> qemu to make it work) which fills the non-written blocks from the new
> snapshot from the old base image and breaks the link to the the old base
> image.

So it sounds like in this case you're talking about moving the disk to a 
different storage device connected to the same host, leaving the VM 
running on the same host.

What I was talking about in this was migrating the VM and storage 
together to a new host.  People who want this typically have all VMs on 
local storage, so (if I'm understanding you right) the snapshot trick 
won't work, because host A (where it's running) can't directly access 
host B's disk (to which we want to migrate it).

Although I suppose one could always hack something together with sshfs 
or something. :-)

  -George

Re: Xen 4.3 development update, 15 Oct

[Alex Bligh]

George,

--On 18 December 2012 16:40:07 +0000 George Dunlap 
<george.dunlap@eu.citrix.com> wrote:

> So it sounds like in this case you're talking about moving the disk to a
> different storage device connected to the same host, leaving the VM
> running on the same host.

Correct, though as you suggest below, in order to achieve the effect
you describe what we do is ensure that between the two migrates
starting and finishing the relevant disk access to both VMs is
maintained (in our case via NFS rather than sshfs :-) ). So it
gives the illusion of working even when the migrate is local disk
to local disk. Generalising this is left as an exercise for the
reader.

Giving xl access to qmp snapshot_blkdev_sync and the qemu-img rebase
would no doubt be useful to those using xl though.

Alex

> What I was talking about in this was migrating the VM and storage
> together to a new host.  People who want this typically have all VMs on
> local storage, so (if I'm understanding you right) the snapshot trick
> won't work, because host A (where it's running) can't directly access
> host B's disk (to which we want to migrate it).
>
> Although I suppose one could always hack something together with sshfs or
> something. :-)



-- 
Alex Bligh