Re: Xen 4.4 development update

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Aug 08, 2013 at 05:09:35PM +0100, George Dunlap wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> 
> Rather than try to predict precisely what will make it into what
> release (which was something of a disaster last release), I'm just
> going to borrow a term from the Agile world and call all uncompleted
> features the "Backlog".  I'll still track who is doing what, and when
> we get close, what state things seem to be in.
> 
> As mentioned in another e-mail, we'll also be working on improving the
> regression tester.  Feel free to join us.
> 
> And as always, if you are working on a feature / bug that you want
> tracked, please respond to this e-mail.
> 
> = Timeline =
> 
> As discussed elsewhere, I am proposing a 6-month release cycle.  Xen
> 4.3 was released on 9 July.  That would give us a release on 9 January
> 2014.  This is fairly close after the Christmas season, so I propose
> to make the estimated release date later, on 21 January, giving a few
> extra weeks for the holiday season:
> 
> * Feature freeze: 18 October 2013
> * Code freezing point: 8 November 2013
> * First RC: 26 November 2013
> * Release: 21 January 2014
> 
> Feedback on the estimated dates  is welcome.
> 
> Last updated: 8 August 2013
> 
> == Completed ==
> 
> [none]
> 
> == Open ==
> 
> * xend still in tree
> 
> * qemu-upstream not freeing pirq
>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>  status: patches posted

They did fix the problem Oracle saw and I believe have been
Tested-and-Acked.

> 
> * __update_vcpu_system_time if system_time comes out negative
> 
> * xl pci-detach broken for pv guests?
>   > http://bugs.xenproject.org/xen/bug/12
>   > kernel doesn't know about moving into states 7 and 8
>   status: External

Didn't I post a patch for this on Linux - and it is in Linux kernel
already.
> 
> * Race in PV shutdown between tool detection and shutdown watch
>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>  > Nothing to do with ACPI
>  status: Probably a bug in Linux xen drivers
> 
> == Backlog ==
> 
> === Testing coverage ===
> 
> * Network driver domains
>  @George
> 
> * new libxl w/ previous versions of xl
>  @IanJ
> 
> * Host S3 suspend
>  @bguthro
> 
> * Default [example] XSM policy
>  @Stefano to ask Daniel D
> 
> * Xen on ARM
>  # hardware
>   @ianc
>    emulator: @stefano to think about it
> 
> * Storage driver domains
>  @roger
> 
> * HVM pci passthrough
>  @anthony
> 
> * Nested virt?
>  @intel (chased by George)
> 
> * Fix SRIOV test (chase intel)
>  @ianj
> 
> * Fix bisector to e-mail blame-worthy parties
>  @ianj
> 
> * Fix xl shutdown
>   @ianj
> 
> * stub domains
>   @athony
> 
> === Big ticket items ===
> 
> * NUMA Memory migration
>   owner: dario@citrix
>   status: in progress
> 
> * Event channel scalability
>   owner: david@citrix
>   status: RFC v5 submitted
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)

aka FIFO thing right?

> 
> * PVH mode (w/ Linux)
>   owner: mukesh@oracle
>   status (Linux): 3rd draft patches posted.

More like v8. And already Acked - just waiting for ABI to be in some way
"nailed down".

>   status (Xen): v10 posted
> 
> * ARM stuff: ??
> 
> * Meta: PVIO NUMA improvements
>  - NUMA affinity for vcpus
>     owner: Dario
>  - PV guest NUMA interface
>     owner: Elena
>  - Sensible dom0 NUMA layout
>  - Toolstack pinning backend thread / virq to appropraite d0 vcpu
> 
> * qemu-upstream stubdom, Linux
>    owner: anthony@citrix
>    status: in progress
>    qemu-upstream needs a more fully-featured libc than exists in
>    mini-os.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
> 
> * qemu-upstream stubdom, BSD libc
>   owner: ianj@citrix
> 
> * Network performance improvements
>   owner: wei@citrix
> 
> * Disk performance improvements

I think you mean blkback and blkfront?
> 
> * Xen EFI feature: Xen can boot from grub.efi
>  owner: Daniel Kiper
>  status: Just begun
>  prognosis: Fair
> 
> * libvirt/libxl integration (external)
>  > need a status update
>  - owner: jfehlig@suse, dario@citrix
> 
> * Default to credit2
>  - cpu pinning
>  - NUMA affinity
>  - cpu "reservation"
> 
> * xenperf
>   Owner: Boris Otovsky
> 
> * blktap3
>   owner: thanos@citrix
>   status: on hold pending XenServer decision
> 
> * Nested virtualization on Intel
> 
> * Nested virtualization on AMD
> 
> * Make storage migration possible
>   owner: ?
>   status: none
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of libxl
>   or xl.
> 
> * Full-VM snapshotting
>   owner: ?
>   status: none
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
> 
> * VM Cloning
>   owner: ?
>   status: none
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
> 
> * xl vm-{export,import}
>   owner: ?
>   status: none
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
> 
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
>   status: none
> 
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>   status: ?
> 
> * Wait queues for mm
>  > Needed for more advanced paging schemes
>   owner: ?
>   status: Draft posted Feb 2012; more work to do.
> 
> * V4V: Inter-domain communication
>   owner (Xen): dominic.curran@citrix.com
>   status (Xen): patches submitted
>   owner (Linux driver):  stefano.panella@citrix
>   status (Linux driver): in progress
> 
> === clean-ups ===
> 
> * Sort out better memory / ballooning / dom0 autoballooning thing
>  > Don't forget NUMA angle
>  - Inaccurate / incomplete info from HV
> 
> * Implement Xen hypervisor dmesg log entry timestamps
> 
> * Make network driver domains easier to set up / more useful
>  - Default backend (submitted)
>  - Remove unnecessary restriction wrt dom0 hotplug scripts (submitted)
>  - Make it easy to make a device assignable (in discussion)
>  - Automatically start/shutdown (xendomains?)
>  - Pause booting of other domains until network driver domain is up
> 
> * libxl: More fine-grained control over when to pass through a device
>  > Some IOMMUs are secure; some are merely functional, some are not present.
>  > Allow the adminitrator to set the default
> 
> * xl does not handle migrate interruption gracefully
>   > If you start a localhost migrate, and press "Ctrl-C" in the middle,
>   > you get two hung domains
>   status: Probably not for 4.3
> 
> * libxl / xl does not handle failure of remote qemu gracefully
>   > Easiest way to reproduce:
>   >  - set "vncunused=0" and do a local migrate
>   >  - The "remote" qemu will fail because the vnc port is in use
>   > The failure isn't the problem, but everything being stuck afterwards is
>   status: Probably not for 4.3
> 
> * mac address changes on reboot if not specified in config file
>   > Needs a robust way to "add" to the config
>   status: Too much for 4.3
> 
> * qxl
>   > http://bugs.xenproject.org/xen/bug/11http://bugs.xenproject.org/xen/bug/11
>   - Uninitialized struct element in qemu
>   - Revert 5479961 to re-enable qxl in xl,libxl
>   - Option in Xen top-level to enable qxl support in qemu tree
>   - Fix sse2 MMIO issue
> 
> * libxl config file
> 
> * libxl: Don't use RAW format for "URL"-based qdisks (e.g., rbd:rbd/foo.img)
>   - Figure out whether to use a generic URL or have a specific type for each one
>   - Check existence of disk file for all RAW
> 
> * Polish up xenbugtool
>   owner: wei.liu2@citrix.com
> 
> * acpi-related xenstore entries not propagated on migrate
>  > http://www.gossamer-threads.com/lists/xen/devel/282466
>  > Only used by hvmloader; only a clean-up, not a bug.
>  status: Not for 4.3
> 
> * Remove hardcoded mobprobe's in xencommons
>   owner: Wei Liu
>   status: still in discussion
> 
> * xl USB pass-through for HVM guests using Qemu USB emulation
>   owner: George
>   status: v6 patch series posted
> 
> * Rationalized backend scripts
>   owner: roger@citrix
>   status: patches posted
> 
> * Scripts for driver domains (depends on backend scripts)
>   owner: roger@citrix
>   status:
> 
> * Multi-vector PCI MSI (support at least for Dom0)
>   owner: jan@suse
>   status: Patches posted for Intel; AMD not yet done

And patches for Linux upstream are missing.

> 
> * xl: passing more defaults in configuration in xl.conf
>   owner: ?
>   There are a number of options for which it might be useful to pass a
>   default in xl.conf.  For example, if we could have a default
>   "backend" parameter for vifs, then it would be easy to switch back
>   and forth between a backend in a driver domain and a backend in dom0.
> 
> * xl PVUSB pass-through for PV guests
> * xl PVUSB pass-through for HVM guests
>   owner: George
>   status: ?
>   xm/xend supports PVUSB pass-through to guests with PVUSB drivers
> (both PV and HVM guests).
>   - port the xm/xend functionality to xl.
>   - this PVUSB feature does not require support or emulation from Qemu.
>   - upstream the Linux frontend/backend drivers. Current
> work-in-progress versions are in Konrad's git tree.

Which is to say I hadn't actually touched them in a year.

>   - James Harper's GPLPV drivers for Windows include PVUSB frontend drivers.
> 
> * Guest EFI booting
>  - status: tianocore in-tree, some build problems.
>    Needs new owner.
> 
> * Xen EFI feature: pvops dom0 able to make use of EFI run-time
> services (external)
>  owner: Daniel Kiper
>  status: Just begun
> 
> * Serial console improvements
>   owner: ?
>   status: Stalled (see below)
>   -xHCI debug port (Needs hardware)
>   -Firewire (needs hardware)
> 

I would like also to put GPU passthrough working here.

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel