GSoC 2012 project brainstorming

GSoC 2012 project brainstorming

[Ian Campbell]

Below are a few ideas which have been floating around as potential
projects (or broad subject areas for projects) for GSoC this year.

If you want any more information on any particular item then please ask.
Likewise if you have any ideas of your own please feel free to chip in.

If you think you might be interested in a project, either as mentor or
potential student, then please add it to the wiki page at
http://wiki.xen.org/wiki/GSoC_2012_Ideas

Ian.

TOOLS
-----
- pv grub2
- xapi support in libvirt
- make xapi use libxl
- compile xapi on ARM
- OpenXenManager
- driver domains
- PV dbus
- HA daemon for Remus
- HA daemon for XCP

PERF
----
- Oprofile
- Linux perf tools in guest

HYPERVISOR
----------
- insmod Xen
- event channel limits
- NUMA

MEMORY
------
- disk based memory sharing
- memory scanner
- paging replacing PoD
- VM Fork
- Copy on read (past VDI boot)


IO
--
- PV OpenGL/Gallium
- PV USB
- PV USB3
- PV SCSI
- PVFB in Xorg
- Infiniband


STORAGE
-------
- gluster/ceph plugins


QEMU
----
- BSD libc
- upstream QEMU stubdoms


TESTS
-----
- better web reporting
- more tests
- upstream linux


DISTROS
-------
- packaging stubdoms
- xen on centos6
- driver domains
- figure out the VM format issue
- XSM in distros

Re: [Xen-devel] GSoC 2012 project brainstorming

[Pasi Kärkkäinen]

On Wed, Feb 29, 2012 at 04:53:57PM +0000, Ian Campbell wrote:
> Below are a few ideas which have been floating around as potential
> projects (or broad subject areas for projects) for GSoC this year.
> 
> If you want any more information on any particular item then please ask.
> Likewise if you have any ideas of your own please feel free to chip in.
> 
> If you think you might be interested in a project, either as mentor or
> potential student, then please add it to the wiki page at
> http://wiki.xen.org/wiki/GSoC_2012_Ideas
> 
> Ian.
> 
> 
> HYPERVISOR
> ----------
> - insmod Xen
> - event channel limits
> - NUMA
> 

See below..

> IO
> --
> - PV OpenGL/Gallium
> - PV USB

Just for reference..
PVUSB (USB 2) driver for pvops Linux 3.x is available from:
http://lists.xen.org/archives/html/xen-devel/2012-02/msg00576.html
http://lists.xen.org/archives/html/xen-devel/2012-02/msg00571.html

http://wiki.xen.org/wiki/Xen_USB_Passthrough


> - PV USB3

Is anyone working on this?


> - PV SCSI

PVSCSI driver for pvops Linux 3.x is available from:
http://git.kernel.org/?p=linux/kernel/git/konrad/xen.git;a=shortlog;h=refs/heads/devel/xen-scsi.v1.0

http://wiki.xen.org/wiki/Paravirtualized_SCSI

> 
> 
> QEMU
> ----
> - BSD libc
> - upstream QEMU stubdoms
> 

Not sure if this should be in HYPERVISOR or QEMU section:

- Xen HVM guest direct kernel+initramfs boot support, just like for PV guests. 
This has been requested a couple of times and would be useful feature to have.
KVM/Qemu supports this.


-- Pasi

Re: GSoC 2012 project brainstorming

[Konrad Rzeszutek Wilk]

On Wed, Feb 29, 2012 at 04:53:57PM +0000, Ian Campbell wrote:
> Below are a few ideas which have been floating around as potential
> projects (or broad subject areas for projects) for GSoC this year.
> 
> If you want any more information on any particular item then please ask.
> Likewise if you have any ideas of your own please feel free to chip in.
> 
> If you think you might be interested in a project, either as mentor or
> potential student, then please add it to the wiki page at
> http://wiki.xen.org/wiki/GSoC_2012_Ideas
> 
> Ian.
> 
> TOOLS
> -----
> - pv grub2
> - xapi support in libvirt
> - make xapi use libxl
> - compile xapi on ARM
> - OpenXenManager
> - driver domains
> - PV dbus
> - HA daemon for Remus
> - HA daemon for XCP
> 
> PERF
> ----
> - Oprofile

So the oprofile mega patch posted by Michale seems to work. But it is
riddled with #ifdef CONFIG_XEN so not very nice.

> - Linux perf tools in guest

 from this e-mail thread:

https://lkml.org/lkml/2012/2/12/74

it actually seems to work in dom0 - well, there is a bug so not exactly good.

But having the tool to work with dom0 and domU would be super.

Sign me up as a mentor for that!


> 
> HYPERVISOR
> ----------
> - insmod Xen
> - event channel limits
> - NUMA

Yeah, we need that. I think there were some patches posted for that.. whatever
happend to them?

> 
> MEMORY
> ------
> - disk based memory sharing
> - memory scanner
> - paging replacing PoD
> - VM Fork
> - Copy on read (past VDI boot)
> 
> 
> IO
> --
> - PV OpenGL/Gallium
> - PV USB
> - PV USB3
> - PV SCSI
> - PVFB in Xorg
> - Infiniband

Huh? InfiniBand PV drivers?

> 
> 
> STORAGE
> -------
> - gluster/ceph plugins
> 
> 
> QEMU
> ----
> - BSD libc
> - upstream QEMU stubdoms
> 
> 
> TESTS
> -----
> - better web reporting
> - more tests
> - upstream linux
> 
> 
> DISTROS
> -------
> - packaging stubdoms
> - xen on centos6
> - driver domains
> - figure out the VM format issue
> - XSM in distros
> 
> 
> 
> _______________________________________________
> xen-api mailing list
> xen-api-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Re: GSoC 2012 project brainstorming

[Ian Campbell]

On Mon, 2012-03-12 at 17:00 +0000, Konrad Rzeszutek Wilk wrote:
> On Wed, Feb 29, 2012 at 04:53:57PM +0000, Ian Campbell wrote:
> > Below are a few ideas which have been floating around as potential
> > projects (or broad subject areas for projects) for GSoC this year.
> > 
> > If you want any more information on any particular item then please ask.
> > Likewise if you have any ideas of your own please feel free to chip in.
> > 
> > If you think you might be interested in a project, either as mentor or
> > potential student, then please add it to the wiki page at
> > http://wiki.xen.org/wiki/GSoC_2012_Ideas
> > 
> > Ian.
> > 
> > TOOLS
> > -----
> > - pv grub2
> > - xapi support in libvirt
> > - make xapi use libxl
> > - compile xapi on ARM
> > - OpenXenManager
> > - driver domains
> > - PV dbus
> > - HA daemon for Remus
> > - HA daemon for XCP
> > 
> > PERF
> > ----
> > - Oprofile
> 
> So the oprofile mega patch posted by Michale seems to work. But it is
> riddled with #ifdef CONFIG_XEN so not very nice.
> 
> > - Linux perf tools in guest
> 
>  from this e-mail thread:
> 
> https://lkml.org/lkml/2012/2/12/74
> 
> it actually seems to work in dom0 - well, there is a bug so not exactly good.
> 
> But having the tool to work with dom0 and domU would be super.
> 
> Sign me up as a mentor for that!

Please update http://wiki.xen.org/wiki/GSoC_2012_Ideas as
appropriate ;-)

> > 
> > HYPERVISOR
> > ----------
> > - insmod Xen
> > - event channel limits
> > - NUMA
> 
> Yeah, we need that. I think there were some patches posted for that.. whatever
> happend to them?

I think there are some patches, which have gone in and various ideas for
ongoing work but I'm not sure what the details were -- hopefully whoever
shouted out NUMA (this was a brainstorm remember ;-)) will chime in.

> > 
> > - Infiniband
> 
> Huh? InfiniBand PV drivers?

I think this is about using Infiniband for extra fast migration, by
pumping the migration data over infiniband instead of a normal network.
But again hopefully someone who knows what that bullet was will chime
in. I seem to remember someone mentioning it at the Munich hackathon.

Ian.

Re: GSoC 2012 project brainstorming

[Konrad Rzeszutek Wilk]

> > Sign me up as a mentor for that!
> 
> Please update http://wiki.xen.org/wiki/GSoC_2012_Ideas as
> appropriate ;-)

Done!
> 
> > > 
> > > HYPERVISOR
> > > ----------
> > > - insmod Xen
> > > - event channel limits
> > > - NUMA
> > 
> > Yeah, we need that. I think there were some patches posted for that.. whatever
> > happend to them?
> 
> I think there are some patches, which have gone in and various ideas for
> ongoing work but I'm not sure what the details were -- hopefully whoever
> shouted out NUMA (this was a brainstorm remember ;-)) will chime in.

Dulloor Rao had some patches: http://old-list-archives.xen.org/archives/html/xen-devel/2010-04/msg00105.html

> 
> > > 
> > > - Infiniband
> > 
> > Huh? InfiniBand PV drivers?
> 
> I think this is about using Infiniband for extra fast migration, by
> pumping the migration data over infiniband instead of a normal network.
> But again hopefully someone who knows what that bullet was will chime
> in. I seem to remember someone mentioning it at the Munich hackathon.
> 
> Ian.

Re: [Xen-API] GSoC 2012 project brainstorming

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1631 bytes --]

On Mon, 2012-03-12 at 13:00 -0400, Konrad Rzeszutek Wilk wrote:
> > HYPERVISOR
> > ----------
> > - insmod Xen
> > - event channel limits
> > - NUMA
> 
> Yeah, we need that.
>
I'm looking into some aspects of this. Basically, I'm (trying to? :-D)
putting some NUMA-aware VM placement logic in xl, basing the decision on
how much free memory we have in each node.

I'm also trying to have a meaningful set of benchmarks to better
understand the performance implications of a sub-optimal placement.

I'm hoping to have the implementation of this (memory-wise only for now)
placemen logic soon, and I have a quite hard deadline for it placed at
the end of the month, so... :-)

As usual, any comment and ideas are more than welcome.

> I think there were some patches posted for that.. whatever
> happend to them?
> 
I've searched the list and the web for things like that and didn't find
anything, but that could of course be my fault. Currently, my reference
is what xm/xend does in order to deply VMs on the various nodes... I
really hope the final result will look better than that, but that's
enough for the first step.

Finally, I'm not yet sure if there will be something suitable for GsoC
here, but again I'm open to discuss any idea! :-)

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] GSoC 2012 project brainstorming

[Ian Campbell]

On Wed, 2012-02-29 at 16:53 +0000, Ian Campbell wrote:
> Below are a few ideas which have been floating around as potential
> projects (or broad subject areas for projects) for GSoC this year.
> 
> If you want any more information on any particular item then please ask.
> Likewise if you have any ideas of your own please feel free to chip in.
> 
> If you think you might be interested in a project, either as mentor or
> potential student, then please add it to the wiki page at
> http://wiki.xen.org/wiki/GSoC_2012_Ideas
> 

I've just added a proposal for "xl to xapi offline VM migration
utility". I know the xl side well and I'm reasonably familiar with
XenAPI etc but it would be really useful to have a co-/backup-mentor
from the xen-api@ side of things.

Ian.

> TOOLS
> -----
> - pv grub2
> - xapi support in libvirt
> - make xapi use libxl
> - compile xapi on ARM
> - OpenXenManager
> - driver domains
> - PV dbus
> - HA daemon for Remus
> - HA daemon for XCP
> 
> PERF
> ----
> - Oprofile
> - Linux perf tools in guest
> 
> HYPERVISOR
> ----------
> - insmod Xen
> - event channel limits
> - NUMA
> 
> MEMORY
> ------
> - disk based memory sharing
> - memory scanner
> - paging replacing PoD
> - VM Fork
> - Copy on read (past VDI boot)
> 
> 
> IO
> --
> - PV OpenGL/Gallium
> - PV USB
> - PV USB3
> - PV SCSI
> - PVFB in Xorg
> - Infiniband
> 
> 
> STORAGE
> -------
> - gluster/ceph plugins
> 
> 
> QEMU
> ----
> - BSD libc
> - upstream QEMU stubdoms
> 
> 
> TESTS
> -----
> - better web reporting
> - more tests
> - upstream linux
> 
> 
> DISTROS
> -------
> - packaging stubdoms
> - xen on centos6
> - driver domains
> - figure out the VM format issue
> - XSM in distros
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> http://lists.xen.org/xen-devel

Re: [Xen-devel] GSoC 2012 project brainstorming

[Dave Scott]

Ian Campbell wrote: 
> On Wed, 2012-02-29 at 16:53 +0000, Ian Campbell wrote:
> > Below are a few ideas which have been floating around as potential
> > projects (or broad subject areas for projects) for GSoC this year.
> >
> > If you want any more information on any particular item then please
> ask.
> > Likewise if you have any ideas of your own please feel free to chip
> in.
> >
> > If you think you might be interested in a project, either as mentor
> or
> > potential student, then please add it to the wiki page at
> > http://wiki.xen.org/wiki/GSoC_2012_Ideas
> >
> 
> I've just added a proposal for "xl to xapi offline VM migration
> utility". I know the xl side well and I'm reasonably familiar with
> XenAPI etc but it would be really useful to have a co-/backup-mentor
> from the xen-api@ side of things.

I could be a co-/backup-mentor for that.

I've been busy recently moving all the domain management parts of xapi
into a separate daemon, which later can be libxl'ed up. For testing
(and for fun) I've made it understand some of xm/xl config file syntax eg

[root@st20 ~]# xn export-metadata-xm win7 win7.xm
[root@st20 ~]# cat win7.xm 
name='win7'
builder='hvmloader'
boot='dc'
vcpus=1
memory=2048
disk=[ 'sm:7af570d8-f8c5-4103-ac1d-969fe28bfc11,hda,w', 'sm:137c8a61-113c-ab46-20fa-5c0574eaff77,hdb:cdrom,r' ]
vif=[  ]
pci=[  ]
pci_msitranslate=1
pci_power_mgmt=0
# transient=true

Another goal of the refactoring is to allow xapi to co-exist with domains
created by someone else (e.g. xl/libxl). This should allow a migration to
be done piecemeal, one VM at a time on the same host.

Cheers,
Dave

> 
> Ian.
> 
> > TOOLS
> > -----
> > - pv grub2
> > - xapi support in libvirt
> > - make xapi use libxl
> > - compile xapi on ARM
> > - OpenXenManager
> > - driver domains
> > - PV dbus
> > - HA daemon for Remus
> > - HA daemon for XCP
> >
> > PERF
> > ----
> > - Oprofile
> > - Linux perf tools in guest
> >
> > HYPERVISOR
> > ----------
> > - insmod Xen
> > - event channel limits
> > - NUMA
> >
> > MEMORY
> > ------
> > - disk based memory sharing
> > - memory scanner
> > - paging replacing PoD
> > - VM Fork
> > - Copy on read (past VDI boot)
> >
> >
> > IO
> > --
> > - PV OpenGL/Gallium
> > - PV USB
> > - PV USB3
> > - PV SCSI
> > - PVFB in Xorg
> > - Infiniband
> >
> >
> > STORAGE
> > -------
> > - gluster/ceph plugins
> >
> >
> > QEMU
> > ----
> > - BSD libc
> > - upstream QEMU stubdoms
> >
> >
> > TESTS
> > -----
> > - better web reporting
> > - more tests
> > - upstream linux
> >
> >
> > DISTROS
> > -------
> > - packaging stubdoms
> > - xen on centos6
> > - driver domains
> > - figure out the VM format issue
> > - XSM in distros
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> > http://lists.xen.org/xen-devel
> 
> 
> 
> _______________________________________________
> xen-api mailing list
> xen-api-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Re: [Xen-devel] GSoC 2012 project brainstorming

[Ian Campbell]

On Wed, 2012-03-14 at 17:55 +0000, Dave Scott wrote:
> Ian Campbell wrote: 
> > I've just added a proposal for "xl to xapi offline VM migration
> > utility". I know the xl side well and I'm reasonably familiar with
> > XenAPI etc but it would be really useful to have a co-/backup-mentor
> > from the xen-api@ side of things.
> 
> I could be a co-/backup-mentor for that.

Excellent, thanks!

> I've been busy recently moving all the domain management parts of xapi
> into a separate daemon, which later can be libxl'ed up. For testing
> (and for fun) I've made it understand some of xm/xl config file syntax eg

Interesting. Pure ocaml I presume?

> [root@st20 ~]# xn export-metadata-xm win7 win7.xm

xn? We're going to run out of letters soon ;-)

Do you handle import as well as export? One of the more interesting use
cases (I think) is handling folks who want to migrate from an xm/xl
based setup to a xapi setup (e.g. by installing Kronos on their existing
Debian system). That's was the primary aim of the proposed project.

> [root@st20 ~]# cat win7.xm 
> name='win7'
> builder='hvmloader'
> boot='dc'
> vcpus=1
> memory=2048
> disk=[ 'sm:7af570d8-f8c5-4103-ac1d-969fe28bfc11,hda,w', 'sm:137c8a61-113c-ab46-20fa-5c0574eaff77,hdb:cdrom,r' ]

Half-assed wondering -- I wonder if sm: (or script=sm or similar)
support could work in xl...

> vif=[  ]
> pci=[  ]
> pci_msitranslate=1
> pci_power_mgmt=0
> # transient=true
> 
> Another goal of the refactoring is to allow xapi to co-exist with domains
> created by someone else (e.g. xl/libxl). This should allow a migration to
> be done piecemeal, one VM at a time on the same host.

The brainstoming list below includes "make xapi use libxl". Is this (or
a subset of this) the sort of thing which could be done by a GSoC
student?

I suppose it is only fair that I offer to be co-/backup-mentor to a main
mentor from the xapi side of things for such a project...

Ian.


> 
> Cheers,
> Dave
> 
> > 
> > Ian.
> > 
> > > TOOLS
> > > -----
> > > - pv grub2
> > > - xapi support in libvirt
> > > - make xapi use libxl
> > > - compile xapi on ARM
> > > - OpenXenManager
> > > - driver domains
> > > - PV dbus
> > > - HA daemon for Remus
> > > - HA daemon for XCP
> > >
> > > PERF
> > > ----
> > > - Oprofile
> > > - Linux perf tools in guest
> > >
> > > HYPERVISOR
> > > ----------
> > > - insmod Xen
> > > - event channel limits
> > > - NUMA
> > >
> > > MEMORY
> > > ------
> > > - disk based memory sharing
> > > - memory scanner
> > > - paging replacing PoD
> > > - VM Fork
> > > - Copy on read (past VDI boot)
> > >
> > >
> > > IO
> > > --
> > > - PV OpenGL/Gallium
> > > - PV USB
> > > - PV USB3
> > > - PV SCSI
> > > - PVFB in Xorg
> > > - Infiniband
> > >
> > >
> > > STORAGE
> > > -------
> > > - gluster/ceph plugins
> > >
> > >
> > > QEMU
> > > ----
> > > - BSD libc
> > > - upstream QEMU stubdoms
> > >
> > >
> > > TESTS
> > > -----
> > > - better web reporting
> > > - more tests
> > > - upstream linux
> > >
> > >
> > > DISTROS
> > > -------
> > > - packaging stubdoms
> > > - xen on centos6
> > > - driver domains
> > > - figure out the VM format issue
> > > - XSM in distros
> > >
> > >
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> > > http://lists.xen.org/xen-devel
> > 
> > 
> > 
> > _______________________________________________
> > xen-api mailing list
> > xen-api-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Re: [Xen-devel] GSoC 2012 project brainstorming

[Dave Scott]

Ian Campbell wrote:
> On Wed, 2012-03-14 at 17:55 +0000, Dave Scott wrote:
> > I've been busy recently moving all the domain management parts of
> xapi
> > into a separate daemon, which later can be libxl'ed up. For testing
> > (and for fun) I've made it understand some of xm/xl config file
> syntax eg
> 
> Interesting. Pure ocaml I presume?

Yep.

> > [root@st20 ~]# xn export-metadata-xm win7 win7.xm
> 
> xn? We're going to run out of letters soon ;-)

:-)

> Do you handle import as well as export? One of the more interesting use
> cases (I think) is handling folks who want to migrate from an xm/xl
> based setup to a xapi setup (e.g. by installing Kronos on their
> existing
> Debian system). That's was the primary aim of the proposed project.

IIRC it can import simple things, but it's quite incomplete. If the
goal is to migrate from an xm/xl setup to xapi then it probably makes
more sense to use the existing (and by definition correct) xl config
parser and then talk the XenAPI directly. Or emit a xapi "metadata export".

One of the interesting areas will be storage...

> > [root@st20 ~]# cat win7.xm
> > name='win7'
> > builder='hvmloader'
> > boot='dc'
> > vcpus=1
> > memory=2048
> > disk=[ 'sm:7af570d8-f8c5-4103-ac1d-969fe28bfc11,hda,w', 'sm:137c8a61-
> 113c-ab46-20fa-5c0574eaff77,hdb:cdrom,r' ]
> 
> Half-assed wondering -- I wonder if sm: (or script=sm or similar)
> support could work in xl...

Yeah I've been wondering that too. As well as tidying up the domain
handling code in xapi I've also been trying to generate docs for the
xapi <-> storage interface (aka "SMAPI"). The current version is here:

http://dave.recoil.org/xen/storage.html

I'd also like to make the current storage plugins run standalone (currently
they require xapi to be running). If we did that then we could potentially
add support for XCP storage types directly into libxl (or the hotplug scripts).

As well as generate docs for the SMAPI I can also generate python skeleton
code, to make it easier to write storage plugins. A custom one of these
might make it easier to migrate from xm/xl to xapi too, by leaving the
disks where they are, rather than moving them into a regular SR.

> 
> > vif=[  ]
> > pci=[  ]
> > pci_msitranslate=1
> > pci_power_mgmt=0
> > # transient=true
> >
> > Another goal of the refactoring is to allow xapi to co-exist with
> domains
> > created by someone else (e.g. xl/libxl). This should allow a
> migration to
> > be done piecemeal, one VM at a time on the same host.
> 
> The brainstoming list below includes "make xapi use libxl". Is this (or
> a subset of this) the sort of thing which could be done by a GSoC
> student?

I think a subset could probably be done in the GSoC timeframe. Before the
summer starts I should have merged my refactoring into the xapi mainline.
A student could then fire up the ocaml libxl bindings (they might need a bit
of tweaking here or there) and then start patching things through. All the
critical libxc code is now called (indirectly) via a single module:

https://github.com/djs55/xen-api/blob/cooper/ocaml/xenops/xenops_server_xen.ml

If that were made to use libxl then the job would be basically done. All that
would be left would be little things like statistics gathering.


> 
> I suppose it is only fair that I offer to be co-/backup-mentor to a
> main
> mentor from the xapi side of things for such a project...

Great :-)

Cheers,
Dave

> 
> Ian.
> 
> 
> >
> > Cheers,
> > Dave
> >
> > >
> > > Ian.
> > >
> > > > TOOLS
> > > > -----
> > > > - pv grub2
> > > > - xapi support in libvirt
> > > > - make xapi use libxl
> > > > - compile xapi on ARM
> > > > - OpenXenManager
> > > > - driver domains
> > > > - PV dbus
> > > > - HA daemon for Remus
> > > > - HA daemon for XCP
> > > >
> > > > PERF
> > > > ----
> > > > - Oprofile
> > > > - Linux perf tools in guest
> > > >
> > > > HYPERVISOR
> > > > ----------
> > > > - insmod Xen
> > > > - event channel limits
> > > > - NUMA
> > > >
> > > > MEMORY
> > > > ------
> > > > - disk based memory sharing
> > > > - memory scanner
> > > > - paging replacing PoD
> > > > - VM Fork
> > > > - Copy on read (past VDI boot)
> > > >
> > > >
> > > > IO
> > > > --
> > > > - PV OpenGL/Gallium
> > > > - PV USB
> > > > - PV USB3
> > > > - PV SCSI
> > > > - PVFB in Xorg
> > > > - Infiniband
> > > >
> > > >
> > > > STORAGE
> > > > -------
> > > > - gluster/ceph plugins
> > > >
> > > >
> > > > QEMU
> > > > ----
> > > > - BSD libc
> > > > - upstream QEMU stubdoms
> > > >
> > > >
> > > > TESTS
> > > > -----
> > > > - better web reporting
> > > > - more tests
> > > > - upstream linux
> > > >
> > > >
> > > > DISTROS
> > > > -------
> > > > - packaging stubdoms
> > > > - xen on centos6
> > > > - driver domains
> > > > - figure out the VM format issue
> > > > - XSM in distros
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> > > > http://lists.xen.org/xen-devel
> > >
> > >
> > >
> > > _______________________________________________
> > > xen-api mailing list
> > > xen-api-GuqFBffKawuEi8DpZVb4nw@public.gmane.org
> > > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
> 

Re: [Xen-devel] GSoC 2012 project brainstorming

[Ian Campbell]

On Thu, 2012-03-15 at 10:02 +0000, Dave Scott wrote:
> Ian Campbell wrote:
> > On Wed, 2012-03-14 at 17:55 +0000, Dave Scott wrote:
> > Do you handle import as well as export? One of the more interesting use
> > cases (I think) is handling folks who want to migrate from an xm/xl
> > based setup to a xapi setup (e.g. by installing Kronos on their
> > existing
> > Debian system). That's was the primary aim of the proposed project.
> 
> IIRC it can import simple things, but it's quite incomplete. If the
> goal is to migrate from an xm/xl setup to xapi then it probably makes
> more sense to use the existing (and by definition correct) xl config
> parser and then talk the XenAPI directly.

That was my line of thinking.

>  Or emit a xapi "metadata export".

Hadn't considered this one -- how well specified is that format?

Another thing I'd wondered about was the ability to consume/exhume OVA
(or is it OVF?) thing.

> One of the interesting areas will be storage...
> 
> > > [root@st20 ~]# cat win7.xm
> > > name='win7'
> > > builder='hvmloader'
> > > boot='dc'
> > > vcpus=1
> > > memory=2048
> > > disk=[ 'sm:7af570d8-f8c5-4103-ac1d-969fe28bfc11,hda,w', 'sm:137c8a61-
> > 113c-ab46-20fa-5c0574eaff77,hdb:cdrom,r' ]
> > 
> > Half-assed wondering -- I wonder if sm: (or script=sm or similar)
> > support could work in xl...
> 
> Yeah I've been wondering that too. As well as tidying up the domain
> handling code in xapi I've also been trying to generate docs for the
> xapi <-> storage interface (aka "SMAPI"). The current version is here:
> 
> http://dave.recoil.org/xen/storage.html
> 
> I'd also like to make the current storage plugins run standalone (currently
> they require xapi to be running). If we did that then we could potentially
> add support for XCP storage types directly into libxl (or the hotplug scripts).
> 
> As well as generate docs for the SMAPI I can also generate python skeleton
> code, to make it easier to write storage plugins. A custom one of these
> might make it easier to migrate from xm/xl to xapi too, by leaving the
> disks where they are, rather than moving them into a regular SR.

That could be a good plan. Given such an SR plugin could you then do
some sort of "xe vdi-move" to move a VDI from that plugin to one of the
"standard" ones?

> 
> > 
> > > vif=[  ]
> > > pci=[  ]
> > > pci_msitranslate=1
> > > pci_power_mgmt=0
> > > # transient=true
> > >
> > > Another goal of the refactoring is to allow xapi to co-exist with
> > domains
> > > created by someone else (e.g. xl/libxl). This should allow a
> > migration to
> > > be done piecemeal, one VM at a time on the same host.
> > 
> > The brainstoming list below includes "make xapi use libxl". Is this (or
> > a subset of this) the sort of thing which could be done by a GSoC
> > student?
> 
> I think a subset could probably be done in the GSoC timeframe. Before the
> summer starts I should have merged my refactoring into the xapi mainline.
> A student could then fire up the ocaml libxl bindings (they might need a bit
> of tweaking here or there) and then start patching things through. All the
> critical libxc code is now called (indirectly) via a single module:
> 
> https://github.com/djs55/xen-api/blob/cooper/ocaml/xenops/xenops_server_xen.ml

That's a surprisingly (in a good way) small amount of code!

> If that were made to use libxl then the job would be basically done. All that
> would be left would be little things like statistics gathering.
> 
> 
> > 
> > I suppose it is only fair that I offer to be co-/backup-mentor to a
> > main
> > mentor from the xapi side of things for such a project...
> 
> Great :-)

Was that the sound of you offering to be (or to find a) main mentor ;-)

Ian.

Re: [Xen-devel] GSoC 2012 project brainstorming

[Stefano Stabellini]

On Thu, 15 Mar 2012, Dave Scott wrote:
> > Do you handle import as well as export? One of the more interesting use
> > cases (I think) is handling folks who want to migrate from an xm/xl
> > based setup to a xapi setup (e.g. by installing Kronos on their
> > existing
> > Debian system). That's was the primary aim of the proposed project.
> 
> IIRC it can import simple things, but it's quite incomplete. If the
> goal is to migrate from an xm/xl setup to xapi then it probably makes
> more sense to use the existing (and by definition correct) xl config
> parser and then talk the XenAPI directly. Or emit a xapi "metadata export".

Maybe we could interface XAPI with xl via JSON?
We were considering a machine readable input/output system for xl
anyway...
This way you could reuse all the existing code in xl as well as libxl
and have XAPI being a drop in addon on top of xl.
At that point xn could be the glue between xl and XAPI via JSON and
XenAPI.


> I'd also like to make the current storage plugins run standalone (currently
> they require xapi to be running). If we did that then we could potentially
> add support for XCP storage types directly into libxl (or the hotplug scripts).
>
> As well as generate docs for the SMAPI I can also generate python skeleton
> code, to make it easier to write storage plugins. A custom one of these
> might make it easier to migrate from xm/xl to xapi too, by leaving the
> disks where they are, rather than moving them into a regular SR.

That would certainly make a lot of sense.
It would be great if the XCP storage plugins could be used from libxl,
so maybe XAPI could actually use libxl to setup storage too.
We are currently rewriting the interface between libxl and the hotplug
scripts, introducing the concept of a xenbackend daemon that takes care
of calling these scripts and setup the xenstore backend entries.
Alternatively this could be another place where the SM could hook into.