[FW: FYI: The plan for Xen kernels in Fedora 9]

[FW: FYI: The plan for Xen kernels in Fedora 9]

[Daniel P. Berrange]

In the spirit of improving communication of Fedora Xen plans to the world,
below is a mail I recently circulated in the Fedora community about the 
direction for Xen-ified kernels from Fedora 9 and onwards. 

The short story, is that we intend to ship hypervisor & userspace based on
Xen 3.2.0 tree, and a kernel based on latest LKML pv_ops tree and patches
ontop of that to support Dom0 and x86_64. With some short term pain and
instability, we hope to get significant long term benefits for support of
Xen Linux kernels.

The long story is the mail below..

We have a number of kernel guys working on this project, and Stephen will
shortly followup with details of his current patch queue, for benefit of 
anyone else who wishes to track this / get involved.

Regards,
Dan.

----- Forwarded message from "Daniel P. Berrange" <berrange@redhat.com> -----

> Date: Fri, 30 Nov 2007 18:59:09 +0000
> From: "Daniel P. Berrange" <berrange@redhat.com>
> To: fedora-xen@redhat.com
> Subject: FYI: The plan for Xen kernels in Fedora 9
> 
> This is a friendly alert of the major plans we have for Xen kernels in 
> Fedora 9 timeframe...
> 
> Since we first added Xen in Fedora Core 5, our kernels have been based on
> a forward-port of XenSource's upstream Xen kernels, to new LKML. For a
> long time we ported their 2.6.16 tree to 2.6.18. Now we do ports of their
> 2.6.18 tree to 2.6.21/22/23, etc.  At the same time, upstream Linux gained
> Xen support for i386 DomU, and shortly x86_64  DomU, and is generally
> getting ever more virtualization capabilities.
> 
> As everyone knows, we have tended to lag behind Fedora's state-of-the-art 
> bare metal kernels by several releases due to the effort required to port
> Xen to newer LKML releases. Despite our best efforts, this lag has been 
> getting worse, not better.
> 
> We have taken the decision, that this situation is unacceptable for Fedora 9.
> We simply cannot spend more time forward porting Xen kernels. Either Xen has
> to be dropped entirely, or we need a different strategy for dealing with the
> kernels. Since people seeem to use Xen, we have decided not to drop it :-)
> 
> So the plan is to re-focus 100% of all Xen kernel efforts onto paravirt_ops.
> LKML already has i386 pv_ops + Xen DomU. We intend to build on this to
> add:
> 
>   - x64_64 pv_ops
>   - x86_64 Xen DomU on pv_ops
>   - i386 & x86_64  Xen Dom0  on pv_ops
>   - memory balloon
>   - paravirt framebuffer
>   - save/restore
> 
> All of this based on same LKML release as Fedora bare metal. If all goes to
> plan it may even be in the base kernel RPM, instead of kernel-xen, but thats
> a minor concern compared to the actual coding.
> 
> Getting all this done for Fedora 9 is seriously ambitious, but it is the only
> long term sustainable option, other than dropping Xen entirely.
> 
> What this means though, is that Fedora 9 Xen will certainly be going through
> periods of instability and will certainly be even buggier than normal. F9
> may well end up lacking features compared to Xen in Fedora 8 & earlier (eg no
> PCI device passthrough, or CPU hotplug). On the plus side though we will be
> 100% back in sync with bare metal kernel versions & hopefully even have a
> lot of this stuff merged in LKML to make ongoing maintainence sustainable.
> Short term pain; Long term gain!
> 
> I have not got any ETA on when any of these kernel changes will appear in
> rawhide - some time before the F9 feature freeze date is best guesstimate.
> We will alert people when the time comes. There is a F9 feature page
> with some amount of info about the plan...
> 
>    http://fedoraproject.org/wiki/Features/XenPvops
> 
> In terms of Fedora 6/7/8 maintainence... The kernel-xen in these existing 
> releases already lags behind the bare metal kernel version by 2-3 releases.
> We do not intend to continue trying to rebase the kernel-xen in existing
> Fedora releases. It will be essentially important bug-fix mode only. This
> is neccessary to enable maximum resources to be focused on the critical
> Fedora 9 Xen work.
> 
> Regards,
> Dan ...on behalf of some very busy Fedora Xen kernel developers :-)
> -- 
> |=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
> |=-           Perl modules: http://search.cpan.org/~danberr/              -=|
> |=-               Projects: http://freshmeat.net/~danielpb/               -=|
> |=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 
> 
----- End forwarded message -----

-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Keir Fraser]

This sounds like an excellent plan to me!

 -- Keir

On 10/12/07 15:20, "Daniel P. Berrange" <berrange@redhat.com> wrote:

> In the spirit of improving communication of Fedora Xen plans to the world,
> below is a mail I recently circulated in the Fedora community about the
> direction for Xen-ified kernels from Fedora 9 and onwards.
> 
> The short story, is that we intend to ship hypervisor & userspace based on
> Xen 3.2.0 tree, and a kernel based on latest LKML pv_ops tree and patches
> ontop of that to support Dom0 and x86_64. With some short term pain and
> instability, we hope to get significant long term benefits for support of
> Xen Linux kernels.
> 
> The long story is the mail below..
> 
> We have a number of kernel guys working on this project, and Stephen will
> shortly followup with details of his current patch queue, for benefit of
> anyone else who wishes to track this / get involved.
> 
> Regards,
> Dan.
> 
> ----- Forwarded message from "Daniel P. Berrange" <berrange@redhat.com> -----
> 
>> Date: Fri, 30 Nov 2007 18:59:09 +0000
>> From: "Daniel P. Berrange" <berrange@redhat.com>
>> To: fedora-xen@redhat.com
>> Subject: FYI: The plan for Xen kernels in Fedora 9
>> 
>> This is a friendly alert of the major plans we have for Xen kernels in
>> Fedora 9 timeframe...
>> 
>> Since we first added Xen in Fedora Core 5, our kernels have been based on
>> a forward-port of XenSource's upstream Xen kernels, to new LKML. For a
>> long time we ported their 2.6.16 tree to 2.6.18. Now we do ports of their
>> 2.6.18 tree to 2.6.21/22/23, etc.  At the same time, upstream Linux gained
>> Xen support for i386 DomU, and shortly x86_64  DomU, and is generally
>> getting ever more virtualization capabilities.
>> 
>> As everyone knows, we have tended to lag behind Fedora's state-of-the-art
>> bare metal kernels by several releases due to the effort required to port
>> Xen to newer LKML releases. Despite our best efforts, this lag has been
>> getting worse, not better.
>> 
>> We have taken the decision, that this situation is unacceptable for Fedora 9.
>> We simply cannot spend more time forward porting Xen kernels. Either Xen has
>> to be dropped entirely, or we need a different strategy for dealing with the
>> kernels. Since people seeem to use Xen, we have decided not to drop it :-)
>> 
>> So the plan is to re-focus 100% of all Xen kernel efforts onto paravirt_ops.
>> LKML already has i386 pv_ops + Xen DomU. We intend to build on this to
>> add:
>> 
>>   - x64_64 pv_ops
>>   - x86_64 Xen DomU on pv_ops
>>   - i386 & x86_64  Xen Dom0  on pv_ops
>>   - memory balloon
>>   - paravirt framebuffer
>>   - save/restore
>> 
>> All of this based on same LKML release as Fedora bare metal. If all goes to
>> plan it may even be in the base kernel RPM, instead of kernel-xen, but thats
>> a minor concern compared to the actual coding.
>> 
>> Getting all this done for Fedora 9 is seriously ambitious, but it is the only
>> long term sustainable option, other than dropping Xen entirely.
>> 
>> What this means though, is that Fedora 9 Xen will certainly be going through
>> periods of instability and will certainly be even buggier than normal. F9
>> may well end up lacking features compared to Xen in Fedora 8 & earlier (eg no
>> PCI device passthrough, or CPU hotplug). On the plus side though we will be
>> 100% back in sync with bare metal kernel versions & hopefully even have a
>> lot of this stuff merged in LKML to make ongoing maintainence sustainable.
>> Short term pain; Long term gain!
>> 
>> I have not got any ETA on when any of these kernel changes will appear in
>> rawhide - some time before the F9 feature freeze date is best guesstimate.
>> We will alert people when the time comes. There is a F9 feature page
>> with some amount of info about the plan...
>> 
>>    http://fedoraproject.org/wiki/Features/XenPvops
>> 
>> In terms of Fedora 6/7/8 maintainence... The kernel-xen in these existing
>> releases already lags behind the bare metal kernel version by 2-3 releases.
>> We do not intend to continue trying to rebase the kernel-xen in existing
>> Fedora releases. It will be essentially important bug-fix mode only. This
>> is neccessary to enable maximum resources to be focused on the critical
>> Fedora 9 Xen work.
>> 
>> Regards,
>> Dan ...on behalf of some very busy Fedora Xen kernel developers :-)
>> -- 
>> |=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
>> |=-           Perl modules: http://search.cpan.org/~danberr/              -=|
>> |=-               Projects: http://freshmeat.net/~danielpb/               -=|
>> |=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|
>> 
> ----- End forwarded message -----

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Mon, 2007-12-10 at 15:20 +0000, Daniel P. Berrange wrote:

> We have a number of kernel guys working on this project, and Stephen will
> shortly followup with details of his current patch queue, for benefit of 
> anyone else who wishes to track this / get involved.

I just tidied up my current tree, and I've pushed patches to 

	http://people.redhat.com/sct/patches/dom0-pvops/2.6.24-rc4-A0/

These include Jeremy Fitzhardinge's existing dom0 work (except for the
chunk that was a start towards adding the ELF note stuff to vmlinux,
I'll add that back once it's at least compiling properly.)

So far everything is done within the pvops framework, so there's nothing
added that stops a single kernel image running both baremetal and as
dom0.  New since Jeremy's patches are a basic mtrr plugin (not full
runtime mtrr, but enough to satisfy SMP boot), a bit of ACPI tweaking
and ioremap pv_ops.

Juan Quintela and I are working on IRQ routing next, as that's the
current big stumbling block during dom0 boot.

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Mon, 2007-12-10 at 17:06 +0000, Stephen C. Tweedie wrote:

> I just tidied up my current tree, and I've pushed patches to 
> 
> 	http://people.redhat.com/sct/patches/dom0-pvops/2.6.24-rc4-A0/

btw, I'm maintaining this myself as a local git repository, with two
active branches: one working branch which I'll be rebasing regularly to
keep it synced up and patched against Linus' latest tree, and which will
be getting commits reordered and merged to keep things as a sane linear
patch queue against Linus's tree; and a full-history branch, which
contains all the history of the rebases for sanity checking.

This setup works extremely well for me, but I'm not sure how useful it
is for other folks.  Because it is regularly rebased, the active branch
isn't useful as a git merge source; and because it keeps all the
historical cruft that is being pruned from the active branch, the
history branch isn't all that useful as a development base.

But I'll push these to a public git repo if anybody is interested in
using the tree in that form.  (Just as long as I can put up a big hazard
warning about the dangers of using a git tree that loses history!)

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Chuck Short]

I would appreciate the git tree and Im pretty sure that members of the
Ubuntu community would as well.

Thanks
chuck

On Dec 10, 2007 12:38 PM, Stephen C. Tweedie <sct@redhat.com> wrote:
> Hi,
>
> On Mon, 2007-12-10 at 17:06 +0000, Stephen C. Tweedie wrote:
>
> > I just tidied up my current tree, and I've pushed patches to
> >
> >       http://people.redhat.com/sct/patches/dom0-pvops/2.6.24-rc4-A0/
>
> btw, I'm maintaining this myself as a local git repository, with two
> active branches: one working branch which I'll be rebasing regularly to
> keep it synced up and patched against Linus' latest tree, and which will
> be getting commits reordered and merged to keep things as a sane linear
> patch queue against Linus's tree; and a full-history branch, which
> contains all the history of the rebases for sanity checking.
>
> This setup works extremely well for me, but I'm not sure how useful it
> is for other folks.  Because it is regularly rebased, the active branch
> isn't useful as a git merge source; and because it keeps all the
> historical cruft that is being pruned from the active branch, the
> history branch isn't all that useful as a development base.
>
> But I'll push these to a public git repo if anybody is interested in
> using the tree in that form.  (Just as long as I can put up a big hazard
> warning about the dangers of using a git tree that loses history!)
>
>
> --Stephen
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Mon, 2007-12-10 at 19:31 -0500, Chuck Short wrote:

> I would appreciate the git tree and Im pretty sure that members of the
> Ubuntu community would as well.

OK, although the fact that the tree is getting rebased means that it
will not behave like a traditional git tree in some ways.

I'll probably reorganise the tree for public consumption, with the
master branch just being Linus's main kernel HEAD, and the patch queues
coming off that as separate branches.  That way we won't run into
trouble with repeated pulls; and the full latest version is always going
to be available on the full-history branch anyway (albeit with all the
historical baggage on that branch.)

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Stephen C. Tweedie wrote:
> I just tidied up my current tree, and I've pushed patches to 
>
> 	http://people.redhat.com/sct/patches/dom0-pvops/2.6.24-rc4-A0/
>
> These include Jeremy Fitzhardinge's existing dom0 work (except for the
> chunk that was a start towards adding the ELF note stuff to vmlinux,
> I'll add that back once it's at least compiling properly.)
>
> So far everything is done within the pvops framework, so there's nothing
> added that stops a single kernel image running both baremetal and as
> dom0.  New since Jeremy's patches are a basic mtrr plugin (not full
> runtime mtrr, but enough to satisfy SMP boot), a bit of ACPI tweaking
> and ioremap pv_ops.

I just had a quick look through this, and it looks good to me.  One
thing though: I'm wondering if we shouldn't have CONFIG_XEN_DOM0 protect
this stuff, so that its possible to build a domU-only kernel.

    J

RE: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Tian, Kevin]

>From: Jeremy Fitzhardinge
>Sent: 2007年12月11日 8:37
>
>Stephen C. Tweedie wrote:
>> I just tidied up my current tree, and I've pushed patches to 
>>
>> 	http://people.redhat.com/sct/patches/dom0-pvops/2.6.24-rc4-A0/
>>
>> These include Jeremy Fitzhardinge's existing dom0 work 
>(except for the
>> chunk that was a start towards adding the ELF note stuff to vmlinux,
>> I'll add that back once it's at least compiling properly.)
>>
>> So far everything is done within the pvops framework, so 
>there's nothing
>> added that stops a single kernel image running both baremetal and as
>> dom0.  New since Jeremy's patches are a basic mtrr plugin (not full
>> runtime mtrr, but enough to satisfy SMP boot), a bit of ACPI tweaking
>> and ioremap pv_ops.
>
>I just had a quick look through this, and it looks good to me.  One
>thing though: I'm wondering if we shouldn't have 
>CONFIG_XEN_DOM0 protect
>this stuff, so that its possible to build a domU-only kernel.
>
>    J

Or take a neutral one like CONFIG_XEN_PRIV?

Thanks,
Kevin

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Tian, Kevin wrote:
> Or take a neutral one like CONFIG_XEN_PRIV?

Why is that an improvement? I could see a finer grain config to do
driver domains, perhaps, but it doesn't seem worthwhile at this point.
But it might be useful to have some marker of which code is necessary
for Dom0 functionality, and which is core Xen support.

J

RE: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Tian, Kevin]

>From: Jeremy Fitzhardinge [mailto:jeremy@goop.org] 
>Sent: 2007年12月11日 9:36
>
>Tian, Kevin wrote:
>> Or take a neutral one like CONFIG_XEN_PRIV?
>
>Why is that an improvement? I could see a finer grain config to do
>driver domains, perhaps, but it doesn't seem worthwhile at this point.
>But it might be useful to have some marker of which code is necessary
>for Dom0 functionality, and which is core Xen support.
>
>J
>

En.. yes, finer grain config is better.

Thanks,
Kevin

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Mon, 2007-12-10 at 16:36 -0800, Jeremy Fitzhardinge wrote:

> I just had a quick look through this, and it looks good to me.  One
> thing though: I'm wondering if we shouldn't have CONFIG_XEN_DOM0 protect
> this stuff, so that its possible to build a domU-only kernel.

Yep, I did wonder about that.

But... there's actually quite a lot that isn't really dom0-specific.
Rather, it's IO-domain-specific.  Configure a PV guest with PCI
passthrough and you'd want much of the same functionality in it.  

Also, adding CONFIG entries just increases the size of the source right
now.  I certainly think it's worth having eventually, but for now I'm
aiming for minimal invasiveness, so I haven't bothered with domU-only
configs.

If people think it's important I can add them sooner rather than later,
of course.

Cheers,
 Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Stephen C. Tweedie wrote:
> Hi,
>
> On Mon, 2007-12-10 at 16:36 -0800, Jeremy Fitzhardinge wrote:
>
>   
>> I just had a quick look through this, and it looks good to me.  One
>> thing though: I'm wondering if we shouldn't have CONFIG_XEN_DOM0 protect
>> this stuff, so that its possible to build a domU-only kernel.
>>     
>
> Yep, I did wonder about that.
>
> But... there's actually quite a lot that isn't really dom0-specific.
> Rather, it's IO-domain-specific.  Configure a PV guest with PCI
> passthrough and you'd want much of the same functionality in it.  
>
> Also, adding CONFIG entries just increases the size of the source right
> now.  I certainly think it's worth having eventually, but for now I'm
> aiming for minimal invasiveness, so I haven't bothered with domU-only
> configs.
>
> If people think it's important I can add them sooner rather than later,
> of course.

I think they're useful as documentation, so you can tell whether a piece
of code is dom0/io-specific vs generic.  On the other hand, #ifdefs are
undesirable, and more config options just means more combinatorial build
testing.

So put me down as uselessly indecisive on this one.

    J

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Tue, 2007-12-11 at 08:57 -0800, Jeremy Fitzhardinge wrote:

> > Also, adding CONFIG entries just increases the size of the source right
> > now.  I certainly think it's worth having eventually, but for now I'm
> > aiming for minimal invasiveness, so I haven't bothered with domU-only
> > configs.

> I think they're useful as documentation, so you can tell whether a piece
> of code is dom0/io-specific vs generic.  On the other hand, #ifdefs are
> undesirable, and more config options just means more combinatorial build
> testing.

> So put me down as uselessly indecisive on this one.

:-)  Well, we can wait and see if anybody complains; if the config
options are actually useful to people in real life, then that's worth
knowing.  Certainly, though, all the Red Hat and Fedora kernels just
compile with as much enabled as possible --- it's unnecessary complexity
to build separate dom0 and domU kernels.

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi all,

On Mon, 2007-12-10 at 15:20 +0000, Daniel P. Berrange wrote:

> > So the plan is to re-focus 100% of all Xen kernel efforts onto paravirt_ops.
> > LKML already has i386 pv_ops + Xen DomU. We intend to build on this to
> > add:
> > 
> >   - x64_64 pv_ops
> >   - x86_64 Xen DomU on pv_ops
> >   - i386 & x86_64  Xen Dom0  on pv_ops
> >   - memory balloon
> >   - paravirt framebuffer
> >   - save/restore

There are a few other bits and pieces missing in the current pv_ops
code:

 CPU hotplug
 ia64 support (this has already been brought up on xen-ia64-devel)
 kexec
 xenoprof

but the fact that we've got a functional (albeit 32-bit-only)
implementation already upstream to work against is a great help in
getting some of these efforts going on in parallel.

64-bit support and functional dom0 are definitely our top priorities
right now for Fedora.

At some point, the Xen community is going to have to think about turning
off the air supply to the old 2.6.18-based tree. :)  

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Daniel P. Berrange wrote:
> The short story, is that we intend to ship hypervisor & userspace based on
> Xen 3.2.0 tree, and a kernel based on latest LKML pv_ops tree and patches
> ontop of that to support Dom0 and x86_64. With some short term pain and
> instability, we hope to get significant long term benefits for support of
> Xen Linux kernels.
>   

Excellent, excellent news.

    J

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Mark Williamson]

> In the spirit of improving communication of Fedora Xen plans to the world,
> below is a mail I recently circulated in the Fedora community about the
> direction for Xen-ified kernels from Fedora 9 and onwards.
>
> The short story, is that we intend to ship hypervisor & userspace based on
> Xen 3.2.0 tree, and a kernel based on latest LKML pv_ops tree and patches
> ontop of that to support Dom0 and x86_64. With some short term pain and
> instability, we hope to get significant long term benefits for support of
> Xen Linux kernels.

Sounds like the best possible and longterm-sustainable plan and good for 
everybody involved.

Question: do the dom0-compatibility patches have any chance of getting into 
kernel.org?  Or would they continue to live as a patchset?

Cheers,
Mark

> The long story is the mail below..
>
> We have a number of kernel guys working on this project, and Stephen will
> shortly followup with details of his current patch queue, for benefit of
> anyone else who wishes to track this / get involved.
>
> Regards,
> Dan.
>
> ----- Forwarded message from "Daniel P. Berrange" <berrange@redhat.com>
> -----
>
> > Date: Fri, 30 Nov 2007 18:59:09 +0000
> > From: "Daniel P. Berrange" <berrange@redhat.com>
> > To: fedora-xen@redhat.com
> > Subject: FYI: The plan for Xen kernels in Fedora 9
> >
> > This is a friendly alert of the major plans we have for Xen kernels in
> > Fedora 9 timeframe...
> >
> > Since we first added Xen in Fedora Core 5, our kernels have been based on
> > a forward-port of XenSource's upstream Xen kernels, to new LKML. For a
> > long time we ported their 2.6.16 tree to 2.6.18. Now we do ports of their
> > 2.6.18 tree to 2.6.21/22/23, etc.  At the same time, upstream Linux
> > gained Xen support for i386 DomU, and shortly x86_64  DomU, and is
> > generally getting ever more virtualization capabilities.
> >
> > As everyone knows, we have tended to lag behind Fedora's state-of-the-art
> > bare metal kernels by several releases due to the effort required to port
> > Xen to newer LKML releases. Despite our best efforts, this lag has been
> > getting worse, not better.
> >
> > We have taken the decision, that this situation is unacceptable for
> > Fedora 9. We simply cannot spend more time forward porting Xen kernels.
> > Either Xen has to be dropped entirely, or we need a different strategy
> > for dealing with the kernels. Since people seeem to use Xen, we have
> > decided not to drop it :-)
> >
> > So the plan is to re-focus 100% of all Xen kernel efforts onto
> > paravirt_ops. LKML already has i386 pv_ops + Xen DomU. We intend to build
> > on this to add:
> >
> >   - x64_64 pv_ops
> >   - x86_64 Xen DomU on pv_ops
> >   - i386 & x86_64  Xen Dom0  on pv_ops
> >   - memory balloon
> >   - paravirt framebuffer
> >   - save/restore
> >
> > All of this based on same LKML release as Fedora bare metal. If all goes
> > to plan it may even be in the base kernel RPM, instead of kernel-xen, but
> > thats a minor concern compared to the actual coding.
> >
> > Getting all this done for Fedora 9 is seriously ambitious, but it is the
> > only long term sustainable option, other than dropping Xen entirely.
> >
> > What this means though, is that Fedora 9 Xen will certainly be going
> > through periods of instability and will certainly be even buggier than
> > normal. F9 may well end up lacking features compared to Xen in Fedora 8 &
> > earlier (eg no PCI device passthrough, or CPU hotplug). On the plus side
> > though we will be 100% back in sync with bare metal kernel versions &
> > hopefully even have a lot of this stuff merged in LKML to make ongoing
> > maintainence sustainable. Short term pain; Long term gain!
> >
> > I have not got any ETA on when any of these kernel changes will appear in
> > rawhide - some time before the F9 feature freeze date is best
> > guesstimate. We will alert people when the time comes. There is a F9
> > feature page with some amount of info about the plan...
> >
> >    http://fedoraproject.org/wiki/Features/XenPvops
> >
> > In terms of Fedora 6/7/8 maintainence... The kernel-xen in these existing
> > releases already lags behind the bare metal kernel version by 2-3
> > releases. We do not intend to continue trying to rebase the kernel-xen in
> > existing Fedora releases. It will be essentially important bug-fix mode
> > only. This is neccessary to enable maximum resources to be focused on the
> > critical Fedora 9 Xen work.
> >
> > Regards,
> > Dan ...on behalf of some very busy Fedora Xen kernel developers :-)
> > --
> >
> > |=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496
> > | -=| =-           Perl modules: http://search.cpan.org/~danberr/        
> > |      -=| =-               Projects: http://freshmeat.net/~danielpb/    
> > |           -=| =-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF
> > | F742 7D3B 9505  -=|
>
> ----- End forwarded message -----



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Mark Williamson wrote:
> Question: do the dom0-compatibility patches have any chance of getting into 
> kernel.org?  Or would they continue to live as a patchset?
>   

I'm actually optimistic we can beat them into an upstreamable state, at
least eventually.  Devil's in the details, of course, but the
pre-existing Xen foothold in the kernel and x86 unification should make
it easier to add interfaces to allow the dom0 stuff to work, so long as
we're careful and exercise good taste.

    J

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Stephen C. Tweedie]

Hi,

On Mon, 2007-12-10 at 13:48 -0800, Jeremy Fitzhardinge wrote:

> I'm actually optimistic we can beat them into an upstreamable state, at
> least eventually.  Devil's in the details, of course, but the
> pre-existing Xen foothold in the kernel and x86 unification should make
> it easier to add interfaces to allow the dom0 stuff to work, so long as
> we're careful and exercise good taste.

Right.  But getting something in shape soon to wean us all off the
2.6.18-xen tree is the priority right now.

As I pull stuff into the dom0 pv-ops tree, I'm being careful to try to
make things as maintainable as possible, and to do nothing that will
make upstreaming harder.  That means no magic *-xen.c copies of
mainstream files, etc.  

But there are definitely places where the right upstream answer isn't
obvious (eg, where mtrr meets pv_ops... both subsystems try to hide
their internals behind an abstraction layer, so we need to break the
abstractions somewhere to let pv_ops install an mtrr back-end.)  In such
cases I'm having to make a decision quickly as to how things will go in
just to get the tree progressing; but we'll have to go back and
potentially rework a lot of that before it's actually upstreamable.

So ... even if we do get everything upstream eventually, it will take a
while (look how long the existing pv_ops took.)  But making the code as
upstreamable as possible pays dividends even while things are still out
of Linus's tree, just by making things more maintainable.  And that's a
BIG bonus.

--Stephen

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Jeremy Fitzhardinge]

Stephen C. Tweedie wrote:
> But there are definitely places where the right upstream answer isn't
> obvious (eg, where mtrr meets pv_ops... both subsystems try to hide
> their internals behind an abstraction layer, so we need to break the
> abstractions somewhere to let pv_ops install an mtrr back-end.)  In such
> cases I'm having to make a decision quickly as to how things will go in
> just to get the tree progressing; but we'll have to go back and
> potentially rework a lot of that before it's actually upstreamable.
>   

Right.  pv_ops is all about adding interfaces where nothing suitable
existed before.  If there's an existing interface which is usable or
nearly usable, we've gone with that rather than extending pv_ops.  The
clocksource/clockevent interfaces are a good example of that.  If we can
hook into the mtrr machinery by registering a pseudo-cpu type, then that
would be neat.

Similarly, I'm hoping that the work on apic unification will give us an
interface we can hook the Xen apic machinery into without too much gross
hackery.

    J

Re: [FW: FYI: The plan for Xen kernels in Fedora 9]

[Daniel P. Berrange]

On Mon, Dec 10, 2007 at 09:38:49PM +0000, Mark Williamson wrote:
> > In the spirit of improving communication of Fedora Xen plans to the world,
> > below is a mail I recently circulated in the Fedora community about the
> > direction for Xen-ified kernels from Fedora 9 and onwards.
> >
> > The short story, is that we intend to ship hypervisor & userspace based on
> > Xen 3.2.0 tree, and a kernel based on latest LKML pv_ops tree and patches
> > ontop of that to support Dom0 and x86_64. With some short term pain and
> > instability, we hope to get significant long term benefits for support of
> > Xen Linux kernels.
> 
> Sounds like the best possible and longterm-sustainable plan and good for 
> everybody involved.
> 
> Question: do the dom0-compatibility patches have any chance of getting into 
> kernel.org?  Or would they continue to live as a patchset?

That's a huge open ended question & you can't take anything for granted when
its comes to LKML review / feedback. I'd like to think they have a reasonable
chance, but its all depends how intrusive the changes become. Even if they
don't get accepted upstream, at least they'll be reconciled against current
accepted  DomU patches. So at worst, we have an incremental improvement in
the maintainability, at best a dramatic improvement.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|