Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[George Dunlap]

* Executive summary

- We've agreed on a "convergence" point for PV shim functionality that
  covers as many users as possible:
 - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
   event channels, &c, booted via 'sidecar'
 - 'PVH' functionality: boots in PVH mode, booted via toolstack
   changes

- "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
  each cover some users and not others; neither one (yet) covers all
  users

- Vixen is ready for immediate release; PVH shim still needs some
  minor revision, and the toolstack component

- Proposal:
 - Release "HOWTO" for Amazon shim v1 immediately (including a signed
   tag on xenbits with the requisite patches)
 - Release "HOWTO" for PVH shim as soon as it's ready (including a
   signed tag or tags on xenbits with the requisite patches)
 - Base future development on the PVH shim (porting over functionality
   from the Amazon series)

* Discussion

In our discussions, we've generally identified four kinds of users /
consumers of Xen:

1. Those for whom upgrading to a newer version of Xen is the biggest
problem

This could be people who have local changes they can't forward-port,
people whose SLAs forbid updating versions, or any number of other
things that make updating to a newer version of Xen unpalatable or
impossible.

2. Those for whom adding a QEMU instance to all PV guests is the
biggest problem

Many users I've talked to extremely dislike the idea of adding QEMU
instances to all their PV guests, and would rather do upgrades and/or
modify guest types to avoid this

3. Those for whom modifying guest parameters is the biggest problem

Many users I've talked to are using software which severely limits the
flexibility they have in terms of how guests are created

4. Those for whom some subset of features (migration, ballooning, vcpu
hotplug, &c) is the biggest problem.

This includes users who rely on these features, as well as software
providers that provide those features.

Amazon is in the first category, and have developed a "PV shim"
series that addresses their needs.  They have also very kindly
shared the patches publicly for the community to be able to use.

Citrix is in the fourth category, and have developed a "PV shim"
series that addresses their needs.  This shim also satisfies category 2,
and with some work could support category 3.

Going forward, the best solutions for new hypervisors (Xen 4.11+)
would be to avoid needing QEMU, building the sidecar, and so on.  But
we still want to be able to support people running the 'HVM shim with
sidecar' version going forward.  So the "convergence point" we've all
seemed to agree on is to have a shim capable of satisfying all groups:

- Can run in HVM mode back as far as Xen 3.4
- Can run in PVH mode, without QEMU or "sidecar", and doesn't require
  many toolstack changes
- Has feature parity with PV mode (migration, ballooning, vcpu
  migration, &c)

The issue has been made public for nearly a week now, so there is a
lot of pressure to get a solution to our users as soon as possible; by
the end of the week at absolute latest, but today if possible.

Amazon's v1 series:
- Has support for 'legacy' event channels

- Has been tested by Amazon over a wide range of their hypervisor
  versions and guest types (including pvgrub types) back as far as Xen
  3.4

- Uses HVM mode, and so has the overhead and security implications of
  running QEMU

- Requires no L0 hypervisor or toolstack changes

- Requires users to build a "boot sidecar" image for each unique guest
  boot configuration

- Is missing many features, such as migration, memory ballooning, and
  vcpu hotplug

- Has accurate 'stolen time' support

Citrix's series:
- Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
  PVH backports) and 4.10
- Uses PVH mode, and so doesn't have the overhead and security
  implications of running QEMU
- No need to build "boot sidecar"
- Requires hypervisor changes for versions for versions before 4.10
  which cannot reasonably be backported by the open-source team beyond
  Xen 4.8
- Requires toolstack changes in all versions
- Has migration, memory ballooning, and vcpu hotplug (extensively
  tested by XenRT)
- Doesn't have accurate 'stolen time' support
- Currently is missing a clean toolstack interface

With some tweaks, Citrix's version has been made to boot in HVM mode.
However, this modified version:
- Doesn't support the 'legacy' event channel mode, and so won't work
  for versions as old as Amazon's
- Hasn't been tested extensively on older versions

Ian suggested that we get something and check it into 4.10-staging
as soon as possible.  But there has been a debate about whether we
should start with Amazon's series and add PVH support, or start with
Citrix's series and add HVM support (including legacy event channels,
and so on).

No matter what, individual users will need to decide whether to take
the 'HVM + sidecar' option or the 'PVH' option..

I'd like to propose a new appraoch:

1. Immediately release Amazon's v1 series for people who can / prefer
to use the HVM + sidecar option.
- Advisory and HOWTO should include who should use this option, and
  how to do it.
- Check the series into a branch on xenbits, and add a signed tag
- HOWTO-vixen, and sidecar script, to be included in the advisory.

2. As soon as possible, release Citrix's series for people who can /
prefer to use the PVH + toolstack option.
- Immediate work should focus on getting PVH + toolstack functionality
  ready to release
- When ready, advisory and HOWTO should include who should use this
  option and how
- Either check the series into a branch on xenbits, or into
  staging-4.10
- HOWTO-pvh to be included in the advisory.

This should allow us to get a solution to the widest number of users
in the shortest amount of time; it also allows us to leverage the
testing efforts of both Amazon (for breadth of Xen versions) and
Citrix (for depth of functionality).

That takes the pressure off us to check in one or the other version of
the patch series.

Going forward, we could work towards the convergence of functionality
from either patch series.  But it looks superficially at least like
the Citrix series is closer to the convergence point, and so it seems
like using that as a starting point would make the most sense.

Regardless of what we think of step 2, I think we should take step 1
immediately.

Let me know what you think.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Jan Beulich]

>>> On 10.01.18 at 14:34, <george.dunlap@citrix.com> wrote:
> I'd like to propose a new appraoch:
> 
> 1. Immediately release Amazon's v1 series for people who can / prefer
> to use the HVM + sidecar option.
> - Advisory and HOWTO should include who should use this option, and
>   how to do it.
> - Check the series into a branch on xenbits, and add a signed tag
> - HOWTO-vixen, and sidecar script, to be included in the advisory.
> 
> 2. As soon as possible, release Citrix's series for people who can /
> prefer to use the PVH + toolstack option.
> - Immediate work should focus on getting PVH + toolstack functionality
>   ready to release
> - When ready, advisory and HOWTO should include who should use this
>   option and how
> - Either check the series into a branch on xenbits, or into
>   staging-4.10
> - HOWTO-pvh to be included in the advisory.
> 
> This should allow us to get a solution to the widest number of users
> in the shortest amount of time; it also allows us to leverage the
> testing efforts of both Amazon (for breadth of Xen versions) and
> Citrix (for depth of functionality).
> 
> That takes the pressure off us to check in one or the other version of
> the patch series.
> 
> Going forward, we could work towards the convergence of functionality
> from either patch series.  But it looks superficially at least like
> the Citrix series is closer to the convergence point, and so it seems
> like using that as a starting point would make the most sense.

There are a couple of instances of "a branch", and I'm not really
clear on which one that would be, yet in part my opinion depends
on that, as this will affect what state certain branches will be in
for subsequent work. As I agree with the PVH shim being the
better baseline for work going forward, in particular I wouldn't like
to see the Vixen series becoming the base of any branch going to
be maintained going forward.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Anthony Liguori]

On Wed, Jan 10, 2018 at 5:50 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 10.01.18 at 14:34, <george.dunlap@citrix.com> wrote:
>> I'd like to propose a new appraoch:
>>
>> 1. Immediately release Amazon's v1 series for people who can / prefer
>> to use the HVM + sidecar option.
>> - Advisory and HOWTO should include who should use this option, and
>>   how to do it.
>> - Check the series into a branch on xenbits, and add a signed tag
>> - HOWTO-vixen, and sidecar script, to be included in the advisory.
>>
>> 2. As soon as possible, release Citrix's series for people who can /
>> prefer to use the PVH + toolstack option.
>> - Immediate work should focus on getting PVH + toolstack functionality
>>   ready to release
>> - When ready, advisory and HOWTO should include who should use this
>>   option and how
>> - Either check the series into a branch on xenbits, or into
>>   staging-4.10
>> - HOWTO-pvh to be included in the advisory.
>>
>> This should allow us to get a solution to the widest number of users
>> in the shortest amount of time; it also allows us to leverage the
>> testing efforts of both Amazon (for breadth of Xen versions) and
>> Citrix (for depth of functionality).
>>
>> That takes the pressure off us to check in one or the other version of
>> the patch series.
>>
>> Going forward, we could work towards the convergence of functionality
>> from either patch series.  But it looks superficially at least like
>> the Citrix series is closer to the convergence point, and so it seems
>> like using that as a starting point would make the most sense.
>
> There are a couple of instances of "a branch", and I'm not really
> clear on which one that would be, yet in part my opinion depends
> on that, as this will affect what state certain branches will be in
> for subsequent work. As I agree with the PVH shim being the
> better baseline for work going forward, in particular I wouldn't like
> to see the Vixen series becoming the base of any branch going to
> be maintained going forward.

What I would suggest is the following:

1) Merge Vixen into staging

2) Backport Vixen into stable-4.10 and cut a release

3) Immediately begin bringing in PVH shim into staging

4) While doing (3), avoid breaking HVM shim but take full liberty to
    remove Vixen bits and replace.

5) Release 4.11 once PVH shim is ready

I think releasing something to users and then doing a totally
different implementation is going to be extremely painful.

The advantage of merging to staging is that it becomes possible to
bisect regressions.

There isn't really much of a user facing difference here so if the entirety
of Vixen was replaced under the covers for 4.11, as long as we are able
to test continuously and avoid regressions, no one would notice the
difference.

However, if you start from a different base for 4.11 and release something
that regresses from a unique Vixen branch, it's going to be painful trying
to move people off of that branch.

Regards,

Anthony Liguori

> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon [and 1 more messages]

[Ian Jackson]

Jan Beulich writes ("Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
> There are a couple of instances of "a branch", and I'm not really
> clear on which one that would be, yet in part my opinion depends
> on that, as this will affect what state certain branches will be in
> for subsequent work. As I agree with the PVH shim being the
> better baseline for work going forward, in particular I wouldn't like
> to see the Vixen series becoming the base of any branch going to
> be maintained going forward.

Anthony Liguori writes ("Re: [Xen-devel] Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
> What I would suggest is the following:
> 1) Merge Vixen into staging
> 2) Backport Vixen into stable-4.10 and cut a release

We do not have time any longer (if we had time to start with) to
reconcile these divergent views.


Hence George's suggestion, which bypasses the problem.  By "a branch"
we mean some git branch on xenbits which is not any of our usual git
branches, and which we expect to die fairly soon.

Jan, I suggest

   https://xenbits.xen.org/git-http/xen.git
     refs/heads/4.10.meltdown.vixen
     refs/tags/4.10.meltdown.vixen.1   (signed by usual key)

If we can't agree to that[1] then I intend the following:

   https://xenbits.xen.org/git-http/people/iwj/xen.git
     refs/heads/meltdown/4.10.meltdown.vixen
     refs/tags/4.10.meltdown.vixen.1   (signed by me personally)

[1] I am happy to accept any reasonable counterproposal for the exact
branch and tag names.


Separately, I would like to say that this branch will receive security
support from the Xen Project Security Team, but that security support
will be withdrawn at no less than 2 months' notice when a final
solution is availabler.

If anyone objects to that then we can go ahead with sending out
information ASAP and argue about security support status later.


Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon [and 1 more messages]

[Roger Pau Monné]

On Wed, Jan 10, 2018 at 04:39:11PM +0000, Ian Jackson wrote:
> Jan Beulich writes ("Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
> > There are a couple of instances of "a branch", and I'm not really
> > clear on which one that would be, yet in part my opinion depends
> > on that, as this will affect what state certain branches will be in
> > for subsequent work. As I agree with the PVH shim being the
> > better baseline for work going forward, in particular I wouldn't like
> > to see the Vixen series becoming the base of any branch going to
> > be maintained going forward.
> 
> Anthony Liguori writes ("Re: [Xen-devel] Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
> > What I would suggest is the following:
> > 1) Merge Vixen into staging
> > 2) Backport Vixen into stable-4.10 and cut a release
> 
> We do not have time any longer (if we had time to start with) to
> reconcile these divergent views.
> 
> 
> Hence George's suggestion, which bypasses the problem.  By "a branch"
> we mean some git branch on xenbits which is not any of our usual git
> branches, and which we expect to die fairly soon.
> 
> Jan, I suggest
> 
>    https://xenbits.xen.org/git-http/xen.git
>      refs/heads/4.10.meltdown.vixen
>      refs/tags/4.10.meltdown.vixen.1   (signed by usual key)
> 
> If we can't agree to that[1] then I intend the following:
> 
>    https://xenbits.xen.org/git-http/people/iwj/xen.git
>      refs/heads/meltdown/4.10.meltdown.vixen
>      refs/tags/4.10.meltdown.vixen.1   (signed by me personally)
> 
> [1] I am happy to accept any reasonable counterproposal for the exact
> branch and tag names.
> 
> 
> Separately, I would like to say that this branch will receive security
> support from the Xen Project Security Team, but that security support
> will be withdrawn at no less than 2 months' notice when a final
> solution is availabler.

I would add that security support will be limited to issues which
affect the security of the shim guest (ie: like being able to break
from user to kernel level or similar). IMHO a warning should be added
somewhere that this branch should not be used as a bare-metal
hypervisor.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon [and 1 more messages]

[George Dunlap]

On 01/10/2018 04:39 PM, Ian Jackson wrote:
> Jan Beulich writes ("Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
>> There are a couple of instances of "a branch", and I'm not really
>> clear on which one that would be, yet in part my opinion depends
>> on that, as this will affect what state certain branches will be in
>> for subsequent work. As I agree with the PVH shim being the
>> better baseline for work going forward, in particular I wouldn't like
>> to see the Vixen series becoming the base of any branch going to
>> be maintained going forward.
> 
> Anthony Liguori writes ("Re: [Xen-devel] Radical proposal v2: Publish Amazon's verison now, Citrix's version soon"):
>> What I would suggest is the following:
>> 1) Merge Vixen into staging
>> 2) Backport Vixen into stable-4.10 and cut a release
> 
> We do not have time any longer (if we had time to start with) to
> reconcile these divergent views.
> 
> 
> Hence George's suggestion, which bypasses the problem.  By "a branch"
> we mean some git branch on xenbits which is not any of our usual git
> branches, and which we expect to die fairly soon.
> 
> Jan, I suggest
> 
>    https://xenbits.xen.org/git-http/xen.git
>      refs/heads/4.10.meltdown.vixen
>      refs/tags/4.10.meltdown.vixen.1   (signed by usual key)

I would use 'shim' somewhere; and I don't think 'meltdown' is necessary,
but I'm not terribly picky.

+1 to putting it on xenbits.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon [and 1 more messages]

[Jan Beulich]

>>> On 10.01.18 at 17:39, <ian.jackson@eu.citrix.com> wrote:
> Jan Beulich writes ("Re: Radical proposal v2: Publish Amazon's verison now, 
> Citrix's version soon"):
>> There are a couple of instances of "a branch", and I'm not really
>> clear on which one that would be, yet in part my opinion depends
>> on that, as this will affect what state certain branches will be in
>> for subsequent work. As I agree with the PVH shim being the
>> better baseline for work going forward, in particular I wouldn't like
>> to see the Vixen series becoming the base of any branch going to
>> be maintained going forward.
> 
> Anthony Liguori writes ("Re: [Xen-devel] Radical proposal v2: Publish 
> Amazon's verison now, Citrix's version soon"):
>> What I would suggest is the following:
>> 1) Merge Vixen into staging
>> 2) Backport Vixen into stable-4.10 and cut a release
> 
> We do not have time any longer (if we had time to start with) to
> reconcile these divergent views.
> 
> 
> Hence George's suggestion, which bypasses the problem.  By "a branch"
> we mean some git branch on xenbits which is not any of our usual git
> branches, and which we expect to die fairly soon.
> 
> Jan, I suggest
> 
>    https://xenbits.xen.org/git-http/xen.git 
>      refs/heads/4.10.meltdown.vixen
>      refs/tags/4.10.meltdown.vixen.1   (signed by usual key)

Fine with me.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Juergen Gross]

On 10/01/18 14:34, George Dunlap wrote:
> * Executive summary
> 
> - We've agreed on a "convergence" point for PV shim functionality that
>   covers as many users as possible:
>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>    event channels, &c, booted via 'sidecar'
>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>    changes
> 
> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>   each cover some users and not others; neither one (yet) covers all
>   users
> 
> - Vixen is ready for immediate release; PVH shim still needs some
>   minor revision, and the toolstack component
> 
> - Proposal:
>  - Release "HOWTO" for Amazon shim v1 immediately (including a signed
>    tag on xenbits with the requisite patches)
>  - Release "HOWTO" for PVH shim as soon as it's ready (including a
>    signed tag or tags on xenbits with the requisite patches)
>  - Base future development on the PVH shim (porting over functionality
>    from the Amazon series)
> 
> * Discussion
> 
> In our discussions, we've generally identified four kinds of users /
> consumers of Xen:
> 
> 1. Those for whom upgrading to a newer version of Xen is the biggest
> problem
> 
> This could be people who have local changes they can't forward-port,
> people whose SLAs forbid updating versions, or any number of other
> things that make updating to a newer version of Xen unpalatable or
> impossible.
> 
> 2. Those for whom adding a QEMU instance to all PV guests is the
> biggest problem
> 
> Many users I've talked to extremely dislike the idea of adding QEMU
> instances to all their PV guests, and would rather do upgrades and/or
> modify guest types to avoid this
> 
> 3. Those for whom modifying guest parameters is the biggest problem
> 
> Many users I've talked to are using software which severely limits the
> flexibility they have in terms of how guests are created
> 
> 4. Those for whom some subset of features (migration, ballooning, vcpu
> hotplug, &c) is the biggest problem.
> 
> This includes users who rely on these features, as well as software
> providers that provide those features.
> 
> Amazon is in the first category, and have developed a "PV shim"
> series that addresses their needs.  They have also very kindly
> shared the patches publicly for the community to be able to use.
> 
> Citrix is in the fourth category, and have developed a "PV shim"
> series that addresses their needs.  This shim also satisfies category 2,
> and with some work could support category 3.
> 
> Going forward, the best solutions for new hypervisors (Xen 4.11+)
> would be to avoid needing QEMU, building the sidecar, and so on.  But
> we still want to be able to support people running the 'HVM shim with
> sidecar' version going forward.  So the "convergence point" we've all
> seemed to agree on is to have a shim capable of satisfying all groups:
> 
> - Can run in HVM mode back as far as Xen 3.4
> - Can run in PVH mode, without QEMU or "sidecar", and doesn't require
>   many toolstack changes
> - Has feature parity with PV mode (migration, ballooning, vcpu
>   migration, &c)
> 
> The issue has been made public for nearly a week now, so there is a
> lot of pressure to get a solution to our users as soon as possible; by
> the end of the week at absolute latest, but today if possible.
> 
> Amazon's v1 series:
> - Has support for 'legacy' event channels
> 
> - Has been tested by Amazon over a wide range of their hypervisor
>   versions and guest types (including pvgrub types) back as far as Xen
>   3.4
> 
> - Uses HVM mode, and so has the overhead and security implications of
>   running QEMU
> 
> - Requires no L0 hypervisor or toolstack changes
> 
> - Requires users to build a "boot sidecar" image for each unique guest
>   boot configuration
> 
> - Is missing many features, such as migration, memory ballooning, and
>   vcpu hotplug
> 
> - Has accurate 'stolen time' support
> 
> Citrix's series:
> - Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
>   PVH backports) and 4.10
> - Uses PVH mode, and so doesn't have the overhead and security
>   implications of running QEMU
> - No need to build "boot sidecar"
> - Requires hypervisor changes for versions for versions before 4.10
>   which cannot reasonably be backported by the open-source team beyond
>   Xen 4.8
> - Requires toolstack changes in all versions
> - Has migration, memory ballooning, and vcpu hotplug (extensively
>   tested by XenRT)
> - Doesn't have accurate 'stolen time' support
> - Currently is missing a clean toolstack interface
> 
> With some tweaks, Citrix's version has been made to boot in HVM mode.
> However, this modified version:
> - Doesn't support the 'legacy' event channel mode, and so won't work
>   for versions as old as Amazon's
> - Hasn't been tested extensively on older versions
> 
> Ian suggested that we get something and check it into 4.10-staging
> as soon as possible.  But there has been a debate about whether we
> should start with Amazon's series and add PVH support, or start with
> Citrix's series and add HVM support (including legacy event channels,
> and so on).
> 
> No matter what, individual users will need to decide whether to take
> the 'HVM + sidecar' option or the 'PVH' option..
> 
> I'd like to propose a new appraoch:
> 
> 1. Immediately release Amazon's v1 series for people who can / prefer
> to use the HVM + sidecar option.
> - Advisory and HOWTO should include who should use this option, and
>   how to do it.
> - Check the series into a branch on xenbits, and add a signed tag
> - HOWTO-vixen, and sidecar script, to be included in the advisory.
> 
> 2. As soon as possible, release Citrix's series for people who can /
> prefer to use the PVH + toolstack option.
> - Immediate work should focus on getting PVH + toolstack functionality
>   ready to release
> - When ready, advisory and HOWTO should include who should use this
>   option and how
> - Either check the series into a branch on xenbits, or into
>   staging-4.10
> - HOWTO-pvh to be included in the advisory.
> 
> This should allow us to get a solution to the widest number of users
> in the shortest amount of time; it also allows us to leverage the
> testing efforts of both Amazon (for breadth of Xen versions) and
> Citrix (for depth of functionality).
> 
> That takes the pressure off us to check in one or the other version of
> the patch series.
> 
> Going forward, we could work towards the convergence of functionality
> from either patch series.  But it looks superficially at least like
> the Citrix series is closer to the convergence point, and so it seems
> like using that as a starting point would make the most sense.
> 
> Regardless of what we think of step 2, I think we should take step 1
> immediately.
> 
> Let me know what you think.

I'm absolutely in favor of that idea.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Lars Kurth]

> On 10 Jan 2018, at 13:51, Juergen Gross <jgross@suse.com> wrote:
> 
> On 10/01/18 14:34, George Dunlap wrote:
>> * Executive summary
>> 
[snip]

>> 
>> Regardless of what we think of step 2, I think we should take step 1
>> immediately.
>> 
>> Let me know what you think.

Thank you for putting this proposal together.

> I'm absolutely in favor of that idea.

Me too. We should aim for a release tomorrow morning. Maybe getting as much as we need to get in place today.
It is probably to late to get all the bits and pieces done today.

Regards
Lars
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Anthony Liguori]

On Wed, Jan 10, 2018 at 5:51 AM, Juergen Gross <jgross@suse.com> wrote:
> On 10/01/18 14:34, George Dunlap wrote:
>> * Executive summary
>>
>> - We've agreed on a "convergence" point for PV shim functionality that
>>   covers as many users as possible:
>>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>    event channels, &c, booted via 'sidecar'
>>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>    changes
>>
>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>   each cover some users and not others; neither one (yet) covers all
>>   users
>>
>> - Vixen is ready for immediate release; PVH shim still needs some
>>   minor revision, and the toolstack component
>>
>> - Proposal:
>>  - Release "HOWTO" for Amazon shim v1 immediately (including a signed
>>    tag on xenbits with the requisite patches)
>>  - Release "HOWTO" for PVH shim as soon as it's ready (including a
>>    signed tag or tags on xenbits with the requisite patches)
>>  - Base future development on the PVH shim (porting over functionality
>>    from the Amazon series)
>>
>> * Discussion
>>
>> In our discussions, we've generally identified four kinds of users /
>> consumers of Xen:
>>
>> 1. Those for whom upgrading to a newer version of Xen is the biggest
>> problem
>>
>> This could be people who have local changes they can't forward-port,
>> people whose SLAs forbid updating versions, or any number of other
>> things that make updating to a newer version of Xen unpalatable or
>> impossible.
>>
>> 2. Those for whom adding a QEMU instance to all PV guests is the
>> biggest problem
>>
>> Many users I've talked to extremely dislike the idea of adding QEMU
>> instances to all their PV guests, and would rather do upgrades and/or
>> modify guest types to avoid this
>>
>> 3. Those for whom modifying guest parameters is the biggest problem
>>
>> Many users I've talked to are using software which severely limits the
>> flexibility they have in terms of how guests are created
>>
>> 4. Those for whom some subset of features (migration, ballooning, vcpu
>> hotplug, &c) is the biggest problem.
>>
>> This includes users who rely on these features, as well as software
>> providers that provide those features.
>>
>> Amazon is in the first category, and have developed a "PV shim"
>> series that addresses their needs.  They have also very kindly
>> shared the patches publicly for the community to be able to use.
>>
>> Citrix is in the fourth category, and have developed a "PV shim"
>> series that addresses their needs.  This shim also satisfies category 2,
>> and with some work could support category 3.
>>
>> Going forward, the best solutions for new hypervisors (Xen 4.11+)
>> would be to avoid needing QEMU, building the sidecar, and so on.  But
>> we still want to be able to support people running the 'HVM shim with
>> sidecar' version going forward.  So the "convergence point" we've all
>> seemed to agree on is to have a shim capable of satisfying all groups:
>>
>> - Can run in HVM mode back as far as Xen 3.4
>> - Can run in PVH mode, without QEMU or "sidecar", and doesn't require
>>   many toolstack changes
>> - Has feature parity with PV mode (migration, ballooning, vcpu
>>   migration, &c)
>>
>> The issue has been made public for nearly a week now, so there is a
>> lot of pressure to get a solution to our users as soon as possible; by
>> the end of the week at absolute latest, but today if possible.
>>
>> Amazon's v1 series:
>> - Has support for 'legacy' event channels
>>
>> - Has been tested by Amazon over a wide range of their hypervisor
>>   versions and guest types (including pvgrub types) back as far as Xen
>>   3.4
>>
>> - Uses HVM mode, and so has the overhead and security implications of
>>   running QEMU
>>
>> - Requires no L0 hypervisor or toolstack changes
>>
>> - Requires users to build a "boot sidecar" image for each unique guest
>>   boot configuration
>>
>> - Is missing many features, such as migration, memory ballooning, and
>>   vcpu hotplug
>>
>> - Has accurate 'stolen time' support
>>
>> Citrix's series:
>> - Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
>>   PVH backports) and 4.10
>> - Uses PVH mode, and so doesn't have the overhead and security
>>   implications of running QEMU
>> - No need to build "boot sidecar"
>> - Requires hypervisor changes for versions for versions before 4.10
>>   which cannot reasonably be backported by the open-source team beyond
>>   Xen 4.8
>> - Requires toolstack changes in all versions
>> - Has migration, memory ballooning, and vcpu hotplug (extensively
>>   tested by XenRT)
>> - Doesn't have accurate 'stolen time' support
>> - Currently is missing a clean toolstack interface
>>
>> With some tweaks, Citrix's version has been made to boot in HVM mode.
>> However, this modified version:
>> - Doesn't support the 'legacy' event channel mode, and so won't work
>>   for versions as old as Amazon's
>> - Hasn't been tested extensively on older versions
>>
>> Ian suggested that we get something and check it into 4.10-staging
>> as soon as possible.  But there has been a debate about whether we
>> should start with Amazon's series and add PVH support, or start with
>> Citrix's series and add HVM support (including legacy event channels,
>> and so on).
>>
>> No matter what, individual users will need to decide whether to take
>> the 'HVM + sidecar' option or the 'PVH' option..
>>
>> I'd like to propose a new appraoch:
>>
>> 1. Immediately release Amazon's v1 series for people who can / prefer
>> to use the HVM + sidecar option.
>> - Advisory and HOWTO should include who should use this option, and
>>   how to do it.
>> - Check the series into a branch on xenbits, and add a signed tag
>> - HOWTO-vixen, and sidecar script, to be included in the advisory.
>>
>> 2. As soon as possible, release Citrix's series for people who can /
>> prefer to use the PVH + toolstack option.
>> - Immediate work should focus on getting PVH + toolstack functionality
>>   ready to release
>> - When ready, advisory and HOWTO should include who should use this
>>   option and how
>> - Either check the series into a branch on xenbits, or into
>>   staging-4.10
>> - HOWTO-pvh to be included in the advisory.
>>
>> This should allow us to get a solution to the widest number of users
>> in the shortest amount of time; it also allows us to leverage the
>> testing efforts of both Amazon (for breadth of Xen versions) and
>> Citrix (for depth of functionality).
>>
>> That takes the pressure off us to check in one or the other version of
>> the patch series.
>>
>> Going forward, we could work towards the convergence of functionality
>> from either patch series.  But it looks superficially at least like
>> the Citrix series is closer to the convergence point, and so it seems
>> like using that as a starting point would make the most sense.
>>
>> Regardless of what we think of step 2, I think we should take step 1
>> immediately.
>>
>> Let me know what you think.
>
> I'm absolutely in favor of that idea.

I've pushed https://github.com/aliguori/xen/tree/vixen-upstream-v1.5 which
contains the original series + one fix from Wei for xl shutdown.

I think this is the branch to merge.

Regards,

Anthony Liguori

>
>
> Juergen
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Stefano Stabellini]

On Wed, 10 Jan 2018, George Dunlap wrote:
> * Executive summary
> 
> - We've agreed on a "convergence" point for PV shim functionality that
>   covers as many users as possible:
>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>    event channels, &c, booted via 'sidecar'
>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>    changes
> 
> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>   each cover some users and not others; neither one (yet) covers all
>   users

Sorry for being punctilious, but neither one can cover all users: there
are users without VT-x on their platform, and both approaches require
VT-x.

Both can only be temporary stop-gaps. I made a couple of more comments
in this respect below.



> - Vixen is ready for immediate release; PVH shim still needs some
>   minor revision, and the toolstack component
> 
> - Proposal:
>  - Release "HOWTO" for Amazon shim v1 immediately (including a signed
>    tag on xenbits with the requisite patches)
>  - Release "HOWTO" for PVH shim as soon as it's ready (including a
>    signed tag or tags on xenbits with the requisite patches)
>  - Base future development on the PVH shim (porting over functionality
>    from the Amazon series)
> 
> * Discussion
> 
> In our discussions, we've generally identified four kinds of users /
> consumers of Xen:
> 
> 1. Those for whom upgrading to a newer version of Xen is the biggest
> problem
> 
> This could be people who have local changes they can't forward-port,
> people whose SLAs forbid updating versions, or any number of other
> things that make updating to a newer version of Xen unpalatable or
> impossible.
> 
> 2. Those for whom adding a QEMU instance to all PV guests is the
> biggest problem
> 
> Many users I've talked to extremely dislike the idea of adding QEMU
> instances to all their PV guests, and would rather do upgrades and/or
> modify guest types to avoid this
> 
> 3. Those for whom modifying guest parameters is the biggest problem
> 
> Many users I've talked to are using software which severely limits the
> flexibility they have in terms of how guests are created
> 
> 4. Those for whom some subset of features (migration, ballooning, vcpu
> hotplug, &c) is the biggest problem.
> 
> This includes users who rely on these features, as well as software
> providers that provide those features.

5. Those that cannot deploy neither Vixen nor PVShim due to lack of VT-x



> Amazon is in the first category, and have developed a "PV shim"
> series that addresses their needs.  They have also very kindly
> shared the patches publicly for the community to be able to use.
> 
> Citrix is in the fourth category, and have developed a "PV shim"
> series that addresses their needs.  This shim also satisfies category 2,
> and with some work could support category 3.
> 
> Going forward, the best solutions for new hypervisors (Xen 4.11+)
> would be to avoid needing QEMU, building the sidecar, and so on.  But
> we still want to be able to support people running the 'HVM shim with
> sidecar' version going forward.  So the "convergence point" we've all
> seemed to agree on is to have a shim capable of satisfying all groups:

Going forward, the best solutions for new hypervisors (Xen 4.11+) is to
fix PV in a way that is secure without needing VT-x.



> - Can run in HVM mode back as far as Xen 3.4
> - Can run in PVH mode, without QEMU or "sidecar", and doesn't require
>   many toolstack changes
> - Has feature parity with PV mode (migration, ballooning, vcpu
>   migration, &c)
>
> The issue has been made public for nearly a week now, so there is a
> lot of pressure to get a solution to our users as soon as possible; by
> the end of the week at absolute latest, but today if possible.
> 
> Amazon's v1 series:
> - Has support for 'legacy' event channels
> 
> - Has been tested by Amazon over a wide range of their hypervisor
>   versions and guest types (including pvgrub types) back as far as Xen
>   3.4
> 
> - Uses HVM mode, and so has the overhead and security implications of
>   running QEMU
> 
> - Requires no L0 hypervisor or toolstack changes
> 
> - Requires users to build a "boot sidecar" image for each unique guest
>   boot configuration
> 
> - Is missing many features, such as migration, memory ballooning, and
>   vcpu hotplug
> 
> - Has accurate 'stolen time' support
> 
> Citrix's series:
> - Has been extensively tested by XenServer's XenRT for Xen 4.7 (with
>   PVH backports) and 4.10
> - Uses PVH mode, and so doesn't have the overhead and security
>   implications of running QEMU
> - No need to build "boot sidecar"
> - Requires hypervisor changes for versions for versions before 4.10
>   which cannot reasonably be backported by the open-source team beyond
>   Xen 4.8
> - Requires toolstack changes in all versions
> - Has migration, memory ballooning, and vcpu hotplug (extensively
>   tested by XenRT)
> - Doesn't have accurate 'stolen time' support
> - Currently is missing a clean toolstack interface
> 
> With some tweaks, Citrix's version has been made to boot in HVM mode.
> However, this modified version:
> - Doesn't support the 'legacy' event channel mode, and so won't work
>   for versions as old as Amazon's
> - Hasn't been tested extensively on older versions
> 
> Ian suggested that we get something and check it into 4.10-staging
> as soon as possible.  But there has been a debate about whether we
> should start with Amazon's series and add PVH support, or start with
> Citrix's series and add HVM support (including legacy event channels,
> and so on).
> 
> No matter what, individual users will need to decide whether to take
> the 'HVM + sidecar' option or the 'PVH' option..
> 
> I'd like to propose a new appraoch:
> 
> 1. Immediately release Amazon's v1 series for people who can / prefer
> to use the HVM + sidecar option.
> - Advisory and HOWTO should include who should use this option, and
>   how to do it.
> - Check the series into a branch on xenbits, and add a signed tag
> - HOWTO-vixen, and sidecar script, to be included in the advisory.
> 
> 2. As soon as possible, release Citrix's series for people who can /
> prefer to use the PVH + toolstack option.
> - Immediate work should focus on getting PVH + toolstack functionality
>   ready to release
> - When ready, advisory and HOWTO should include who should use this
>   option and how
> - Either check the series into a branch on xenbits, or into
>   staging-4.10
> - HOWTO-pvh to be included in the advisory.
> 
> This should allow us to get a solution to the widest number of users
> in the shortest amount of time; it also allows us to leverage the
> testing efforts of both Amazon (for breadth of Xen versions) and
> Citrix (for depth of functionality).
> 
> That takes the pressure off us to check in one or the other version of
> the patch series.
> 
> Going forward, we could work towards the convergence of functionality
> from either patch series.  But it looks superficially at least like
> the Citrix series is closer to the convergence point, and so it seems
> like using that as a starting point would make the most sense.
> 
> Regardless of what we think of step 2, I think we should take step 1
> immediately.
> 
> Let me know what you think.
> 
>  -George
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Jan Beulich]

>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
> On Wed, 10 Jan 2018, George Dunlap wrote:
>> * Executive summary
>> 
>> - We've agreed on a "convergence" point for PV shim functionality that
>>   covers as many users as possible:
>>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>    event channels, &c, booted via 'sidecar'
>>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>    changes
>> 
>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>   each cover some users and not others; neither one (yet) covers all
>>   users
> 
> Sorry for being punctilious, but neither one can cover all users: there
> are users without VT-x on their platform, and both approaches require
> VT-x.

For the record, yesterday I've decided to make an attempt to
create a very simplistic patch to deal with the issue in the
hypervisor, ignoring (almost) all performance considerations
(not all, because I didn't want to go the "disable caching" route).
I've dealt with some of the to-be-expected early bugs, but I'm
now debugging a host hang (note: not a triple fault apparently,
as the box doesn't reboot, yet triple faults is what I would have
expected to occur if anything is wrong here or missing).

I know that's late, and I have to admit that I don't understand
myself why I didn't consider doing such earlier on, but the
much increased pressure to get something like the shim out,
which
- doesn't address all cases
- requires changes to how VMs are being created (which likely will
  be a problem for various customers)
- later will want those changes undone
plus the pretty obvious impossibility to backport something like
Andrew's (not yet complete) series to baselines as old as 3.2
made it seem to me that some (measurable!) performance
overhead can't be all that bad in the given situation.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Stefano Stabellini]

On Thu, 11 Jan 2018, Jan Beulich wrote:
> >>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
> > On Wed, 10 Jan 2018, George Dunlap wrote:
> >> * Executive summary
> >> 
> >> - We've agreed on a "convergence" point for PV shim functionality that
> >>   covers as many users as possible:
> >>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
> >>    event channels, &c, booted via 'sidecar'
> >>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
> >>    changes
> >> 
> >> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
> >>   each cover some users and not others; neither one (yet) covers all
> >>   users
> > 
> > Sorry for being punctilious, but neither one can cover all users: there
> > are users without VT-x on their platform, and both approaches require
> > VT-x.
> 
> For the record, yesterday I've decided to make an attempt to
> create a very simplistic patch to deal with the issue in the
> hypervisor, ignoring (almost) all performance considerations
> (not all, because I didn't want to go the "disable caching" route).
> I've dealt with some of the to-be-expected early bugs, but I'm
> now debugging a host hang (note: not a triple fault apparently,
> as the box doesn't reboot, yet triple faults is what I would have
> expected to occur if anything is wrong here or missing).
> 
> I know that's late, and I have to admit that I don't understand
> myself why I didn't consider doing such earlier on, but the
> much increased pressure to get something like the shim out,
> which
> - doesn't address all cases
> - requires changes to how VMs are being created (which likely will
>   be a problem for various customers)
> - later will want those changes undone
> plus the pretty obvious impossibility to backport something like
> Andrew's (not yet complete) series to baselines as old as 3.2
> made it seem to me that some (measurable!) performance
> overhead can't be all that bad in the given situation.

Thank you for giving it a look! I completely agree with you on these
points. I think we should approach this problem with the assumption that
this is going to be the only long term solution to SP3, while Vixen (or
PVshim) incomplete stopgaps for now.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[George Dunlap]

On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
> On Thu, 11 Jan 2018, Jan Beulich wrote:
>>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
>>> On Wed, 10 Jan 2018, George Dunlap wrote:
>>>> * Executive summary
>>>>
>>>> - We've agreed on a "convergence" point for PV shim functionality that
>>>>   covers as many users as possible:
>>>>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>>>    event channels, &c, booted via 'sidecar'
>>>>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>>>    changes
>>>>
>>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>>>   each cover some users and not others; neither one (yet) covers all
>>>>   users
>>>
>>> Sorry for being punctilious, but neither one can cover all users: there
>>> are users without VT-x on their platform, and both approaches require
>>> VT-x.
>>
>> For the record, yesterday I've decided to make an attempt to
>> create a very simplistic patch to deal with the issue in the
>> hypervisor, ignoring (almost) all performance considerations
>> (not all, because I didn't want to go the "disable caching" route).
>> I've dealt with some of the to-be-expected early bugs, but I'm
>> now debugging a host hang (note: not a triple fault apparently,
>> as the box doesn't reboot, yet triple faults is what I would have
>> expected to occur if anything is wrong here or missing).
>>
>> I know that's late, and I have to admit that I don't understand
>> myself why I didn't consider doing such earlier on, but the
>> much increased pressure to get something like the shim out,
>> which
>> - doesn't address all cases
>> - requires changes to how VMs are being created (which likely will
>>   be a problem for various customers)
>> - later will want those changes undone
>> plus the pretty obvious impossibility to backport something like
>> Andrew's (not yet complete) series to baselines as old as 3.2
>> made it seem to me that some (measurable!) performance
>> overhead can't be all that bad in the given situation.
> 
> Thank you for giving it a look! I completely agree with you on these
> points. I think we should approach this problem with the assumption that
> this is going to be the only long term solution to SP3, while Vixen (or
> PVshim) incomplete stopgaps for now.

Well the pvshim is a feature for people who want to be able to eliminate
all PV interfaces to the hypervisor whatsover for security / maintenance
purposes.  I do agree a "proper" fix for PV would be good, assuming the
overhead is lower than pvshim.

 -George


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Stefano Stabellini]

On Thu, 11 Jan 2018, George Dunlap wrote:
> On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
> > On Thu, 11 Jan 2018, Jan Beulich wrote:
> >>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
> >>> On Wed, 10 Jan 2018, George Dunlap wrote:
> >>>> * Executive summary
> >>>>
> >>>> - We've agreed on a "convergence" point for PV shim functionality that
> >>>>   covers as many users as possible:
> >>>>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
> >>>>    event channels, &c, booted via 'sidecar'
> >>>>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
> >>>>    changes
> >>>>
> >>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
> >>>>   each cover some users and not others; neither one (yet) covers all
> >>>>   users
> >>>
> >>> Sorry for being punctilious, but neither one can cover all users: there
> >>> are users without VT-x on their platform, and both approaches require
> >>> VT-x.
> >>
> >> For the record, yesterday I've decided to make an attempt to
> >> create a very simplistic patch to deal with the issue in the
> >> hypervisor, ignoring (almost) all performance considerations
> >> (not all, because I didn't want to go the "disable caching" route).
> >> I've dealt with some of the to-be-expected early bugs, but I'm
> >> now debugging a host hang (note: not a triple fault apparently,
> >> as the box doesn't reboot, yet triple faults is what I would have
> >> expected to occur if anything is wrong here or missing).
> >>
> >> I know that's late, and I have to admit that I don't understand
> >> myself why I didn't consider doing such earlier on, but the
> >> much increased pressure to get something like the shim out,
> >> which
> >> - doesn't address all cases
> >> - requires changes to how VMs are being created (which likely will
> >>   be a problem for various customers)
> >> - later will want those changes undone
> >> plus the pretty obvious impossibility to backport something like
> >> Andrew's (not yet complete) series to baselines as old as 3.2
> >> made it seem to me that some (measurable!) performance
> >> overhead can't be all that bad in the given situation.
> > 
> > Thank you for giving it a look! I completely agree with you on these
> > points. I think we should approach this problem with the assumption that
> > this is going to be the only long term solution to SP3, while Vixen (or
> > PVshim) incomplete stopgaps for now.
> 
> Well the pvshim is a feature for people who want to be able to eliminate
> all PV interfaces to the hypervisor whatsover for security / maintenance
> purposes.  I do agree a "proper" fix for PV would be good, assuming the
> overhead is lower than pvshim.

Why "assuming the overhead is lower than pvshim"? What if the overhead
is higher?  As I said, there are users that *cannot* deploy HVM because
it is not available to them.

In other words, PVshim is irrelevant to me because I cannot use it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Rich Persaud]

On Jan 11, 2018, at 11:36, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
>> On Thu, 11 Jan 2018, George Dunlap wrote:
>>> On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
>>> On Thu, 11 Jan 2018, Jan Beulich wrote:
>>>>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
>>>>>> On Wed, 10 Jan 2018, George Dunlap wrote:
>>>>>> * Executive summary
>>>>>> 
>>>>>> - We've agreed on a "convergence" point for PV shim functionality that
>>>>>>  covers as many users as possible:
>>>>>> - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>>>>>   event channels, &c, booted via 'sidecar'
>>>>>> - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>>>>>   changes
>>>>>> 
>>>>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>>>>>  each cover some users and not others; neither one (yet) covers all
>>>>>>  users
>>>>> 
>>>>> Sorry for being punctilious, but neither one can cover all users: there
>>>>> are users without VT-x on their platform, and both approaches require
>>>>> VT-x.
>>>> 
>>>> For the record, yesterday I've decided to make an attempt to
>>>> create a very simplistic patch to deal with the issue in the
>>>> hypervisor, ignoring (almost) all performance considerations
>>>> (not all, because I didn't want to go the "disable caching" route).
>>>> I've dealt with some of the to-be-expected early bugs, but I'm
>>>> now debugging a host hang (note: not a triple fault apparently,
>>>> as the box doesn't reboot, yet triple faults is what I would have
>>>> expected to occur if anything is wrong here or missing).
>>>> 
>>>> I know that's late, and I have to admit that I don't understand
>>>> myself why I didn't consider doing such earlier on, but the
>>>> much increased pressure to get something like the shim out,
>>>> which
>>>> - doesn't address all cases
>>>> - requires changes to how VMs are being created (which likely will
>>>>  be a problem for various customers)
>>>> - later will want those changes undone
>>>> plus the pretty obvious impossibility to backport something like
>>>> Andrew's (not yet complete) series to baselines as old as 3.2
>>>> made it seem to me that some (measurable!) performance
>>>> overhead can't be all that bad in the given situation.
>>> 
>>> Thank you for giving it a look! I completely agree with you on these
>>> points. I think we should approach this problem with the assumption that
>>> this is going to be the only long term solution to SP3, while Vixen (or
>>> PVshim) incomplete stopgaps for now.
>> 
>> Well the pvshim is a feature for people who want to be able to eliminate
>> all PV interfaces to the hypervisor whatsover for security / maintenance
>> purposes.  I do agree a "proper" fix for PV would be good, assuming the
>> overhead is lower than pvshim.
> 
> Why "assuming the overhead is lower than pvshim"? What if the overhead
> is higher?  As I said, there are users that *cannot* deploy HVM because
> it is not available to them.
> 
> In other words, PVshim is irrelevant to me because I cannot use it.

Would a “proper” PV fix (does this have a codename?) benefit stubdoms?  These are needed to isolate Qemu, e.g. on an HVM driver domain.  PVshim does not yet support driver domains.

Rich
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[George Dunlap]

On 01/11/2018 04:52 PM, Rich Persaud wrote:
> On Jan 11, 2018, at 11:36, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>>> On Thu, 11 Jan 2018, George Dunlap wrote:
>>>> On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
>>>> On Thu, 11 Jan 2018, Jan Beulich wrote:
>>>>>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
>>>>>>> On Wed, 10 Jan 2018, George Dunlap wrote:
>>>>>>> * Executive summary
>>>>>>>
>>>>>>> - We've agreed on a "convergence" point for PV shim functionality that
>>>>>>>  covers as many users as possible:
>>>>>>> - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>>>>>>>   event channels, &c, booted via 'sidecar'
>>>>>>> - 'PVH' functionality: boots in PVH mode, booted via toolstack
>>>>>>>   changes
>>>>>>>
>>>>>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>>>>>>>  each cover some users and not others; neither one (yet) covers all
>>>>>>>  users
>>>>>>
>>>>>> Sorry for being punctilious, but neither one can cover all users: there
>>>>>> are users without VT-x on their platform, and both approaches require
>>>>>> VT-x.
>>>>>
>>>>> For the record, yesterday I've decided to make an attempt to
>>>>> create a very simplistic patch to deal with the issue in the
>>>>> hypervisor, ignoring (almost) all performance considerations
>>>>> (not all, because I didn't want to go the "disable caching" route).
>>>>> I've dealt with some of the to-be-expected early bugs, but I'm
>>>>> now debugging a host hang (note: not a triple fault apparently,
>>>>> as the box doesn't reboot, yet triple faults is what I would have
>>>>> expected to occur if anything is wrong here or missing).
>>>>>
>>>>> I know that's late, and I have to admit that I don't understand
>>>>> myself why I didn't consider doing such earlier on, but the
>>>>> much increased pressure to get something like the shim out,
>>>>> which
>>>>> - doesn't address all cases
>>>>> - requires changes to how VMs are being created (which likely will
>>>>>  be a problem for various customers)
>>>>> - later will want those changes undone
>>>>> plus the pretty obvious impossibility to backport something like
>>>>> Andrew's (not yet complete) series to baselines as old as 3.2
>>>>> made it seem to me that some (measurable!) performance
>>>>> overhead can't be all that bad in the given situation.
>>>>
>>>> Thank you for giving it a look! I completely agree with you on these
>>>> points. I think we should approach this problem with the assumption that
>>>> this is going to be the only long term solution to SP3, while Vixen (or
>>>> PVshim) incomplete stopgaps for now.
>>>
>>> Well the pvshim is a feature for people who want to be able to eliminate
>>> all PV interfaces to the hypervisor whatsover for security / maintenance
>>> purposes.  I do agree a "proper" fix for PV would be good, assuming the
>>> overhead is lower than pvshim.
>>
>> Why "assuming the overhead is lower than pvshim"? What if the overhead
>> is higher?  As I said, there are users that *cannot* deploy HVM because
>> it is not available to them.
>>
>> In other words, PVshim is irrelevant to me because I cannot use it.
> 
> Would a “proper” PV fix (does this have a codename?) benefit stubdoms?  These are needed to isolate Qemu, e.g. on an HVM driver domain.  PVshim does not yet support driver domains.

We'd have to see the actual code before we could be sure, but I can't
think of a reason why a "proper" PV fix wouldn't work for driver domains.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3527 bytes --]

On Thu, 11 Jan 2018, Rich Persaud wrote:
> On Jan 11, 2018, at 11:36, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> >> On Thu, 11 Jan 2018, George Dunlap wrote:
> >>> On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
> >>> On Thu, 11 Jan 2018, Jan Beulich wrote:
> >>>>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
> >>>>>> On Wed, 10 Jan 2018, George Dunlap wrote:
> >>>>>> * Executive summary
> >>>>>> 
> >>>>>> - We've agreed on a "convergence" point for PV shim functionality that
> >>>>>>  covers as many users as possible:
> >>>>>> - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
> >>>>>>   event channels, &c, booted via 'sidecar'
> >>>>>> - 'PVH' functionality: boots in PVH mode, booted via toolstack
> >>>>>>   changes
> >>>>>> 
> >>>>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
> >>>>>>  each cover some users and not others; neither one (yet) covers all
> >>>>>>  users
> >>>>> 
> >>>>> Sorry for being punctilious, but neither one can cover all users: there
> >>>>> are users without VT-x on their platform, and both approaches require
> >>>>> VT-x.
> >>>> 
> >>>> For the record, yesterday I've decided to make an attempt to
> >>>> create a very simplistic patch to deal with the issue in the
> >>>> hypervisor, ignoring (almost) all performance considerations
> >>>> (not all, because I didn't want to go the "disable caching" route).
> >>>> I've dealt with some of the to-be-expected early bugs, but I'm
> >>>> now debugging a host hang (note: not a triple fault apparently,
> >>>> as the box doesn't reboot, yet triple faults is what I would have
> >>>> expected to occur if anything is wrong here or missing).
> >>>> 
> >>>> I know that's late, and I have to admit that I don't understand
> >>>> myself why I didn't consider doing such earlier on, but the
> >>>> much increased pressure to get something like the shim out,
> >>>> which
> >>>> - doesn't address all cases
> >>>> - requires changes to how VMs are being created (which likely will
> >>>>  be a problem for various customers)
> >>>> - later will want those changes undone
> >>>> plus the pretty obvious impossibility to backport something like
> >>>> Andrew's (not yet complete) series to baselines as old as 3.2
> >>>> made it seem to me that some (measurable!) performance
> >>>> overhead can't be all that bad in the given situation.
> >>> 
> >>> Thank you for giving it a look! I completely agree with you on these
> >>> points. I think we should approach this problem with the assumption that
> >>> this is going to be the only long term solution to SP3, while Vixen (or
> >>> PVshim) incomplete stopgaps for now.
> >> 
> >> Well the pvshim is a feature for people who want to be able to eliminate
> >> all PV interfaces to the hypervisor whatsover for security / maintenance
> >> purposes.  I do agree a "proper" fix for PV would be good, assuming the
> >> overhead is lower than pvshim.
> > 
> > Why "assuming the overhead is lower than pvshim"? What if the overhead
> > is higher?  As I said, there are users that *cannot* deploy HVM because
> > it is not available to them.
> > 
> > In other words, PVshim is irrelevant to me because I cannot use it.
> 
> Would a “proper” PV fix (does this have a codename?) benefit stubdoms?  These are needed to isolate Qemu, e.g. on an HVM driver domain.  PVshim does not yet support driver domains.

Yes, good point. A "proper" fix should support stubdoms too. I think
that Jan's approach above should be able to cover them.

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Jan Beulich]

>>> On 11.01.18 at 17:56, <sstabellini@kernel.org> wrote:
> On Thu, 11 Jan 2018, Rich Persaud wrote:
>> On Jan 11, 2018, at 11:36, Stefano Stabellini <sstabellini@kernel.org> wrote:
>> > 
>> >> On Thu, 11 Jan 2018, George Dunlap wrote:
>> >>> On 01/11/2018 04:23 PM, Stefano Stabellini wrote:
>> >>> On Thu, 11 Jan 2018, Jan Beulich wrote:
>> >>>>>>> On 10.01.18 at 18:25, <sstabellini@kernel.org> wrote:
>> >>>>>> On Wed, 10 Jan 2018, George Dunlap wrote:
>> >>>>>> * Executive summary
>> >>>>>> 
>> >>>>>> - We've agreed on a "convergence" point for PV shim functionality that
>> >>>>>>  covers as many users as possible:
>> >>>>>> - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>> >>>>>>   event channels, &c, booted via 'sidecar'
>> >>>>>> - 'PVH' functionality: boots in PVH mode, booted via toolstack
>> >>>>>>   changes
>> >>>>>> 
>> >>>>>> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>> >>>>>>  each cover some users and not others; neither one (yet) covers all
>> >>>>>>  users
>> >>>>> 
>> >>>>> Sorry for being punctilious, but neither one can cover all users: there
>> >>>>> are users without VT-x on their platform, and both approaches require
>> >>>>> VT-x.
>> >>>> 
>> >>>> For the record, yesterday I've decided to make an attempt to
>> >>>> create a very simplistic patch to deal with the issue in the
>> >>>> hypervisor, ignoring (almost) all performance considerations
>> >>>> (not all, because I didn't want to go the "disable caching" route).
>> >>>> I've dealt with some of the to-be-expected early bugs, but I'm
>> >>>> now debugging a host hang (note: not a triple fault apparently,
>> >>>> as the box doesn't reboot, yet triple faults is what I would have
>> >>>> expected to occur if anything is wrong here or missing).
>> >>>> 
>> >>>> I know that's late, and I have to admit that I don't understand
>> >>>> myself why I didn't consider doing such earlier on, but the
>> >>>> much increased pressure to get something like the shim out,
>> >>>> which
>> >>>> - doesn't address all cases
>> >>>> - requires changes to how VMs are being created (which likely will
>> >>>>  be a problem for various customers)
>> >>>> - later will want those changes undone
>> >>>> plus the pretty obvious impossibility to backport something like
>> >>>> Andrew's (not yet complete) series to baselines as old as 3.2
>> >>>> made it seem to me that some (measurable!) performance
>> >>>> overhead can't be all that bad in the given situation.
>> >>> 
>> >>> Thank you for giving it a look! I completely agree with you on these
>> >>> points. I think we should approach this problem with the assumption that
>> >>> this is going to be the only long term solution to SP3, while Vixen (or
>> >>> PVshim) incomplete stopgaps for now.
>> >> 
>> >> Well the pvshim is a feature for people who want to be able to eliminate
>> >> all PV interfaces to the hypervisor whatsover for security / maintenance
>> >> purposes.  I do agree a "proper" fix for PV would be good, assuming the
>> >> overhead is lower than pvshim.
>> > 
>> > Why "assuming the overhead is lower than pvshim"? What if the overhead
>> > is higher?  As I said, there are users that *cannot* deploy HVM because
>> > it is not available to them.
>> > 
>> > In other words, PVshim is irrelevant to me because I cannot use it.
>> 
>> Would a “proper” PV fix (does this have a codename?) benefit stubdoms?  
> These are needed to isolate Qemu, e.g. on an HVM driver domain.  PVshim does 
> not yet support driver domains.
> 
> Yes, good point. A "proper" fix should support stubdoms too. I think
> that Jan's approach above should be able to cover them.

Well, any in-hypervisor workaround for PV will - naturally - cover
all forms of PV guests. I don't view my patch (which allows Dom0
to come up as of five minutes ago) as a permanent solution though;
I'm pretty convinced Andrew's series, once completed, would have
much better performance characteristics. But for backporting
purpose I think a single patch mostly using infrastructure which
have been around forever is a better basis, and the performance
impact at least on really old versions then would simply need to be
accepted.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[George Dunlap]

On 01/10/2018 01:34 PM, George Dunlap wrote:
> * Executive summary
> 
> - We've agreed on a "convergence" point for PV shim functionality that
>   covers as many users as possible:
>  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
>    event channels, &c, booted via 'sidecar'
>  - 'PVH' functionality: boots in PVH mode, booted via toolstack
>    changes
> 
> - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
>   each cover some users and not others; neither one (yet) covers all
>   users

Proposed codenames we can use for shorthand:

"Vixen": Amazon's HVM+sidecar shim solution
"Comet": Citrix's PVH+toolstack shim solution
"Rudolph": Long-term solution combining both

In the Xen mythos then, Rudolph could be the child of Vixen and Comet. :-)

Cultural references (given that 'Vixen' and 'Comet' were both developed
in the run up to Christmas):

* https://www.teachervision.com/twas-night-christmas-full-text
*
http://www.metrolyrics.com/rudolph-the-red-nosed-reindeer-lyrics-christmas-carols.html

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Radical proposal v2: Publish Amazon's verison now, Citrix's version soon

[Konrad Rzeszutek Wilk]

On Thu, Jan 11, 2018 at 03:17:45PM +0000, George Dunlap wrote:
> On 01/10/2018 01:34 PM, George Dunlap wrote:
> > * Executive summary
> > 
> > - We've agreed on a "convergence" point for PV shim functionality that
> >   covers as many users as possible:
> >  - 'HVM' functionality: boots in HVM mode, has support for Xen 3.4
> >    event channels, &c, booted via 'sidecar'
> >  - 'PVH' functionality: boots in PVH mode, booted via toolstack
> >    changes
> > 
> > - "Vixen" (the Amazon shim) and PVH shim (mostly developed by Citrix)
> >   each cover some users and not others; neither one (yet) covers all
> >   users
> 
> Proposed codenames we can use for shorthand:
> 
> "Vixen": Amazon's HVM+sidecar shim solution
> "Comet": Citrix's PVH+toolstack shim solution
> "Rudolph": Long-term solution combining both
> 
> In the Xen mythos then, Rudolph could be the child of Vixen and Comet. :-)

<groans>

> 
> Cultural references (given that 'Vixen' and 'Comet' were both developed
> in the run up to Christmas):
> 
> * https://www.teachervision.com/twas-night-christmas-full-text
> *
> http://www.metrolyrics.com/rudolph-the-red-nosed-reindeer-lyrics-christmas-carols.html
> 
>  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel