pvops kernel branches - 2.6.36 options, stable/next/etc?

pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 500 bytes --]

Is there now or will there be a 2.6.36 stable branch, or is the only option
for this version the next-2.6.36 branch?

In the past Jeremy has posted details on the state and forward goals for the
major consolidation branches.
I haven't seen any such details regarding the .36/.37 branches.
Does any one know what the plans are?

We've been hanging off of Konrad's  stable/xen-pcifront-0.8.2 branch with
good results. I'm wondering what our options are now with the recent merge
storm.

Thanks

-Bruce

[-- Attachment #1.2: Type: text/html, Size: 569 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Jeremy Fitzhardinge]

On 11/08/2010 10:19 AM, Bruce Edge wrote:
> Is there now or will there be a 2.6.36 stable branch, or is the only
> option for this version the next-2.6.36 branch?

No.  xen/next-2.6.36 doesn't exist any more, and it never really
contained anything useful.  xen/next-2.6.37, however, is closely
tracking upstream and I hope will reach functional parity with the
2.6.32-based branches.

>
> In the past Jeremy has posted details on the state and forward goals
> for the major consolidation branches.
> I haven't seen any such details regarding the .36/.37 branches.
> Does any one know what the plans are?

I'll post something the next week or so when we work out what needs to
happen with .37 to get it up to speed.

>
> We've been hanging off of Konrad's  stable/xen-pcifront-0.8.2 branch
> with good results. I'm wondering what our options are now with the
> recent merge storm.

That has been merged into mainline Linux.

    J

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Thomas Goirand]

On 11/09/2010 03:14 AM, Jeremy Fitzhardinge wrote:
> On 11/08/2010 10:19 AM, Bruce Edge wrote:
>   
>> Is there now or will there be a 2.6.36 stable branch, or is the only
>> option for this version the next-2.6.36 branch?
>>     
> No.  xen/next-2.6.36 doesn't exist any more, and it never really
> contained anything useful.  xen/next-2.6.37, however, is closely
> tracking upstream and I hope will reach functional parity with the
> 2.6.32-based branches.
>   
So, you do believe that it's not too late so that mainline 2.6.37 will
include the backend drivers for HDD? If so, that's great!

Thomas

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Jeremy Fitzhardinge]

On 11/09/2010 02:26 AM, Thomas Goirand wrote:
> On 11/09/2010 03:14 AM, Jeremy Fitzhardinge wrote:
>> On 11/08/2010 10:19 AM, Bruce Edge wrote:
>>   
>>> Is there now or will there be a 2.6.36 stable branch, or is the only
>>> option for this version the next-2.6.36 branch?
>>>     
>> No.  xen/next-2.6.36 doesn't exist any more, and it never really
>> contained anything useful.  xen/next-2.6.37, however, is closely
>> tracking upstream and I hope will reach functional parity with the
>> 2.6.32-based branches.
>>   
> So, you do believe that it's not too late so that mainline 2.6.37 will
> include the backend drivers for HDD? If so, that's great!

No, mainline won't get any device backends until .38 at the earliest -
.37 is closed for anything except bugfixes.  But I'll maintain a .37
based tree with all the extra bits in it for full use.

However, with block backends specifically, we're considering doing away
with a kernel-based backend altogether and deferring it to a usermode
process, so that will be independent of the kernel.

    J

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1741 bytes --]

> But I'll maintain a .37
> based tree with all the extra bits in it for full use.

Would it be possible to know when backend drivers
will come into xen/next-2.6.37 ?

Boris.

--- On Wed, 11/10/10, Jeremy Fitzhardinge <jeremy@goop.org> wrote:

From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Thomas Goirand" <zigo@debian.org>
Cc: xen-devel@lists.xensource.com
Date: Wednesday, November 10, 2010, 1:13 PM

On 11/09/2010 02:26 AM, Thomas Goirand wrote:
> On 11/09/2010 03:14 AM, Jeremy Fitzhardinge wrote:
>> On 11/08/2010 10:19 AM, Bruce Edge wrote:
>>   
>>> Is there now or will there be a 2.6.36 stable branch, or is the only
>>> option for this version the next-2.6.36 branch?
>>>     
>> No.  xen/next-2.6.36 doesn't exist any more, and it never really
>> contained anything useful.  xen/next-2.6.37, however, is closely
>> tracking upstream and I hope will reach functional parity with the
>> 2.6.32-based branches.
>>   
> So, you do believe that it's not too late so that mainline 2.6.37 will
> include the backend drivers for HDD? If so, that's great!

No, mainline won't get any device backends until .38 at the earliest -
.37 is closed for anything except bugfixes.  But I'll maintain a .37
based tree with all the extra bits in it for full use.

However, with block backends specifically, we're considering doing away
with a kernel-based backend altogether and deferring it to a usermode
process, so that will be independent of the kernel.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 2454 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1842 bytes --]

> No.  xen/next-2.6.36 doesn't exist any more, and it never really
> contained anything useful.  xen/next-2.6.37, however, is closely
> tracking upstream and I hope will reach functional parity with the
> 2.6.32-based branches.

Is 2.6.38 merge window right estimate for time frame to reach functional 
parity with the 2.6.32-based branches ?

Boris.

--- On Mon, 11/8/10, Jeremy Fitzhardinge <jeremy@goop.org> wrote:

From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Monday, November 8, 2010, 2:14 PM

On 11/08/2010 10:19 AM, Bruce Edge wrote:
> Is there now or will there be a 2.6.36 stable branch, or is the only
> option for this version the next-2.6.36 branch?

No.  xen/next-2.6.36 doesn't exist any more, and it never really
contained anything useful.  xen/next-2.6.37, however, is closely
tracking upstream and I hope will reach functional parity with the
2.6.32-based branches.

>
> In the past Jeremy has posted details on the state and forward goals
> for the major consolidation branches.
> I haven't seen any such details regarding the .36/.37 branches.
> Does any one know what the plans are?

I'll post something the next week or so when we work out what needs to
happen with .37 to get it up to speed.

>
> We've been hanging off of Konrad's  stable/xen-pcifront-0.8.2 branch
> with good results. I'm wondering what our options are now with the
> recent merge storm.

That has been merged into mainline Linux.

    J


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 2492 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

On Tue, Nov 09, 2010 at 03:20:13AM -0800, Boris Derzhavets wrote:
> > No.  xen/next-2.6.36 doesn't exist any more, and it never really
> > contained anything useful.  xen/next-2.6.37, however, is closely
> > tracking upstream and I hope will reach functional parity with the
> > 2.6.32-based branches.
> 
> Is 2.6.38 merge window right estimate for time frame to reach functional 
> parity with the 2.6.32-based branches ?

Not exactly. We are aiming for the basic functionality for running a PV guest
in that timeframe. Right now, you probably can run an HVM guests, thought we
have been so focused on shaking out bringup bugs that it has not been on our
list of high priorities.

However, something which I did not know until last week, is that there is a
blkback and netback built in QEMU. Which means you could boot up a PV guest
with the blkback/netback utilizing the QEMU back-ends - how well they do, I don't
know. I am not even sure how to utilize it - it might be worth digging in that
and finding out how well it works?

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1768 bytes --]

> Right now, you probably can run an HVM guests

Attempt to create HVM via virt-install causes kernel crash (2.6.37-rc1)

Boris.

--- On Tue, 11/9/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Tuesday, November 9, 2010, 10:00 AM

On Tue, Nov 09, 2010 at 03:20:13AM -0800, Boris Derzhavets wrote:
> > No.  xen/next-2.6.36 doesn't exist any more, and it never really
> > contained anything useful.  xen/next-2.6.37, however, is closely
> > tracking upstream and I hope will reach functional parity with the
> > 2.6.32-based branches.
> 
> Is 2.6.38 merge window right estimate for time frame to reach functional 
> parity with the 2.6.32-based branches ?

Not exactly. We are aiming for the basic functionality for running a PV guest
in that timeframe. Right now, you probably can run an HVM guests, thought we
have been so focused on shaking out bringup bugs that it has not been on our
list of high priorities.

However, something which I did not know until last week, is that there is a
blkback and netback built in QEMU. Which means you could boot up a PV guest
with the blkback/netback utilizing the QEMU back-ends - how well they do, I don't
know. I am not even sure how to utilize it - it might be worth digging in that
and finding out how well it works?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 2380 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 2059 bytes --]

On Mon, Nov 8, 2010 at 11:14 AM, Jeremy Fitzhardinge <jeremy@goop.org>wrote:

> On 11/08/2010 10:19 AM, Bruce Edge wrote:
> > Is there now or will there be a 2.6.36 stable branch, or is the only
> > option for this version the next-2.6.36 branch?
>
> No.  xen/next-2.6.36 doesn't exist any more, and it never really
> contained anything useful.  xen/next-2.6.37, however, is closely
> tracking upstream and I hope will reach functional parity with the
> 2.6.32-based branches.
>
> >
> > In the past Jeremy has posted details on the state and forward goals
> > for the major consolidation branches.
> > I haven't seen any such details regarding the .36/.37 branches.
> > Does any one know what the plans are?
>
> I'll post something the next week or so when we work out what needs to
> happen with .37 to get it up to speed.
>
> >
> > We've been hanging off of Konrad's  stable/xen-pcifront-0.8.2 branch
> > with good results. I'm wondering what our options are now with the
> > recent merge storm.
>
> That has been merged into mainline Linux.
>

I've tried the mainline 2.6.37-rc1 branch and it's highly unstable as a
pvops domU with PCI passthough.
It's not a patch on konrad's stable/xen-pcifront-0.8.2 branch which was
looking very good around the 10/21 timeframe, but which has also developed
it's own set of instabilities recently.
I haven't reported any of the symptoms as I thought I'd let the dist settle
before starting the bug reporting.

Is there some other branch that developers working on PCI device drivers in
a pvops domU should be following?

I realize that we're tracking unstable trees, but it's either that or hvm.
We'd like to stick with the benefits that pvops provides, but I'm running
out of branches to try.

The last stable domU I built was:
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git -
stable/xen-pcifront-0.8.2
 8029255fcc3a7925ad8ee9f5fdc0764b6662301e
Thu Oct 21 09:36:07 2010 -0400

Do we need to stick with this for a while until the mainline or some other
xen/pvops branch stabilizes?

-Bruce


>
>    J
>
>

[-- Attachment #1.2: Type: text/html, Size: 2885 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Jeremy Fitzhardinge]

On 11/10/2010 10:17 AM, Bruce Edge wrote:
> I've tried the mainline 2.6.37-rc1 branch and it's highly unstable as
> a pvops domU with PCI passthough.
> It's not a patch on konrad's stable/xen-pcifront-0.8.2 branch which
> was looking very good around the 10/21 timeframe, but which has also
> developed it's own set of instabilities recently.
> I haven't reported any of the symptoms as I thought I'd let the dist
> settle before starting the bug reporting.

Ah, no, please report bugs, the earlier the better.  We need to get
bugfixes into mainline ASAP to make sure that .37 is a useful release,
at least within its expected range of functionality.  (Linus is
generally unhappy about non-regression bugfixes beyond about -rc3 -
bugfixes for a new feature indicate that the feature wasn't ready for
upstream and should simply be disabled or reverted for that release.)

    J

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

> It's not a patch on konrad's stable/xen-pcifront-0.8.2 branch which was
> looking very good around the 10/21 timeframe, but which has also developed
> it's own set of instabilities recently.

Oooh, share please.

> I haven't reported any of the symptoms as I thought I'd let the dist settle
> before starting the bug reporting.
> 
> Is there some other branch that developers working on PCI device drivers in
> a pvops domU should be following?

Well, I am working on a PCI E820 hole thingie, but those patches are in the
infancy period. There are some fixes:

stable/xen-pcifront-fixes

but they aren't that .. sophisticated.
> 
> I realize that we're tracking unstable trees, but it's either that or hvm.
> We'd like to stick with the benefits that pvops provides, but I'm running
> out of branches to try.
> 
> The last stable domU I built was:
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git -
> stable/xen-pcifront-0.8.2
>  8029255fcc3a7925ad8ee9f5fdc0764b6662301e

Hmm, so the difference between that and the upstream is the PVonHVM and
the initial domain 0 support (all in Stefano's tree).

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 1396 bytes --]

On Wed, Nov 10, 2010 at 11:43 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:

> > It's not a patch on konrad's stable/xen-pcifront-0.8.2 branch which was
> > looking very good around the 10/21 timeframe, but which has also
> developed
> > it's own set of instabilities recently.
>
> Oooh, share please.
>
> > I haven't reported any of the symptoms as I thought I'd let the dist
> settle
> > before starting the bug reporting.
> >
> > Is there some other branch that developers working on PCI device drivers
> in
> > a pvops domU should be following?
>
> Well, I am working on a PCI E820 hole thingie, but those patches are in the
> infancy period. There are some fixes:
>
> stable/xen-pcifront-fixes
>
> but they aren't that .. sophisticated.
> >
> > I realize that we're tracking unstable trees, but it's either that or
> hvm.
> > We'd like to stick with the benefits that pvops provides, but I'm running
> > out of branches to try.
> >
> > The last stable domU I built was:
> > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git -
> > stable/xen-pcifront-0.8.2
> >  8029255fcc3a7925ad8ee9f5fdc0764b6662301e
>
> Hmm, so the difference between that and the upstream is the PVonHVM and
> the initial domain 0 support (all in Stefano's tree).
>

I'll try disabling CONFIG_XEN_PVHVM and see if that helps.

Either way I'll start filing the problems I'm seeing.

Thanks

-Bruce

[-- Attachment #1.2: Type: text/html, Size: 2027 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 1536 bytes --]

On Wed, Nov 10, 2010 at 11:43 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:

> > It's not a patch on konrad's stable/xen-pcifront-0.8.2 branch which was
> > looking very good around the 10/21 timeframe, but which has also
> developed
> > it's own set of instabilities recently.
>
> Oooh, share please.
>
> > I haven't reported any of the symptoms as I thought I'd let the dist
> settle
> > before starting the bug reporting.
> >
> > Is there some other branch that developers working on PCI device drivers
> in
> > a pvops domU should be following?
>
> Well, I am working on a PCI E820 hole thingie,


If this for the 4GB mem limit on pvops domUs?



> but those patches are in the
> infancy period. There are some fixes:
>
> stable/xen-pcifront-fixes
>
> but they aren't that .. sophisticated.
> >
> > I realize that we're tracking unstable trees, but it's either that or
> hvm.
> > We'd like to stick with the benefits that pvops provides, but I'm running
> > out of branches to try.
> >
> > The last stable domU I built was:
> > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git -
> > stable/xen-pcifront-0.8.2
> >  8029255fcc3a7925ad8ee9f5fdc0764b6662301e
>
> Hmm, so the difference between that and the upstream is the PVonHVM and
> the initial domain 0 support (all in Stefano's tree).
>

I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
mainline domU - BUG: unable to handle kernel paging request" thread.
Is that sufficient to eliminate the PnonHVM from the equation?

-Bruce

[-- Attachment #1.2: Type: text/html, Size: 2514 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

> > Well, I am working on a PCI E820 hole thingie,
> 
> 
> If this for the 4GB mem limit on pvops domUs?

Yup. I got it to boot nicely yesterday too. The patches are in
devel/e820-hole, _but_ you need patches for the toolstack. Those
aren't ready yet (they are quite skanky right now).

> > Hmm, so the difference between that and the upstream is the PVonHVM and
> > the initial domain 0 support (all in Stefano's tree).
> >
> 
> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
> mainline domU - BUG: unable to handle kernel paging request" thread.
> Is that sufficient to eliminate the PnonHVM from the equation?

Yes should be. And based on your output it doesn't seem to make much difference.

I think bisecting the code and just running as DomU PV (without PCI passthrough)
and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
same or similar hang can help us a bit. If you can get hold of Boris you could
both coordinate this?

I've got to fix this AMD bootup hang, but once I am done I can switch over to this,
unless somebody else picks it up (Jeremy, Stefano, Ian?)

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

On Nov 11, 2010, at 8:25 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

>>> Well, I am working on a PCI E820 hole thingie,
>> 
>> 
>> If this for the 4GB mem limit on pvops domUs?
> 
> Yup. I got it to boot nicely yesterday too. The patches are in
> devel/e820-hole, _but_ you need patches for the toolstack. Those
> aren't ready yet (they are quite skanky right now).

Fantastic. I'm looking forward to this.
> 
>>> Hmm, so the difference between that and the upstream is the PVonHVM and
>>> the initial domain 0 support (all in Stefano's tree).
>>> 
>> 
>> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
>> mainline domU - BUG: unable to handle kernel paging request" thread.
>> Is that sufficient to eliminate the PnonHVM from the equation?
> 
> Yes should be. And based on your output it doesn't seem to make much difference.
> 
> I think bisecting the code and just running as DomU PV (without PCI passthrough)
> and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
> same or similar hang can help us a bit. If you can get hold of Boris you could
> both coordinate this?

Any particular pvops branch, or mainline Linux?

-Bruce
> 
> I've got to fix this AMD bootup hang, but once I am done I can switch over to this,
> unless somebody else picks it up (Jeremy, Stefano, Ian?)

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

> > I think bisecting the code and just running as DomU PV (without PCI passthrough)
> > and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
> > same or similar hang can help us a bit. If you can get hold of Boris you could
> > both coordinate this?
> 
> Any particular pvops branch, or mainline Linux?

Mainline please.

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 2171 bytes --]

Bruce,

 I believe , Konrad wants us to test vanilla 2.6.36(5)(4) for the issue with NFS folder.

Boris


--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>
Date: Thursday, November 11, 2010, 11:48 AM



On Nov 11, 2010, at 8:25 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

>>> Well, I am working on a PCI E820 hole thingie,
>> 
>> 
>> If this for the 4GB mem limit on pvops domUs?
> 
> Yup. I got it to boot nicely yesterday too. The patches are in
> devel/e820-hole, _but_ you need patches for the toolstack. Those
> aren't ready yet (they are quite skanky right now).

Fantastic. I'm looking forward to this.
> 
>>> Hmm, so the difference between that and the upstream is the PVonHVM and
>>> the initial domain 0 support (all in Stefano's tree).
>>> 
>> 
>> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
>> mainline domU - BUG: unable to handle kernel paging request" thread.
>> Is that sufficient to eliminate the PnonHVM from the equation?
> 
> Yes should be. And based on your output it doesn't seem to make much difference.
> 
> I think bisecting the code and just running as DomU PV (without PCI passthrough)
> and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
> same or similar hang can help us a bit. If you can get hold of Boris you could
> both coordinate this?

Any particular pvops branch, or mainline Linux?

-Bruce
> 
> I've got to fix this AMD bootup hang, but once I am done I can switch over to this,
> unless somebody else picks it up (Jeremy, Stefano, Ian?)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 3016 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 2882 bytes --]

On Thu, Nov 11, 2010 at 10:09 AM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:

> Bruce,
>
>  I believe , Konrad wants us to test vanilla 2.6.36(5)(4) for the issue
> with NFS folder.
>

I built a mainline 2.6.36 and it's not crashing with the same config.
Of course it doesn't have pci passthrough as it's not available in that
kernel.

So that puts it in the 36 -> 37-rc1 region. Is there any reason to test
anything before 2.6.36?

I'll reboot it a few more times to confirm that it's stable.

-Bruce

>
> Boris
>
>
> --- On *Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
>
> Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options,
> stable/next/etc?
> To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, "
> xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "
> ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <
> stefano.stabellini@eu.citrix.com>
> Date: Thursday, November 11, 2010, 11:48 AM
>
>
>
> On Nov 11, 2010, at 8:25 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>>
> wrote:
>
> >>> Well, I am working on a PCI E820 hole thingie,
> >>
> >>
> >> If this for the 4GB mem limit on pvops domUs?
> >
> > Yup. I got it to boot nicely yesterday too. The patches are in
> > devel/e820-hole, _but_ you need patches for the toolstack. Those
> > aren't ready yet (they are quite skanky right now).
>
> Fantastic. I'm looking forward to this.
> >
> >>> Hmm, so the difference between that and the upstream is the PVonHVM and
> >>> the initial domain 0 support (all in Stefano's tree).
> >>>
> >>
> >> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the
> "2.6.37-rc1
> >> mainline domU - BUG: unable to handle kernel paging request" thread.
> >> Is that sufficient to eliminate the PnonHVM from the equation?
> >
> > Yes should be. And based on your output it doesn't seem to make much
> difference.
> >
> > I think bisecting the code and just running as DomU PV (without PCI
> passthrough)
> > and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you
> get the
> > same or similar hang can help us a bit. If you can get hold of Boris you
> could
> > both coordinate this?
>
> Any particular pvops branch, or mainline Linux?
>
> -Bruce
> >
> > I've got to fix this AMD bootup hang, but once I am done I can switch
> over to this,
> > unless somebody else picks it up (Jeremy, Stefano, Ian?)
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 4979 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 3532 bytes --]


> Of course it doesn't have pci passthrough as it's not available in that kernel

I believe, this feature (per Konrad)  is not core reason of crash .37-rc1 

Boris.


--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>
Date: Thursday, November 11, 2010, 1:32 PM

On Thu, Nov 11, 2010 at 10:09 AM, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

Bruce,

 I believe , Konrad wants us to test vanilla 2.6.36(5)(4) for the issue with NFS folder.


I built a mainline 2.6.36 and it's not crashing with the same config.
Of course it doesn't have pci passthrough as it's not available in that kernel. 

So that puts it in the 36 -> 37-rc1 region. Is there any reason to test anything before 2.6.36?


I'll reboot it a few more times to confirm that it's stable.

-Bruce


Boris


--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:


From: Bruce Edge <bruce.edge@gmail.com>

Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>

Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>

Date: Thursday, November 11, 2010, 11:48 AM



On Nov 11, 2010, at 8:25
 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

>>> Well, I am working on a PCI E820 hole thingie,
>> 

>> 
>> If this for the 4GB mem limit on pvops domUs?
> 
> Yup. I got it to boot nicely yesterday too. The patches are in
> devel/e820-hole, _but_ you need patches for the toolstack. Those

> aren't ready yet (they are quite skanky right now).

Fantastic. I'm looking forward to this.
> 
>>> Hmm, so the difference between that and the upstream is the PVonHVM and
>>> the initial domain 0 support (all in Stefano's tree).

>>> 
>> 
>> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
>> mainline domU - BUG: unable to handle kernel paging request" thread.
>> Is that
 sufficient to eliminate the PnonHVM from the equation?
> 
> Yes should be. And based on your output it doesn't seem to make much difference.
> 
> I think bisecting the code and just running as DomU PV (without PCI passthrough)

> and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
> same or similar hang can help us a bit. If you can get hold of Boris you could
> both coordinate this?

Any particular pvops branch, or mainline Linux?


-Bruce
> 
> I've got to fix this AMD bootup hang, but once I am done I can switch over to this,
> unless somebody else picks it up (Jeremy, Stefano, Ian?)

_______________________________________________

Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel










      
_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel







      

[-- Attachment #1.2: Type: text/html, Size: 6874 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 3571 bytes --]

Konrad,

2.6.35.8 (mainline) works as PV DomU kernel ( at Xen 4.0.1 Dom0)  with NFS remote folder  with no issues. So,  2.6.35.8, 2.6.36 been built via upstream source are OK. Seems  to be .37-rc1 bug

Boris

--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>
Date: Thursday, November 11, 2010, 1:32 PM

On Thu, Nov 11, 2010 at 10:09 AM, Boris Derzhavets <bderzhavets@yahoo.com> wrote:

Bruce,

 I believe , Konrad wants us to test vanilla 2.6.36(5)(4) for the issue with NFS folder.


I built a mainline 2.6.36 and it's not crashing with the same config.
Of course it doesn't have pci passthrough as it's not available in that kernel. 

So that puts it in the 36 -> 37-rc1 region. Is there any reason to test anything before 2.6.36?


I'll reboot it a few more times to confirm that it's stable.

-Bruce


Boris


--- On Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com> wrote:


From: Bruce Edge <bruce.edge@gmail.com>

Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options, stable/next/etc?
To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>

Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <ian.campbell@citrix.com>, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com>

Date: Thursday, November 11, 2010, 11:48 AM



On Nov 11, 2010, at 8:25
 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

>>> Well, I am working on a PCI E820 hole thingie,
>> 

>> 
>> If this for the 4GB mem limit on pvops domUs?
> 
> Yup. I got it to boot nicely yesterday too. The patches are in
> devel/e820-hole, _but_ you need patches for the toolstack. Those

> aren't ready yet (they are quite skanky right now).

Fantastic. I'm looking forward to this.
> 
>>> Hmm, so the difference between that and the upstream is the PVonHVM and
>>> the initial domain 0 support (all in Stefano's tree).

>>> 
>> 
>> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the "2.6.37-rc1
>> mainline domU - BUG: unable to handle kernel paging request" thread.
>> Is that
 sufficient to eliminate the PnonHVM from the equation?
> 
> Yes should be. And based on your output it doesn't seem to make much difference.
> 
> I think bisecting the code and just running as DomU PV (without PCI passthrough)

> and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you get the
> same or similar hang can help us a bit. If you can get hold of Boris you could
> both coordinate this?

Any particular pvops branch, or mainline Linux?


-Bruce
> 
> I've got to fix this AMD bootup hang, but once I am done I can switch over to this,
> unless somebody else picks it up (Jeremy, Stefano, Ian?)

_______________________________________________

Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel










      
_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com

http://lists.xensource.com/xen-devel







      

[-- Attachment #1.2: Type: text/html, Size: 6928 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 4678 bytes --]

On Thu, Nov 11, 2010 at 12:03 PM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:

> Konrad,
>
> 2.6.35.8 (mainline) works as PV DomU kernel ( at Xen 4.0.1 Dom0)  with NFS
> remote folder  with no issues. So,  2.6.35.8, 2.6.36 been built via upstream
> source are OK. Seems  to be .37-rc1 bug
>
>
I agree. I reinstalled a 2.6.36 domU and am unable to get it to crash using
the same actions that reliably crashed the 2.6.37-rc1 domU.

Would it be worth bisecting the konrad-pcifront-2.6.32 branch to find which
it started failing?
I think that 8029255fcc3a7925ad8ee9f5fdc0764b6662301e (on 10/24) was the
last good sync that I got.

-Bruce


>
> Boris
>
> --- On *Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options,
> stable/next/etc?
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, "Jeremy
> Fitzhardinge" <jeremy@goop.org>, "xen-devel@lists.xensource.com" <
> xen-devel@lists.xensource.com>, "ian.campbell@citrix.com" <
> ian.campbell@citrix.com>, "Stefano Stabellini" <
> stefano.stabellini@eu.citrix.com>
> Date: Thursday, November 11, 2010, 1:32 PM
>
>
> On Thu, Nov 11, 2010 at 10:09 AM, Boris Derzhavets <bderzhavets@yahoo.com<http://mc/compose?to=bderzhavets@yahoo.com>
> > wrote:
>
> Bruce,
>
>  I believe , Konrad wants us to test vanilla 2.6.36(5)(4) for the issue
> with NFS folder.
>
>
> I built a mainline 2.6.36 and it's not crashing with the same config.
> Of course it doesn't have pci passthrough as it's not available in that
> kernel.
>
> So that puts it in the 36 -> 37-rc1 region. Is there any reason to test
> anything before 2.6.36?
>
> I'll reboot it a few more times to confirm that it's stable.
>
> -Bruce
>
>
> Boris
>
>
> --- On *Thu, 11/11/10, Bruce Edge <bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com<http://mc/compose?to=bruce.edge@gmail.com>
> >
>
> Subject: Re: [Xen-devel] Re: pvops kernel branches - 2.6.36 options,
> stable/next/etc?
> To: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>
> >
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org<http://mc/compose?to=jeremy@goop.org>>,
> "xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>"
> <xen-devel@lists.xensource.com<http://mc/compose?to=xen-devel@lists.xensource.com>>,
> "ian.campbell@citrix.com <http://mc/compose?to=ian.campbell@citrix.com>" <
> ian.campbell@citrix.com <http://mc/compose?to=ian.campbell@citrix.com>>,
> "Stefano Stabellini" <stefano.stabellini@eu.citrix.com<http://mc/compose?to=stefano.stabellini@eu.citrix.com>
> >
> Date: Thursday, November 11, 2010, 11:48 AM
>
>
>
> On Nov 11, 2010, at 8:25 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com<http://mc/compose?to=konrad.wilk@oracle.com>>
> wrote:
>
> >>> Well, I am working on a PCI E820 hole thingie,
> >>
> >>
> >> If this for the 4GB mem limit on pvops domUs?
> >
> > Yup. I got it to boot nicely yesterday too. The patches are in
> > devel/e820-hole, _but_ you need patches for the toolstack. Those
> > aren't ready yet (they are quite skanky right now).
>
> Fantastic. I'm looking forward to this.
> >
> >>> Hmm, so the difference between that and the upstream is the PVonHVM and
> >>> the initial domain 0 support (all in Stefano's tree).
> >>>
> >>
> >> I tried disabling the  CONFIG_XEN_PVHVM with no change. (see the
> "2.6.37-rc1
> >> mainline domU - BUG: unable to handle kernel paging request" thread.
> >> Is that sufficient to eliminate the PnonHVM from the equation?
> >
> > Yes should be. And based on your output it doesn't seem to make much
> difference.
> >
> > I think bisecting the code and just running as DomU PV (without PCI
> passthrough)
> > and trying different kernels: 2.6.36, 2.6.35, 2.6.34 and seeing if you
> get the
> > same or similar hang can help us a bit. If you can get hold of Boris you
> could
> > both coordinate this?
>
> Any particular pvops branch, or mainline Linux?
>
> -Bruce
> >
> > I've got to fix this AMD bootup hang, but once I am done I can switch
> over to this,
> > unless somebody else picks it up (Jeremy, Stefano, Ian?)
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 8244 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

On Thu, Nov 11, 2010 at 12:20:25PM -0800, Bruce Edge wrote:
> On Thu, Nov 11, 2010 at 12:03 PM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:
> 
> > Konrad,
> >
> > 2.6.35.8 (mainline) works as PV DomU kernel ( at Xen 4.0.1 Dom0)  with NFS
> > remote folder  with no issues. So,  2.6.35.8, 2.6.36 been built via upstream
> > source are OK. Seems  to be .37-rc1 bug
> >
> >
> I agree. I reinstalled a 2.6.36 domU and am unable to get it to crash using
> the same actions that reliably crashed the 2.6.37-rc1 domU.
> 
> Would it be worth bisecting the konrad-pcifront-2.6.32 branch to find which
> it started failing?

I think you meant another branch....but I don't believe the problem is related
to the pcifront. Boris was able to trigger this without using the XenPCI front.

It might make more sense to bisect from 2.6.36 through 2.6.36-rc1 timeframe.
During the merge window time when a lot of patches went in. It seems to point
to network since that is how you guys are triggering it..

> I think that 8029255fcc3a7925ad8ee9f5fdc0764b6662301e (on 10/24) was the
> last good sync that I got.

Ah, that would be the stable/xen-pcifront-0.8.1 tree. And it was based off
v2.6.36-rc7. There is also the stable/xen-pcifront-0.8.2 tree 
(5bba6c56dc99ff88f79a79572e29ecf445710878) - which for some mysterious
reasons on git.kernel.org was tracking the #master branch, so it ended up
pointing to stable/xen-pcifront-fixes.. I fixed it, but that one (xen-pcifront-0.8.2)
is based on v2.6.36-rc6, so even older (that was required as it was based on sparse_irq
rework).

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Bruce Edge]

On Thu, Nov 11, 2010 at 1:11 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Thu, Nov 11, 2010 at 12:20:25PM -0800, Bruce Edge wrote:
>> On Thu, Nov 11, 2010 at 12:03 PM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:
>>
>> > Konrad,
>> >
>> > 2.6.35.8 (mainline) works as PV DomU kernel ( at Xen 4.0.1 Dom0)  with NFS
>> > remote folder  with no issues. So,  2.6.35.8, 2.6.36 been built via upstream
>> > source are OK. Seems  to be .37-rc1 bug
>> >
>> >
>> I agree. I reinstalled a 2.6.36 domU and am unable to get it to crash using
>> the same actions that reliably crashed the 2.6.37-rc1 domU.
>>
>> Would it be worth bisecting the konrad-pcifront-2.6.32 branch to find which
>> it started failing?
>
> I think you meant another branch....but I don't believe the problem is related
> to the pcifront. Boris was able to trigger this without using the XenPCI front.

Right, good point. I fell back to that branch as we'd been running OK
with it for a while.

>
> It might make more sense to bisect from 2.6.36 through 2.6.36-rc1 timeframe.

I'm assuming 37-rc1 not 36-rc1.

> During the merge window time when a lot of patches went in. It seems to point
> to network since that is how you guys are triggering it..
>
>> I think that 8029255fcc3a7925ad8ee9f5fdc0764b6662301e (on 10/24) was the
>> last good sync that I got.
>
> Ah, that would be the stable/xen-pcifront-0.8.1 tree. And it was based off
> v2.6.36-rc7. There is also the stable/xen-pcifront-0.8.2 tree
> (5bba6c56dc99ff88f79a79572e29ecf445710878) - which for some mysterious
> reasons on git.kernel.org was tracking the #master branch, so it ended up
> pointing to stable/xen-pcifront-fixes.. I fixed it, but that one (xen-pcifront-0.8.2)
> is based on v2.6.36-rc6, so even older (that was required as it was based on sparse_irq
> rework).

I'll look for it in the mainline.

-Bruce

>
>

Re: Re: pvops kernel branches - 2.6.36 options, stable/next/etc?

[Konrad Rzeszutek Wilk]

> > It might make more sense to bisect from 2.6.36 through 2.6.36-rc1 timeframe.
> 
> I'm assuming 37-rc1 not 36-rc1.

Yes!