xen-unstable on OL6 (RHEL6 clone) problems

xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

Due to some other commitments, it's been awhile since I've built
a xen-unstable-based system, and with Oracle Linux 6 (a RHEL6
clone) just released and with Xen 4.1 in release candidates, I
decided to install and test on a new machine.

As I sort-of expected, it's not going real well due to tools
changes that I don't understand how to get around or fix.  I'm
not a sysadmin guru so some or all of the problems may be
trivial, and various googling helped a bit.

One that I can't figure out is when I try to start xend,
I get a long sequence of:

/sys/class/net/virbr0/bridge: No such file or directory
/sys/class/net/virbr0/bridge: No such file or directory
/sys/class/net/virbr0/bridge: No such file or directory
/sys/class/net/virbr0/bridge: No such file or directory

and networking in a guest fails to work.  This directory
DOES exist.  And "brctl show" seems to be the source of
those messages.

In one semi-related googled link, I found that libvirtd
may cause some problems, so I disabled that.  No luck,
but the messages change to:

/sys/class/net/tmpbridge/bridge: No such file or directory

and starting xend caused my dom0 networking to go down.

Any clues?  I'm guessing that I have to manually set up some
network bridges that I never had to deal with before, but
don't have any idea how to do that.

Thanks,
Dan

P.S. Don't know if it is related, but I see in dmesg several:
	XENBUS: Unable to read cpu state
and a

virbr0: starting userspace STP failed, starting kernel STP

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Wed, Feb 9, 2011 at 5:26 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
> Due to some other commitments, it's been awhile since I've built
> a xen-unstable-based system, and with Oracle Linux 6 (a RHEL6
> clone) just released and with Xen 4.1 in release candidates, I
> decided to install and test on a new machine.
>
> As I sort-of expected, it's not going real well due to tools
> changes that I don't understand how to get around or fix.  I'm
> not a sysadmin guru so some or all of the problems may be
> trivial, and various googling helped a bit.
>
> One that I can't figure out is when I try to start xend,
> I get a long sequence of:
>
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
>
> and networking in a guest fails to work.  This directory
> DOES exist.  And "brctl show" seems to be the source of
> those messages.
>
> In one semi-related googled link, I found that libvirtd
> may cause some problems, so I disabled that.  No luck,
> but the messages change to:
>
> /sys/class/net/tmpbridge/bridge: No such file or directory
>
> and starting xend caused my dom0 networking to go down.
>
> Any clues?  I'm guessing that I have to manually set up some
> network bridges that I never had to deal with before, but
> don't have any idea how to do that.
>

Disable bridging in the xend config and set up a xen bridge like this:
http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging

Then, just make sure your domU config files use the same bridge name
you created.

Hope that helps.

Thanks,
Todd

P.S. We have a work in progress wiki page to help you through the 4.1+ migration
http://wiki.xensource.com/xenwiki/MigrationGuideToXen4.1+

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> > Any clues?  I'm guessing that I have to manually set up some
> > network bridges that I never had to deal with before, but
> > don't have any idea how to do that.
> >
> 
> Disable bridging in the xend config and set up a xen bridge like this:
> http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
> 
> Then, just make sure your domU config files use the same bridge name
> you created.
> 
> Hope that helps.
> 
> Thanks,
> Todd
> 
> P.S. We have a work in progress wiki page to help you through the 4.1+
> migration
> http://wiki.xensource.com/xenwiki/MigrationGuideToXen4.1+


Thanks... I think.

I have to ask... this is progress?

<flame on>
Um, never mind, waste of breath
<flame off>

:-)
Dan

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> > > Any clues?  I'm guessing that I have to manually set up some
> > > network bridges that I never had to deal with before, but
> > > don't have any idea how to do that.
> > >
> >
> > Disable bridging in the xend config and set up a xen bridge like
> this:
> > http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
> >
> > Then, just make sure your domU config files use the same bridge name
> > you created.

Wait... I just realized... this means that there is NO backwards
compatibility of domU config files between 4.0 and 4.1?  EVERY
vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?

If this is true, /me raises a red flag on 4.1.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Wed, Feb 9, 2011 at 7:22 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
> Wait... I just realized... this means that there is NO backwards
> compatibility of domU config files between 4.0 and 4.1?  EVERY
> vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?
>
> If this is true, /me raises a red flag on 4.1.
>

There is almost full compatibility, with the exception of being able
to have arbitrary python code embedded. Most should work directly.
Name the bridge the same as before and it is likely to completely work
with most built to work with Xen 4.0

I'll be sure to clarify that a bit on the wiki page.

Thanks for the feedback.

Todd

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Fajar A. Nugraha]

On Thu, Feb 10, 2011 at 8:38 AM, Todd Deshane <deshantm@gmail.com> wrote:
> On Wed, Feb 9, 2011 at 7:22 PM, Dan Magenheimer
> <dan.magenheimer@oracle.com> wrote:
>> Wait... I just realized... this means that there is NO backwards
>> compatibility of domU config files between 4.0 and 4.1?  EVERY
>> vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?
>>
>> If this is true, /me raises a red flag on 4.1.
>>
>
> There is almost full compatibility, with the exception of being able
> to have arbitrary python code embedded. Most should work directly.
> Name the bridge the same as before and it is likely to completely work
> with most built to work with Xen 4.0
>
> I'll be sure to clarify that a bit on the wiki page.

Reading the wiki, isn't the change only required when using xl?
Shouldn't xm and xend work just as it has always been?

-- 
Fajar

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Wed, Feb 9, 2011 at 8:54 PM, Fajar A. Nugraha <list@fajar.net> wrote:
> On Thu, Feb 10, 2011 at 8:38 AM, Todd Deshane <deshantm@gmail.com> wrote:
>> On Wed, Feb 9, 2011 at 7:22 PM, Dan Magenheimer
>> <dan.magenheimer@oracle.com> wrote:
>>> Wait... I just realized... this means that there is NO backwards
>>> compatibility of domU config files between 4.0 and 4.1?  EVERY
>>> vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?
>>>
>>> If this is true, /me raises a red flag on 4.1.
>>>
>>
>> There is almost full compatibility, with the exception of being able
>> to have arbitrary python code embedded. Most should work directly.
>> Name the bridge the same as before and it is likely to completely work
>> with most built to work with Xen 4.0
>>
>> I'll be sure to clarify that a bit on the wiki page.
>
> Reading the wiki, isn't the change only required when using xl?
> Shouldn't xm and xend work just as it has always been?

xm and xend are still available (for now), but using xl and libxl is preferred.

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> From: Todd Deshane [mailto:deshantm@gmail.com]
> Sent: Wednesday, February 09, 2011 7:15 PM
> To: Fajar A. Nugraha
> Cc: Dan Magenheimer; xen-devel@lists.xensource.com; keir@xen.org
> Subject: Re: [Xen-devel] xen-unstable on OL6 (RHEL6 clone) problems
> 
> On Wed, Feb 9, 2011 at 8:54 PM, Fajar A. Nugraha <list@fajar.net>
> wrote:
> > On Thu, Feb 10, 2011 at 8:38 AM, Todd Deshane <deshantm@gmail.com>
> wrote:
> >>
> >> There is almost full compatibility, with the exception of being able
> >> to have arbitrary python code embedded. Most should work directly.
> >> Name the bridge the same as before and it is likely to completely
> work
> >> with most built to work with Xen 4.0
> >>
> >> I'll be sure to clarify that a bit on the wiki page.
> >
> > Reading the wiki, isn't the change only required when using xl?
> > Shouldn't xm and xend work just as it has always been?
> 
> xm and xend are still available (for now), but using xl and libxl is
> preferred.

But the answer to Fajar's question is NO, correct?  This
bridging stuff is irrelevant to xm vs xl and must be done
even if you are using xm/xend on 4.1, correct?

BTW, I ran into this because "xl vncviewer" wasn't working
for me so I had to use "xm vncviewer" (which worked).

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Wed, Feb 9, 2011 at 9:38 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
>> From: Todd Deshane [mailto:deshantm@gmail.com]
>> Sent: Wednesday, February 09, 2011 7:15 PM
>> To: Fajar A. Nugraha
>> Cc: Dan Magenheimer; xen-devel@lists.xensource.com; keir@xen.org
>> Subject: Re: [Xen-devel] xen-unstable on OL6 (RHEL6 clone) problems
>>
>> On Wed, Feb 9, 2011 at 8:54 PM, Fajar A. Nugraha <list@fajar.net>
>> wrote:
>> > On Thu, Feb 10, 2011 at 8:38 AM, Todd Deshane <deshantm@gmail.com>
>> wrote:
>> >>
>> >> There is almost full compatibility, with the exception of being able
>> >> to have arbitrary python code embedded. Most should work directly.
>> >> Name the bridge the same as before and it is likely to completely
>> work
>> >> with most built to work with Xen 4.0
>> >>
>> >> I'll be sure to clarify that a bit on the wiki page.
>> >
>> > Reading the wiki, isn't the change only required when using xl?
>> > Shouldn't xm and xend work just as it has always been?
>>
>> xm and xend are still available (for now), but using xl and libxl is
>> preferred.
>
> But the answer to Fajar's question is NO, correct?

I believe the answer to his question should have been yes (xend/xm are
still there and should work as before (for now)). You may have just
come across a bug that was tested. xm is not automatically tested by
the automated testing system (example here:
http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/)

> bridging stuff is irrelevant to xm vs xl and must be done
> even if you are using xm/xend on 4.1, correct?
>

The old xen bridge code is currently still in the tree, but it is not
the preferred use.

> BTW, I ran into this because "xl vncviewer" wasn't working
> for me so I had to use "xm vncviewer" (which worked).
>

I believe there was a patch for this recently
see:
http://xen.markmail.org/search/?q=xl+vncviewer#query:xl%20vncviewer+page:1+mid:nxhvjkalvjn2iolk+state:results

Hope that helps.

Thanks,
Todd

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Thu, 2011-02-10 at 03:18 +0000, Todd Deshane wrote:
> On Wed, Feb 9, 2011 at 9:38 PM, Dan Magenheimer
> <dan.magenheimer@oracle.com> wrote:
> >> From: Todd Deshane [mailto:deshantm@gmail.com]
> >> Sent: Wednesday, February 09, 2011 7:15 PM
> >> To: Fajar A. Nugraha
> >> Cc: Dan Magenheimer; xen-devel@lists.xensource.com; keir@xen.org
> >> Subject: Re: [Xen-devel] xen-unstable on OL6 (RHEL6 clone) problems
> >>
> >> On Wed, Feb 9, 2011 at 8:54 PM, Fajar A. Nugraha <list@fajar.net>
> >> wrote:
> >> > On Thu, Feb 10, 2011 at 8:38 AM, Todd Deshane <deshantm@gmail.com>
> >> wrote:
> >> >>
> >> >> There is almost full compatibility, with the exception of being able
> >> >> to have arbitrary python code embedded. Most should work directly.
> >> >> Name the bridge the same as before and it is likely to completely
> >> work
> >> >> with most built to work with Xen 4.0
> >> >>
> >> >> I'll be sure to clarify that a bit on the wiki page.
> >> >
> >> > Reading the wiki, isn't the change only required when using xl?
> >> > Shouldn't xm and xend work just as it has always been?
> >>
> >> xm and xend are still available (for now), but using xl and libxl is
> >> preferred.
> >
> > But the answer to Fajar's question is NO, correct?
> 
> I believe the answer to his question should have been yes (xend/xm are
> still there and should work as before (for now)).

There is one subtle difference in 4.1 wrt xend/xm which is that the
network scripts now look to see if there are any existing bridges
configured and if there are then they do not blow away the existing
configuration and try to setup the network themselves (IOW they exit
without doing anything if they think someone has already done it).

The assumption is that if the admin has configured the system with a
bridge already then xend has no business blowing it away.

This also allows for smoother transitions back and forth between xm and
xl since it means you can safely make the necessary configuration
changes to use xl and not have to undo them if you switch back to
xm/xend, assuming you name your bridge in a compatible way.

 You may have just
> come across a bug that was tested. xm is not automatically tested by
                            ^not?
> the automated testing system (example here:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/)

I think xend is tested, AFAIK the sequences (columns in the above) which
do not explicitly say "xl" are all xend tests. It least one of them is:
http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/test-amd64-amd64-pair/info.html

Ian.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Jackson]

Todd Deshane writes ("Re: [Xen-devel] xen-unstable on OL6 (RHEL6 clone) problems"):
> I believe the answer to his question should have been yes (xend/xm are
> still there and should work as before (for now)). You may have just
> come across a bug that was tested. xm is not automatically tested by
> the automated testing system (example here:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/)

It is tested.  But the configuration where we let the xend startup
create the bridge is not (it was too fragile ...)

Ian.

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> >> > Reading the wiki, isn't the change only required when using xl?
> >> > Shouldn't xm and xend work just as it has always been?
> >>
> >> xm and xend are still available (for now), but using xl and libxl is
> >> preferred.
> >
> > But the answer to Fajar's question is NO, correct?
> 
> I believe the answer to his question should have been yes (xend/xm are
> still there and should work as before (for now)). You may have just
> come across a bug that was tested. xm is not automatically tested by
> the automated testing system (example here:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/)

OK, thanks.  Indeed, after setting up the bridging, I am continuing to
have the same problem ("brctl show" complains repeatedly) on a machine
with Xen installed, but the bridging works fine on the same OS (Oracle
Linux 6) without Xen installed.

Will follow up with more info when available.
 
> > bridging stuff is irrelevant to xm vs xl and must be done
> > even if you are using xm/xend on 4.1, correct?
> >
> 
> The old xen bridge code is currently still in the tree, but it is not
> the preferred use.

Is this commented out and easy to turn on?  Or maybe it's NOT
commented out and is causing the breakage I am seeing?

> > BTW, I ran into this because "xl vncviewer" wasn't working
> > for me so I had to use "xm vncviewer" (which worked).
> 
> I believe there was a patch for this recently
> see:
> http://xen.markmail.org/search/?q=xl+vncviewer#query:xl%20vncviewer+pag
> e:1+mid:nxhvjkalvjn2iolk+state:results

Excellent!  Pulled the latest bits and xl works now!  (Still
no networking in my guests though due to the above.)

Thanks very much for your help!
Dan

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Stefano Stabellini]

On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> > >> > Reading the wiki, isn't the change only required when using xl?
> > >> > Shouldn't xm and xend work just as it has always been?
> > >>
> > >> xm and xend are still available (for now), but using xl and libxl is
> > >> preferred.
> > >
> > > But the answer to Fajar's question is NO, correct?
> > 
> > I believe the answer to his question should have been yes (xend/xm are
> > still there and should work as before (for now)). You may have just
> > come across a bug that was tested. xm is not automatically tested by
> > the automated testing system (example here:
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/5680/)
> 
> OK, thanks.  Indeed, after setting up the bridging, I am continuing to
> have the same problem ("brctl show" complains repeatedly) on a machine
> with Xen installed, but the bridging works fine on the same OS (Oracle
> Linux 6) without Xen installed.
> 
> Will follow up with more info when available.
>  
> > > bridging stuff is irrelevant to xm vs xl and must be done
> > > even if you are using xm/xend on 4.1, correct?
> > >
> > 
> > The old xen bridge code is currently still in the tree, but it is not
> > the preferred use.
> 
> Is this commented out and easy to turn on?  Or maybe it's NOT
> commented out and is causing the breakage I am seeing?

The script is still installed but nobody is calling it.
If you are using xl you need to add

/etc/xen/scripts/network-bridge start

somewhere in your init scripts.

If you use xend you need to uncomment the line

# (network-script network-bridge)

in /etc/xen/xend-config.sxp.


It is worth mentioning this in the wiki so that people can still have
the old behaviour if they want to.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Thu, 10 Feb 2011, Dan Magenheimer wrote:
>> Is this commented out and easy to turn on?  Or maybe it's NOT
>> commented out and is causing the breakage I am seeing?
>
> The script is still installed but nobody is calling it.
> If you are using xl you need to add
>
> /etc/xen/scripts/network-bridge start
>
> somewhere in your init scripts.

With xl you need to do this?
So like in /etc/rc.local?

If that is the case, why don't we have some xen init script that loads
things like this?

>
> If you use xend you need to uncomment the line
>
> # (network-script network-bridge)
>
> in /etc/xen/xend-config.sxp.
>
>
> It is worth mentioning this in the wiki so that people can still have
> the old behaviour if they want to.
>
Added this second note. I'll add the other once I get
confirmation/clarification (from the above questions)

Thanks,
Todd

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Fri, 2011-02-11 at 03:09 +0000, Todd Deshane wrote: 
> On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> >> Is this commented out and easy to turn on?  Or maybe it's NOT
> >> commented out and is causing the breakage I am seeing?
> >
> > The script is still installed but nobody is calling it.
> > If you are using xl you need to add
> >
> > /etc/xen/scripts/network-bridge start
> >
> > somewhere in your init scripts.
> 
> With xl you need to do this?
> So like in /etc/rc.local?

This is a quick and dirty hack which brings back the old xend solution,
with all the brokenness and caveats described in IanJ's mail from last
night (it's worse than the previous xend solution if anything).

We should _not_ be recommending to users that they do this. Stefano,
please do not muddy the waters for people trying to upgrade by doing so.

The upgrade notes should continue to consistently point to the relevant
distro documentation and to provide associated guidance.

> If that is the case, why don't we have some xen init script that loads
> things like this?
> 
> >
> > If you use xend you need to uncomment the line
> >
> > # (network-script network-bridge)
> >
> > in /etc/xen/xend-config.sxp.
> >
> >
> > It is worth mentioning this in the wiki so that people can still have
> > the old behaviour if they want to.
> >
> Added this second note.

This option is not commented out in the default xend-config.sxp in the
xen-unstable.hg tree. There should be no need to uncomment it.

Ian.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Stefano Stabellini]

On Fri, 11 Feb 2011, Ian Campbell wrote:
> On Fri, 2011-02-11 at 03:09 +0000, Todd Deshane wrote: 
> > On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
> > <stefano.stabellini@eu.citrix.com> wrote:
> > > On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> > >> Is this commented out and easy to turn on?  Or maybe it's NOT
> > >> commented out and is causing the breakage I am seeing?
> > >
> > > The script is still installed but nobody is calling it.
> > > If you are using xl you need to add
> > >
> > > /etc/xen/scripts/network-bridge start
> > >
> > > somewhere in your init scripts.
> > 
> > With xl you need to do this?
> > So like in /etc/rc.local?
> 
> This is a quick and dirty hack which brings back the old xend solution,
> with all the brokenness and caveats described in IanJ's mail from last
> night (it's worse than the previous xend solution if anything).
> 
> We should _not_ be recommending to users that they do this. Stefano,
> please do not muddy the waters for people trying to upgrade by doing so.

In no way I meant to recommend this option to the users, but it might be
worth mentioning it in the wiki.


> > If that is the case, why don't we have some xen init script that loads
> > things like this?
> > 
> > >
> > > If you use xend you need to uncomment the line
> > >
> > > # (network-script network-bridge)
> > >
> > > in /etc/xen/xend-config.sxp.
> > >
> > >
> > > It is worth mentioning this in the wiki so that people can still have
> > > the old behaviour if they want to.
> > >
> > Added this second note.
> 
> This option is not commented out in the default xend-config.sxp in the
> xen-unstable.hg tree. There should be no need to uncomment it.
 
Ooops, sorry for the confusion! I double checked and Ian is right :-/

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Fri, 2011-02-11 at 11:44 +0000, Stefano Stabellini wrote:
> On Fri, 11 Feb 2011, Ian Campbell wrote:
> > On Fri, 2011-02-11 at 03:09 +0000, Todd Deshane wrote: 
> > > On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
> > > <stefano.stabellini@eu.citrix.com> wrote:
> > > > On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> > > >> Is this commented out and easy to turn on?  Or maybe it's NOT
> > > >> commented out and is causing the breakage I am seeing?
> > > >
> > > > The script is still installed but nobody is calling it.
> > > > If you are using xl you need to add
> > > >
> > > > /etc/xen/scripts/network-bridge start
> > > >
> > > > somewhere in your init scripts.
> > > 
> > > With xl you need to do this?
> > > So like in /etc/rc.local?
> > 
> > This is a quick and dirty hack which brings back the old xend solution,
> > with all the brokenness and caveats described in IanJ's mail from last
> > night (it's worse than the previous xend solution if anything).
> > 
> > We should _not_ be recommending to users that they do this. Stefano,
> > please do not muddy the waters for people trying to upgrade by doing so.
> 
> In no way I meant to recommend this option to the users, but it might be
> worth mentioning it in the wiki.

I don't think so.

The wiki should contains a clear statement of the correct and
recommended best practice.

The only thing worse for users than making a change like this and
failing to adequately document it would be to make a change and then
document it with a set of inconsistent and incompatible recommendations.

Ian.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Stefano Stabellini]

On Fri, 11 Feb 2011, Ian Campbell wrote:
> On Fri, 2011-02-11 at 11:44 +0000, Stefano Stabellini wrote:
> > On Fri, 11 Feb 2011, Ian Campbell wrote:
> > > On Fri, 2011-02-11 at 03:09 +0000, Todd Deshane wrote: 
> > > > On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
> > > > <stefano.stabellini@eu.citrix.com> wrote:
> > > > > On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> > > > >> Is this commented out and easy to turn on?  Or maybe it's NOT
> > > > >> commented out and is causing the breakage I am seeing?
> > > > >
> > > > > The script is still installed but nobody is calling it.
> > > > > If you are using xl you need to add
> > > > >
> > > > > /etc/xen/scripts/network-bridge start
> > > > >
> > > > > somewhere in your init scripts.
> > > > 
> > > > With xl you need to do this?
> > > > So like in /etc/rc.local?
> > > 
> > > This is a quick and dirty hack which brings back the old xend solution,
> > > with all the brokenness and caveats described in IanJ's mail from last
> > > night (it's worse than the previous xend solution if anything).
> > > 
> > > We should _not_ be recommending to users that they do this. Stefano,
> > > please do not muddy the waters for people trying to upgrade by doing so.
> > 
> > In no way I meant to recommend this option to the users, but it might be
> > worth mentioning it in the wiki.
> 
> I don't think so.
> 
> The wiki should contains a clear statement of the correct and
> recommended best practice.
> 
> The only thing worse for users than making a change like this and
> failing to adequately document it would be to make a change and then
> document it with a set of inconsistent and incompatible recommendations.

Ok

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> > The only thing worse for users than making a change like this and
> > failing to adequately document it would be to make a change and then
> > document it with a set of inconsistent and incompatible
> recommendations.

Well, there IS one thing worse, and that's documenting the
new way WRONG.  And that's what the current draft is.
The examples inconsistently use "br0" and "xenbr0".
All need to be one or the other.  While that may be obvious
to all of you networking alpha-geeks (humor intended), it
was not obvious to me... I assumed there was some Xen
magic that turned one into the other!

Further, I *think* xenbr0 is required, not br0.  Not sure,
but it worked for me as soon as I switched everything to
xenbr0.  If this is true, the libvirt wiki that Todd pointed
to is wrong for Xen also.

Finally, since John's post dropped the cc list and I'm not
sure everyone reads every xen-devel message, I'm repeating
that John found that my problem that started all this, namely
seeing this message many times:

/sys/class/net/br0/bridge: No such file or directory

is due to an old bridge package shipped with RHEL6 (and
clones such as OL6).  John's post is here and I can confirm
that I updated to the newer RPM and the messages went away.

http://lists.xensource.com/archives/html/xen-devel/2011-02/msg00725.html 

Thanks,
Dan

P.S. Thanks very much to everyone for contributions to the
migration wiki... I'm not trying to be critical at all, just
trying to provide corrections and debug one of the sources of
my frustration that led to my on-list tirade.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Fri, Feb 11, 2011 at 5:30 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
>> > The only thing worse for users than making a change like this and
>> > failing to adequately document it would be to make a change and then
>> > document it with a set of inconsistent and incompatible
>> recommendations.
>
> Well, there IS one thing worse, and that's documenting the
> new way WRONG.  And that's what the current draft is.
> The examples inconsistently use "br0" and "xenbr0".
> All need to be one or the other.  While that may be obvious
> to all of you networking alpha-geeks (humor intended), it
> was not obvious to me... I assumed there was some Xen
> magic that turned one into the other!
>
> Further, I *think* xenbr0 is required, not br0.  Not sure,
> but it worked for me as soon as I switched everything to
> xenbr0.  If this is true, the libvirt wiki that Todd pointed
> to is wrong for Xen also.
>
> Finally, since John's post dropped the cc list and I'm not
> sure everyone reads every xen-devel message, I'm repeating
> that John found that my problem that started all this, namely
> seeing this message many times:
>
> /sys/class/net/br0/bridge: No such file or directory
>
> is due to an old bridge package shipped with RHEL6 (and
> clones such as OL6).  John's post is here and I can confirm
> that I updated to the newer RPM and the messages went away.
>
> http://lists.xensource.com/archives/html/xen-devel/2011-02/msg00725.html
>
> Thanks,
> Dan
>
> P.S. Thanks very much to everyone for contributions to the
> migration wiki... I'm not trying to be critical at all, just
> trying to provide corrections and debug one of the sources of
> my frustration that led to my on-list tirade.
>

Thanks for pointing things out Dan. I think it is smarter to go with
xenbr0 consistently since that is what RHEL5 had.

I think I have now fixed the items that you pointed out.

Let us know if you find anything else.

Thanks,
Todd

Re: xen-unstable on OL6 (RHEL6 clone) problems

[John Haxby]

[-- Attachment #1.1: Type: text/plain, Size: 976 bytes --]


On 11 Feb 2011, at 17:30, Dan Magenheimer wrote:

> Finally, since John's post dropped the cc list and I'm not
> sure everyone reads every xen-devel message, I'm repeating
> that John found that my problem that started all this, namely
> seeing this message many times:
> 
> /sys/class/net/br0/bridge: No such file or directory
> 
> is due to an old bridge package shipped with RHEL6 (and
> clones such as OL6).  John's post is here and I can confirm
> that I updated to the newer RPM and the messages went away.
> 
> http://lists.xensource.com/archives/html/xen-devel/2011-02/msg00725.html 


I've kept the cc list this time :-)

RHEL6 has a _newer_ bridge utils , or, I should say, that it has stuff for IGMP snooping which isn't in the base version.

I would hope that by the time the IGMP snooping stuff makes it into the upstream bridge-utils that  it will be tolerant of kernels that don't have the corresponding support from the kernel.



jch

[-- Attachment #1.2: Type: text/html, Size: 2049 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: xen-unstable on OL6 (RHEL6 clone) problems

[John Haxby]

On 11/02/11 07:49, Ian Campbell wrote:
> The upgrade notes should continue to consistently point to the relevant
> distro documentation and to provide associated guidance.

I hesitate to just say "me too" but but Ian is dead right here.  The
examples in http://wiki.xensource.com/xenwiki/MigrationGuideToXen4.1+
that Dan has just referred me to are exactly what's needed here.  The
real documentation for RHEL et alia
(/usr/share/doc/initscripts-*/sysconfig.txt) are as close as you'll get
to definitive without reading the shell scripts, but the examples are
the perfect start.

While I'm talking about bridges though, RHEL6 (and clones including OL6)
have a problem with brctl.  RHEL6 adds IGMP snooping support for bridges
and this is provided by kernel patches.  If you're using the xen dom0
kernel this support is not present and brctl complains bitterly and
repeatedly that /sys/class/net/xenbr0/bridge (or similar) doesn't
exist.  What is actually means is that a various files in that directory
don't exist.   Luckily, Fedora 14's bridge-utils-1.2-9.fc13 RPM doesn't
expect the IGMP snooping stuff and works perfectly on the dom0 kernel.  
You could also rebuild the RHEL6 bridge-utils RPM and just miss out
bridge-utils-1.2-Add_IGMP_snooping_support.patch

jch

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Pasi Kärkkäinen]

On Fri, Feb 11, 2011 at 04:53:14PM +0000, John Haxby wrote:
> On 11/02/11 07:49, Ian Campbell wrote:
> > The upgrade notes should continue to consistently point to the relevant
> > distro documentation and to provide associated guidance.
> 
> I hesitate to just say "me too" but but Ian is dead right here.  The
> examples in http://wiki.xensource.com/xenwiki/MigrationGuideToXen4.1+
> that Dan has just referred me to are exactly what's needed here.  The
> real documentation for RHEL et alia
> (/usr/share/doc/initscripts-*/sysconfig.txt) are as close as you'll get
> to definitive without reading the shell scripts, but the examples are
> the perfect start.
> 
> While I'm talking about bridges though, RHEL6 (and clones including OL6)
> have a problem with brctl.  RHEL6 adds IGMP snooping support for bridges
> and this is provided by kernel patches.  If you're using the xen dom0
> kernel this support is not present and brctl complains bitterly and
> repeatedly that /sys/class/net/xenbr0/bridge (or similar) doesn't
> exist.  What is actually means is that a various files in that directory
> don't exist.   Luckily, Fedora 14's bridge-utils-1.2-9.fc13 RPM doesn't
> expect the IGMP snooping stuff and works perfectly on the dom0 kernel.  
> You could also rebuild the RHEL6 bridge-utils RPM and just miss out
> bridge-utils-1.2-Add_IGMP_snooping_support.patch
> 

.. and we need someone to start maintaining xen dom0 capable kernel based
on the el6 2.6.32 kernel.. so all the security fixes and extra features 
are there out-of-the-box :)

-- Pasi

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

On Fri, 11 Feb 2011, Todd Deshane wrote:
> On Thu, Feb 10, 2011 at 12:56 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Thu, 10 Feb 2011, Dan Magenheimer wrote:
> >> Is this commented out and easy to turn on?  Or maybe it's NOT
> >> commented out and is causing the breakage I am seeing?
> >
> > The script is still installed but nobody is calling it.
> > If you are using xl you need to add
> >
> > /etc/xen/scripts/network-bridge start
> >
> > somewhere in your init scripts.
> 
> With xl you need to do this?
> So like in /etc/rc.local?

Users don't need to do this: the recommended way of solving this problem
is using the distro's bridge scripts.


> If that is the case, why don't we have some xen init script that loads
> things like this?

This is only for people that wants the old behaviour.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Thu, 2011-02-10 at 01:38 +0000, Todd Deshane wrote:
> On Wed, Feb 9, 2011 at 7:22 PM, Dan Magenheimer
> <dan.magenheimer@oracle.com> wrote:
> > Wait... I just realized... this means that there is NO backwards
> > compatibility of domU config files between 4.0 and 4.1?  EVERY
> > vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?
> >
> > If this is true, /me raises a red flag on 4.1.
> >
> 
> There is almost full compatibility, with the exception of being able
> to have arbitrary python code embedded. Most should work directly.
> Name the bridge the same as before and it is likely to completely work
> with most built to work with Xen 4.0
> 
> I'll be sure to clarify that a bit on the wiki page.

Since it is one of the biggest changes it might be worth embedding a
simple example (i.e. the most basic possible) of the network
configuration required for each of the distro types, as well as
referencing the distro documentation.

A hint as to what bridge name(s) to prefer for best xend compatibility
would also be useful.

Ian.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Todd Deshane]

On Thu, Feb 10, 2011 at 8:32 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2011-02-10 at 01:38 +0000, Todd Deshane wrote:
>> On Wed, Feb 9, 2011 at 7:22 PM, Dan Magenheimer
>> <dan.magenheimer@oracle.com> wrote:
>> > Wait... I just realized... this means that there is NO backwards
>> > compatibility of domU config files between 4.0 and 4.1?  EVERY
>> > vm.cfg file MUST be changed when upgrading from 4.0 to 4.1?
>> >
>> > If this is true, /me raises a red flag on 4.1.
>> >
>>
>> There is almost full compatibility, with the exception of being able
>> to have arbitrary python code embedded. Most should work directly.
>> Name the bridge the same as before and it is likely to completely work
>> with most built to work with Xen 4.0
>>
>> I'll be sure to clarify that a bit on the wiki page.
>
> Since it is one of the biggest changes it might be worth embedding a
> simple example (i.e. the most basic possible) of the network
> configuration required for each of the distro types, as well as
> referencing the distro documentation.
>
> A hint as to what bridge name(s) to prefer for best xend compatibility
> would also be useful.
>

Thanks. I added this and the other xend-specific note.

Any other comments just let me know (or feel free to make edits)

Thanks,
Todd

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Wed, 2011-02-09 at 23:49 +0000, Dan Magenheimer wrote:
> > > Any clues?  I'm guessing that I have to manually set up some
> > > network bridges that I never had to deal with before, but
> > > don't have any idea how to do that.
> > >
> > 
> > Disable bridging in the xend config and set up a xen bridge like this:
> > http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging
> > 
> > Then, just make sure your domU config files use the same bridge name
> > you created.
> > 
> > Hope that helps.
> > 
> > Thanks,
> > Todd
> > 
> > P.S. We have a work in progress wiki page to help you through the 4.1+
> > migration
> > http://wiki.xensource.com/xenwiki/MigrationGuideToXen4.1+
> 
> 
> Thanks... I think.
> 
> I have to ask... this is progress?

Yes. We consider integrating with the distro's standard configuration
mechanisms/utilities/tools in a way which is consistent with each
distro's own documentation, instead of Xen rolling its own and blowing
away any network configuration the system administrator has performed,
to be progress.

> <flame on>
> Um, never mind, waste of breath
> <flame off>

If you have constructive comments to make, as opposed to vague allusions
to not liking some change or other and snide comments, we would be happy
to hear them.

Ian.

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> > I have to ask... this is progress?
> 
> Yes. We consider integrating with the distro's standard configuration
> mechanisms/utilities/tools in a way which is consistent with each
> distro's own documentation, instead of Xen rolling its own and blowing
> away any network configuration the system administrator has performed,
> to be progress.

Hi Ian --

While I agree with your good intentions, I think what happened is
that you broke something that was working fine for some subset of
users/developers (RH/Fedora0-based) in order to unbreak something
for another subset (Debian-based).

In the Xen upstream development community, that may be a fine
tradeoff because of the high sysadmin knowledge and the high
percentage of developers using a "bleeding edge" distro.

I suspect that it will be much more of a problem for a released
Xen where a much higher percentage of people installing Xen
are less savvy and prefer stable distros.
 
> > <flame on>
> > Um, never mind, waste of breath
> > <flame off>
> 
> If you have constructive comments to make, as opposed to vague
> allusions
> to not liking some change or other and snide comments, we would be
> happy
> to hear them.

My comment (and even if you consider it snide or non-constructive,
I'd appreciate it if you would seriously consider it):

I think we need to make every effort to make Xen as drop-dead
easy to install and use as possible for as many developers and
users as possible, NOT a series of test questions on the
renew-my-alpha-geek-merit-badge exam.

Thanks,
Dan

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Jackson]

Dan Magenheimer writes ("RE: [Xen-devel] xen-unstable on OL6 (RHEL6 clone) problems"):
> While I agree with your good intentions, I think what happened is
> that you broke something that was working fine for some subset of
> users/developers (RH/Fedora0-based) in order to unbreak something
> for another subset (Debian-based).

No, that's not the case.  There is nothing RH-vs-Debian in this.
RH/Fedora/etc. and Debian/Ubuntu/etc. are affected by this change in
the same way, to the same extent, and with the same upsides and
downsides.

Both rpm-based distros and Debian-derived distros have facilities for
creating bridges as part of their system configuration.  The fact that
they do it in different ways is somewhat unfortunate but irrelevant.

The old bridge setup code (/etc/xen/scripts/network-bridge) was
completely insane.  It attempted to rename the system's eth0, create a
new bridge called eth0, and transfer all of the configuration done by
other unwitting parts of the dom0 from the real physical eth0 to the
bridge.

This was of course incredibly fragile.  Amongst the problems it caused
were races where different bits of the startup ran in different
orders, causing nondeterministic failures (especially if the dom0
interface was being configured by dhcp, as it increasingly is even for
servers).  Also it led to an ever-increasing amount of ad-hoc code for
transferring interface properties from peth0 to eth0; this code kept
rotting, had distro-specific bugs, needed updating every time the
system grew a new networking feature.

And the whole thing didn't work properly if you wanted to do anything
unusual, in which case the admin would end up fighting the nightmarish
Xen bootup scripts and no doubt cursing our name!

> I think we need to make every effort to make Xen as drop-dead
> easy to install and use as possible for as many developers and
> users as possible, NOT a series of test questions on the
> renew-my-alpha-geek-merit-badge exam.

I think this kind of comment is extremely unhelpful.  I have to say
that I don't appreciate the sarcastic attitude.

Of course we are trying to make Xen easier to install and use.  I'm
personally very conscious of the barriers to casual deployment of Xen.

One of the ways we are working on this problem is by removing crazy
fragile hacks from our bootup scripts and instead reusing existing
functionality that distros provide and document.

This does mean that there is some pain when upgrading.  But the payoff
is less pain for new users (who can just be told to use their distro's
bridging setup) and less pain for all future upgrades (because the
bridging setup will be much less likely to break in the future, as it
often has for an unlucky subset of people during Xen upgrades).

Ian.

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

> No, that's not the case.  There is nothing RH-vs-Debian in this.
>  :
> I think this kind of comment is extremely unhelpful.  I have to say
> that I don't appreciate the sarcastic attitude.
> 
> Of course we are trying to make Xen easier to install and use.  I'm
> personally very conscious of the barriers to casual deployment of Xen.

OK.  Thanks for the very thorough explanation.

And I apologize to everyone for the unkind comment and sarcastic
attitude.  As I'm sure we all know, it is very frustrating to
be trying to urgently get something (or several things) done and
something unrelated that has always worked fine stops working.
Please understand my tirade in light of this frustration.

Dan

Re: xen-unstable on OL6 (RHEL6 clone) problems

[George Dunlap]

On Thu, Feb 10, 2011 at 7:45 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
> And I apologize to everyone for the unkind comment and sarcastic
> attitude.  As I'm sure we all know, it is very frustrating to
> be trying to urgently get something (or several things) done and
> something unrelated that has always worked fine stops working.
> Please understand my tirade in light of this frustration.

Sure -- and given that you feel frustration, it's good that you
expressed it and the reason for it, so that we could let you know
where why we made the decisions we did rather than silently fuming.

Hmm, maybe there should be some blog entries on some of these things...

 -George

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Ian Campbell]

On Thu, 2011-02-10 at 19:25 +0000, Ian Jackson wrote: 
> the whole thing didn't work properly if you wanted to do anything
> unusual, in which case the admin would end up fighting the nightmarish
> Xen bootup scripts and no doubt cursing our name!

Agreed, the debugability of the previous solution was basically nil so,
while it was ok if it it worked, it actually required a far greater
level of arcane knowledge (worse: Xen specific arcane knowledge) than
the new scheme to fix when it didn't.

> Of course we are trying to make Xen easier to install and use.  I'm
> personally very conscious of the barriers to casual deployment of Xen.

Making casual deployment as easy as we are able to is certainly
extremely important and something we should all be mindful of.

Another aspect we should bear in mind is that one of the best ways we
can support end users (particularly the less savvy or those with a
preference for a stable distro) is by making it as easy as possible for
downstream distros to do a first rate job of supporting Xen themselves
by making it easy to create packages that are well integrated with their
distro and providing the support which they need to achieve this etc.

Ian.

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Pasi Kärkkäinen]

On Wed, Feb 09, 2011 at 02:26:00PM -0800, Dan Magenheimer wrote:
> Due to some other commitments, it's been awhile since I've built
> a xen-unstable-based system, and with Oracle Linux 6 (a RHEL6
> clone) just released and with Xen 4.1 in release candidates, I
> decided to install and test on a new machine.
> 
> As I sort-of expected, it's not going real well due to tools
> changes that I don't understand how to get around or fix.  I'm
> not a sysadmin guru so some or all of the problems may be
> trivial, and various googling helped a bit.
> 
> One that I can't figure out is when I try to start xend,
> I get a long sequence of:
> 
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
> /sys/class/net/virbr0/bridge: No such file or directory
> 
> and networking in a guest fails to work.  This directory
> DOES exist.  And "brctl show" seems to be the source of
> those messages.
> 
> In one semi-related googled link, I found that libvirtd
> may cause some problems, so I disabled that.  No luck,
> but the messages change to:
> 
> /sys/class/net/tmpbridge/bridge: No such file or directory
> 
> and starting xend caused my dom0 networking to go down.
> 
> Any clues?  
>

Dunno if this helps: http://wiki.xen.org/xenwiki/RHEL6Xen4Tutorial


> I'm guessing that I have to manually set up some
> network bridges that I never had to deal with before, but
> don't have any idea how to do that.
> 

Yep, when you use xl toolstack you need to set up the bridges yourself. 
The recommended way in el6 is to use /etc/sysconfig/network-scripts/ifcfg-br* scripts. 
Those are documented in the user manuals. 

Something like:

/etc/sysconfig/network-scripts/ifcfg-br0:

DEVICE="br0"
TYPE="Bridge"
BOOTPROTO="dhcp"
ONBOOT="yes"
DELAY=0

/etc/sysconfig/network-scripts/ifcfg-eth0:
DEVICE="eth0"
BOOTPROTO="none"
HWADDR="00:xx:yy:zz:xx:yy"
IPV6INIT="no"
IPV6_AUTOCONF="no"
NM_CONTROLLED="no"
ONBOOT="yes"
BRIDGE="br0"


-- Pasi

> Thanks,
> Dan
> 
> P.S. Don't know if it is related, but I see in dmesg several:
> 	XENBUS: Unable to read cpu state
> and a
> 
> virbr0: starting userspace STP failed, starting kernel STP
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: xen-unstable on OL6 (RHEL6 clone) problems

[Dan Magenheimer]

Hi Pasi!

> Dunno if this helps: http://wiki.xen.org/xenwiki/RHEL6Xen4Tutorial

Actually I meant to mention in the original email, yes
this WAS very helpful, thanks very much for writing that
or I wouldn't have gotten as far as I did.

However, I am trying to install xen-unstable rather than from
rpms so this is Xen 4.1 (rc) not Xen 4.0 and I think that's
where my problems are.

> > /sys/class/net/virbr0/bridge: No such file or directory
>
> > I'm guessing that I have to manually set up some
> > network bridges that I never had to deal with before, but
> > don't have any idea how to do that.
> >
> 
> Yep, when you use xl toolstack you need to set up the bridges yourself.
> The recommended way in el6 is to use /etc/sysconfig/network-
> scripts/ifcfg-br* scripts.
> Those are documented in the user manuals.
> 
> Something like:

Yep, did that both on an OL6 machine without Xen installed and
an OL6 machine with Xen-4.1-rc3 installed (built from scratch).
The bridging looks to be working fine on the machine without Xen.
On the Xen machine, I am still seeing a long sequence of

/sys/class/net/virbr0/bridge: No such file or directory

and that directory shows up fine with "ls -l".  That message
also shows up many times with "brctl show", so I suspect something
in the Xen install/startup is changing something that is confusing
something.  Or it is possible I committed some pilot error elsewhere.

Thanks for the help!
Dan

Re: xen-unstable on OL6 (RHEL6 clone) problems

[Pasi Kärkkäinen]

On Thu, Feb 10, 2011 at 09:55:55AM -0800, Dan Magenheimer wrote:
> Hi Pasi!
> 
> > Dunno if this helps: http://wiki.xen.org/xenwiki/RHEL6Xen4Tutorial
> 
> Actually I meant to mention in the original email, yes
> this WAS very helpful, thanks very much for writing that
> or I wouldn't have gotten as far as I did.
> 

Good to hear it helped :)

> However, I am trying to install xen-unstable rather than from
> rpms so this is Xen 4.1 (rc) not Xen 4.0 and I think that's
> where my problems are.
> 

Yep.

> > > /sys/class/net/virbr0/bridge: No such file or directory
> >
> > > I'm guessing that I have to manually set up some
> > > network bridges that I never had to deal with before, but
> > > don't have any idea how to do that.
> > >
> > 
> > Yep, when you use xl toolstack you need to set up the bridges yourself.
> > The recommended way in el6 is to use /etc/sysconfig/network-
> > scripts/ifcfg-br* scripts.
> > Those are documented in the user manuals.
> > 
> > Something like:
> 
> Yep, did that both on an OL6 machine without Xen installed and
> an OL6 machine with Xen-4.1-rc3 installed (built from scratch).
> The bridging looks to be working fine on the machine without Xen.
> On the Xen machine, I am still seeing a long sequence of
> 
> /sys/class/net/virbr0/bridge: No such file or directory
> 
> and that directory shows up fine with "ls -l".

So the directory is there, but is the "bridge" file there aswell? 


> That message also shows up many times with "brctl show", so I suspect something
> in the Xen install/startup is changing something that is confusing
> something.  Or it is possible I committed some pilot error elsewhere.
>

Hmm, so does "brctl show" list the bridges, even when it gives the error?

Btw do you have SElinux enabled? 

-- Pasi