Re: Network-bridge script with bonding and vlan

[Ted Kaczmarek <tedkaz@optonline.net>]

On Sat, 2005-10-22 at 00:23 +1300, Greg Brackley wrote:
> ----- Original Message ----- 
> From: "Ewan Mellor" <ewan@xensource.com>
> 
> 
> > The new script allows for one bridge per outgoing interface, I think, so 
> > in
> > your case one bridge per VLAN ought to be possible.  Why do you need one
> > bridge per domain?  What are you bridging in this case?  The topology we 
> > use
> > uses a bridge to plex all of the guests onto a specific outgoing 
> > interface --
> > so what are you bridging to require one per domain?
> 
> The main outcome I'm trying to achieve is isolation between the domains. 
> Typically we would have this separation between our production servers (not 
> all lumped together on one LAN).  Having the separation at layer 2 would 
> seem like to be a positive thing to maintain.  Using the VLAN interfaces 
> gives the impression that the machine has many network adapters [this would 
> seem to me to be a good fit with a virtualisation technology].
> 
> >> In the trivial case the xen support is vif0.0->veth0, and vifN.0->eth0
> >> (where N is a non-zero domain number).
> > Do your arrows refer to incoming packets?  If so, I think that is right,
> > though veth0 is renamed to eth0 now, so that dom0 sees a "normal" eth0
> > interface.
> 
> Sorry, I wasn't very clear there. Yes, incoming packets, or 
> backend<->frontend.
> 
> 
> >> The scripts seem to treat xen0 as a special case, and I'm not sure why.
> > Domain 0 is the privileged domain i.e. the one that manages the creation 
> > of
> > other domains.  It comes up on boot, and is where things like the script 
> > in
> > question are running.  That is why it is a special case.  Also, it is 
> > arranged
> > so that dom0's interface appears to have the MAC address of the underlying
> > NIC -- you can't do this with the guest domains, of course.
> 
> Got that.  But could the part of the dom0 script that configures the dom0 
> vif0.0<->veth0 part of the networking be extracted/pluggable just like the 
> domU setup of the vifN.0 interface.
> 
> > I've no idea whether that makes sense, but why are you trying to do it?
> 
> I guess it comes from the desire not to configure an interface one way, and 
> then moments later tear it all down and configure it another way.  I can see 
> that the current way is great for development, and integrates well with an 
> existing machine - not sure I'd want to head in that direction on a 
> non-development system.
> 
> > The bottom line, I think, is that Xen does not try to do everything with
> > network configuration.  We are happy to help, and happy to provide example
> 
> Thanks very much for your feedback. I lot of my comments are probably due to 
> being a relative newbie.
> 
> Most of the configuration I have been doing has been with the fc4 init 
> scripts. I'm looking for less, rather than more. However I am having 
> problems getting packets forwarded/bridged correctly.  ICMP pings seem to 
> work just fine, but the only way to get TCP/UDP packets to forward is to 
> attach tcpdump to the VLAN interface. I probably missing something very 
> obvious. I don't have the machine in front of me (and the networking isn't 
> working), so can't post the current config.
> 
> I'm unclear as to why the physical/bridge/backend interfaces need to have a 
> special mac address (fe:ff:ff:ff:ff:ff), and why they need arp disabled 
> (since they have no IP address).  What is special about fe:ff:ff:ff:ff:ff? 
> Could it be any locally assigned unicast mac address?
> 
> Greg.
> 

I am still seeing very erratic network behavior in general, on first
reboot of the dom0 things will generally behave well, but after the dom0
has been up for a while, with domU's brought up and down along the way
all kinds of random failures start to happen.

DomU oops 
[<c02644b2>] xenbus_dev_ok+0x32/0x60

After domU is brought down
mac address for domU is still showing up on xen-br0

domU is brought up
vif is not mapped to domU

I am no longer seeing the issue with peth0 seeing packet with its own
address though.

xen_changeset : Wed Oct 19 06:53:00 2005 +0100 7425:7c951e3eb5ab

I am not seeing any issues like this at all on my UP P4, just my SMP
Athlon. None of these are easy to replicate :-(


xen console output, the issues correlate to the smp_send_stop
disable_local_APIC message.

(XEN) DOM1: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM1: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
smp_send_stop disable_local_APIC
smp_send_stop disable_local_APIC
(XEN) DOM4: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM4: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
smp_send_stop disable_local_APIC
(XEN) DOM5: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM5: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM22: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM22: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM23: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000
(XEN) DOM23: (file=mm.c, line=460) Non-privileged attempt to map I/O
space 00000000


Regards,
Ted