vif-bridge is driving me nuts

vif-bridge is driving me nuts

[Matt Ayres]

Hi,

I'm having a few problems with vif-bridge...

1) I wanted to change the bridge interface names to be more user
friendly and be named after the domain name, not the id.

I figured making this change in /etc/xen/scripts/vif-bridge would do it:

# Add/remove vif to/from bridge.
#brctl ${brcmd} ${bridge} ${vif}
#ifconfig ${vif} $OP
brctl ${brcmd} ${bridge} ${domain}
ifconfig ${domain} $OP

I tried running it by hand and it says the interface doesn't exist.  My
question is... where/when is the tunnel device created?  This is mostly
a cosmetic change compared to the second problem, which is a
functionality problem..

2) In the example usage it shows the last parameter as ip= with a list
of subnets.  How do I pass this ip information to vif-bridge from the
domains configuration file?  I'd really like to use the functionality it
has to limit which domain will be forwarded IP's (to prevent IP
stealing), but for the life of my I can't figure it out... it definitely
does NOT pass the ip variable from the config file as that is already
set.

I'm wondering where vif-script is executed by xend.. I did a grep over
xend itself along with the entire /usr/lib/python/xen directory and only
found a single reference to vif-script and that is where it loads in
from the config file.  Perhaps just knowing more about how xend deals
with vif-script will help me along here.

-- 
Thanks,
Matt



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt

Re: vif-bridge is driving me nuts

[Matt Ayres]

On Tue, 2005-01-04 at 22:52 -0500, Matt Ayres wrote:

> 
> 2) In the example usage it shows the last parameter as ip= with a list
> of subnets.  How do I pass this ip information to vif-bridge from the
> domains configuration file?  I'd really like to use the functionality it
> has to limit which domain will be forwarded IP's (to prevent IP
> stealing), but for the life of my I can't figure it out... it definitely
> does NOT pass the ip variable from the config file as that is already
> set.
> 
> I'm wondering where vif-script is executed by xend.. I did a grep over
> xend itself along with the entire /usr/lib/python/xen directory and only
> found a single reference to vif-script and that is where it loads in
> from the config file.  Perhaps just knowing more about how xend deals
> with vif-script will help me along here.
> 

In response to my own post, I noticed the parameters for 'vif' in the
config file match up to those for vif-bridge... except for the 'ip'
parameter.  Would something like this in the domain configuration file
make sense?

vif = [ 'ip="192.168.1.80/32"' ]

or to better follow the example, even this:

vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0, ip="192.168.1.80/32"' ]

Of course, no scripts see ip as a valid argument so it doesn't work
now... it returns something like "Error: Invalid vif specifier:
ip=192.168.1.80/32".  Interestingly enough i've tried modifying
create.py (in the build tree so it builds create.pyc) to include 'ip' as
valid, same error.  Then I completely commented the 2 lines of code that
check and return the error and it STILL does it so i'm obviously missing
something, I guess I just need either a pointer from someone or to bury
myself in the code for a little bit more.  

-- 
Thanks,
Matt



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt

Re: vif-bridge is driving me nuts

[Mike Wray]

Matt Ayres wrote:
> Hi,
> 
> I'm having a few problems with vif-bridge...
> 
> 1) I wanted to change the bridge interface names to be more user
> friendly and be named after the domain name, not the id.

Unfortunately you will have trouble with this. The names of network
interfaces are limited to 15 characters - and domain names are not.

> 
> I figured making this change in /etc/xen/scripts/vif-bridge would do it:
> 
> # Add/remove vif to/from bridge.
> #brctl ${brcmd} ${bridge} ${vif}
> #ifconfig ${vif} $OP
> brctl ${brcmd} ${bridge} ${domain}
> ifconfig ${domain} $OP
> 
> I tried running it by hand and it says the interface doesn't exist.  My
> question is... where/when is the tunnel device created?  This is mostly
> a cosmetic change compared to the second problem, which is a
> functionality problem..
> 
> 2) In the example usage it shows the last parameter as ip= with a list
> of subnets.  How do I pass this ip information to vif-bridge from the
> domains configuration file?  I'd really like to use the functionality it
> has to limit which domain will be forwarded IP's (to prevent IP
> stealing), but for the life of my I can't figure it out... it definitely
> does NOT pass the ip variable from the config file as that is already
> set.
> 
> I'm wondering where vif-script is executed by xend.. I did a grep over
> xend itself along with the entire /usr/lib/python/xen directory and only
> found a single reference to vif-script and that is where it loads in
> from the config file.  Perhaps just knowing more about how xend deals
> with vif-script will help me along here.
> 

Mike


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt

Re: vif-bridge is driving me nuts

[Matt Ayres]

On Wed, 2005-01-05 at 00:08 -0500, Matt Ayres wrote:
> > 
> 
> In response to my own post, I noticed the parameters for 'vif' in the
> config file match up to those for vif-bridge... except for the 'ip'
> parameter.  Would something like this in the domain configuration file
> make sense?
> 
> vif = [ 'ip="192.168.1.80/32"' ]
> 
> or to better follow the example, even this:
> 
> vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0, ip="192.168.1.80/32"' ]
> 
> Of course, no scripts see ip as a valid argument so it doesn't work
> now... it returns something like "Error: Invalid vif specifier:
> ip=192.168.1.80/32".  Interestingly enough i've tried modifying
> create.py (in the build tree so it builds create.pyc) to include 'ip' as
> valid, same error.  Then I completely commented the 2 lines of code that
> check and return the error and it STILL does it so i'm obviously missing
> something, I guess I just need either a pointer from someone or to bury
> myself in the code for a little bit more.  
> 

In yet another response to my own post, it looks like it's looking to
pass the variable 'ipaddr' to the vif-script (aka. vif-bridge) script as
the last argument.  Does this sound correct?  Of course it doesn't work,
but the code looks like this should be what should happen.  From
Vifctl.py:

def vifctl(op, vif=None, script=None, domain=None, mac=None,
bridge=None, ipaddr=[]):
    """Call a vif control script.
    Xend calls this when bringing vifs up or down.

    @param op:     vif operation (up, down)
    @param vif:    vif name
    @param script: name of control script
    @param domain: name of domain the vif is on
    @param mac:    vif MAC address
    @param bridge: bridge to add the vif to
    @param ipaddr: list of ipaddrs the vif may use

...

    if ipaddr:
        ips = ' '.join(ipaddr)
        args.append("ip='%s'" % ips)


vifctl is called from server/netif.py and it also retrieves the ipaddr
information from there.

Unfortunately, it doesn't work... even though ipaddr is defined in the
domain config file it's still not passing the argument to vif-bridge.

I have to admit I know absolutely no Python and very little OO design so
perhaps i'm missing something... it looks like it should just work
though.  Can whoever wrote this part of Xen perhaps provide some
insight?


Here is an example configuration file I am trying to use:
#  -*- mode: python; -*-
kernel = "/xen/servers/vs1/kernel"
memory = 64
name = "vs1"
disk = [ 'file:/xen/servers/vs1/rootfs,sda1,w',
'file:/xen/servers/vs1/swapfs,sda2,w' ]
netmask="255.255.255.0"
gateway="192.168.1.1"
hostname="www.vs1.net"
ipaddr="192.168.1.80"
ip="192.168.1.80"
console=500
root = "/dev/sda1 ro"
extra = "4"
restart = 'onreboot'

antispoof is set to yes in xend-config.sxp.

I'll post more if I get this working before I hear back from someone
else, but at the moment I think i've exhausted my resources.

-- 



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt

Re: vif-bridge is driving me nuts

[Mike Wray]

Matt Ayres wrote:
> On Wed, 2005-01-05 at 00:08 -0500, Matt Ayres wrote:
> 
>>In response to my own post, I noticed the parameters for 'vif' in the
>>config file match up to those for vif-bridge... except for the 'ip'
>>parameter.  Would something like this in the domain configuration file
>>make sense?
>>
>>vif = [ 'ip="192.168.1.80/32"' ]
>>
>>or to better follow the example, even this:
>>
>>vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0, ip="192.168.1.80/32"' ]
>>
>>Of course, no scripts see ip as a valid argument so it doesn't work
>>now... it returns something like "Error: Invalid vif specifier:
>>ip=192.168.1.80/32".  Interestingly enough i've tried modifying
>>create.py (in the build tree so it builds create.pyc) to include 'ip' as
>>valid, same error.  Then I completely commented the 2 lines of code that
>>check and return the error and it STILL does it so i'm obviously missing
>>something, I guess I just need either a pointer from someone or to bury
>>myself in the code for a little bit more.  
>>

The code in xm/create.py to do with ip addrs for vifs dates from the old
vfr stuff and appears to have bit-rotted. FWIW, there is an 'ipaddr'
parameter that sets IP address(es) for vifs, but it is not currently
used. The vif handling in xm/create will error on trying to use
an unsupported vif parameter. The supported ones are mac, bridge, script,
backend (see 'xm help create').

Xend supports the 'ip' field in a vif spec, but this is not currently
passed-in by xm/create.

I'm about to take a look at the handling of device specs, so I could
take a look at this. It seems to make most sense to add an 'ip' parameter
to the vif spec in xm/create the way you tried to use it above.

> In yet another response to my own post, it looks like it's looking to
> pass the variable 'ipaddr' to the vif-script (aka. vif-bridge) script as
> the last argument.  Does this sound correct?  Of course it doesn't work,
> but the code looks like this should be what should happen.  From
> Vifctl.py:
> 
> def vifctl(op, vif=None, script=None, domain=None, mac=None,
> bridge=None, ipaddr=[]):
>     """Call a vif control script.
>     Xend calls this when bringing vifs up or down.
> 
>     @param op:     vif operation (up, down)
>     @param vif:    vif name
>     @param script: name of control script
>     @param domain: name of domain the vif is on
>     @param mac:    vif MAC address
>     @param bridge: bridge to add the vif to
>     @param ipaddr: list of ipaddrs the vif may use
> 
> ...
> 
>     if ipaddr:
>         ips = ' '.join(ipaddr)
>         args.append("ip='%s'" % ips)
> 
> 
> vifctl is called from server/netif.py and it also retrieves the ipaddr
> information from there.
> 
> Unfortunately, it doesn't work... even though ipaddr is defined in the
> domain config file it's still not passing the argument to vif-bridge.
> 
> I have to admit I know absolutely no Python and very little OO design so
> perhaps i'm missing something... it looks like it should just work
> though.  Can whoever wrote this part of Xen perhaps provide some
> insight?
> 
> 
> Here is an example configuration file I am trying to use:
> #  -*- mode: python; -*-
> kernel = "/xen/servers/vs1/kernel"
> memory = 64
> name = "vs1"
> disk = [ 'file:/xen/servers/vs1/rootfs,sda1,w',
> 'file:/xen/servers/vs1/swapfs,sda2,w' ]
> netmask="255.255.255.0"
> gateway="192.168.1.1"
> hostname="www.vs1.net"
> ipaddr="192.168.1.80"
> ip="192.168.1.80"
> console=500
> root = "/dev/sda1 ro"
> extra = "4"
> restart = 'onreboot'
> 
> antispoof is set to yes in xend-config.sxp.
> 
> I'll post more if I get this working before I hear back from someone
> else, but at the moment I think i've exhausted my resources.
> 

Mike


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt

Re: vif-bridge is driving me nuts

[Matt Ayres]

On Thu, 2005-01-06 at 14:51 +0000, Mike Wray wrote:
> Matt Ayres wrote:
> > On Wed, 2005-01-05 at 00:08 -0500, Matt Ayres wrote:
> > 
> >>In response to my own post, I noticed the parameters for 'vif' in the
> >>config file match up to those for vif-bridge... except for the 'ip'
> >>parameter.  Would something like this in the domain configuration file
> >>make sense?
> >>
> >>vif = [ 'ip="192.168.1.80/32"' ]
> >>
> >>or to better follow the example, even this:
> >>
> >>vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0, ip="192.168.1.80/32"' ]
> >>
> >>Of course, no scripts see ip as a valid argument so it doesn't work
> >>now... it returns something like "Error: Invalid vif specifier:
> >>ip=192.168.1.80/32".  Interestingly enough i've tried modifying
> >>create.py (in the build tree so it builds create.pyc) to include 'ip' as
> >>valid, same error.  Then I completely commented the 2 lines of code that
> >>check and return the error and it STILL does it so i'm obviously missing
> >>something, I guess I just need either a pointer from someone or to bury
> >>myself in the code for a little bit more.  
> >>
> 
> The code in xm/create.py to do with ip addrs for vifs dates from the old
> vfr stuff and appears to have bit-rotted. FWIW, there is an 'ipaddr'
> parameter that sets IP address(es) for vifs, but it is not currently
> used. The vif handling in xm/create will error on trying to use
> an unsupported vif parameter. The supported ones are mac, bridge, script,
> backend (see 'xm help create').
> 
> Xend supports the 'ip' field in a vif spec, but this is not currently
> passed-in by xm/create.
> 
> I'm about to take a look at the handling of device specs, so I could
> take a look at this. It seems to make most sense to add an 'ip' parameter
> to the vif spec in xm/create the way you tried to use it above.

Well, I did try modifying create.py to accept an ip/ipaddr field (as
stated)... it didn't seem to take it too well.  I know very little
bash/python so what I could do is limited, but I did end up being able
to fix this for myself.  I wrote a small perl script that returns the
IP's assigned to a domain (via ipaddr config variable).  In vif-bridge I
then just added "ip=`/path/to/script ${domain}`" and that fills in the
ip variable and everything works fine.  Once the IP's are actually
passed to vif-bridge it will work great.

One thing I noticed... the iptables code prevents outside machines from
accessing the IP assigned to the domain if it's not explicitly allowed,
but the host itself and other domains on the same host see the IP as
going to to the invalid domain.  I don't know if this is an
iptables/kernel limit or if it can be fixed... still working on that
part and if I find rules that work better I will pass them along.



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt