Net containers config and usage

Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

All,

  I'm new to linux containers and am interested in the network containers
support, but am having some troubles getting it working & could use some help.

  I saw that the code released to 2.6.26 was incompatible with sysfs, but
that 2.6.29 should have support for both.  

  So, I built:
    linux-2.6.29-rc1
    iproute2.2.6.25 plus the patch for containers support
    lxc-0.5.2

  I can now run lxc-unshare -n /bin/tcsh
  And, as expected, that process only sees loopback.

  But I'm having trouble assigning an interface over to that container.

  I tried doing:
     # echo <pid> > /sys/class/net/eth4/new_ns_pid

  But there are no entries in sysfs called new_ns_pid.

  I also tried doing ip link set eth4 newns <pid>
  but I get a netlink error:
    RTNETLINK answers: Invalid argument 


Any tips on what I might be doing wrong, or how to get this working?  Since
I can do the lxc-unshare, it seems like I'm part way there, but can't move
network devices between namespaces.

Thanks,
Chris

Re: Net containers config and usage

[Serge E. Hallyn]

Quoting chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org (chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org):
> All,
> 
>   I'm new to linux containers and am interested in the network containers
> support, but am having some troubles getting it working & could use some help.
> 
>   I saw that the code released to 2.6.26 was incompatible with sysfs, but
> that 2.6.29 should have support for both.  
> 
>   So, I built:
>     linux-2.6.29-rc1
>     iproute2.2.6.25 plus the patch for containers support
>     lxc-0.5.2
> 
>   I can now run lxc-unshare -n /bin/tcsh
>   And, as expected, that process only sees loopback.

Since you're using the liblxc tools, can you just use one
of the configs it comes with which sets up networking for
you?

>   But I'm having trouble assigning an interface over to that container.
> 
>   I tried doing:
>      # echo <pid> > /sys/class/net/eth4/new_ns_pid
> 
>   But there are no entries in sysfs called new_ns_pid.

Yeah I'm not sure where that command came from.  To move eth4 into a new
netns, you'd want to do

	ip link set eth4 netns $pid_of_shell_in_ns

See http://lxc.sourceforge.net/network/configuration.php for
more  info.

>   I also tried doing ip link set eth4 newns <pid>
>   but I get a netlink error:
>     RTNETLINK answers: Invalid argument 
> 
> 
> Any tips on what I might be doing wrong, or how to get this working?  Since
> I can do the lxc-unshare, it seems like I'm part way there, but can't move
> network devices between namespaces.

-serge

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

On Mon, Jan 12, 2009 at 04:51:40PM -0600, Serge E. Hallyn wrote:
> 
> >   But I'm having trouble assigning an interface over to that container.
> > 
> >   I tried doing:
> >      # echo <pid> > /sys/class/net/eth4/new_ns_pid
> > 
> >   But there are no entries in sysfs called new_ns_pid.
> 
> Yeah I'm not sure where that command came from.

OK - just FYI, I got that command from http://lxc.sourceforge.net/network.php

> To move eth4 into a new
> netns, you'd want to do
> 
> 	ip link set eth4 netns $pid_of_shell_in_ns
> 
> See http://lxc.sourceforge.net/network/configuration.php for
> more  info.
> 

Trying to use the "ip link set" command, I get a different problem - it
reports an RTNETLINK error:

 [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
 [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
 RTNETLINK answers: Invalid argument

Any idea what could cause that error?

Thanks,
Chris

Re: Net containers config and usage

[Daniel Lezcano]

chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> On Mon, Jan 12, 2009 at 04:51:40PM -0600, Serge E. Hallyn wrote:
>   
>>>   But I'm having trouble assigning an interface over to that container.
>>>
>>>   I tried doing:
>>>      # echo <pid> > /sys/class/net/eth4/new_ns_pid
>>>
>>>   But there are no entries in sysfs called new_ns_pid.
>>>       
>> Yeah I'm not sure where that command came from.
>>     
>
> OK - just FYI, I got that command from http://lxc.sourceforge.net/network.php
>   
You are right, it is not up-to-date.
>   
>> To move eth4 into a new
>> netns, you'd want to do
>>
>> 	ip link set eth4 netns $pid_of_shell_in_ns
>>
>> See http://lxc.sourceforge.net/network/configuration.php for
>> more  info.
>>
>>     
>
> Trying to use the "ip link set" command, I get a different problem - it
> reports an RTNETLINK error:
>
>  [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
>  root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
>  root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
>  [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
>  RTNETLINK answers: Invalid argument
>
> Any idea what could cause that error?
>   
Assuming your prompt is showing the right version, try a version of 
iproute2 >= 2.6.26

Regards
    -- Daniel

Re: Net containers config and usage

[Serge E. Hallyn]

Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>> On Mon, Jan 12, 2009 at 04:51:40PM -0600, Serge E. Hallyn wrote:
>>   
>>>>   But I'm having trouble assigning an interface over to that container.
>>>>
>>>>   I tried doing:
>>>>      # echo <pid> > /sys/class/net/eth4/new_ns_pid
>>>>
>>>>   But there are no entries in sysfs called new_ns_pid.
>>>>       
>>> Yeah I'm not sure where that command came from.
>>>     
>>
>> OK - just FYI, I got that command from http://lxc.sourceforge.net/network.php
>>   
> You are right, it is not up-to-date.
>>   
>>> To move eth4 into a new
>>> netns, you'd want to do
>>>
>>> 	ip link set eth4 netns $pid_of_shell_in_ns
>>>
>>> See http://lxc.sourceforge.net/network/configuration.php for
>>> more  info.
>>>
>>>     
>>
>> Trying to use the "ip link set" command, I get a different problem - it
>> reports an RTNETLINK error:
>>
>>  [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
>>  root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
>>  root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
>>  [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
>>  RTNETLINK answers: Invalid argument
>>
>> Any idea what could cause that error?
>>   
> Assuming your prompt is showing the right version, try a version of  
> iproute2 >= 2.6.26

Right, this was a bit of a pain for me - when you install iproute,
by default it goes into another location, so you probably have to
specify /usr/local/sbin/ip or something.  Just a guess, but that
may be your problem here :)

-serge

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

> >>
> >>  [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
> >>  root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
> >>  root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
> >>  [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
> >>  RTNETLINK answers: Invalid argument
> >>
> >> Any idea what could cause that error?
> >>   
> > Assuming your prompt is showing the right version, try a version of  
> > iproute2 >= 2.6.26
> 
> Right, this was a bit of a pain for me - when you install iproute,
> by default it goes into another location, so you probably have to
> specify /usr/local/sbin/ip or something.  Just a guess, but that
> may be your problem here :)
> 
> -serge

I was running 2.6.25 plus the patch from here:
    http://marc.info/?l=linux-netdev&m=120905226611742&q=raw

But I tried getting 2.6.26 and I have the same problem:

[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755
RTNETLINK answers: Invalid argument

I'm pretty certain I'm running the right "version" of the command.  It
does install it to /usr/sbin/ip, but if I run the version that came with the
distribution (in /sbin/ip), I get a different error reporting netns as garbage.
So.. since it at least parses the netns argument and sends the netlink request,
it at least seems to be making it to the kernel.

Does the kernel.org version of 2.6.29-rc1 have everything I need?

Any other ideas?

Thanks,
Chris

Re: Net containers config and usage

[Guenter Roeck]

Works for me in 2.6.27. 
Did you enable network namespace support in the kernel ?

Guenter

On Tue, 2009-01-13 at 11:18 -0800, chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> > >>
> > >>  [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
> > >>  root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
> > >>  root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
> > >>  [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
> > >>  RTNETLINK answers: Invalid argument
> > >>
> > >> Any idea what could cause that error?
> > >>
> > > Assuming your prompt is showing the right version, try a version of
> > > iproute2 >= 2.6.26
> >
> > Right, this was a bit of a pain for me - when you install iproute,
> > by default it goes into another location, so you probably have to
> > specify /usr/local/sbin/ip or something.  Just a guess, but that
> > may be your problem here :)
> >
> > -serge
> 
> I was running 2.6.25 plus the patch from here:
>     http://marc.info/?l=linux-netdev&m=120905226611742&q=raw
> 
> But I tried getting 2.6.26 and I have the same problem:
> 
> [root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755
> RTNETLINK answers: Invalid argument
> 
> I'm pretty certain I'm running the right "version" of the command.  It
> does install it to /usr/sbin/ip, but if I run the version that came with the
> distribution (in /sbin/ip), I get a different error reporting netns as garbage.
> So.. since it at least parses the netns argument and sends the netlink request,
> it at least seems to be making it to the kernel.
> 
> Does the kernel.org version of 2.6.29-rc1 have everything I need?
> 
> Any other ideas?
> 
> Thanks,
> Chris
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linux-foundation.org/mailman/listinfo/containers

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

On Tue, Jan 13, 2009 at 11:37:48AM -0800, Guenter Roeck wrote:
> Works for me in 2.6.27. 
> Did you enable network namespace support in the kernel ?
> 
> Guenter

Yes, and I'm able to do an "lxc-unshare -n /bin/tcsh", so I guess that's
proof that network namespace support is there in the kernel.  But then when
I go to re-assign an interface, I get the error.

Chris

Re: Net containers config and usage

[Daniel Lezcano]

chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>>>>  [root@C100273 iproute2-2.6.25]# ps aux | grep tcsh
>>>>  root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
>>>>  root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh
>>>>  [root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918
>>>>  RTNETLINK answers: Invalid argument
>>>>
>>>> Any idea what could cause that error?
>>>>   
>>>>         
>>> Assuming your prompt is showing the right version, try a version of  
>>> iproute2 >= 2.6.26
>>>       
>> Right, this was a bit of a pain for me - when you install iproute,
>> by default it goes into another location, so you probably have to
>> specify /usr/local/sbin/ip or something.  Just a guess, but that
>> may be your problem here :)
>>
>> -serge
>>     
>
> I was running 2.6.25 plus the patch from here:
>     http://marc.info/?l=linux-netdev&m=120905226611742&q=raw
>
> But I tried getting 2.6.26 and I have the same problem:
>
> [root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755
> RTNETLINK answers: Invalid argument
>
> I'm pretty certain I'm running the right "version" of the command.  It
> does install it to /usr/sbin/ip, but if I run the version that came with the
> distribution (in /sbin/ip), I get a different error reporting netns as garbage.
> So.. since it at least parses the netns argument and sends the netlink request,
> it at least seems to be making it to the kernel.
>
> Does the kernel.org version of 2.6.29-rc1 have everything I need?
>
> Any other ideas?
>   
Is eth4 a physical network device or a virtual device (like a tunnel) 
renamed eth4 ?

Can you send the output of the "ip link" command and the value of
/sys/class/net/eth4/features to check if the network device is not 
flagged as a non-movable.

thanks.
  -- Daniel

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

On Tue, Jan 13, 2009 at 08:52:58PM +0100, Daniel Lezcano wrote:
> >I tried getting 2.6.26 and I have the same problem:
> >
> >[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755
> >RTNETLINK answers: Invalid argument
> >
> >I'm pretty certain I'm running the right "version" of the command.  It
> >does install it to /usr/sbin/ip, but if I run the version that came with 
> >the
> >distribution (in /sbin/ip), I get a different error reporting netns as 
> >garbage.
> >So.. since it at least parses the netns argument and sends the netlink 
> >request,
> >it at least seems to be making it to the kernel.
> >
> >Does the kernel.org version of 2.6.29-rc1 have everything I need?
> >
> >Any other ideas?
> >  
> Is eth4 a physical network device or a virtual device (like a tunnel) 
> renamed eth4 ?
> 
> Can you send the output of the "ip link" command and the value of
> /sys/class/net/eth4/features to check if the network device is not 
> flagged as a non-movable.
> 
> thanks.
>  -- Daniel

It's a physical device - uses the e1000e driver.
Here's the output of both commands.

[root@C100273 iproute2-2.6.26]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1c:c4:dc:44:b2 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f5 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:dc:44:b0 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f4 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1c:c4:48:35:f7 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f6 brd ff:ff:ff:ff:ff:ff
8: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue 
    link/ether 66:f4:32:9f:12:df brd ff:ff:ff:ff:ff:ff

[root@C100273 iproute2-2.6.26]# cat /sys/class/net/eth4/features
0x111ba9

Thanks,
Chris

Re: Net containers config and usage

[Guenter Roeck]

As far as I recall, if you have sysfs active and use the sysfs patch to
let you configure both sysfs and network namespaces, you can only move
virtual interfaces into a network namespace.

Guenter

On Tue, 2009-01-13 at 13:47 -0800, chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> On Tue, Jan 13, 2009 at 08:52:58PM +0100, Daniel Lezcano wrote:
> > >I tried getting 2.6.26 and I have the same problem:
> > >
> > >[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755
> > >RTNETLINK answers: Invalid argument
> > >
> > >I'm pretty certain I'm running the right "version" of the command.  It
> > >does install it to /usr/sbin/ip, but if I run the version that came with
> > >the
> > >distribution (in /sbin/ip), I get a different error reporting netns as
> > >garbage.
> > >So.. since it at least parses the netns argument and sends the netlink
> > >request,
> > >it at least seems to be making it to the kernel.
> > >
> > >Does the kernel.org version of 2.6.29-rc1 have everything I need?
> > >
> > >Any other ideas?
> > >
> > Is eth4 a physical network device or a virtual device (like a tunnel)
> > renamed eth4 ?
> >
> > Can you send the output of the "ip link" command and the value of
> > /sys/class/net/eth4/features to check if the network device is not
> > flagged as a non-movable.
> >
> > thanks.
> >  -- Daniel
> 
> It's a physical device - uses the e1000e driver.
> Here's the output of both commands.
> 
> [root@C100273 iproute2-2.6.26]# ip link
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
>     link/ether 00:1c:c4:dc:44:b2 brd ff:ff:ff:ff:ff:ff
> 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
>     link/ether 00:1c:c4:48:35:f5 brd ff:ff:ff:ff:ff:ff
> 4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
>     link/ether 00:1c:c4:dc:44:b0 brd ff:ff:ff:ff:ff:ff
> 5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
>     link/ether 00:1c:c4:48:35:f4 brd ff:ff:ff:ff:ff:ff
> 6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
>     link/ether 00:1c:c4:48:35:f7 brd ff:ff:ff:ff:ff:ff
> 7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
>     link/ether 00:1c:c4:48:35:f6 brd ff:ff:ff:ff:ff:ff
> 8: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>     link/ether 66:f4:32:9f:12:df brd ff:ff:ff:ff:ff:ff
> 
> [root@C100273 iproute2-2.6.26]# cat /sys/class/net/eth4/features
> 0x111ba9
> 
> Thanks,
> Chris
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linux-foundation.org/mailman/listinfo/containers

Re: Net containers config and usage

[Daniel Lezcano]

Guenter Roeck wrote:
> As far as I recall, if you have sysfs active and use the sysfs patch to
> let you configure both sysfs and network namespaces, you can only move
> virtual interfaces into a network namespace.
>
> Guenter
>   
Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist 
together has one restriction, the physical network devices can not be 
moved if sysfs is enabled in the kernel. This is why Chris can not move 
the physical network device with this version of the kernel.
This restriction will be set until the sysfs per namespace is fully 
supported.

This restriction does not exist with with the previous kernel version 
with the sysfs per namespace patchset.

 -- Daniel

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
> Guenter Roeck wrote:
> >As far as I recall, if you have sysfs active and use the sysfs patch to
> >let you configure both sysfs and network namespaces, you can only move
> >virtual interfaces into a network namespace.
> >
> >Guenter
> >  
> Ah ! yes, you are right :)
> 
> The current upstream implementation allowing sysfs and netns to coexist 
> together has one restriction, the physical network devices can not be 
> moved if sysfs is enabled in the kernel. This is why Chris can not move 
> the physical network device with this version of the kernel.
> This restriction will be set until the sysfs per namespace is fully 
> supported.
> 
> This restriction does not exist with with the previous kernel version 
> with the sysfs per namespace patchset.
> 
> -- Daniel
> 

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will
be added in the upstream?

Thanks again,
Chris

Re: Net containers config and usage

[Daniel Lezcano]

chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
>   
>> Guenter Roeck wrote:
>>     
>>> As far as I recall, if you have sysfs active and use the sysfs patch to
>>> let you configure both sysfs and network namespaces, you can only move
>>> virtual interfaces into a network namespace.
>>>
>>> Guenter
>>>  
>>>       
>> Ah ! yes, you are right :)
>>
>> The current upstream implementation allowing sysfs and netns to coexist 
>> together has one restriction, the physical network devices can not be 
>> moved if sysfs is enabled in the kernel. This is why Chris can not move 
>> the physical network device with this version of the kernel.
>> This restriction will be set until the sysfs per namespace is fully 
>> supported.
>>
>> This restriction does not exist with with the previous kernel version 
>> with the sysfs per namespace patchset.
>>
>> -- Daniel
>>
>>     
>
> Ah, great, thanks to all for your help on this.
> Do you have any rough estimate when the support for sysfs per namespace will
>   
The sysfs per namespace has been rejected because of some design 
problems related with the sysfs itself.
Perhaps Eric can tell more about that...

Re: Net containers config and usage

[Serge E. Hallyn]

Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> > On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
> >   
> >> Guenter Roeck wrote:
> >>     
> >>> As far as I recall, if you have sysfs active and use the sysfs patch to
> >>> let you configure both sysfs and network namespaces, you can only move
> >>> virtual interfaces into a network namespace.
> >>>
> >>> Guenter
> >>>  
> >>>       
> >> Ah ! yes, you are right :)
> >>
> >> The current upstream implementation allowing sysfs and netns to coexist 
> >> together has one restriction, the physical network devices can not be 
> >> moved if sysfs is enabled in the kernel. This is why Chris can not move 
> >> the physical network device with this version of the kernel.
> >> This restriction will be set until the sysfs per namespace is fully 
> >> supported.
> >>
> >> This restriction does not exist with with the previous kernel version 
> >> with the sysfs per namespace patchset.
> >>
> >> -- Daniel
> >>
> >>     
> >
> > Ah, great, thanks to all for your help on this.
> > Do you have any rough estimate when the support for sysfs per namespace will
> >   
> The sysfs per namespace has been rejected because of some design 
> problems related with the sysfs itself.
> Perhaps Eric can tell more about that...

Chris, in the meantime, is using the physical device an absolute
necessity, or could you work around it for now using a veth tunnel?

Even if Eric has been working on the sysfs locking rework quietly
the last few months, i'd expect several months of back-and-forth
trying to prove that the rework is correct...

-serge

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

On Wed, Jan 14, 2009 at 01:26:34PM -0600, Serge E. Hallyn wrote:
> Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
> > chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> > > On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
> > >   
> > >> Guenter Roeck wrote:
> > >>     
> > >>> As far as I recall, if you have sysfs active and use the sysfs patch to
> > >>> let you configure both sysfs and network namespaces, you can only move
> > >>> virtual interfaces into a network namespace.
> > >>>
> > >>> Guenter
> > >>>  
> > >>>       
> > >> Ah ! yes, you are right :)
> > >>
> > >> The current upstream implementation allowing sysfs and netns to coexist 
> > >> together has one restriction, the physical network devices can not be 
> > >> moved if sysfs is enabled in the kernel. This is why Chris can not move 
> > >> the physical network device with this version of the kernel.
> > >> This restriction will be set until the sysfs per namespace is fully 
> > >> supported.
> > >>
> > >> This restriction does not exist with with the previous kernel version 
> > >> with the sysfs per namespace patchset.
> > >>
> > >> -- Daniel
> > >>
> > >>     
> > >
> > > Ah, great, thanks to all for your help on this.
> > > Do you have any rough estimate when the support for sysfs per namespace will
> > >   
> > The sysfs per namespace has been rejected because of some design 
> > problems related with the sysfs itself.
> > Perhaps Eric can tell more about that...
> 
> Chris, in the meantime, is using the physical device an absolute
> necessity, or could you work around it for now using a veth tunnel?
> 
> Even if Eric has been working on the sysfs locking rework quietly
> the last few months, i'd expect several months of back-and-forth
> trying to prove that the rework is correct...
> 
> -serge

Yes, ultimately we'll need the physical device inside the same namespace
as our application.  Our application does a lot of management on the interface,
monitoring things like the interface's link-pulse and such, and that wouldn't
be available through a virtual interface.  We can always redesign things
to have the management portion run in the namespace with the physical interface,
but for performance reasons we'd eventually want the physical interface to be
directly inside the namespace anyway - so that would probably be wasted effort.

Thanks,
Chris

Re: Net containers config and usage

[Daniel Lezcano]

chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> On Wed, Jan 14, 2009 at 01:26:34PM -0600, Serge E. Hallyn wrote:
>   
>> Quoting Daniel Lezcano (daniel.lezcano-GANU6spQydw@public.gmane.org):
>>     
>>> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>>>       
>>>> On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
>>>>   
>>>>         
>>>>> Guenter Roeck wrote:
>>>>>     
>>>>>           
>>>>>> As far as I recall, if you have sysfs active and use the sysfs patch to
>>>>>> let you configure both sysfs and network namespaces, you can only move
>>>>>> virtual interfaces into a network namespace.
>>>>>>
>>>>>> Guenter
>>>>>>  
>>>>>>       
>>>>>>             
>>>>> Ah ! yes, you are right :)
>>>>>
>>>>> The current upstream implementation allowing sysfs and netns to coexist 
>>>>> together has one restriction, the physical network devices can not be 
>>>>> moved if sysfs is enabled in the kernel. This is why Chris can not move 
>>>>> the physical network device with this version of the kernel.
>>>>> This restriction will be set until the sysfs per namespace is fully 
>>>>> supported.
>>>>>
>>>>> This restriction does not exist with with the previous kernel version 
>>>>> with the sysfs per namespace patchset.
>>>>>
>>>>> -- Daniel
>>>>>
>>>>>     
>>>>>           
>>>> Ah, great, thanks to all for your help on this.
>>>> Do you have any rough estimate when the support for sysfs per namespace will
>>>>   
>>>>         
>>> The sysfs per namespace has been rejected because of some design 
>>> problems related with the sysfs itself.
>>> Perhaps Eric can tell more about that...
>>>       
>> Chris, in the meantime, is using the physical device an absolute
>> necessity, or could you work around it for now using a veth tunnel?
>>
>> Even if Eric has been working on the sysfs locking rework quietly
>> the last few months, i'd expect several months of back-and-forth
>> trying to prove that the rework is correct...
>>
>> -serge
>>     
>
> Yes, ultimately we'll need the physical device inside the same namespace
> as our application.  Our application does a lot of management on the interface,
> monitoring things like the interface's link-pulse and such, and that wouldn't
> be available through a virtual interface.  We can always redesign things
> to have the management portion run in the namespace with the physical interface,
> but for performance reasons we'd eventually want the physical interface to be
> directly inside the namespace anyway - so that would probably be wasted effort.
>   
Did you tried with the macvlan ?

Re: Net containers config and usage

[Eric W. Biederman]

Daniel Lezcano <daniel.lezcano-GANU6spQydw@public.gmane.org> writes:

> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>> On Wed, Jan 14, 2009 at 09:50:29AM +0100, Daniel Lezcano wrote:
>>
>>> Guenter Roeck wrote:
>>>
>>>> As far as I recall, if you have sysfs active and use the sysfs patch to
>>>> let you configure both sysfs and network namespaces, you can only move
>>>> virtual interfaces into a network namespace.
>>>>
>>>> Guenter
>>>>
>>> Ah ! yes, you are right :)
>>>
>>> The current upstream implementation allowing sysfs and netns to coexist
>>> together has one restriction, the physical network devices can not be moved
>>> if sysfs is enabled in the kernel. This is why Chris can not move the
>>> physical network device with this version of the kernel.
>>> This restriction will be set until the sysfs per namespace is fully
> supported.
>>>
>>> This restriction does not exist with with the previous kernel version with
>>> the sysfs per namespace patchset.
>>>
>>> -- Daniel
>>>
>>>
>>
>> Ah, great, thanks to all for your help on this.
>> Do you have any rough estimate when the support for sysfs per namespace will
>>
> The sysfs per namespace has been rejected because of some design problems
> related with the sysfs itself.
> Perhaps Eric can tell more about that...

Short version.  Before you can change code that is a mess you get the pleasure
of cleaning up someone else's mess.

I have seen no viable alternative to sysfs per namespace.

I have something like 22 patches that I hope to start pushing in the next
day or two to clean up sysfs, and then when that is done I will come back
and see what can be done about the kobject layer and sysfs.

To do all of this right I need to step back and see reworking the
sysfs and the kobject layers as worthy goals in and of themselves.  Instead
of just some barrier in the way of getting network namespaces to work
properly.

Eric

Re: Net containers config and usage

[chris-SqNQQPNds68nxqbYAscKCQ]

> >
> >Yes, ultimately we'll need the physical device inside the same namespace
> >as our application.  Our application does a lot of management on the 
> >interface,
> >monitoring things like the interface's link-pulse and such, and that 
> >wouldn't
> >be available through a virtual interface.  We can always redesign things
> >to have the management portion run in the namespace with the physical 
> >interface,
> >but for performance reasons we'd eventually want the physical interface to 
> >be
> >directly inside the namespace anyway - so that would probably be wasted 
> >effort.
> >  
> Did you tried with the macvlan ?

I just tried it - and it does work - I can move the vlan interfaces into the
namespace.   But that's not a solution for us - we really need to support
communications out to other nodes on the network that we can't count on using
vlan.  

Thanks,
Chris

Re: Net containers config and usage

[Daniel Lezcano]

chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>>> Yes, ultimately we'll need the physical device inside the same namespace
>>> as our application.  Our application does a lot of management on the 
>>> interface,
>>> monitoring things like the interface's link-pulse and such, and that 
>>> wouldn't
>>> be available through a virtual interface.  We can always redesign things
>>> to have the management portion run in the namespace with the physical 
>>> interface,
>>> but for performance reasons we'd eventually want the physical interface to 
>>> be
>>> directly inside the namespace anyway - so that would probably be wasted 
>>> effort.
>>>  
>>>       
>> Did you tried with the macvlan ?
>>     
>
> I just tried it - and it does work - I can move the vlan interfaces into the
> namespace.   But that's not a solution for us - we really need to support
> communications out to other nodes on the network that we can't count on using
> vlan.  
>   
Perhaps I am misunderstand but the macvlan is just an "eth aliased" 
interface, so you should but able to communicate with the outside world 
and keep the same performances as the physical device, no ? The only 
drawback with the macvlan is you can not communicate on the same host 
between different namespaces. We did some benchmark last year wih the 
macvlan and we got very good results.

Re: Net containers config and usage

[Chris R. Jones]

On Thu, Jan 15, 2009 at 11:21:43PM +0100, Daniel Lezcano wrote:
> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
> >>Did you tried with the macvlan ?
> >>    
> >
> >I just tried it - and it does work - I can move the vlan interfaces into 
> >the
> >namespace.   But that's not a solution for us - we really need to support
> >communications out to other nodes on the network that we can't count on 
> >using
> >vlan.  
> >  
> Perhaps I am misunderstand but the macvlan is just an "eth aliased" 
> interface, so you should but able to communicate with the outside world 
> and keep the same performances as the physical device, no ? The only 
> drawback with the macvlan is you can not communicate on the same host 
> between different namespaces. We did some benchmark last year wih the 
> macvlan and we got very good results.

Oh great, I confused the macvlan support with 802.1q vlan interfaces 
(as added through vconfig).  macvlan does look very promising - I was able 
to create a macvlan interface and assign it to my namespace - looks pretty good!

Thanks,
Chris

Re: Net containers config and usage

[Eric W. Biederman]

"Chris R. Jones" <chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org> writes:

> On Thu, Jan 15, 2009 at 11:21:43PM +0100, Daniel Lezcano wrote:
>> chris-SqNQQPNds68nxqbYAscKCQ@public.gmane.org wrote:
>> >>Did you tried with the macvlan ?
>> >>    
>> >
>> >I just tried it - and it does work - I can move the vlan interfaces into 
>> >the
>> >namespace.   But that's not a solution for us - we really need to support
>> >communications out to other nodes on the network that we can't count on 
>> >using
>> >vlan.  
>> >  
>> Perhaps I am misunderstand but the macvlan is just an "eth aliased" 
>> interface, so you should but able to communicate with the outside world 
>> and keep the same performances as the physical device, no ? The only 
>> drawback with the macvlan is you can not communicate on the same host 
>> between different namespaces. 

And that drawback (when all the macvlan are off of the same network device is
actually a bug).

>We did some benchmark last year wih the 
>> macvlan and we got very good results.
>
> Oh great, I confused the macvlan support with 802.1q vlan interfaces 
> (as added through vconfig).  macvlan does look very promising - I was able 
> to create a macvlan interface and assign it to my namespace - looks pretty good!

Good to hear you have a solution.  Putting physical hardware devices in network
namespaces is definitely on the todo.  But it looks to be a while before we get
all of the details sorted out.

Eric