Bonding with tun-devices

Bonding with tun-devices

[Linus Gasser]

Dear list,

I'm trying to create a bonded interface using tun-devices created by the

ssh -NTCf -w 0:0

command. The reason is that I'm behind a satellite connection with 
per-stream limited bandwidth...

On some older kernels everything was fine, and I could do:

# ssh -NTCf -w 0:0 web
# ssh -NTCf -w 1:1 web
# modprobe bonding mode=0 miimon=100
# ifconfig bond0 172.16.0.1 netmask 255.255.255.0
# ifconfig bond0 hw ether 12:34:56:78:9a
# ifenslave bond0 tun0 tun1

on one side and

# modprobe bonding mode=0 miimon=100
# ifconfig bond0 172.16.0.2 netmask 255.255.255.0
# ifconfig bond0 hw ether 12:35
# ifenslave -f bond0 tun0 tun1
# ping 172.16.0.1

on the other side. On a newer kernel (>3.14) this doesn't work (at least 
not on linux-armv7), while on 3.13 it does work under linux-armv7. Now 
I've been told to use the iproute2-suite, as ifenslave is deprecated. 
But if I try to do

# ssh -NTCf -w 0:0 web
# ssh -NTCf -w 1:1 web
# ip link add name bond1 type bond
# ip link set dev tun0 master bond1

it gives me an error

RTNETLINK answers: Operation not supported

Any idea what I'm doing wrong?

Linus

Re: Bonding with tun-devices

[Veaceslav Falico]

On Wed, May 21, 2014 at 02:59:22PM +0100, Linus Gasser wrote:
>Dear list,
>
>I'm trying to create a bonded interface using tun-devices created by the
>
>ssh -NTCf -w 0:0
>
>command. The reason is that I'm behind a satellite connection with 
>per-stream limited bandwidth...
>
>On some older kernels everything was fine, and I could do:
>
># ssh -NTCf -w 0:0 web
># ssh -NTCf -w 1:1 web
># modprobe bonding mode=0 miimon=100
># ifconfig bond0 172.16.0.1 netmask 255.255.255.0
># ifconfig bond0 hw ether 12:34:56:78:9a
># ifenslave bond0 tun0 tun1

It should fail here, as tun (from the first approach*) doesn't have
ndo_set_mac_address, and bonding relies on that, *unless* fail_over_mac is
set to 1.

Could you try setting fail_over_mac = 1 on modprobe or via sysfs and
retesting?

* I've tried to quickly recreate your setup but hit a panic (already
submitted a fix).

Also, enabling debug for bonding (via dynamic_debug or anything else) and
attaching the output would help a lot.

Thank you!

>
>on one side and
>
># modprobe bonding mode=0 miimon=100
># ifconfig bond0 172.16.0.2 netmask 255.255.255.0
># ifconfig bond0 hw ether 12:35
># ifenslave -f bond0 tun0 tun1
># ping 172.16.0.1
>
>on the other side. On a newer kernel (>3.14) this doesn't work (at 
>least not on linux-armv7), while on 3.13 it does work under 
>linux-armv7. Now I've been told to use the iproute2-suite, as 
>ifenslave is deprecated. But if I try to do
>
># ssh -NTCf -w 0:0 web
># ssh -NTCf -w 1:1 web
># ip link add name bond1 type bond
># ip link set dev tun0 master bond1
>
>it gives me an error
>
>RTNETLINK answers: Operation not supported
>
>Any idea what I'm doing wrong?
>
>Linus
>--
>To unsubscribe from this list: send the line "unsubscribe netdev" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bonding with tun-devices

[Nikolay Aleksandrov]

On 05/21/2014 03:59 PM, Linus Gasser wrote:
> Dear list,
> 
> I'm trying to create a bonded interface using tun-devices created by the
> 
> ssh -NTCf -w 0:0
> 
> command. The reason is that I'm behind a satellite connection with per-stream
> limited bandwidth...
> 
> On some older kernels everything was fine, and I could do:
> 
> # ssh -NTCf -w 0:0 web
> # ssh -NTCf -w 1:1 web
> # modprobe bonding mode=0 miimon=100
> # ifconfig bond0 172.16.0.1 netmask 255.255.255.0
> # ifconfig bond0 hw ether 12:34:56:78:9a
> # ifenslave bond0 tun0 tun1
> 
> on one side and
> 
> # modprobe bonding mode=0 miimon=100
> # ifconfig bond0 172.16.0.2 netmask 255.255.255.0
> # ifconfig bond0 hw ether 12:35
> # ifenslave -f bond0 tun0 tun1
> # ping 172.16.0.1
> 
> on the other side. On a newer kernel (>3.14) this doesn't work (at least not on
> linux-armv7), while on 3.13 it does work under linux-armv7. Now I've been told
> to use the iproute2-suite, as ifenslave is deprecated. But if I try to do
> 
> # ssh -NTCf -w 0:0 web
> # ssh -NTCf -w 1:1 web
> # ip link add name bond1 type bond
> # ip link set dev tun0 master bond1
> 
> it gives me an error
> 
> RTNETLINK answers: Operation not supported
> 
> Any idea what I'm doing wrong?
> 
> Linus
> -- 
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Hi,
Could you please try modprobe bonding mode=0 miimon=100 fail_over_mac=active ?
I know that fail_over_mac doesn't affect that mode, but I think I can see why
it's happening.

Cheers,
 Nik

Re: Bonding with tun-devices

[Linus Gasser]

Le 21/05/14 16:54, Veaceslav Falico a écrit :
> On Wed, May 21, 2014 at 02:59:22PM +0100, Linus Gasser wrote:
>> Dear list,
>>
>> I'm trying to create a bonded interface using tun-devices created by the
>>
>> ssh -NTCf -w 0:0
>>
>> command. The reason is that I'm behind a satellite connection with
>> per-stream limited bandwidth...
>>
>> On some older kernels everything was fine, and I could do:
>>
>> # ssh -NTCf -w 0:0 web
>> # ssh -NTCf -w 1:1 web
>> # modprobe bonding mode=0 miimon=100
>> # ifconfig bond0 172.16.0.1 netmask 255.255.255.0
>> # ifconfig bond0 hw ether 12:34:56:78:9a
>> # ifenslave bond0 tun0 tun1
>
> It should fail here, as tun (from the first approach*) doesn't have
> ndo_set_mac_address, and bonding relies on that, *unless* fail_over_mac is
> set to 1.
>
> Could you try setting fail_over_mac = 1 on modprobe or via sysfs and
> retesting?
>
> * I've tried to quickly recreate your setup but hit a panic (already
> submitted a fix).
>
> Also, enabling debug for bonding (via dynamic_debug or anything else) and
> attaching the output would help a lot.

Hi,

OK, I tried to add "fail_over_mac=1" to the modprobe-call, but both with 
"ifenslave" and "ip add" I get the same error. In the logs I found (with 
fail_over_mac):

kernel: bonding: Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
kernel: bonding: MII link monitoring set to 100 ms
kernel: bonding: Warning: fail_over_mac only affects active-backup mode.
bonding: bond0: Warning: The first slave device specified does not 
support setting the MAC address.
bonding: bond0: Warning: The first slave device specified does not 
support setting the MAC address.

As far as I see, my kernel doesn't have dynamic_debug enabled. I'm in 
the midst of downloading it - hold on for a day or two ;) If you can 
tell me how I can debug otherwise, I'll be glad to do so.

Linus

>
> Thank you!
>
>>
>> on one side and
>>
>> # modprobe bonding mode=0 miimon=100
>> # ifconfig bond0 172.16.0.2 netmask 255.255.255.0
>> # ifconfig bond0 hw ether 12:35
>> # ifenslave -f bond0 tun0 tun1
>> # ping 172.16.0.1
>>
>> on the other side. On a newer kernel (>3.14) this doesn't work (at
>> least not on linux-armv7), while on 3.13 it does work under
>> linux-armv7. Now I've been told to use the iproute2-suite, as
>> ifenslave is deprecated. But if I try to do
>>
>> # ssh -NTCf -w 0:0 web
>> # ssh -NTCf -w 1:1 web
>> # ip link add name bond1 type bond
>> # ip link set dev tun0 master bond1
>>
>> it gives me an error
>>
>> RTNETLINK answers: Operation not supported
>>
>> Any idea what I'm doing wrong?
>>
>> Linus
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Bonding with tun-devices

[Veaceslav Falico]

On Wed, May 21, 2014 at 09:36:34PM +0100, Linus Gasser wrote:
>Le 21/05/14 16:54, Veaceslav Falico a écrit :
>>On Wed, May 21, 2014 at 02:59:22PM +0100, Linus Gasser wrote:
>>>Dear list,
>>>
>>>I'm trying to create a bonded interface using tun-devices created by the
>>>
>>>ssh -NTCf -w 0:0
>>>
>>>command. The reason is that I'm behind a satellite connection with
>>>per-stream limited bandwidth...
>>>
>>>On some older kernels everything was fine, and I could do:
>>>
>>># ssh -NTCf -w 0:0 web
>>># ssh -NTCf -w 1:1 web
>>># modprobe bonding mode=0 miimon=100
>>># ifconfig bond0 172.16.0.1 netmask 255.255.255.0
>>># ifconfig bond0 hw ether 12:34:56:78:9a
>>># ifenslave bond0 tun0 tun1
>>
>>It should fail here, as tun (from the first approach*) doesn't have
>>ndo_set_mac_address, and bonding relies on that, *unless* fail_over_mac is
>>set to 1.
>>
>>Could you try setting fail_over_mac = 1 on modprobe or via sysfs and
>>retesting?
>>
>>* I've tried to quickly recreate your setup but hit a panic (already
>>submitted a fix).
>>
>>Also, enabling debug for bonding (via dynamic_debug or anything else) and
>>attaching the output would help a lot.
>
>Hi,
>
>OK, I tried to add "fail_over_mac=1" to the modprobe-call, but both 
>with "ifenslave" and "ip add" I get the same error. In the logs I 
>found (with fail_over_mac):
>
>kernel: bonding: Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
>kernel: bonding: MII link monitoring set to 100 ms
>kernel: bonding: Warning: fail_over_mac only affects active-backup mode.
>bonding: bond0: Warning: The first slave device specified does not 
>support setting the MAC address.
>bonding: bond0: Warning: The first slave device specified does not 
>support setting the MAC address.
>
>As far as I see, my kernel doesn't have dynamic_debug enabled. I'm in 
>the midst of downloading it - hold on for a day or two ;) If you can 
>tell me how I can debug otherwise, I'll be glad to do so.

No worries, I figured it out why it's misbehaving. Basically now bonding
expects every slave to have support of setting the mac address (that's in
short), but tun device is a IP device, which doesn't support mac addresses.

A workaround would be to use tap devices, as they're more "real" and
support mac address setting.

I'll, though, try to figure out how to make bonding work even on NOARP
devices if it doesn't need to fiddle with OSI level 2...

Hope that helps.

>
>Linus
>
>>
>>Thank you!
>>
>>>
>>>on one side and
>>>
>>># modprobe bonding mode=0 miimon=100
>>># ifconfig bond0 172.16.0.2 netmask 255.255.255.0
>>># ifconfig bond0 hw ether 12:35
>>># ifenslave -f bond0 tun0 tun1
>>># ping 172.16.0.1
>>>
>>>on the other side. On a newer kernel (>3.14) this doesn't work (at
>>>least not on linux-armv7), while on 3.13 it does work under
>>>linux-armv7. Now I've been told to use the iproute2-suite, as
>>>ifenslave is deprecated. But if I try to do
>>>
>>># ssh -NTCf -w 0:0 web
>>># ssh -NTCf -w 1:1 web
>>># ip link add name bond1 type bond
>>># ip link set dev tun0 master bond1
>>>
>>>it gives me an error
>>>
>>>RTNETLINK answers: Operation not supported
>>>
>>>Any idea what I'm doing wrong?
>>>
>>>Linus
>>>--
>>>To unsubscribe from this list: send the line "unsubscribe netdev" in
>>>the body of a message to majordomo@vger.kernel.org
>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>--
>>To unsubscribe from this list: send the line "unsubscribe netdev" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: Bonding with tun-devices

[Eric Dumazet]

On Thu, 2014-05-22 at 15:24 +0200, Veaceslav Falico wrote:

> No worries, I figured it out why it's misbehaving. Basically now bonding
> expects every slave to have support of setting the mac address (that's in
> short), but tun device is a IP device, which doesn't support mac addresses.
> 
> A workaround would be to use tap devices, as they're more "real" and
> support mac address setting.
> 
> I'll, though, try to figure out how to make bonding work even on NOARP
> devices if it doesn't need to fiddle with OSI level 2...

Yes, it sounds some regression was added at some point.

It shouldnt be difficult to fix this.
 

Re: Bonding with tun-devices

[Linus Gasser]

Le 22/05/14 14:24, Veaceslav Falico a écrit :
> On Wed, May 21, 2014 at 09:36:34PM +0100, Linus Gasser wrote:
>> Le 21/05/14 16:54, Veaceslav Falico a écrit :
>>> On Wed, May 21, 2014 at 02:59:22PM +0100, Linus Gasser wrote:
>>>> Dear list,
>>>>
>>>> I'm trying to create a bonded interface using tun-devices created by
>>>> the
>>>>
>>>> ssh -NTCf -w 0:0
>>>>
>>>> command. The reason is that I'm behind a satellite connection with
>>>> per-stream limited bandwidth...
>>>>
>>>> On some older kernels everything was fine, and I could do:
>>>>
>>>> # ssh -NTCf -w 0:0 web
>>>> # ssh -NTCf -w 1:1 web
>>>> # modprobe bonding mode=0 miimon=100
>>>> # ifconfig bond0 172.16.0.1 netmask 255.255.255.0
>>>> # ifconfig bond0 hw ether 12:34:56:78:9a
>>>> # ifenslave bond0 tun0 tun1
>>>
>>> It should fail here, as tun (from the first approach*) doesn't have
>>> ndo_set_mac_address, and bonding relies on that, *unless*
>>> fail_over_mac is
>>> set to 1.
>>>
>>> Could you try setting fail_over_mac = 1 on modprobe or via sysfs and
>>> retesting?
>>>
>>> * I've tried to quickly recreate your setup but hit a panic (already
>>> submitted a fix).
>>>
>>> Also, enabling debug for bonding (via dynamic_debug or anything else)
>>> and
>>> attaching the output would help a lot.
>>
>> Hi,
>>
>> OK, I tried to add "fail_over_mac=1" to the modprobe-call, but both
>> with "ifenslave" and "ip add" I get the same error. In the logs I
>> found (with fail_over_mac):
>>
>> kernel: bonding: Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
>> kernel: bonding: MII link monitoring set to 100 ms
>> kernel: bonding: Warning: fail_over_mac only affects active-backup mode.
>> bonding: bond0: Warning: The first slave device specified does not
>> support setting the MAC address.
>> bonding: bond0: Warning: The first slave device specified does not
>> support setting the MAC address.
>>
>> As far as I see, my kernel doesn't have dynamic_debug enabled. I'm in
>> the midst of downloading it - hold on for a day or two ;) If you can
>> tell me how I can debug otherwise, I'll be glad to do so.
>
> No worries, I figured it out why it's misbehaving. Basically now bonding
> expects every slave to have support of setting the mac address (that's in
> short), but tun device is a IP device, which doesn't support mac addresses.
>
> A workaround would be to use tap devices, as they're more "real" and
> support mac address setting.

OK, what can I say - thank you for helping me not to have fiddled around 
with it more than needed ;) If you manage to support 
non-mac-connections, this would be great, of course. Else I'll have to 
try Openvpn which supports tap-devices.

Thanks for your efforts

>
> I'll, though, try to figure out how to make bonding work even on NOARP
> devices if it doesn't need to fiddle with OSI level 2...
>
> Hope that helps.
>
>>
>> Linus
>>
>>>
>>> Thank you!
>>>
>>>>
>>>> on one side and
>>>>
>>>> # modprobe bonding mode=0 miimon=100
>>>> # ifconfig bond0 172.16.0.2 netmask 255.255.255.0
>>>> # ifconfig bond0 hw ether 12:35
>>>> # ifenslave -f bond0 tun0 tun1
>>>> # ping 172.16.0.1
>>>>
>>>> on the other side. On a newer kernel (>3.14) this doesn't work (at
>>>> least not on linux-armv7), while on 3.13 it does work under
>>>> linux-armv7. Now I've been told to use the iproute2-suite, as
>>>> ifenslave is deprecated. But if I try to do
>>>>
>>>> # ssh -NTCf -w 0:0 web
>>>> # ssh -NTCf -w 1:1 web
>>>> # ip link add name bond1 type bond
>>>> # ip link set dev tun0 master bond1
>>>>
>>>> it gives me an error
>>>>
>>>> RTNETLINK answers: Operation not supported
>>>>
>>>> Any idea what I'm doing wrong?
>>>>
>>>> Linus
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Bonding with tun-devices

[Veaceslav Falico]

On Thu, May 22, 2014 at 03:50:05PM +0100, Linus Gasser wrote:
>Le 22/05/14 14:24, Veaceslav Falico a écrit :
>>A workaround would be to use tap devices, as they're more "real" and
>>support mac address setting.
>
>OK, what can I say - thank you for helping me not to have fiddled 
>around with it more than needed ;) If you manage to support 
>non-mac-connections, this would be great, of course. Else I'll have 
>to try Openvpn which supports tap-devices.

Quick googling says that ssh -o Tunnel=ethernet creates a tap device, so no
need for openvpn. Disclaimer - I've never actually tried it, so dunno if
it'll work.

I'll add you on CC once the patch supporting tun is sent.

Hope that helps.

>
>Thanks for your efforts

Re: Bonding with tun-devices

[Linus Gasser]

Le 22/05/14 16:00, Veaceslav Falico a écrit :
> On Thu, May 22, 2014 at 03:50:05PM +0100, Linus Gasser wrote:
>> Le 22/05/14 14:24, Veaceslav Falico a écrit :
>>> A workaround would be to use tap devices, as they're more "real" and
>>> support mac address setting.
>>
>> OK, what can I say - thank you for helping me not to have fiddled
>> around with it more than needed ;) If you manage to support
>> non-mac-connections, this would be great, of course. Else I'll have to
>> try Openvpn which supports tap-devices.
>
> Quick googling says that ssh -o Tunnel=ethernet creates a tap device, so no
> need for openvpn. Disclaimer - I've never actually tried it, so dunno if
> it'll work.

I tried that, but I still only get tun-devices. Don't know why it 
doesn't want to generate tap-devices, as both man-pages and 
google-results say I could. Waiting for your patch then,

Linus