2.6.29 regression? Bonding tied to IPV6 in 29-rc5

2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]

Forward to bonding and netdev

On 17 of February 2009 11:52:32 J.A. Magallón wrote:
> Hi all...
>
> Don't know if this is specific for -rc5, I have jumped from 28.4 to
> 29-rc5. In this latest kernel, I can not install 'bonding' module if
> 'ipv6' is disabled to load via modprobe.conf:
>
> install ipv6 /bin/true
>
> Trying bonding gives this dmesg:
>
> bonding: Unknown symbol ndisc_build_skb
> bonding: Unknown symbol in6_dev_finish_destroy
> bonding: Unknown symbol ndisc_send_skb
> bonding: Unknown symbol unregister_inet6addr_notifier
> bonding: Unknown symbol register_inet6addr_notifier
>
> Commenting the line in modprobe.conf makes things smooth again.
> We can not disable ipv6 anymore ?
>

If IPv6 is disable in kernel config bonding loads. But I think it is 
regression, it should be possible to disable IPv6 if not required.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On 17 of February 2009 20:01:38 Andrey Borzenkov wrote:
> Forward to bonding and netdev
>
> On 17 of February 2009 11:52:32 J.A. Magallón wrote:
> > Hi all...
> >
> > Don't know if this is specific for -rc5, I have jumped from 28.4 to
> > 29-rc5. In this latest kernel, I can not install 'bonding' module
> > if 'ipv6' is disabled to load via modprobe.conf:
> >
> > install ipv6 /bin/true
> >
> > Trying bonding gives this dmesg:
> >
> > bonding: Unknown symbol ndisc_build_skb
> > bonding: Unknown symbol in6_dev_finish_destroy
> > bonding: Unknown symbol ndisc_send_skb
> > bonding: Unknown symbol unregister_inet6addr_notifier
> > bonding: Unknown symbol register_inet6addr_notifier
> >
> > Commenting the line in modprobe.conf makes things smooth again.
> > We can not disable ipv6 anymore ?
>
> If IPv6 is disable in kernel config bonding loads. But I think it is
> regression, it should be possible to disable IPv6 if not required.

This hard dependency was apparently introduced by this commit:

commit 305d552accae6afb859c493ebc7d98ca3371dae2
Author: Brian Haley <brian.haley@hp.com>
Date:   Tue Nov 4 17:51:14 2008 -0800

    bonding: send IPv6 neighbor advertisement on failover


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Jay Vosburgh]

Andrey Borzenkov <arvidjaar@mail.ru> wrote:

>> If IPv6 is disable in kernel config bonding loads. But I think it is
>> regression, it should be possible to disable IPv6 if not required.
>
>This hard dependency was apparently introduced by this commit:
>
>commit 305d552accae6afb859c493ebc7d98ca3371dae2
>Author: Brian Haley <brian.haley@hp.com>
>Date:   Tue Nov 4 17:51:14 2008 -0800
>
>    bonding: send IPv6 neighbor advertisement on failover

	I'm not really sure how to work around this.  If bonding is
compiled with CONFIG_IPV6, then the IPv6 support is compiled in, and
bonding will need the ipv6 module loaded to resolve its symbols.

	The simple answer (don't turn on CONFIG_IPV6) isn't really
useful for the common case of distro kernels, which will generally have
CONFIG_IPV6 enabled.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Jay Vosburgh <fubar@us.ibm.com>
Date: Tue, 17 Feb 2009 12:08:03 -0800

> 	The simple answer (don't turn on CONFIG_IPV6) isn't really
> useful for the common case of distro kernels, which will generally have
> CONFIG_IPV6 enabled.

This whole "disable ipv6 module load in /etc/modules.conf" was
bound to eventually cause problems like this.

There are so many chicken-and-egg issues wrt. this it isn't
even funny, for example:

1) If we provide a sysctl which is a bitmask of protocols
   to disallow registering, we have the same problem we have
   now since ipv6 won't be able to load when the sock_register()
   fails.

2) If we add some ipv6 sysctl that says "don't do ipv6" so the
   module can load yet not automatically add link local ipv6
   addresses to interfaces and solicit for routers on the network,
   the sysctl can't be set until the module is loaded.

And this bonding stuff in particular wants to get into the
neighbour discovery state of the ipv6 module.  Therefore it's
not like we can split out a few functions into some kind
of "ipv6_helpers.ko" kernel module to eliminate the problem
either.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Brian Haley]

Andrey Borzenkov wrote:
> On 17 of February 2009 20:01:38 Andrey Borzenkov wrote:
>> Forward to bonding and netdev
>>
>> On 17 of February 2009 11:52:32 J.A. Magallón wrote:
>>> Hi all...
>>>
>>> Don't know if this is specific for -rc5, I have jumped from 28.4 to
>>> 29-rc5. In this latest kernel, I can not install 'bonding' module
>>> if 'ipv6' is disabled to load via modprobe.conf:
>>>
>>> install ipv6 /bin/true
>>>
>>> Trying bonding gives this dmesg:
>>>
>>> bonding: Unknown symbol ndisc_build_skb
>>> bonding: Unknown symbol in6_dev_finish_destroy
>>> bonding: Unknown symbol ndisc_send_skb
>>> bonding: Unknown symbol unregister_inet6addr_notifier
>>> bonding: Unknown symbol register_inet6addr_notifier
>>>
>>> Commenting the line in modprobe.conf makes things smooth again.
>>> We can not disable ipv6 anymore ?
>> If IPv6 is disable in kernel config bonding loads. But I think it is
>> regression, it should be possible to disable IPv6 if not required.
> 
> This hard dependency was apparently introduced by this commit:
> 
> commit 305d552accae6afb859c493ebc7d98ca3371dae2
> Author: Brian Haley <brian.haley@hp.com>
> Date:   Tue Nov 4 17:51:14 2008 -0800
> 
>     bonding: send IPv6 neighbor advertisement on failover

I initially had bonding IPv6 support as a Kconfig option, but it was 
decided it would be cleaner if it just got built-in whenever CONFIG_IPV6 
was set like SCTP, with the assumption you might want it.

Is it a common configuration to not allow a module to load like you're 
doing in modprobe.conf?  I don't know how hard it would be to rip this 
out into it's own bonding_ipv6.ko module, simply turning-off CONFIG_IPV6 
seems better.

-Brian

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Thomas Backlund]

Brian Haley skrev:
> Andrey Borzenkov wrote:
>> On 17 of February 2009 20:01:38 Andrey Borzenkov wrote:
>>> Forward to bonding and netdev
>>>
>>> On 17 of February 2009 11:52:32 J.A. Magallón wrote:
>>>> Hi all...
>>>>
>>>> Don't know if this is specific for -rc5, I have jumped from 28.4 to
>>>> 29-rc5. In this latest kernel, I can not install 'bonding' module
>>>> if 'ipv6' is disabled to load via modprobe.conf:
>>>>
>>>> install ipv6 /bin/true
>>>>
>>>> Trying bonding gives this dmesg:
>>>>
>>>> bonding: Unknown symbol ndisc_build_skb
>>>> bonding: Unknown symbol in6_dev_finish_destroy
>>>> bonding: Unknown symbol ndisc_send_skb
>>>> bonding: Unknown symbol unregister_inet6addr_notifier
>>>> bonding: Unknown symbol register_inet6addr_notifier
>>>>
>>>> Commenting the line in modprobe.conf makes things smooth again.
>>>> We can not disable ipv6 anymore ?
>>> If IPv6 is disable in kernel config bonding loads. But I think it is
>>> regression, it should be possible to disable IPv6 if not required.
>>
>> This hard dependency was apparently introduced by this commit:
>>
>> commit 305d552accae6afb859c493ebc7d98ca3371dae2
>> Author: Brian Haley <brian.haley@hp.com>
>> Date:   Tue Nov 4 17:51:14 2008 -0800
>>
>>     bonding: send IPv6 neighbor advertisement on failover
> 
> I initially had bonding IPv6 support as a Kconfig option, but it was 
> decided it would be cleaner if it just got built-in whenever CONFIG_IPV6 
> was set like SCTP, with the assumption you might want it.
> 
> Is it a common configuration to not allow a module to load like you're 
> doing in modprobe.conf?  I don't know how hard it would be to rip this 

Yep.

> out into it's own bonding_ipv6.ko module, simply turning-off CONFIG_IPV6 
> seems better.

That's not an option for distro kernels ...

--
Thomas

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Jay Vosburgh]

Brian Haley <brian.haley@hp.com> wrote:

>Andrey Borzenkov wrote:
[...]
>> This hard dependency was apparently introduced by this commit:
>> 
>> commit 305d552accae6afb859c493ebc7d98ca3371dae2
>> Author: Brian Haley <brian.haley@hp.com>
>> Date:   Tue Nov 4 17:51:14 2008 -0800
>> 
>>     bonding: send IPv6 neighbor advertisement on failover
>
>I initially had bonding IPv6 support as a Kconfig option, but it was 
>decided it would be cleaner if it just got built-in whenever CONFIG_IPV6 
>was set like SCTP, with the assumption you might want it.
>
>Is it a common configuration to not allow a module to load like you're 
>doing in modprobe.conf?  I don't know how hard it would be to rip this 
>out into it's own bonding_ipv6.ko module, simply turning-off CONFIG_IPV6 
>seems better.

	I'm not sure either of those really helps.  Distro kernels are
built with CONFIG_IPV6 (and would have the CONFIG_BONDING_IPV6_DINGUS
enabled as well), so the common case users would have it enabled, too.

	Putting the ipv6 bits into a different module might not help,
either, because the "core" bonding code would still have the call to the
ipv6 functions.  Unless there's some magic way to somehow know at
runtime whether or not the ipv6 module is loaded, and only try to
resolve those symbols if ipv6 is loaded.  That seems complicated.

	To answer your question, I have come across this (aliasing ipv6
to nothing in modprobe.conf to disable IPv6) from time to time, but
didn't think of it when the NA code was added to bonding.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Brian Haley]

Jay Vosburgh wrote:
> Brian Haley <brian.haley@hp.com> wrote:
> 
>> Andrey Borzenkov wrote:
> [...]
>>> This hard dependency was apparently introduced by this commit:
>>>
>>> commit 305d552accae6afb859c493ebc7d98ca3371dae2
>>> Author: Brian Haley <brian.haley@hp.com>
>>> Date:   Tue Nov 4 17:51:14 2008 -0800
>>>
>>>     bonding: send IPv6 neighbor advertisement on failover
>> I initially had bonding IPv6 support as a Kconfig option, but it was 
>> decided it would be cleaner if it just got built-in whenever CONFIG_IPV6 
>> was set like SCTP, with the assumption you might want it.
>>
>> Is it a common configuration to not allow a module to load like you're 
>> doing in modprobe.conf?  I don't know how hard it would be to rip this 
>> out into it's own bonding_ipv6.ko module, simply turning-off CONFIG_IPV6 
>> seems better.
> 
> 	I'm not sure either of those really helps.  Distro kernels are
> built with CONFIG_IPV6 (and would have the CONFIG_BONDING_IPV6_DINGUS
> enabled as well), so the common case users would have it enabled, too.

I think that was one of the reasons too.

> 	Putting the ipv6 bits into a different module might not help,
> either, because the "core" bonding code would still have the call to the
> ipv6 functions.  Unless there's some magic way to somehow know at
> runtime whether or not the ipv6 module is loaded, and only try to
> resolve those symbols if ipv6 is loaded.  That seems complicated.

This separate bonding_ipv6 module would have to register itself with the 
"core" one with a new proto_ops of some sort.  Calls are made for the 
appropriate method, for example bond_ops->send_gratuitous(bond).  We'd 
change the IPv4 code too.  It's just a theory, does make things more 
complicated.

> 	To answer your question, I have come across this (aliasing ipv6
> to nothing in modprobe.conf to disable IPv6) from time to time, but
> didn't think of it when the NA code was added to bonding.

So I guess I'll start hacking the above, unless someone has a better 
suggestion.

-Brian

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Jay Vosburgh]

Brian Haley <brian.haley@hp.com> wrote:

>Jay Vosburgh wrote:
[...]
>> 	Putting the ipv6 bits into a different module might not help,
>> either, because the "core" bonding code would still have the call to the
>> ipv6 functions.  Unless there's some magic way to somehow know at
>> runtime whether or not the ipv6 module is loaded, and only try to
>> resolve those symbols if ipv6 is loaded.  That seems complicated.
>
>This separate bonding_ipv6 module would have to register itself with the
>"core" one with a new proto_ops of some sort.  Calls are made for the
>appropriate method, for example bond_ops->send_gratuitous(bond).  We'd
>change the IPv4 code too.  It's just a theory, does make things more
>complicated.

	I don't see any reason to change the IPv4 bits; there won't ever
be a case of ipv4 not being loaded, and this would just add the
complexity of a registration gizmo with no real benefit.  A "bonding
ipv6 ops" would be strictly a hack to deal with ipv6 module dependencies
for cases when the kernel is built with CONFIG_IPV6 but the ipv6 module
itself is prevented from loading.

	A registration gizmo doesn't need to be especially complicated;
there's only three functions in bond_ipv6.c that are called from the
bonding core: bond_send_unsolicited_na, and the ipv6 notifier register /
unregister.  The bonding_ipv6 module can simply be bond_ipv6.c, which
calls some exported "hey, bond_send_unsol_na is here" thing in the
bonding core during init and another "hey, send_unsol_na is gone" during
unload.  The bonding_ipv6 module can do its own notifier registration
handing.

>> 	To answer your question, I have come across this (aliasing ipv6
>> to nothing in modprobe.conf to disable IPv6) from time to time, but
>> didn't think of it when the NA code was added to bonding.
>
>So I guess I'll start hacking the above, unless someone has a better
>suggestion.

	Well, I think this is pretty heinous, but I don't have a better
idea at the moment.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Jan Engelhardt]

On Tuesday 2009-02-17 23:24, Jay Vosburgh wrote:
>Brian Haley <brian.haley@hp.com> wrote:
>
>	I don't see any reason to change the IPv4 bits;
>
>there won't ever be a case of ipv4 not being loaded,

I would not be so sure of that.

>>> 	To answer your question, I have come across this (aliasing ipv6
>>> to nothing in modprobe.conf to disable IPv6) from time to time, but
>>> didn't think of it when the NA code was added to bonding.
>>
>>So I guess I'll start hacking the above, unless someone has a better
>>suggestion.
>
>	Well, I think this is pretty heinous, but I don't have a better
>idea at the moment.

IMO, a better solution could be to add a dummy ipv6 module that returns 
-EOPNOTSOPP or appropriate for most calls.

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Nicolas de Pesloüan]

Jay Vosburgh wrote:
> 	I'm not sure either of those really helps.  Distro kernels are
> built with CONFIG_IPV6 (and would have the CONFIG_BONDING_IPV6_DINGUS
> enabled as well), so the common case users would have it enabled, too.
> 
> 	Putting the ipv6 bits into a different module might not help,
> either, because the "core" bonding code would still have the call to the
> ipv6 functions.  Unless there's some magic way to somehow know at
> runtime whether or not the ipv6 module is loaded, and only try to
> resolve those symbols if ipv6 is loaded.  That seems complicated.

What about aliasing ipv6 to a dummy module "dummy-ipv6-for-bonding" that 
only provide the required symbols and do (close to) nothing ?

Just my two cents.

     Nicolas.

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Jay Vosburgh <fubar@us.ibm.com>
Date: Tue, 17 Feb 2009 13:06:28 -0800

> 	Putting the ipv6 bits into a different module might not help,
> either, because the "core" bonding code would still have the call to the
> ipv6 functions.  Unless there's some magic way to somehow know at
> runtime whether or not the ipv6 module is loaded, and only try to
> resolve those symbols if ipv6 is loaded.  That seems complicated.

This is a non-starter, as I described in one of my other replies.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Brian Haley <brian.haley@hp.com>
Date: Tue, 17 Feb 2009 15:10:23 -0500

> Is it a common configuration to not allow a module to load like
> you're doing in modprobe.conf?  I don't know how hard it would be to
> rip this out into it's own bonding_ipv6.ko module, simply
> turning-off CONFIG_IPV6 seems better.

It's unfortunately how users get told to turn off ipv6.

There were some back-compat issues with how GLIBC emitted DNS requests
a few months ago that required turning off ipv6 completely otherwise
DNS requests would timeout.

Sticking an ipv6.ko module load disable into /etc/modprobe.conf
was the only tenable workaround.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Andrey Borzenkov <arvidjaar@mail.ru>
Date: Tue, 17 Feb 2009 20:01:38 +0300

> Forward to bonding and netdev
 ...
> If IPv6 is disable in kernel config bonding loads. But I think it is 
> regression, it should be possible to disable IPv6 if not required.

Don't configure ipv6 into your kernel, really.

There is no other way to handle this.  If we want to support
IPV6 layer things in the bonding driver, it is going to
call helper functions in the ipv6 module and therefore must
be able to load it and use functions in it.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1325 bytes --]

On Tue, 17 Feb 2009 14:29:46 PST, David Miller said:
> Don't configure ipv6 into your kernel, really.
> 
> There is no other way to handle this.  If we want to support
> IPV6 layer things in the bonding driver, it is going to
> call helper functions in the ipv6 module and therefore must
> be able to load it and use functions in it.

What does a poor corporate user do if they're running a distro kernel that
was built with CONFIG_IPV6, but local security policy says "Disable IPv6
because we don't do it yet, or because it breaks mission-critical software
package XYZ?"  There's a *lot* of people who implement that by the "block
the ipv6 module from loading" trick.  And building a kernel that doesn't
include IPv6 may not be feasible due to vendor certification issues...

Heck, *I*'m almost in that boat - probably need to use bonded ethernet on some
servers because we can't get 10GigE, but the software used in the project the
servers were bought for blows chunks if it gets a whiff of an IPv6 address.
Ended up spending 3 weeks doing a massive kludgery of one sort in DNS for the
rest of the world, and equally massive lying in /etc/hosts for the hosts...
(Don't ask - it was long and ugly, and just disabling the module would have
saved me about 2.95 weeks of work, so I know where those people are coming
from...)


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Valdis.Kletnieks@vt.edu
Date: Tue, 17 Feb 2009 23:41:16 -0500

> What does a poor corporate user do if they're running a distro kernel that
> was built with CONFIG_IPV6, but local security policy says "Disable IPv6
> because we don't do it yet, or because it breaks mission-critical software
> package XYZ?"  There's a *lot* of people who implement that by the "block
> the ipv6 module from loading" trick.  And building a kernel that doesn't
> include IPv6 may not be feasible due to vendor certification issues...
> 
> Heck, *I*'m almost in that boat - probably need to use bonded ethernet on some
> servers because we can't get 10GigE, but the software used in the project the
> servers were bought for blows chunks if it gets a whiff of an IPv6 address.
> Ended up spending 3 weeks doing a massive kludgery of one sort in DNS for the
> rest of the world, and equally massive lying in /etc/hosts for the hosts...
> (Don't ask - it was long and ugly, and just disabling the module would have
> saved me about 2.95 weeks of work, so I know where those people are coming
> from...)

Well, first of all, if you keep trying to push the box into the round
hole you get what you ask for :-)

Next, if it's just an issue of IPV6 traffic, install a packet
scheduler rule that rejects all packets with ethernet proto
ETH_P_IPV6

If openning up ipv6 sockets is problematic, that can be blocked
using the security layer, which your super-duper distro kernel
is guarenteed to have enabled. :-)

I'm sure there is someone who has legacy problems with ipv4
and that can't be disabled, and somehow people cope.  Amazing.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Roland Dreier]

 > Next, if it's just an issue of IPV6 traffic, install a packet
 > scheduler rule that rejects all packets with ethernet proto
 > ETH_P_IPV6
 > 
 > If openning up ipv6 sockets is problematic, that can be blocked
 > using the security layer, which your super-duper distro kernel
 > is guarenteed to have enabled. :-)

This reminds me of a related issue that some users have run into with
IP-over-InfiniBand -- the IPoIB module also depends on ipv6 symbols if
ipv6 is enabled, so when ipoib is loaded then ipv6 gets loaded.  And
this causes a problem from some people in the IB case because assigning
an ipv6 address to the ipoib interface actually consumes some network
resources (the number of multicast groups that the network supports may
be limited and having a solicited node multicast group for each node may
use them all up).

Anyway, this leads to the folowing question: is there a way to prevent a
link-local ipv6 address from being configured automatically for the
ipoib interface?

Thanks,
  Roland

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Theodore Tso]

On Tue, Feb 17, 2009 at 09:29:19PM -0800, David Miller wrote:
> Next, if it's just an issue of IPV6 traffic, install a packet
> scheduler rule that rejects all packets with ethernet proto
> ETH_P_IPV6
> 
> If openning up ipv6 sockets is problematic, that can be blocked
> using the security layer, which your super-duper distro kernel
> is guarenteed to have enabled. :-)
> 
> I'm sure there is someone who has legacy problems with ipv4
> and that can't be disabled, and somehow people cope.  Amazing.

The reality is that there are far more people who have legacy problems
with ipv6 than ipv4 (which has been around and in active use for about
3 decades, after all), whereas ipv6 has been around and largely
ignored for about a decade.  :-/

I'll admit that I ran into some wierd sh*t problems with some open
source software or another failing mysteriously when IPv6 was enabled,
and I dealt with it by simply disabling IPv6 (yeah, I blocked the
module).  I was in a hurry, and it just didn't work, and I had better
thing to do than to spend time trying to debug why the presense of an
IPv6 enabled interface caused programs to misbehave in random ways.

I think I can pretty much guarantee that distro users will be
clamoring for a quick and easy way to block ipv6, and it's in our
interest to document the recomended way to block it that doesn't cause
weird problems with bonding, etc.

						- Ted

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Chuck Lever]

On Feb 18, 2009, at Feb 18, 2009, 8:55 AM, Theodore Tso wrote:
> On Tue, Feb 17, 2009 at 09:29:19PM -0800, David Miller wrote:
>> Next, if it's just an issue of IPV6 traffic, install a packet
>> scheduler rule that rejects all packets with ethernet proto
>> ETH_P_IPV6
>>
>> If openning up ipv6 sockets is problematic, that can be blocked
>> using the security layer, which your super-duper distro kernel
>> is guarenteed to have enabled. :-)
>>
>> I'm sure there is someone who has legacy problems with ipv4
>> and that can't be disabled, and somehow people cope.  Amazing.

Here's another "me too".

Kernel RPC support also has this problem.  We hit it just a couple of  
weeks ago now that we have IPv6 enabled NFS in prototype.  If PF_INET6  
listener creation fails (eg. because ipv6.ko is blacklisted), our  
workaround right now is to retry the listener socket creation with  
PF_INET.  There are plenty of somewhat rare corner cases that make  
this less than ideal.

> The reality is that there are far more people who have legacy problems
> with ipv6 than ipv4 (which has been around and in active use for about
> 3 decades, after all), whereas ipv6 has been around and largely
> ignored for about a decade.  :-/
>
> I'll admit that I ran into some wierd sh*t problems with some open
> source software or another failing mysteriously when IPv6 was enabled,
> and I dealt with it by simply disabling IPv6 (yeah, I blocked the
> module).  I was in a hurry, and it just didn't work, and I had better
> thing to do than to spend time trying to debug why the presense of an
> IPv6 enabled interface caused programs to misbehave in random ways.
>
> I think I can pretty much guarantee that distro users will be
> clamoring for a quick and easy way to block ipv6, and it's in our
> interest to document the recomended way to block it that doesn't cause
> weird problems with bonding, etc.

A better solution would be to design kernel and user space networking  
to handle this use case, instead of providing a workaround.  From the  
variety of comments I've heard, this use case is pretty common.

Considering the government mandates requiring IPv6 support (and the  
advertisements by Linux vendors claiming IPv6 support), IPv6 needs to  
become a first-class citizen in Linux in fairly short order.  It still  
feels a little piecemeal to me to be called "production ready."

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Vlad Yasevich]

Chuck Lever wrote:
> On Feb 18, 2009, at Feb 18, 2009, 8:55 AM, Theodore Tso wrote:
>> On Tue, Feb 17, 2009 at 09:29:19PM -0800, David Miller wrote:
>>> Next, if it's just an issue of IPV6 traffic, install a packet
>>> scheduler rule that rejects all packets with ethernet proto
>>> ETH_P_IPV6
>>>
>>> If openning up ipv6 sockets is problematic, that can be blocked
>>> using the security layer, which your super-duper distro kernel
>>> is guarenteed to have enabled. :-)
>>>
>>> I'm sure there is someone who has legacy problems with ipv4
>>> and that can't be disabled, and somehow people cope.  Amazing.
> 
> Here's another "me too".

And another "me too" for SCTP.  When IPv6 is turned on in the kernel
config file, we include the necessary pieces into the SCTP module and
sctp.ko will not load if ipv6.ko is blacklisted.

Having worked in other environments where ipv6 has to be explicitly
enabled per interface, I've thought that this level of control was
always missing from linux.  Being able to configure only the interface
that users want seems like a good thing to have.
Would a module parameter that disables ipv6 or at least addrconf be
enough of a solution?

-vlad

> 
> Kernel RPC support also has this problem.  We hit it just a couple of
> weeks ago now that we have IPv6 enabled NFS in prototype.  If PF_INET6
> listener creation fails (eg. because ipv6.ko is blacklisted), our
> workaround right now is to retry the listener socket creation with
> PF_INET.  There are plenty of somewhat rare corner cases that make this
> less than ideal.
> 
>> The reality is that there are far more people who have legacy problems
>> with ipv6 than ipv4 (which has been around and in active use for about
>> 3 decades, after all), whereas ipv6 has been around and largely
>> ignored for about a decade.  :-/
>>
>> I'll admit that I ran into some wierd sh*t problems with some open
>> source software or another failing mysteriously when IPv6 was enabled,
>> and I dealt with it by simply disabling IPv6 (yeah, I blocked the
>> module).  I was in a hurry, and it just didn't work, and I had better
>> thing to do than to spend time trying to debug why the presense of an
>> IPv6 enabled interface caused programs to misbehave in random ways.
>>
>> I think I can pretty much guarantee that distro users will be
>> clamoring for a quick and easy way to block ipv6, and it's in our
>> interest to document the recomended way to block it that doesn't cause
>> weird problems with bonding, etc.
> 
> A better solution would be to design kernel and user space networking to
> handle this use case, instead of providing a workaround.  From the
> variety of comments I've heard, this use case is pretty common.
> 
> Considering the government mandates requiring IPv6 support (and the
> advertisements by Linux vendors claiming IPv6 support), IPv6 needs to
> become a first-class citizen in Linux in fairly short order.  It still
> feels a little piecemeal to me to be called "production ready."
> 
> -- 
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
> -- 
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Brian Haley]

[-- Attachment #1: Type: text/plain, Size: 1289 bytes --]

Vlad Yasevich wrote:
> Having worked in other environments where ipv6 has to be explicitly
> enabled per interface, I've thought that this level of control was
> always missing from linux.  Being able to configure only the interface
> that users want seems like a good thing to have.
> Would a module parameter that disables ipv6 or at least addrconf be
> enough of a solution?

There does seem to be a sysctl for it, just doesn't seem to work. 
Possible patch below.

This actually brings up the issue that the "all" ipv6 sysctl, for 
example net.ipv6.conf.all.disable_ipv6, doesn't actually do anything (at 
least it didn't seem to for me).  Maybe it's time to fix that too to be 
like IPv4, things like IN_DEV_RPFILTER() and friends aren't looking so 
bad...

I tested this patch on lo and a few Ethernet devices and saw no IPv6 
addresses.  Don't know if EPERM is the right errno since we don't know 
if the user set this or DAD failed.


The disable_ipv6 knob was meant to be used for the kernel to disable 
IPv6 on an interface when DAD failed for the link-local address based on 
the MAC, but we should also be able to administratively disable it on an 
interface, or the entire system.  This patch fixes the per-interface 
problem.

Signed-off-by: Brian Haley <brian.haley@hp.com>

[-- Attachment #2: noipv6.patch --]
[-- Type: text/x-diff, Size: 421 bytes --]

diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index 03e2a1a..9bc761f 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -603,6 +603,11 @@ ipv6_add_addr(struct inet6_dev *idev, const struct in6_addr *addr, int pfxlen,
 		goto out2;
 	}
 
+	if (idev->cnf.disable_ipv6) {
+		err = -EPERM;
+		goto out2;
+	}
+
 	write_lock(&addrconf_hash_lock);
 
 	/* Ignore adding duplicate addresses on an interface */

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[John Dykstra]

On Wed, 2009-02-18 at 14:57 -0500, Brian Haley wrote: 
> Vlad Yasevich wrote:
> > Having worked in other environments where ipv6 has to be explicitly
> > enabled per interface, I've thought that this level of control was
> > always missing from linux.  
> 
> There does seem to be a sysctl for it, just doesn't seem to work.

While this sysctl is definitely useful, I don't think it meets the needs
of those users and distributions that are currently blacklisting the
ipv6 module.  Even if you disable IPv6 on all interfaces, apps will
still be able to create AF_INET6 sockets, and thus some apps will break
or do inappropriate things because there isn't real IPv6 connectivity.

I've tried to put together an RFC patch that turned off all
externally-visible IPv6 behavior that could break something, but I keep
running into distasteful choices.

The current default behavior must be preserved--if you drop this patch
into an existing distribution, the IPv6 stack must come up the way it
always has.  Thus the knob (let's call it ip6_disable) must default
false.

It seems desirable to make this a per-net-namespace knob.  But that
means each new net will be initialized before the distribution has a
chance to disable the IPv6 part.  This will lead to neighbor solicits
going out for link-local addresses, which some people can't accept.

The only alternative I can think of is to make it a module parameter,
which would leverage people's current habit to disable IPv6 via the
module configuration files, but doesn't fit well into a virtualized
world.

Exactly what to disable is unclear too.  Turning off neighbor and router
solicits, responding to same, etc.--almost certainly.  Refusing to
create AF_INET6 sockets--definitely.  Erroring on ioctl() and netlink
API calls related to IPv6--probably.  Hiding /proc entries for IPv6--I
don't know.  

Ideally, once ip6_disable was set true, every API, /proc and /sys
entries, etc. would look just like the ipv6 module was not loaded.  But
to do this, while still initializing most of the IPv6 code (so that
those hooks from other modules won't do unexpected things) is very
invasive.

So it seems to me that the only practical solution is to live with
blacklisting module ipv6 until the apps are fixed and sending IPv6
packets isn't considered a crime.

  --  John

Re: [Bonding-devel] 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Stephen Hemminger]

On Wed, 18 Feb 2009 21:21:07 +0000
John Dykstra <john.dykstra1@gmail.com> wrote:

> On Wed, 2009-02-18 at 14:57 -0500, Brian Haley wrote: 
> > Vlad Yasevich wrote:
> > > Having worked in other environments where ipv6 has to be explicitly
> > > enabled per interface, I've thought that this level of control was
> > > always missing from linux.  
> > 
> > There does seem to be a sysctl for it, just doesn't seem to work.
> 
> While this sysctl is definitely useful, I don't think it meets the needs
> of those users and distributions that are currently blacklisting the
> ipv6 module.  Even if you disable IPv6 on all interfaces, apps will
> still be able to create AF_INET6 sockets, and thus some apps will break
> or do inappropriate things because there isn't real IPv6 connectivity.
> 
> I've tried to put together an RFC patch that turned off all
> externally-visible IPv6 behavior that could break something, but I keep
> running into distasteful choices.
> 
> The current default behavior must be preserved--if you drop this patch
> into an existing distribution, the IPv6 stack must come up the way it
> always has.  Thus the knob (let's call it ip6_disable) must default
> false.
> 
> It seems desirable to make this a per-net-namespace knob.  But that
> means each new net will be initialized before the distribution has a
> chance to disable the IPv6 part.  This will lead to neighbor solicits
> going out for link-local addresses, which some people can't accept.
> 
> The only alternative I can think of is to make it a module parameter,
> which would leverage people's current habit to disable IPv6 via the
> module configuration files, but doesn't fit well into a virtualized
> world.
> 
> Exactly what to disable is unclear too.  Turning off neighbor and router
> solicits, responding to same, etc.--almost certainly.  Refusing to
> create AF_INET6 sockets--definitely.  Erroring on ioctl() and netlink
> API calls related to IPv6--probably.  Hiding /proc entries for IPv6--I
> don't know.  
> 
> Ideally, once ip6_disable was set true, every API, /proc and /sys
> entries, etc. would look just like the ipv6 module was not loaded.  But
> to do this, while still initializing most of the IPv6 code (so that
> those hooks from other modules won't do unexpected things) is very
> invasive.
> 
> So it seems to me that the only practical solution is to live with
> blacklisting module ipv6 until the apps are fixed and sending IPv6
> packets isn't considered a crime.
> 
>   --  John

There are also ipv6 purists who would like to see the same mechanism
exist to force ipv6 only (no ipv4). So any long term solution should
support both.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Herbert Xu]

Brian Haley <brian.haley@hp.com> wrote:
> 
> There does seem to be a sysctl for it, just doesn't seem to work. 

OK we should definitely fix that.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[David Miller]

From: Vlad Yasevich <vladislav.yasevich@hp.com>
Date: Wed, 18 Feb 2009 13:33:42 -0500

> Would a module parameter that disables ipv6 or at least addrconf be
> enough of a solution?

We have it, it's just that (as others have stated) it doesn't
prevent IPV6 sockets from being openned by applications.

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Vlad Yasevich]

David Miller wrote:
> From: Vlad Yasevich <vladislav.yasevich@hp.com>
> Date: Wed, 18 Feb 2009 13:33:42 -0500
> 
>> Would a module parameter that disables ipv6 or at least addrconf be
>> enough of a solution?
> 
> We have it, it's just that (as others have stated) it doesn't
> prevent IPV6 sockets from being openned by applications.
> 

Is that what people really want to block?

Or do they want to prevent the use of IPv6 in environments where
IPv6 is not supported?  If it's this case, then simply not configuring
any IPv6 addresses on the system interfaces will make it seem as if
IPv6 is not there.

Without IPv6 addresses, AF_INET6 sockets are the same as AF_INET.
I really see no reason to block them.  Any legacy apps that people
might worry about don't use this type socket any way.

One doesn't even need to worry about processing IPv6 traffic,
since the system would never join any IPv6 multicast groups and thus
would never see 99% of the traffic.

-vlad

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Herbert Xu]

Theodore Tso <tytso@mit.edu> wrote:
> 
> I think I can pretty much guarantee that distro users will be
> clamoring for a quick and easy way to block ipv6, and it's in our
> interest to document the recomended way to block it that doesn't cause
> weird problems with bonding, etc.

We already have a way:

echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Frank Blaschka]

We have the same issue with the qeth_l3 driver (it requires IPv6 symbols).
Distributors compile with IPv6 but some customes want to disable IPv6 without
building a custom kernel. If there would be a generic solution to address
this kind of runtime IPv6 dependencies this would be creat.  

Valdis.Kletnieks@vt.edu schrieb:
> On Tue, 17 Feb 2009 14:29:46 PST, David Miller said:
>> Don't configure ipv6 into your kernel, really.
>>
>> There is no other way to handle this.  If we want to support
>> IPV6 layer things in the bonding driver, it is going to
>> call helper functions in the ipv6 module and therefore must
>> be able to load it and use functions in it.
> 
> What does a poor corporate user do if they're running a distro kernel that
> was built with CONFIG_IPV6, but local security policy says "Disable IPv6
> because we don't do it yet, or because it breaks mission-critical software
> package XYZ?"  There's a *lot* of people who implement that by the "block
> the ipv6 module from loading" trick.  And building a kernel that doesn't
> include IPv6 may not be feasible due to vendor certification issues...
> 
> Heck, *I*'m almost in that boat - probably need to use bonded ethernet on some
> servers because we can't get 10GigE, but the software used in the project the
> servers were bought for blows chunks if it gets a whiff of an IPv6 address.
> Ended up spending 3 weeks doing a massive kludgery of one sort in DNS for the
> rest of the world, and equally massive lying in /etc/hosts for the hosts...
> (Don't ask - it was long and ugly, and just disabling the module would have
> saved me about 2.95 weeks of work, so I know where those people are coming
> from...)
> 

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Randy Dunlap]

Andrey Borzenkov wrote:
> Forward to bonding and netdev
> 
> On 17 of February 2009 11:52:32 J.A. Magallón wrote:
>> Hi all...
>>
>> Don't know if this is specific for -rc5, I have jumped from 28.4 to
>> 29-rc5. In this latest kernel, I can not install 'bonding' module if
>> 'ipv6' is disabled to load via modprobe.conf:
>>
>> install ipv6 /bin/true
>>
>> Trying bonding gives this dmesg:
>>
>> bonding: Unknown symbol ndisc_build_skb
>> bonding: Unknown symbol in6_dev_finish_destroy
>> bonding: Unknown symbol ndisc_send_skb
>> bonding: Unknown symbol unregister_inet6addr_notifier
>> bonding: Unknown symbol register_inet6addr_notifier
>>
>> Commenting the line in modprobe.conf makes things smooth again.
>> We can not disable ipv6 anymore ?
>>
> 
> If IPv6 is disable in kernel config bonding loads. But I think it is 
> regression, it should be possible to disable IPv6 if not required.

Just for clarification, is this a run-time (module load-time) error
but not a build error?

-- 
~Randy

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]

On 19 of February 2009 21:15:07 Randy Dunlap wrote:
> Andrey Borzenkov wrote:
> > Forward to bonding and netdev
> >
> > On 17 of February 2009 11:52:32 J.A. Magallón wrote:
> >> Hi all...
> >>
> >> Don't know if this is specific for -rc5, I have jumped from 28.4
> >> to 29-rc5. In this latest kernel, I can not install 'bonding'
> >> module if 'ipv6' is disabled to load via modprobe.conf:
> >>
> >> install ipv6 /bin/true
> >>
> >> Trying bonding gives this dmesg:
> >>
> >> bonding: Unknown symbol ndisc_build_skb
> >> bonding: Unknown symbol in6_dev_finish_destroy
> >> bonding: Unknown symbol ndisc_send_skb
> >> bonding: Unknown symbol unregister_inet6addr_notifier
> >> bonding: Unknown symbol register_inet6addr_notifier
> >>
> >> Commenting the line in modprobe.conf makes things smooth again.
> >> We can not disable ipv6 anymore ?
> >
> > If IPv6 is disable in kernel config bonding loads. But I think it
> > is regression, it should be possible to disable IPv6 if not
> > required.
>
> Just for clarification, is this a run-time (module load-time) error
> but not a build error?

Yes, this is run-time error. If IPV6 is disabled during kernel build, 
everything works; but the error was seen on distribution kernel which 
includes IPV6.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 2.6.29 regression? Bonding tied to IPV6 in 29-rc5

[Jay Vosburgh]

Randy Dunlap <randy.dunlap@oracle.com> wrote:

>Andrey Borzenkov wrote:
>> Forward to bonding and netdev
>> 
>> On 17 of February 2009 11:52:32 J.A. Magallón wrote:
>>> Hi all...
>>>
>>> Don't know if this is specific for -rc5, I have jumped from 28.4 to
>>> 29-rc5. In this latest kernel, I can not install 'bonding' module if
>>> 'ipv6' is disabled to load via modprobe.conf:
>>>
>>> install ipv6 /bin/true
>>>
>>> Trying bonding gives this dmesg:
>>>
>>> bonding: Unknown symbol ndisc_build_skb
>>> bonding: Unknown symbol in6_dev_finish_destroy
>>> bonding: Unknown symbol ndisc_send_skb
>>> bonding: Unknown symbol unregister_inet6addr_notifier
>>> bonding: Unknown symbol register_inet6addr_notifier
>>>
>>> Commenting the line in modprobe.conf makes things smooth again.
>>> We can not disable ipv6 anymore ?
>>>
>> 
>> If IPv6 is disable in kernel config bonding loads. But I think it is 
>> regression, it should be possible to disable IPv6 if not required.
>
>Just for clarification, is this a run-time (module load-time) error
>but not a build error?

	Yes, module load error, but not a build error.

	The build with CONFIG_IPV6 is fine, and bonding loads fine as
long as ipv6 is loaded.  The issue arises when the ipv6 module is
prevented from loading; in that case, bonding cannot load because it now
requires functionality from ipv6.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com