[PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

When roaming between different networks (e.g., changing wireless
SSIDs, or plugging in to different wired networks), IPv6 addresses and
routes are not cleared. If the two networks have different IPv6
subnets assigned, the host maintains both the old and new IPv6
addresses and gateways, but only the new ones works. If the host
chooses the wrong source address or gateway, or if the new network
does not have IPv6 but the old one did, IPv6 connections time out,
leading to long delays when trying to connect to IPv6 hosts.

Fix this by ensuring that autoconfigured IPv6 addresses and routes are
purged when link is lost, not only when the interface goes down.

Signed-off-by: Lorenzo Colitti <lorenzo@google.com>

--- a/net/ipv6/addrconf.c	2010-10-20 13:30:22.000000000 -0700
+++ b/net/ipv6/addrconf.c	2010-10-25 13:55:15.000000000 -0700
@@ -2524,6 +2524,14 @@
 		} else {
 			if (!addrconf_qdisc_ok(dev)) {
 				/* device is still not ready. */
+				if (idev && (idev->if_flags & IF_READY)) {
+					/* Link lost. Clear addresses and
+					   routes, the device might come back
+					   on a link where they are no longer
+					   valid. */
+					addrconf_ifdown(dev, 0);
+					idev->if_flags &= ~IF_READY;
+				}
 				break;
 			}

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Stephen Hemminger]

On Mon, 25 Oct 2010 19:08:27 -0700
Lorenzo Colitti <lorenzo@google.com> wrote:

> When roaming between different networks (e.g., changing wireless
> SSIDs, or plugging in to different wired networks), IPv6 addresses and
> routes are not cleared. If the two networks have different IPv6
> subnets assigned, the host maintains both the old and new IPv6
> addresses and gateways, but only the new ones works. If the host
> chooses the wrong source address or gateway, or if the new network
> does not have IPv6 but the old one did, IPv6 connections time out,
> leading to long delays when trying to connect to IPv6 hosts.
> 
> Fix this by ensuring that autoconfigured IPv6 addresses and routes are
> purged when link is lost, not only when the interface goes down.
> 
> Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
> 
> --- a/net/ipv6/addrconf.c	2010-10-20 13:30:22.000000000 -0700
> +++ b/net/ipv6/addrconf.c	2010-10-25 13:55:15.000000000 -0700
> @@ -2524,6 +2524,14 @@
>  		} else {
>  			if (!addrconf_qdisc_ok(dev)) {
>  				/* device is still not ready. */
> +				if (idev && (idev->if_flags & IF_READY)) {
> +					/* Link lost. Clear addresses and
> +					   routes, the device might come back
> +					   on a link where they are no longer
> +					   valid. */
> +					addrconf_ifdown(dev, 0);
> +					idev->if_flags &= ~IF_READY;
> +				}
>  				break;
>  			}

This is incorrect. When link is lost, routes and address should not be
flushed. They should be marked as tentative and then go through DAD again
on the new network.

If you do it this way, you break routing protocols when link is brought
down and back up.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Mon, Oct 25, 2010 at 9:38 PM, Stephen Hemminger
<shemminger@vyatta.com> wrote:
> This is incorrect. When link is lost, routes and address should not be
> flushed. They should be marked as tentative and then go through DAD again
> on the new network.

That won't help the case I am trying to fix, which is the case where
the new link has a global prefix different than the old link. Marking
the addresses as tentative will simply make them pass DAD and come
back as soon as link comes back. But since they don't match the prefix
that is assigned to the new link, they are unusable, because packets
can't be routed back to them.

> If you do it this way, you break routing protocols when link is brought
> down and back up.

The only addresses and routes flushed in this way should be ones that
aren't manually configured, i.e., the ones created by autoconf
(addrconf.c:2720 onwards). These won't be used by routing protocols,
except for link-local addresses. So I assume you're talking about
link-local here.

Link-local addresses are immediately recreated in a tentative state as
soon as link comes back, because on NETDEV_UP addrconf_notify calls
addrconf_dev_config. So, this patch only makes it so that they become
tentative when link goes away and comes back. In that time, the router
that temporarily loses link is unable to send packets for the brief
period of time that the link is performing DAD, but if the router has
lost link, it will also fail to send the packet while link is lost.
What's the additional failure scenario? Will it help if I make it so
that link-local addresses aren't touched at all?

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Stephen Hemminger]

On Mon, 25 Oct 2010 22:44:03 -0700
Lorenzo Colitti <lorenzo@google.com> wrote:

> On Mon, Oct 25, 2010 at 9:38 PM, Stephen Hemminger
> <shemminger@vyatta.com> wrote:
> > This is incorrect. When link is lost, routes and address should not be
> > flushed. They should be marked as tentative and then go through DAD again
> > on the new network.
> 
> That won't help the case I am trying to fix, which is the case where
> the new link has a global prefix different than the old link. Marking
> the addresses as tentative will simply make them pass DAD and come
> back as soon as link comes back. But since they don't match the prefix
> that is assigned to the new link, they are unusable, because packets
> can't be routed back to them.

For IPv4 this is already handled by network manager.
Why couldn't the same apply to IPv6?

> > If you do it this way, you break routing protocols when link is brought
> > down and back up.
> 
> The only addresses and routes flushed in this way should be ones that
> aren't manually configured, i.e., the ones created by autoconf
> (addrconf.c:2720 onwards). These won't be used by routing protocols,
> except for link-local addresses. So I assume you're talking about
> link-local here.

Not sure, let me do test it.

> Link-local addresses are immediately recreated in a tentative state as
> soon as link comes back, because on NETDEV_UP addrconf_notify calls
> addrconf_dev_config. So, this patch only makes it so that they become
> tentative when link goes away and comes back. In that time, the router
> that temporarily loses link is unable to send packets for the brief
> period of time that the link is performing DAD, but if the router has
> lost link, it will also fail to send the packet while link is lost.
> What's the additional failure scenario? Will it help if I make it so
> that link-local addresses aren't touched at all?

Link-local works fine.

-- 

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 8:28 AM, Stephen Hemminger
<shemminger@vyatta.com> wrote:
>> That won't help the case I am trying to fix, which is the case where
>> the new link has a global prefix different than the old link. Marking
>> the addresses as tentative will simply make them pass DAD and come
>> back as soon as link comes back. But since they don't match the prefix
>> that is assigned to the new link, they are unusable, because packets
>> can't be routed back to them.
>
> For IPv4 this is already handled by network manager.
> Why couldn't the same apply to IPv6?

I think addresses should be deleted by the entity that configured
them. In IPv4, the addresses are created by userspace daemons like
networkmanager or DHCP, and so they should be deleted by
networkmanager as well. In IPv6, autoconf addresses are created by the
kernel, and so should be deleted by the kernel.

Otherwise, userspace daemons that are not IPv6 aware must listen to
see what addresses are created, deleting them if they think they need
to go away, and possibly even racing with the kernel (e.g., imagine
networkmanager purging IPv6 addresses just before the kernel processes
an RA that adds new IPv6 addresses).

>> What's the additional failure scenario? Will it help if I make it so
>> that link-local addresses aren't touched at all?
>
> Link-local works fine.

Ok, so if global addresses work, then the patch doesn't need to do
anything special with link-local?

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Dan Williams]

On Tue, 2010-10-26 at 08:28 -0700, Stephen Hemminger wrote:
> On Mon, 25 Oct 2010 22:44:03 -0700
> Lorenzo Colitti <lorenzo@google.com> wrote:
> 
> > On Mon, Oct 25, 2010 at 9:38 PM, Stephen Hemminger
> > <shemminger@vyatta.com> wrote:
> > > This is incorrect. When link is lost, routes and address should not be
> > > flushed. They should be marked as tentative and then go through DAD again
> > > on the new network.
> > 
> > That won't help the case I am trying to fix, which is the case where
> > the new link has a global prefix different than the old link. Marking
> > the addresses as tentative will simply make them pass DAD and come
> > back as soon as link comes back. But since they don't match the prefix
> > that is assigned to the new link, they are unusable, because packets
> > can't be routed back to them.
> 
> For IPv4 this is already handled by network manager.
> Why couldn't the same apply to IPv6?

For this sort of thing, I tend to think that only higher-level network
management daemons like NM have the *possibility* to get this sort of
thing right, because only they have overall knowledge of the networking
situation, like whether you've actually connected to a different network
or not.  Specifically in the case of wifi, only the connection manager
(or wpa_supplicant) knows whether you're on a different network or  not,
anything else is a layering violation.

Remember that carrier loss does *not* indicate that you've switched
networks, both for wired and especially for wifi.  The only way you know
that you've really switched is when you've reconnected (or timed out the
reconnect) and inspected the new network situation.  It seems only
appropriate that the policy gets applied from higher levels based on a
more nuanced view of network state.

Dan

> 
> > > If you do it this way, you break routing protocols when link is brought
> > > down and back up.
> > 
> > The only addresses and routes flushed in this way should be ones that
> > aren't manually configured, i.e., the ones created by autoconf
> > (addrconf.c:2720 onwards). These won't be used by routing protocols,
> > except for link-local addresses. So I assume you're talking about
> > link-local here.
> 
> Not sure, let me do test it.
> 
> > Link-local addresses are immediately recreated in a tentative state as
> > soon as link comes back, because on NETDEV_UP addrconf_notify calls
> > addrconf_dev_config. So, this patch only makes it so that they become
> > tentative when link goes away and comes back. In that time, the router
> > that temporarily loses link is unable to send packets for the brief
> > period of time that the link is performing DAD, but if the router has
> > lost link, it will also fail to send the packet while link is lost.
> > What's the additional failure scenario? Will it help if I make it so
> > that link-local addresses aren't touched at all?
> 
> Link-local works fine.
> 

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Nov 2, 2010 at 10:54 AM, Dan Williams <dcbw@redhat.com> wrote:
> For this sort of thing, I tend to think that only higher-level network
> management daemons like NM have the *possibility* to get this sort of
> thing right, because only they have overall knowledge of the networking
> situation, like whether you've actually connected to a different network
> or not.  Specifically in the case of wifi, only the connection manager
> (or wpa_supplicant) knows whether you're on a different network or  not,
> anything else is a layering violation.

Maybe you're right and everything should be done in userspace. But for
now, the kernel is still processing RAs itself and adding IPv6
addresses to interfaces itself. If we don't intend to rip that all out
and replace it with a userspace implementation, then we should fix the
kernel to be a little smarter and not keep addresses around when they
are not useful any more.

This can be done relatively in a hitless fashion by putting old
addresses into the deprecated state until they are refreshed by an RA,
etc.

I don't think it's reasonable to say that we need a network management
daemon because we need to delete addresses that the kernel created and
the kernel can't be bothered to clean up after itself.

Cheers,
Lorenzo

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Brian Haley]

On 10/26/2010 01:44 AM, Lorenzo Colitti wrote:
> On Mon, Oct 25, 2010 at 9:38 PM, Stephen Hemminger
> <shemminger@vyatta.com> wrote:
>> This is incorrect. When link is lost, routes and address should not be
>> flushed. They should be marked as tentative and then go through DAD again
>> on the new network.
> 
> That won't help the case I am trying to fix, which is the case where
> the new link has a global prefix different than the old link. Marking
> the addresses as tentative will simply make them pass DAD and come
> back as soon as link comes back. But since they don't match the prefix
> that is assigned to the new link, they are unusable, because packets
> can't be routed back to them.

The old addresses will become deprecated, and eventually get removed, but
it will take 2 hours.

>> If you do it this way, you break routing protocols when link is brought
>> down and back up.
> 
> The only addresses and routes flushed in this way should be ones that
> aren't manually configured, i.e., the ones created by autoconf
> (addrconf.c:2720 onwards). These won't be used by routing protocols,
> except for link-local addresses. So I assume you're talking about
> link-local here.

I posted a very similar patch recently:

http://marc.info/?l=linux-netdev&m=128415231909522&w=2

But the first response pointed out that I didn't test this with just a
simple link flap, in which case all the IPv6 addresses are deleted,
and all sessions using them die.  Not good.  This changes the current
behavior, and isn't what happens with IPv4 either.

Having these addresses restart DAD is probably about as much as we
can do I think, unless we add a per-device sysctl to remove the addresses
(which I think has been shot-down before).  Is this a mobile device that
is actually changing it's point of attachment?

-Brian

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 9:58 AM, Brian Haley <brian.haley@hp.com> wrote:
> > That won't help the case I am trying to fix, which is the case where
> > the new link has a global prefix different than the old link. Marking
> > the addresses as tentative will simply make them pass DAD and come
> > back as soon as link comes back. But since they don't match the prefix
> > that is assigned to the new link, they are unusable, because packets
> > can't be routed back to them.
>
> The old addresses will become deprecated, and eventually get removed, but
> it will take 2 hours.

Yes, but they become deprecated only after the preferred lifetime is
expires. Until that happens, the kernel considers them fair game and
will use them for outgoing connections, without knowing that they
won't work. So the user just sees connection timeouts and thinks that
IPv6 is slow.

> http://marc.info/?l=linux-netdev&m=128415231909522&w=2
>
> But the first response pointed out that I didn't test this with just a
> simple link flap, in which case all the IPv6 addresses are deleted,
> and all sessions using them die.  Not good.  This changes the current
> behavior, and isn't what happens with IPv4 either.

Actually, I just tested this and it works fine. I opened a telnet
session to ipv6.google.com port 80, and while the TCP connection was
open I reassociated with the same wifi link. During the flap, the
patch removed and then readded the same global IPv6 address. While it
was doing it, I typed GET / HTTP/1.0\n\n in the telnet window. When
the address came back, the response came back fine. The connection was
not reset.

> Having these addresses restart DAD is probably about as much as we
> can do I think, unless we add a per-device sysctl to remove the addresses
> (which I think has been shot-down before).

As before, just setting them tentative won't help that case I am
trying to fix. They have to be removed.

> Is this a mobile device that is actually changing it's point of attachment?

This is a laptop that is changing SSIDs, plugging into wired/wireless,
etc. At work we have multiple wireless networks with their own IPv6
ranges. The typical case is that I am connected to the corp network,
then I associate with the guest network... and IPv6 is broken, because
the kernel wants to use my old address and gateway, which don't work.

I'm also doing lab testing of IPv6-capable home routers, each of which
has its own wireless network. As you can imagine, after a couple of
switches, I end up with three IPv6 addresses and between three sets of
default gateways, only one of which works.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Brian Haley <brian.haley@hp.com>
Date: Tue, 26 Oct 2010 12:58:54 -0400

> Having these addresses restart DAD is probably about as much as we
> can do I think, unless we add a per-device sysctl to remove the addresses
> (which I think has been shot-down before).

It would be fundamentally flawed, in my opinion, if the normal
mechanisms of ipv6 cannot handle this situation adequately.

That's really why I don't even want to consider the "blow away
the addresses" sysctl option.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 10:10 AM, David Miller <davem@davemloft.net> wrote:
> It would be fundamentally flawed, in my opinion, if the normal
> mechanisms of ipv6 cannot handle this situation adequately.

I think they can. When you associate with a new link, you naturally
set up link-local addresses, perform DAD for them, and send router
solicitations. You just need to make sure you don't have old addresses
from a previous link you were on before.

Removing autoconfigured addresses is what the kernel currently does
when the link goes down. The patch just makes sure it happens when
carrier is lost as well, since losing link might mean that when
carrier comes back, the host might be on a different link than it was
before.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 10:13:45 -0700

> Removing autoconfigured addresses is what the kernel currently does
> when the link goes down. The patch just makes sure it happens when
> carrier is lost as well, since losing link might mean that when
> carrier comes back, the host might be on a different link than it was
> before.

I just think validating that fact would be a better approach than
simply assuming link down/up means link change.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 10:21 AM, David Miller <davem@davemloft.net> wrote:
>> Removing autoconfigured addresses is what the kernel currently does
>> when the link goes down. The patch just makes sure it happens when
>> carrier is lost as well, since losing link might mean that when
>> carrier comes back, the host might be on a different link than it was
>> before.
>
> I just think validating that fact would be a better approach than
> simply assuming link down/up means link change.

So when the link comes back, we'd have to wait for all RAs to come in
(there could be multiple routers on a link, each announcing different
prefixes), and when this is all done, compare the list of addresses to
the prefixes that are currently on the link and delete the ones that
don't match. We'd have to do the same with default routers and /64
routes to all the link prefixes.

Also, we'd have to figure out how long to wait for RAs, and when to
time out if there are no RAs. Bear in mind that for as long as you
have an invalid IPv6 address, you can't reach dual-stack websites at
all (well, technically you can, but only if you're prepared to wait 3
minutes for the connection to time out). The damage starts when
associating to the link and only stops when the addresses and routes
are removed.

Is the extra complexity worth it? This patch is three lines, deals
with what I think is the common case, and has no obvious downsides
we've spotted so far. And at least in my testing, it's a definite
improvement. Would you be opposed to doing something like this as a
first cut and iterating or backing out if it turns out to be
insufficient or broken?

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 10:37:53 -0700

> So when the link comes back, we'd have to wait for all RAs to come in
> (there could be multiple routers on a link, each announcing different
> prefixes), and when this is all done, compare the list of addresses to
> the prefixes that are currently on the link and delete the ones that
> don't match. We'd have to do the same with default routers and /64
> routes to all the link prefixes.

Why would you do this when there are link layer indications that
we are attached to a different network?

The only thing missing is a new device notification type, some code to
generate it in the wireless subsystem, and an ipv6 handler to zap the
addresses when received.

Please think out of the box for a second, thanks :-)

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 10:47 AM, David Miller <davem@davemloft.net> wrote:
> Why would you do this when there are link layer indications that
> we are attached to a different network?

Are there? For wifi you could look at the SSID (though even that is no
guarantee - for example enterprise networks commonly use just a few
SSIDs, but have different IP address ranges on a per-controller or
per-radio basis). But what about wired? Remember, the same thing
happens when you unplug your laptop's ethernet port and plug it in to
another network.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 10:50:20 -0700

> On Tue, Oct 26, 2010 at 10:47 AM, David Miller <davem@davemloft.net> wrote:
>> Why would you do this when there are link layer indications that
>> we are attached to a different network?
> 
> Are there? For wifi you could look at the SSID (though even that is no

I'm saying to check the AP's MAC.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 10:55 AM, David Miller <davem@davemloft.net> wrote:
>> Are there? For wifi you could look at the SSID (though even that is no
>
> I'm saying to check the AP's MAC.

Ah yes, that works better. But for wired? Or do you think it should be
fixed only for wifi?

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 11:02:45 -0700

> On Tue, Oct 26, 2010 at 10:55 AM, David Miller <davem@davemloft.net> wrote:
>>> Are there? For wifi you could look at the SSID (though even that is no
>>
>> I'm saying to check the AP's MAC.
> 
> Ah yes, that works better. But for wired? Or do you think it should be
> fixed only for wifi?

For now the case where it is most important to do something, and
the situation is common, seems to be wireless.

BTW, we already have a NETDEV_NOTIFY_PEERS notifier for faster
failover.  It's used in ipv4 to generate gratuitous ARPs.

Maybe it could be used for your case too somehow.  Just an idea.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Dan Williams]

On Tue, 2010-10-26 at 11:02 -0700, Lorenzo Colitti wrote:
> On Tue, Oct 26, 2010 at 10:55 AM, David Miller <davem@davemloft.net> wrote:
> >> Are there? For wifi you could look at the SSID (though even that is no
> >
> > I'm saying to check the AP's MAC.
> 
> Ah yes, that works better. But for wired? Or do you think it should be
> fixed only for wifi?

Only looking at the SSID tells you if you're on a different network.
Just looking at the AP MAC is like plugging into a different port on the
same switch: it's still the same network.

Dan

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 10:47 AM, David Miller <davem@davemloft.net> wrote:
> Why would you do this when there are link layer indications that
> we are attached to a different network?

Thinking about this some more, I don't think it's necessarily better.

The only thing that really knows the state of the network is the
network itself. IPv6 provides a mechanism that hosts can use to find
out what they should be doing: send an RA and process the resulting
RS. I think this is much cleaner than keeping local state, performing
layering violations (looking at MAC addresses) and getting information
from other systems in order to second-guess information that the host
can get simply by sending another RS.

If you think about it, this is what we do today in IPv4: when link is
lost, networkmanager *deletes IP addresses and routes* from the
interfaces, and then starts DHCP again when link comes back. That is,
asks the network for configuration information.

This works pretty well. Why not do the same for IPv6? The only
difference in IPv6 is that the kernel is the one doing
autoconfiguration, so it's the kernel that should be doing the asking.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 11:17:27 -0700

> This works pretty well. Why not do the same for IPv6? The only
> difference in IPv6 is that the kernel is the one doing
> autoconfiguration, so it's the kernel that should be doing the asking.

That should be fine, but if I have static IPv6 addresses assigned to
an interface I really don't want them changing on a simple link flap.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Tue, Oct 26, 2010 at 11:23 AM, David Miller <davem@davemloft.net> wrote:
>> This works pretty well. Why not do the same for IPv6? The only
>> difference in IPv6 is that the kernel is the one doing
>> autoconfiguration, so it's the kernel that should be doing the asking.
>
> That should be fine, but if I have static IPv6 addresses assigned to
> an interface I really don't want them changing on a simple link flap.

As the patch stands, they don't. Only autoconfigured addresses will be
cleared, because addrconf_ifdown() does not remove any addresses that
are permanent (unless they are link-local, in which case they are
recreated as soon as link comes back).

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Tue, 26 Oct 2010 15:53:50 -0700

> As the patch stands, they don't. Only autoconfigured addresses will be
> cleared, because addrconf_ifdown() does not remove any addresses that
> are permanent (unless they are link-local, in which case they are
> recreated as soon as link comes back).

Ok, and that brings us back to the issue of losing a TCP connection
over a link-local et al. address during a minor link flap.

I think some financial services people will really dislike that
behavior :)

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Wed, Oct 27, 2010 at 8:51 AM, David Miller <davem@davemloft.net> wrote:
>> As the patch stands, they don't. Only autoconfigured addresses will be
>> cleared, because addrconf_ifdown() does not remove any addresses that
>> are permanent (unless they are link-local, in which case they are
>> recreated as soon as link comes back).
>
> Ok, and that brings us back to the issue of losing a TCP connection
> over a link-local et al. address during a minor link flap.

TCP connections don't go away when the address they are using goes
away. They go away if their address goes away forever. As mentioned
above in the thread, I did test this :)

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[David Miller]

From: Lorenzo Colitti <lorenzo@google.com>
Date: Wed, 27 Oct 2010 09:01:25 -0700

> On Wed, Oct 27, 2010 at 8:51 AM, David Miller <davem@davemloft.net> wrote:
>>> As the patch stands, they don't. Only autoconfigured addresses will be
>>> cleared, because addrconf_ifdown() does not remove any addresses that
>>> are permanent (unless they are link-local, in which case they are
>>> recreated as soon as link comes back).
>>
>> Ok, and that brings us back to the issue of losing a TCP connection
>> over a link-local et al. address during a minor link flap.
> 
> TCP connections don't go away when the address they are using goes
> away. They go away if their address goes away forever. As mentioned
> above in the thread, I did test this :)

Ok, that's good enough for me :)

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Brian Haley]

Hi Lorenzo,

On 10/25/2010 10:08 PM, Lorenzo Colitti wrote:
> When roaming between different networks (e.g., changing wireless
> SSIDs, or plugging in to different wired networks), IPv6 addresses and
> routes are not cleared. If the two networks have different IPv6
> subnets assigned, the host maintains both the old and new IPv6
> addresses and gateways, but only the new ones works. If the host
> chooses the wrong source address or gateway, or if the new network
> does not have IPv6 but the old one did, IPv6 connections time out,
> leading to long delays when trying to connect to IPv6 hosts.
> 
> Fix this by ensuring that autoconfigured IPv6 addresses and routes are
> purged when link is lost, not only when the interface goes down.
> 
> Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
> 
> --- a/net/ipv6/addrconf.c	2010-10-20 13:30:22.000000000 -0700
> +++ b/net/ipv6/addrconf.c	2010-10-25 13:55:15.000000000 -0700
> @@ -2524,6 +2524,14 @@
>  		} else {
>  			if (!addrconf_qdisc_ok(dev)) {
>  				/* device is still not ready. */
> +				if (idev && (idev->if_flags & IF_READY)) {
> +					/* Link lost. Clear addresses and
> +					   routes, the device might come back
> +					   on a link where they are no longer
> +					   valid. */
> +					addrconf_ifdown(dev, 0);
> +					idev->if_flags &= ~IF_READY;
> +				}

Just taking another look at this, you don't need that ~IF_READY line,
addrconf_ifdown() is already doing that when how==0.

Could you give my previous patch a try?  I believe it will work the same
way as yours, but also fixes a case where DAD is started twice for some
addresses.

Thanks,

-Brian

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Maciej Żenczykowski]

So how does all this work with privacy addresses, established
connections, and a link flap?

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Wed, Oct 27, 2010 at 1:35 AM, Maciej Żenczykowski
<zenczykowski@gmail.com> wrote:
> So how does all this work with privacy addresses, established
> connections, and a link flap?

The current privacy address comes back because it's a time-based hash.
I think the old ones are gone. Still, I think it's better that
connections from 1 day ago don't work any more (the default for
privacy addresses is 1 day), than if all new and all old connections
don't work any more.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Maciej Żenczykowski]

> The current privacy address comes back because it's a time-based hash.

I would guess it uses jiffies (or some other equally fine-grained
time, or just pulls from a random entropy source),

I see no reason why the second time around it would generate the same
privacy address.
In other words: are you absolutely certain of this?  It's certainly
not how I would expect it to function.

v2.6.36/net/ipv6/addrconf.c
#ifdef CONFIG_IPV6_PRIVACY
#include <linux/random.h>
#endif

and

http://lxr.linux.no/linux+v2.6.36/net/ipv6/addrconf.c#L1608

static int __ipv6_regen_rndid(struct inet6_dev *idev) {
        get_random_bytes(idev->rndid, sizeof(idev->rndid));
        idev->rndid[0] &= ~0x02;

certainly seem to point towards it being totally random.

> I think the old ones are gone. Still, I think it's better that
> connections from 1 day ago don't work any more (the default for
> privacy addresses is 1 day), than if all new and all old connections
> don't work any more.

No, the default is _no more_ than 1 day preferred and 7 days valid.
But those are the upper maximums, the actual values are the
preferred/validity values taken from the RA (limited to whatever the
max limit sysctls are set to).

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Wed, Oct 27, 2010 at 1:39 PM, Maciej Żenczykowski
<zenczykowski@gmail.com> wrote:
> static int __ipv6_regen_rndid(struct inet6_dev *idev) {
>        get_random_bytes(idev->rndid, sizeof(idev->rndid));
>        idev->rndid[0] &= ~0x02;
>
> certainly seem to point towards it being totally random.

It's random, but the random interface ID is a property of the
interface, not of the address, and thus survives interface up/down:

# ip -6 a li dev wlan0 secondary
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2620:0:1000:fd01:952f:2586:ed6f:408c/64 scope global
secondary dynamic
       valid_lft 604363sec preferred_lft 85363sec
# ip link set wlan0 down
# ip -6 a li dev wlan0 secondary
# ip link set wlan0 up
# ip -6 a li dev wlan0 secondary
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2620:0:1000:167d:952f:2586:ed6f:408c/64 scope global
secondary dynamic
       valid_lft 604799sec preferred_lft 85799sec

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Lorenzo Colitti]

On Thu, Oct 28, 2010 at 3:23 PM, Lorenzo Colitti <lorenzo@google.com> wrote:
> It's random, but the random interface ID is a property of the
> interface, not of the address, and thus survives interface up/down:

Also note that it's fairly trivial, instead of deleting all privacy
addresses when link is lost, to simply deprecate them so they
automatically come back after link comes back if an RA refreshes their
lifetime.

I'd rather not do this, because it introduces a brief window on link
up where the system has broken IPv6 connectivity and all connection
attempts hang for three minutes. But if this is the only alternative
to the current behaviour (where all connection attempts hang forever
until all old addresses have exhausted their lifetimes), then we can
do it.

Re: [PATCH] ipv6: addrconf: clear IPv6 addresses and routes when losing link

[Maciej Żenczykowski]

> It's random, but the random interface ID is a property of the
> interface, not of the address, and thus survives interface up/down:

Interesting, that actually means a link down/up will break the
temporary address lifetimes (reset them).

# uname -a
Linux nike 2.6.34.7-61.fc13.x86_64 #1 SMP Tue Oct 19 04:06:30 UTC 2010
x86_64 x86_64 x86_64 GNU/Linux

# echo --Dump--; ip -6 addr list dev eth0 secondary; echo --Down--; ip
link set eth0 down; echo --Dump--; ip -6 addr list dev eth0 secondary;
echo --Up--; ip link set eth0 up; echo --Dump--; ip -6 addr list dev
eth0 secondary; echo --Sleep30--; sleep 30; echo --Dump--; ip -6 addr
list dev eth0 secondary; echo --Sleep60--; sleep 60; echo --Dump--; ip
-6 addr list dev eth0 secondary; echo --Done--

--Dump--
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2620:0:1000:1301:ab:5938:64ce:ae20/64 scope global temporary dynamic
       valid_lft 604667sec preferred_lft 85667sec
--Down--
--Dump--
--Up--
--Dump--
--Sleep30--
--Dump--
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2620:0:1000:1301:ab:5938:64ce:ae20/64 scope global temporary dynamic
       valid_lft 604775sec preferred_lft 85775sec
--Sleep60--
--Dump--
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2620:0:1000:1301:ab:5938:64ce:ae20/64 scope global temporary dynamic
       valid_lft 604715sec preferred_lft 85715sec
--Done--

Notice that when the v6 temp address comes back (presumably after
receiving an RA) it comes back with fresh valid/preferred lifetimes,
instead of the already partially expired ones that we had before the
link went down.

I was already aware that v6 temp addresses didn't work all that well
with short RA timeouts, but it looks like the logic needs a bit more
tweaking...