why are IPv6 addresses removed on link down

why are IPv6 addresses removed on link down

[David Ahern]

We noticed that IPv6 addresses are removed on a link down. e.g.,
   ip link set dev eth1


Looking at the code it appears to be this code path in addrconf.c:

         case NETDEV_DOWN:
         case NETDEV_UNREGISTER:
                 /*
                  *      Remove all addresses from this interface.
                  */
                 addrconf_ifdown(dev, event != NETDEV_DOWN);
                 break;

IPv4 addresses are NOT removed on a link down. Is there a particular 
reason IPv6 addresses are?

Thanks,
David

Re: why are IPv6 addresses removed on link down

[Stephen Hemminger]

On Mon, 12 Jan 2015 22:06:44 -0700
David Ahern <dsahern@gmail.com> wrote:

> We noticed that IPv6 addresses are removed on a link down. e.g.,
>    ip link set dev eth1
> 
> 
> Looking at the code it appears to be this code path in addrconf.c:
> 
>          case NETDEV_DOWN:
>          case NETDEV_UNREGISTER:
>                  /*
>                   *      Remove all addresses from this interface.
>                   */
>                  addrconf_ifdown(dev, event != NETDEV_DOWN);
>                  break;
> 
> IPv4 addresses are NOT removed on a link down. Is there a particular 
> reason IPv6 addresses are?
> 
> Thanks,
> David

See RFC's which describes how IPv6 does Duplicate Address Detection.
Address is not valid when link is down, since DAD is not possible.

Re: why are IPv6 addresses removed on link down

[Hannes Frederic Sowa]

On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
> On Mon, 12 Jan 2015 22:06:44 -0700
> David Ahern <dsahern@gmail.com> wrote:
> 
> > We noticed that IPv6 addresses are removed on a link down. e.g.,
> >    ip link set dev eth1
> > 
> > 
> > Looking at the code it appears to be this code path in addrconf.c:
> > 
> >          case NETDEV_DOWN:
> >          case NETDEV_UNREGISTER:
> >                  /*
> >                   *      Remove all addresses from this interface.
> >                   */
> >                  addrconf_ifdown(dev, event != NETDEV_DOWN);
> >                  break;
> > 
> > IPv4 addresses are NOT removed on a link down. Is there a particular 
> > reason IPv6 addresses are?
> > 
> > Thanks,
> > David
> 
> See RFC's which describes how IPv6 does Duplicate Address Detection.
> Address is not valid when link is down, since DAD is not possible.

It should be no problem if the kernel would reacquire them on ifup and
do proper DAD. We simply must not use them while the interface is dead
(also making sure they don't get used for loopback routing).

The problem the IPv6 addresses get removed is much more a historical
artifact nowadays, I think. It is part of user space API and scripts
deal with that already.

Bye,
Hannes

Re: why are IPv6 addresses removed on link down

[YOSHIFUJI Hideaki]

Hi,

Hannes Frederic Sowa wrote:
> On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
>> On Mon, 12 Jan 2015 22:06:44 -0700
>> David Ahern <dsahern@gmail.com> wrote:
>>
>>> We noticed that IPv6 addresses are removed on a link down. e.g.,
>>>     ip link set dev eth1
>>>
>>>
>>> Looking at the code it appears to be this code path in addrconf.c:
>>>
>>>           case NETDEV_DOWN:
>>>           case NETDEV_UNREGISTER:
>>>                   /*
>>>                    *      Remove all addresses from this interface.
>>>                    */
>>>                   addrconf_ifdown(dev, event != NETDEV_DOWN);
>>>                   break;
>>>
>>> IPv4 addresses are NOT removed on a link down. Is there a particular
>>> reason IPv6 addresses are?
>>>
>>> Thanks,
>>> David
>>
>> See RFC's which describes how IPv6 does Duplicate Address Detection.
>> Address is not valid when link is down, since DAD is not possible.
>
> It should be no problem if the kernel would reacquire them on ifup and
> do proper DAD. We simply must not use them while the interface is dead
> (also making sure they don't get used for loopback routing).
>
> The problem the IPv6 addresses get removed is much more a historical
> artifact nowadays, I think. It is part of user space API and scripts
> deal with that already.

We might have another "detached" state which essintially drops
outgoing packets while link is down.  Just after recovering link,
we could start receiving packet from the link and perform optimistic
DAD. And then, after it succeeds, we may start sending packets.

Since "detached" state is like the state just before completing
Optimistic DAD, it is not so difficult to implement this extended
behavior, I guess.

-- 
Hideaki Yoshifuji <hideaki.yoshifuji@miraclelinux.com>
Technical Division, MIRACLE LINUX CORPORATION

Re: why are IPv6 addresses removed on link down

[YOSHIFUJI Hideaki]

YOSHIFUJI Hideaki wrote:
> Hi,
>
> Hannes Frederic Sowa wrote:
>> On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
>>> On Mon, 12 Jan 2015 22:06:44 -0700
>>> David Ahern <dsahern@gmail.com> wrote:
>>>
>>>> We noticed that IPv6 addresses are removed on a link down. e.g.,
>>>>     ip link set dev eth1
>>>>
>>>>
>>>> Looking at the code it appears to be this code path in addrconf.c:
>>>>
>>>>           case NETDEV_DOWN:
>>>>           case NETDEV_UNREGISTER:
>>>>                   /*
>>>>                    *      Remove all addresses from this interface.
>>>>                    */
>>>>                   addrconf_ifdown(dev, event != NETDEV_DOWN);
>>>>                   break;
>>>>
>>>> IPv4 addresses are NOT removed on a link down. Is there a particular
>>>> reason IPv6 addresses are?
>>>>
>>>> Thanks,
>>>> David
>>>
>>> See RFC's which describes how IPv6 does Duplicate Address Detection.
>>> Address is not valid when link is down, since DAD is not possible.
>>
>> It should be no problem if the kernel would reacquire them on ifup and
>> do proper DAD. We simply must not use them while the interface is dead
>> (also making sure they don't get used for loopback routing).
>>
>> The problem the IPv6 addresses get removed is much more a historical
>> artifact nowadays, I think. It is part of user space API and scripts
>> deal with that already.
>
> We might have another "detached" state which essintially drops
> outgoing packets while link is down.  Just after recovering link,
> we could start receiving packet from the link and perform optimistic
> DAD. And then, after it succeeds, we may start sending packets.
>
> Since "detached" state is like the state just before completing
> Optimistic DAD, it is not so difficult to implement this extended
> behavior, I guess.
>

Note that node is allowed to send packets to neighbours or default
routers if the node knows their link-layer addresses during Optimistic
DAD.

-- 
Hideaki Yoshifuji <hideaki.yoshifuji@miraclelinux.com>
Technical Division, MIRACLE LINUX CORPORATION

Re: why are IPv6 addresses removed on link down

[Hannes Frederic Sowa]

Hi,

On Di, 2015-01-13 at 21:15 +0900, YOSHIFUJI Hideaki wrote:
> YOSHIFUJI Hideaki wrote:
> > Hi,
> >
> > Hannes Frederic Sowa wrote:
> >> On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
> >>> On Mon, 12 Jan 2015 22:06:44 -0700
> >>> David Ahern <dsahern@gmail.com> wrote:
> >>>
> >>>> We noticed that IPv6 addresses are removed on a link down. e.g.,
> >>>>     ip link set dev eth1
> >>>>
> >>>>
> >>>> Looking at the code it appears to be this code path in addrconf.c:
> >>>>
> >>>>           case NETDEV_DOWN:
> >>>>           case NETDEV_UNREGISTER:
> >>>>                   /*
> >>>>                    *      Remove all addresses from this interface.
> >>>>                    */
> >>>>                   addrconf_ifdown(dev, event != NETDEV_DOWN);
> >>>>                   break;
> >>>>
> >>>> IPv4 addresses are NOT removed on a link down. Is there a particular
> >>>> reason IPv6 addresses are?
> >>>>
> >>>> Thanks,
> >>>> David
> >>>
> >>> See RFC's which describes how IPv6 does Duplicate Address Detection.
> >>> Address is not valid when link is down, since DAD is not possible.
> >>
> >> It should be no problem if the kernel would reacquire them on ifup and
> >> do proper DAD. We simply must not use them while the interface is dead
> >> (also making sure they don't get used for loopback routing).
> >>
> >> The problem the IPv6 addresses get removed is much more a historical
> >> artifact nowadays, I think. It is part of user space API and scripts
> >> deal with that already.
> >
> > We might have another "detached" state which essintially drops
> > outgoing packets while link is down.  Just after recovering link,
> > we could start receiving packet from the link and perform optimistic
> > DAD. And then, after it succeeds, we may start sending packets.
> >
> > Since "detached" state is like the state just before completing
> > Optimistic DAD, it is not so difficult to implement this extended
> > behavior, I guess.
> >
> 
> Note that node is allowed to send packets to neighbours or default
> routers if the node knows their link-layer addresses during Optimistic
> DAD.
> 

I don't think it should be a problem from internal state handling of the
addresses.

I am much more concerned with scripts expecting the addresses to be
flushed on interface down/up and not reacting appropriate.

Bye,
Hannes

Re: why are IPv6 addresses removed on link down

[David Ahern]

On 1/13/15 5:36 AM, Hannes Frederic Sowa wrote:
> Hi,
>
> On Di, 2015-01-13 at 21:15 +0900, YOSHIFUJI Hideaki wrote:
>> YOSHIFUJI Hideaki wrote:
>>> Hi,
>>>
>>> Hannes Frederic Sowa wrote:
>>>> On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
>>>>> On Mon, 12 Jan 2015 22:06:44 -0700
>>>>> David Ahern <dsahern@gmail.com> wrote:
>>>>>
>>>>>> We noticed that IPv6 addresses are removed on a link down. e.g.,
>>>>>>      ip link set dev eth1
>>>>>>
>>>>>>
>>>>>> Looking at the code it appears to be this code path in addrconf.c:
>>>>>>
>>>>>>            case NETDEV_DOWN:
>>>>>>            case NETDEV_UNREGISTER:
>>>>>>                    /*
>>>>>>                     *      Remove all addresses from this interface.
>>>>>>                     */
>>>>>>                    addrconf_ifdown(dev, event != NETDEV_DOWN);
>>>>>>                    break;
>>>>>>
>>>>>> IPv4 addresses are NOT removed on a link down. Is there a particular
>>>>>> reason IPv6 addresses are?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>
>>>>> See RFC's which describes how IPv6 does Duplicate Address Detection.
>>>>> Address is not valid when link is down, since DAD is not possible.
>>>>
>>>> It should be no problem if the kernel would reacquire them on ifup and
>>>> do proper DAD. We simply must not use them while the interface is dead
>>>> (also making sure they don't get used for loopback routing).
>>>>
>>>> The problem the IPv6 addresses get removed is much more a historical
>>>> artifact nowadays, I think. It is part of user space API and scripts
>>>> deal with that already.
>>>
>>> We might have another "detached" state which essintially drops
>>> outgoing packets while link is down.  Just after recovering link,
>>> we could start receiving packet from the link and perform optimistic
>>> DAD. And then, after it succeeds, we may start sending packets.
>>>
>>> Since "detached" state is like the state just before completing
>>> Optimistic DAD, it is not so difficult to implement this extended
>>> behavior, I guess.
>>>
>>
>> Note that node is allowed to send packets to neighbours or default
>> routers if the node knows their link-layer addresses during Optimistic
>> DAD.
>>
>
> I don't think it should be a problem from internal state handling of the
> addresses.
>
> I am much more concerned with scripts expecting the addresses to be
> flushed on interface down/up and not reacting appropriate.

The current code seems inconsistent: I can put an IPv6 address on a link 
in the down state. On a link up the address is retained. Only on a 
subsequent link down is it removed. If DAD or anything else is the 
reason for the current logic then why allow an address to be assigned in 
the down state? Similarly that it currently seems to work ok then it 
suggests the right thing is done on a link up in which case a flush is 
not needed.

Bottom line is there a harm in removing the flush? If there is no harm 
will mainline kernel take a patch to do that or is your backward 
compatibility concern enough to block it?

David

Re: why are IPv6 addresses removed on link down

[Hannes Frederic Sowa]

On Di, 2015-01-13 at 07:53 -0700, David Ahern wrote:
> On 1/13/15 5:36 AM, Hannes Frederic Sowa wrote:
> > Hi,
> >
> > On Di, 2015-01-13 at 21:15 +0900, YOSHIFUJI Hideaki wrote:
> >> YOSHIFUJI Hideaki wrote:
> >>> Hi,
> >>>
> >>> Hannes Frederic Sowa wrote:
> >>>> On Mo, 2015-01-12 at 23:10 -0800, Stephen Hemminger wrote:
> >>>>> On Mon, 12 Jan 2015 22:06:44 -0700
> >>>>> David Ahern <dsahern@gmail.com> wrote:
> >>>>>
> >>>>>> We noticed that IPv6 addresses are removed on a link down. e.g.,
> >>>>>>      ip link set dev eth1
> >>>>>>
> >>>>>>
> >>>>>> Looking at the code it appears to be this code path in addrconf.c:
> >>>>>>
> >>>>>>            case NETDEV_DOWN:
> >>>>>>            case NETDEV_UNREGISTER:
> >>>>>>                    /*
> >>>>>>                     *      Remove all addresses from this interface.
> >>>>>>                     */
> >>>>>>                    addrconf_ifdown(dev, event != NETDEV_DOWN);
> >>>>>>                    break;
> >>>>>>
> >>>>>> IPv4 addresses are NOT removed on a link down. Is there a particular
> >>>>>> reason IPv6 addresses are?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> David
> >>>>>
> >>>>> See RFC's which describes how IPv6 does Duplicate Address Detection.
> >>>>> Address is not valid when link is down, since DAD is not possible.
> >>>>
> >>>> It should be no problem if the kernel would reacquire them on ifup and
> >>>> do proper DAD. We simply must not use them while the interface is dead
> >>>> (also making sure they don't get used for loopback routing).
> >>>>
> >>>> The problem the IPv6 addresses get removed is much more a historical
> >>>> artifact nowadays, I think. It is part of user space API and scripts
> >>>> deal with that already.
> >>>
> >>> We might have another "detached" state which essintially drops
> >>> outgoing packets while link is down.  Just after recovering link,
> >>> we could start receiving packet from the link and perform optimistic
> >>> DAD. And then, after it succeeds, we may start sending packets.
> >>>
> >>> Since "detached" state is like the state just before completing
> >>> Optimistic DAD, it is not so difficult to implement this extended
> >>> behavior, I guess.
> >>>
> >>
> >> Note that node is allowed to send packets to neighbours or default
> >> routers if the node knows their link-layer addresses during Optimistic
> >> DAD.
> >>
> >
> > I don't think it should be a problem from internal state handling of the
> > addresses.
> >
> > I am much more concerned with scripts expecting the addresses to be
> > flushed on interface down/up and not reacting appropriate.
> 
> The current code seems inconsistent: I can put an IPv6 address on a link 
> in the down state. On a link up the address is retained. Only on a 
> subsequent link down is it removed. If DAD or anything else is the 
> reason for the current logic then why allow an address to be assigned in 
> the down state? Similarly that it currently seems to work ok then it 
> suggests the right thing is done on a link up in which case a flush is 
> not needed.
> 
> Bottom line is there a harm in removing the flush? If there is no harm 
> will mainline kernel take a patch to do that or is your backward 
> compatibility concern enough to block it?

This was already discussed several times here, e.g. one patch I just 
found:

http://lists.openwall.net/netdev/2011/01/24/8
and
http://patchwork.ozlabs.org/patch/17558/

Albeit I hate sysctls for things like this, it might I tend to find it
acceptable because it solves a problem which happened to lots of people.
And I don't like the current behavior neither.

I think this can work, but we should follow up all the old discussions
to not introduce any kind of new undesired behavior this time.

Thanks,
Hannes

Re: why are IPv6 addresses removed on link down

[Nicolas Dichtel]

Le 13/01/2015 16:00, Hannes Frederic Sowa a écrit :
[snip]
>
> This was already discussed several times here, e.g. one patch I just
> found:
>
> http://lists.openwall.net/netdev/2011/01/24/8
> and
> http://patchwork.ozlabs.org/patch/17558/
>
> Albeit I hate sysctls for things like this, it might I tend to find it
> acceptable because it solves a problem which happened to lots of people.
> And I don't like the current behavior neither.
>
> I think this can work, but we should follow up all the old discussions
> to not introduce any kind of new undesired behavior this time.
If I remember well, one of the problem was that it was not possible anymore to
unload the IPv6 module after the patches. But since commit 8ce440610357 ("ipv6:
do not allow ipv6 module to be removed"), it's not possible anymore.

Re: why are IPv6 addresses removed on link down

[Sowmini Varadhan]

On (01/13/15 07:53), David Ahern wrote:
> 
> The current code seems inconsistent: I can put an IPv6 address on a
> link in the down state. On a link up the address is retained. Only
> on a subsequent link down is it removed. If DAD or anything else is
> the reason for the current logic then why allow an address to be
> assigned in the down state? Similarly that it currently seems to
> work ok then it suggests the right thing is done on a link up in
> which case a flush is not needed.
> 
> Bottom line is there a harm in removing the flush? If there is no
> harm will mainline kernel take a patch to do that or is your
> backward compatibility concern enough to block it?

Does some of this have to do with the manner in which this interacts
with SLAAC? I recall that there were two schools of thought for doing
DAD when SLAAC is present: one says it is sufficient to just do DAD
on the interface-id, the other requies DAD on the whole 128-bit IPv6
address. I'm not sure which choice linux makes.

--Sowmini

Re: why are IPv6 addresses removed on link down

[Hannes Frederic Sowa]

On Di, 2015-01-13 at 10:00 -0500, Sowmini Varadhan wrote:
> On (01/13/15 07:53), David Ahern wrote:
> > 
> > The current code seems inconsistent: I can put an IPv6 address on a
> > link in the down state. On a link up the address is retained. Only
> > on a subsequent link down is it removed. If DAD or anything else is
> > the reason for the current logic then why allow an address to be
> > assigned in the down state? Similarly that it currently seems to
> > work ok then it suggests the right thing is done on a link up in
> > which case a flush is not needed.
> > 
> > Bottom line is there a harm in removing the flush? If there is no
> > harm will mainline kernel take a patch to do that or is your
> > backward compatibility concern enough to block it?
> 
> Does some of this have to do with the manner in which this interacts
> with SLAAC? I recall that there were two schools of thought for doing
> DAD when SLAAC is present: one says it is sufficient to just do DAD
> on the interface-id, the other requies DAD on the whole 128-bit IPv6
> address. I'm not sure which choice linux makes.

Yes, it does have something to do with it. But I didn't understand what
you meant by doing DAD on the interface-id.

If you look at the patches I just posted, only addresses which are in
link-local and not in permanent state will be flushed.

I also need to do research on how to safely approach this, I don't know,
yet.

Bye,
Hannes

Re: why are IPv6 addresses removed on link down

[Sowmini Varadhan]

On (01/13/15 16:09), Hannes Frederic Sowa wrote:
> 
> Yes, it does have something to do with it. But I didn't understand what
> you meant by doing DAD on the interface-id.

I have to dig up the RFCs for this, but I recall that, at one point,
the specs assert that it is sufficient to verify that the interface-id
(I think via DAD for the link-local address) is unique, and use
that to infer uniqueness of all the other non-link-local addresses
as well.

I think later specs may have changed that, asserting that the
correct, safe, proper thing to do is to separately DAD each address
by itself.

> If you look at the patches I just posted, only addresses which are in
> link-local and not in permanent state will be flushed.
> 
> I also need to do research on how to safely approach this, I don't know,
> yet.
> 
> Bye,
> Hannes
> 
> 

Re: why are IPv6 addresses removed on link down

[David Miller]

From: David Ahern <dsahern@gmail.com>
Date: Tue, 13 Jan 2015 07:53:59 -0700

> Bottom line is there a harm in removing the flush? If there is no harm
> will mainline kernel take a patch to do that or is your backward
> compatibility concern enough to block it?

Backward compatibility trumps all other concerns here, and I say is
enough to block changing the behavior.  One which we've had for more
than a decade.

Re: why are IPv6 addresses removed on link down

[Hannes Frederic Sowa]

On Di, 2015-01-13 at 12:25 -0500, David Miller wrote:
> From: David Ahern <dsahern@gmail.com>
> Date: Tue, 13 Jan 2015 07:53:59 -0700
> 
> > Bottom line is there a harm in removing the flush? If there is no harm
> > will mainline kernel take a patch to do that or is your backward
> > compatibility concern enough to block it?
> 
> Backward compatibility trumps all other concerns here, and I say is
> enough to block changing the behavior.  One which we've had for more
> than a decade.

Totally agreed, a new sysctl will definitely be needed and the default
should be the old behavior. Otherwise we cannot do it.