Problem with IPv6 privacy addresses in 7.0

Problem with IPv6 privacy addresses in 7.0

[Chris Adams]

My normal use of my desktop system (Fedora 43) is to suspend at night
and wake in the morning, and then immediately SSH to my remote server,
where I stay logged in.  I use NetworkManager and have ipv6.ip6-privacy
set to prefer-temp-addr, and SSH configured to use SSH keepalives.

When I upgraded to kernel 7.0, I started having an issue with this - my
SSH session gets dropped, usually several times, in the first hour or
so, then I usually don't have any problem the rest of the day.  If I run
an IPv4 session at the same time, that seems to be fine, so I don't
think it's a network issue (I know that dual-stack doesn't always take
the same path).

What seems to be happening is that privacy addresses are removed while
the SSH session is still using them, before the timeout even.  I wrote a
script to watch public v6 addresses being added and removed, and this is
what I've seen so far today (the number at the end is the valid_lft
seconds), with the public prefix masked:

2026-05-21 07:39:17 removed xx::f4f/128 4673
2026-05-21 07:39:20 added xx::f4f/128 5400
2026-05-21 07:41:20 removed xx:3e8c:f8ff:fe60:1d5a/64 4922
2026-05-21 07:41:20 removed xx:7cb1:c518:1be0:d81d/64 4922
2026-05-21 07:41:23 added xx:3e8c:f8ff:fe60:1d5a/64 5398
2026-05-21 07:41:23 added xx:596a:f6f5:67b2:1d8f/64 5398
2026-05-21 08:14:43 added xx:fac3:61f6:ad18:d712/64 4987
2026-05-21 08:14:43 removed xx:596a:f6f5:67b2:1d8f/64 4991
2026-05-21 08:30:26 added xx:84b4:244e:bb14:94fd/64 5398
2026-05-21 08:30:26 removed xx:fac3:61f6:ad18:d712/64 5120

I woke the system at 07:39:08 and SSHed at 07:39:39, which used the d81d
source address.  That dropped in 2 minutes and I reconnected, which used
the 1d8f address.  That dropped at 08:14:43, I didn't notice right away,
I reconnected at 08:31:28 which used the 94fd address.

-- 
Chris Adams <linux@cmadams.net>

Re: Problem with IPv6 privacy addresses in 7.0

[Jakub Kicinski]

On Thu, 21 May 2026 08:53:10 -0500 Chris Adams wrote:
> My normal use of my desktop system (Fedora 43) is to suspend at night
> and wake in the morning, and then immediately SSH to my remote server,
> where I stay logged in.  I use NetworkManager and have ipv6.ip6-privacy
> set to prefer-temp-addr, and SSH configured to use SSH keepalives.
> 
> When I upgraded to kernel 7.0, I started having an issue with this - my
> SSH session gets dropped, usually several times, in the first hour or
> so, then I usually don't have any problem the rest of the day.  If I run
> an IPv4 session at the same time, that seems to be fine, so I don't
> think it's a network issue (I know that dual-stack doesn't always take
> the same path).
> 
> What seems to be happening is that privacy addresses are removed while
> the SSH session is still using them, before the timeout even.  I wrote a
> script to watch public v6 addresses being added and removed, and this is
> what I've seen so far today (the number at the end is the valid_lft
> seconds), with the public prefix masked:
> 
> 2026-05-21 07:39:17 removed xx::f4f/128 4673
> 2026-05-21 07:39:20 added xx::f4f/128 5400
> 2026-05-21 07:41:20 removed xx:3e8c:f8ff:fe60:1d5a/64 4922
> 2026-05-21 07:41:20 removed xx:7cb1:c518:1be0:d81d/64 4922
> 2026-05-21 07:41:23 added xx:3e8c:f8ff:fe60:1d5a/64 5398
> 2026-05-21 07:41:23 added xx:596a:f6f5:67b2:1d8f/64 5398
> 2026-05-21 08:14:43 added xx:fac3:61f6:ad18:d712/64 4987
> 2026-05-21 08:14:43 removed xx:596a:f6f5:67b2:1d8f/64 4991
> 2026-05-21 08:30:26 added xx:84b4:244e:bb14:94fd/64 5398
> 2026-05-21 08:30:26 removed xx:fac3:61f6:ad18:d712/64 5120
> 
> I woke the system at 07:39:08 and SSHed at 07:39:39, which used the d81d
> source address.  That dropped in 2 minutes and I reconnected, which used
> the 1d8f address.  That dropped at 08:14:43, I didn't notice right away,
> I reconnected at 08:31:28 which used the 94fd address.

Hi! Adding more people to CC. Do you know if you upgraded from 6.18 
or 6.19?

Would you be able to try testing with some commits reverted?
On a quick look the candidates would be:

cb3de96eea66 ("ipv6: preserve insertion order for same-scope addresses")
c7dc5b522882 ("ipv6: clean up routes when manually removing address with a lifetime")

Less likely:
5023479627e3 ("ipv6: Switch to higher-level SHA-1 functions")
9e371b0ba7f5 ("ipv6: addrconf: reduce default temp_valid_lft to 2 days")
6af51e9f3133 ("ipv6: Remove permanent routes from tb6_gc_hlist when all exceptions expire.")

Re: Problem with IPv6 privacy addresses in 7.0

[Chris Adams]

Heh, I opened a Fedora BZ on this and JUST updated it... RHBZ 2480928

Once upon a time, Jakub Kicinski <kuba@kernel.org> said:
> On Thu, 21 May 2026 08:53:10 -0500 Chris Adams wrote:
> > My normal use of my desktop system (Fedora 43) is to suspend at night
> > and wake in the morning, and then immediately SSH to my remote server,
> > where I stay logged in.  I use NetworkManager and have ipv6.ip6-privacy
> > set to prefer-temp-addr, and SSH configured to use SSH keepalives.
> > 
> > When I upgraded to kernel 7.0, I started having an issue with this - my
> > SSH session gets dropped, usually several times, in the first hour or
> > so, then I usually don't have any problem the rest of the day.  If I run
> > an IPv4 session at the same time, that seems to be fine, so I don't
> > think it's a network issue (I know that dual-stack doesn't always take
> > the same path).
> > 
> > What seems to be happening is that privacy addresses are removed while
> > the SSH session is still using them, before the timeout even.  I wrote a
> > script to watch public v6 addresses being added and removed, and this is
> > what I've seen so far today (the number at the end is the valid_lft
> > seconds), with the public prefix masked:
> > 
> > 2026-05-21 07:39:17 removed xx::f4f/128 4673
> > 2026-05-21 07:39:20 added xx::f4f/128 5400
> > 2026-05-21 07:41:20 removed xx:3e8c:f8ff:fe60:1d5a/64 4922
> > 2026-05-21 07:41:20 removed xx:7cb1:c518:1be0:d81d/64 4922
> > 2026-05-21 07:41:23 added xx:3e8c:f8ff:fe60:1d5a/64 5398
> > 2026-05-21 07:41:23 added xx:596a:f6f5:67b2:1d8f/64 5398
> > 2026-05-21 08:14:43 added xx:fac3:61f6:ad18:d712/64 4987
> > 2026-05-21 08:14:43 removed xx:596a:f6f5:67b2:1d8f/64 4991
> > 2026-05-21 08:30:26 added xx:84b4:244e:bb14:94fd/64 5398
> > 2026-05-21 08:30:26 removed xx:fac3:61f6:ad18:d712/64 5120
> > 
> > I woke the system at 07:39:08 and SSHed at 07:39:39, which used the d81d
> > source address.  That dropped in 2 minutes and I reconnected, which used
> > the 1d8f address.  That dropped at 08:14:43, I didn't notice right away,
> > I reconnected at 08:31:28 which used the 94fd address.
> 
> Hi! Adding more people to CC. Do you know if you upgraded from 6.18 
> or 6.19?

It was 6.19 to 7.0.

> Would you be able to try testing with some commits reverted?
> On a quick look the candidates would be:
> 
> cb3de96eea66 ("ipv6: preserve insertion order for same-scope addresses")

It's this one.

I figured out that it happens after stopping a VM (and I usually
start/stop a VM for a bit in the morning, which is why it happened more
than once).  So I set up a VM with a nested VM, running up-to-date
Fedora 44, and then was able to bisect pretty easily, and it landed on
this commit.

Fedora is using NetworkManager, and IIRC NM does some part of privacy
address management (right?).  NM didn't change, so maybe this commit is
confusing something in NM?

-- 
Chris Adams <linux@cmadams.net>

Re: Problem with IPv6 privacy addresses in 7.0

[Jakub Kicinski]

On Tue, 26 May 2026 20:06:41 -0500 Chris Adams wrote:
> > Hi! Adding more people to CC. Do you know if you upgraded from 6.18 
> > or 6.19?  
> 
> It was 6.19 to 7.0.
> 
> > Would you be able to try testing with some commits reverted?
> > On a quick look the candidates would be:
> > 
> > cb3de96eea66 ("ipv6: preserve insertion order for same-scope addresses")  
> 
> It's this one.

Phew, the second one was mine :)

> I figured out that it happens after stopping a VM (and I usually
> start/stop a VM for a bit in the morning, which is why it happened more
> than once).  So I set up a VM with a nested VM, running up-to-date
> Fedora 44, and then was able to bisect pretty easily, and it landed on
> this commit.
> 
> Fedora is using NetworkManager, and IIRC NM does some part of privacy
> address management (right?).  NM didn't change, so maybe this commit is
> confusing something in NM?

Sounds plausible, pretty sure we knew this commit was risky to begin
with, but we had no direct proof that it'd break real life users.

Revert is the right course of action here. Would you be willing/able
to send the revert with your problem description and a Fixes tag
pointing to the reverted commit?