From: Stefano Brivio <sbrivio@redhat.com>
To: "Íñigo Huguet" <ihuguet@redhat.com>
Cc: Thorsten Leemhuis <regressions@leemhuis.info>,
Fernando Fernandez Mancera <fmancera@suse.de>,
Jakub Kicinski <kuba@kernel.org>,
netdev@vger.kernel.org, Yumei Huang <yuhuang@redhat.com>,
Ido Schimmel <idosch@idosch.org>,
Justin Iurman <justin.iurman@gmail.com>,
David Ahern <dsahern@kernel.org>,
David Gibson <david@gibson.dropbear.id.au>,
Linux kernel regressions list <regressions@lists.linux.dev>,
Beniamino Galvani <bgalvani@redhat.com>
Subject: Re: Problem with IPv6 privacy addresses in 7.0
Date: Thu, 28 May 2026 16:53:21 +0200 (CEST) [thread overview]
Message-ID: <20260528165320.15b90ded@elisabeth> (raw)
In-Reply-To: <CACT4ouf5ZqGgb=RO8_7NovLCvYKLKL4vD70dEe6xUODT1UEcSA@mail.gmail.com>
On Thu, 28 May 2026 16:15:38 +0200
Íñigo Huguet <ihuguet@redhat.com> wrote:
> CCing Beniamino, NetworkManager maintainer.
>
> On Thu, May 28, 2026 at 3:32 PM Stefano Brivio <sbrivio@redhat.com> wrote:
> > Not a bigger regression, but a regression definitely, even just for 'ip
> > address' or 'ip address showdump' which would now go back to display
> > / save addresses in the reversed order for IPv6 (only).
>
> One could argue that the bug had to be fixed in iproute2 and not in
> the kernel, to avoid breaking UAPI.
I think what's considered UAPI in this case isn't so clear-cut. I guess
it would have been pretty hard for anybody not familiar with
NetworkManager's codebase to imagine that an application would rely on
IPv6 addresses (and only IPv6) to be returned in the reverse order.
Pushing this to the extreme, one could argue for example that if we
accidentally byteswapped the last 16 bits of some IPv6 addresses, and
NetworkManager started to work around that, that would start being
considered established UAPI...
> > Actually, an eventually fixed version of NetworkManager doesn't need to
> > know the behaviour of the kernel: it can just order addresses by
> > timestamps instead, as Fernando mentioned.
>
> If the fix must be in NetworkManager, we only need to parse them in
> non-reverse order like IPv4, I guess.
But that would then require some form of detection, and, at least
according to Fernando, isn't the most robust option anyway, as ideally
NetworkManager shouldn't rely on the order at all.
> > And I'm not sure how relevant this is, but if we revert the fix,
> > current combinations of NetworkManager / kernel versions would be
> > anyway affected.
>
> Probably many, for who knows how much time, unless all distros
> backport the NetworkManager patch. Many don't backport much except a
> small bunch of security fixes.
>
> > So at this point the only robust / complete fix would be changing
> > NetworkManager to sort addresses as needed. Are you suggesting that we
> > should anyway try to minimise the temporal impact of this with a revert?
>
> I don't think it's the "only" robust fix. Normally one would think
> first about not breaking UAPI.
Sure... I would just argue that in this case it was pretty hard to
imagine that as an established UAPI trait.
> Kernel's UAPI is not great or intuitive
> in many places, but normally it's never improved to avoid breaking
> backward compatibility. Adding a GIVE_ME_THE_RIGHT_ORDER flag could
> help too, but I agree that it sounds a bit silly to have an opt-in
> option to force the right behaviour.
>
> Then, the only robust fix would be to fix pasta and iproute2.
I was mentioning that we can't really fix this in pasta (unless, again,
we implement some form of detection, or something on the lines of what
Thorsten suggested) because we're just copying addresses to a container
and we don't know much about them.
We could add detection and swap them, which is rather complicated for
us because we currently don't store them as we copy them (we don't use
dynamic memory allocation), but of course that would be on us to figure
out, somehow.
> Here we know of 2 affected userspace programs, but we don't know if
> more could be discovered in the future.
>
> That said, it is true that this case is very unintuitive. Insertion is
> direct-order for both IPv4 and 6. Dump is reversed, but only for IPv6.
Wait... that's how confusing it is. That's also what I thought at the
beginning of https://bugs.passt.top/show_bug.cgi?id=175, but it turned
out that insertion is actually reversed for IPv6. Dump isn't.
So this is also inconsistent in the kernel itself: if everything else
is equal, address selection depends on the order of addresses as
inserted, which is the opposite compared to IPv4. Maybe this is the
most surprising fact in all this.
> Userspace programs won't expect this at all. Programs that do it right
> it's probably because they encountered a bug because they were
> initially reading in direct-order, and changed to reverse-order. I
> haven't dug into NetworkManager's history but I bet this was the case.
I guess so too, yes.
> In this case I would be in favor of not reverting the kernel change,
> and fix NetworkManager, but I'd like to hear Beniamino's opinion, as
> I'm not being much involved in NetworkManager any more.
--
Stefano
next prev parent reply other threads:[~2026-05-28 14:54 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-21 13:53 Problem with IPv6 privacy addresses in 7.0 Chris Adams
2026-05-27 0:57 ` Jakub Kicinski
2026-05-27 1:06 ` Chris Adams
2026-05-27 1:31 ` Jakub Kicinski
2026-05-27 21:13 ` Chris Adams
2026-05-27 21:16 ` Fernando Fernandez Mancera
2026-05-27 21:51 ` Jakub Kicinski
2026-05-27 21:51 ` Chris Adams
2026-05-27 21:59 ` Fernando Fernandez Mancera
2026-05-27 23:07 ` Jakub Kicinski
2026-05-28 5:38 ` Stefano Brivio
2026-05-28 10:46 ` Fernando Fernandez Mancera
2026-05-28 11:12 ` Stefano Brivio
2026-05-28 11:29 ` Fernando Fernandez Mancera
2026-05-28 12:29 ` Thorsten Leemhuis
2026-05-28 13:32 ` Stefano Brivio
2026-05-28 14:02 ` Thorsten Leemhuis
2026-05-28 14:15 ` Íñigo Huguet
2026-05-28 14:53 ` Stefano Brivio [this message]
2026-05-28 15:24 ` Íñigo Huguet
2026-05-28 16:01 ` Beniamino Galvani
2026-05-28 17:21 ` Stefano Brivio
2026-05-28 14:34 ` Andrew Lunn
2026-05-28 15:17 ` Stefano Brivio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260528165320.15b90ded@elisabeth \
--to=sbrivio@redhat.com \
--cc=bgalvani@redhat.com \
--cc=david@gibson.dropbear.id.au \
--cc=dsahern@kernel.org \
--cc=fmancera@suse.de \
--cc=idosch@idosch.org \
--cc=ihuguet@redhat.com \
--cc=justin.iurman@gmail.com \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=regressions@leemhuis.info \
--cc=regressions@lists.linux.dev \
--cc=yuhuang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox