From: Benjamin Marzinski <bmarzins@redhat.com>
To: Martin Wilck <mwilck@suse.com>
Cc: Brian Bunker <brian@purestorage.com>,
Christophe Varoqui <christophe.varoqui@opensvc.com>,
dm-devel@lists.linux.dev
Subject: Re: [PATCH v4 1/1] multipathd: recheck WWIDs of re-added paths with the device
Date: Mon, 16 Nov 2026 14:46:03 -0500 [thread overview]
Message-ID: <avtdexYN1B3pJLmL@redhat.com> (raw)
In-Reply-To: <9259d20188a042cb280bad95f8be9783d1b9c1fd.camel@suse.com>
On Fri, Nov 13, 2026 at 09:52:23PM +0100, Martin Wilck wrote:
> On Fri, 2026-11-13 at 13:00 -0500, Benjamin Marzinski wrote:
> > On Thu, Nov 12, 2026 at 11:24:32PM +0100, Martin Wilck wrote:
> > >
> > > > }
> > > > + /*
> > > > + * This path is being re-added because its udev data was
> > > > missing or
> > > > + * changed. The uevent may have been queued long before
> > > > we got here,
> > > > + * and adding a path to the map of a different LUN
> > > > corrupts data. If
> > > > + * recheck_wwid is set, confirm the WWID with the device
> > > > first. On a
> > > > + * mismatch, leave the path out and let the checker
> > > > retrigger udev
> > > > + * to get a fresh WWID.
> > > > + */
> > > > + if (mode == PATH_ADD_REINIT && pp->recheck_wwid ==
> > > > RECHECK_WWID_ON &&
> > > > + check_path_wwid_change(pp)) {
> > >
> > > check_path_wwid_change() returns false when it fails, is it correct
> > > to keep the
> > > path in that case? IMHO we should treat this like the strlen(pp-
> > > >wwid) == 0 case
> > > above.
> >
> > Not sure I agree with this. It's a paranoia check. If it fails, I
> > don't
> > have a problem with just trusting the data that was given to us. I'm
> > willing to be convinced otherwise, however.
>
> In my mind, a changed WWID followed by a failure to obtain the WWID
> from the device shows that something is really fishy about the path in
> question.
>
> Moreover:
>
> I realize that uev_update_path() calls get_uid() with allow_fallback=0.
> This goes back to your commit ffaf947 ("multipathd: Don't use fallback
> code after getting wwid"), where the rationale you provided was
>
> > once the
> > devices are set up, udev is the definitive source of this information
> > ...
> > Working under the assumption that udev will either never give us this
> > information, or that it usually will.
>
> We have given up this assumption, at least partly. In view of the late
> discussions, it seems reasonable to me to allow fallback to SG_IO if
> udev failed, at least if pp->wwid was non-zero before.
>
> Then, if the fallback failed, the situation would really be almost
> identical to the failing paranoia check.
I wrote my objection assuming that this code would also be called when
when we simply hadn't gotten the WWID on the first uevent. If we
retrigger a uevent and then get the WWID, but the SG_IO fails, we could
end up not creating the multipath device at all, despite the the path
existing with a WWID (albeit with some issues). Perhaps that's the
correct answer for an obviously flakey device. If this code was only run
when we got a wwid before and then it changed, then requiring a matching
WWID from the device seems much more reasonable.
> >
> > makes sense.
> >
> > > But more generally: as this indicates a possible mismatch between
> > > kernel and device
> > > state, should we handle this similar to handle_path_wwid_change()
> > > instead,
> > > i.e. also call rescan_path()? I guess that would need to be done in
> > > uev_update_path(),
> > > i.e. it would touch the regular "wwid changed in uevent" path.
> >
> > Is this not another place were we are opening ourselves up for
> > commands
> > blocking? If we want to do a rescan and it can block, it makes more
> > sense to do it only if recheck_wwids is set, and we know that the
> > path
> > is up.
> >
>
> Right, I didn't consider that rescan_path() may block. But still, why
> do we need to rescan in handle_path_wwid_change(), but not here?
>
> AFAICS, the likelihood that the kernel is using stale information is
> quite high when we handle this situation. I'm not sure if testing
> whether the path is up makes much sense. There are many situations in
> which the path would be down, but rescan/inquiry would still work. The
> only thing that would really hurt us are timeouts during the kernel's
> probing procedure, but I am not aware of an easy way to predict whether
> that will happen.
>
> I suppose we could rescan asynchronously in this case. After all, we
> already have obtained the real WWID from the device, therefore we don't
> need to wait for the rescan to finish.
I'm not against the rescan, I just thought it made more sense in
ev_add_path(), although running it asynchronously works too.
> > > >
> > > > + }
> > > > mpp = find_mp_by_wwid(vecs->mpvec, pp->wwid);
> > > > if (mpp && pp->size && mpp->size != pp->size) {
> > > > condlog(0, "%s: failed to add new path %s,
> > > > device size mismatch", mpp->alias, pp->dev);
> > > > @@ -1375,7 +1390,7 @@ rescan:
> > > > if (ro == 1)
> > > > mpp->force_readonly = 1;
> > > >
> > > > - if (!need_do_map)
> > > > + if (mode == PATH_ADD_NO_MAP)
> > > > return 0;
> > > >
> > > > if (!dm_map_present(mpp->alias)) {
> > > > @@ -1789,7 +1804,7 @@ out:
> > > > }
> > > > /* pp->initialized must not be INIT_PARTIAL if
> > > > needs_reinit is set */
> > > > if (needs_reinit)
> > > > - retval = uev_add_path(uev, vecs, 1);
> > > > + retval = uev_add_path(uev, vecs,
> > > > PATH_ADD_REINIT);
> > >
> > > Is it correct to take this branch also if needs_reinit is set
> > > because
> > > INIT_REQUESTED_UDEV was set? I don't think so.
> >
> > This is a little tricky. I agree that we don't need to add the extra
> > layer of paranoid checking when the path never got a wwid. In that
> > case,
> > what most likely what happened is that the udev worker thread timed
> > out
> > on the previous uevent, so we couldn't get the WWID from uevent
> > variables. However, if we don't recheck on INIT_REQUESTED_UDEV, then
> > we
> > can get into a situation where:
> > 1. uev_update_path() notices that the wwid has changed
> > 2. ev_add_path() confirms that the wwid we got from udev doesn't
> > match
> > the wwid we get directly asking the device
> > 3. We trigger a new uevent
> > 4. We blindly trust udev's wwid.
>
> We need to trust udev at some point, don't we? Assume
>
> 4. we double check again
> 5. ev_add_path() confirms that the wwid we got from udev doesn't match
> 6. goto 3
>
> At least we'd need to avoid getting stuck in a loop.
This will only loop for retrigger_tries. Then it will give up and trust
udev without double-checking. Athough we never reset pp->retriggers once
we have a reasonable WWID, so we only get retrigger_tries for the entire
lifetime of the path. It seems like the retries when we couldn't get a
WWID from udev are different than the retries when the WWIDs don't
match.
> AFAICS there are 2 reasons to do the SG_IO check:
>
> 1) vpd info from sysfs may be stale (affects SUSE only)
> 2) something changed between the VPD detection in udev rules and our
> event handling
>
> Then there is
>
> 3) a bug e.g. in an udev rule causes ID_SERIAL to be consistently
> wrong.
>
> Case 1) would be fixed triggering a rescan (see above). In case 2), it
> makes a certain amount of sense to trust udev the next time. Case 3)
> would cause an infinite loop if we keep double-checking.
>
> > Perhaps we need more initialization states or a path device flag to
> > distinguish between the two cases (never having gotten a WWID, or
> > having
> > our WWID . Or we can decide that we are o.k. with extra checking
> > whenever something goes wrong with getting the wwid from udev (when
> > you
> > have recheck_wwid set). Or am I misunderstanding something about why
> > doing this is wrong? Are we intentionally only double-checking once
> > here?
>
> I don't like the idea of checking the first time around very much, but
> I agree it isn't a big deal. OTOH, see above.
If it's fine to only do one double-check before trusting udev, then
PATH_ADD_REINIT is just for that one recheck, and what to do in that
case is less ambiguous.
> Yet another possibility is to introduce a new mode where we just never
> trust udev. That'd be a 180" turn, but it might be worth trying.
> The longer we discuss this, the more attractive it seems to me. Rather
> than guessing whether or not we can trust udev, just use a "zero trust"
> model :-)
I wouldn't be against it. Device WWIDs could potentially switch between
modes. But as long as users know that they would need to be careful when
switching modes, that should be fine. Were you thinking of always
getting the wwid through SG_IO, or were you thinking of using something
like sysfs that could still potentially give you a stale value?
-Ben
> Regards
> Martin
>
> --
> Dr. Martin Wilck <mwilck@suse.com>
> SUSE Software Solutions Germany GmbH, Frankenstr. 146, 90461 Nürnberg,
> Germany
> Geschäftsführer: Stefan Gaiser, Jochen Jaser, Abhinav Puri (HRB
> 36809,AG Nürnberg)
next prev parent reply other threads:[~2026-11-16 19:46 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 18:16 [PATCH v4 0/1] multipathd: recheck WWIDs of re-added paths with the device Brian Bunker
2026-11-11 18:16 ` [PATCH v4 1/1] " Brian Bunker
2026-11-12 22:24 ` Martin Wilck
2026-11-13 18:00 ` Benjamin Marzinski
2026-11-13 20:52 ` Martin Wilck
2026-11-16 18:43 ` Brian Bunker
2026-11-16 22:19 ` Martin Wilck
2026-11-16 19:46 ` Benjamin Marzinski [this message]
2026-11-16 22:27 ` Martin Wilck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avtdexYN1B3pJLmL@redhat.com \
--to=bmarzins@redhat.com \
--cc=brian@purestorage.com \
--cc=christophe.varoqui@opensvc.com \
--cc=dm-devel@lists.linux.dev \
--cc=mwilck@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.