All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Garry <john.garry@linux.dev>
To: Laurence Oberman <loberman@redhat.com>,
	james.bottomley@hansenpartnership.com, mkp@kernel.org
Cc: hare@suse.com, linux-nvme@lists.infradead.org,
	linux-scsi@vger.kernel.org, snitzer@kernel.org,
	jmeneghi@redhat.com, bmarzins@redhat.com,
	dm-devel@lists.linux.dev, hch@lst.de,
	michael.christie@oracle.com
Subject: Re: [PATCH v9 00/29] Native SCSI Multipath support
Date: Sun, 15 Nov 2026 18:59:37 +0000	[thread overview]
Message-ID: <01eac6bf-8df7-4e7a-8cc6-93e013750bce@linux.dev> (raw)
In-Reply-To: <9b90edc29d5b3be49f5bd0e0ecec5f1701e089bf.camel@redhat.com>

On 11/9/26 20:19, Laurence Oberman wrote:
> 2. ANO paths are ranked the same as AO paths
> 
> With the ALUA state corrected (by writing "activate" to dh_state, or
> with the fix above), I/O still went to active/non-optimized paths.
> scsi_multipath_sdev_init() sets mpath_device->access_state to
> MPATH_STATE_OPTIMIZED and nothing updates it, while
> __mpath_find_path(), mpath_round_robin_path() and
> mpath_queue_depth_path() rank paths by that field. Only
> scsi_mpath_is_optimized()/is_disabled() read sdev->access_state.
> 
> With numa, the first path in dev_list wins for every node, and
> dev_list order follows sd probe completion, so for about a third of
> the LUNs the current path was an ANO path. Example, LUN sdn, reads
> completed per path for a 2000 x 1M O_DIRECT dd (:0/:3 AO, :1/:2 ANO
> at the time):
> 
>    sdn:0 0sdn:1 2000sdn:2 0sdn:3 0
> 
> Every read went through the non-owning controller. From the code, for
> these LUNs the numa recheck of is_optimized() fails on every I/O and
> __mpath_find_path() picks the same ANO path again.
> 
> With the change below (a template callback so the selectors use the
> live state), plus the scsi_dh_alua fix, all 30 LUNs pick an AO
> current path at boot, and on LUN sdl (:0/:2 AO, :1/:3 ANO):
> 
>    numa         2000     0     0     0
>    round-robin  1000     0  1000     0
>    queue-depth 18039     0 13961     0   (8 parallel dd, 64k)
> 
> No I/O reached the ANO paths under any policy. This is a test patch to
> show the problem, not a polished change - feel free to fold it in or
> do it differently.


JFYI, the work which I did for ALUA support a while ago is here:
#https://github.com/johnpgarry/linux/commits/scsi-multipath-v7.3-v7-alua/

I don't think that I shared this previously.

I do notice that it would conflict with your work to do some 
centralising of the ALUA RTPG support.

About the NUMA iopolicy - apart from issues in this series with not 
checking optimised/access_state properly - it does seem less than 
optimal to we only favour one device per NUMA node (when all in same 
optimized + enabled state). NVMe multipath support does the same. I 
suppose that this iopolicy relies on us having one device/path per NUMA 
node.

Thanks,
John


  parent reply	other threads:[~2026-11-15 18:59 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-08 20:47 [PATCH v9 00/29] Native SCSI Multipath support John Garry
2026-11-08 20:47 ` [PATCH v9 01/29] scsi: core: Clear cmd->flags for terminal queuing error John Garry
2026-11-09 20:49   ` Bart Van Assche
2026-11-15 18:37     ` John Garry
2026-11-15 20:42       ` Bart Van Assche
2026-11-08 20:47 ` [PATCH v9 02/29] libmultipath: Add initial framework John Garry
2026-11-08 20:47 ` [PATCH v9 03/29] libmultipath: Add basic gendisk support John Garry
2026-11-08 20:47 ` [PATCH v9 04/29] libmultipath: Add path selection support John Garry
2026-11-08 21:07   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 05/29] libmultipath: Add bio handling John Garry
2026-11-08 20:47 ` [PATCH v9 06/29] libmultipath: Add support for mpath_device management John Garry
2026-11-08 20:47 ` [PATCH v9 07/29] libmultipath: Add delayed removal support John Garry
2026-11-08 21:11   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 08/29] libmultipath: Add sysfs helpers John Garry
2026-11-08 21:02   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 09/29] libmultipath: Add support for block device IOCTL John Garry
2026-11-08 21:09   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 10/29] libmultipath: Add mpath_bdev_getgeo() John Garry
2026-11-08 20:47 ` [PATCH v9 11/29] libmultipath: Add mpath_bdev_get_unique_id() John Garry
2026-11-08 20:47 ` [PATCH v9 12/29] scsi-multipath: introduce basic SCSI device support John Garry
2026-11-08 21:02   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 13/29] scsi-multipath: introduce scsi_device head structure John Garry
2026-11-08 20:47 ` [PATCH v9 14/29] scsi-multipath: provide sysfs link from to scsi_device John Garry
2026-11-08 20:47 ` [PATCH v9 15/29] scsi-multipath: support iopolicy John Garry
2026-11-08 20:47 ` [PATCH v9 16/29] scsi-multipath: clone each bio John Garry
2026-11-08 20:47 ` [PATCH v9 17/29] scsi-multipath: clear path when device is blocked John Garry
2026-11-08 21:06   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 18/29] scsi-multipath: revalidate paths upon device unblock John Garry
2026-11-08 21:08   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 19/29] scsi-multipath: failover handling John Garry
2026-11-08 21:15   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 20/29] scsi-multipath: provide callbacks for path state John Garry
2026-11-08 21:05   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 21/29] scsi-multipath: add scsi_mpath_{start,end}_request() John Garry
2026-11-08 21:28   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 22/29] scsi-multipath: update request for terminal request queuing error John Garry
2026-11-08 21:09   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 23/29] scsi-multipath: add delayed disk removal support John Garry
2026-11-08 20:47 ` [PATCH v9 24/29] scsi: sd: add multipath disk class John Garry
2026-11-08 20:47 ` [PATCH v9 25/29] scsi: sd: add multipath disk attr groups John Garry
2026-11-08 20:47 ` [PATCH v9 26/29] scsi: sd: support multipath disk John Garry
2026-11-08 21:23   ` sashiko-bot
2026-11-08 20:47 ` [PATCH v9 27/29] scsi: sd: add mpath_dev file John Garry
2026-11-08 20:47 ` [PATCH v9 28/29] scsi: sd: add mpath_numa_nodes dev attribute John Garry
2026-11-08 20:47 ` [PATCH v9 29/29] scsi: sd: add mpath_queue_depth " John Garry
2026-11-09 20:19 ` [PATCH v9 00/29] Native SCSI Multipath support Laurence Oberman
2026-11-10  9:05   ` John Garry
2026-11-14 13:03     ` Xose Vazquez Perez
2026-11-14 16:29       ` Laurence Oberman
2026-11-14 19:59         ` Laurence Oberman
2026-11-15 17:43       ` John Garry
2026-11-15 18:59   ` John Garry [this message]
2026-11-15 19:50     ` Laurence Oberman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=01eac6bf-8df7-4e7a-8cc6-93e013750bce@linux.dev \
    --to=john.garry@linux.dev \
    --cc=bmarzins@redhat.com \
    --cc=dm-devel@lists.linux.dev \
    --cc=hare@suse.com \
    --cc=hch@lst.de \
    --cc=james.bottomley@hansenpartnership.com \
    --cc=jmeneghi@redhat.com \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=loberman@redhat.com \
    --cc=michael.christie@oracle.com \
    --cc=mkp@kernel.org \
    --cc=snitzer@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.