Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

[John Garry]

On 24/07/2026 06:01, Christoph Hellwig wrote:
> On Tue, Jul 21, 2026 at 11: 45: 53AM +0000, John Garry wrote: > Setting as an
> RFC as adding this extra bio field is not acceptable, but I > can't see how to
> lookup the original partition. In general it is not, but you should mark this a
> 
> 
> On Tue, Jul 21, 2026 at 11:45:53AM +0000, John Garry wrote:
>> Setting as an RFC as adding this extra bio field is not acceptable, but I
>> can't see how to lookup the original partition.
> 
> In general it is not, but you should mark this a block patch so that
> Jens can better cream at you :)
> 
> I'm also not sure that supporting per-partition diskstats on a multipath
> device makes too much sense, but then again there's a lot of setups
> that are crazy and actually used..
> 

It just seems to me that we should have same behaviour as if it were not 
multipath.

So we can't use bi_private as that can be set by original bio submitter. 
I was thinking that this bdev pointer could be temp stashed in bi_next 
(as it should be originally NULL), but that it dodgy and maybe won't 
even work.

Least worst I can think if is to alloc some temp memory per-bio in 
nvme_ns_head_submit_bio() to hold this bdev pointer and original 
bi_private, set bio->bi_private to that memory, and then set bi_private 
back to original when we end the bio. But this is all crappy, especially 
just for partition diskstats.

Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

[Keith Busch]

On Fri, Jul 24, 2026 at 08:09:51AM +0100, John Garry wrote:
> On 24/07/2026 06:01, Christoph Hellwig wrote:
> > On Tue, Jul 21, 2026 at 11: 45: 53AM +0000, John Garry wrote: > Setting as an
> > RFC as adding this extra bio field is not acceptable, but I > can't see how to
> > lookup the original partition. In general it is not, but you should mark this a
> > 
> > 
> > On Tue, Jul 21, 2026 at 11:45:53AM +0000, John Garry wrote:
> > > Setting as an RFC as adding this extra bio field is not acceptable, but I
> > > can't see how to lookup the original partition.
> > 
> > In general it is not, but you should mark this a block patch so that
> > Jens can better cream at you :)
> > 
> > I'm also not sure that supporting per-partition diskstats on a multipath
> > device makes too much sense, but then again there's a lot of setups
> > that are crazy and actually used..
> > 
> 
> It just seems to me that we should have same behaviour as if it were not
> multipath.
> 
> So we can't use bi_private as that can be set by original bio submitter. I
> was thinking that this bdev pointer could be temp stashed in bi_next (as it
> should be originally NULL), but that it dodgy and maybe won't even work.

Failover is corner case to consider here. We always reset the bio bdev
to the part0, so your stats will be incorrect when that happens. I think
you can quickly fix that in your proposal with the "bi_orig", though.

Can we just thread through partitions for the bio's block_device instead
of assuming part0? I know the hidden path devices skip partition
scanning, but maybe if we let it happen, then those will have the same
partition setup as the head gendisk. Then we can go right to the
disk->part_tbl for what we provide to bio_set_dev() for both submission
and failover, and everything should work out from there.

Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

[John Garry]

On 7/24/26 13:45, Keith Busch wrote:
> On Fri, Jul 24, 2026 at 08:09:51AM +0100, John Garry wrote:
>> On 24/07/2026 06:01, Christoph Hellwig wrote:
>>> On Tue, Jul 21, 2026 at 11: 45: 53AM +0000, John Garry wrote: > Setting as an
>>> RFC as adding this extra bio field is not acceptable, but I > can't see how to
>>> lookup the original partition. In general it is not, but you should mark this a
>>>
>>>
>>> On Tue, Jul 21, 2026 at 11:45:53AM +0000, John Garry wrote:
>>>> Setting as an RFC as adding this extra bio field is not acceptable, but I
>>>> can't see how to lookup the original partition.
>>>
>>> In general it is not, but you should mark this a block patch so that
>>> Jens can better cream at you :)
>>>
>>> I'm also not sure that supporting per-partition diskstats on a multipath
>>> device makes too much sense, but then again there's a lot of setups
>>> that are crazy and actually used..
>>>
>>
>> It just seems to me that we should have same behaviour as if it were not
>> multipath.
>>
>> So we can't use bi_private as that can be set by original bio submitter. I
>> was thinking that this bdev pointer could be temp stashed in bi_next (as it
>> should be originally NULL), but that it dodgy and maybe won't even work.
> 
> Failover is corner case to consider here. We always reset the bio bdev
> to the part0, so your stats will be incorrect when that happens. I think
> you can quickly fix that in your proposal with the "bi_orig", though.
> 
> Can we just thread through partitions for the bio's block_device instead
> of assuming part0? I know the hidden path devices skip partition
> scanning, but maybe if we let it happen, then those will have the same
> partition setup as the head gendisk. Then we can go right to the
> disk->part_tbl for what we provide to bio_set_dev() for both submission
> and failover, and everything should work out from there.

I guess that we would just scan through the per-path gendisk->part_tbl 
and match somehow to lookup the partition. Maybe vs start address of 
bio->bi_bdev. Or is there a simpler (and quicker) way?

Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

[Keith Busch]

On Fri, Jul 24, 2026 at 04:29:08PM +0100, John Garry wrote:
> On 7/24/26 13:45, Keith Busch wrote:
> > Can we just thread through partitions for the bio's block_device instead
> > of assuming part0? I know the hidden path devices skip partition
> > scanning, but maybe if we let it happen, then those will have the same
> > partition setup as the head gendisk. Then we can go right to the
> > disk->part_tbl for what we provide to bio_set_dev() for both submission
> > and failover, and everything should work out from there.
> 
> I guess that we would just scan through the per-path gendisk->part_tbl and
> match somehow to lookup the partition. Maybe vs start address of
> bio->bi_bdev. Or is there a simpler (and quicker) way?
> 

This is the idea, assuming we can get the partition tables of the head
and path disks to be aligned:

---
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -543,7 +543,10 @@ static void nvme_ns_head_submit_bio(struct bio *bio)
        srcu_idx = srcu_read_lock(&head->srcu);
        ns = nvme_find_path(head);
        if (likely(ns)) {
-               bio_set_dev(bio, ns->disk->part0);
+               struct block_device *bdev;
+
+               bdev = xa_load(&ns->disk->part_tbl, bdev_partno(bio->bi_bdev));
+               bio_set_dev(bio, bdev);
                /*
                 * Use BIO_REMAPPED to skip bio_check_eod() when this bio
                 * enters submit_bio_noacct() for the per-path device. The EOD
--

Re: [PATCH RFC] nvme-multipath: fix diskstats for partitions

[John Garry]

On 7/24/26 16:37, Keith Busch wrote:
> On Fri, Jul 24, 2026 at 04:29:08PM +0100, John Garry wrote:
>> On 7/24/26 13:45, Keith Busch wrote:
>>> Can we just thread through partitions for the bio's block_device instead
>>> of assuming part0? I know the hidden path devices skip partition
>>> scanning, but maybe if we let it happen, then those will have the same
>>> partition setup as the head gendisk. Then we can go right to the
>>> disk->part_tbl for what we provide to bio_set_dev() for both submission
>>> and failover, and everything should work out from there.
>>
>> I guess that we would just scan through the per-path gendisk->part_tbl and
>> match somehow to lookup the partition. Maybe vs start address of
>> bio->bi_bdev. Or is there a simpler (and quicker) way?
>>
> 
> This is the idea, assuming we can get the partition tables of the head
> and path disks to be aligned:
> 
> ---
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -543,7 +543,10 @@ static void nvme_ns_head_submit_bio(struct bio *bio)
>          srcu_idx = srcu_read_lock(&head->srcu);
>          ns = nvme_find_path(head);
>          if (likely(ns)) {
> -               bio_set_dev(bio, ns->disk->part0);
> +               struct block_device *bdev;
> +
> +               bdev = xa_load(&ns->disk->part_tbl, bdev_partno(bio->bi_bdev));
> +               bio_set_dev(bio, bdev);
>                  /*
>                   * Use BIO_REMAPPED to skip bio_check_eod() when this bio
>                   * enters submit_bio_noacct() for the per-path device. The EOD
> --

ok, got it.

And I think that we would need to do the reverse lookup in 
nvme_mpath_start_request() and nvme_mpath_end_request() to get the head 
disk partition, like:
bdev = xa_load(&ns->head->disk->part_tbl, bdev_partno(bio->bi_bdev));

I suppose that the tricky part now would be have the per-path disk scan 
run but keep those per-path disks hidden.

I experimented by stop setting GENHD_FL_HIDDEN for the per-path disk, 
and the diskstats look ok, FWIW.