RAID0 & diskstats.

RAID0 & diskstats.

[Dave Jones]

Hi Neil,
   According to Documentation/iostats.txt, the 9th column of
/proc/diskstats (and its modern replacement in sysfs) should go to 0
as IO completes.

I assembled a RAID0 stripe using two SSD's, and saw this..

# mdadm --assemble /dev/md0
mdadm: /dev/md0 has been started with 2 drives.
# cat /sys/block/md0/stat
     167        0     5656        0        5        0     4096        0     172     3408   582825
# cat /sys/block/md0/stat
     167        0     5656        0        5        0     4096        0     172   231469 39809317

The 10th & 11th fields constantly increase, as field 9 remains non-zero.
If I mount and umount a filesystem on that volume, it works as expected,
but the 9th 'IOs inflight' field continues to rise and never decreases even
though the IO has obviously completed.

# umount /mnt/ssd
# cat /sys/block/md0/stat
     167        0     5656        0        9        0     4225        0     176   571384 98278615

The underlying disks have their respective stats entries behaving as
expected, it only seems to affect the upper md layer.

Some missing accounting somewhere in md ?

(Only tested on 4.0rc2 so far, and only on RAID0)

	Dave

Re: RAID0 & diskstats.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1886 bytes --]

On Wed, 4 Mar 2015 16:09:04 -0500 Dave Jones <davej@codemonkey.org.uk> wrote:

> Hi Neil,
>    According to Documentation/iostats.txt, the 9th column of
> /proc/diskstats (and its modern replacement in sysfs) should go to 0
> as IO completes.
> 
> I assembled a RAID0 stripe using two SSD's, and saw this..
> 
> # mdadm --assemble /dev/md0
> mdadm: /dev/md0 has been started with 2 drives.
> # cat /sys/block/md0/stat
>      167        0     5656        0        5        0     4096        0     172     3408   582825
> # cat /sys/block/md0/stat
>      167        0     5656        0        5        0     4096        0     172   231469 39809317
> 
> The 10th & 11th fields constantly increase, as field 9 remains non-zero.
> If I mount and umount a filesystem on that volume, it works as expected,
> but the 9th 'IOs inflight' field continues to rise and never decreases even
> though the IO has obviously completed.
> 
> # umount /mnt/ssd
> # cat /sys/block/md0/stat
>      167        0     5656        0        9        0     4225        0     176   571384 98278615
> 
> The underlying disks have their respective stats entries behaving as
> expected, it only seems to affect the upper md layer.
> 
> Some missing accounting somewhere in md ?
> 
> (Only tested on 4.0rc2 so far, and only on RAID0)
> 
> 	Dave

blockdev stats often aren't really a good match for md/raid...

"in_flight" assumes a queue, and raid0 doesn't have one.  It just redirects
each request to the relevant device and lets the device handle it.

The only useful thing we could do here is make that value always zero. 


I guess I need to add a "generic_end_io_acct()" call to md.c somewhere.
For raid1/5 there is probably somewhere sensible to put it.
For raid0/linear, it probably goes immediately after generic_start_io_acct().

Patches welcome :-)

NeilBrown

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]