test sfdisk, optimal_iosize

test sfdisk, optimal_iosize

[Ruediger Meier]

Hi,

I've got some test failures like

--- /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/expected/sfdisk/dos-activate
+++ /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/output/sfdisk/dos-activate
@@ -6,7 +6,7 @@
 Disk <removed>: 50 MiB, 52428800 bytes, 102400 sectors
 Units: sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 4096 bytes
-I/O size (minimum/optimal): 4096 bytes / 32768 bytes
+I/O size (minimum/optimal): 4096 bytes / 64 bytes
 Disklabel type: dos
 Disk identifier: <removed>


optimal < minimum ... is this a bug?

Where does this optimal iosize come from, scsi_debug module?
Is there a commandline tool to get this iosize, maybe blkid?

cu,
Rudi

Re: test sfdisk, optimal_iosize

[Karel Zak]

On Fri, Jan 29, 2016 at 11:31:28AM +0100, Ruediger Meier wrote:
> Hi,
> 
> I've got some test failures like
> 
> --- /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/expected/sfdisk/dos-activate
> +++ /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/output/sfdisk/dos-activate
> @@ -6,7 +6,7 @@
>  Disk <removed>: 50 MiB, 52428800 bytes, 102400 sectors
>  Units: sectors of 1 * 512 = 512 bytes
>  Sector size (logical/physical): 512 bytes / 4096 bytes
> -I/O size (minimum/optimal): 4096 bytes / 32768 bytes
> +I/O size (minimum/optimal): 4096 bytes / 64 bytes

Very very strange. It seems like the number is in sectors (64 * 512 =
32768) rather than in bytes.

>  Disklabel type: dos
>  Disk identifier: <removed>
> 
> 
> optimal < minimum ... is this a bug?
> 
> Where does this optimal iosize come from, scsi_debug module?
> Is there a commandline tool to get this iosize, maybe blkid?

 cat /sys/block/sda/queue/{optimal,minimum}_io_size
 cat /sys/block/sda/queue/{physical,logical}_block_size

The optimal IO size is often 0 (in /sys) in this case libfdisk
fallback to minimal IO size.

Try 
    LIBFDISK_DEBUG=all fdisk -l /dev/sda

you will see something like:

8694: libfdisk:      CXT: [0x560e0bba9050]: /dev/sda: discovering topology...
8694: libfdisk:      CXT: [0x560e0bba9050]: initialize libblkid prober
8694: libfdisk:      CXT: [0x560e0bba9050]: result: log/phy sector size: 512/512
8694: libfdisk:      CXT: [0x560e0bba9050]: result: fdisk/optimal/minimal io: 512/0/512
8694: libfdisk:      CXT: [0x560e0bba9050]: /dev/sda: discovering geometry...
8694: libfdisk:      CXT: [0x560e0bba9050]: total sectors: 468862128 (ioctl=468862128)
8694: libfdisk:      CXT: [0x560e0bba9050]: result: C/H/S: 29185/255/63

  Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: test sfdisk, optimal_iosize

[Karel Zak]

On Fri, Jan 29, 2016 at 12:31:10PM +0100, Karel Zak wrote:
> > Where does this optimal iosize come from, scsi_debug module?
> > Is there a commandline tool to get this iosize, maybe blkid?
> 
>  cat /sys/block/sda/queue/{optimal,minimum}_io_size
>  cat /sys/block/sda/queue/{physical,logical}_block_size

 And also by ioctl:

   blockdev --getiomin --getioopt /dev/sdc

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: test sfdisk, optimal_iosize

[Ruediger Meier]

On Friday 29 January 2016, Karel Zak wrote:
> On Fri, Jan 29, 2016 at 11:31:28AM +0100, Ruediger Meier wrote:
> > Hi,
> >
> > I've got some test failures like
> >
> > ---
> > /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/expe
> >cted/sfdisk/dos-activate +++
> > /home/abuild/rpmbuild/BUILD/util-linux-2.27.git247.4c01c/tests/outp
> >ut/sfdisk/dos-activate @@ -6,7 +6,7 @@
> >  Disk <removed>: 50 MiB, 52428800 bytes, 102400 sectors
> >  Units: sectors of 1 * 512 = 512 bytes
> >  Sector size (logical/physical): 512 bytes / 4096 bytes
> > -I/O size (minimum/optimal): 4096 bytes / 32768 bytes
> > +I/O size (minimum/optimal): 4096 bytes / 64 bytes
>
> Very very strange. It seems like the number is in sectors (64 * 512 =
> 32768) rather than in bytes.
>
> >  Disklabel type: dos
> >  Disk identifier: <removed>
> >
> >
> > optimal < minimum ... is this a bug?
> >
> > Where does this optimal iosize come from, scsi_debug module?
> > Is there a commandline tool to get this iosize, maybe blkid?
>
>  cat /sys/block/sda/queue/{optimal,minimum}_io_size
>  cat /sys/block/sda/queue/{physical,logical}_block_size
>
> The optimal IO size is often 0 (in /sys) in this case libfdisk
> fallback to minimal IO size.
>
> Try
>     LIBFDISK_DEBUG=all fdisk -l /dev/sda

Thx.

I get this on the "broken systems":

## sector_size=512
$ modprobe -r scsi_debug
$ modprobe scsi_debug dev_size_mb=100 sector_size=512
$ devname=$(grep --with-filename scsi_debug /sys/block/*/device/model | awk -F '/' '{print $4}')
$ cat /sys/block/$devname/queue/{logical_block_size,physical_block_size,minimum_io_size,optimal_io_size}
512
512
512
64

## sector_size=4096
$ modprobe -r scsi_debug
$ modprobe scsi_debug dev_size_mb=100 sector_size=4096
$ devname=$(grep --with-filename scsi_debug /sys/block/*/device/model | awk -F '/' '{print $4}') 
$ cat /sys/block/$devname/queue/{logical_block_size,physical_block_size,minimum_io_size,optimal_io_size}
4096
4096
4096
512

These odd optimal iosizes only happen for "scsi_debug" devices (not the loop
nor md devices used in our test suite).

Reproduced on kernel 4.1.15 and 4.4.0.
Was correct on kernel 3.16.7 (and maybe even 4.1.13).

Maybe it's not only about the kernel version but also something about
other host specific settings.

Could somebody please reproduce this on kernel 4.4.0?

cu,
Rudi

Re: scsi_debug, iosize optimal < minimum (was: test sfdisk, optimal_iosize)

[Ruediger Meier]

Hi,

I've noticed a possible problem with scsi_debug devices where
optimal_io_size < minimum_io_size.

The problem seems reproducable with kernel 4.4.0 and also with
stable version 4.1.15. For stable tree the last known working
version is 4.1.13.

How to reproduce:

uname -mr
modprobe -r scsi_debug 
modprobe scsi_debug dev_size_mb=100 sector_size=512
udevadm settle
devname=$(grep --with-filename scsi_debug /sys/block/*/device/model | awk -F '/' '{print $4}')
cat /sys/block/$devname/queue/{minimum_io_size,optimal_io_size}
modprobe -r scsi_debug


## output kernel 4.1.13 (openSUSE Leap):
4.1.13-5-default x86_64
512
32768


## output kernel 4.1.15 (openSUSE Leap):
4.1.15-8-default x86_64
512
64

cu,
Rudi

Re: scsi_debug, iosize optimal < minimum

[Martin K. Petersen]

>>>>> "Ruediger" == Ruediger Meier <sweet_f_a@gmx.de> writes:

Ruediger> I've noticed a possible problem with scsi_debug devices where
Ruediger> optimal_io_size < minimum_io_size.

Ruediger> The problem seems reproducable with kernel 4.4.0 and also with
Ruediger> stable version 4.1.15. For stable tree the last known working
Ruediger> version is 4.1.13.

http://git.kernel.org/cgit/linux/kernel/git/mkp/scsi.git/commit/?h=4.5/scsi-fixes&id=d0eb20a863ba7dc1d3f4b841639671f134560be2

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: scsi_debug, iosize optimal < minimum

[Ruediger Meier]

On Monday 01 February 2016, Martin K. Petersen wrote:
> >>>>> "Ruediger" == Ruediger Meier <sweet_f_a@gmx.de> writes:
>
> Ruediger> I've noticed a possible problem with scsi_debug devices
> where Ruediger> optimal_io_size < minimum_io_size.
>
> Ruediger> The problem seems reproducable with kernel 4.4.0 and also
> with Ruediger> stable version 4.1.15. For stable tree the last known
> working Ruediger> version is 4.1.13.
>
> http://git.kernel.org/cgit/linux/kernel/git/mkp/scsi.git/commit/?h=4.
>5/scsi-fixes&id=d0eb20a863ba7dc1d3f4b841639671f134560be2


Ah thanks!

But I'm still wondering a bit. Using kernel 4.5-rc2 (where 9c1d9c20 and 
d0eb20a8 is included) I get

$ getconf PAGESIZE
4096
$ modprobe scsi_debug
$  cat /sys/block/$devname/queue/{minimum_io_size,optimal_io_size}
512
524288

In past (kernel<=4.1) optimal_io_size io size was 32768. Is this really 
correct now?

cu,
Rudi

Re: scsi_debug, iosize optimal < minimum

[Martin K. Petersen]

>>>>> "Ruediger" == Ruediger Meier <sweet_f_a@gmx.de> writes:

Ruediger> In past (kernel<=4.1) optimal_io_size io size was 32768. Is
Ruediger> this really correct now?

We recently bumped the scsi_debug optimal I/O size to something akin to
what real-world storage reports.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: scsi_debug, iosize optimal < minimum

[Ruediger Meier]

On Wednesday 03 February 2016, Martin K. Petersen wrote:
> >>>>> "Ruediger" == Ruediger Meier <sweet_f_a@gmx.de> writes:
>
> Ruediger> In past (kernel<=4.1) optimal_io_size io size was 32768. Is
> Ruediger> this really correct now?
>
> We recently bumped the scsi_debug optimal I/O size to something akin
> to what real-world storage reports.

Regarding these kernel changes I've send a pull request via github to 
fix the tests:
https://github.com/karelzak/util-linux/pull/272

cu,
Rudi