Re: optimal io size / custom alignment

Re: optimal io size / custom alignment

[Tom Yan]

I heard about that it matters for RAID but since I don't really know
about RAID so I can't comment.

I do wonder whether the scsi disk driver should derive minimum/optimal
i/o size from VPD at all then. It might still be "tolerable" if it's
the limit of WRITE SAME(10), but definitely not if it's that of WRITE
SAME (16):

[tom@localhost ~]$ sudo fdisk /dev/sdc

Welcome to fdisk (util-linux 2.26.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xccb261a9.

Command (m for help): p
Disk /dev/sdc: 29.2 GiB, 31376707072 bytes, 61282631 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4294966784 bytes
Disklabel type: dos
Disk identifier: 0xccb261a9

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (8388607-61282630, default 8388607):
Last sector, +sectors or +size{K,M,G,T,P} (8388607-61282630, default 61282630):

Created a new partition 1 of type 'Linux' and of size 25.2 GiB.

On 16 June 2015 at 17:43, Karel Zak <kzak-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Tue, Jun 16, 2015 at 01:20:37PM +0800, Tom Yan wrote:
>> The thing is, why any io/transfer size/length should be considered
>> when it comes to partition alignment? From what I understand,
>> partition alignment is only to make sure partition starts at physical
>> boundaries of the disk because of the mismatch between logicial sector
>> (512 bytes) and physical sectors (4096 bytes) or pages/erase blocks of
>> SSDs.
>
> It's more complicated, the I/O limits are the most important for RAIDs
> where optimal I/O size is usually stripe size and you want to use it
> for partitions alignment for better performance (if you align to
> sector size then read/write on RAID maybe performed on more disks on
> unaligned partitions). And it's not only fdisk who cares, it's also
> important for mkfs.<type> (for example XFS align according to I/O limits).
>
> And because all this is mess and sometimes HW does not provide
> relevant information and because people use dd(1) to copy partition
> tables we have decided to use 1MiB granularity if possible. If 1MiB is
> useless then we use optimal_io_size, if undefined then minimal_io_size
> and if undefined then sector_size.
>
> http://people.redhat.com/msnitzer/docs/io-limits.txt
>
>
> Unfortunately the current code does not check if optimal_io_size makes
> sense, so thing like 33553920 for 4k device is blindly accepted ;-(
>
>     Karel
>
>
> --
>  Karel Zak  <kzak-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>  http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: optimal io size / custom alignment

[Tom Yan]

On 17 June 2015 at 01:08, Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
> The two values have nothing to do with each other. They just happen to
> be the same in your case (65535 is the maximum block count for the WRITE
> SAME(10) command).
>
> Your device sets the transfer length granularity to 1 logical block and
> the optimal transfer length to 65535 logical blocks. If it then reports
> a 4096-byte physical block size in response to READ CAPACITY(16) then
> it's clearly on crack.
>
> There's only so much we can do about devices that report garbage.

All drives I have are flash drives so none of them reports 4k physical
sectors. But it does seems possible in the case I linked. The thing is
these VPDs/transfer lengths are probably provided by the USB to
ATA(/SCSI?) bridges. I can't judge if they are wrong to set the
lengths that way but it seem to be a common practice. I have two USB
devices provide the SBC-2 (Block limit VPD), one is a SanDisk Extreme
USB (SDCZ80), another an Intel X25-M Gen1 on an ASMedia SATA adapter,
and both of them set the Optimal transfer length. The usb-storage
driver does not read vpd so it won't be a thing, but the the uas
driver does.

> Also, the kernel only reports things. It is up to Karel to decide
> whether to sanity check the values before he uses them.

I just feel like the kernel shouldn't bind values from totally
different source (raid stripe vs vpd limit) to the same variable. I
don't know if what else would make use of this variable but by only
considering the fdisk case, it seems the scsi disk driver should be
the one who should stop binding.

> The best fix, of course, is to complain to the manufacturer of your
> broken widget and hope for a firmware upgrade.

This is simply too idealistic especially when it seems that this issue
mostly happens on USB bridges. I am not even sure if the SCSI
standards has anything to say about this practice.

> Failing that, adjust your partitions manually.

Yeah that's why I said fdisk should allow custom alignment.

On 17 June 2015 at 01:08, Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>>>>>> "Tom" == Tom Yan <tom.ty89-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> Tom> From the adapter/drive I have, it is the same as the "Maximum
> Tom> transfer length" and they seem to be simply limits of SCSI "WRITE
> Tom> SAME (10/16)" command:
>
> The two values have nothing to do with each other. They just happen to
> be the same in your case (65535 is the maximum block count for the WRITE
> SAME(10) command).
>
> Tom> [tom@localhost ~]$ sudo sg_inq -p 0xb0 /dev/sdb VPD INQUIRY: Block
> Tom> limits page (SBC) Maximum compare and write length: 0 blocks
> Tom> Optimal transfer length granularity: 1 blocks Maximum transfer
> Tom> length: 65535 blocks Optimal transfer length: 65535 blocks
>
> Your device sets the transfer length granularity to 1 logical block and
> the optimal transfer length to 65535 logical blocks. If it then reports
> a 4096-byte physical block size in response to READ CAPACITY(16) then
> it's clearly on crack.
>
> There's only so much we can do about devices that report garbage.
>
> Also, the kernel only reports things. It is up to Karel to decide
> whether to sanity check the values before he uses them.
>
> I would probably err on the side of trusting the physical block size
> reporting more than anything seeded from the Block Limits VPD. And in
> this case, assuming the alignment offset is reported to be 0, I guess
> one could entertain aligning to the nearest 4K boundary. But on the
> other hand it'll quickly get hairy to have to maintain this kind of
> heuristics.
>
> The best fix, of course, is to complain to the manufacturer of your
> broken widget and hope for a firmware upgrade. Failing that, adjust your
> partitions manually.
>
> Tom> The thing is, why any io/transfer size/length should be considered
> Tom> when it comes to partition alignment? From what I understand,
> Tom> partition alignment is only to make sure partition starts at
> Tom> physical boundaries of the disk because of the mismatch between
> Tom> logicial sector (512 bytes) and physical sectors (4096 bytes) or
> Tom> pages/erase blocks of SSDs.
>
> For RAID it makes a big difference to ensure the partition is aligned on
> a stripe boundary.
>
> --
> Martin K. Petersen      Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: optimal io size / custom alignment

[Martin K. Petersen]

>>>>> "Tom" == Tom Yan <tom.ty89@gmail.com> writes:

Tom> All drives I have are flash drives so none of them reports 4k
Tom> physical sectors.

There are plenty of SSDs that report 4K physical sectors, fwiw.

Tom> The usb-storage driver does not read vpd so it won't be a thing,
Tom> but the the uas driver does.

We gave up on USB-SATA bridges long ago. Their designers appear to have
a pretty comprehensive misunderstanding of both the ATA and SCSI
protocols.

We had higher hopes for UAS since it provided a clean slate. So far,
however, the results are equally discouraging.

Tom> I just feel like the kernel shouldn't bind values from totally
Tom> different source (raid stripe vs vpd limit) to the same variable.

RAID devices communicate the stripe width through the Block Limits VPD.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: optimal io size / custom alignment

[Tom Yan]

On 17 June 2015 at 05:28, Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
> There are plenty of SSDs that report 4K physical sectors, fwiw.

Oh didn't know that. Wonder if it's yet another garbage info. Though
4k is often a nice value to make use of.

> We gave up on USB-SATA bridges long ago. Their designers appear to have
> a pretty comprehensive misunderstanding of both the ATA and SCSI
> protocols.

Aren't there tons of thumb drives make use of it anyway?

> Tom> I just feel like the kernel shouldn't bind values from totally
> Tom> different source (raid stripe vs vpd limit) to the same variable.
>
> RAID devices communicate the stripe width through the Block Limits VPD.

No I put it in the wrong way. What I meant was "sd vs md". For
example, couldn't the scsi disk driver bind the value it reads from
the VPD to another variable instead of "optimal i/o size", so that
this value would be exclusively for RAID (and other virtual devices)?
Is it even necessary for it to report? Because it seems only to make
this variable ambiguous.

If it HAS TO BE ambiguous, I see no reason why fdisk should use it to
derive the alignment. It should simply let the users do their
judgement and provide a way for them to adjust manually.
--
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: optimal io size / custom alignment

[Martin K. Petersen]

>>>>> "Tom" == Tom Yan <tom.ty89-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

Tom> No I put it in the wrong way. What I meant was "sd vs md". For
Tom> example, couldn't the scsi disk driver bind the value it reads from
Tom> the VPD to another variable instead of "optimal i/o size", so that
Tom> this value would be exclusively for RAID (and other virtual
Tom> devices)?

Who says that RAID is a virtual device? Hardware RAID controllers as
well as SAS, iSCSI and Fibre Channel disk arrays all use the Block
Limits VPD to communicate their preferred I/O size and alignment to
us. As do enterprise disk drives.

We deal with broken devices by blacklisting them. I suggest you try to
find a way we can reliably identify your UAS devices. If there is a
common pattern, we can entertain adding a workaround.

-- 
Martin K. Petersen	Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: optimal io size / custom alignment

[Tom Yan]

I was not saying RAIDs are virtual devices. I just mentioned it
because I saw things like virtio-blk or zram use blk_queue_io_opt().

I know they all use VPDs, but the main point is whether those hardware
RAIDs or so are handled by sd_mod, and whether those "transfer
lengths" info are still important when it's just a simple drive. To me
they look like to be of different nature. That's why I think it's
inappropraite that they use the same "variable" / "file" to report
because that makes tools like fdisk have trouble determining when does
those values really matters.

In fact, (maybe I am just unlucky :P) VPDs of all my devices are to
some extent broken. I just found out today my Intel 530 SSD connecting
directly to SATA also reports totally garbage values for TRIM : (

To be honest the UAS thing doesn't really affect me a lot, I mostly
use gdisk now (which doesn't care about i/o size AFAIK). I can also
disable uas with the quirk so that VPDs are skipped when I really need
fdisk for msdos/mbr. It's just I think that it kind of reveal a
problem that has to be dealt with sooner or later, though you can
optimistically think that vendors would do better on VPDs in the
future.

On 19 June 2015 at 05:01, Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>>>>>> "Tom" == Tom Yan <tom.ty89-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> Tom> No I put it in the wrong way. What I meant was "sd vs md". For
> Tom> example, couldn't the scsi disk driver bind the value it reads from
> Tom> the VPD to another variable instead of "optimal i/o size", so that
> Tom> this value would be exclusively for RAID (and other virtual
> Tom> devices)?
>
> Who says that RAID is a virtual device? Hardware RAID controllers as
> well as SAS, iSCSI and Fibre Channel disk arrays all use the Block
> Limits VPD to communicate their preferred I/O size and alignment to
> us. As do enterprise disk drives.
>
> We deal with broken devices by blacklisting them. I suggest you try to
> find a way we can reliably identify your UAS devices. If there is a
> common pattern, we can entertain adding a workaround.
>
> --
> Martin K. Petersen      Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe util-linux" in

Re: optimal io size / custom alignment

[Martin K. Petersen]

>>>>> "Tom" == Tom Yan <tom.ty89-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

Tom> I know they all use VPDs, but the main point is whether those
Tom> hardware RAIDs or so are handled by sd_mod, and whether those
Tom> "transfer lengths" info are still important when it's just a simple
Tom> drive. To me they look like to be of different nature.

We don't know whether a discovered device is "a simple drive".

And once again: The whole point of the queue limit is to have an common
abstraction for all block devices.

-- 
Martin K. Petersen	Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe util-linux" in

Re: optimal io size / custom alignment

[Alan Stern]

On Sun, 21 Jun 2015, Tom Yan wrote:

> I was not saying RAIDs are virtual devices. I just mentioned it
> because I saw things like virtio-blk or zram use blk_queue_io_opt().
> 
> I know they all use VPDs, but the main point is whether those hardware
> RAIDs or so are handled by sd_mod, and whether those "transfer
> lengths" info are still important when it's just a simple drive. To me
> they look like to be of different nature. That's why I think it's
> inappropraite that they use the same "variable" / "file" to report
> because that makes tools like fdisk have trouble determining when does
> those values really matters.
> 
> In fact, (maybe I am just unlucky :P) VPDs of all my devices are to
> some extent broken. I just found out today my Intel 530 SSD connecting
> directly to SATA also reports totally garbage values for TRIM : (
> 
> To be honest the UAS thing doesn't really affect me a lot, I mostly
> use gdisk now (which doesn't care about i/o size AFAIK). I can also
> disable uas with the quirk so that VPDs are skipped when I really need
> fdisk for msdos/mbr. It's just I think that it kind of reveal a
> problem that has to be dealt with sooner or later, though you can
> optimistically think that vendors would do better on VPDs in the
> future.

Regardless of all these issues, it is clear that a lot of devices don't 
implement the VPD data correctly.  Therefore the information in the 
kernel will often be wrong.

And consequently, fdisk needs to offer the user an option to override 
the default partition-alignment setting.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe util-linux" in