[PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ibragimov Rinat]

I have a proposal about 4k-sector hard drives support in fs drivers.
Toshiba MK1231GAL is an 1.8" LIF IDE drive. It has 4096-byte physical
and 512-byte logical sectors [1]. Unlike others drives with Advanced
Format, it has not read-modify-write logic, so it can only operate
with 4k blocks. Fortunately some filesystems in linux (I tested jfs, 
reiserfs and btrfs) read and write data by 4096-byte blocks. But 
when I tried to mount ext4, it fails:

[ 2595.093595] ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
[ 2595.097302] ata2.00: BMDMA stat 0x5
[ 2595.100427] ata2.00: failed command: READ DMA
[ 2595.103191] ata2.00: cmd c8/00:02:02:c0:d1/00:00:00:00:00/ed tag 0 dma 1024 in
[ 2595.103196]          res 51/10:02:02:c0:d1/00:00:00:00:00/ed Emask 0x81 (invalid argument)
[ 2595.108796] ata2.00: status: { DRDY ERR }
[ 2595.111981] ata2.00: error: { IDNF }
[ 2595.128438] ata2.00: configured for UDMA/100
[ 2595.131964] ata2: EH complete


Before ext4 reads superblock it calls `sb_min_blocksize', remembers its value and
then use it. By default it returns 1024 (=EXT4_MIN_BLOCK_SIZE).
I propose to return physical block size instead of logical one. With such change 
filesystems that honor `sb_min_blocksize' return value are able to mount and operate.
(At least ext2/3/4, hfs and hfsplus.)
For 512-byte drives nothing changes.

Is such approach correct?


diff -ur linux-2.6.37-rc5-orig/fs/block_dev.c linux-2.6.37-rc5/fs/block_dev.c
--- linux-2.6.37-rc5-orig/fs/block_dev.c	2010-12-07 04:09:04.000000000 +0000
+++ linux-2.6.37-rc5/fs/block_dev.c	2010-12-15 20:11:19.311387630 +0000
@@ -121,7 +121,7 @@
 
 int sb_min_blocksize(struct super_block *sb, int size)
 {
-	int minsize = bdev_logical_block_size(sb->s_bdev);
+	int minsize = bdev_physical_block_size(sb->s_bdev);
 	if (size < minsize)
 		size = minsize;
 	return sb_set_blocksize(sb, size);


Please add linux-fsdevel@ to CC list, I'm subscribed to it only.

[1] http://storage.toshiba.eu/cms/en/hdd/multimedia/product_detail.jsp?productid=258

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Jeff Moyer]

Ibragimov Rinat <ibragimovrinat@mail.ru> writes:

> I have a proposal about 4k-sector hard drives support in fs drivers.
> Toshiba MK1231GAL is an 1.8" LIF IDE drive. It has 4096-byte physical
> and 512-byte logical sectors [1]. Unlike others drives with Advanced
> Format, it has not read-modify-write logic, so it can only operate
> with 4k blocks. Fortunately some filesystems in linux (I tested jfs, 

Then it should report a logical block size of 4k.  You have a drive with
broken firmware.

Cheers,
Jeff

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Martin K. Petersen]

>>>>> "Jeff" == Jeff Moyer <jmoyer@redhat.com> writes:

>> I have a proposal about 4k-sector hard drives support in fs drivers.
>> Toshiba MK1231GAL is an 1.8" LIF IDE drive. It has 4096-byte physical
>> and 512-byte logical sectors [1]. Unlike others drives with Advanced
>> Format, it has not read-modify-write logic, so it can only operate
>> with 4k blocks. Fortunately some filesystems in linux (I tested jfs,

Jeff> Then it should report a logical block size of 4k.  You have a
Jeff> drive with broken firmware.

Several vendors are contemplating something similar.

I.e. keep the 512-byte addressing to avoid implementing support for 4KB
logical blocks in the OS ATA stack. But the drive will reject I/Os that
are not naturally aligned multiples of 4KB.

I agree it's totally broken. But I suspect we'll be seeing more of
these... :(

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ted Ts'o]

On Mon, Feb 28, 2011 at 09:58:55AM -0500, Martin K. Petersen wrote:
> >>>>> "Jeff" == Jeff Moyer <jmoyer@redhat.com> writes:
> 
> >> I have a proposal about 4k-sector hard drives support in fs drivers.
> >> Toshiba MK1231GAL is an 1.8" LIF IDE drive. It has 4096-byte physical
> >> and 512-byte logical sectors [1]. Unlike others drives with Advanced
> >> Format, it has not read-modify-write logic, so it can only operate
> >> with 4k blocks. Fortunately some filesystems in linux (I tested jfs,
> 
> Jeff> Then it should report a logical block size of 4k.  You have a
> Jeff> drive with broken firmware.
> 
> Several vendors are contemplating something similar.
> 
> I.e. keep the 512-byte addressing to avoid implementing support for 4KB
> logical blocks in the OS ATA stack. But the drive will reject I/Os that
> are not naturally aligned multiples of 4KB.

It's fine for them to do this; but if they do, they should be
reporting a logical block size of 4k, yes?  That's the definition of
what logical block size means....

Ibragimov, the problem with your patch is that it means that users who
had previously created a 1k block file system on an advanced format
disk with 4k physical sectors but 512 byte logical sectors would no
longer be able to mount that file system.  Yes, it was stupid for them
to do so, and they are deserving of plenty of mocking.  But
technically, it is something that should work, even if it's not
advisable from a performance point of view.

Maybe we can create a blacklist of broken drives that report a logical
blocksize of 512 but which really have a logical block size of 4k?

						- Ted

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Martin K. Petersen]

>>>>> "Ted" == Ted Ts'o <tytso@mit.edu> writes:

>> I.e. keep the 512-byte addressing to avoid implementing support for
>> 4KB logical blocks in the OS ATA stack. But the drive will reject
>> I/Os that are not naturally aligned multiples of 4KB.

Ted> It's fine for them to do this; but if they do, they should be
Ted> reporting a logical block size of 4k, yes?  That's the definition
Ted> of what logical block size means....

Kind of, yes. But that involves teaching every ATA driver in $LEGACY_OS
about 4KB logical blocks. There is little value is switching to 4KB
logical blocks in the first place. So the claim is that it is easier to
stick with 512-byte addressing, leave the I/O stack intact, and require
the filesystem to always issue aligned units of 4KB.

That saves the drive vendors from implementing RMW logic and puts the
burden on us. Perfect deal.


Ted> Maybe we can create a blacklist of broken drives that report a
Ted> logical blocksize of 512 but which really have a logical block size
Ted> of 4k?

It's not that easy. The logical block size is the unit used when we fill
out the commands to send to the device. Using 4KB addressing and issuing
block lengths in multiples of 4KB is not what this device wants.

*sigh*

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ted Ts'o]

On Mon, Feb 28, 2011 at 03:19:21PM -0500, Martin K. Petersen wrote:
> Kind of, yes. But that involves teaching every ATA driver in $LEGACY_OS
> about 4KB logical blocks. There is little value is switching to 4KB
> logical blocks in the first place. So the claim is that it is easier to
> stick with 512-byte addressing, leave the I/O stack intact, and require
> the filesystem to always issue aligned units of 4KB.
> 
> That saves the drive vendors from implementing RMW logic and puts the
> burden on us. Perfect deal.

So it there a reliable way to detect that disk drives that don't have
the Read-Modify-Write logic?  If that can be reflected up the I/O
layer I can have mke2fs enforce that restriction, which currently we
only enforce if the logical blocksize is 4k.

						- Ted

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Jeff Moyer]

"Ted Ts'o" <tytso@mit.edu> writes:

> On Mon, Feb 28, 2011 at 03:19:21PM -0500, Martin K. Petersen wrote:
>> Kind of, yes. But that involves teaching every ATA driver in $LEGACY_OS
>> about 4KB logical blocks. There is little value is switching to 4KB
>> logical blocks in the first place. So the claim is that it is easier to
>> stick with 512-byte addressing, leave the I/O stack intact, and require
>> the filesystem to always issue aligned units of 4KB.
>> 
>> That saves the drive vendors from implementing RMW logic and puts the
>> burden on us. Perfect deal.
>
> So it there a reliable way to detect that disk drives that don't have
> the Read-Modify-Write logic?  If that can be reflected up the I/O
> layer I can have mke2fs enforce that restriction, which currently we
> only enforce if the logical blocksize is 4k.

That's not enough, I'm afraid.  How do you deal with O_DIRECT, for
example?

Cheers,
Jeff

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ted Ts'o]

On Mon, Feb 28, 2011 at 04:10:44PM -0500, Jeff Moyer wrote:
> > So it there a reliable way to detect that disk drives that don't have
> > the Read-Modify-Write logic?  If that can be reflected up the I/O
> > layer I can have mke2fs enforce that restriction, which currently we
> > only enforce if the logical blocksize is 4k.
> 
> That's not enough, I'm afraid.  How do you deal with O_DIRECT, for
> example?

The drives do the read/modify/write, so O_DIRECT works fine from the
OS's perspective.  There is some risk of torn pages if you get a crash
at the wrong moment, but that's true whenver you have a misaligned
partition.  (At least for the drives that do the RMW in the disk
firmware.)

					-	 Ted

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Jeff Moyer]

"Ted Ts'o" <tytso@mit.edu> writes:

> On Mon, Feb 28, 2011 at 04:10:44PM -0500, Jeff Moyer wrote:
>> > So it there a reliable way to detect that disk drives that don't have
>> > the Read-Modify-Write logic?  If that can be reflected up the I/O
>> > layer I can have mke2fs enforce that restriction, which currently we
>> > only enforce if the logical blocksize is 4k.
>> 
>> That's not enough, I'm afraid.  How do you deal with O_DIRECT, for
>> example?
>
> The drives do the read/modify/write, so O_DIRECT works fine from the
> OS's perspective.  There is some risk of torn pages if you get a crash
> at the wrong moment, but that's true whenver you have a misaligned
> partition.  (At least for the drives that do the RMW in the disk
> firmware.)

Sorry, I thought we were discussing devices that didn't do
read-modify-write:

> >> Unlike others drives with Advanced Format, it has not
> >> read-modify-write logic, so it can only operate with 4k
> >> blocks.

Also, mkp noted that the theory of operation was to issue aligned units
of 4k, but using 512 byte addressing.  So, unless you want to keep a
blacklist AND write a remapping layer, I'd say don't buy those drives.

Cheers,
Jeff

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Martin K. Petersen]

>>>>> "Jeff" == Jeff Moyer <jmoyer@redhat.com> writes:

Jeff> Also, mkp noted that the theory of operation was to issue aligned
Jeff> units of 4k, but using 512 byte addressing.

Yeah, we'd have to expose a 4KB lbs to the filesystem and then do a
translation to 512-byte units when filling out the command.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[James Bottomley]

On Mon, 2011-02-28 at 16:05 -0500, Ted Ts'o wrote:
> On Mon, Feb 28, 2011 at 03:19:21PM -0500, Martin K. Petersen wrote:
> > Kind of, yes. But that involves teaching every ATA driver in $LEGACY_OS
> > about 4KB logical blocks. There is little value is switching to 4KB
> > logical blocks in the first place. So the claim is that it is easier to
> > stick with 512-byte addressing, leave the I/O stack intact, and require
> > the filesystem to always issue aligned units of 4KB.
> > 
> > That saves the drive vendors from implementing RMW logic and puts the
> > burden on us. Perfect deal.
> 
> So it there a reliable way to detect that disk drives that don't have
> the Read-Modify-Write logic?  If that can be reflected up the I/O
> layer I can have mke2fs enforce that restriction, which currently we
> only enforce if the logical blocksize is 4k.

Well, we could blacklist it to force the logical block size up to 4k
based on identification by the inquiry strings (that, of course,
involves finding all the inquiry strings).  It's just that we're
practically out of blacklist bits ... it's just I thought we'd finally
got all manufacturers on the same page as standards compliance ... doing
something like this would send us back to the dark ages again.

Could we just have a campaign to send drives that fail like this back?
That might actually get the message across to the manufacturers .. and
it could arguably be covered under the warranty.

James

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ted Ts'o]

On Mon, Feb 28, 2011 at 03:20:51PM -0600, James Bottomley wrote:
> Could we just have a campaign to send drives that fail like this back?
> That might actually get the message across to the manufacturers .. and
> it could arguably be covered under the warranty.

I bet the manufacturers will say the drive works just fine --- as long
as you use the Windows drivers that they thoughtfully provided with
the HDD.  :-)

						- Ted

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Martin K. Petersen]

>>>>> "Ted" == Ted Ts'o <tytso@mit.edu> writes:

>> That saves the drive vendors from implementing RMW logic and puts the
>> burden on us. Perfect deal.

Ted> So it there a reliable way to detect that disk drives that don't
Ted> have the Read-Modify-Write logic?  If that can be reflected up the
Ted> I/O layer I can have mke2fs enforce that restriction, which
Ted> currently we only enforce if the logical blocksize is 4k.

Not really :(

I am aware of a few devices that report 512/512 but actually reject
anything that's not an aligned multiple of 4KB. At least this Toshiba
drive reports the physical block size correctly.

The problem with detection is that we typically just get an I/O
error. No explanation. The standards body dogma is that a storage device
can abort any command at any time for any reason.

Maybe a smoke test for mkfs would be to attempt to write a single
logical block. If that gives an -EIO then attempt the physical block
size instead. That assumes that the partition is aligned, but that
should be taken care of with the latest tooling.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-writesupport

[Ibragimov Rinat]

> Ibragimov, the problem with your patch is that it means that users who
> had previously created a 1k block file system on an advanced format
> disk with 4k physical sectors but 512 byte logical sectors would no
> longer be able to mount that file system.  Yes, it was stupid for them
> to do so, and they are deserving of plenty of mocking.  But
> technically, it is something that should work, even if it's not
> advisable from a performance point of view.

I haven't thought about such case. It renders the patch useless.

> Maybe we can create a blacklist of broken drives that report a logical
> blocksize of 512 but which really have a logical block size of 4k?

That would be great, but then one must recalculate LBA and sector count
on every read and write request to hardware. I tried first to implement
a kind of blacklist and simply changed ATA ID (to report 4k logical),
but realized it was a bad idea. 

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-writesupport

[Ibragimov Rinat]

> Ibragimov, the problem with your patch is that it means that users who
> had previously created a 1k block file system on an advanced format
> disk with 4k physical sectors but 512 byte logical sectors would no
> longer be able to mount that file system. Yes, it was stupid for them
> to do so, and they are deserving of plenty of mocking. But
> technically, it is something that should work, even if it's not
> advisable from a performance point of view.

I haven't found anyone with AF disk who agreed to run test of 1k-block
fs mount with modified kernel, so I tried to emulate such disk on
available hardware. AF disk with 4kb sector reports its size in 
106th word of ATA ID. This is the only difference, except write 
performace. So if I change 106th word, I will get emulated AF disk.
As you can see below, it reports 4096-byte physical block size.
ext4 with 1k block size mounts and operates on such disk without
any notable problem.


diff -ur linux-2.6.38-rc4-orig/drivers/ata/libata-core.c linux-2.6.38-rc4/drivers/ata/libata-core.c
--- linux-2.6.38-rc4-orig/drivers/ata/libata-core.c	2011-02-08 00:03:55.000000000 +0000
+++ linux-2.6.38-rc4/drivers/ata/libata-core.c	2011-03-01 18:01:31.859956475 +0000
@@ -1907,6 +1907,8 @@
 	else
 		err_mask = ata_do_dev_read_id(dev, &tf, id);
 
+	id[106] = 0x6003; // 4kb imitation
+
 	if (err_mask) {
 		if (err_mask & AC_ERR_NODEV_HINT) {
 			ata_dev_printk(dev, KERN_DEBUG,
diff -ur linux-2.6.38-rc4-orig/fs/block_dev.c linux-2.6.38-rc4/fs/block_dev.c
--- linux-2.6.38-rc4-orig/fs/block_dev.c	2011-02-08 00:03:55.000000000 +0000
+++ linux-2.6.38-rc4/fs/block_dev.c	2011-03-01 18:43:50.126957299 +0000
@@ -121,7 +121,7 @@
 
 int sb_min_blocksize(struct super_block *sb, int size)
 {
-	int minsize = bdev_logical_block_size(sb->s_bdev);
+	int minsize = bdev_physical_block_size(sb->s_bdev);
 	if (size < minsize)
 		size = minsize;
 	return sb_set_blocksize(sb, size);

===============

Script started on Срд 02 Мар 2011 00:45:22
root@debv:~# dmesg | grep sdb
[    3.778499] sd 3:0:0:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
[    3.778505] sd 3:0:0:0: [sdb] 4096-byte physical blocks
[    3.778647] sd 3:0:0:0: [sdb] Write Protect is off
[    3.778653] sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    3.778712] sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.779394]  sdb: sdb1
[    3.795441] sd 3:0:0:0: [sdb] Attached SCSI disk
[   62.307185] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
root@debv:~# tune2fs -l /dev/sdb1 | grep "Block size"
Block size:               1024
root@debv:~# umount /mnt
root@debv:~# mount /dev/sdb1 /mnt
root@debv:~# ls /mnt
archives  lost+found
root@debv:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             7.4G  3.4G  3.7G  49% /
tmpfs                 506M     0  506M   0% /lib/init/rw
udev                  501M  156K  500M   1% /dev
tmpfs                 506M     0  506M   0% /dev/shm
/dev/sdb1            1007M  480M  477M  51% /mnt
root@debv:~# exit

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-writesupport

[Ibragimov Rinat]

> > Ibragimov, the problem with your patch is that it means that users who
> > had previously created a 1k block file system on an advanced format
> > disk with 4k physical sectors but 512 byte logical sectors would no
> > longer be able to mount that file system. Yes, it was stupid for them
> > to do so, and they are deserving of plenty of mocking. But
> > technically, it is something that should work, even if it's not
> > advisable from a performance point of view.
> 
> I haven't found anyone with AF disk who agreed to run test of 1k-block
> fs mount with modified kernel, so I tried to emulate such disk on
> available hardware. AF disk with 4kb sector reports its size in 
> 106th word of ATA ID. This is the only difference, except write 
> performace. So if I change 106th word, I will get emulated AF disk.
> As you can see below, it reports 4096-byte physical block size.
> ext4 with 1k block size mounts and operates on such disk without
> any notable problem.

Ping? The problem you mentioned (with 1k blocks on AF drive) do not
exist, I think.
Value returned by sb_min_blocksize used only until superblock is read 
from disk, then driver recalculates 'blocksize' from on-disk sb. So that
change can't interfere with other code.
Please correct me if I'm wrong.

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Ibragimov Rinat]

 
> Then it should report a logical block size of 4k.  You have a drive with
> broken firmware.
> 

It is unlikely Toshiba will release a correct firmware. That drive was
indended for use in external hdd case. 512-byte sector emulation 
implemented in USB-IDE controller.

Re: [PATCH][RFC]ext4 on 4k-sector drives without read-modify-write support

[Mike]

Ibragimov Rinat <ibragimovrinat <at> mail.ru> writes:

> 
> 
> > Then it should report a logical block size of 4k.  You have a drive with
> > broken firmware.
> > 
> 
> It is unlikely Toshiba will release a correct firmware. That drive was
> indended for use in external hdd case. 512-byte sector emulation 
> implemented in USB-IDE controller.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo <at> vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 


Hi, has anyone tried with the utility of the site TOSCHIBA?

http://storage.toshiba.com/storage-services-support/warranty-support/software-
utilities

greetings