[BUG] xfs.mkfs recognizes wrong disk size

[BUG] xfs.mkfs recognizes wrong disk size

[Xingbo Wu]

/dev/sdf is a 3TB hard drive.
It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
Checking for bad blocks in read-write mode
>From block 0 to 2930266583

However mkfs.xfs reports a larger block count:
`sudo mkfs.xfs /dev/sdf`:
meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=0, rmapbt=0
data     =                       bsize=4096   blocks=732566646, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=357698, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

It later leads to IO error:
`dmesg`:
[1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
[1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
[1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
("xlog_iodone") error 5 numblks 64
[1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
line 1200 of file fs/xfs/xfs_log.c.  Return address =
0xffffffffa024090c
[1807875.742381] XFS (sdf): Log I/O Error Detected.  Shutting down filesystem
[1807875.742416] XFS (sdf): xfs_log_force: error -5 returned.
[1807875.742426] XFS (sdf): Please umount the filesystem and rectify
the problem(s)
[1807881.868117] XFS (sdf): xfs_log_force: error -5 returned.
[1807910.701836] XFS (sdf): xfs_log_force: error -5 returned.
[1807910.701841] XFS (sdf): Unmounting Filesystem
[1807910.701854] XFS (sdf): xfs_log_force: error -5 returned.
[1807910.701866] XFS (sdf): xfs_log_force: error -5 returned.



The hard drive information:
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-9YN166
Serial Number:    W1F03GXY

-- 

Regards,
        Wu, Xingbo (吴兴博)  <wuxb45@gmail.com>

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Eric Sandeen]

On 1/5/17 12:02 PM, Xingbo Wu wrote:
> /dev/sdf is a 3TB hard drive.
> It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
> Checking for bad blocks in read-write mode
> From block 0 to 2930266583

2930266583 x 1024 == 3000592980992

I'd rather see what /proc/partitions and/or blockdev says about it...

> However mkfs.xfs reports a larger block count:
> `sudo mkfs.xfs /dev/sdf`:
> meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
>          =                       sectsz=4096  attr=2, projid32bit=1
>          =                       crc=1        finobt=1, sparse=0, rmapbt=0
> data     =                       bsize=4096   blocks=732566646, imaxpct=5
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
> log      =internal log           bsize=4096   blocks=357698, version=2
>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> realtime =none                   extsz=4096   blocks=0, rtextents=0

732566646 x 4k blocks == 3000592982016 bytes

Ok, that is 1024 bytes bigger.  But the weird thing is that
mkfs.xfs actually writes the last block, and fails if that fails.

So let's start with what /proc/partitions says, as well as

# blkdev --getsize --getsize64 /dev/sdf

-Eric

> It later leads to IO error:
> `dmesg`:
> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
> [1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
> ("xlog_iodone") error 5 numblks 64
> [1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
> line 1200 of file fs/xfs/xfs_log.c.  Return address =
> 0xffffffffa024090c
> [1807875.742381] XFS (sdf): Log I/O Error Detected.  Shutting down filesystem
> [1807875.742416] XFS (sdf): xfs_log_force: error -5 returned.
> [1807875.742426] XFS (sdf): Please umount the filesystem and rectify
> the problem(s)
> [1807881.868117] XFS (sdf): xfs_log_force: error -5 returned.
> [1807910.701836] XFS (sdf): xfs_log_force: error -5 returned.
> [1807910.701841] XFS (sdf): Unmounting Filesystem
> [1807910.701854] XFS (sdf): xfs_log_force: error -5 returned.
> [1807910.701866] XFS (sdf): xfs_log_force: error -5 returned.
> 
> 
> 
> The hard drive information:
> Model Family:     Seagate Barracuda 7200.14 (AF)
> Device Model:     ST3000DM001-9YN166
> Serial Number:    W1F03GXY
> 

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Xingbo Wu]

On Thu, Jan 5, 2017 at 12:58 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> On 1/5/17 12:02 PM, Xingbo Wu wrote:
>> /dev/sdf is a 3TB hard drive.
>> It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
>> Checking for bad blocks in read-write mode
>> From block 0 to 2930266583
>
> 2930266583 x 1024 == 3000592980992
>
> I'd rather see what /proc/partitions and/or blockdev says about it...
>

$ cat /proc/partitions
major minor  #blocks  name

   8        0  244198584 sda
   8        1   31457280 sda1
   8        2  212740096 sda2
   8       48 3907018584 sdd
   8       64 3907018584 sde
   8       32 1953514584 sdc
   8       16 1953514584 sdb
   8       80 2930266584 sdf


>> However mkfs.xfs reports a larger block count:
>> `sudo mkfs.xfs /dev/sdf`:
>> meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
>>          =                       sectsz=4096  attr=2, projid32bit=1
>>          =                       crc=1        finobt=1, sparse=0, rmapbt=0
>> data     =                       bsize=4096   blocks=732566646, imaxpct=5
>>          =                       sunit=0      swidth=0 blks
>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
>> log      =internal log           bsize=4096   blocks=357698, version=2
>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>
> 732566646 x 4k blocks == 3000592982016 bytes
>
> Ok, that is 1024 bytes bigger.  But the weird thing is that
> mkfs.xfs actually writes the last block, and fails if that fails.
>
> So let's start with what /proc/partitions says, as well as
>
> # blkdev --getsize --getsize64 /dev/sdf
>

The machines runs a up-to-dated Archlinux with Kernel 4.8.13 (64bit)
I didn't found blkdev but instead found blockdev in package
util-linux. It looks the samething.
$ sudo blockdev --getsize --getsize64 /dev/sdf
5860533168
3000592982016

> -Eric
>
>> It later leads to IO error:
>> `dmesg`:
>> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
>> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
>> [1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
>> ("xlog_iodone") error 5 numblks 64
>> [1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
>> line 1200 of file fs/xfs/xfs_log.c.  Return address =
>> 0xffffffffa024090c
>> [1807875.742381] XFS (sdf): Log I/O Error Detected.  Shutting down filesystem
>> [1807875.742416] XFS (sdf): xfs_log_force: error -5 returned.
>> [1807875.742426] XFS (sdf): Please umount the filesystem and rectify
>> the problem(s)
>> [1807881.868117] XFS (sdf): xfs_log_force: error -5 returned.
>> [1807910.701836] XFS (sdf): xfs_log_force: error -5 returned.
>> [1807910.701841] XFS (sdf): Unmounting Filesystem
>> [1807910.701854] XFS (sdf): xfs_log_force: error -5 returned.
>> [1807910.701866] XFS (sdf): xfs_log_force: error -5 returned.
>>
>>
>>
>> The hard drive information:
>> Model Family:     Seagate Barracuda 7200.14 (AF)
>> Device Model:     ST3000DM001-9YN166
>> Serial Number:    W1F03GXY
>>



-- 

Regards,
        Wu, Xingbo (吴兴博)  <wuxb45@gmail.com>

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Darrick J. Wong]

On Thu, Jan 05, 2017 at 12:58:23PM -0600, Eric Sandeen wrote:
> On 1/5/17 12:02 PM, Xingbo Wu wrote:
> > /dev/sdf is a 3TB hard drive.
> > It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
> > Checking for bad blocks in read-write mode
> > From block 0 to 2930266583
>
> 2930266583 x 1024 == 3000592980992
> 
> I'd rather see what /proc/partitions and/or blockdev says about it...

>From the badblocks manpage,

"-b block_size: Specify the size of blocks in bytes.  The default is 1024."

(So far so good.)

> > However mkfs.xfs reports a larger block count:
> > `sudo mkfs.xfs /dev/sdf`:
> > meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
> >          =                       sectsz=4096  attr=2, projid32bit=1
> >          =                       crc=1        finobt=1, sparse=0, rmapbt=0
> > data     =                       bsize=4096   blocks=732566646, imaxpct=5
> >          =                       sunit=0      swidth=0 blks
> > naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
> > log      =internal log           bsize=4096   blocks=357698, version=2
> >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > realtime =none                   extsz=4096   blocks=0, rtextents=0
> 
> 732566646 x 4k blocks == 3000592982016 bytes
> 
> Ok, that is 1024 bytes bigger.  But the weird thing is that
> mkfs.xfs actually writes the last block, and fails if that fails.
> 
> So let's start with what /proc/partitions says, as well as
> 
> # blkdev --getsize --getsize64 /dev/sdf
> 
> -Eric
> 
> > It later leads to IO error:
> > `dmesg`:
> > [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
> > [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664

The kernel complains about "sector 2930266664", which is in 512-byte blocks.

2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.

Makes sense since the log generally ends up halfway across the disk.
Not sure if there's any output before 1807875.741674 but this sort of
looks like a disk error or something?

--D

> > [1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
> > ("xlog_iodone") error 5 numblks 64
> > [1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
> > line 1200 of file fs/xfs/xfs_log.c.  Return address =
> > 0xffffffffa024090c
> > [1807875.742381] XFS (sdf): Log I/O Error Detected.  Shutting down filesystem
> > [1807875.742416] XFS (sdf): xfs_log_force: error -5 returned.
> > [1807875.742426] XFS (sdf): Please umount the filesystem and rectify
> > the problem(s)
> > [1807881.868117] XFS (sdf): xfs_log_force: error -5 returned.
> > [1807910.701836] XFS (sdf): xfs_log_force: error -5 returned.
> > [1807910.701841] XFS (sdf): Unmounting Filesystem
> > [1807910.701854] XFS (sdf): xfs_log_force: error -5 returned.
> > [1807910.701866] XFS (sdf): xfs_log_force: error -5 returned.
> > 
> > 
> > 
> > The hard drive information:
> > Model Family:     Seagate Barracuda 7200.14 (AF)
> > Device Model:     ST3000DM001-9YN166
> > Serial Number:    W1F03GXY
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Eric Sandeen]

On 1/5/17 1:13 PM, Darrick J. Wong wrote:
>>> It later leads to IO error:
>>> `dmesg`:
>>> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
>>> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
> The kernel complains about "sector 2930266664", which is in 512-byte blocks.
> 
> 2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.
> 
> Makes sense since the log generally ends up halfway across the disk.
> Not sure if there's any output before 1807875.741674 but this sort of
> looks like a disk error or something?

Oh, right.  Like my physics teacher said, "think units before you think
numbers!"  ;)  I usually remember ...

This doesn't look like a size mismatch problem, this looks like a failing
disk problem.

-Eric

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Xingbo Wu]

On Thu, Jan 5, 2017 at 1:19 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> On 1/5/17 1:13 PM, Darrick J. Wong wrote:
>>>> It later leads to IO error:
>>>> `dmesg`:
>>>> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
>>>> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
>> The kernel complains about "sector 2930266664", which is in 512-byte blocks.
>>
>> 2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.
>>
>> Makes sense since the log generally ends up halfway across the disk.
>> Not sure if there's any output before 1807875.741674 but this sort of
>> looks like a disk error or something?
>
> Oh, right.  Like my physics teacher said, "think units before you think
> numbers!"  ;)  I usually remember ...
>
> This doesn't look like a size mismatch problem, this looks like a failing
> disk problem.
>
> -Eric

It looks I need to scan for bad blocks. Thanks for your time :)

-- 

Regards,
        Wu, Xingbo (吴兴博)  <wuxb45@gmail.com>

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Xingbo Wu]

On Thu, Jan 5, 2017 at 1:13 PM, Darrick J. Wong <darrick.wong@oracle.com> wrote:
> On Thu, Jan 05, 2017 at 12:58:23PM -0600, Eric Sandeen wrote:
>> On 1/5/17 12:02 PM, Xingbo Wu wrote:
>> > /dev/sdf is a 3TB hard drive.
>> > It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
>> > Checking for bad blocks in read-write mode
>> > From block 0 to 2930266583
>>
>> 2930266583 x 1024 == 3000592980992
>>
>> I'd rather see what /proc/partitions and/or blockdev says about it...
>
> From the badblocks manpage,
>
> "-b block_size: Specify the size of blocks in bytes.  The default is 1024."
>
> (So far so good.)
>
>> > However mkfs.xfs reports a larger block count:
>> > `sudo mkfs.xfs /dev/sdf`:
>> > meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
>> >          =                       sectsz=4096  attr=2, projid32bit=1
>> >          =                       crc=1        finobt=1, sparse=0, rmapbt=0
>> > data     =                       bsize=4096   blocks=732566646, imaxpct=5
>> >          =                       sunit=0      swidth=0 blks
>> > naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
>> > log      =internal log           bsize=4096   blocks=357698, version=2
>> >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
>> > realtime =none                   extsz=4096   blocks=0, rtextents=0
>>
>> 732566646 x 4k blocks == 3000592982016 bytes
>>
>> Ok, that is 1024 bytes bigger.  But the weird thing is that
>> mkfs.xfs actually writes the last block, and fails if that fails.
>>
>> So let's start with what /proc/partitions says, as well as
>>
>> # blkdev --getsize --getsize64 /dev/sdf
>>
>> -Eric
>>
>> > It later leads to IO error:
>> > `dmesg`:
>> > [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
>> > [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
>
> The kernel complains about "sector 2930266664", which is in 512-byte blocks.
>
> 2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.
>
> Makes sense since the log generally ends up halfway across the disk.
> Not sure if there's any output before 1807875.741674 but this sort of
> looks like a disk error or something?
>

I grepped "sdf" from dmesg:
[    7.158847] sd 0:0:6:0: [sdf] 5860533168 512-byte logical blocks:
(3.00 TB/2.73 TiB)
[    7.158851] sd 0:0:6:0: [sdf] 4096-byte physical blocks
[    7.259937] sd 0:0:6:0: [sdf] Write Protect is off
[    7.259943] sd 0:0:6:0: [sdf] Mode Sense: 9b 00 10 08
[    7.282316] sd 0:0:6:0: [sdf] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.418959] sd 0:0:6:0: [sdf] Attached SCSI disk
[1807850.471959] XFS (sdf): Mounting V5 Filesystem
[1807850.605057] XFS (sdf): Ending clean mount
[1807875.741657] sd 0:0:6:0: [sdf] tag#0 UNKNOWN(0x2003) Result:
hostbyte=0x07 driverbyte=0x00
[1807875.741670] sd 0:0:6:0: [sdf] tag#0 CDB: opcode=0x8a 8a 08 00 00
00 00 ae a8 52 28 00 00 00 08 00 00
[1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
[1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
[1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
("xlog_iodone") error 5 numblks 64
[1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
line 1200 of file fs/xfs/xfs_log.c.  Return address =
0xffffffffa024090c

Earlier I tried to format the disk with ext4. It also had similar
errors. I thought it was the disk's problem. Would that be something
wrong with mkfs' math? I will try ext4 again and let you know if I
found something relevant.

-- 

Regards,
        Wu, Xingbo (吴兴博)  <wuxb45@gmail.com>

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Eric Sandeen]

On 1/5/17 1:58 PM, Xingbo Wu wrote:
> On Thu, Jan 5, 2017 at 1:13 PM, Darrick J. Wong <darrick.wong@oracle.com> wrote:
>> On Thu, Jan 05, 2017 at 12:58:23PM -0600, Eric Sandeen wrote:
>>> On 1/5/17 12:02 PM, Xingbo Wu wrote:
>>>> /dev/sdf is a 3TB hard drive.
>>>> It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
>>>> Checking for bad blocks in read-write mode
>>>> From block 0 to 2930266583
>>>
>>> 2930266583 x 1024 == 3000592980992
>>>
>>> I'd rather see what /proc/partitions and/or blockdev says about it...
>>
>> From the badblocks manpage,
>>
>> "-b block_size: Specify the size of blocks in bytes.  The default is 1024."
>>
>> (So far so good.)
>>
>>>> However mkfs.xfs reports a larger block count:
>>>> `sudo mkfs.xfs /dev/sdf`:
>>>> meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
>>>>          =                       sectsz=4096  attr=2, projid32bit=1
>>>>          =                       crc=1        finobt=1, sparse=0, rmapbt=0
>>>> data     =                       bsize=4096   blocks=732566646, imaxpct=5
>>>>          =                       sunit=0      swidth=0 blks
>>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
>>>> log      =internal log           bsize=4096   blocks=357698, version=2
>>>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
>>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>>>
>>> 732566646 x 4k blocks == 3000592982016 bytes
>>>
>>> Ok, that is 1024 bytes bigger.  But the weird thing is that
>>> mkfs.xfs actually writes the last block, and fails if that fails.
>>>
>>> So let's start with what /proc/partitions says, as well as
>>>
>>> # blkdev --getsize --getsize64 /dev/sdf
>>>
>>> -Eric
>>>
>>>> It later leads to IO error:
>>>> `dmesg`:
>>>> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
>>>> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
>>
>> The kernel complains about "sector 2930266664", which is in 512-byte blocks.
>>
>> 2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.
>>
>> Makes sense since the log generally ends up halfway across the disk.
>> Not sure if there's any output before 1807875.741674 but this sort of
>> looks like a disk error or something?
>>
> 
> I grepped "sdf" from dmesg:
> [    7.158847] sd 0:0:6:0: [sdf] 5860533168 512-byte logical blocks:
> (3.00 TB/2.73 TiB)
> [    7.158851] sd 0:0:6:0: [sdf] 4096-byte physical blocks
> [    7.259937] sd 0:0:6:0: [sdf] Write Protect is off
> [    7.259943] sd 0:0:6:0: [sdf] Mode Sense: 9b 00 10 08
> [    7.282316] sd 0:0:6:0: [sdf] Write cache: enabled, read cache:
> enabled, supports DPO and FUA
> [    7.418959] sd 0:0:6:0: [sdf] Attached SCSI disk
> [1807850.471959] XFS (sdf): Mounting V5 Filesystem
> [1807850.605057] XFS (sdf): Ending clean mount
> [1807875.741657] sd 0:0:6:0: [sdf] tag#0 UNKNOWN(0x2003) Result:
> hostbyte=0x07 driverbyte=0x00
> [1807875.741670] sd 0:0:6:0: [sdf] tag#0 CDB: opcode=0x8a 8a 08 00 00
> 00 00 ae a8 52 28 00 00 00 08 00 00
> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664

Your disk is failing.  To solve this, get a new disk.

-Eric

> [1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
> ("xlog_iodone") error 5 numblks 64
> [1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
> line 1200 of file fs/xfs/xfs_log.c.  Return address =
> 0xffffffffa024090c
> 
> Earlier I tried to format the disk with ext4. It also had similar
> errors. I thought it was the disk's problem. Would that be something
> wrong with mkfs' math? I will try ext4 again and let you know if I
> found something relevant.
> 

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Darrick J. Wong]

On Thu, Jan 05, 2017 at 02:01:40PM -0600, Eric Sandeen wrote:
> On 1/5/17 1:58 PM, Xingbo Wu wrote:
> > On Thu, Jan 5, 2017 at 1:13 PM, Darrick J. Wong <darrick.wong@oracle.com> wrote:
> >> On Thu, Jan 05, 2017 at 12:58:23PM -0600, Eric Sandeen wrote:
> >>> On 1/5/17 12:02 PM, Xingbo Wu wrote:
> >>>> /dev/sdf is a 3TB hard drive.
> >>>> It has 2930266584 blocks, as reported in `badblocks -wsv /dev/sdf`:
> >>>> Checking for bad blocks in read-write mode
> >>>> From block 0 to 2930266583
> >>>
> >>> 2930266583 x 1024 == 3000592980992
> >>>
> >>> I'd rather see what /proc/partitions and/or blockdev says about it...
> >>
> >> From the badblocks manpage,
> >>
> >> "-b block_size: Specify the size of blocks in bytes.  The default is 1024."
> >>
> >> (So far so good.)
> >>
> >>>> However mkfs.xfs reports a larger block count:
> >>>> `sudo mkfs.xfs /dev/sdf`:
> >>>> meta-data=/dev/sdf               isize=512    agcount=4, agsize=183141662 blks
> >>>>          =                       sectsz=4096  attr=2, projid32bit=1
> >>>>          =                       crc=1        finobt=1, sparse=0, rmapbt=0
> >>>> data     =                       bsize=4096   blocks=732566646, imaxpct=5
> >>>>          =                       sunit=0      swidth=0 blks
> >>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
> >>>> log      =internal log           bsize=4096   blocks=357698, version=2
> >>>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> >>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
> >>>
> >>> 732566646 x 4k blocks == 3000592982016 bytes
> >>>
> >>> Ok, that is 1024 bytes bigger.  But the weird thing is that
> >>> mkfs.xfs actually writes the last block, and fails if that fails.
> >>>
> >>> So let's start with what /proc/partitions says, as well as
> >>>
> >>> # blkdev --getsize --getsize64 /dev/sdf
> >>>
> >>> -Eric
> >>>
> >>>> It later leads to IO error:
> >>>> `dmesg`:
> >>>> [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
> >>>> [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
> >>
> >> The kernel complains about "sector 2930266664", which is in 512-byte blocks.
> >>
> >> 2930266664 * 512 = 1500296531968, or halfway across your 3TB disk.
> >>
> >> Makes sense since the log generally ends up halfway across the disk.
> >> Not sure if there's any output before 1807875.741674 but this sort of
> >> looks like a disk error or something?
> >>
> > 
> > I grepped "sdf" from dmesg:
> > [    7.158847] sd 0:0:6:0: [sdf] 5860533168 512-byte logical blocks:
> > (3.00 TB/2.73 TiB)
> > [    7.158851] sd 0:0:6:0: [sdf] 4096-byte physical blocks
> > [    7.259937] sd 0:0:6:0: [sdf] Write Protect is off
> > [    7.259943] sd 0:0:6:0: [sdf] Mode Sense: 9b 00 10 08
> > [    7.282316] sd 0:0:6:0: [sdf] Write cache: enabled, read cache:
> > enabled, supports DPO and FUA
> > [    7.418959] sd 0:0:6:0: [sdf] Attached SCSI disk
> > [1807850.471959] XFS (sdf): Mounting V5 Filesystem
> > [1807850.605057] XFS (sdf): Ending clean mount
> > [1807875.741657] sd 0:0:6:0: [sdf] tag#0 UNKNOWN(0x2003) Result:
> > hostbyte=0x07 driverbyte=0x00
> > [1807875.741670] sd 0:0:6:0: [sdf] tag#0 CDB: opcode=0x8a 8a 08 00 00
> > 00 00 ae a8 52 28 00 00 00 08 00 00

WRITE(16) with FUA set... on a SATA drive?

A dumb Google search of "ST3000DM001 FUA" pulls up a lot of dmesg
that show the drive not advertising DPO or FUA support.

0:0:6:0 ... SCSI host0, ctrl0, target6, lun0, huh?  Is this SATA drive
connected to a SAS/RAID controller or something?  It could be
advertising FUA support to Linux but the drive (or even the SATL I
guess) rejects an actual command w/ the FUA bit set.

--D

> > [1807875.741674] blk_update_request: I/O error, dev sdf, sector 2930266664
> > [1807875.741732] blk_update_request: I/O error, dev sdf, sector 2930266664
> 
> Your disk is failing.  To solve this, get a new disk.
> 
> -Eric
> 
> > [1807875.742306] XFS (sdf): metadata I/O error: block 0xaea85228
> > ("xlog_iodone") error 5 numblks 64
> > [1807875.742369] XFS (sdf): xfs_do_force_shutdown(0x2) called from
> > line 1200 of file fs/xfs/xfs_log.c.  Return address =
> > 0xffffffffa024090c
> > 
> > Earlier I tried to format the disk with ext4. It also had similar
> > errors. I thought it was the disk's problem. Would that be something
> > wrong with mkfs' math? I will try ext4 again and let you know if I
> > found something relevant.
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] xfs.mkfs recognizes wrong disk size

[Xingbo Wu]

>> > [    7.158847] sd 0:0:6:0: [sdf] 5860533168 512-byte logical blocks:
>> > (3.00 TB/2.73 TiB)
>> > [    7.158851] sd 0:0:6:0: [sdf] 4096-byte physical blocks
>> > [    7.259937] sd 0:0:6:0: [sdf] Write Protect is off
>> > [    7.259943] sd 0:0:6:0: [sdf] Mode Sense: 9b 00 10 08
>> > [    7.282316] sd 0:0:6:0: [sdf] Write cache: enabled, read cache:
>> > enabled, supports DPO and FUA
>> > [    7.418959] sd 0:0:6:0: [sdf] Attached SCSI disk
>> > [1807850.471959] XFS (sdf): Mounting V5 Filesystem
>> > [1807850.605057] XFS (sdf): Ending clean mount
>> > [1807875.741657] sd 0:0:6:0: [sdf] tag#0 UNKNOWN(0x2003) Result:
>> > hostbyte=0x07 driverbyte=0x00
>> > [1807875.741670] sd 0:0:6:0: [sdf] tag#0 CDB: opcode=0x8a 8a 08 00 00
>> > 00 00 ae a8 52 28 00 00 00 08 00 00
>
> WRITE(16) with FUA set... on a SATA drive?
>
> A dumb Google search of "ST3000DM001 FUA" pulls up a lot of dmesg
> that show the drive not advertising DPO or FUA support.
>
> 0:0:6:0 ... SCSI host0, ctrl0, target6, lun0, huh?  Is this SATA drive
> connected to a SAS/RAID controller or something?  It could be
> advertising FUA support to Linux but the drive (or even the SATL I
> guess) rejects an actual command w/ the FUA bit set.
>
> --D
>

It's a Dell T630 Tower server. All disks are connected through a panel
that looks like some controller. Anyway no RAID was configured and I
just plugged in disks/ssds. Never had a problem with other disks.

Here is a longer part of the dmesg:
[    4.378499] scsi host0: Avago SAS based MegaRAID driver
[    4.379725] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6
Gbps 0x3f impl SATA mode
[    4.379730] ahci 0000:00:1f.2: flags: 64bit ncq stag led clo pio
slum part ems apst
[    4.384933] scsi 0:0:0:0: Direct-Access     ATA      Samsung SSD
840  BB6Q PQ: 0 ANSI: 6
[    4.429659] scsi host5: ahci
[    4.429847] scsi host6: ahci
[    4.430065] scsi host7: ahci
[    4.430246] scsi host8: ahci
[    4.430434] scsi host9: ahci
[    4.430650] scsi host10: ahci
[    4.430757] ata5: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00100 irq 63
[    4.430760] ata6: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00180 irq 63
[    4.430762] ata7: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00200 irq 63
[    4.430765] ata8: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00280 irq 63
[    4.430767] ata9: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00300 irq 63
[    4.430770] ata10: SATA max UDMA/133 abar m2048@0x91f00000 port
0x91f00380 irq 63
[    4.446366] scsi 0:0:2:0: Direct-Access     ATA
ST2000NM0055-1V4 DA03 PQ: 0 ANSI: 6
[    4.520634] scsi 0:0:3:0: Direct-Access     ATA
ST2000NM0055-1V4 DA03 PQ: 0 ANSI: 6
[    4.566866] scsi 0:0:4:0: Direct-Access     ATA      WDC
WD40EZRX-00S 0A80 PQ: 0 ANSI: 6
[    4.603520] scsi 0:0:5:0: Direct-Access     ATA      WDC
WD40EZRX-00S 0A80 PQ: 0 ANSI: 6
[    4.639943] scsi 0:0:6:0: Direct-Access     ATA
ST3000DM001-9YN1 CC98 PQ: 0 ANSI: 6
[    4.648847] ata1: SATA link down (SStatus 0 SControl 300)
[    4.671763] usb 1-1: new high-speed USB device number 2 using ehci-pci
[    4.691709] usb 2-1: new high-speed USB device number 2 using ehci-pci
[    4.745205] ata5: SATA link down (SStatus 0 SControl 300)
[    4.815850] hub 1-1:1.0: USB hub found
[    4.815982] hub 1-1:1.0: 6 ports detected
[    4.842352] hub 2-1:1.0: USB hub found
[    4.842393] hub 2-1:1.0: 8 ports detected
[    4.911744] tsc: Refined TSC clocksource calibration: 2197.454 MHz
[    4.911764] clocksource: tsc: mask: 0xffffffffffffffff max_cycles:
0x1faccdc500e, max_idle_ns: 440795287451 ns
[    4.961561] ata2: SATA link down (SStatus 0 SControl 300)
[    5.098384] usb 1-1.6: new high-speed USB device number 3 using ehci-pci
[    5.123235] random: fast init done
[    5.196708] hub 1-1.6:1.0: USB hub found
[    5.196863] hub 1-1.6:1.0: 6 ports detected
[    5.275437] ata3: SATA link down (SStatus 0 SControl 300)
[    5.588809] ata4: SATA link down (SStatus 0 SControl 300)
[    5.902119] ata6: SATA link down (SStatus 0 SControl 300)
[    5.925244] clocksource: Switched to clocksource tsc
[    6.215354] ata7: SATA link down (SStatus 0 SControl 300)
[    6.528669] ata8: SATA link down (SStatus 0 SControl 300)
[    6.841961] ata9: SATA link down (SStatus 0 SControl 300)
[    7.155272] ata10: SATA link down (SStatus 0 SControl 300)
[    7.158789] sd 0:0:5:0: [sde] 7814037168 512-byte logical blocks:
(4.00 TB/3.64 TiB)
[    7.158793] sd 0:0:5:0: [sde] 4096-byte physical blocks
[    7.158801] sd 0:0:4:0: [sdd] 7814037168 512-byte logical blocks:
(4.00 TB/3.64 TiB)
[    7.158805] sd 0:0:4:0: [sdd] 4096-byte physical blocks
[    7.158847] sd 0:0:6:0: [sdf] 5860533168 512-byte logical blocks:
(3.00 TB/2.73 TiB)
[    7.158851] sd 0:0:6:0: [sdf] 4096-byte physical blocks
[    7.159772] sd 0:0:0:0: [sda] 488397168 512-byte logical blocks:
(250 GB/233 GiB)
[    7.163014] sd 0:0:3:0: [sdc] 3907029168 512-byte logical blocks:
(2.00 TB/1.82 TiB)
[    7.165425] sd 0:0:4:0: [sdd] Write Protect is off
[    7.165431] sd 0:0:4:0: [sdd] Mode Sense: 9b 00 10 08
[    7.165507] sd 0:0:0:0: [sda] Write Protect is off
[    7.165512] sd 0:0:0:0: [sda] Mode Sense: 9b 00 10 08
[    7.165524] sd 0:0:5:0: [sde] Write Protect is off
[    7.165530] sd 0:0:5:0: [sde] Mode Sense: 9b 00 10 08
[    7.165910] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.166445] sd 0:0:4:0: [sdd] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.166637] sd 0:0:5:0: [sde] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.167238] sd 0:0:2:0: [sdb] 3907029168 512-byte logical blocks:
(2.00 TB/1.82 TiB)
[    7.172598]  sda: sda1 sda2
[    7.179065] sd 0:0:0:0: [sda] Attached SCSI disk
[    7.191963] sd 0:0:3:0: [sdc] Write Protect is off
[    7.191969] sd 0:0:3:0: [sdc] Mode Sense: 9b 00 10 08
[    7.193469] sd 0:0:3:0: [sdc] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.195788] sd 0:0:2:0: [sdb] Write Protect is off
[    7.195793] sd 0:0:2:0: [sdb] Mode Sense: 9b 00 10 08
[    7.197287] sd 0:0:2:0: [sdb] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.243312] sd 0:0:3:0: [sdc] Attached SCSI disk
[    7.247197] sd 0:0:2:0: [sdb] Attached SCSI disk
[    7.259937] sd 0:0:6:0: [sdf] Write Protect is off
[    7.259943] sd 0:0:6:0: [sdf] Mode Sense: 9b 00 10 08
[    7.282316] sd 0:0:6:0: [sdf] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[    7.418959] sd 0:0:6:0: [sdf] Attached SCSI disk
[    7.971468] sd 0:0:4:0: [sdd] Attached SCSI disk
[    8.010606] sd 0:0:5:0: [sde] Attached SCSI disk
[    8.270177] SGI XFS with ACLs, security attributes, realtime, no
debug enabled
[    8.274002] XFS (sda1): Mounting V4 Filesystem
[    8.274707] random: crng init done
[    8.286898] XFS (sda1): Ending clean mount



-- 

Regards,
        Wu, Xingbo (吴兴博)  <wuxb45@gmail.com>