Re: BTRFS scrub reports an error but check doesn't find any errors.

[Andrei Borzenkov <arvidjaar@gmail.com>]

On 28.07.2021 19:21, Dave T wrote:
> On Wed, Jul 28, 2021 at 12:11 PM Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>
>> On 28.07.2021 18:15, Dave T wrote:
>> ...
>>>
>>> Jul 27 21:54:39 server kernel: ata10.00: exception Emask 0x0 SAct
>>> 0xffffffff SErr 0x0 action 0x0
>>> Jul 27 21:54:39 server kernel: ata10.00: irq_stat 0x40000008
>>> Jul 27 21:54:39 server kernel: ata10.00: failed command: READ FPDMA QUEUED
>>> Jul 27 21:54:39 server kernel: ata10.00: cmd
>>> 60/00:90:98:2f:9f/03:00:a4:00:00/40 tag 18 ncq dma 393216 in
>>>                                          res
>>> 41/40:00:20:32:9f/00:03:a4:00:00/00 Emask 0x409 (media error) <F>
>>> Jul 27 21:54:39 server kernel: ata10.00: status: { DRDY ERR }
>>> Jul 27 21:54:39 server kernel: ata10.00: error: { UNC }
>>> Jul 27 21:54:39 server kernel: ata10.00: configured for UDMA/133
>>> Jul 27 21:54:39 server kernel: sd 9:0:0:0: [sde] tag#18 FAILED Result:
>>> hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=3s
>>> Jul 27 21:54:39 server kernel: sd 9:0:0:0: [sde] tag#18 Sense Key :
>>> Medium Error [current]
>>> Jul 27 21:54:39 server kernel: sd 9:0:0:0: [sde] tag#18 Add. Sense:
>>> Unrecovered read error - auto reallocate failed
>>> Jul 27 21:54:39 server kernel: sd 9:0:0:0: [sde] tag#18 CDB: Read(16)
>>> 88 00 00 00 00 00 a4 9f 2f 98 00 00 03 00 00 00
>>> Jul 27 21:54:39 server kernel: blk_update_request: I/O error, dev sde,
>>> sector 2761896480 op 0x0:(READ) flags 0x0 phys_seg 15 prio class 0
>>> Jul 27 21:54:39 server kernel: ata10: EH complete
>>> Jul 27 21:54:45 server kernel: ata10.00: exception Emask 0x0 SAct
>>> 0x4000000 SErr 0x0 action 0x0
>>> Jul 27 21:54:45 server kernel: ata10.00: irq_stat 0x40000008
>>> Jul 27 21:54:45 server kernel: ata10.00: failed command: READ FPDMA QUEUED
>>> Jul 27 21:54:45 server kernel: ata10.00: cmd
>>> 60/08:d0:20:32:9f/00:00:a4:00:00/40 tag 26 ncq dma 4096 in
>>>                                          res
>>> 41/40:08:20:32:9f/00:00:a4:00:00/00 Emask 0x409 (media error) <F>
>>> Jul 27 21:54:45 server kernel: ata10.00: status: { DRDY ERR }
>>> Jul 27 21:54:45 server kernel: ata10.00: error: { UNC }
>>> Jul 27 21:54:45 server kernel: ata10.00: configured for UDMA/133
>>> Jul 27 21:54:45 server kernel: sd 9:0:0:0: [sde] tag#26 FAILED Result:
>>> hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=4s
>>> Jul 27 21:54:45 server kernel: sd 9:0:0:0: [sde] tag#26 Sense Key :
>>> Medium Error [current]
>>> Jul 27 21:54:45 server kernel: sd 9:0:0:0: [sde] tag#26 Add. Sense:
>>> Unrecovered read error - auto reallocate failed
>>> Jul 27 21:54:45 server kernel: sd 9:0:0:0: [sde] tag#26 CDB: Read(16)
>>> 88 00 00 00 00 00 a4 9f 32 20 00 00 00 08 00 00
>>> Jul 27 21:54:45 server kernel: blk_update_request: I/O error, dev sde,
>>> sector 2761896480 op 0x0:(READ) flags 0x800 phys_seg 1 prio class 0
>>> Jul 27 21:54:45 server kernel: ata10: EH complete
>>> Jul 27 21:54:45 server kernel: BTRFS warning (device dm-2): i/o error
>>> at logical 1567691653120 on dev /dev/mapper/userluks, physical
>>> 1414087852032, root 19911, inode 624993, offset 5717954560, length
>>> 4096, links 1 (path: path/to/file/filename.ext)
>>> Jul 27 21:54:45 server kernel: BTRFS warning (device dm-2): i/o error
>>> at logical 1567691653120 on dev /dev/mapper/userluks, physical
>>> 1414087852032, root 19989, inode 624993, offset 5717954560, length
>>> 4096, links 1 (path: path/to/file/filename.ext)
>>> Jul 27 21:54:45 server kernel: BTRFS warning (device dm-2): i/o error
>>> at logical 1567691653120 on dev /dev/mapper/userluks, physical
>>> 1414087852032, root 20199, inode 624993, offset 5717954560, length
>>> 4096, links 1 (path: path/to/file/filename.ext)
>>> Jul 27 21:54:45 server kernel: BTRFS error (device dm-2): bdev
>>> /dev/mapper/userluks errs: wr 0, rd 2, flush 0, corrupt 0, gen 0
>>> Jul 27 21:54:45 server kernel: BTRFS error (device dm-2): unable to
>>> fixup (regular) error at logical 1567691653120 on dev
>>> /dev/mapper/userluks
>>> Jul 27 21:55:42 server kernel: BTRFS info (device dm-2): scrub:
>>> finished on devid 1 with status: 0
>>>
>> ...>
>>> The volume is a 3TB disk, model ST3000DM001-1CH166 (Seagate Barracuda
>>> SATA HDD).
>>>
>>> Is there a way to mark sectors on the disk as bad? If so, is it
>>
>> Directly overwriting sector may "fix" it (of course, data is still lost)
>> or trigger sector replacement. hdparm has --write-sector command
>> although I do not have any experience with it. Or simple dd may suffice.
>> Difference is that hdparm will bypass any kernel block layer recovery.
>> If you had redundant data profile, btrfs scrub would likely have fixed
>> it for you.
> 
> I never knew BTRFS could duplicate data without RAID. This looks like
> a great feature for my situation. I think I may upgrade this disk to a
> larger one and enable DUP for data.
> 
> Is this a good tutorial to follow?
> https://zejn.net/b/2017/04/30/single-device-data-redundancy-with-btrfs/
> 
> Should I expect my data to take twice as much space after enabling DUP?
> 

Yes, of course. Your data will be stored twice.

>>
>>> advisable to keep using this physical disk?
>>>
>>
>> Well, this happens, if this is just one sector so far I would say yes.
>> You probably need to keep an eye on it though.
> 
> Thanks. My guess is that this is an isolated issue. It doesn't seem to
> be growing, but I will watch it.
>