How to delete/rewrite a corrupted file in a read only snapshot?

How to delete/rewrite a corrupted file in a read only snapshot?

[Marc MERLIN]

howdy,

I'm seeing this during a background check:
[374402.156920] BTRFS warning (device dm-18): checksum error at logical 4401320624128 on dev /dev/dm-18, physical 2939697954816, root 63534, inode 595460, offset 1506283520, length 4096, links 1 (path: nobck/file2)

this is in a read only btrfs send snapshot, so I can't just delete the
file or the snspshot (I have 20 historical ones all with the same broken
fine).

can I either
1) force delete it with some admin tool
2) even better force/overwrite it with the correct file from source?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: How to delete/rewrite a corrupted file in a read only snapshot?

[Holger Hoffstätte]

On 2024-02-06 20:12, Marc MERLIN wrote:
> howdy,
> 
> I'm seeing this during a background check:
> [374402.156920] BTRFS warning (device dm-18): checksum error at logical 4401320624128 on dev /dev/dm-18, physical 2939697954816, root 63534, inode 595460, offset 1506283520, length 4096, links 1 (path: nobck/file2)
> 
> this is in a read only btrfs send snapshot, so I can't just delete the
> file or the snspshot (I have 20 historical ones all with the same broken
> fine).
> 
> can I either
> 1) force delete it with some admin tool
> 2) even better force/overwrite it with the correct file from source?

You can flip subvolumes between read-only and read-write:

   $btrfs prop set yoursubvolume ro false

See "man btrfs-property" for details.

This should allow you to delete the broken file or replace it.

Good luck!
Holger

Re: How to delete/rewrite a corrupted file in a read only snapshot?

[Andrei Borzenkov]

On 06.02.2024 22:29, Holger Hoffstätte wrote:
> On 2024-02-06 20:12, Marc MERLIN wrote:
>> howdy,
>>
>> I'm seeing this during a background check:
>> [374402.156920] BTRFS warning (device dm-18): checksum error at logical 4401320624128 on dev /dev/dm-18, physical 2939697954816, root 63534, inode 595460, offset 1506283520, length 4096, links 1 (path: nobck/file2)
>>
>> this is in a read only btrfs send snapshot, so I can't just delete the
>> file or the snspshot (I have 20 historical ones all with the same broken
>> fine).
>>
>> can I either
>> 1) force delete it with some admin tool
>> 2) even better force/overwrite it with the correct file from source?
> 
> You can flip subvolumes between read-only and read-write:
> 

This will break replication chain.

Re: How to delete/rewrite a corrupted file in a read only snapshot?

[Qu Wenruo]

On 2024/2/7 05:42, Marc MERLIN wrote:
> howdy,
>
> I'm seeing this during a background check:
> [374402.156920] BTRFS warning (device dm-18): checksum error at logical 4401320624128 on dev /dev/dm-18, physical 2939697954816, root 63534, inode 595460, offset 1506283520, length 4096, links 1 (path: nobck/file2)
>
> this is in a read only btrfs send snapshot, so I can't just delete the
> file or the snspshot (I have 20 historical ones all with the same broken
> fine).
>
> can I either
> 1) force delete it with some admin tool

That would break any later incremental receive.

> 2) even better force/overwrite it with the correct file from source?

That's possible manually.

If you can get the correct content of that sector, you can manually
write it back to the location (all needed info is already in that line,
including where the device is, what's the physical offset, the size)

And later verify with scrub to make sure it's properly fixed.

Thanks,
Qu
>
> Thanks,
> Marc

Re: How to delete/rewrite a corrupted file in a read only snapshot?

[Marc MERLIN]

On Wed, Feb 07, 2024 at 03:37:04PM +1030, Qu Wenruo wrote:
> > can I either
> > 1) force delete it with some admin tool
> 
> That would break any later incremental receive.
> 
> > 2) even better force/overwrite it with the correct file from source?
> 
> That's possible manually.

If I change the snapshot to readwrite, and cat the good file into the
corrupted one 
cat good > /path/to/bad
will that work, or will that allocate new COW blocks and do weird things
that would break later brfs send/receive?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: How to delete/rewrite a corrupted file in a read only snapshot?

[Qu Wenruo]

On 2024/2/8 07:55, Marc MERLIN wrote:
> On Wed, Feb 07, 2024 at 03:37:04PM +1030, Qu Wenruo wrote:
>>> can I either
>>> 1) force delete it with some admin tool
>>
>> That would break any later incremental receive.
>>
>>> 2) even better force/overwrite it with the correct file from source?
>>
>> That's possible manually.
>
> If I change the snapshot to readwrite, and cat the good file into the
> corrupted one
> cat good > /path/to/bad
> will that work, or will that allocate new COW blocks and do weird things
> that would break later brfs send/receive?

As long as you changed the snapshot to RW, it would break later
incremental receive IIRC, but I'm not 100% sure.

And for that cat, it would definitely lead to COW.

So, no, that would not fix the corrupted sector, you have to go the
dangerous way.

Thanks,
Qu

>
> Thanks,
> Marc