[f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[John Kehayias via Linux-f2fs-devel]

Hello,

Apologies if this is not the correct mailing list, but it was the only one I could find.

I just resized an f2fs partition to the remaining free space on the disk. This seemed fine and resize.f2fs also seemed fine. However, the partition will no longer mount, due to this message:

Aug 12 00:10:09 localhost vmunix: [22056.879920] F2FS-fs (sdb2): invalid crc_offset: 0
Aug 12 00:10:09 localhost vmunix: [22056.881202] F2FS-fs (sdb2): Wrong valid_user_blocks: 7163307, user_block_count: 5227520
Aug 12 00:10:09 localhost vmunix: [22056.881206] F2FS-fs (sdb2): Failed to get valid F2FS checkpoint

Is there any way to fix this error?

I see this error reported before and supposedly fixed in current versions, though I had use f2fs-tools v1.14. I tried using 1.15 and from git, but resize reports nothing to do and fsck is successful. Here is that output:

Info: [/dev/sdb2] Disk Model: SD  Transcend   
Info: MKFS version
  "Linux version 5.5.2-arch1-1 (linux@archlinux) (gcc version 9.2.0 (GCC)) #1 SMP PREEMPT Tue, 04 Feb 2020 18:56:18 +0000"
Info: FSCK version
  from "Linux version 5.18.14 (guix@guix) (gcc (GCC) 10.3.0, GNU ld (GNU Binutils) 2.37) #1 SMP PREEMPT_DYNAMIC 1"
    to "Linux version 5.18.14 (guix@guix) (gcc (GCC) 10.3.0, GNU ld (GNU Binutils) 2.37) #1 SMP PREEMPT_DYNAMIC 1"
Info: superblock features = 0 : 
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: Segments per section = 1
Info: Sections per zone = 1
Info: total FS sectors = 124528600 (60804 MB)
	Invalid CP CRC offset: 0
Info: CKPT version = 6ea966da
Info: Checked valid nat_bits in checkpoint
Info: checkpoint state = 81 :  nat_bits unmount
[FSCK] Check node 1 / 402988 (0.00%)
[FSCK] Check node 40299 / 402988 (10.00%)
[FSCK] Check node 80597 / 402988 (20.00%)
[FSCK] Check node 120895 / 402988 (30.00%)
[FSCK] Check node 161193 / 402988 (40.00%)
[FSCK] Check node 201491 / 402988 (50.00%)
[FSCK] Check node 241789 / 402988 (60.00%)
[FSCK] Check node 282087 / 402988 (70.00%)
[FSCK] Check node 322385 / 402988 (80.00%)
[FSCK] Check node 362683 / 402988 (90.00%)
[FSCK] Check node 402981 / 402988 (100.00%)

[FSCK] Max image size: 30099 MB, Free space: 16769654 MB
[FSCK] Unreachable nat entries                        [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking                [Ok..]
[FSCK] Hard link checking for regular file            [Ok..] [0x474]
[FSCK] valid_block_count matching with CP             [Ok..] [0x6d4dab]
[FSCK] valid_node_count matching with CP (de lookup)  [Ok..] [0x6262c]
[FSCK] valid_node_count matching with CP (nat lookup) [Ok..] [0x6262c]
[FSCK] valid_inode_count matched with CP              [Ok..] [0x61367]
[FSCK] free segment_count matched with CP             [Ok..] [0x3cc0]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Ok..]

Done: 0.000000 secs
sudo fsck.f2fs /dev/sdb2  6.20s user 5.41s system 4% cpu 4:24.44 total

I hope someone can help. It seems to just be a metadata issue? Is there a way to manually fix the misreported block count to resolve this?

Thanks,
John


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[John Kehayias via Linux-f2fs-devel]

Hello again,

As this is the main disk for a homeserver, I was anxious to get it working again. After reading about what seemed like the same exact problem here: https://old.reddit.com/r/linuxquestions/comments/whomdc/expanded_f2fs_cant_mount/ I thought using a newer f2fs-tools would help (I think I ended up with latest git commit). As I reported earlier though, resize won't do anything (already resized). So I thought about shrinking, though it might be risky.

First, I tested with a disk image, made with dd and mounted with sudo udisksctl loop-setup -f disk.img. Then I could operate on the loop device with that partition, /dev/loop0p2 in my case. So I tried shrinking, bolstered by the good experiences reported in https://www.mail-archive.com/linux-f2fs-devel@lists.sourceforge.net/msg23214.html Since the size was something like twice the data on it, I didn't worry too much about how much smaller I made it. That was then mountable! So I resized back to the full size, and that still worked. Bonus: working on the disk image was much faster on my SSD than the original memory card.

I made another fresh image for backup and then repeated these steps on the actual disk (memory card): resize.f2fs to shrink slightly (with -t <some sector amount less than max>) and then resize it fully. Everything worked!

I have logs of the resizes and fscks happy to share though I know this has been reported and seems fixed in current versions.

John


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[Jaegeuk Kim]

Hi John,

On 08/12, John Kehayias via Linux-f2fs-devel wrote:
> Hello,
> 
> Apologies if this is not the correct mailing list, but it was the only one I could find.
> 
> I just resized an f2fs partition to the remaining free space on the disk. This seemed fine and resize.f2fs also seemed fine. However, the partition will no longer mount, due to this message:
> 
> Aug 12 00:10:09 localhost vmunix: [22056.879920] F2FS-fs (sdb2): invalid crc_offset: 0
> Aug 12 00:10:09 localhost vmunix: [22056.881202] F2FS-fs (sdb2): Wrong valid_user_blocks: 7163307, user_block_count: 5227520
> Aug 12 00:10:09 localhost vmunix: [22056.881206] F2FS-fs (sdb2): Failed to get valid F2FS checkpoint
> 
> Is there any way to fix this error?
> 
> I see this error reported before and supposedly fixed in current versions, though I had use f2fs-tools v1.14. I tried using 1.15 and from git, but resize reports nothing to do and fsck is successful. Here is that output:
> 
> Info: [/dev/sdb2] Disk Model: SD  Transcend   
> Info: MKFS version
>   "Linux version 5.5.2-arch1-1 (linux@archlinux) (gcc version 9.2.0 (GCC)) #1 SMP PREEMPT Tue, 04 Feb 2020 18:56:18 +0000"
> Info: FSCK version
>   from "Linux version 5.18.14 (guix@guix) (gcc (GCC) 10.3.0, GNU ld (GNU Binutils) 2.37) #1 SMP PREEMPT_DYNAMIC 1"
>     to "Linux version 5.18.14 (guix@guix) (gcc (GCC) 10.3.0, GNU ld (GNU Binutils) 2.37) #1 SMP PREEMPT_DYNAMIC 1"
> Info: superblock features = 0 : 
> Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
> Info: Segments per section = 1
> Info: Sections per zone = 1
> Info: total FS sectors = 124528600 (60804 MB)
> 	Invalid CP CRC offset: 0
> Info: CKPT version = 6ea966da
> Info: Checked valid nat_bits in checkpoint
> Info: checkpoint state = 81 :  nat_bits unmount
> [FSCK] Check node 1 / 402988 (0.00%)
> [FSCK] Check node 40299 / 402988 (10.00%)
> [FSCK] Check node 80597 / 402988 (20.00%)
> [FSCK] Check node 120895 / 402988 (30.00%)
> [FSCK] Check node 161193 / 402988 (40.00%)
> [FSCK] Check node 201491 / 402988 (50.00%)
> [FSCK] Check node 241789 / 402988 (60.00%)
> [FSCK] Check node 282087 / 402988 (70.00%)
> [FSCK] Check node 322385 / 402988 (80.00%)
> [FSCK] Check node 362683 / 402988 (90.00%)
> [FSCK] Check node 402981 / 402988 (100.00%)
> 
> [FSCK] Max image size: 30099 MB, Free space: 16769654 MB
> [FSCK] Unreachable nat entries                        [Ok..] [0x0]
> [FSCK] SIT valid block bitmap checking                [Ok..]
> [FSCK] Hard link checking for regular file            [Ok..] [0x474]
> [FSCK] valid_block_count matching with CP             [Ok..] [0x6d4dab]
> [FSCK] valid_node_count matching with CP (de lookup)  [Ok..] [0x6262c]
> [FSCK] valid_node_count matching with CP (nat lookup) [Ok..] [0x6262c]
> [FSCK] valid_inode_count matched with CP              [Ok..] [0x61367]
> [FSCK] free segment_count matched with CP             [Ok..] [0x3cc0]
> [FSCK] next block offset is free                      [Ok..]
> [FSCK] fixing SIT types
> [FSCK] other corrupted bugs                           [Ok..]
> 
> Done: 0.000000 secs
> sudo fsck.f2fs /dev/sdb2  6.20s user 5.41s system 4% cpu 4:24.44 total
> 
> I hope someone can help. It seems to just be a metadata issue? Is there a way to manually fix the misreported block count to resolve this?

Yeah, I guess so. Can I get the full fsck log by "fsck.f2fs -d 3"?

> 
> Thanks,
> John
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[Jaegeuk Kim]

On 08/12, John Kehayias via Linux-f2fs-devel wrote:
> Hello again,
> 
> As this is the main disk for a homeserver, I was anxious to get it working again. After reading about what seemed like the same exact problem here: https://old.reddit.com/r/linuxquestions/comments/whomdc/expanded_f2fs_cant_mount/ I thought using a newer f2fs-tools would help (I think I ended up with latest git commit). As I reported earlier though, resize won't do anything (already resized). So I thought about shrinking, though it might be risky.
> 
> First, I tested with a disk image, made with dd and mounted with sudo udisksctl loop-setup -f disk.img. Then I could operate on the loop device with that partition, /dev/loop0p2 in my case. So I tried shrinking, bolstered by the good experiences reported in https://www.mail-archive.com/linux-f2fs-devel@lists.sourceforge.net/msg23214.html Since the size was something like twice the data on it, I didn't worry too much about how much smaller I made it. That was then mountable! So I resized back to the full size, and that still worked. Bonus: working on the disk image was much faster on my SSD than the original memory card.
> 
> I made another fresh image for backup and then repeated these steps on the actual disk (memory card): resize.f2fs to shrink slightly (with -t <some sector amount less than max>) and then resize it fully. Everything worked!
> 
> I have logs of the resizes and fscks happy to share though I know this has been reported and seems fixed in current versions.

Ah, so the recent f2fs-tools addressed your problem, right?

> 
> John
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] How to recover from "Wrong valid_user_blocks"?

[John Kehayias via Linux-f2fs-devel]

Hi,

------- Original Message -------
On Friday, August 12th, 2022 at 2:13 PM, Jaegeuk Kim wrote:

> 
> On 08/12, John Kehayias via Linux-f2fs-devel wrote:
> 
> > Hello again,
> > 
> > As this is the main disk for a homeserver, I was anxious to get it working again. After reading about what seemed like the same exact problem here: https://old.reddit.com/r/linuxquestions/comments/whomdc/expanded_f2fs_cant_mount/ I thought using a newer f2fs-tools would help (I think I ended up with latest git commit). As I reported earlier though, resize won't do anything (already resized). So I thought about shrinking, though it might be risky.
> > 
> > First, I tested with a disk image, made with dd and mounted with sudo udisksctl loop-setup -f disk.img. Then I could operate on the loop device with that partition, /dev/loop0p2 in my case. So I tried shrinking, bolstered by the good experiences reported in https://www.mail-archive.com/linux-f2fs-devel@lists.sourceforge.net/msg23214.html Since the size was something like twice the data on it, I didn't worry too much about how much smaller I made it. That was then mountable! So I resized back to the full size, and that still worked. Bonus: working on the disk image was much faster on my SSD than the original memory card.
> > 
> > I made another fresh image for backup and then repeated these steps on the actual disk (memory card): resize.f2fs to shrink slightly (with -t <some sector amount less than max>) and then resize it fully. Everything worked!
> > 
> > I have logs of the resizes and fscks happy to share though I know this has been reported and seems fixed in current versions.
> 
> 
> Ah, so the recent f2fs-tools addressed your problem, right?
> 

Yes, that's correct. I used the latest (git version I believe, not just 1.15) of f2fs-tools to shrink and re-expand the partition which seems to have fixed it up. If I didn't change the size then resize wouldn't do anything, while fsck did not find or correct any problems. I would have though fsck would be able to fix this, at least with the latest version after this bug was first pointed out and fixed in the resizing.

Anyway, happy to have the partition working again with no ill effects I can see.

John


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel