btrfs-progs test error on fsck/012-leaf-corruption

btrfs-progs test error on fsck/012-leaf-corruption

[Sidong Yang]

Hi!

I found some error when I run unittest code in btrfs-progs.
fsck/012-leaf-corruption test corrupt leaf and check that it's recovered.
but the test was failed and demsg below

[   47.284095] BTRFS error (device loop5): device total_bytes should be at most 27660288 but found 67108864
[   47.284207] BTRFS error (device loop5): failed to read chunk tree: -22
[   47.286465] BTRFS error (device loop5): open_ctree failed

I'm using kernel version 5.11 and there is no error in old version kernel.
I traced the kernel code and found the code that prints error message.
When it tried to mount btrfs, the function read_one_dev() failed.
I found that code added by the commit 3a160a9331112 cause this problem.
The unittest in btrfs-progs should be changed or kernel code should be patched?

Re: btrfs-progs test error on fsck/012-leaf-corruption

[David Sterba]

On Thu, Feb 18, 2021 at 02:56:14AM +0000, Sidong Yang wrote:
> I found some error when I run unittest code in btrfs-progs.
> fsck/012-leaf-corruption test corrupt leaf and check that it's recovered.
> but the test was failed and demsg below
> 
> [   47.284095] BTRFS error (device loop5): device total_bytes should be at most 27660288 but found 67108864
> [   47.284207] BTRFS error (device loop5): failed to read chunk tree: -22
> [   47.286465] BTRFS error (device loop5): open_ctree failed
> 
> I'm using kernel version 5.11 and there is no error in old version kernel.
> I traced the kernel code and found the code that prints error message.
> When it tried to mount btrfs, the function read_one_dev() failed.
> I found that code added by the commit 3a160a9331112 cause this problem.
> The unittest in btrfs-progs should be changed or kernel code should be patched?

The kernel check makes sense. The unit test fails because the image is
restored from a dump and not extended to the full size automatically.

After 'extract_image' the image is

-rw-r--r-- 1 root root 27660288 Feb 19 17:47 good.img.restored
-rw-r--r-- 1 root root   186392 Jul 27  2020 good.img.xz
-rwxr-xr-x 1 root root     2788 Feb 19 17:46 test.sh

but with a manual 'truncate -s 67108864 good.img.restored' the test
succeeds.

btrfs-image enlarges the file but it's probably taking the wrong size

2281         dev_size = key.offset + btrfs_dev_extent_length(path.nodes[0], dev_ext);
2282         btrfs_release_path(&path);
2283
2284         btrfs_set_stack_device_total_bytes(dev_item, dev_size);
2285         btrfs_set_stack_device_bytes_used(dev_item, mdres->alloced_chunks);
2286         ret = fstat(out_fd, &buf);
2287         if (ret < 0) {
2288                 error("failed to stat result image: %m");
2289                 return -errno;
2290         }
2291         if (S_ISREG(buf.st_mode)) {
2292                 /* Don't forget to enlarge the real file */
2293                 ret = ftruncate64(out_fd, dev_size);
2294                 if (ret < 0) {
2295                         error("failed to enlarge result image: %m");
2296                         return -errno;
2297                 }
2298         }

here it's the 'dev_size'. In the superblock dump, the sb.total_size and
sb.dev_item.total_size are both 67108864, which is the correct value.

The size as obtained from the device item in the device tree also matches the
right value

        item 6 key (1 DEV_EXTENT 61210624) itemoff 3667 itemsize 48
                dev extent chunk_tree 3
                chunk_objectid 256 chunk_offset 61210624 length 5898240
                chunk_tree_uuid b2834867-4e78-47ee-9877-94d4e39bda43

Which is the key.offset + length = 61210624 + 5898240 = 67108864.

But the code is not called in restore_metadump because of condition
"btrfs_super_num_devices(mdrestore.original_super) != 1"

Re: btrfs-progs test error on fsck/012-leaf-corruption

[Sidong Yang]

On Fri, Feb 19, 2021 at 05:17:07PM +0100, David Sterba wrote:
> On Thu, Feb 18, 2021 at 02:56:14AM +0000, Sidong Yang wrote:
> > I found some error when I run unittest code in btrfs-progs.
> > fsck/012-leaf-corruption test corrupt leaf and check that it's recovered.
> > but the test was failed and demsg below
> > 
> > [   47.284095] BTRFS error (device loop5): device total_bytes should be at most 27660288 but found 67108864
> > [   47.284207] BTRFS error (device loop5): failed to read chunk tree: -22
> > [   47.286465] BTRFS error (device loop5): open_ctree failed
> > 
> > I'm using kernel version 5.11 and there is no error in old version kernel.
> > I traced the kernel code and found the code that prints error message.
> > When it tried to mount btrfs, the function read_one_dev() failed.
> > I found that code added by the commit 3a160a9331112 cause this problem.
> > The unittest in btrfs-progs should be changed or kernel code should be patched?
> 
> The kernel check makes sense. The unit test fails because the image is
> restored from a dump and not extended to the full size automatically.
> 
> After 'extract_image' the image is
> 
> -rw-r--r-- 1 root root 27660288 Feb 19 17:47 good.img.restored
> -rw-r--r-- 1 root root   186392 Jul 27  2020 good.img.xz
> -rwxr-xr-x 1 root root     2788 Feb 19 17:46 test.sh
> 
> but with a manual 'truncate -s 67108864 good.img.restored' the test
> succeeds.
> 
> btrfs-image enlarges the file but it's probably taking the wrong size
> 
> 2281         dev_size = key.offset + btrfs_dev_extent_length(path.nodes[0], dev_ext);
> 2282         btrfs_release_path(&path);
> 2283
> 2284         btrfs_set_stack_device_total_bytes(dev_item, dev_size);
> 2285         btrfs_set_stack_device_bytes_used(dev_item, mdres->alloced_chunks);
> 2286         ret = fstat(out_fd, &buf);
> 2287         if (ret < 0) {
> 2288                 error("failed to stat result image: %m");
> 2289                 return -errno;
> 2290         }
> 2291         if (S_ISREG(buf.st_mode)) {
> 2292                 /* Don't forget to enlarge the real file */
> 2293                 ret = ftruncate64(out_fd, dev_size);
> 2294                 if (ret < 0) {
> 2295                         error("failed to enlarge result image: %m");
> 2296                         return -errno;
> 2297                 }
> 2298         }
> 
> here it's the 'dev_size'. In the superblock dump, the sb.total_size and
> sb.dev_item.total_size are both 67108864, which is the correct value.
> 
> The size as obtained from the device item in the device tree also matches the
> right value
> 
>         item 6 key (1 DEV_EXTENT 61210624) itemoff 3667 itemsize 48
>                 dev extent chunk_tree 3
>                 chunk_objectid 256 chunk_offset 61210624 length 5898240
>                 chunk_tree_uuid b2834867-4e78-47ee-9877-94d4e39bda43
> 
> Which is the key.offset + length = 61210624 + 5898240 = 67108864.
> 
> But the code is not called in restore_metadump because of condition
> "btrfs_super_num_devices(mdrestore.original_super) != 1"

Thanks for reply. I read the commit 73dd4e3c87c and I understood a
purpose of the commit. but I'm confused the code block that isn't called
in restore_metadump should be called in multi device?

I also checked that test goes good when removing the condition in
restore_metadump().