[devel branch]Btrfs-convert assert at volumes.c:1846

[devel branch]Btrfs-convert assert at volumes.c:1846

[Qu Wenruo]

Hi David and all,

Recently I'm reworking the btrfs-convert chunk allocater to make it able 
to separate data and meta chunk.

But when I was testing, I found that, your devel branch, with the 
following commit just fails to convert an empty ext4 filesystem:

commit 0002085c53a5ac9ad1c538bfc3e39c80891c4a1a
Author: Zhao Lei <zhaolei@cn.fujitsu.com>
Date:   Mon Oct 12 21:23:04 2015 +0800

     btrfs-progs: fragments: use btrfs_open_dir for btrfs-fragments command

     We can use btrfs_open_dir() to check whether target dir is
     in btrfs's mount point before open, instead of checking it in
     deeper code, and return fuzzy error message.

     Before patch:
       ./btrfs-fragments -o 123 /mnt/tmp1
       ERROR: can't perform the search

     After patch:
       # ./btrfs-fragments -o 123 /mnt/tmp1
       ERROR: not a btrfs filesystem: /mnt/tmp1

     Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
     Signed-off-by: David Sterba <dsterba@suse.com>


The error output is:
create btrfs filesystem:
	blocksize: 4096
	nodesize:  16384
	features:  extref, skinny-metadata (default)
volumes.c:1846: btrfs_read_sys_array: Assertion failed.
./btrfs-convert(btrfs_read_sys_array+0x1b8)[0x42422c]
./btrfs-convert(btrfs_setup_chunk_tree_and_device_map+0x41)[0x41176b]
./btrfs-convert[0x411a22]
./btrfs-convert(open_ctree_fd+0x10)[0x411b59]
./btrfs-convert(main+0x1814)[0x4382d7]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f5571479610]
./btrfs-convert(_start+0x29)[0x407979]

Any ideas?
I hope it won't block me too long to test my rework.

Thanks,
Qu

Re: [devel branch]Btrfs-convert assert at volumes.c:1846

[David Sterba]

On Mon, Oct 19, 2015 at 05:30:16PM +0800, Qu Wenruo wrote:
> Recently I'm reworking the btrfs-convert chunk allocater to make it able 
> to separate data and meta chunk.
> 
> But when I was testing, I found that, your devel branch, with the 
> following commit just fails to convert an empty ext4 filesystem:
> 
> commit 0002085c53a5ac9ad1c538bfc3e39c80891c4a1a
> Author: Zhao Lei <zhaolei@cn.fujitsu.com>
> Date:   Mon Oct 12 21:23:04 2015 +0800
> 
>      btrfs-progs: fragments: use btrfs_open_dir for btrfs-fragments command

That's the top commit, but it's not causing the breakage. I've bisected
the breakage to the patch

commit 893b7a6815a735449613afbb4040f486a4a4e1d0
Author: Qu Wenruo <quwenruo@cn.fujitsu.com>
Date:   Wed May 13 17:15:34 2015 +0800

    btrfs-progs: Read the whole superblock instead of struct btrfs_super_block.

Looks familiar? :)

The patch caused some trouble in the past (on ppc64 machines) and I've
been holding it back due to that. The plan is to add it to 4.3 devel
branch and get it fixed before the final release, so the convert could
fail in the meantime. I'm trying to put together all pending branches
for the next release so I can't say when the convert will be fixed.

Re: [devel branch]Btrfs-convert assert at volumes.c:1846

[Qu Wenruo]

在 2015年10月19日 19:21, David Sterba 写道:
> On Mon, Oct 19, 2015 at 05:30:16PM +0800, Qu Wenruo wrote:
>> Recently I'm reworking the btrfs-convert chunk allocater to make it able
>> to separate data and meta chunk.
>>
>> But when I was testing, I found that, your devel branch, with the
>> following commit just fails to convert an empty ext4 filesystem:
>>
>> commit 0002085c53a5ac9ad1c538bfc3e39c80891c4a1a
>> Author: Zhao Lei <zhaolei@cn.fujitsu.com>
>> Date:   Mon Oct 12 21:23:04 2015 +0800
>>
>>       btrfs-progs: fragments: use btrfs_open_dir for btrfs-fragments command
>
> That's the top commit, but it's not causing the breakage. I've bisected
> the breakage to the patch
>
> commit 893b7a6815a735449613afbb4040f486a4a4e1d0
> Author: Qu Wenruo <quwenruo@cn.fujitsu.com>
> Date:   Wed May 13 17:15:34 2015 +0800
>
>      btrfs-progs: Read the whole superblock instead of struct btrfs_super_block.
>
> Looks familiar? :)
>
> The patch caused some trouble in the past (on ppc64 machines) and I've
> been holding it back due to that. The plan is to add it to 4.3 devel
> branch and get it fixed before the final release, so the convert could
> fail in the meantime. I'm trying to put together all pending branches
> for the next release so I can't say when the convert will be fixed.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Thanks for the bisect, David.

Very interesting, I'll try to fix it in recent days at least to make 
sure that btrfs-convert can be executed without assert.

Hopes the convert rework for separated chunk profile will arrive in time.

Thanks,
Qu

Re: [devel branch]Btrfs-convert assert at volumes.c:1846

[Qu Wenruo]

Qu Wenruo wrote on 2015/10/19 20:38 +0800:
> 在 2015年10月19日 19:21, David Sterba 写道:
>> On Mon, Oct 19, 2015 at 05:30:16PM +0800, Qu Wenruo wrote:
>>> Recently I'm reworking the btrfs-convert chunk allocater to make it able
>>> to separate data and meta chunk.
>>>
>>> But when I was testing, I found that, your devel branch, with the
>>> following commit just fails to convert an empty ext4 filesystem:
>>>
>>> commit 0002085c53a5ac9ad1c538bfc3e39c80891c4a1a
>>> Author: Zhao Lei <zhaolei@cn.fujitsu.com>
>>> Date:   Mon Oct 12 21:23:04 2015 +0800
>>>
>>>       btrfs-progs: fragments: use btrfs_open_dir for btrfs-fragments
>>> command
>>
>> That's the top commit, but it's not causing the breakage. I've bisected
>> the breakage to the patch
>>
>> commit 893b7a6815a735449613afbb4040f486a4a4e1d0
>> Author: Qu Wenruo <quwenruo@cn.fujitsu.com>
>> Date:   Wed May 13 17:15:34 2015 +0800
>>
>>      btrfs-progs: Read the whole superblock instead of struct
>> btrfs_super_block.
>>
>> Looks familiar? :)
>>
>> The patch caused some trouble in the past (on ppc64 machines) and I've
>> been holding it back due to that. The plan is to add it to 4.3 devel
>> branch and get it fixed before the final release, so the convert could
>> fail in the meantime. I'm trying to put together all pending branches
>> for the next release so I can't say when the convert will be fixed.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> Thanks for the bisect, David.
>
> Very interesting, I'll try to fix it in recent days at least to make
> sure that btrfs-convert can be executed without assert.
>
> Hopes the convert rework for separated chunk profile will arrive in time.
>
> Thanks,
> Qu

Finally, the bug is found.

A SUPER AWKWARD TYPO....

See the patch just submitted:
[PATCH] btrfs-progs: Fix a typo which causes super_copy corrupted

And, I think that's also the reason why PPC64 report errors.
But unfortunately, there is no PPC64 machine here for me to test.

Hopes someone can help testing it on PPC64.

Thanks,
Qu

Re: [devel branch]Btrfs-convert assert at volumes.c:1846

[David Sterba]

On Tue, Oct 20, 2015 at 10:33:37AM +0800, Qu Wenruo wrote:
> Finally, the bug is found.

Great, thanks!
> 
> A SUPER AWKWARD TYPO....

Happens ... :)

> See the patch just submitted:
> [PATCH] btrfs-progs: Fix a typo which causes super_copy corrupted
> 
> And, I think that's also the reason why PPC64 report errors.
> But unfortunately, there is no PPC64 machine here for me to test.

For now we can use convert and mkfs to test it a bit. Convert crashes on
4k sectors, and with "mkfs.btrfs -n 64k -s 64k --rootdir " we can
partially simulate the ppc64 parameters.