Re: btrfs crash on armv7

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2021/4/8 下午7:15, riteshh wrote:
>
> Please excuse my silly queries here.
>
> On 21/04/08 04:38PM, Qu Wenruo wrote:
>>
>>
>> On 2021/4/8 下午4:16, Joe Hermaszewski wrote:
>>> It took a while but I managed to get hold of another one of these
>>> arm32 boards. Very disappointingly this exact "bitflip" is still
>>> present (log enclosed).
>>
>> Yeah, we got to the conclusion it's not bitflip, but completely 32bit
>> limit on armv7.
>>
>> For ARMv7, it's a 32bit system, where unsigned long is only 32bit.
>>
>> This means, things like page->index is only 32bit long, and for 4K page
>> size, it also means all filesystems (not only btrfs) can only utilize at
>> most 16T bytes.
>>
>> But there is pitfall for btrfs, btrfs uses its internal address space
>> for its meatadata, and the address space is U64.
>
> Can you pls point me to the code you are referring here?
> So IIUC, you mean since page->index can hold a value which can be upto 32bit in
> size so the maximum FS address range which can be accessed is 16T.
> This should be true in general for any FS no?


The code is in definition of "struct page",from "include/linux/mm_types.h"

Yes, for all fs.

But no other fs has another internal address space, unlike btrfs.

Btrfs uses its internal space to implement multi-device support.

>
>>
>> And furthermore, for btrfs it can have metadata at bytenr way larger
>> than the total device size.
>
> Is this because of multi-device support?

Yes.

>
>> This is possible because btrfs maps part of its address space to real
>> disks, thus it can have bytenr way larger than device size.
>
> Please a code pointing to that will help me understand this better.
> Thanks.

You need to understand btrfs chunk tree first.

Each btrfs chunk item is a mapping from btrfs logical address to each
real device.

The easiest way to understand it is not code, but "btrfs ins dump-tree
-t chunk <device>" to experience it by yourself.

>
>>
>> But this brings to a problem, 32bit Linux can only handle 16T, but in
>> your case, some of your metadata is already beyond 16T in btrfs address
>> space.
>
> Sorry I am not much aware of the history. Was this disk mkfs on 64-bit system
> and then connected to a 32bit board?

Possible.

But there are other cases to go beyond that limit, especially with balance.

So I'm not confident enough to say what's the exact event to make the fs
cross the line.

>
> This also brings me to check with you about other filesystems.
> See the capacity section from below wiki[1]. Depending upon the host OS
> limitation on the max size of the filesystem may vary right?
>
> [1] https://en.wikipedia.org/wiki/XFS

The last time Dave Chineer said, for xfs larger than 16T, 32bit kernel
will just refuse to mount.

Thanks,
Qu

>
> -ritesh
>
>>
>> Then a lot of things are going to be wrong.
>>
>> I have submitted a patch to do extra check, at least allowing user to
>> know this is the limit of 32bit:
>> https://patchwork.kernel.org/project/linux-btrfs/patch/20210225011814.24009-1-wqu@suse.com/
>>
>> Unfortunately, this will not help existing fs though.
>>