Re: [btrfs-progs] Bug in mkfs.btrfs -r

[Duncan <1i5t5.duncan@cox.net>]

Duncan posted on Sat, 02 Sep 2017 04:03:06 +0000 as excerpted:

> Austin S. Hemmelgarn posted on Fri, 01 Sep 2017 10:07:47 -0400 as
> excerpted:
> 
>>  On 2017-09-01 09:54, Qu Wenruo wrote:
>>> 
>>> On 2017年09月01日 20:47, Austin S. Hemmelgarn wrote:
> 
>>>> On 2017-09-01 08:19, Qu Wenruo wrote:
>>>>>
>>>>> Current kernel (and btrfs-progs also tries to follow kernel chunk
>>>>> allocator's behavior) will not make a chunk larger than 10% of RW
>>>>> space.
>>>>> So for small filesystem chunk won't be too maximally sized.
> 
>>>> Are you sure about this?  I've got a couple of sub 10GB BTRFS volumes
>>>> that definitely have more than one 1GB data chunk.
>>> 
>>> Yes, check the following code:
>>> 
>>>          /* we don't want a chunk larger than 10% of writeable
>>>          space */
>>>          max_chunk_size = min(div_factor(fs_devices->total_rw_bytes,
>>>          1),
>>>                               max_chunk_size);
>>>
>>> Which is in __btrfs_alloc_chunk() function in fs/btrfs/volumes.c
> 
>> Huh, I may have the remnants of an old bug present on those filesystems
>> then, I'll have to look further into this.
> 
> Please do.

> Meanwhile, I might have some free time to do my own checks tomorrow.
> It'd be worth it just to my own peace of mind to settle the issue,
> since it frustrated me for so long.

Just finished testing, and no, as of btrfs-progs 4.12, mkfs.btrfs does 
*NOT* always limit chunk sizes to 1/10 filesystem size, despite smaller 
chunk sizes being available.

Try this.

256 MiB partition.

mkfs.btrfs -d dup -m dup -O extref,skinny-metadata,no-holes --mixed 
<device>

mkfs.btrfs will create a 64 MiB mixed-mode chunk, a quarter the 256 MiB 
filesystem size, duped to 128 MiB, half the filesystem size.

That chunk cannot be balanced because there's no room for the write-into 
chunk and its dup on the filesystem, due to the system chunk of several 
MiB taking up part of the other half.

This despite further writes creating smaller chunks, so it's definitely 
possible to have 32 MiB and I believe 16 MiB chunks (at least, don't know 
if smaller is possible).

So as of -progs 4.12 at least, whatever code is there to try to keep 
mkfs.btrfs max chunk size under 10% of the filesystem size, is broken, at 
least for some configurations (check mixed-mode, sub-GiB).

So I'm glad I upsized /boot to half a GiB when I upgraded SSDs and redid 
my layout.  At least with half a gig, 64 MiB (still 12.5%, over the 10% 
limit) doubled to 128 MiB is still only a quarter the filesystem size, so 
rebalancing is actually possible.

(FWIW my /var/log is still 256 MiB per device, but it's raid1, so only a 
single copy of each chunk per device, and the 64 MiB chunk size is only a 
quarter of the device, so it can at least still be balanced, even with 
chunks more than double the broken 10% ceiling.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman