Re: Fixing Btrfs Filesystem Full Problems typo?

[Robert White <rwhite@pobox.com>]

On 12/11/2014 01:55 AM, Patrik Lundquist wrote:
> On 11 December 2014 at 09:42, Robert White <rwhite@pobox.com> wrote:
>> On 12/10/2014 05:36 AM, Patrik Lundquist wrote:
>>>
>>> On 10 December 2014 at 13:17, Robert White <rwhite@pobox.com> wrote:
>>>>
>>>> On 12/09/2014 11:19 PM, Patrik Lundquist wrote:
>>>>>
>>>>>
>>>> BUT FIRST UNDERSTAND: you do _not_ need to balance a newly converted
>>>> filesystem. That is, the recommended balance (and recursive defrag) is
>>>> _not_
>>>> a useability issue, its an efficiency issue.
>>>
>>>
>>> But if I can't start with an efficient filesystem I'd rather start
>>> over now/soon. I intend to add four more old disks for a RAID1 and it
>>> will be problematic to start over later on (I'd have to buy new, large
>>> disks).
>>
>>
>> Nope, not an issue.
>>
>> When you add the space and rebalance with the conversions by adding all
>> those other disks and such it will _completely_ _obliterate_ the current
>> balance.
>
> But if the issue is too large extents, why would they fit on any added
> btrfs space?

Because that added btrfs space will be _empty_. It's not that the extent 
is "too big" by some absolute measure. It's that it's too big to fit in 
the available space at the _extent_ _tree_ level.

You can't put two feet into one shoe.



>> You are cleaning the house before the maid comes.
>
> Indeed, as a health check. And the patient is slightly ill.

Not really...

So lets say I have a bunch of things that are all size 10-inches

And lets say I space them along a rail with 9-inches between each object.

And I glue them down (because Copy On Write only)

And I do that until the rail is "full", say it takes 100 to fill the rail.

So I still have 900 inches of "free space" but I don't have _any_ _more_ 
_room_ available if I need to mount another 10-inch item.

There's plenty of space but there is no room.

This is what youve got going on.

The conversion hoovered up all the block groups from the ext4 donor 
image more-or-less, and then it built the metadata blocks

(see btrfs-convert at about line 1486)
/* for each block group, create device extent and chunk item */
etc...




>> If you are going to add four more volumes, if those volumes are big enough
>> just make a new filesystem on them then copy the files over.
>
> As it looks now, I will, but I also think there's a bug which I'm
> trying to zero in on.

It doesn't exist. There is no bug that I can see from anything you've shown.

You are confusing the word "extent" as used in ext4, which is a per-file 
thing, with the word "extent" as used differently in btrfs which is a 
raw storage region into which other structures or data is placed.

>>> I deleted the subvolume after being satisfied with the conversion,
>>> defragged recursively, and balanced. In that order.
>>
>> Yea, but your file system is full and you are out of space so get on with
>> the adding space.
>
> I don't think it is full. balance -musage=100 -dusage=99 completes
> with ~1.5TB free space. The remaining unbalanced data is using full or
> close to full blocks. Still can't speak for contiguous space though.
>
>
>> (looking back through my mail spool) You haven't sent the output of /bin/df
>> or btrfs fi df yet, I'd like to see what those two commands say.
>
> I have posted these before, but not /bin/df (no access at the moment).

Ah, yes, I remember these, but the /bin/df is what's going to be 
dispositive.


> btrfs fi show
> Label: none  uuid: 770fe01d-6a45-42b9-912e-
> e8f8b413f6a4
>      Total devices 1 FS bytes used 1.35TiB
>      devid    1 size 2.73TiB used 1.36TiB path /dev/sdc1
>
>
> btrfs fi df /mnt
> Data, single: total=1.35TiB, used=1.35TiB
> System, single: total=32.00MiB, used=112.00KiB
> Metadata, single: total=3.00GiB, used=1.55GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
> btrfs check /dev/sdc1
> Checking filesystem on /dev/sdc1
> UUID: 770fe01d-6a45-42b9-912e-e8f8b413f6a4
> found 825003219475 bytes used err is 0
> total csum bytes: 1452612464
> total tree bytes: 1669943296
> total fs tree bytes: 39600128
> total extent tree bytes: 52903936
> btree space waste bytes: 79921034
> file data blocks allocated: 1487627730944
>   referenced 1487627730944




>>>> This would
>>>> be quadruply true if you'd tweaked the block group ratios when you made
>>>> the original file system.
>>>
>>> Ext4 created with defaults, but I think it has been completely full at one
>>> time.
>>
>> Did you use e4defrag before you did the conversion or is this the result of
>> converting chaos most profound?
>
> Didn't use e4defrag.

Probably doesn't matter. Now that I've read more of btrfs-convert.c I 
think I can see how this is shaking out. e4defrag might have packed the 
block groups tighter but it doesn't really try to maximize free space 
within the extent.

>>>> Think of the time and worry you'd have saved if you'd copied the thing in
>>>> the first place. 8-)
>>>
>>> But then I wouldn't learn as much. :-)
>>
>> Learning not to cut corners is a lesson... 8-)
>
> This is more of an experiment than cutting corners, but yeah.
>
>
>> TRUTH BE TOLD :: After two very "eventful" conversions not too long ago I
>> just don't do those any more. The total amount of time I "saved" by not
>> copying the files was in the negative numbers before I just copied the files
>> onto an external media and reformatted and restored.
>
> Conversion probably should be discouraged on the wiki then.

I didn't pursue the wiki on the matter, but conversion of anything to 
anything always requires living with the limits of both, at least to 
start. In this case you are suffering under the burden of the block 
group alignment and layout that was selected by mkfs.ext4, which is 
based on assumptions optimal to ext4.

Systems are _rarely_ replaced by other systems based on the same 
assumptions.

As a terrible aside example, EXT4 says it can support file extent sizes 
up to two gig. But that assumes your CPU memory page size is 64k. On a 
typical Intel PC the page size is 4k, so your maximum extent size is 
1/8th that size. I filed a bug on that some time ago because e4defrag 
output didn't take that into account.

e.g. http://sourceforge.net/p/e2fsprogs/bugs/314/

The mythology of that two-gig file extent has people allocating VM drive 
stripes and rdbms files (etc) in two-gig chunks thinking they are 
optimally alinging things with their drive allocations. But when they do 
it on an intel box they are wrong. Those extents should have been 128Meg 
if they wanted one file equals one extent layouts.

So assumptions in systems can become pernicious, and when you try to do 
any sort of in-place conversion you are likely to end up with the least 
of all worlds.

The devils are always in the details.

Heck, we are still dragging around head/track/sector disk geometry 
nonsense despite variable pitch recording performed on modern drives. 
Thats because we just keep converting old ideas to new.

My "eventful" conversions of those two disks may well have (and probably 
were) completely my own doing. It's a poor craftsman that blames his tools.

My house is a mess. My computers tend to be scrupulously organized. And 
the result from btrfs-convert just doesn't seem optimal for all future 
geometries. After all if default extent size of 0x1000 and 0x8000 were 
chosen for optimal cause (instead of beauty) ending up with a buch of 
two-gig-ish extents would oppose that cause.

It just feels ookie to use btrfs-convert in _my_ _humble_ _opinion_.


>> It's like a choose-your-own-adventure book! 8-)
>
> I like that! :-)