Way to force allocation of more metadata?

Way to force allocation of more metadata?

[E V]

It would be nice if there was an easy way to tell btrfs to allocate
another metadata chunk. For example, the below fs is full due to
exhausted metadata:

    Device size:                1013.28GiB
    Device allocated:           1013.28GiB
    Device unallocated:            2.00MiB
    Device missing:                  0.00B
    Used:                        981.94GiB
    Free (estimated):             15.16GiB      (min: 15.16GiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              510.31MiB      (used: 0.00B)

             Data      Metadata  System
Id Path      RAID1     RAID1     RAID1    Unallocated
-- --------- --------- --------- -------- -----------
 1 /dev/sdv1 505.63GiB   1.00GiB  8.00MiB     1.00MiB
 2 /dev/sdw1 505.63GiB   1.00GiB  8.00MiB     1.00MiB
-- --------- --------- --------- -------- -----------
   Total     505.63GiB   1.00GiB  8.00MiB     2.00MiB
   Used      490.47GiB 510.88MiB 96.00KiB

I can delete a multi GB file and get several GB of unallocated space,
however if I try and copy big files to it again the same exact thing
happens. However, if I play with balance and deleting files and such
and manage to get it to allocate another metadata chunk while there is
unallocated space then the filesystem will happily fill up all of the
data chunks. Failing an automatic allocation out of global reserve, or
saving metadata as soon as unallocated space is available it would be
nice if I could just delete a file and then tell btrfs to allocate
more metadata immediately. Makes sense? No idea how easy this would be
to do, but seems like it should be a simple thing btrfs file could do.

Re: Way to force allocation of more metadata?

[Austin S. Hemmelgarn]

On 2017-02-16 15:13, E V wrote:
> It would be nice if there was an easy way to tell btrfs to allocate
> another metadata chunk. For example, the below fs is full due to
> exhausted metadata:
>
>     Device size:                1013.28GiB
>     Device allocated:           1013.28GiB
>     Device unallocated:            2.00MiB
>     Device missing:                  0.00B
>     Used:                        981.94GiB
>     Free (estimated):             15.16GiB      (min: 15.16GiB)
>     Data ratio:                       2.00
>     Metadata ratio:                   2.00
>     Global reserve:              510.31MiB      (used: 0.00B)
>
>              Data      Metadata  System
> Id Path      RAID1     RAID1     RAID1    Unallocated
> -- --------- --------- --------- -------- -----------
>  1 /dev/sdv1 505.63GiB   1.00GiB  8.00MiB     1.00MiB
>  2 /dev/sdw1 505.63GiB   1.00GiB  8.00MiB     1.00MiB
> -- --------- --------- --------- -------- -----------
>    Total     505.63GiB   1.00GiB  8.00MiB     2.00MiB
>    Used      490.47GiB 510.88MiB 96.00KiB
>
> I can delete a multi GB file and get several GB of unallocated space,
> however if I try and copy big files to it again the same exact thing
> happens. However, if I play with balance and deleting files and such
> and manage to get it to allocate another metadata chunk while there is
> unallocated space then the filesystem will happily fill up all of the
> data chunks. Failing an automatic allocation out of global reserve, or
> saving metadata as soon as unallocated space is available it would be
> nice if I could just delete a file and then tell btrfs to allocate
> more metadata immediately. Makes sense? No idea how easy this would be
> to do, but seems like it should be a simple thing btrfs file could do.
The potentially tricky bit about this is that BTRFS (at least, with 
recent kernels) will deallocate completely empty chunks.  The horribly 
ironic part of this is that that behavior got added to help avoid 
situations like this.  There is technically logic to allocate extra 
metadata chunks for every few data chunks that get allocated (and I 
think the reverse too), but because of the auto-deallocation behavior, 
this ends up just spinning and wasting cycles (and bandwidth).  I would 
personally love to see the following happen in regard to this all:
1. Add a switch to disable the auto-deallocation (ideally a mount 
option).  This bit _should_ be pretty easy, but I'm not certain.
2. Disable the auto-allocation of chunks of type X every N chunks of 
type Y that get allocated when the auto-deallocation is enabled.  This 
one should in theory be pretty easy too.
3. Add a tool (not sure where exactly makes the most sense) to force 
allocation of a specific chunk type.  This would also be insanely useful 
for debugging, but is probably the hardest part (would likely need a new 
ioctl).
4. Possibly add an option to reserve some percentage of the space to 
only be used for System and Metadata chunks.  This would help prevent 
this kind of thing from happening.  I'm not entirely sold on this right 
now, but it _seems_ like a generically good idea.

Sadly, I don't currently have the time to work on any of that myself, 
and there are other things that are higher priorities, so I don't know 
when (if ever) this may actually happen.  I might have some time to look 
into the first two things I listed some time soon, but I don't know how 
soon that might be.

Re: Way to force allocation of more metadata?

[Duncan]

E V posted on Thu, 16 Feb 2017 15:13:40 -0500 as excerpted:

> I can delete a multi GB file and get several GB of unallocated space,
> however if I try and copy big files to it again the same exact thing
> happens. However, if I play with balance and deleting files and such and
> manage to get it to allocate another metadata chunk while there is
> unallocated space then the filesystem will happily fill up all of the
> data chunks. Failing an automatic allocation out of global reserve, or
> saving metadata as soon as unallocated space is available it would be
> nice if I could just delete a file and then tell btrfs to allocate more
> metadata immediately. Makes sense? No idea how easy this would be to do,
> but seems like it should be a simple thing btrfs file could do.

You should be able to trigger metadata allocation by writing enough tiny 
files, say 1 KiB each.  Small files (typically upto slightly under 2 KiB) 
are inlined into the metadata, thus using it up.  Writing enough of them 
in for instance a shellscript loop to trigger a new metadata chunk 
allocation shouldn't be too difficult, but keep in mind when doing the 
math that global reserve is allocated from metadata as well, tho it's 
single even when metadata is dup or (as here) raid1.

Also, if you're looking at the space output as you write them, keep in 
mind the btrfs 30-second by default commit timing, and call btrfs fi sync 
(or just sync but that's system-wide) on the filesystem every N files or 
so, before checking the usage, so it's accurate without waiting 30 
seconds for the commit-clock to timeout.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman