Re: [4.4.1] btrfs-transacti frequent high CPU usage despite little fragmentation

[Ole Langbehn <neurolabs.de@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3271 bytes --]

Duncan,

thanks again for your effort, I highly appreciate it.

On 19.03.2016 00:06, Duncan wrote:
> autodefrag

Got it, thanks.

> Nocow interacts with snapshots.  

Thanks for presenting that in that much detail.

> What can happen then, and used to happen frequently before 3.17, tho much 
> less frequently but it can still happen now, is that over time and with 
> use, the filesystem will allocate all available space as one type, 
> typically data chunks, and then run out of space in the other type of 
> chunk, typically metadata, and have no unallocated space from which to 
> allocate more.   So you'll have lots of space left, but it'll be all tied 
> up in only partially used chunks of the one type and you'll be out of 
> space in the other type.
> 
> And by the time you actually start getting ENOSPC errors as a result of 
> the situation, there's often too little space left to create even the one 
> additional chunk necessary for a balance to write the data from other 
> chunks into, in ordered to combine some of the less used chunks into 
> fewer chunks at 100% usage (but for the last one, of course).
> 
> And you were already in a tight spot in that regard and may well have had 
> errors if you had simply tried an unfiltered balance, because data chunks 
> are typically 1 GiB in size (and can be upto 10 GiB in some circumstances 
> on large enough filesystems, tho I think the really large sizes require 
> multi-device), and you were down to 300-ish MiB of unallocated space, not 
> enough to create a new 1 GiB data chunk.
>
> And considering the filesystem's near terabyte scale, to be down to under 
> a GiB of unallocated space is even more startling, particularly on newer 
> kernels where empty chunks are normally reclaimed automatically (tho as 
> the usage=0 balances reclaimed some space for you, obviously not all of 
> them had been reclaimed in your case).

As I said before, this fs has (with 99.9% probability) never seen
kernels <3.18. I'm curious why it came to the point of only having
300MiB unallocated, or what could potentially lead to this.

> Meanwhile, discussion in another thread reminded me of another factor, 
> quotas.

Sure thing I had quotas enabled without the direct need for them ;).
I've been using

https://github.com/agronick/btrfs-size/

which uses quotas in order to display human readable snapshot sizes.

As a wrap up to the chunk allocation issue (the balance has finished):

# btrfs filesystem usage /
Overall:
    Device size:                 915.32GiB
    Device allocated:            169.04GiB
    Device unallocated:          746.28GiB
    Device missing:                  0.00B
    Used:                        155.51GiB
    Free (estimated):            758.33GiB      (min: 758.33GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,single: Size:164.01GiB, Used:151.95GiB
   /dev/sda2     164.01GiB

Metadata,single: Size:5.00GiB, Used:3.55GiB
   /dev/sda2       5.00GiB

System,single: Size:32.00MiB, Used:48.00KiB
   /dev/sda2      32.00MiB

Unallocated:
   /dev/sda2     746.28GiB

Cheers,

Ole


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]