Re: ENOSPC with mkdir and rename

[Austin S Hemmelgarn <ahferroin7@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4437 bytes --]

On 2014-08-04 09:17, Peter Waller wrote:
> For anyone else having this problem, this article is fairly useful for
> understanding disk full problems and rebalance:
> 
> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
> 
> It actually covers the problem that I had, which is that a rebalance
> can't take place because it is full.
> 
> I still am unsure what is really wrong with this whole situation. Is
> it that I wasn't careful to do a rebalance when I should have been
> doing? Is it that BTRFS doesn't do a rebalance automatically when it
> could in principle?
> 
> It's pretty bad to end up in a situation (with spare space) where the
> only way out is to add more storage, which may be impractical,
> difficult or expensive.
I really disagree with the statement that adding more storage is
difficult or expensive, all you need to do is plug in a 2G USB flash
drive, or allocate a ramdisk, and add the device to the filesystem only
long enough to do a full balance.
> 
> The other thing that I still don't understand I've seen repeated in a
> few places, from the above article:
> 
> "because the filesystem is only 55% full, I can ask balance to rewrite
> all chunks that are more than 55% full"
> 
> Then he uses `btrfs balance start -dusage=55 /mnt/btrfs_pool1`. I
> don't understand the relationship between "the FS is 55% full" and
> "chunks more than 55% full". What's going on here?
To understand this, you have to understand that BTRFS uses a two level
allocation scheme, at the top level, you have chunks, which are
contiguous regions of the disk that get used for storing a specific
block type.  For data chunks, these default to 1G in size, for metadata,
they default to 256M in size.  When a filesystem is created, you get the
minimum number of chunks of each type based on the replication profiles
chosen for each chunk type; with no extra options, this means 1 data
chunk and 2 metadata chunks for a single disk filesystem.  Within each
chunk, BTRFS then allocates and frees individual blocks on demand, these
blocks are the analogue of blocks in most other filesystems.  When there
are no free blocks in any chunks of a given type, BTRFS then allocates
new chunks of that type based on the replication profile.  Unlike blocks
however, chunks aren't freed automatically (there are good reasons for
this behavior, but they are kind of long to explain here), this is where
balance comes in, it takes all of the blocks in the filesystem, and
sends them back through the block allocator.  This usually causes all of
the free blocks to end up in a single chunk, and frees the unneeded chunks.

When someone talks about a chunk being x% full, they mean that x% of the
space in that chunk is used by allocated blocks.  Talking about how full
the filesystem is can get tricky because of the replication profiles,
but the usual consensus is to treat that as the percentage of the
filesystem that contains blocks that are being used.

It should say LESS than 55% full in the various articles, as the
-dusage=x option tells balance to only consider chunks that are less
than 55% full for balancing.  In general, if your filesystem is totally
full, you should use numbers starting with 0, and working your way up
from there.  You may even get lucky, and using -dusage=0 -musage=0 may
free up enough chunks that you don't need to add more storage.
> 
> I conclude that now since I have added more storage, the rebalance
> won't fail and if I keep rebalancing from a cron job I won't hit this
> problem again (unless the filesystem fills up very fast! what then?).
> I don't know however what value to assign to `-dusage` in general for
> the cron rebalance. Any hints?
I've found that something between 25 and 50 tends to do well, much
outside of that range and you start to get diminishing returns.  The
exact value tends to be more personal preference, I use 25 on most of my
systems, because I don't like saturating the disks with I/O for very
long.  Do make sure however to add -musage=x as well, metadata also
should be balanced (especially if you have very large numbers of small
files).
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]