Re: [Not TLS] Re: Reducing impact of periodic btrfs balance

[Graham Cobb <g.btrfs@cobb.uk.net>]

On 19/05/16 05:09, Duncan wrote:
> So to Graham, are these 1.5K snapshots all of the same subvolume, or 
> split into snapshots of several subvolumes?  If it's all of the same 
> subvolume or of only 2-3 subvolumes, you still have some work to do in 
> terms of getting down to recommended snapshot levels.  Also, if you have 
> quotas on and don't specifically need them, try turning them off and see 
> if that alone makes it workable.

I have just under 20 subvolumes but the snapshots are only taken if
something has changed (actually I use btrbk: I am not sure if it takes
the snapshot and then removes it if nothing changed or whether it knows
not to even take it).  The most frequently changing subvolumes have just
under 400 snapshots each.  I have played with snapshot retention and
think it unlikely I would want to reduce it further.

I have quotas turned off.  At least, I am not using quotas -- how can I
double check it is really turned off?

I know that very large numbers of snapshots are not recommended, and I
expected the balance to be slow.  I was quite prepared for it to take
many days.  My full backups take several days and even incrementals take
several hours. What I did not expect, and think is a MUCH more serious
problem, is that the balance prevented use of the disk, holding up all
writes to the disk for (quite literally) hours each.  I have not seen
that effect mentioned anywhere!

That means that for a large, busy data disk, it is impossible to do a
balance unless the server is taken down to single-user mode for the time
the balance takes (presumably still days).  I assume this would also
apply to doing a RAID rebuild (I am not using multiple disks at the moment).

At the moment I am still using my previous backup strategy, alongside
the snapshots (that is: rsync-based rsnapshots to another disk daily and
with fairly long retentions, and separate daily full/incremental backups
using dar to a nas in another building).  I was hoping the btrfs
snapshots might replace the daily rsync snapshots but it doesn't look
like that will work out.

Thanks to all for the replies.

Graham