Re: btrfs space used issue

["Austin S. Hemmelgarn" <ahferroin7@gmail.com>]

On 2018-02-28 14:54, Duncan wrote:
> Austin S. Hemmelgarn posted on Wed, 28 Feb 2018 14:24:40 -0500 as
> excerpted:
> 
>>> I believe this effect is what Austin was referencing when he suggested
>>> the defrag, tho defrag won't necessarily /entirely/ clear it up.  One
>>> way to be /sure/ it's cleared up would be to rewrite the entire file,
>>> deleting the original, either by copying it to a different filesystem
>>> and back (with the off-filesystem copy guaranteeing that it can't use
>>> reflinks to the existing extents), or by using cp's --reflink=never
>>> option.
>>> (FWIW, I prefer the former, just to be sure, using temporary copies to
>>> a suitably sized tmpfs for speed where possible, tho obviously if the
>>> file is larger than your memory size that's not possible.)
> 
>> Correct, this is why I recommended trying a defrag.  I've actually never
>> seen things so bad that a simple defrag didn't fix them however (though
>> I have seen a few cases where the target extent size had to be set
>> higher than the default of 20MB).
> 
> Good to know.  I knew larger target extent sizes could help, but between
> not being sure they'd entirely fix it and not wanting to get too far down
> into the detail when the copy-off-the-filesystem-and-back option is
> /sure/ to fix the problem, I decided to handwave that part of it. =:^)
FWIW, a target size of 128M has fixed it on all 5 cases I've seen where 
the default didn't.  In theory, there's probably some really 
pathological case where that won't work, but I've just gotten into the 
habit of using that by default on all my systems now and haven't seen 
any issues so far (but like you I'm pretty much exclusively on SSD's, 
and the small handful of things I have on traditional hard disks are all 
archival storage with WORM access patterns).
> 
>> Also, as counter-intuitive as it
>> might sound, autodefrag really doesn't help much with this, and can
>> actually make things worse.
> 
> I hadn't actually seen that here, but suspect I might, now, as previous
> autodefrag behavior on my system tended to rewrite the entire file[1],
> thereby effectively giving me the benefit of the copy-away-and-back
> technique without actually bothering, while that "bug" has now been fixed.
> 
> I sort of wish the old behavior remained an option, maybe
> radicalautodefrag or something, and must confess to being a bit concerned
> over the eventual impact here now that autodefrag does /not/ rewrite the
> entire file any more, but oh, well...  Chances are it's not going to be
> /that/ big a deal since I /am/ on fast ssd, and if it becomes one, I
> guess I can just setup say firefox-profile-defrag.timer jobs or whatever,
> as necessary.
> 
> ---
> [1] I forgot whether it was ssd behavior, or compression, or what, but
> something I'm using here apparently forced autodefrag to rewrite the
> entire file, and a recent "bugfix" changed that so it's more in line with
> the normal autodefrag behavior.  I rather preferred the old behavior,
> especially since I'm on fast ssd and all my large files tend to be write-
> once no-rewrite anyway, but I understand the performance implications on
> large active-rewrite files such as gig-plus database and VM-image files,
> so...
Hmm.  I've actually never seen such behavior myself.  I do know that 
compression impacts how autodefrag works (autodefrag tries to rewrite up 
to 64k around a random write, but compression operates in 128k blocks), 
but beyond that I'm not sure what might have caused this.