Re: Feature requests: online backup - defrag - change RAID level

webmaster
2019-09-09T19:26:31Z

This post is a reply to Remi Gauvin's post, but the email got lost so
I can't reply to him directly.

Remi Gauvin wrote on 2019-09-09 17:24 :
>
> On 2019-09-09 11:29 a.m., Graham Cobb wrote:
>
>> and does anyone really care about
>> defrag any more?).
>>
>
>
> Err, yes, yes absolutely.
>
> I don't have any issues with the current btrfs defrag implementions, but
> it's *vital* for btrfs. (which works just as the OP requested, as far as
> I can tell, recursively for a subvolume)
>
> Just booting Windows on a BTRFS virtual image, for example, will create
> almost 20,000 file fragments. Even on SSD's, you get into problems
> trying to work with files that are over 200,000 fragments.
>
> Another huge problem is rsync --inplace. which is perfect backup
> solution to take advantage of BTRFS snapshots, but fragments larges
> files into tiny pieces (and subsequently creates files that are very
> slow to read.).. for some reason, autodefrag doesn't catch that one either.
>
> But the wiki could do a beter job of trying to explain that the snapshot
> duplication of defrag only affects the fragmented portions. As I
> understand, it's really only a problem when using defrag to change
> compression.

Ok, a few things.

First, my defrag suggestion doesn't EVER unshare extents. The defrag
should never unshare, not even a single extent. Why? Because it
violates the expectation that defrag would not decrease free space.

Defrag may break up extents. Defrag may fuse extents. But it shouln't
ever unshare extents.

Therefore, I doubt that the current defrag does "just as the OP
requested". Nonsense. The current implementation does the unsharing
all the time.

Second, I never used btrfs defrag in my life, despite mananging at
least 10 btrfs filesystems. I can't. Because, all my btrfs volumes
have lot of subvolumes, so I'm afraid that defrag will unshare much
more than I can tolerate. In my subvolumes, over 90% of data is
shared. If all subvolumes were to be unshared, the disk usage would
likely increase tenfold, and that I cannot afford.

I agree that btrfs defrag is vital. But currently, it's unusable for
many use cases.

Also, I don't quite understand what the poster means by "the snapshot
duplication of defrag only affects the fragmented portions". Possibly
it means approximately: if a file wasn't modified in the current
(latest) subvolume, it doesn't need to be unshared. But, that would
still unshare all the log files, for example, even all files that have
been appended, etc... that's quite bad. Even if just one byte was
appended to a log file, then defrag will unshare the entire file (I
suppose).