Re: Feature requests: online backup - defrag - change RAID level

webmaster
2019-09-12T03:05:49Z

Quoting Remi Gauvin <remi@georgianit.com>:

> On 2019-09-11 7:21 p.m., webmaster@zedlx.com wrote:
>
>> For example, lets examine the typical home user. If he is using btrfs,
>> it means he probably wants snapshots of his data. And, after a few
>> snapshots, his data is fragmented, and the current defrag can't help
>> because it does a terrible job in this particualr case.
>>
>
> I shouldn't be replying to your provocative posts, but this is just
> nonsense.

I really hope that I'm not such a bad person as that sentence is suggesting.
About the provocative posts, I don't know of any other way to get my
thoughts across.
If I offended people, I appologize, but I cannot change the way I communicate.

Certainly, I like the btrfs filesystem and the features it offers, and
I'll continue using it, and no matter what you think of me I want to
say thanks to you guys who are making it all work.

> Not to say that Defragmentation can't be better, smarter,, it happens
> to work very well for typical use.

My thought is that the only reason why it appears that it works is
that a typical home user rarely needs defragmentation. He runs the
"btrfs fi defrag", virtually nothing happens (a few log files get
defragged, if they were shared than they are unshared), prints out
"Done", the user is happy. Placebo effect.

> This sounds like you're implying that snapshots fragment data... can you
> explain that? as far as I know, snapshotting has nothing to do with
> fragmentation of data. All data is COW, and all files that are subject
> to random read write will be fragmented, with or without snapshots.

Close, but essentially: yes. I'm implying that snapshots induce future
fragmentation. The mere act of snapshoting won't create fragments
immediately, but if there are any future writes to previously
snapshoted files, those writes are likely to cause fragmentation. I
think that this is not hard to figure out, but if you wish, I can
elaborate further.

The real question is: does it really matter? Looking at the typical
home user, most of his files rarely change, they are rarely written
to. More likely, most new writes will go to new files. So, maybe the
"home user" is not the best study-case for defragmentation. He has to
be at least some kind of power-user, or content-creator to experience
any significant fragmentation.

> And running defrag on your system regularly works just fine. There's a
> little overhead of space if you are taking regular snapshots, (say
> hourly snapshots with snapper.) If you have more control/liberty when
> you take your snapshots, ideally, you would defrag before taking the
> snaptshop/reflink copy. Again, this only matters to files that are
> subject to fragmentation in the first place.

Btrfs defrag works just fine until you get some serious fragmentation.
At that point, if you happen to have some snapshots, you better delete
them before running defrag. Because, if you do run defrag on
snapshoted and heavily fragmented filesystem, you are going to run out
of disk space really fast.

> I suspect if you actually tried using the btrfs defrag, you would find
> you are making a mountain of a molehill.. There are lots of far more
> important problems to solve.

About importance, well, maybe you are right there, maybe not. Somehow
I guess that after so many years in development and a stable feature
set, most remaining problems are bugs and trivialities. So you are
fixing them, one by one, many of those are urgent. I see you are
working on deduplication, that's a hell of a work which actually won't
end up well if it is not supplemented by a good defrag.

Didn't someone say, earlier in this discussion, that the defrag is
important for btrfs. I would guess that it is. On many OSes defrag is
run automatically. All older filesystems have a pretty good defrag.

What I would say is that btrfs can have a much better defrag than it
has now. If defrag is deemed important, thay why are improvements to
defrag unimportant?