Re: A possible way to reduce free space fragmentation?

["Theodore Ts'o" <tytso@mit.edu>]

On Sat, Feb 01, 2025 at 03:48:16PM +0000, Artem S. Tashkinov wrote:
> 
> 
> On 2/1/25 3:38 PM, Andreas Dilger wrote:
> > It should be possible to run "find $DIR -type f -size -1M | xargs e4defrag" to only defragment files below 1MB (or whatever you consider "small").
> 
> I have smaller files completely defragmented already.
> 
> The issue is a dozen of 50-250MB files that span multiple extents (up to
> 30).

How big are the extents?  If you are performing large sequential
reads, a few seeks every few megabytes is really not a big deal from a
performance perspective, and it's certainly not worth the huge amount
of time that a perfect defragmentation would take (since that would
require moving smaller files out of the way to free up enough
contiguous space for a big file).

This is why Windows defraggers have mostly fallen out of faver, and
why no one has really found it worthwhile to invest more effort in
improving e4defrag (either the userspace program or the underlying
kernel infrastructure).

> > > ext4 has no free space defragmentation and at most you can use e4defrag
> > > to defragment individual files. I now have a 24GB ext4 filesystem that
> > > has only 7GB of space occupied however it has small files scattered all
> > > over it and now bigger files occupy more than one extent and I cannot
> > > reduce fragmentation to zero. One way to approach that would be to
> > > shrink the volume and then defragment it but that will involve a ton of
> > > disk writes and unnecessary tear and wear. Is it possible to modify the
> > > e4degrag utility to move small defragmented files, so that they were
> > > placed consecutively instead of being randomly spread all over the disk?

Anything is *possible*.  Whether anyone thinks its worth their
development time is a different question.  Many years ago, at a
face-to-face ext4 developer's get together, we had sketched out some
ideas for how we might do this.  It included ways to block certain
areas of the disk from being used for normal block allocation, and an
extended ext4-specific fallocate-like ioctl which e4defrag could use
to allocate blocks in a specific portion of the file system.

But no company has a business case where implementing this feature
would have a positive return on investment; no hobbyist has been
interested in doing in their free time; and unfortunately, this is too
complicated of a project for a Google Summer of Code, Outreachy, or
other Intern project.

If you're interested, I'm happy to chat.  But basically, this is a
"patches are welcome; send us code and we'll be happy to review them"
sort of situation.

Cheers,

						- Ted