Re: mount compress=zstd leaves files uncompressed, that used to compress well with before

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

在 2025/3/26 22:45, Dimitrios Apostolou 写道:
> CC'ing Qu Wenruo from this thread
>
> On Mon, 24 Mar 2025, Gerhard Wiesinger wrote:
>>
>> It's a known bug I also ran into, see the disucssion here:
>> https://lore.kernel.org/all/b7995589-35a4-4595-
>> baea-1dcdf1011d68@wiesinger.com/T/
>> (It can't be easily fixed)
>
> Hi Qu, reading this thread I understand that posix_fallocate() ends up
> leaving files uncompressed forever in btrfs, regardless of mount options.
> I see this in PostgreSQL that recently started using posix_fallocate().
>
> You gave some information on why the solution is hard:
>> There is a long existing problem with compression with preallocation.
>>
>> One easy example is, if we go compression for the preallocated range,
>> what we do with the gap (compressed size is always smaller than the real
>> size).
>>
>> If we leave the gap, then the read performance can be even worse, as now
>> we have to read several small extents with gaps between them, vs a large
>> contig read.
>
> Can't the solution/workaround be way more simple, or stupid even?
>
> * Either have fallocate(2) return EOPNOTSUPP on a force-compress
>    filesystem, and leave the work-around to userspace,

Unfortunately fallocate has higher priority, not vise-verse.

In most cases, compression is a good to have feature, but even with
force-compression, we can still have cases that won't be compressed.

On the other hand, all major upstream fses have support for fallocate,
and although I understand preallocation is no longer as simple as
non-COW filesystems, not supporting it would still be a big surprise to
a lot of user space tools.

Although emotionally I agree with you. Fallocation on btrfs is just
looking for extra problems, and if I have the final call, I will be more
than happier to nuke fallocation support.

>
> * or fill up the holes with compressed zeros, basically implementing the
>    work-around in kernelspace. I suspect this would be very cheap in a
>    deduplicating filesystem like btrfs, since all the zero-filled
>    compressed extents are essentially identical.

Not that easily either. Fallocate itself should mean the next write into
the fallocated range will not fail with ENOSPC.

Although that assumption itself is no longer correct on btrfs, (e.g.
fallocate, then snapshot).

But doing compressed zeros means we got nothing from the old
preallocation behavior, and still waste space on holes.

>
>
> In any case, I think this should be documented in the btrfs documentation
> about compression, [1] it would have saved me time. I can try to submit a
> patch if you point me to the docs repo. Any other known cases where
> compression is unexpectedly skipped?

The document repo is btrfs-progs:

https://github.com/kdave/btrfs-progs

You can either submit regular email patches to this ml, or submit a
github PR.

Thanks,
Qu

>
> [1] https://btrfs.readthedocs.io/en/latest/Compression.html
>
>
> Thank you in advance,
> Dimitris
>