* compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
@ 2022-10-26 10:21 Nemcev Aleksey
2022-10-26 19:07 ` Chris Murphy
0 siblings, 1 reply; 6+ messages in thread
From: Nemcev Aleksey @ 2022-10-26 10:21 UTC (permalink / raw)
To: linux-btrfs
Hello.
Recently I decided to compress the existing Btrfs volume with zstd.
To do this, I set property `compression=zstd` for all subvolumes to
compress new files with the default zstd:3 compression level.
Then I run `btrfs filesystem defragment -c zstd:12 -v -r` on all
subvolumes to compress existing files using zstd:12 compression level
(to spend more time now and save more space later, but don't want to
keep slow zstd:12 level all the time).
After defragment, I run `btrfs filesystem balance` on all subvolumes to
make Btrfs happy.
And after all, I run `compsize` on the root subvolume to check the
compression ratio, and compsize shows me the following:
Processed 1100578 files, 1393995 regular extents (1649430 refs), 666312
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 77% 169G 217G 226G
none 100% 99G 99G 100G
zlib 52% 54G 102G 109G
zstd 19% 12M 65M 66M
prealloc 100% 15G 15G 16G
I'm very confused why almost all compressed data is compressed with zlib
while I haven't used zlib at any step.
Why do compsize reports this?
Should I worry about this? It seems zstd offers a much faster
decompression speed than zlib.
Thank you.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
2022-10-26 10:21 compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere Nemcev Aleksey
@ 2022-10-26 19:07 ` Chris Murphy
2022-10-26 22:29 ` Nemcev Aleksey
0 siblings, 1 reply; 6+ messages in thread
From: Chris Murphy @ 2022-10-26 19:07 UTC (permalink / raw)
To: Nemcev Aleksey, Btrfs BTRFS
On Wed, Oct 26, 2022, at 6:21 AM, Nemcev Aleksey wrote:
> Hello.
>
> Recently I decided to compress the existing Btrfs volume with zstd.
>
> To do this, I set property `compression=zstd` for all subvolumes to
> compress new files with the default zstd:3 compression level.
>
> Then I run `btrfs filesystem defragment -c zstd:12 -v -r` on all
> subvolumes to compress existing files using zstd:12 compression level
> (to spend more time now and save more space later, but don't want to
> keep slow zstd:12 level all the time).
>
> After defragment, I run `btrfs filesystem balance` on all subvolumes to
> make Btrfs happy.
>
> And after all, I run `compsize` on the root subvolume to check the
> compression ratio, and compsize shows me the following:
> Processed 1100578 files, 1393995 regular extents (1649430 refs), 666312
> inline.
> Type Perc Disk Usage Uncompressed Referenced
> TOTAL 77% 169G 217G 226G
> none 100% 99G 99G 100G
> zlib 52% 54G 102G 109G
> zstd 19% 12M 65M 66M
> prealloc 100% 15G 15G 16G
>
> I'm very confused why almost all compressed data is compressed with zlib
> while I haven't used zlib at any step.
> Why do compsize reports this?
> Should I worry about this? It seems zstd offers a much faster
> decompression speed than zlib.
> Thank you.
If you use chattr +c anywhere, it uses the btrfs default compression algo which is zlib. There is a way to set a compression algo property, but I'm uncertain if the kernel honors it.
So you'll want to recursively remove the compression file attribute. I'm not sure it's worth recompressing but you can use defrag -z for that.
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
2022-10-26 19:07 ` Chris Murphy
@ 2022-10-26 22:29 ` Nemcev Aleksey
2022-10-27 1:03 ` Zygo Blaxell
0 siblings, 1 reply; 6+ messages in thread
From: Nemcev Aleksey @ 2022-10-26 22:29 UTC (permalink / raw)
To: Chris Murphy, Btrfs BTRFS
26.10.2022 22:07, Chris Murphy пишет:
>
> On Wed, Oct 26, 2022, at 6:21 AM, Nemcev Aleksey wrote:
>> Hello.
>>
>> Recently I decided to compress the existing Btrfs volume with zstd.
>>
>> To do this, I set property `compression=zstd` for all subvolumes to
>> compress new files with the default zstd:3 compression level.
>>
>> Then I run `btrfs filesystem defragment -c zstd:12 -v -r` on all
>> subvolumes to compress existing files using zstd:12 compression level
>> (to spend more time now and save more space later, but don't want to
>> keep slow zstd:12 level all the time).
>>
>> After defragment, I run `btrfs filesystem balance` on all subvolumes to
>> make Btrfs happy.
>>
>> And after all, I run `compsize` on the root subvolume to check the
>> compression ratio, and compsize shows me the following:
>> Processed 1100578 files, 1393995 regular extents (1649430 refs), 666312
>> inline.
>> Type Perc Disk Usage Uncompressed Referenced
>> TOTAL 77% 169G 217G 226G
>> none 100% 99G 99G 100G
>> zlib 52% 54G 102G 109G
>> zstd 19% 12M 65M 66M
>> prealloc 100% 15G 15G 16G
>>
>> I'm very confused why almost all compressed data is compressed with zlib
>> while I haven't used zlib at any step.
>> Why do compsize reports this?
>> Should I worry about this? It seems zstd offers a much faster
>> decompression speed than zlib.
>> Thank you.
>
> If you use chattr +c anywhere, it uses the btrfs default compression algo which is zlib. There is a way to set a compression algo property, but I'm uncertain if the kernel honors it.
>
> So you'll want to recursively remove the compression file attribute. I'm not sure it's worth recompressing but you can use defrag -z for that.
>
Thank you for the idea, I understood where I failed.
`btrfs filesystem defragment -c zstd -r /` is not correct command, but
`btrfs filesystem defragment -czstd -r /` is correct one (note the space).
And the first command will try to defragment the file named zstd using
default compression (zlib), not the compression from the `compression`
subvolume property.
And also it's not possible to specify compression level in defragment -
so, I'll change the level in subvolume properties, defragment with
-czstd, and change the level back.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
2022-10-26 22:29 ` Nemcev Aleksey
@ 2022-10-27 1:03 ` Zygo Blaxell
2022-10-27 13:21 ` Chris Murphy
0 siblings, 1 reply; 6+ messages in thread
From: Zygo Blaxell @ 2022-10-27 1:03 UTC (permalink / raw)
To: Nemcev Aleksey; +Cc: Chris Murphy, Btrfs BTRFS
On Thu, Oct 27, 2022 at 01:29:35AM +0300, Nemcev Aleksey wrote:
> 26.10.2022 22:07, Chris Murphy пишет:
> >
> > On Wed, Oct 26, 2022, at 6:21 AM, Nemcev Aleksey wrote:
> > > Hello.
> > >
> > > Recently I decided to compress the existing Btrfs volume with zstd.
> > >
> > > To do this, I set property `compression=zstd` for all subvolumes to
> > > compress new files with the default zstd:3 compression level.
> > >
> > > Then I run `btrfs filesystem defragment -c zstd:12 -v -r` on all
> > > subvolumes to compress existing files using zstd:12 compression level
> > > (to spend more time now and save more space later, but don't want to
> > > keep slow zstd:12 level all the time).
> > >
> > > After defragment, I run `btrfs filesystem balance` on all subvolumes to
> > > make Btrfs happy.
> > >
> > > And after all, I run `compsize` on the root subvolume to check the
> > > compression ratio, and compsize shows me the following:
> > > Processed 1100578 files, 1393995 regular extents (1649430 refs), 666312
> > > inline.
> > > Type Perc Disk Usage Uncompressed Referenced
> > > TOTAL 77% 169G 217G 226G
> > > none 100% 99G 99G 100G
> > > zlib 52% 54G 102G 109G
> > > zstd 19% 12M 65M 66M
> > > prealloc 100% 15G 15G 16G
> > >
> > > I'm very confused why almost all compressed data is compressed with zlib
> > > while I haven't used zlib at any step.
> > > Why do compsize reports this?
> > > Should I worry about this? It seems zstd offers a much faster
> > > decompression speed than zlib.
> > > Thank you.
> >
> > If you use chattr +c anywhere, it uses the btrfs default compression algo which is zlib. There is a way to set a compression algo property, but I'm uncertain if the kernel honors it.
> >
> > So you'll want to recursively remove the compression file attribute. I'm not sure it's worth recompressing but you can use defrag -z for that.
> >
> Thank you for the idea, I understood where I failed.
>
> `btrfs filesystem defragment -c zstd -r /` is not correct command, but
> `btrfs filesystem defragment -czstd -r /` is correct one (note the space).
>
> And the first command will try to defragment the file named zstd using
> default compression (zlib), not the compression from the `compression`
> subvolume property.
>
> And also it's not possible to specify compression level in defragment - so,
> I'll change the level in subvolume properties, defragment with -czstd, and
> change the level back.
It's not possible to set the level in subvol properties either. Only the
compress= mount option can set the level.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
2022-10-27 1:03 ` Zygo Blaxell
@ 2022-10-27 13:21 ` Chris Murphy
2022-10-27 14:32 ` Zygo Blaxell
0 siblings, 1 reply; 6+ messages in thread
From: Chris Murphy @ 2022-10-27 13:21 UTC (permalink / raw)
To: Zygo Blaxell, Nemcev Aleksey; +Cc: Btrfs BTRFS
On Wed, Oct 26, 2022, at 9:03 PM, Zygo Blaxell wrote:
>> And also it's not possible to specify compression level in defragment - so,
>> I'll change the level in subvolume properties, defragment with -czstd, and
>> change the level back.
>
> It's not possible to set the level in subvol properties either. Only the
> compress= mount option can set the level.
$ sudo btrfs property set test compression zstd:1
$ sudo btrfs property get test
compression=zstd:1
$ sudo btrfs property set test compression zstd:f
$ sudo btrfs property get test
compression=zstd:f
It's allowing the arbitrary setting of compression values. This is with btrfs-progs 6.0. Progs is being too permissive, it shouldn't silently set invalid values. But then even with valid values, I'm pretty sure right now the kernel ignores the compression properties.
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere
2022-10-27 13:21 ` Chris Murphy
@ 2022-10-27 14:32 ` Zygo Blaxell
0 siblings, 0 replies; 6+ messages in thread
From: Zygo Blaxell @ 2022-10-27 14:32 UTC (permalink / raw)
To: Chris Murphy; +Cc: Nemcev Aleksey, Btrfs BTRFS
On Thu, Oct 27, 2022 at 09:21:53AM -0400, Chris Murphy wrote:
>
>
> On Wed, Oct 26, 2022, at 9:03 PM, Zygo Blaxell wrote:
>
> >> And also it's not possible to specify compression level in defragment - so,
> >> I'll change the level in subvolume properties, defragment with -czstd, and
> >> change the level back.
> >
> > It's not possible to set the level in subvol properties either. Only the
> > compress= mount option can set the level.
>
> $ sudo btrfs property set test compression zstd:1
> $ sudo btrfs property get test
> compression=zstd:1
> $ sudo btrfs property set test compression zstd:f
> $ sudo btrfs property get test
> compression=zstd:f
>
> It's allowing the arbitrary setting of compression values. This is with
> btrfs-progs 6.0. Progs is being too permissive, it shouldn't silently
> set invalid values. But then even with valid values, I'm pretty sure
> right now the kernel ignores the compression properties.
Correct. You can put whatever you want in the string, but the kernel
only ever looks at fs_info for compress_level, which is set from the
mount options.
e.g. if you mount with 'compress=zlib:5' and you have a file with
btrfs.compression property with 'zstd:7', what you get is zstd compression
with level 5.
> --
> Chris Murphy
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-10-27 14:32 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-26 10:21 compsize reports that filesystem uses zlib compression, while I set zstd compression everywhere Nemcev Aleksey
2022-10-26 19:07 ` Chris Murphy
2022-10-26 22:29 ` Nemcev Aleksey
2022-10-27 1:03 ` Zygo Blaxell
2022-10-27 13:21 ` Chris Murphy
2022-10-27 14:32 ` Zygo Blaxell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox