Running compsize utility on btrfs compress mount point

Running compsize utility on btrfs compress mount point

[shally verma]

Hi

This email is in context of another thread initiated by me (with
Subject: using fio to test btrfs compression).

Just as brief background, I am trying to verify btrfs compression
where I am writing data using
fio, dd or cp command.

In response to my question in another thread:
> Is there any command that i can run to confirm file has been compressed?

I've gotten response from Duncan:
>>There is the quite recently posted (and actively updated since then)
>>compsize command.

>>https://github.com/kilobyte/compsize

On running compsize on btrfs mount point created, I get following:

Type       Perc     Disk Usage   Uncompressed Referenced
Data        50%      128K         256K         256K
zlib          50%      128K         256K         256K
Data       100%       10G          10G          10G
none       100%       10G          10G          10G

but then how do I interpret above information?
if I go by each line, then Does ..

1st entry means
Data        50%      128K         256K         256K
 - It is just raw uncompressed data but occupying only 50% of disk
space w.r.t its size (may be because of its CoW or deduplication?)

zlib          50%      128K         256K         256K
- This is zlib compressed data occupying 50% of disk usage ?

Data       100%       10G          10G          10G
- This is uncompressed data with no compression/cow/deduplication
occupying 100% of disk of its size?

none       100%       10G          10G          10G
- Did not understand how to interpret none entry here.. any help?

Thanks
Shally

Re: Running compsize utility on btrfs compress mount point

[Duncan]

shally verma posted on Fri, 22 Sep 2017 12:01:32 +0530 as excerpted:

> In response to my question in another thread:
>> Is there any command that i can run to confirm file has been compressed?
> 
> I've gotten response from Duncan:
>>>There is the quite recently posted (and actively updated since then)
>>>compsize command.
> 
>>>https://github.com/kilobyte/compsize
> 
> On running compsize on btrfs mount point created, I get following:
> 
> Type       Perc     Disk Usage   Uncompressed Referenced
> Data        50%      128K         256K         256K
> zlib        50%      128K         256K         256K
> Data       100%       10G          10G          10G
> none       100%       10G          10G          10G
> 
> but then how do I interpret above information?

Hmm... Looks like you got hit by the live-repo phenomenon.  I've not
updated for a few days (reported sync time was last Sat), but was
just running it a few hours ago, and my output was a bit more
intuitively ordered.

The way it's /supposed/ to work (unless it's changing, it /is/ a rather
new tool after all), the first line (Data) should be the aggregate
total and average compression percentage of all used compression types,
with the following lines displaying values for each compression type
compsize found, on its own.

IOW, for the above I'd have expected just three lines (plus header),
something like this:

Data	100%	 10G	 10G	 10G
zlib	 50%	128K	256K	256K
none	100%	 10G	 10G	 10G

(In this case the 128K compressed is so small it's a rounding
error in the 10G full size, so the combined data line and
the none line will appear the same.)

I'd suggest trying to pull any updates and rebuilding.  If that
doesn't fix it, try checking out a specific commit, say the 160c335e7
that was HEAD last Saturday (16th) when I built, and see if the output
comes out a bit more sane.  If that works, then the problem must be in
either 94d98028b from Tuesday, or caed4fdcd, HEAD as I'm looking at it
now, from Thursday, if you pulled after it.  If the author doesn't
popup in this thread, you might bisect to the specific commit and
file a github issue on it.

If 160c335e7 doesn't give you saner output, then there's some other
difference between our systems that must account for your output
above.  FWIW, gcc 6.4.0 (gentoo) here, amd64, kernel 4.13, glibc
2.25-r5 (the -r5 indicating gentoo patch level), btrfs-progs 4.12.
I have CFLAGS available too if you need 'em. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman