From: Sun YangKai <sunk67188@gmail.com>
To: linux-btrfs@vger.kernel.org
Cc: sunk67188@gmail.com
Subject: Sharing my faster compsize implementation
Date: Thu, 24 Apr 2025 20:45:15 +0800 [thread overview]
Message-ID: <2235418.irdbgypaU6@saltykitkat> (raw)
I love compsize a lot. However, It is a little slow for me. So I'm trying to
write a faster compsize implementation, just for my use case, which gets a
great speedup by both reducing syscalls and multi-threading.
I'm sharing this in case it might be of interest to others in the community.
All feedback and questions are welcome. The project is on
https://github.com/SaltyKitkat/xsz
And followings are the performance comparison on my machine.
On a SATA SSD device, with some snapshots for backing up, and some large media
files.
Cache is cleared before each run.
```
$ sudo time ./xsz -j1 /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
4.27user 37.46system 1:14.13elapsed 56%CPU (0avgtext+0avgdata
31420maxresident)k
5060832inputs+0outputs (0major+46434minor)pagefaults 0swaps
$ sudo time ./xsz -j4 /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
4.89user 37.88system 0:20.42elapsed 209%CPU (0avgtext+0avgdata
35820maxresident)k
5060832inputs+0outputs (0major+79430minor)pagefaults 0swaps
$ sudo time compsize /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
3.72user 72.98system 1:50.07elapsed 69%CPU (0avgtext+0avgdata
80132maxresident)k
5254008inputs+0outputs (1major+24967minor)pagefaults 0swaps
```
On a HDD device, with a lot of small files, and some program files.
Cache is cleared before each run.
```
$ sudo time ./xsz -j1 /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.37user 3.47system 1:07.54elapsed 5%CPU (0avgtext+0avgdata 5436maxresident)k
830912inputs+0outputs (0major+7825minor)pagefaults 0swaps
$ sudo time ./xsz -j4 /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.40user 3.97system 0:57.72elapsed 7%CPU (0avgtext+0avgdata 8064maxresident)k
830912inputs+0outputs (0major+5093minor)pagefaults 0swaps
$ sudo time compsize /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.42user 9.45system 2:49.78elapsed 5%CPU (0avgtext+0avgdata 12648maxresident)k
954080inputs+0outputs (0major+2990minor)pagefaults 0swaps
```
reply other threads:[~2025-04-24 12:45 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2235418.irdbgypaU6@saltykitkat \
--to=sunk67188@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox