* Sharing my faster compsize implementation
@ 2025-04-24 12:45 Sun YangKai
0 siblings, 0 replies; only message in thread
From: Sun YangKai @ 2025-04-24 12:45 UTC (permalink / raw)
To: linux-btrfs; +Cc: sunk67188
I love compsize a lot. However, It is a little slow for me. So I'm trying to
write a faster compsize implementation, just for my use case, which gets a
great speedup by both reducing syscalls and multi-threading.
I'm sharing this in case it might be of interest to others in the community.
All feedback and questions are welcome. The project is on
https://github.com/SaltyKitkat/xsz
And followings are the performance comparison on my machine.
On a SATA SSD device, with some snapshots for backing up, and some large media
files.
Cache is cleared before each run.
```
$ sudo time ./xsz -j1 /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
4.27user 37.46system 1:14.13elapsed 56%CPU (0avgtext+0avgdata
31420maxresident)k
5060832inputs+0outputs (0major+46434minor)pagefaults 0swaps
$ sudo time ./xsz -j4 /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
4.89user 37.88system 0:20.42elapsed 209%CPU (0avgtext+0avgdata
35820maxresident)k
5060832inputs+0outputs (0major+79430minor)pagefaults 0swaps
$ sudo time compsize /mnt/1
Processed 5663444 files, 1097233 regular extents (3859780 refs), 3459956
inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 95% 827G 865G 1.3T
none 100% 811G 811G 1.1T
zstd 29% 15G 54G 192G
3.72user 72.98system 1:50.07elapsed 69%CPU (0avgtext+0avgdata
80132maxresident)k
5254008inputs+0outputs (1major+24967minor)pagefaults 0swaps
```
On a HDD device, with a lot of small files, and some program files.
Cache is cleared before each run.
```
$ sudo time ./xsz -j1 /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.37user 3.47system 1:07.54elapsed 5%CPU (0avgtext+0avgdata 5436maxresident)k
830912inputs+0outputs (0major+7825minor)pagefaults 0swaps
$ sudo time ./xsz -j4 /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.40user 3.97system 0:57.72elapsed 7%CPU (0avgtext+0avgdata 8064maxresident)k
830912inputs+0outputs (0major+5093minor)pagefaults 0swaps
$ sudo time compsize /mnt/guest
Processed 393486 files, 215207 regular extents (215207 refs), 178308 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 100% 29G 29G 29G
none 100% 29G 29G 29G
0.42user 9.45system 2:49.78elapsed 5%CPU (0avgtext+0avgdata 12648maxresident)k
954080inputs+0outputs (0major+2990minor)pagefaults 0swaps
```
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2025-04-24 12:45 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-04-24 12:45 Sharing my faster compsize implementation Sun YangKai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox