Re: Worse performance than SquashFS for small filesystems

[Gao Xiang <xiang@kernel.org>]

On Mon, May 30, 2022 at 08:41:34PM +0200, Paul Spooren wrote:
> Hi,
> 
> I’m an OpenWrt developer and we work on an operation system for routers and overall network facing embedded devices. Since storage is often limited on our targeted devices we use SquashFS to compress everything, total image sizes are less than 10 MegaBytes.
> 
> Reading about erofs I’m very keen to adopt it for our case, however giving it a first try the storage performance seems to be significantly _worse_ than the default SquashFS implementation. Since you ran benchmarks in the past, could you give me advise if I’m missing anything?

What the meaning of `performance'? I don't think the size of images
mean runtime performance anyway.

> 
> For the test I used erofs-utils as of a134ce7c39a5427343029e97a62665157bef9bc3 (2022-05-17) and compressed the x86/64 root filesystem of a standard OpenWrt image[1]. I’m seeing a difference of one MegaByte which is about 30% worse in this context.
> 
> My test:
> 
> $ ./staging_dir/host/bin/mkfs.erofs -zlzma erofs.root ./build_dir/target-x86_64_musl/root-x86

Have you try with

-zlzma -C 65536 -Eztailpacking

by default, EROFS uses 4k compression rather than Squashfs 128k.

> mkfs.erofs 1.4
> <W> erofs: EXPERIMENTAL MicroLZMA feature in use. Use at your own risk!
> <W> erofs: Note that it may take more time since the compressor is still single-threaded for now.
> Build completed.
> 
> $ mksquashfs -comp xz ./build_dir/target-x86_64_musl/root-x86 squashfs.root
> 
> $ ls -lh *.root
> -rw-r--r-- 1 ubuntu ubuntu 4.3M May 30 20:27 erofs.root
> -rw-r--r-- 1 ubuntu ubuntu 3.3M May 30 20:28 squashfs.root

Even EROFS now supports big pcluster and ztailpacking features,
Squashfs supports a feature called fragments which compresses
several tails of files in a fragment. It's obviously beneficial
to the final size of images but it can read unrelated data of
other files even such files are very small.

You can try a big file with EROFS and Squashfs, and you can
see the difference.

Btw, MicroLZMA is still an experimental feature for now, due to 
three reasons:
 - XZ utils hasn't release a stable version for now, which I think
   Lasse will release a stable version this year;

 - EROFS has a finalized on-disk MircoLZMA support since Linux 5.16,
   so users can mount MicroLZMA since 5.16.  Yet currently EROFS is
   not quite optimized for such slow algorithm, since it needs a
   completely different strategy from LZ4.  I'm working on that
   together with folio work;

 - We need a similiar `fragment' feature to catch squashfs under a
   lot of files.

> 
> Is erofs just not meant for such small storages?

If you care more about the size of images, I personally prefer to
squashfs for now until we handle all the things above.

Thanks,
Gao Xiang

> 
> Thanks for all further comments!
> 
> Best,
> Paul
> 
> [1]: https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-rootfs.tar.gz