Re: [RFC PATCH 03/24] erofs: add Errno in Rust

[Gao Xiang <hsiangkao@linux.alibaba.com>]

On 2024/9/26 20:50, Ariel Miculas wrote:
> On 24/09/26 07:23, Gao Xiang wrote:

...

>>>
>>> It might be a fair comparison, but that's not how container images are
>>> distributed. You're trying to argue that I should just use EROFS and I'm
>>
>> First, OCI layer is just distributed like what I said.
>>
>> For example, I could introduce some common blobs to keep
>> chunks as chunk dictionary.   And then the each image
>> will be just some index, and all data will be
>> deduplicated.  That is also what Nydus works.
> 
> I don't really follow what Nydus does. Here [1] it says they're using
> fixed size chunks of 1 MB. Where is the CDC step exactly?

Dragonfly Nydus uses fixed-size chunks of 1MiB by default with
limited external blobs as chunk dictionaries.  And ComposeFS
uses per-file blobs.

Currently, Both are all EROFS users using different EROFS
features.  EROFS itself supports fixed-size chunks (unencoded),
variable-sized extents (encoded, CDC optional) and limited
external blobs.

Honestly, for your testload (10 versions of ubuntu:jammy), I
don't think CDC made a significant difference in the final
result compared to per-file blobs likewise.  Because most of
the files in these images are identical, I think there are
only binary differences due to CVE fixes or similar issues.
Maybe delta compression could do more help, but I never try
this.  So as I asked in [1], does ComposeFS already meet your
requirement?

Again, EROFS could keep every different extent (or each chunk,
whatever) as a seperate file with minor update, it's a trivial
stuff for some userspace archive system, but IMO it's
controversal for an in-tree kernel filesystem.

[1] https://github.com/project-machine/puzzlefs/issues/114#issuecomment-2369971291

Thanks,
Gao Xiang