Re: [PATCH v2 2/2] fsck.erofs: implement concurrent directory traversal

[Nithurshen <nithurshen.dev@gmail.com>]

Hi Xiang,

I have implemented the concurrent directory traversal and ran
the benchmarks to compare it against our single-threaded baseline.
All integrity checks passed successfully. Note that this data is an
average of 5 runs of extraction of the same file with purging all
cache before each extraction.

Here is the extraction time breakdown and performance improvement:

| Alg   | Cluster | ST Baseline | MT Traversal | Improvement |
|-------|---------|-------------|--------------|-------------|
| lz4hc | 4096    | 4.17s       | 3.62s        | 13.2%       |
| lz4hc | 8192    | 6.02s       | 3.73s        | 38.0%       |
| lz4hc | 16384   | 5.34s       | 4.61s        | 13.7%       |
| lz4hc | 32768   | 5.24s       | 3.79s        | 27.7%       |
| lz4hc | 65536   | 4.23s       | 3.61s        | 14.7%       |
| zstd  | 4096    | 4.63s       | 4.62s        | 0.2%        |
| zstd  | 8192    | 4.78s       | 3.95s        | 17.4%       |
| zstd  | 16384   | 4.29s       | 3.68s        | 14.2%       |
| zstd  | 32768   | 4.33s       | 3.68s        | 15.0%       |
| zstd  | 65536   | 4.34s       | 3.82s        | 12.0%       |
| lzma  | 4096    | 56.37s      | 36.57s       | 35.1%       |
| lzma  | 8192    | 64.48s      | 45.00s       | 30.2%       |
| lzma  | 16384   | 69.83s      | 52.34s       | 25.0%       |
| lzma  | 32768   | 74.88s      | 55.37s       | 26.1%       |
| lzma  | 65536   | 80.66s      | 63.64s       | 21.1%       |

Average Improvements:
* LZMA: ~27.5% improvement overall.
* LZ4HC: ~21.5% improvement overall.
* ZSTD: ~11.8% improvement overall.

Regarding bottlenecks:
There is a bottleneck regarding the extraction logic which I think can
be solved by parallelizing the decompression logic. We can adapt our
existing Phase 1 parallel decompression design (the dynamic pcluster
batching worker queue) to handle this. By integrating that existing
design with this concurrent traversal, I hope we can effectively 
decouple the CPU-heavy decompression from the I/O-bound extraction.

Please let me know your thoughts on this patch.

Thanks,
Nithurshen