Re: (PING) btrfs sequential 8K read()s from compressed files are not merging

[Anand Jain <anand.jain@oracle.com>]

> # dd if=/dev/zero of=blah bs=1G count=16
> 16+0 records in
> 16+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 14.2627 s, 1.2 GB/s
> 
> I verified the file is well compressed:
> 
> # compsize blah
> Processed 1 file, 131073 regular extents (131073 refs), 0 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL        3%      512M          16G          16G
> zstd         3%      512M          16G          16G
> 
> I'm surprised that such a file needed 128Kextents and required 512MB of
> disk space (the filesystem is mounted with compress=zstd:3) but it is what
> it is. On to reading the file:
> 
> # dd if=blah of=/dev/null bs=512k
> 32768+0 records in
> 32768+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 7.40493 s, 2.3 GB/s
> ### iostat showed 30MB/s to 100MB/s device read speed
> 
> # dd if=blah of=/dev/null bs=32k
> 524288+0 records in
> 524288+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 8.34762 s, 2.1 GB/s
> ### iostat showed 30MB/s to 90MB/s device read speed
> 
> # dd if=blah of=/dev/null bs=8k
> 2097152+0 records in
> 2097152+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 18.7143 s, 918 MB/s
> ### iostat showed very variable 8MB/s to 60MB/s device read speed
> ### average maybe around 40MB/s
> 
> 
> Also worth noting is the IO request size that iostat is reporting. For
> bs=8k it reports a request size of about 4 (KB?), while it's order of
> magnitudes higher for all the other measurements in this email.
> 

The sector size is 4k, and the compression block size is 128k. There 
will be a lot more read IO, which may not be mergeable for reads with 
lower block sizes.

> 
> ==== Same test with uncompressable file
> 
> I performed the same experiments with a urandom-filled file. I assume here
> that btrfs is detecting the file can't be compressed, so it's treating it
> differently. This is what the measurements are showing here, that the
> device speed limits are reached in all cases
> (this host has an HDD with limit 200MB/s).
> 
> # dd if=/dev/urandom of=blah-random bs=1G count=16
> 16+0 records in
> 16+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 84.0045 s, 205 MB/s
> 
> # compsize blah-random
> Processed 1 file, 133 regular extents (133 refs), 0 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL      100%       15G          15G          15G
> none       100%       15G          15G          15G
> 
> # dd if=blah-random of=/dev/null bs=512k
> 32768+0 records in
> 32768+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 87.82 s, 196 MB/s
> ### iostat showed 180-205MB/s device read speed
> 
> # dd if=blah-random of=/dev/null bs=32k
> 524288+0 records in
> 524288+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 88.3785 s, 194 MB/s
> ### iostat showed 180-205MB/s device read speed
> 
> # dd if=blah-random of=/dev/null bs=8k
> 2097152+0 records in
> 2097152+0 records out
> 17179869184 bytes (17 GB, 16 GiB) copied, 88.7887 s, 193 MB/s
> ### iostat showed 180-205MB/s device read speed


The heuristic will disable compression on the file if the data is 
incompressible, such as that from /dev/urandom.

Generally, to test compression in fstests, we use the 'dd' command as below.

od /dev/urandom | dd iflag=fullblock of=.. bs=.. count=..

Thanks, Anand