* newer /bin/cp have worse btrfs fiemap performance.
@ 2022-11-05 23:30 Wang Yugui
2022-11-06 0:16 ` Wang Yugui
2022-11-07 8:36 ` Filipe Manana
0 siblings, 2 replies; 3+ messages in thread
From: Wang Yugui @ 2022-11-05 23:30 UTC (permalink / raw)
To: linux-btrfs
Hi,
newer /bin/cp have worse btrfs fiemap performance.
btrfs version: misc-next 750de989d367(the lastest of 2022/11/05)
/bin/cp versions(run them on the same server)
8.22 copy from centos/7.9
8.30 rocklinux/9.0
9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/
test case:
/bin/cp /mnt/test/file1 /dev/null
/mnt/test/file1 is created by https://lore.kernel.org/linux-btrfs/YuwUw2JLKtIa9X+S@localhost.localdomain/T/#T
and /mnt/test/file1 is 256M.
file is not cached: 'echo 3 >/proc/sys/vm/drop_caches'
file is cached: run '/bin/cp /mnt/test/file1 /dev/null' again.
performance result(/bin/cp /mnt/test/file1 /dev/null):
/bin/cp 9.1
file is not cached: 94.85(1:34.85)
file is cached: 1982.43(33:02.43)
/bin/cp 8.30
file is not cached: 1.48(0:01.48)
file is cached:14.07(0:14.07)
/bin/cp 8.22
file is not cached: 0.53(0:00.53)
file is cached: 0.10(0:00.10)
as a compare, we test it on xfs too.
1) /bin/cp 8.22/8.30/9.1 have almost same performance.
2) the case(the file is cached) is faster than the case(the file is not
cached).
strace show that the logical of /bin/cp 8.30 and 9.1 are different.
/bin/cp 8.30
lseek(3, 198737920, SEEK_SET) = 198737920
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
lseek(3, 198746112, SEEK_SET) = 198746112
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
/bin/cp 9.1
lseek(3, 880640, SEEK_DATA) = 884736
lseek(3, 884736, SEEK_HOLE) = 888832
lseek(3, 884736, SEEK_SET) = 884736
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
lseek(3, 888832, SEEK_DATA) = 892928
lseek(3, 892928, SEEK_HOLE) = 897024
lseek(3, 892928, SEEK_SET) = 892928
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
Do we need some job in btrfs to support /bin/cp 9.1 well,
or /bin/cp 9.1 is just wrong?
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2022/11/06
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: newer /bin/cp have worse btrfs fiemap performance.
2022-11-05 23:30 newer /bin/cp have worse btrfs fiemap performance Wang Yugui
@ 2022-11-06 0:16 ` Wang Yugui
2022-11-07 8:36 ` Filipe Manana
1 sibling, 0 replies; 3+ messages in thread
From: Wang Yugui @ 2022-11-06 0:16 UTC (permalink / raw)
To: linux-btrfs
Hi,
a fix.
> Hi,
>
> newer /bin/cp have worse btrfs fiemap performance.
>
> btrfs version: misc-next 750de989d367(the lastest of 2022/11/05)
> /bin/cp versions(run them on the same server)
> 8.22 copy from centos/7.9
this
> 8.30 rocklinux/9.0
should be
8.30 rocklinux/8
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2022/11/06
> 9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/
>
> test case:
> /bin/cp /mnt/test/file1 /dev/null
> /mnt/test/file1 is created by https://lore.kernel.org/linux-btrfs/YuwUw2JLKtIa9X+S@localhost.localdomain/T/#T
> and /mnt/test/file1 is 256M.
>
> file is not cached: 'echo 3 >/proc/sys/vm/drop_caches'
> file is cached: run '/bin/cp /mnt/test/file1 /dev/null' again.
>
> performance result(/bin/cp /mnt/test/file1 /dev/null):
> /bin/cp 9.1
> file is not cached: 94.85(1:34.85)
> file is cached: 1982.43(33:02.43)
> /bin/cp 8.30
> file is not cached: 1.48(0:01.48)
> file is cached:14.07(0:14.07)
> /bin/cp 8.22
> file is not cached: 0.53(0:00.53)
> file is cached: 0.10(0:00.10)
>
> as a compare, we test it on xfs too.
> 1) /bin/cp 8.22/8.30/9.1 have almost same performance.
> 2) the case(the file is cached) is faster than the case(the file is not
> cached).
>
> strace show that the logical of /bin/cp 8.30 and 9.1 are different.
> /bin/cp 8.30
> lseek(3, 198737920, SEEK_SET) = 198737920
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> lseek(3, 198746112, SEEK_SET) = 198746112
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
>
> /bin/cp 9.1
> lseek(3, 880640, SEEK_DATA) = 884736
> lseek(3, 884736, SEEK_HOLE) = 888832
> lseek(3, 884736, SEEK_SET) = 884736
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> lseek(3, 888832, SEEK_DATA) = 892928
> lseek(3, 892928, SEEK_HOLE) = 897024
> lseek(3, 892928, SEEK_SET) = 892928
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
>
> Do we need some job in btrfs to support /bin/cp 9.1 well,
> or /bin/cp 9.1 is just wrong?
>
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2022/11/06
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: newer /bin/cp have worse btrfs fiemap performance.
2022-11-05 23:30 newer /bin/cp have worse btrfs fiemap performance Wang Yugui
2022-11-06 0:16 ` Wang Yugui
@ 2022-11-07 8:36 ` Filipe Manana
1 sibling, 0 replies; 3+ messages in thread
From: Filipe Manana @ 2022-11-07 8:36 UTC (permalink / raw)
To: Wang Yugui; +Cc: linux-btrfs
On Sun, Nov 6, 2022 at 12:18 AM Wang Yugui <wangyugui@e16-tech.com> wrote:
>
> Hi,
>
> newer /bin/cp have worse btrfs fiemap performance.
New versions of coreutils (9.0 and 9.1) have worse cp performance, but
it has nothing to do with fiemap.
This was pointed out in another thread ([1]) sometime ago, that
starting with coreutils 9.0, cp no longer uses fiemap.
That happened in coreutils' commit:
https://github.com/coreutils/coreutils/commit/26eccf6c98696c50f4416ba2967edc8676870716
You can also confirm that by checking that strace no longer shows any
lines with:
ioctl(3, FS_IOC_FIEMAP, (...)
>
> btrfs version: misc-next 750de989d367(the lastest of 2022/11/05)
> /bin/cp versions(run them on the same server)
> 8.22 copy from centos/7.9
> 8.30 rocklinux/9.0
> 9.1 local build, https://kojipkgs.fedoraproject.org/packages/coreutils/9.1/8.eln121/
>
> test case:
> /bin/cp /mnt/test/file1 /dev/null
> /mnt/test/file1 is created by https://lore.kernel.org/linux-btrfs/YuwUw2JLKtIa9X+S@localhost.localdomain/T/#T
> and /mnt/test/file1 is 256M.
>
> file is not cached: 'echo 3 >/proc/sys/vm/drop_caches'
> file is cached: run '/bin/cp /mnt/test/file1 /dev/null' again.
>
> performance result(/bin/cp /mnt/test/file1 /dev/null):
> /bin/cp 9.1
> file is not cached: 94.85(1:34.85)
> file is cached: 1982.43(33:02.43)
> /bin/cp 8.30
> file is not cached: 1.48(0:01.48)
> file is cached:14.07(0:14.07)
> /bin/cp 8.22
> file is not cached: 0.53(0:00.53)
> file is cached: 0.10(0:00.10)
>
> as a compare, we test it on xfs too.
> 1) /bin/cp 8.22/8.30/9.1 have almost same performance.
> 2) the case(the file is cached) is faster than the case(the file is not
> cached).
>
> strace show that the logical of /bin/cp 8.30 and 9.1 are different.
Yes, as pointed out before.
Before coreutils 9.0, cp used fiemap to determine where holes and
extents are in a file.
However starting with 9.0 it uses SEEK_DATA/SEEK_HOLE from lseek to do
that, completely
dropping fiemap usage.
> /bin/cp 8.30
> lseek(3, 198737920, SEEK_SET) = 198737920
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> lseek(3, 198746112, SEEK_SET) = 198746112
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
>
> /bin/cp 9.1
> lseek(3, 880640, SEEK_DATA) = 884736
> lseek(3, 884736, SEEK_HOLE) = 888832
> lseek(3, 884736, SEEK_SET) = 884736
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> lseek(3, 888832, SEEK_DATA) = 892928
> lseek(3, 892928, SEEK_HOLE) = 897024
> lseek(3, 892928, SEEK_SET) = 892928
> write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> read(3, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
> write(4, "a\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
>
> Do we need some job in btrfs to support /bin/cp 9.1 well,
> or /bin/cp 9.1 is just wrong?
There's nothing wrong with cp using lseek's SEEK_DATA/HOLE features.
It's just that the data/hole seek calls are where cp is spending most
time, so the way to go is
to improve the efficiency of those calls in btrfs. I'll do some work
to make it better.
Keep in mind that the case being tested is pretty much the worst
possible scenario. It's an extremely
sparse file, where each extent has a hole before it and another hole
after it. More often than not,
there are significantly more extents than holes in a typical file.
Thanks.
[1] https://lore.kernel.org/linux-btrfs/CAL3q7H5NSVicm7nYBJ7x8fFkDpno8z3PYt5aPU43Bajc1H0h1Q@mail.gmail.com/
>
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2022/11/06
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-11-07 8:37 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-11-05 23:30 newer /bin/cp have worse btrfs fiemap performance Wang Yugui
2022-11-06 0:16 ` Wang Yugui
2022-11-07 8:36 ` Filipe Manana
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox