Ceph Kernel client bad performance for 5.4

Ceph Kernel client bad performance for 5.4

[Norman.Kern]

Hi, guys,


I'm using ceph nautilus in my production,  the kernel clients include 5.4 and 4.15, I found a problem in 5.4 sometimes: It's slower than fuse, but when I changed it
to 4.15, it's recovered.
for 5.4:
root@WXRG0432:/mnt/test# rsync -ahHv --progress /root/test test-1
sending incremental file list
test
         58.56M   5%    2.82MB/s    0:05:43  ^C
for 4.15:
root@WXRG0433:/mnt/test# rsync -ahHv --progress /root/test  test-2
sending incremental file list
test
          1.05G 100%  316.25MB/s    0:00:03 (xfr#1, to-chk=0/1)

sent 1.05G bytes  received 35 bytes  299.67M bytes/sec
Anyone have met the same problems with me?

Re: Ceph Kernel client bad performance for 5.4

[Jeff Layton]

On Sat, 2021-05-08 at 10:13 +0800, Norman.Kern wrote:
> Hi, guys,
> 
> 
> I'm using ceph nautilus in my production,  the kernel clients include 5.4 and 4.15, I found a problem in 5.4 sometimes: It's slower than fuse, but when I changed it
> to 4.15, it's recovered.
> for 5.4:
> root@WXRG0432:/mnt/test# rsync -ahHv --progress /root/test test-1
> sending incremental file list
> test
>          58.56M   5%    2.82MB/s    0:05:43  ^C
> for 4.15:
> root@WXRG0433:/mnt/test# rsync -ahHv --progress /root/test  test-2
> sending incremental file list
> test
>           1.05G 100%  316.25MB/s    0:00:03 (xfr#1, to-chk=0/1)
> 
> sent 1.05G bytes  received 35 bytes  299.67M bytes/sec
> Anyone have met the same problems with me?

v5.4 is quite old at this point. It would be good to also test something
newer if you're able. Something v5.12-ish would be ideal.

It's not clear what, exactly, you're testing here, but it looks like the
slowdown is in write activity. A slowdown of that magnitude sounds like
the client has stopped doing buffered I/O, but it's hard to say for
sure.

You may want to run both of these under strace, collecting syscall
timing and see if you can narrow down which syscalls are seeing the
biggest slowdowns. That may help us narrow down what's happening.

Thanks,
-- 
Jeff Layton <jlayton@redhat.com>