Re: Ceph performance

[Sam Lang <sam.lang@inktank.com>]

Hi Roman,

Is this with the ceph fuse client or the ceph kernel module?

Its not surprising that the local file system (/home) is so much faster 
than a mounted ceph volume, especially the first time the directory tree 
is traversed (metadata results are cached at the client to improve 
performance).  Try running the same find command on the ceph volume and 
see if the cached results at the client improve performance at all.

In order to understand what the performance of ceph should be capable of 
doing with your deployment for this specific workload, you should run 
iperf between two nodes to get an idea of your latency limits.

Also, I noticed that the real timings you listed for ceph and /home are 
offset by exactly 17 minutes (user and sys are identical).  Was that a 
copy/paste error, by chance?

-sam

On 10/29/2012 09:01 AM, Roman Alekseev wrote:
> Hi,
>
> Kindly guide me how to improve performance on the cluster which consist
> of 5 dedicated servers:
>
> - ceph.conf: http://pastebin.com/hT3qEhUF
> - file system on all drives is ext4
> - mount options "user_xattr"
> - each server has :
> CPU:Intel® Xeon® Processor E5335(8M Cache, 2.00 GHz, 1333 MHz FSB) x2
> MEM: 4Gb DDR2
> - 1Gb network
>
> Simple test:
>
> mounted as ceph
> root@client1:/mnt/mycephfs# time find . | wc -l
> 83932
>
> real    17m55.399s
> user    0m0.152s
> sys    0m1.528s
>
> on 1 HDD:
>
> root@client1:/home# time find . | wc -l
> 83932
>
> real    0m55.399s
> user    0m0.152s
> sys    0m1.528s
>
> Please help me to find out the issue. Thanks.
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html