Re: Ceph Hammer OSD Shard Tuning Test Results

[Mark Nelson <mnelson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

Hi Alex,

I see I even responded in the same thread!  This would be a good thing 
to bring up in the meeting on Wednesday.  Those are far faster single 
OSD results than I've been able to muster with simplemessenger.  I 
wonder how much effect flow-control and header/data crc had.  He did 
have quite a bit more CPU (Intel specs say 14 cores @ 2.6GHz, 28 if you 
count hyperthreading).  Depending on whether there were 1 or 2 CPUs in 
that node, that might be around 3x the CPU power I have here.

Some other thoughts:  Were the simplemessenger tests on IPoIB or native? 
  How big was the RBD volume that was created (could some data be 
locally cached)?  Did network data transfer statistics match the 
benchmark result numbers?

I also did some tests on fdcache, though just glancing at the results it 
doesn't look like tweaking those parameters had much effect.

Mark

On 03/01/2015 08:38 AM, Alexandre DERUMIER wrote:
> Hi Mark,
>
> I found an previous bench from Vu Pham (it's was about simplemessenger vs xiomessenger)
>
> http://www.spinics.net/lists/ceph-devel/msg22414.html
>
> and with 1 osd, he was able to reach 105k iops with simple messenger
>
> . ~105k iops (4K random read, 20 cores used, numjobs=8, iopdepth=32)
>
> this was with more powerfull nodes, but the difference seem to be quite huge
>
>
>
> ----- Mail original -----
> De: "aderumier" <aderumier@odiso.com>
> À: "Mark Nelson" <mnelson@redhat.com>
> Cc: "ceph-devel" <ceph-devel@vger.kernel.org>, "ceph-users" <ceph-users@lists.ceph.com>
> Envoyé: Vendredi 27 Février 2015 07:10:42
> Objet: Re: [ceph-users] Ceph Hammer OSD Shard Tuning Test Results
>
> Thanks Mark for the results,
> default values seem to be quite resonable indeed.
>
>
> I also wonder is cpu frequency can have an impact on latency or not.
> I'm going to benchmark on dual xeon 10-cores 3,1ghz nodes in coming weeks,
> I'll try replay your benchmark to compare
>
>
>
> ----- Mail original -----
> De: "Mark Nelson" <mnelson@redhat.com>
> À: "ceph-devel" <ceph-devel@vger.kernel.org>, "ceph-users" <ceph-users@lists.ceph.com>
> Envoyé: Jeudi 26 Février 2015 05:44:15
> Objet: [ceph-users] Ceph Hammer OSD Shard Tuning Test Results
>
> Hi Everyone,
>
> In the Ceph Dumpling/Firefly/Hammer SSD/Memstore performance comparison
> thread, Alexandre DERUMIER wondered if changing the default shard and
> threads per shard OSD settings might have a positive effect on
> performance in our tests. I went back and used one of the PCIe SSDs
> from our previous tests to experiment with a recent master pull. I
> wanted to know how performance was affected by changing these parameters
> and also to validate that the default settings still appear to be correct.
>
> I plan to conduct more tests (potentially across multiple SATA SSDs in
> the same box), but these initial results seem to show that the default
> settings that were chosen are quite reasonable.
>
> Mark
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com