Re: speed decrease since firefly,giant,hammer the 2nd try

[Stefan Priebe <s.priebe@profihost.ag>]

Am 10.02.2015 um 20:05 schrieb Gregory Farnum:
> On Tue, Feb 10, 2015 at 10:55 AM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>> Hello,
>>
>> last year in june i already reported this but there was no real result.
>> (http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-July/041070.html)
>>
>> I then had the hope that this will be fixed itself when hammer is released.
>> Now i tried hammer an the results are bad as before.
>>
>> Since firefly librbd1 / librados2 are 20% slower for 4k random iop/s than
>> dumpling - this is also the reason why i still stick to dumpling.
>>
>> I've now modified my test again to be a bit more clear.
>>
>> Ceph cluster itself completely dumpling.
>>
>> librbd1 / librados from dumpling (fio inside qemu): 23k iop/s for random 4k
>> writes
>>
>> - stopped qemu
>> - cp -ra firefly_0.80.8/usr/lib/librados.so.2.0.0 /usr/lib/
>> - cp -ra firefly_0.80.8/usr/lib/librbd.so.1.0.0 /usr/lib/
>> - start qemu
>>
>> same fio, same qemu, same vm, same host, same ceph dumpling storage,
>> different librados / librbd: 16k iop/s for random 4k writes
>>
>> What's wrong with librbd / librados2 since firefly?
>
> We're all going to have the same questions now as we did last time,
> about what the cluster looks like, what the perfcounters are reporting
> on both versions of librados, etc.

I try to answer all your questions - not sue how easy this is.

6 Nodes each with:
- Single Intel E5-1650 v3
- 48GB RAM
- 4x 800GB Samsung SSD
- 2x 10Gbit/s bonded storage network

- Client side
- Dual Xeon E5
- 256GB RAM
- 2x 10Gbit/s bonded storage network

Regarding perf counters - i'm willing to make tests. Just tell me how.

> Also, please give us the results from Giant rather than Firefly, for
> the reasons I mentioned previously.

As giant is not a long term release and we have a support contract it's 
not an option to me. Even tough i tried hammer git master three days 
ago. Same results.

Stefan