Re: rados bench single instance vs. multiple instances

[Mark Nelson <mnelson@redhat.com>]

On 05/13/2015 10:05 AM, Deneau, Tom wrote:
>
>
>> -----Original Message-----
>> From: Sage Weil [mailto:sage@newdream.net]
>> Sent: Monday, May 11, 2015 12:04 PM
>> To: Deneau, Tom
>> Cc: ceph-devel
>> Subject: Re: rados bench single instance vs. multiple instances
>>
>> On Mon, 11 May 2015, Deneau, Tom wrote:
>>> I have noticed the following while running rados bench seq read tests
>>> with a 40M object size
>>>
>>>      single rados bench, 4 concurrent ops,                     bandwidth =
>> 190 MB/s
>>>      4 copies of rados bench, 1 concurrent op each,  aggregate
>>> bandwidth = 310 MB/s
>>>
>>> and in fact the single rados bench seems limited to 190, no matter how many
>> concurrent ops.
>>>
>>> I don't see this kind of behavior with a 4M object size.
>>>
>>> (The above are with caches dropped on the osd targets)
>>>
>>> It doesn't seem to be related to the total number of bytes being
>>> processed by the single because if I don't drop the caches, both the
>>> single rados bench and the 4-copy rados bench get much higher numbers
>>> (600 vs. 900) but still the single rados bench appears limited, no matter
>> how many concurrent ops are used.
>>>
>>> Is there kind of throttling going on by design here?
>>
>> It might be the librados throttles:
>>
>> OPTION(objecter_inflight_op_bytes, OPT_U64, 1024*1024*100) // max in-flight
>> data (both directions)
>> OPTION(objecter_inflight_ops, OPT_U64, 1024)               // max in-flight
>> ios
>>
>> IIRC these only affect librados.. which would include 'rados bench'.
>>
>> sage
>>
>
> Just a follow-up that changing the limits of the two options mentioned above
> did indeed solve my problem.

Yay!  I suspect max bytes more so than max ops?


Mark

>
> Also, my naïve understanding of the architecture was that things like RBD and RGW were
> layered on librados as shown in http://ceph.com/docs/master/architecture/.  So wouldn't these
> throttles apply to those stacks as well?
>
> -- Tom Deneau
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html