Re: less cores more iops / speed

[Mark Nelson <mark.nelson@inktank.com>]

On 11/07/2012 06:00 PM, Joao Eduardo Luis wrote:
> On 11/07/2012 10:02 PM, Stefan Priebe wrote:
>> Hello again,
>>
>> I've noticed something really interesting.
>>
>> I get 5000 iops / VM for rand. 4k writes while assigning 4 cores on a
>> 2.5 Ghz Xeon.
>>
>> When i move this VM to another kvm host with 3.6Ghz i get 8000 iops
>> (still 8 cores) when i then LOWER the assigned cores from 8 to 4 i get
>> 14.500 iops. If i assign only 2 cores i get 16.000 iops...
>>
>> Why does less kvm cores mean more speed?
>
> Totally going on a limb here, but might be related to the cache maybe?
> When you have more cores your threads may bounce around the cores and
> invalidate cache entries as they go by; will less cores you might end up
> with some sort of twisted, forced cpu affinity that allows you to take
> advantage of caching.

There's also the context switching overhead.  It'd be interesting to 
know how much the writer processes were shifting around on cores. 
Stefan, what tool were you using to do writes?

>
> But I don't know, really. I would be amazed if what I just wrote had an
> ounce of truth, and would be completely astonished if that was the cause
> for such a sudden increase on iops.

Yeah, it's seems pretty surprising that there would be any significant 
effect at this level of performance.

>
>    -Joao
>
>>
>> Greets,
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>