Disk iops performance scalability

Disk iops performance scalability

[John Morrison]

Hi,

Seeing a drop-off in iops when more vcpu's are added:-

3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0

dom0_max_vcpus=2 dom0_vcpus_pin

domU  8 cores  fio result 145k iops
domU 10 cores fio result 99k iops
domU 12 cores fio result 89k iops 
domU 14 cores fio result 81k iops

ioping . -c 3
4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms
4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms
4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms

--- . (ext4 /dev/xvda1) ioping statistics ---
3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s
min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms

The initial ioping response is good, then a lot of latency with later ones. Any ideas ?

Thanks

John

Re: Disk iops performance scalability

[Konrad Rzeszutek Wilk]

On Mon, Mar 18, 2013 at 11:23:00PM +0000, John Morrison wrote:
> Hi,
> 
> Seeing a drop-off in iops when more vcpu's are added:-
> 
> 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0
> 
> dom0_max_vcpus=2 dom0_vcpus_pin
> 
> domU  8 cores  fio result 145k iops
> domU 10 cores fio result 99k iops
> domU 12 cores fio result 89k iops 
> domU 14 cores fio result 81k iops
> 
> ioping . -c 3
> 4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms
> 4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms
> 4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms
> 
> --- . (ext4 /dev/xvda1) ioping statistics ---
> 3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s
> min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms
> 
> The initial ioping response is good, then a lot of latency with later ones. Any ideas ?

Is your guest compiled with PARAVIRT_SPINLOCK and are you using a PV or PVHVM guest?

> 
> Thanks
> 
> John
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: Disk iops performance scalability

[John Morrison]

Hi,

Yes PARAVIRT_SPINLOCK and its a PV guest.
iops are consistent now as I had a typo in the config file (cpus=)

ioping still showing first 0.1s the rest 0.7/0.8ms

Thanks

John
On 19 Mar 2013, at 12:58, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> On Mon, Mar 18, 2013 at 11:23:00PM +0000, John Morrison wrote:
>> Hi,
>> 
>> Seeing a drop-off in iops when more vcpu's are added:-
>> 
>> 3.8.2 kernel/xen-4.2.1/single domU/LVM backend/8GB RAM domU/2GB RAM dom0
>> 
>> dom0_max_vcpus=2 dom0_vcpus_pin
>> 
>> domU  8 cores  fio result 145k iops
>> domU 10 cores fio result 99k iops
>> domU 12 cores fio result 89k iops 
>> domU 14 cores fio result 81k iops
>> 
>> ioping . -c 3
>> 4096 bytes from . (ext4 /dev/xvda1): request=1 time=0.1 ms
>> 4096 bytes from . (ext4 /dev/xvda1): request=2 time=0.7 ms
>> 4096 bytes from . (ext4 /dev/xvda1): request=3 time=0.8 ms
>> 
>> --- . (ext4 /dev/xvda1) ioping statistics ---
>> 3 requests completed in 2002.0 ms, 1836 iops, 7.2 mb/s
>> min/avg/max/mdev = 0.1/0.5/0.8/0.3 ms
>> 
>> The initial ioping response is good, then a lot of latency with later ones. Any ideas ?
> 
> Is your guest compiled with PARAVIRT_SPINLOCK and are you using a PV or PVHVM guest?
> 
>> 
>> Thanks
>> 
>> John
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>