[Xen-users] Performance test regarding xenstored

[Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 1010 bytes --]

Hello,

I am running a mail benchmark test using smtp-source[1] with VM running
postfix on Xen hypervisor. My system configuration is:

uname -a ->
Linux cadlab 3.1.10-1.19-xen #1 SMP Mon Feb 25 10:32:50 UTC 2013 (f0b13a3)
x86_64 x86_64 x86_64 GNU/Linux

I am observing a very high disk write usage by xenstored (some 5 Mbps)
without even anything running on the VM. Is it normal?

During test, I increase the mail request rate and I see that xenstored
usage going down as I increase the workload more and more. Basically, after
certain point, delays in the mail starts getting increased and I am
suspecting the disk usage to be the reason for this. However, at the time
of saturation, disk usage by VM comes out to be around 1 Mbps. So, I am not
sure that why it is not able to take more disk bandwidth. Has it something
to do with xenstored. Can someone please give some insights?



[1] http://www.postfix.org/smtp-source.1.html

*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*

[-- Attachment #1.2: Type: text/html, Size: 1381 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[George Dunlap]

On Wed, Sep 4, 2013 at 11:17 AM, Mohit Dhingra <mohitdhingras@gmail.com> wrote:
> Hello,
>
> I am running a mail benchmark test using smtp-source[1] with VM running
> postfix on Xen hypervisor. My system configuration is:
>
> uname -a ->
> Linux cadlab 3.1.10-1.19-xen #1 SMP Mon Feb 25 10:32:50 UTC 2013 (f0b13a3)
> x86_64 x86_64 x86_64 GNU/Linux
>
> I am observing a very high disk write usage by xenstored (some 5 Mbps)
> without even anything running on the VM. Is it normal?

Just to check, what version of Xen are you running?

I think that XenServer has xenstored write all transactions to a log
by default; I didn't think that upstream Xen did that.

If you are using XenServer, then you need to ask this question on the
XenServer development list:
http://www.xenserver.org/discuss-virtualization/mailing-lists.html

 -George

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 1539 bytes --]

Hi,

I am using xen kernel 3.1.10. I think it is xenstored only, following is
the log from "iotop" command:


22:21:36  4493 be/4 root        0.00 K/s 4562.15 K/s  0.00 %  0.00 %
xenstored --pid-file=/var/run/xenstored.pid

It says 4562.15 K/s as the disk write speed and no read. I am not sure why
is it happening. I have enabled sync option on the disk partition where VM
file resides.

/dev/disk/by-id/ata-ST31000528AS_9VP9NG6C-part9 /srv                 ext4
    defaults,sync              1 2


*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 16:40, George Dunlap <George.Dunlap@eu.citrix.com>wrote:

> On Wed, Sep 4, 2013 at 11:17 AM, Mohit Dhingra <mohitdhingras@gmail.com>
> wrote:
> > Hello,
> >
> > I am running a mail benchmark test using smtp-source[1] with VM running
> > postfix on Xen hypervisor. My system configuration is:
> >
> > uname -a ->
> > Linux cadlab 3.1.10-1.19-xen #1 SMP Mon Feb 25 10:32:50 UTC 2013
> (f0b13a3)
> > x86_64 x86_64 x86_64 GNU/Linux
> >
> > I am observing a very high disk write usage by xenstored (some 5 Mbps)
> > without even anything running on the VM. Is it normal?
>
> Just to check, what version of Xen are you running?
>
> I think that XenServer has xenstored write all transactions to a log
> by default; I didn't think that upstream Xen did that.
>
> If you are using XenServer, then you need to ask this question on the
> XenServer development list:
> http://www.xenserver.org/discuss-virtualization/mailing-lists.html
>
>  -George
>

[-- Attachment #1.2: Type: text/html, Size: 2651 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[George Dunlap]

[-- Attachment #1.1: Type: text/plain, Size: 357 bytes --]

On 04/09/13 13:01, Mohit Dhingra wrote:
> Hi,
>
> I am using xen kernel 3.1.10. I think it is xenstored only, following 
> is the log from "iotop" command:

I'm still not sure what you're saying here.  Xen 3.1 only had point 
releases up until 3.1.4, as far as I can see:

http://xenbits.xen.org/hg/xen-3.1-testing.hg/

How did you install Xen?

  -George


[-- Attachment #1.2: Type: text/html, Size: 1092 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 644 bytes --]

I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares" and
then booted into xen kernel one.


*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com>wrote:

>  On 04/09/13 13:01, Mohit Dhingra wrote:
>
>  Hi,
>
>  I am using xen kernel 3.1.10. I think it is xenstored only, following is
> the log from "iotop" command:
>
>
> I'm still not sure what you're saying here.  Xen 3.1 only had point
> releases up until 3.1.4, as far as I can see:
>
> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
>
> How did you install Xen?
>
>  -George
>
>

[-- Attachment #1.2: Type: text/html, Size: 1660 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 996 bytes --]

I think the one that I have installed is:
http://rpm.pbone.net/index.php3/stat/4/idpl/20420199/dir/opensuse_12.x/com/kernel-xen-3.1.10-1.19.1.i586.rpm.html

*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 18:34, Mohit Dhingra <mohitdhingras@gmail.com> wrote:

> I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares" and
> then booted into xen kernel one.
>
>
> *
> ----------------------------
> Thanks & Regards
> Mohit Dhingra
> +919611190435*
>
>
> On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com>wrote:
>
>>  On 04/09/13 13:01, Mohit Dhingra wrote:
>>
>>  Hi,
>>
>>  I am using xen kernel 3.1.10. I think it is xenstored only, following
>> is the log from "iotop" command:
>>
>>
>> I'm still not sure what you're saying here.  Xen 3.1 only had point
>> releases up until 3.1.4, as far as I can see:
>>
>> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
>>
>> How did you install Xen?
>>
>>  -George
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 2534 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[Shakeel Butt]

On Wed, Sep 4, 2013 at 7:10 AM, Mohit Dhingra <mohitdhingras@gmail.com> wrote:
> I think the one that I have installed is:
> http://rpm.pbone.net/index.php3/stat/4/idpl/20420199/dir/opensuse_12.x/com/kernel-xen-3.1.10-1.19.1.i586.rpm.html

This is the Linux kernel. I think what George was asking which Xen
hypervisor version are you using?

>
> ----------------------------
> Thanks & Regards
> Mohit Dhingra
> +919611190435
>
>
> On 4 September 2013 18:34, Mohit Dhingra <mohitdhingras@gmail.com> wrote:
>>
>> I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares" and
>> then booted into xen kernel one.
>>
>>
>> ----------------------------
>> Thanks & Regards
>> Mohit Dhingra
>> +919611190435
>>
>>
>> On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com>
>> wrote:
>>>
>>> On 04/09/13 13:01, Mohit Dhingra wrote:
>>>
>>> Hi,
>>>
>>> I am using xen kernel 3.1.10. I think it is xenstored only, following is
>>> the log from "iotop" command:
>>>
>>>
>>> I'm still not sure what you're saying here.  Xen 3.1 only had point
>>> releases up until 3.1.4, as far as I can see:
>>>
>>> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
>>>
>>> How did you install Xen?
>>>
>>>  -George
>>>
>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 2948 bytes --]

I think xen-3.0. xm info says:

cadlab:/var/lib/xenstored # xm info
host                   : cadlab
release                : 3.1.10-1.19-xen
version                : #1 SMP Mon Feb 25 10:32:50 UTC 2013 (f0b13a3)
machine                : x86_64
nr_cpus                : 8
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 3066
hw_caps                :
bfebfbff:28100800:00000000:00003b40:0098e3bd:00000000:00000001:00000000
virt_caps              : hvm hvm_directio
total_memory           : 8180
free_memory            : 8
free_cpus              : 0
max_free_memory        : 5473
max_para_memory        : 5469
max_hvm_memory         : 5448
xen_major              : 4
xen_minor              : 1
xen_extra              : .4_02-1.29.1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : 23432
xen_commandline        : vga=mode-0x317
cc_compiler            : gcc version 4.6.2 (SUSE Linux)
cc_compile_by          : abuild
cc_compile_domain      :
cc_compile_date        : Sat Mar 30 12:48:56 UTC 2013
xend_config_format     : 4



Please let me know if I answered it correctly?

*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 19:48, Shakeel Butt <shakeel.butt@gmail.com> wrote:

> On Wed, Sep 4, 2013 at 7:10 AM, Mohit Dhingra <mohitdhingras@gmail.com>
> wrote:
> > I think the one that I have installed is:
> >
> http://rpm.pbone.net/index.php3/stat/4/idpl/20420199/dir/opensuse_12.x/com/kernel-xen-3.1.10-1.19.1.i586.rpm.html
>
> This is the Linux kernel. I think what George was asking which Xen
> hypervisor version are you using?
>
> >
> > ----------------------------
> > Thanks & Regards
> > Mohit Dhingra
> > +919611190435
> >
> >
> > On 4 September 2013 18:34, Mohit Dhingra <mohitdhingras@gmail.com>
> wrote:
> >>
> >> I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares" and
> >> then booted into xen kernel one.
> >>
> >>
> >> ----------------------------
> >> Thanks & Regards
> >> Mohit Dhingra
> >> +919611190435
> >>
> >>
> >> On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com>
> >> wrote:
> >>>
> >>> On 04/09/13 13:01, Mohit Dhingra wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am using xen kernel 3.1.10. I think it is xenstored only, following
> is
> >>> the log from "iotop" command:
> >>>
> >>>
> >>> I'm still not sure what you're saying here.  Xen 3.1 only had point
> >>> releases up until 3.1.4, as far as I can see:
> >>>
> >>> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
> >>>
> >>> How did you install Xen?
> >>>
> >>>  -George
> >>>
> >>
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> >
>

[-- Attachment #1.2: Type: text/html, Size: 4861 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 3312 bytes --]

My bad! Major is 4 and Minor is 1. So, 4.1 probably ?

*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 19:53, Mohit Dhingra <mohitdhingras@gmail.com> wrote:

> I think xen-3.0. xm info says:
>
> cadlab:/var/lib/xenstored # xm info
> host                   : cadlab
> release                : 3.1.10-1.19-xen
> version                : #1 SMP Mon Feb 25 10:32:50 UTC 2013 (f0b13a3)
> machine                : x86_64
> nr_cpus                : 8
> nr_nodes               : 1
> cores_per_socket       : 4
> threads_per_core       : 2
> cpu_mhz                : 3066
> hw_caps                :
> bfebfbff:28100800:00000000:00003b40:0098e3bd:00000000:00000001:00000000
> virt_caps              : hvm hvm_directio
> total_memory           : 8180
> free_memory            : 8
> free_cpus              : 0
> max_free_memory        : 5473
> max_para_memory        : 5469
> max_hvm_memory         : 5448
> xen_major              : 4
> xen_minor              : 1
> xen_extra              : .4_02-1.29.1
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : 23432
> xen_commandline        : vga=mode-0x317
> cc_compiler            : gcc version 4.6.2 (SUSE Linux)
> cc_compile_by          : abuild
> cc_compile_domain      :
> cc_compile_date        : Sat Mar 30 12:48:56 UTC 2013
> xend_config_format     : 4
>
>
>
> Please let me know if I answered it correctly?
>
> *
> ----------------------------
> Thanks & Regards
> Mohit Dhingra
> +919611190435*
>
>
> On 4 September 2013 19:48, Shakeel Butt <shakeel.butt@gmail.com> wrote:
>
>> On Wed, Sep 4, 2013 at 7:10 AM, Mohit Dhingra <mohitdhingras@gmail.com>
>> wrote:
>> > I think the one that I have installed is:
>> >
>> http://rpm.pbone.net/index.php3/stat/4/idpl/20420199/dir/opensuse_12.x/com/kernel-xen-3.1.10-1.19.1.i586.rpm.html
>>
>> This is the Linux kernel. I think what George was asking which Xen
>> hypervisor version are you using?
>>
>> >
>> > ----------------------------
>> > Thanks & Regards
>> > Mohit Dhingra
>> > +919611190435
>> >
>> >
>> > On 4 September 2013 18:34, Mohit Dhingra <mohitdhingras@gmail.com>
>> wrote:
>> >>
>> >> I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares" and
>> >> then booted into xen kernel one.
>> >>
>> >>
>> >> ----------------------------
>> >> Thanks & Regards
>> >> Mohit Dhingra
>> >> +919611190435
>> >>
>> >>
>> >> On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com>
>> >> wrote:
>> >>>
>> >>> On 04/09/13 13:01, Mohit Dhingra wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> I am using xen kernel 3.1.10. I think it is xenstored only, following
>> is
>> >>> the log from "iotop" command:
>> >>>
>> >>>
>> >>> I'm still not sure what you're saying here.  Xen 3.1 only had point
>> >>> releases up until 3.1.4, as far as I can see:
>> >>>
>> >>> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
>> >>>
>> >>> How did you install Xen?
>> >>>
>> >>>  -George
>> >>>
>> >>
>> >
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@lists.xen.org
>> > http://lists.xen.org/xen-devel
>> >
>>
>
>

[-- Attachment #1.2: Type: text/html, Size: 5560 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] Performance test regarding xenstored

[Mohit Dhingra]

[-- Attachment #1.1: Type: text/plain, Size: 3885 bytes --]

Probably, there is a similar question here:
http://xen.1045712.n5.nabble.com/xenstored-is-doing-a-lot-of-IO-td2591647.html


I am not sure though. Sometime back, I set the memory of one VM to
experiment with restricted resources. Could this high disk writes be
because of this?

*
----------------------------
Thanks & Regards
Mohit Dhingra
+919611190435*


On 4 September 2013 19:56, Mohit Dhingra <mohitdhingras@gmail.com> wrote:

> My bad! Major is 4 and Minor is 1. So, 4.1 probably ?
>
> *
> ----------------------------
> Thanks & Regards
> Mohit Dhingra
> +919611190435*
>
>
> On 4 September 2013 19:53, Mohit Dhingra <mohitdhingras@gmail.com> wrote:
>
>> I think xen-3.0. xm info says:
>>
>> cadlab:/var/lib/xenstored # xm info
>> host                   : cadlab
>> release                : 3.1.10-1.19-xen
>> version                : #1 SMP Mon Feb 25 10:32:50 UTC 2013 (f0b13a3)
>> machine                : x86_64
>> nr_cpus                : 8
>> nr_nodes               : 1
>> cores_per_socket       : 4
>> threads_per_core       : 2
>> cpu_mhz                : 3066
>> hw_caps                :
>> bfebfbff:28100800:00000000:00003b40:0098e3bd:00000000:00000001:00000000
>> virt_caps              : hvm hvm_directio
>> total_memory           : 8180
>> free_memory            : 8
>> free_cpus              : 0
>> max_free_memory        : 5473
>> max_para_memory        : 5469
>> max_hvm_memory         : 5448
>> xen_major              : 4
>> xen_minor              : 1
>> xen_extra              : .4_02-1.29.1
>> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
>> hvm-3.0-x86_32p hvm-3.0-x86_64
>> xen_scheduler          : credit
>> xen_pagesize           : 4096
>> platform_params        : virt_start=0xffff800000000000
>> xen_changeset          : 23432
>> xen_commandline        : vga=mode-0x317
>> cc_compiler            : gcc version 4.6.2 (SUSE Linux)
>> cc_compile_by          : abuild
>> cc_compile_domain      :
>> cc_compile_date        : Sat Mar 30 12:48:56 UTC 2013
>> xend_config_format     : 4
>>
>>
>>
>> Please let me know if I answered it correctly?
>>
>> *
>> ----------------------------
>> Thanks & Regards
>> Mohit Dhingra
>> +919611190435*
>>
>>
>> On 4 September 2013 19:48, Shakeel Butt <shakeel.butt@gmail.com> wrote:
>>
>>> On Wed, Sep 4, 2013 at 7:10 AM, Mohit Dhingra <mohitdhingras@gmail.com>
>>> wrote:
>>> > I think the one that I have installed is:
>>> >
>>> http://rpm.pbone.net/index.php3/stat/4/idpl/20420199/dir/opensuse_12.x/com/kernel-xen-3.1.10-1.19.1.i586.rpm.html
>>>
>>> This is the Linux kernel. I think what George was asking which Xen
>>> hypervisor version are you using?
>>>
>>> >
>>> > ----------------------------
>>> > Thanks & Regards
>>> > Mohit Dhingra
>>> > +919611190435
>>> >
>>> >
>>> > On 4 September 2013 18:34, Mohit Dhingra <mohitdhingras@gmail.com>
>>> wrote:
>>> >>
>>> >> I installed kernel-xen in OpenSUSE 12.1 through "Install Softwares"
>>> and
>>> >> then booted into xen kernel one.
>>> >>
>>> >>
>>> >> ----------------------------
>>> >> Thanks & Regards
>>> >> Mohit Dhingra
>>> >> +919611190435
>>> >>
>>> >>
>>> >> On 4 September 2013 18:23, George Dunlap <george.dunlap@eu.citrix.com
>>> >
>>> >> wrote:
>>> >>>
>>> >>> On 04/09/13 13:01, Mohit Dhingra wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> I am using xen kernel 3.1.10. I think it is xenstored only,
>>> following is
>>> >>> the log from "iotop" command:
>>> >>>
>>> >>>
>>> >>> I'm still not sure what you're saying here.  Xen 3.1 only had point
>>> >>> releases up until 3.1.4, as far as I can see:
>>> >>>
>>> >>> http://xenbits.xen.org/hg/xen-3.1-testing.hg/
>>> >>>
>>> >>> How did you install Xen?
>>> >>>
>>> >>>  -George
>>> >>>
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Xen-devel mailing list
>>> > Xen-devel@lists.xen.org
>>> > http://lists.xen.org/xen-devel
>>> >
>>>
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 6588 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel