RE: performance problems

RE: performance problems

[Ian Pratt]

 

> yup. the only difference was that I didn't touch dumU's 
> kernel config; in
> dom0 I already disabled the debugging flags (except magic 
> sysreq), but in
> default domU's config they were still enabled.
> 
> Overall impression from my newer benchmark experiments:
> 
> - Harddisk IO is about 30% faster in XenU than in vmware ws 
> (both accessing
>   the same raw partition)
> - Network IO is at the same level in xenU and vmware, which 
> is about 30%
>   slower than dom0/native net io (netpipe, 2 machines using 
> 1000Mbit ethernet
>   connected via a switch, xenU and vmware bridged).

It's odd to see such a drop in network performance unless you're on a
machine with a slow CPU. It should be possible to saturate a Gigabit
Ethernet (900Mb/s) with a relatively modern CPU. What are you using?

Ian





-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Henning Glawe]

On Fri, Feb 04, 2005 at 01:08:31PM -0000, Ian Pratt wrote:
>  
> 
> > yup. the only difference was that I didn't touch dumU's 
> > kernel config; in
> > dom0 I already disabled the debugging flags (except magic 
> > sysreq), but in
> > default domU's config they were still enabled.
> > 
> > Overall impression from my newer benchmark experiments:
> > 
> > - Harddisk IO is about 30% faster in XenU than in vmware ws 
> > (both accessing
> >   the same raw partition)
> > - Network IO is at the same level in xenU and vmware, which 
> > is about 30%
> >   slower than dom0/native net io (netpipe, 2 machines using 
> > 1000Mbit ethernet
> >   connected via a switch, xenU and vmware bridged).
> 
> It's odd to see such a drop in network performance unless you're on a
> machine with a slow CPU. It should be possible to saturate a Gigabit
> Ethernet (900Mb/s) with a relatively modern CPU. What are you using?

its an athlon XP 2100+, so it isn't _so_ slow...

if someone wants to take a look at the netpipe-tcp results:
http://www.physik.fu-berlin.de/~glaweh/netpipe-xen-benchmark.pdf


-- 
c u
henning


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Nivedita Singhvi]

Henning Glawe wrote:

>>It's odd to see such a drop in network performance unless you're on a
>>machine with a slow CPU. It should be possible to saturate a Gigabit
>>Ethernet (900Mb/s) with a relatively modern CPU. What are you using?
> 
> 
> its an athlon XP 2100+, so it isn't _so_ slow...
> 
> if someone wants to take a look at the netpipe-tcp results:
> http://www.physik.fu-berlin.de/~glaweh/netpipe-xen-benchmark.pdf

I took a look at your results. Could you possibly
make available any stats that you collected, and
your system settings? (netstat -s, sysctl -a, ...)

One of the causes for poor performance might be that
the virtual domains are unable to utilize the greater
bandwidth across the virtual bridge using default
socket buffer sizes. Did you bump up the buffer sizes,
queue lengths, etc? A little tuning might help and
give a clearer picture of the real bottlenecks..

thanks,
Nivedita




-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: Performance problems

[Olivier Bonvalet]

Le vendredi 12 avril 2013 à 19:45 +0200, Olivier Bonvalet a écrit :
> Le vendredi 12 avril 2013 à 10:04 -0500, Mark Nelson a écrit :
> > On 04/11/2013 07:25 PM, Ziemowit Pierzycki wrote:
> > > No, I'm not using RDMA in this configuration since this will eventually
> > > get deployed to production with 10G ethernet (yes RDMA is faster).  I
> > > would prefer Ceph because it has a storage drive built into OpenNebula
> > > which my company is using and as you mentioned individual drives.
> > >
> > > I'm not sure what the problem is but it appears to me that one of the
> > > hosts may be holding up the rest... with Ceph if the performance of one
> > > of the hosts is much faster than others could this potentially slow down
> > > the cluster to this level?
> > 
> > Definitely!  Even 1 slow OSD can cause dramatic slow downs.  This is 
> > because we (by default) try to distribute data evenly to every OSD in 
> > the cluster.  If even 1 OSD is really slow, it will accumulate more and 
> > more outstanding operations while all of the other OSDs complete their 
> > requests.  What will happen is that eventually you will have all of your 
> > outstanding operations waiting on that slow OSD, and all of the other 
> > OSDs will sit idle waiting for new requests.
> > 
> > If you know that some OSDs are permanently slower than others, you can 
> > re-weight them so that they receive fewer requests than the others which 
> > can mitigate this, but that isn't always an optimal solution.  Some 
> > times a slow OSD can be a sign of other hardware problems too.
> > 
> > Mark
> > 
> 
> and does response time of OSD are log somewhere, to identify that "weak
> link" ?
> 
> 

I think I found the answer with the admin socket :
    ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok perf dump

For example in the output I can see op_latency, op_w_latency,
op_r_latency, op_rw_latency, etc. Great.


_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Performance problems

[Boscovich, Maximiliano]

Hi People,

We are running FoxPro compiled application and it really slow on performance. 
We have 15 users running dosemu ( version 1.4.0+svn.1828-1 ) in the same server, 
with freedos ( "FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]")    

and the statistics from sysstat are:

09:45:02          CPU     %user     %nice   %system   %iowait    %steal
 %idle
 09:55:06          all     28,68      0,00     71,32      0,00      0,00
 0,00
 10:05:02          all     28,81      0,00     71,19      0,00      0,00
 0,00
 10:15:20          all     26,96      0,00     73,03      0,00      0,00
 0,01
 10:25:02          all     26,42      0,00     73,24      0,00      0,00
 0,34
 10:35:03          all     26,59      0,00     71,90      0,00      0,00
 1,50

Always %system is more than 70% .

The server is a HP Proliat coreduo, and /proc/cpuinfo show this:

mboscovich@psico:~$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
stepping        : 6
cpu MHz         : 1866.678
cache size      : 4096 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4669.09
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
stepping        : 6
cpu MHz         : 1866.678
cache size      : 4096 KB
physical id     : 1
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4669.09
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

The Distro is Debian GNU/Linux, kernel :

Linux elmu 2.6.22-2-686 #1 SMP Fri Aug 31 00:24:01 UTC 2007 i686 GNU/Linux
	

The configuration is :

cat /etc/dosemu/dosemu.conf

$_cpu = "80686"
$_cpu_emu = "off"
$_rdtsc = (on)
$_mathco = (off)
$_hogthreshold = (1)
$_full_file_locks = (off)
$_xms = (16384)
$_ems = (0)
$_dpmi = (0x5000)
$_joy_device = ""
$_printer_timeout = (20)
$_speaker = "off"
$_sound = (off)
$_X_title = "Sipefco on GNU/Linux"
$_X_font = "-xos4-terminus-bold-r-normal-*-*-240-*-*-c-*-ibm-cp850"
$_X_mitshm = (off)
$_X_vgaemu_memsize = (512)

The users connects to the server with ssh -X.

Any idea how to improve the performance?. 

Well, Thanks and sorry for my english, i'm from Argentina.



-- 


"Si hablas con Dios, eres religioso; Sí el habla contigo, eres un psicótico."
 - El Doctor House parafraseando a Thomas Szasz.

-----------------------------------------------------------------
                         Boscovich, Maximiliano
GNU/Linux Counter user #402782
Blog: http://maximilianoboscovich.wordpress.com
Jabber: boscovich@jabberes.org
MSN: joelboscovich@hotmail.com
GnuPG Public Key: http://keyserver.veridis.com:11371/export?id=-8543752951164951938&created=1152911209000
Command in GNU/Linux:  gpg --keyserver wwwkeys.eu.pgp.net --search-keys mboscovich
Key fingerprint: 7A88 1636 BDA4 F680 AB00 0D8E 896E 7DF5 291E 227E
------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Performance problems

[Frantisek Hanzlik]

Hello Maximiliano,

Your dosemu.conf seems normal. Did You verified (by top/ps), that system
load is caused by "dosemu.bin" processes? How load increase with
increasing number of dosemu clients?

I run several servers with dosemu, both 1.2.2 and 1.4 version, with 5-10
users concurrently running, without load problems.

Curiously, we have one DOS app too, which is able take almost 100% CPU
time - when is configured for using its own screensaver. When this
screensaver start, this was able eat almost all CPU power. Disabling
this feature, all work OK.

Best regards, Franta Hanzlik

Boscovich, Maximiliano wrote:
> Hi People,
> 
> We are running FoxPro compiled application and it really slow on performance.
> We have 15 users running dosemu ( version 1.4.0+svn.1828-1 ) in the same server,
> with freedos ( "FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]")
> 
> and the statistics from sysstat are:
> 
> 09:45:02          CPU     %user     %nice   %system   %iowait    %steal
>  %idle
>  09:55:06          all     28,68      0,00     71,32      0,00      0,00
>  0,00
>  10:05:02          all     28,81      0,00     71,19      0,00      0,00
>  0,00
>  10:15:20          all     26,96      0,00     73,03      0,00      0,00
>  0,01
>  10:25:02          all     26,42      0,00     73,24      0,00      0,00
>  0,34
>  10:35:03          all     26,59      0,00     71,90      0,00      0,00
>  1,50
> 
> Always %system is more than 70% .
> 
> The server is a HP Proliat coreduo, and /proc/cpuinfo show this:
> 
> mboscovich@psico:~$ cat /proc/cpuinfo
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 15
> model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
> stepping        : 6
> cpu MHz         : 1866.678
> cache size      : 4096 KB
> physical id     : 0
> siblings        : 1
> core id         : 0
> cpu cores       : 1
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 10
> wp              : yes
> flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
> bogomips        : 4669.09
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 36 bits physical, 48 bits virtual
> power management:
> 
> processor       : 1
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 15
> model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
> stepping        : 6
> cpu MHz         : 1866.678
> cache size      : 4096 KB
> physical id     : 1
> siblings        : 1
> core id         : 0
> cpu cores       : 1
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 10
> wp              : yes
> flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
> bogomips        : 4669.09
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 36 bits physical, 48 bits virtual
> power management:
> 
> The Distro is Debian GNU/Linux, kernel :
> 
> Linux elmu 2.6.22-2-686 #1 SMP Fri Aug 31 00:24:01 UTC 2007 i686 GNU/Linux
> 	
> 
> The configuration is :
> 
> cat /etc/dosemu/dosemu.conf
> 
> $_cpu = "80686"
> $_cpu_emu = "off"
> $_rdtsc = (on)
> $_mathco = (off)
> $_hogthreshold = (1)
> $_full_file_locks = (off)
> $_xms = (16384)
> $_ems = (0)
> $_dpmi = (0x5000)
> $_joy_device = ""
> $_printer_timeout = (20)
> $_speaker = "off"
> $_sound = (off)
> $_X_title = "Sipefco on GNU/Linux"
> $_X_font = "-xos4-terminus-bold-r-normal-*-*-240-*-*-c-*-ibm-cp850"
> $_X_mitshm = (off)
> $_X_vgaemu_memsize = (512)
> 
> The users connects to the server with ssh -X.
> 
> Any idea how to improve the performance?.
> 
> Well, Thanks and sorry for my english, i'm from Argentina.
> 

Re: Performance problems

[Frank Cox]

On Fri, 26 Oct 2007 20:33:44 -0300
"Boscovich, Maximiliano" <mboscovich@rectorado.unl.edu.ar> wrote:

> We are running FoxPro compiled application and it really slow on performance. 

I wonder if TAME would help.

It seems to me that there was another program that does pretty much the same
thing as TAME but without the send-money nag screen, i.e. a free program.  But
I can't recall its name at the moment....

-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

Re: Performance problems

[Frank Cox]

On Sat, 27 Oct 2007 12:19:54 -0600
Frank Cox <theatre@sasktel.net> wrote:

> It seems to me that there was another program that does pretty much the same
> thing as TAME but without the send-money nag screen, i.e. a free program.  But
> I can't recall its name at the moment....

I found it.

Here is the readme file:

QUOTE:
DVPTAME   Released to the Public Domain, 1991.

DVPTAME lets DESQview suspend programs before their time slice is up if
they are frequently polling the keyboard to check for a keystroke.  You
specify how many times the program is allowed to check for a keystroke in
one clock tick (1/18 of a second), and DESQview does the rest.  No extra
programs to load, no shared programs, etc.

Usage 1:   DVPTAME = <dvp-file>
     Report the current polling limit

Usage 2:   DVPTAME <n> <dvp-file>
     Set the polling limit to <n> keyboard polls per clock tick

In both cases, <dvp-file> is the FULL filename of the .DVP for the program
you wish to tame.

In many cases, DVPTAME will perform just as well as the shareware TAME.
However, TAME can suspend programs which don't poll the keyboard often
but make their idleness known in other ways, and can tame programs which
poll the keyboard frequently even while they are still doing useful work.


        Ralf Brown
        ralf@cs.cmu.edu
        1:129/26.1
END OF QUOTE

As you can see, it appears to be DESQvew-specific at the moment.  However,
complete asm source code is included in the archive that I have so someone who
is more incentivized than I am at the moment may be able to make it work with
DOSEMU.

DVPTAME doesn't appear to be easily available online at the moment; at least,
Google didn't find it for me.

Accordingly, I have put it online here for anyone who may want to take a look
at it:

http://www.melvilletheatre.com/dvptame.tar.bz2


-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

Re: Performance problems

[Boscovich, Maximiliano]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Cox escribió:
> On Sat, 27 Oct 2007 12:19:54 -0600
> Frank Cox <theatre@sasktel.net> wrote:
>
>> It seems to me that there was another program that does pretty much
the same
>> thing as TAME but without the send-money nag screen, i.e. a free
program.  But
>> I can't recall its name at the moment....
>
> I found it.
>
> Here is the readme file:
>
> QUOTE:
> DVPTAME   Released to the Public Domain, 1991.
>
> DVPTAME lets DESQview suspend programs before their time slice is up if
> they are frequently polling the keyboard to check for a keystroke.  You
> specify how many times the program is allowed to check for a keystroke in
> one clock tick (1/18 of a second), and DESQview does the rest.  No extra
> programs to load, no shared programs, etc.
>
> Usage 1:   DVPTAME = <dvp-file>
>      Report the current polling limit
>
> Usage 2:   DVPTAME <n> <dvp-file>
>      Set the polling limit to <n> keyboard polls per clock tick
>
> In both cases, <dvp-file> is the FULL filename of the .DVP for the program
> you wish to tame.
>
> In many cases, DVPTAME will perform just as well as the shareware TAME.
> However, TAME can suspend programs which don't poll the keyboard often
> but make their idleness known in other ways, and can tame programs which
> poll the keyboard frequently even while they are still doing useful work.
>
>
>         Ralf Brown
>         ralf@cs.cmu.edu
>         1:129/26.1
> END OF QUOTE
>
> As you can see, it appears to be DESQvew-specific at the moment.  However,
> complete asm source code is included in the archive that I have so
someone who
> is more incentivized than I am at the moment may be able to make it
work with
> DOSEMU.
>
> DVPTAME doesn't appear to be easily available online at the moment; at
least,
> Google didn't find it for me.
>
> Accordingly, I have put it online here for anyone who may want to take
a look
> at it:
>
> http://www.melvilletheatre.com/dvptame.tar.bz2
>
>
Thanks for all comments!!, i solved it!, the problem was hogthreshold
it was 0, now i put it in 1 and all it's ok. Thanks very much !.


- --


"One of the penalties for refusing to participate in politics is
 that you end up being governed by your
inferiors."                                      
- - Plato

- -----------------------------------------------------------------
                         Boscovich, Maximiliano
GNU/Linux Counter user #402782
Blog: http://maximilianoboscovich.wordpress.com
Jabber: boscovich@jabberes.org
MSN: joelboscovich@hotmail.com
GnuPG Public Key:
http://keyserver.veridis.com:11371/export?id=-8543752951164951938&created=1152911209000
Command in GNU/Linux:  gpg --keyserver wwwkeys.eu.pgp.net
- --search-keys mboscovich
Key fingerprint: 7A88 1636 BDA4 F680 AB00 0D8E 896E 7DF5 291E 227E
- ------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHMIrWiW599SkeIn4RAjYWAKCDYMzq93SxknI+X+pf8hGRTBWpgwCcCgu3
BLTmaO87NYObT/udyCoYdGY=
=fswD
-----END PGP SIGNATURE-----

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: performance problems...

[James Harper]

> 
> The default is 75% CPU for domain0, which would explain the unfair
> sharing. :-)
> 

I would have said that DomU had less than 25%, based on its nonexistent
performance.

Is it a generally held belief that this arrangement is a good thing? Or
does it just assume that I won't do anything silly like xen recompiles
under dom0?

Thanks

James

Re: performance problems...

[Keir Fraser]

On 13 Jan 2006, at 09:56, James Harper wrote:

>> The default is 75% CPU for domain0, which would explain the unfair
>> sharing. :-)
>>
>
> I would have said that DomU had less than 25%, based on its nonexistent
> performance.
>
> Is it a generally held belief that this arrangement is a good thing? Or
> does it just assume that I won't do anything silly like xen recompiles
> under dom0?

The default basically assumes that dom0 is a server for other domains, 
so you want to favour it if it's runnable. There isn't much science 
behind that choice of default, I can assure you. :-)

  -- Keir

RE: performance problems...

[James Harper]

Ha! Look at that. I did:

xm sched-sedf Domain-0 0 0 0 1 1
xm sched-sedf mail 0 0 0 1 1

and now I can do a cpuburn (burnP6) in dom0 and you barely notice it.

So... questions...

Sedf must be the default scheduler? I always thought it was bvt? What
are the default settings for it? Obviously they aren't optimal in my
case!

How can I query the current scheduler settings?

Thanks

James

> -----Original Message-----
> From: Mark Nijmeijer [mailto:mark@nijmeijer.com]
> Sent: Friday, 13 January 2006 12:17
> To: James Harper
> Subject: Re: [Xen-devel] performance problems...
> 
>  From the xen-users mailing list, can you try this ?
> 
> 2) Scheduling issue
> 
> The system was unusable with high CPU load in dom0, all other domains
> were almost dead, and there's apparantly no comprehensive
documentation
> of xen's schedulers (please point out the relevant docs to me if I'm
> wrong, all I found was sedf_scheduler_mini-HOWTO.txt,
> xenwiki/Scheduling and `man xm`, none of which helps to understand
> scheduling well enough to change its parameters). Thanks to Tim
Freeman
> I found an easy setup that works for me: I set the scheduling
parameters
> for all domains via "xm sched-sedf <domID> 0 0 0 1 1" or something
> similar corresponding to example "4 domains with weights 2:3:4:2" from
> the sedf_scheduler_mini-HOWTO.txt with an additional 1 as the
extratime
> parameter if you want to allow a domain to allocate more CPU time if
> other domains are idle.
> 
> 
> 
> James Harper wrote:
> 
> >>>Any suggestions as to where to start? Or is this a known and
> >>>solved problem in the latest -testing?
> >>>
> >>>
> >>There have been no changes in 3.0-testing that are likely relevant.
I
> >>don't think anyone else has reported this, so I'd look closely at
your
> >>bridging changes. Also please report the test setup in more detail.
> >>
> >>
> >You
> >
> >
> >>haven't connected dom0 directly to the bridge rather than vif0.0 to
> >>
> >>
> >the
> >
> >
> >>bridge have you?
> >>
> >>
> >>
> >
> >I just upgraded both my xen machines to the latest (yesterdays hg
pull)
> >and both appear to be doing the same thing now. One of them is SMP
> >though, so the problem is less apparent, but if do a 'cpuburnP6' on
> >dom0, the domU's slow down to a crawl.
> >
> >The problem really does appear to be scheduling related, not network,
so
> >I haven't adjusted the network bridge configuration yet, but will do
so
> >if you tell me it's worth a go.
> >
> >Thanks
> >
> >James
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xensource.com
> >http://lists.xensource.com/xen-devel
> >
> >

Re: performance problems...

[Keir Fraser]

On 13 Jan 2006, at 01:33, James Harper wrote:

> Sedf must be the default scheduler? I always thought it was bvt? What
> are the default settings for it? Obviously they aren't optimal in my
> case!

The default is 75% CPU for domain0, which would explain the unfair 
sharing. :-)

  -- Keir

RE: performance problems...

[James Harper]

> are the guests SMP?

They are all SMP capable (default -xen kernel config), but except for
one they are all running with 1 vcpu.

> What does 'xm list' show about the relative CPU burn rates?

As I only rebooted recently I don't have any actual figures, but Dom0 is
highish (which makes sense as it was doing a compile), and DomU was low.
Neither had unexpected values in them.

If you can give me the syntax for rolling back to a specific changeset
or time (or I can look it up, but I'm sure you'll know it off the top of
your head :) I can start a binary search on when it all went wrong, and
definitely confirm that the problem didn't exist earlier.

Thanks

James

RE: performance problems...

[Ian Pratt]

> I just upgraded both my xen machines to the latest 
> (yesterdays hg pull) and both appear to be doing the same 
> thing now. One of them is SMP though, so the problem is less 
> apparent, but if do a 'cpuburnP6' on dom0, the domU's slow 
> down to a crawl.

are the guests SMP?

What does 'xm list' show about the relative CPU burn rates?

Ian

RE: performance problems...

[James Harper]

> > Any suggestions as to where to start? Or is this a known and
> > solved problem in the latest -testing?
> 
> There have been no changes in 3.0-testing that are likely relevant. I
> don't think anyone else has reported this, so I'd look closely at your
> bridging changes. Also please report the test setup in more detail.
You
> haven't connected dom0 directly to the bridge rather than vif0.0 to
the
> bridge have you?
> 

I just upgraded both my xen machines to the latest (yesterdays hg pull)
and both appear to be doing the same thing now. One of them is SMP
though, so the problem is less apparent, but if do a 'cpuburnP6' on
dom0, the domU's slow down to a crawl.

The problem really does appear to be scheduling related, not network, so
I haven't adjusted the network bridge configuration yet, but will do so
if you tell me it's worth a go.

Thanks

James

RE: performance problems...

[James Harper]

>  > I'm using xen-3.0-testing from a week or two ago, and am
> > finding that a recompile of xen-3.0-testing from hg now is
> > all but killing the performance in the other domains.
> >
> > I am running an almost default setup, I've just made some
> > small changes to bridging.
> >
> > Any suggestions as to where to start? Or is this a known and
> > solved problem in the latest -testing?
> 
> There have been no changes in 3.0-testing that are likely relevant. I
> don't think anyone else has reported this, so I'd look closely at your
> bridging changes. Also please report the test setup in more detail.
You
> haven't connected dom0 directly to the bridge rather than vif0.0 to
the
> bridge have you?

Hmmm... I'm not quite sure what you mean here... I always thought that
in dom0 you just connected eth0 (or whatever you might have renamed it
to) into the bridge. Is this not the case?

'brctl show' gives me:
br0             8000.00508bea6159       no              trunk
br1             8000.00508bea6159       no              br0.2
                                                        vif2.0

I have an Ethernet interface called 'trunk', which is bridged to br0.
br0 then has vlan's on it (the vlan's have to be on the bridge
interface, not on the Ethernet interface, or it doesn't work!), giving
br0.2, which is in turn bridged into br1. The affected DomU is using
br1. None of br0, br1, or trunk have an ip address on them in Dom0. I
have another Ethernet interface called 'lan', which is a gigabit
interface and give's dom0 its connection to the lan, including ATA over
Ethernet. Anything hanging off of 'trunk' is purely for the use of
domU's. Only Dom0 does AoE, All domU's get their disk access via block
devices (which are themselves AoE) exported from dom0.

The slowdown is very very obvious, running a 'make world' in dom0 brings
domU to an almost standstill. Ctrl-Z on the make in dom0 instantly
brings domU back to life. Even if I disable the Ethernet interface in
domU (ifdown eth0), it still runs really really slowly. The system load
goes up in domU too, if that means anything.

Thanks

James

RE: performance problems...

[Ian Pratt]

 > I'm using xen-3.0-testing from a week or two ago, and am 
> finding that a recompile of xen-3.0-testing from hg now is 
> all but killing the performance in the other domains.
> 
> I am running an almost default setup, I've just made some 
> small changes to bridging.
> 
> Any suggestions as to where to start? Or is this a known and 
> solved problem in the latest -testing?

There have been no changes in 3.0-testing that are likely relevant. I
don't think anyone else has reported this, so I'd look closely at your
bridging changes. Also please report the test setup in more detail. You
haven't connected dom0 directly to the bridge rather than vif0.0 to the
bridge have you?

Ian

performance problems...

[James Harper]

I'm using xen-3.0-testing from a week or two ago, and am finding that a
recompile of xen-3.0-testing from hg now is all but killing the
performance in the other domains.

I am running an almost default setup, I've just made some small changes
to bridging.

Any suggestions as to where to start? Or is this a known and solved
problem in the latest -testing?

Thanks

James

RE: performance problems

[Ian Pratt]

I think you're going to have to do a few IO microbenchmarks such as ttcp
and dd to figure out what's going slow on your system -- something is
badly wrong. Are you sure the same hardware drivers are being used in
both cases?

Ian

> -----Original Message-----
> From: xen-devel-admin@lists.sourceforge.net 
> [mailto:xen-devel-admin@lists.sourceforge.net] On Behalf Of 
> Henning Glawe
> Sent: 19 January 2005 20:36
> To: xen-devel@lists.sourceforge.net
> Subject: [Xen-devel] performance problems
> 
> Moin,
> I just did a quick comparison regarding speed of xen vs. 
> vmware-workstation
> using a FAI installation test (FAI is a debian autoinstaller).
> 
> Test setup:
> host "burns", athlon xp 2100+, 512MB, via chipset
> - provides an nfsroot (on hda)
> - second disk is partitioned into hdb1, hdb5, hdb6
> - hdb5 and hdb6 are exported to vmware or xenU
> - installation pumps 4.1G of software from a local debian 
> mirror to hdb6
>   (seen as hda6 in xenU/vmware ws) using apt and configures 
> the software
> - assigned 192 M of Memory in both cases to the virtual machines
> - burns is connected to the local debian mirror using a 1000 
> mbit NIC and a
>   switch
> 
> xen setup:
> - xen 2.0.3, kernel 2.6.10, dma support enabled
> - the rest is an out-of-the-box-setup 
> - support scripts are installed to move /lib/tls out of the way (BTW:
>   wouldn't LD_ASSUME_KERNEL=2.4.10 have the same effect)
> - The installation took 7509 seconds
> 
> vmware ws setup:
> - locally configured kernel 2.6.9, dma enabled
> - bridged networking
> - booting from a virtual floppy drive
> - accesses hdb{5,6} directly as hda{5,6}
> - /lib/tls is left in-place
> - The installation took 3068 seconds
> 
> 
> Why takes the installation within Xen 7500 sec and in vmware 
> ws 3000 sec ? I
> expected Xen to be faster because of the lower overhead...
> What could cause this difference ?
> Could the scheduling be responsible ?
> 
> -- 
> c u
> henning
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive 
> Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
> 


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Henning Glawe]

On Wed, Jan 19, 2005 at 09:02:02PM -0000, Ian Pratt wrote:
> 
> I think you're going to have to do a few IO microbenchmarks such as ttcp
> and dd to figure out what's going slow on your system -- something is
> badly wrong. Are you sure the same hardware drivers are being used in
> both cases?

yup. the only difference was that I didn't touch dumU's kernel config; in
dom0 I already disabled the debugging flags (except magic sysreq), but in
default domU's config they were still enabled.

Overall impression from my newer benchmark experiments:

- Harddisk IO is about 30% faster in XenU than in vmware ws (both accessing
  the same raw partition)
- Network IO is at the same level in xenU and vmware, which is about 30%
  slower than dom0/native net io (netpipe, 2 machines using 1000Mbit ethernet
  connected via a switch, xenU and vmware bridged).

-- 
c u
henning


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

performance problems

[Henning Glawe]

Moin,
I just did a quick comparison regarding speed of xen vs. vmware-workstation
using a FAI installation test (FAI is a debian autoinstaller).

Test setup:
host "burns", athlon xp 2100+, 512MB, via chipset
- provides an nfsroot (on hda)
- second disk is partitioned into hdb1, hdb5, hdb6
- hdb5 and hdb6 are exported to vmware or xenU
- installation pumps 4.1G of software from a local debian mirror to hdb6
  (seen as hda6 in xenU/vmware ws) using apt and configures the software
- assigned 192 M of Memory in both cases to the virtual machines
- burns is connected to the local debian mirror using a 1000 mbit NIC and a
  switch

xen setup:
- xen 2.0.3, kernel 2.6.10, dma support enabled
- the rest is an out-of-the-box-setup 
- support scripts are installed to move /lib/tls out of the way (BTW:
  wouldn't LD_ASSUME_KERNEL=2.4.10 have the same effect)
- The installation took 7509 seconds

vmware ws setup:
- locally configured kernel 2.6.9, dma enabled
- bridged networking
- booting from a virtual floppy drive
- accesses hdb{5,6} directly as hda{5,6}
- /lib/tls is left in-place
- The installation took 3068 seconds


Why takes the installation within Xen 7500 sec and in vmware ws 3000 sec ? I
expected Xen to be faster because of the lower overhead...
What could cause this difference ?
Could the scheduling be responsible ?

-- 
c u
henning


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Ronald G. Minnich]

well, that's interesting, I wonder if disk I/o performance had a hand in 
those results. 

I ought to mention that full boot time for Plan 9 in xen is < 10 seconds. 

ron



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Andrew Theurer]

On Wednesday 19 January 2005 14:35, Henning Glawe wrote:
> Moin,
> I just did a quick comparison regarding speed of xen vs. vmware-workstation
> using a FAI installation test (FAI is a debian autoinstaller).
>
> Test setup:
> host "burns", athlon xp 2100+, 512MB, via chipset
> - provides an nfsroot (on hda)
> - second disk is partitioned into hdb1, hdb5, hdb6
> - hdb5 and hdb6 are exported to vmware or xenU
> - installation pumps 4.1G of software from a local debian mirror to hdb6
>   (seen as hda6 in xenU/vmware ws) using apt and configures the software
> - assigned 192 M of Memory in both cases to the virtual machines
> - burns is connected to the local debian mirror using a 1000 mbit NIC and a
>   switch

I really don't know how this installation works, but just in case:  Is it 
possible "burns" only pulls from the mirror the first time, and on subsequent 
installations it is cached?  I am wondering if the first test (Xen) it had to 
pull data across the network, and on the second test (Vmware) already had the 
data cached on burns.

So far I haven't seem much degrade at all in the tests I have run (dbench3, 
SDET, kernel compiles).  Actually I was quite surprised how well the VBD's IO 
performance was so far.  The only "significant" degrade I have seen was SDET, 
around 12.5% lower than bare metal linux.  I have not looked into network 
performance yet, but I think that'll be next on my list...

-Andrew Theurer



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: performance problems

[Luciano Miguel Ferreira Rocha]

On Wed, Jan 19, 2005 at 09:35:51PM +0100, Henning Glawe wrote:
> Moin,
> I just did a quick comparison regarding speed of xen vs. vmware-workstation
> using a FAI installation test (FAI is a debian autoinstaller).
> 
> Test setup:
> host "burns", athlon xp 2100+, 512MB, via chipset
> - provides an nfsroot (on hda)
> - second disk is partitioned into hdb1, hdb5, hdb6
> - hdb5 and hdb6 are exported to vmware or xenU

Don't forget that hdd perfomance decreases as you reach the end of the
disc, and it could have caused a noticeable performance hit.

Regards,
Luciano Rocha

-- 
1/16


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl