Re: Benchmarking Xen (results and questions)

[Andrew Theurer <habanero@us.ibm.com>]

This is much better.  I think my email client just didn't format your 
first (html?) message.  For JBB, I suspect the degrade is mostly cache 
thrashing, and the increased timeslice = better cache warmth.  Perhaps 
there a lot of overhead in the domain context switch as well.  What is 
the cpu cache size?

I will soon be doing tests like these on a dozen or so benchmarks, so 
hopefully we can compare results.  Have you looked at using xenoprofile 
while running your tests?

As for the WebBench numbers, do you have cpu utilization stats for Host 
and each of the domains?  Is it possible the Host and 1 VM are not using 
much cpu at all, and adding domains/clients just increases cpu util & TPS?

-Andrew

David_Wolinsky@Dell.com wrote:

>That's funny, They were sent out formatted nicely...  Tests were run
>multiple times for consistency purposes...  Using BEA Jrockit 1.5.
>
>	SPECJBB	WebBench			
>	1 Thread	1 Client	2 Clients	4 Clients
>8 Clients
>	BOPS	TPS	TPS	TPS	TPS
>Host	32403.5	213.45	416.86	814.62	1523.78
>1 VM	32057		205.4		380.91	569.24	733.8
>2 VM	24909.25	NA		399.29	695.1		896.04
>4 VM	17815.75	NA		NA		742.78	950.63
>8 VM	10216.25	NA		NA		NA
>1002.81
>
>	Period	Slice		BOPs
>8 VM	1 ms		125 us	6858
>8 VM	10 ms		1.25 ms	14287
>8 VM	100 ms	12.5 ms	18912
>8 VM	1 Sec		.125 Sec	20695
>8 VM	2 Sec		.25 Sec	21072
>8 VM	10 Sec	1.25 Sec	21797
>8 VM	100 Sec	12.5 Sec	11402
>
>Hope it works this time.  If not, I'll submit as an attachment.
>
>David
>
>-----Original Message-----
>From: Andrew Theurer [mailto:habanero@us.ibm.com] 
>Sent: Wednesday, August 03, 2005 8:43 PM
>To: Wolinsky, David
>Cc: xen-devel@lists.xensource.com
>Subject: Re: [Xen-devel] Benchmarking Xen (results and questions)
>
>David_Wolinsky@Dell.com wrote:
>
>  
>
>>Hi all,
>>
>>Here are some benchmarks that I've done using Xen.
>>
>>However, before I get started, let me explain some of configuration 
>>details...
>>
>>Xen Version SPECjbb
>>WebBench
>>Linux Distribution Debian 3.1
>>HT disabled
>>Linux Kernel 2.6.12.2
>>Host Patch CK3s
>>
>>Here are the initial benchmarks
>>
>>SPECJBB WebBench
>>1 Thread 1 Client 2 Clients 4 Clients 8 Clients BOPS TPS TPS TPS TPS 
>>Host 32403.5 213.45 416.86 814.62 1523.78
>>1 VM 32057 205.4 380.91 569.24 733.8
>>2 VM 24909.25 NA 399.29 695.1 896.04
>>4 VM 17815.75 NA NA 742.78 950.63
>>8 VM 10216.25 NA NA NA 1002.81
>>
>>(and some more notes.... BOPS - business operations per second, TPS - 
>>transactions per second... SPECjbb tests CPU and Memory WebBench (the 
>>way we configured it) tests Network I/O and Disk I/O
>>
>>Values = AVG * VM count
>>Domain configurations
>>1 VM - 1660 MB - SPECJBB 1500MB
>>2 VM - 1280 MB - SPECJBB - 1024MB
>>4 VM - 640 MB - SPECJBB - 512 MB
>>8 VM - 320 MB - SPECJBB - 256 MB
>>
>>Seeing how the SPECjbb numbers declined so bizarrely, I did some 
>>scheduling tests and found this out...
>>
>>Test1: Examine Xen's scheduling to determine if context switching is 
>>causing the overhead Period Slice BOPs Modified 8 VM 1 ms 125 us 6858
>>8 VM 10 ms 1.25 ms 14287
>>8 VM 100 ms 12.5 ms 18912
>>8 VM 1 Sec .125 Sec 20695
>>8 VM 2 Sec .25 Sec 21072
>>8 VM 10 Sec 1.25 Sec 21797
>>8 VM 100 Sec 12.5 Sec 11402
>>
>>    
>>
>Did you run each JBB test config several times to ensure consistent
>results? What JVM is this?
>
>Would it be possible to format these more appropriately for email? I am
>having a little trouble reading this :)
>
>Thanks,
>
>-Andrew
>
>  
>