Hyperthreading network benchmark

Hyperthreading network benchmark

[kuas]

Hi all,

I found enabling hyperthreading for Xen (the hypervisor layer) would 
degrade network performance. I wonder if the synchronization cost of Xen 
(the hypervisor) running on multi cores/multiprocessors environment is 
high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz 
Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using 
WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package).  I 
was also running 2.4.27 for the domains kernel. The result I have:
1. Native:
    a. Hyperthread enabled: 370 Mb/sec
    b. Hyperthread disabled: 270-290 Mb/sec
2. Domain 0 has the same performance around: 270 Mb/sec
3. Domain 1:
    a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec
    b. Hyperthreaded disabled or even if enabled domain 1 is forced to 
be in the same Core with domain 0: 255-265 Mb/sec

Please note, I was only sending request to one domain at a time. It 
seems when the two domain ran in the same core we had higher 
performance. I wonder if any other people has done almost similar test 
and have similiar/opposite behavior. Are these behaviors make sense?

Thanks in advance for any comments.

Kuas.




-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Hyperthreading network benchmark

[Kip Macy]

They've seen anywhere from a 30% decrease to 30% increase in performance
on SPECint with hyperthreading turned on. Bear in mind you're now
sharing a single cache between two processes. For many workloads the
trace cache on the P4 was too small to begin with.

				-Kip


On Sat, 30 Oct 2004, kuas wrote:

> Hi all,
>
> I found enabling hyperthreading for Xen (the hypervisor layer) would
> degrade network performance. I wonder if the synchronization cost of Xen
> (the hypervisor) running on multi cores/multiprocessors environment is
> high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz
> Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using
> WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package).  I
> was also running 2.4.27 for the domains kernel. The result I have:
> 1. Native:
>     a. Hyperthread enabled: 370 Mb/sec
>     b. Hyperthread disabled: 270-290 Mb/sec
> 2. Domain 0 has the same performance around: 270 Mb/sec
> 3. Domain 1:
>     a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec
>     b. Hyperthreaded disabled or even if enabled domain 1 is forced to
> be in the same Core with domain 0: 255-265 Mb/sec
>
> Please note, I was only sending request to one domain at a time. It
> seems when the two domain ran in the same core we had higher
> performance. I wonder if any other people has done almost similar test
> and have similiar/opposite behavior. Are these behaviors make sense?
>
> Thanks in advance for any comments.
>
> Kuas.
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Sybase ASE Linux Express Edition - download now for FREE
> LinuxWorld Reader's Choice Award Winner for best database on Linux.
> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Hyperthreading network benchmark

[Adam Heath]

On Sat, 30 Oct 2004, Kip Macy wrote:

> They've seen anywhere from a 30% decrease to 30% increase in performance
> on SPECint with hyperthreading turned on. Bear in mind you're now
> sharing a single cache between two processes. For many workloads the
> trace cache on the P4 was too small to begin with.

Not just 2 processes.  2 whole operating systems.  An operating system has a
much higher overhead, and you'll probably get lots of cache trashing.

Note, once xen supports SMP, then it might help if it's scheduler were
HT-aware.  Also, why not use normal linux schedulers(don't know if you
currently are)?


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click

Re: Hyperthreading network benchmark

[Mark A. Williamson]

> I found enabling hyperthreading for Xen (the hypervisor layer) would
> degrade network performance. I wonder if the synchronization cost of Xen
> (the hypervisor) running on multi cores/multiprocessors environment is
> high (around 30 %).

If the domain controlling the network interface is on a different (logical) 
processor to the domain which is currently using it, there is an overhead 
because inter-processor interrupts (IPIs) must be sent to notify the other 
CPU when data is ready.

Xen's pipelined, asynchronous device channels attempt to avoid this problem 
but some workloads (e.g. small packets at a very high rate) will still 
unavoidably stress the system.

We found in tests that hyperthreading improved performance over simply using 
one core, when running dom0 on one hyperthread and dom1 on another.  An IPI 
is still sent between the two logical CPUs, which still has overhead, however 
the performace seemed to be improved because context switches were avoided, 
relative to time-slicing both domains on one CPU.

It's interesting that performance reduced for you when enabling HT.  Perhaps 
the context-swtich vs IPI tradeoff is different for you, either for hardware 
reasons (we were using 3GHz Xeons, so had a different core, larger caches, 
etc.) or because of some software change.

My $0.02
Mark


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click