Re: x86-64 Net Performance

[Andrew Theurer <habanero@us.ibm.com>]

Ian Pratt wrote:

> 
>  
>
>>Update: I went ahead and compiled glibc without TLS / NPTL.  
>>    
>>
>
>For 64 bit Xen you don't need to worry about TLS/NPTL.
>
>  
>
>>Also, I realized that I wasn't allocating all my physical 
>>memory to Domain0 for these tests, so I was giving Xen a 
>>disadvantage.  It didn't change much, but those tests have 
>>been re-done.
>>
>>Here's the odd thing: disabling TLS for 64-bit improved 
>>native performance by around 10%, but had no effect on Xen.
>>    
>>
>
>How did you disable TLS on 64 bit? It's odd that it would have any
>effect on native or xen, let alone such a dramatic one. Are you sure
>about this test?
>
>  
>
>>                Native  Domain0 Penalty
>>64-bit TLS      50.2    65.0    29.5%
>>64-bit no TLS   44.8    65.0    45.1%
>>32-bit TLS      59.6    59.5    -0.2%
>>32-bit no TLS   59.6    60.2     1.0%
>>    
>>
>
>It looks like we need to investigate x86_64 networking performance. We
>may well not be getting the pipelining that we do in 32bit, though I
>can't immediately think why,
>  
>
This is a bit surprising as most newer cpus (really any amd_64) has 
plenty of processing power, and the Ethernet adapter is almost always 
the bottleneck.  This was with netperf, but I would not expect a http 
get of a single file to be much worse..  I have also seen more overhead 
in ia32 than amd_64.  Xenoprofile showed (for me) about a 13% cpu 
overhead (more than baremetal linux) in amd_64, 3% in xen and about 10% 
for the sw bridge.  On ia32 (intel xeon) this was about 23%, 13% in xen 
and 10% for the bridge.  I guess I should run these over again and 
check; maybe something have dramatically changed things.

You might want to try this test without starting xend, so the network 
bridge is not created and eth0 is put on it.  Perhaps that is an issue.  
Come to think of it, my tests were not with veth0/vif0.0 -that may 
indeed be an issue, going through the front/back-end virtual interfaces, 
then on the bridge, then on eth0.  I'll kick off some tests tomorrow and 
see what I get.

-Andrew