Re: Opteron server and NUMA

[Alexander Charbonnet <alexander@charbonnet.com>]

The traffic over the local adapter between my app and MySQL may be 
significant.  I've tried both the socket connection and the TCP connection 
without a big difference.

I recompiled with the necessary kernel option and ran:
tc qdisc add dev eth0 root tbf rate 50mbit latency 20ms burst 50k
as suggested by one of your links, but there wasn't a significant difference.

I used ab to benchmark grabbing a 100MB file 100 times from localhost.  
Natively it took 15.9 seconds.  Under Domain0 it took 41% longer, 22.4 
seconds.  That sure looks like it makes up a huge chunk of the overall 
slowdown I'm noticing.

Any other tricks to try to get network performance closer to native?

Thanks!

Alex


On Saturday 08 October 2005 06:16 am, Nicholas Lee wrote:
> On 10/8/05, Alexander Charbonnet <alexander@charbonnet.com> wrote:
> > I now think that lack of NUMA support isn't the problem.  I've done some
> > benchmarking of my database-backed Perl Web app with the Apache benchmark
> > tool, "ab".  I ran the same query with differing total requests and
> > concurrency.  After the first time, which I did manually before the
> > tests, all the results will be in the database query cache.  Therefore, I
> > don't expect any disk I/O.
>
> Could be Net IO.  One thing I note with my Xen 2.0 setup is that it
> wasn't very high performance.
>
> There are a couple of emails I've noted but haven't had the time to try out
> yet:
>
> http://lists.xensource.com/archives/html/xen-users/2005-08/msg00211.html
>
> "virtual eth devices had packet queueing disabled (txqueuelen:0 on
> ifconfig), and had a 4k size max. ring buffer for
> transfering packets between domains. Whenever that buffer would run
> full, packets are dropped..."
>
> http://lists.xensource.com/archives/html/xen-users/2005-07/msg00708.html
>
> "The problem is that the data structure in which the ring buffer is
> organized has to fit into one memory page (4096 bytes)"
>
> http://lists.xensource.com/archives/html/xen-devel/2005-09/msg00196.html
>
> "default qdisc being inadequate"
> "tc qdisc add dev eth0 root tbf rate 50mbit latency 20ms burst 50k"
>
>
> One of the  problems with Xen at the moment is the lack of
> documentation/cluebooks for stuff like this at the moment. Best
> practice guides etc.  Still a learning process.
>
> --
> Nicholas Lee
> http://stateless.geek.nz
> gpg 8072 4F86 EDCD 4FC1 18EF  5BDD 07B0 9597 6D58 D70C
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel