From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Ronaldo C. A. Chaves" Subject: Re: Re: network performance drop heavily in xen 4.0 release Date: Sat, 24 Apr 2010 12:07:22 -0300 Message-ID: References: <20100418185506.GN1878@reaktio.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1237576758==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: yingbin wang Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --===============1237576758== Content-Type: multipart/alternative; boundary=001636e1f81f9892f30484fce661 --001636e1f81f9892f30484fce661 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I compared the config-2.6.31-13xen* *with the config-2.6.31.13-high performance. 2010/4/24 yingbin wang > which .config you compare with? > > 2010/4/24 Ronaldo C. A. Chaves : > > I compared the files and the difference is > > > > CONFIG_BLK_DEV_LOOP=3Dm > > CONFIG_ATA_PIIX=3Dm > > CONFIG_XEN_DEV_EVTCHN=3Dy > > > > in config-2.6.31.13-high performance. > > > > > > 2010/4/24 yingbin wang > >> > >> of course. > >> the attachment is the dom0 kernel compile config that fix the problem. > >> I don't know the exact config option which cause the problem, so I > >> don't test 2.6.18.8 with debug.you can compare it with the previous > >> config to find the differences. > >> > >> Here are my test results: > >> > >> test tool: iperf-2.0.4 > >> command: > >> root@10.250.6.25 : iperf -s > >> root@10.250.6.28 : iperf -c 10.250.6.25 -i 1 -t 100 > >> > >> network performance: > >> > >> xen4.0+kernel2.6.31.13(with debug): > >> [ ID] Interval Transfer Bandwidth > >> [ 4] 0.0- 9.5 sec 249 MBytes 219 Mbits/sec > >> > >> xen4.0+kernel2.6.31.13(without debug): > >> [ ID] Interval Transfer Bandwidth > >> [ 4] 0.0-100.0 sec 10.7 GBytes 920 Mbits/sec > >> > >> xen3.4.2+kernel2.6.18.8(without debug): > >> [ ID] Interval Transfer Bandwidth > >> [ 4] 0.0-15.0 sec 1.64 GBytes 941 Mbits/sec > >> > >> xen4.0+kernel2.6.18.8(without debug): > >> [ ID] Interval Transfer Bandwidth > >> [ 4] 0.0-16.3 sec 1.79 GBytes 941 Mbits/sec > >> > >> Cheers, > >> wyb > >> > >> 2010/4/19 Pasi K=E4rkk=E4inen : > >> > On Fri, Apr 16, 2010 at 04:49:42PM +0800, yingbin wang wrote: > >> >> the problem is solved. > >> >> > >> >> we closed most of the debug config options. a miracle happened. the > >> >> performance returned to the level before. > >> >> we compared the .config of 2.6.18.8 with 2.6.31.13. the differences > >> >> are the debug options. > >> >> I think the default .config in 2.6.31.13 should close the debug > >> >> options or provide a way to turn off. > >> >> > >> > > >> > Could you please post the exact .config options you turned off to fi= x > >> > the problem? > >> > I can add that info to the wiki page. > >> > > >> > Also can you please post the performance numbers with 2.6.18.8 and > >> > pvops dom0 with and without debug? This would be interesting to know= . > >> > > >> > Thanks! > >> > > >> > -- Pasi > >> > > >> >> thanks all > >> >> > >> >> Cheers, > >> >> wyb > >> >> > >> >> 2010/4/16 yingbin wang : > >> >> > Hi: > >> >> > I report a Bug !!! We have just upgraded to > >> >> > xen4.0+kernel2.6.31.13 recently. however , fond that the network > >> >> > performance drop heavily in dom0 (nearly Reduced by 2/3 vs > >> >> > xen3.4.2+kernel2.6.18.8 ) . > >> >> > > >> >> > our env : > >> >> > hardware : > >> >> > Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > >> >> > 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II > >> >> > BCM5709 Gigabit Ethernet (rev 20) > >> >> > 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II > >> >> > BCM5709 Gigabit Ethernet (rev 20) > >> >> > compile env and filesystem : > >> >> > Redhat AS 5.4 > >> >> > > >> >> > xm info : > >> >> > ----------------------------------------------------------------- > >> >> > host : r02k08015 > >> >> > release : 2.6.31.13xen > >> >> > version : #1 SMP Tue Apr 13 20:38:51 CST 2010 > >> >> > machine : x86_64 > >> >> > nr_cpus : 16 > >> >> > nr_nodes : 2 > >> >> > cores_per_socket : 4 > >> >> > threads_per_core : 2 > >> >> > cpu_mhz : 2266 > >> >> > hw_caps : > >> >> > > >> >> > > bfebfbff:28100800:00000000:00001b40:009ce3bd:00000000:00000001:00000000 > >> >> > virt_caps : hvm > >> >> > total_memory : 24539 > >> >> > free_memory : 15596 > >> >> > node_to_cpu : node0:0,2,4,6,8,10,12,14 > >> >> > node1:1,3,5,7,9,11,13,15 > >> >> > node_to_memory : node0:3589 > >> >> > node1:12007 > >> >> > node_to_dma32_mem : node0:2584 > >> >> > node1:0 > >> >> > max_node_id : 1 > >> >> > xen_major : 4 > >> >> > xen_minor : 0 > >> >> > xen_extra : .0 > >> >> > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p > >> >> > hvm-3.0-x86_32 > >> >> > hvm-3.0-x86_32p hvm-3.0-x86_64 > >> >> > xen_scheduler : credit > >> >> > xen_pagesize : 4096 > >> >> > platform_params : virt_start=3D0xffff800000000000 > >> >> > xen_changeset : unavailable > >> >> > xen_commandline : dom0_mem=3D10240M > >> >> > cc_compiler : gcc version 4.1.2 20080704 (Red Hat > >> >> > 4.1.2-46) > >> >> > cc_compile_by : root > >> >> > cc_compile_domain : > >> >> > cc_compile_date : Tue Apr 13 23:04:16 CST 2010 > >> >> > xend_config_format : 4 > >> >> > > >> >> > > -------------------------------------------------------------------------= -- > >> >> > > >> >> > > >> >> > test tool: iperf-2.0.4 > >> >> > command: > >> >> > root@10.250.6.25 : iperf -s > >> >> > root@10.250.6.28 : iperf -c 10.250.6.25 -i 1 -t 100 > >> >> > > >> >> > network performance: > >> >> > > >> >> > xen4.0+kernel2.6.31.13: > >> >> > [ ID] Interval Transfer Bandwidth > >> >> > [ 4] 0.0- 9.5 sec 249 MBytes 219 Mbits/sec > >> >> > > >> >> > xen3.4.2+kernel2.6.18.8: > >> >> > [ ID] Interval Transfer Bandwidth > >> >> > [ 4] 0.0-15.0 sec 1.64 GBytes 941 Mbits/sec > >> >> > > >> >> > BTW ,1 the disk IO performance also reduce from 90MB/s to 60MB/s. > >> >> > 2 the attachment is the dom0 kernel compile config. > >> >> > > >> >> > Cheers, > >> >> > wyb > >> >> > > >> >> > >> >> _______________________________________________ > >> >> Xen-devel mailing list > >> >> Xen-devel@lists.xensource.com > >> >> http://lists.xensource.com/xen-devel > >> > > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel > >> > > > > > --001636e1f81f9892f30484fce661 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I compared the config-2.6.31-13xen with the config-2.6.31.13-high pe= rformance.


2010/4/24 yingbin wang <yingbin.wang= yb@gmail.com>
which .config you= compare with?

2010/4/24 Ronaldo C. A. Chaves <xarq= ui@gmail.com>:
> I compared the files and the differe= nce is
>
> CONFIG_BLK_DEV_LOOP=3Dm
> CONFIG_ATA_PIIX=3Dm
> CONFIG_XEN_DEV_EVTCHN=3Dy
>
> in config-2.6.31.13-high performance.
>
>
> 2010/4/24 yingbin wang <yingbin.wangyb@gmail.com>
>>
>> of course.
>> the attachment is the dom0 kernel compile config that fix the prob= lem.
>> I don't know the exact config option which cause the problem, = so I
>> don't test 2.6.18.8 with debug.you can compare it with the pre= vious
>> config to find the differences.
>>
>> Here are my test results:
>>
>> test tool: =A0iperf-2.0.4
>> command:
>> root@10.250.6.25 : =A0 =A0= =A0iperf -s
>> root@10.250.6.28 : =A0 =A0= =A0iperf -c 10.250.6.25 -i 1 -t 100
>>
>> network performance:
>>
>> xen4.0+kernel2.6.31.13(with debug):
>> [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidth
>> [ =A04] =A00.0- 9.5 sec =A0 =A0249 MBytes =A0 =A0219 Mbits/sec
>>
>> xen4.0+kernel2.6.31.13(without debug):
>> [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidth
>> [ =A04] =A00.0-100.0 sec =A010.7 GBytes =A0 =A0920 Mbits/sec
>>
>> xen3.4.2+kernel2.6.18.8(without debug):
>> [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidth
>> [ =A04] =A00.0-15.0 sec =A01.64 GBytes =A0 =A0941 Mbits/sec
>>
>> xen4.0+kernel2.6.18.8(without debug):
>> [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidth
>> [ =A04] =A00.0-16.3 sec =A01.79 GBytes =A0 =A0941 Mbits/sec
>>
>> Cheers,
>> wyb
>>
>> 2010/4/19 Pasi K=E4rkk=E4inen <= pasik@iki.fi>:
>> > On Fri, Apr 16, 2010 at 04:49:42PM +0800, yingbin wang wrote:=
>> >> the problem is solved.
>> >>
>> >> we closed most of the debug config options. a miracle hap= pened. the
>> >> performance returned to the level before.
>> >> we compared the .config of 2.6.18.8 with 2.6.31.13. the d= ifferences
>> >> are the debug options.
>> >> I think the default .config in 2.6.31.13 should close the= debug
>> >> options or provide a way to turn off.
>> >>
>> >
>> > Could you please post the exact .config options you turned of= f to fix
>> > the problem?
>> > I can add that info to the wiki page.
>> >
>> > Also can you please post the performance numbers with 2.6.18.= 8 and
>> > pvops dom0 with and without debug? This would be interesting = to know.
>> >
>> > Thanks!
>> >
>> > -- Pasi
>> >
>> >> thanks all
>> >>
>> >> Cheers,
>> >> wyb
>> >>
>> >> 2010/4/16 yingbin wang <yingbin.wangyb@gmail.com>:
>> >> > Hi:
>> >> > =A0 =A0 I report a Bug !!! =A0We have just upgraded = to
>> >> > xen4.0+kernel2.6.31.13 recently. =A0however , fond t= hat the network
>> >> > performance drop heavily in dom0 =A0(nearly Reduced = by 2/3 vs
>> >> > xen3.4.2+kernel2.6.18.8 ) .
>> >> >
>> >> > our env :
>> >> > hardware :
>> >> > =A0 Intel(R) Xeon(R) CPU =A0 =A0 =A0 =A0 =A0 E5520 = =A0@ 2.27GHz
>> >> > =A0 01:00.0 Ethernet controller: Broadcom Corporatio= n NetXtreme II
>> >> > BCM5709 Gigabit Ethernet (rev 20)
>> >> > =A0 01:00.1 Ethernet controller: Broadcom Corporatio= n NetXtreme II
>> >> > BCM5709 Gigabit Ethernet (rev 20)
>> >> > compile env and filesystem :
>> >> > =A0 Redhat AS 5.4
>> >> >
>> >> > xm info :
>> >> > ----------------------------------------------------= -------------
>> >> > host =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 : r02k08015=
>> >> > release =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 2.6.31.13xe= n
>> >> > version =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: #1 SMP Tue = Apr 13 20:38:51 CST 2010
>> >> > machine =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: x86_64
>> >> > nr_cpus =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 16
>> >> > nr_nodes =A0 =A0 =A0 =A0 =A0 =A0 =A0 : 2
>> >> > cores_per_socket =A0 =A0 =A0 : 4
>> >> > threads_per_core =A0 =A0 =A0 : 2
>> >> > cpu_mhz =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 2266
>> >> > hw_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:
>> >> >
>> >> > bfebfbff:28100800:00000000:00001b40:009ce3bd:0000000= 0:00000001:00000000
>> >> > virt_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0: hvm
>> >> > total_memory =A0 =A0 =A0 =A0 =A0 : 24539
>> >> > free_memory =A0 =A0 =A0 =A0 =A0 =A0: 15596
>> >> > node_to_cpu =A0 =A0 =A0 =A0 =A0 =A0: node0:0,2,4,6,8= ,10,12,14
>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 node= 1:1,3,5,7,9,11,13,15
>> >> > node_to_memory =A0 =A0 =A0 =A0 : node0:3589
>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 node= 1:12007
>> >> > node_to_dma32_mem =A0 =A0 =A0: node0:2584
>> >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 node= 1:0
>> >> > max_node_id =A0 =A0 =A0 =A0 =A0 =A0: 1
>> >> > xen_major =A0 =A0 =A0 =A0 =A0 =A0 =A0: 4
>> >> > xen_minor =A0 =A0 =A0 =A0 =A0 =A0 =A0: 0
>> >> > xen_extra =A0 =A0 =A0 =A0 =A0 =A0 =A0: .0
>> >> > xen_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0 : xen-3.0-x86_6= 4 xen-3.0-x86_32p
>> >> > hvm-3.0-x86_32
>> >> > hvm-3.0-x86_32p hvm-3.0-x86_64
>> >> > xen_scheduler =A0 =A0 =A0 =A0 =A0: credit
>> >> > xen_pagesize =A0 =A0 =A0 =A0 =A0 : 4096
>> >> > platform_params =A0 =A0 =A0 =A0: virt_start=3D0xffff= 800000000000
>> >> > xen_changeset =A0 =A0 =A0 =A0 =A0: unavailable
>> >> > xen_commandline =A0 =A0 =A0 =A0: dom0_mem=3D10240M >> >> > cc_compiler =A0 =A0 =A0 =A0 =A0 =A0: gcc version 4.1= .2 20080704 (Red Hat
>> >> > 4.1.2-46)
>> >> > cc_compile_by =A0 =A0 =A0 =A0 =A0: root
>> >> > cc_compile_domain =A0 =A0 =A0:
>> >> > cc_compile_date =A0 =A0 =A0 =A0: Tue Apr 13 23:04:16= CST 2010
>> >> > xend_config_format =A0 =A0 : 4
>> >> >
>> >> > ----------------------------------------------------= -----------------------
>> >> >
>> >> >
>> >> > test tool: =A0iperf-2.0.4
>> >> > command:
>> >> > root@10.250.6.25= : =A0 =A0 =A0iperf -s
>> >> > root@10.250.6.28= : =A0 =A0 =A0iperf -c 10.250.6.25 -i 1 -t 100
>> >> >
>> >> > network performance:
>> >> >
>> >> > xen4.0+kernel2.6.31.13:
>> >> > [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidt= h
>> >> > [ =A04] =A00.0- 9.5 sec =A0 =A0249 MBytes =A0 =A0219= Mbits/sec
>> >> >
>> >> > xen3.4.2+kernel2.6.18.8:
>> >> > [ ID] Interval =A0 =A0 =A0 Transfer =A0 =A0 Bandwidt= h
>> >> > [ =A04] =A00.0-15.0 sec =A01.64 GBytes =A0 =A0941 Mb= its/sec
>> >> >
>> >> > BTW ,1 the disk IO performance also reduce from 90MB= /s to 60MB/s.
>> >> > =A0 =A0 =A0 =A0 2 the attachment is the dom0 kernel = compile config.
>> >> >
>> >> > Cheers,
>> >> > wyb
>> >> >
>> >>
>> >> _______________________________________________
>> >> Xen-devel mailing list
>> >> Xen-deve= l@lists.xensource.com
>> >> http://lists.xensource.com/xen-devel
>> >
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.x= ensource.com
>> http://lists.xensource.com/xen-devel
>>
>
>

--001636e1f81f9892f30484fce661-- --===============1237576758== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1237576758==--