From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= Subject: Re: [Xen-devel] RE: How to: Improve Network Throughput of a XEN kernel? Date: Wed, 3 Mar 2010 11:18:44 +0200 Message-ID: <20100303091844.GI2761@reaktio.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-users-bounces@lists.xensource.com Errors-To: xen-users-bounces@lists.xensource.com To: Mike Viau Cc: bderzhavets@yahoo.com, xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Tue, Mar 02, 2010 at 07:50:59PM -0500, Mike Viau wrote: > >Any just first argument dummy=3Ddummy should on kenel line. > > Read Wiki one more time. It works. >=20 > I believe you that it does work. I was able to see (XEN) messages coming = =66rom the hypervisor, but no matter how I changed the option I kept gettin= g a kernel fault. >=20 > I feel like it was time to stop mucking around with getting Ubuntu and th= e grub2 boot loader to work with XEN so I decided to install a headless Ope= nSUSE 11.2 system. >=20 > The openSUSE distribution supported Xen 3.4.1 tools, and had a 2.6.31.12 = xenified kernel. On this setup I experience identical network performance. >=20 > ~1MB/s - w/ xen kernel > ~4.5 MB/s - w/ non-xen kernel >=20 > I use the e1000e kernel module driver for my Ethernet card. >=20 > Is this typical Dom0 (or Xen Host) network performance with a xenified ke= rnel or is just in my two setups that one encounters such a network perform= ance hit to the Dom0? > No, it's not typical. How are you measuring the performance? In a gigabit network you should be a= ble to push around 110 MB/sec.. 4.5 MB/sec sounds really slow. =20 -- Pasi > -M >=20 > --- On Mon, 3/1/10, Mike Viau wrote: >=20 > From: Mike Viau > Subject: RE: [Xen-devel] RE: [Xen-users] How to: Improve Network Throug= hput of a XEN kernel? > To: bderzhavets@yahoo.com > Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com > Date: Monday, March 1, 2010, 2:52 PM >=20 > Why using grub2 vs grub is baffling am not sure.... >=20 > I think I'd prefer to use the kernel command line in grub2 so that the = options may be changed easier laster on. However I still have been having s= o success with boot > XEN with Ubuntu's grub2 boot loader. >=20 > Am I correct to conclude CONFIG_CMDLINE_BOL must =3D N to allow GRUB2 t= o pass kernel options? > [1]http://cateee.net/lkddb/web-lkddb/CMDLINE_BOOL.html >=20 > I have read through the Wiki page at [2]http://wiki.xensource.com/xenwi= ki/XenCommonProblems. I am running a 64-bit Xen Hypervisor. >=20 > Thus far I have been having no success with using the example entry fou= nd at [3]http://old.nabble.com/Strange-interaction-from-grub2-and-XEN-td264= 64067.html >=20 > A question regarding that entry is what would be the root device when t= here are separate /boot and root partitions? I am using the partition with = the /boot (grub2) as > my 'set root' device right? >=20 > I am also using the dummy=3Ddummy as the first option on the first modu= le line, again as discovered on the wiki. >=20 > Additionally posted on another thread here in the Xen mailing list was: >=20 > > On 1 March 2010 08:09, Luke Carrier wrote: > >> > >> Not sure about Debian, but I know Ubuntu isn*t ideal for Xen any more > >> because of the presence of GRUB2 * the new boot loader doesn*t support > >> passing certain command lines and the current multiboot implementation > >> breaks the method Xen uses to load the > Dom0 kernel * you can*t use Ubuntu > >> 9.10 Server with Xen. >=20 > I am confused now because Boris clearly created a wordpress.com weblog i= n where Ubuntu 9.10 Server was > used with XEN. >=20 > Lastly, out of curiosity, what (or what kind) of commands cannot be pass= ed with GRUB2? >=20 > ThX! >=20 >=20 > -M >=20 > -------------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > Date: Sat, 27 Feb 2010 11:20:31 -0800 > From: bderzhavets@yahoo.com > Subject: Re: [Xen-devel] RE: [Xen-users] How to: Improve Network Throug= hput of a XEN kernel? > To: mike.viau@sheridanc.on.ca > CC: xen-devel@lists.xensource.com; xen-users@lists.xensource.com >=20 > In Xen Wiki is mentioned , that parameters for kernel may be set via co= mmand line as follows:- > ***********************************************************************= ************************************* > module (hd1,10)/vmlinuz-2.6.31.8 dummy=3Ddummy root=3D/dev/sda7 ro co= nsole=3Dtty0 > ***********************************************************************= ************************************* > # cat /proc/cmdline >=20 > shows that GRUB2 skips first argument, what causes problem. >=20 > Compiling built-in CMD_LINE is just an option. > Read all Wiki written by Pasi and you would never be surprised ;) >=20 > > P.S: What is your feeling on using a recent Ubuntu system with > > grub-legacy as the boot loader? I noticed that Squeeze has adopted gr= ub2 > package as well too... >=20 > I have development Xen 3.4.3 instance on top Ubuntu 9.10 Server with GR= UB2 ( baremetall) loading any from 3 mentioned bellow kernels under Xen 3.4= =2E3 >=20 > 1.) 2.6.32.9 PVOPS > 2.) 2.6.31.6 PVOPS > 3.) 2.6.31.12 xenified >=20 > with no problems with dummy=3Ddummy as first parameter ( thanks Pasi). > It may also may be done via compiling kernels with built-in CMD_LINE. > It works fine as well. >=20 > Boris. >=20 > --- On Sat, 2/27/10, Mike Viau wrote: >=20 > From: Mike Viau > Subject: [Xen-devel] RE: [Xen-users] How to: Improve Network Throughp= ut of a XEN kernel? > To: bderzhavets@yahoo.com > Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com > Date: Saturday, February 27, 2010, 12:03 PM >=20 > Yes, thank you Boris for pointing that out. I found that was defiantl= y true. The 3.2.1 xen hypervisor could only boot the 2.6.26-2 xen kernel an= d the xenified > 2.6.31.12. >=20 > Additionally I found that: >=20 > Xend does not start when using pv_ops dom0 kernel? >=20 > In December 2009 pv_ops dom0 kernel modules were renamed to have a "x= en-" prefix in them, ie. "evtchn.ko" became "xen-evtchn.ko". >=20 > This makes Xen 3.4.x xend fail to start, because it tried to load "ev= tchn.ko", but that doesn't exist. You need to load "xen-evtchn.ko" and then= start xend. Fedora 12 > xen-3.4.2-2 rpms have this problem fixed. >=20 > Also make sure you have xenfs mounted to "/proc/xen", that's needed a= swell. >=20 > [Source: [4]http://wiki.xensource.com/xenwiki/XenParavirtOps] >=20 > And it appears the Debian xen-3.4.2-2 debs (in squeeze) have not fixe= d this problem. >=20 > Lastly I am still doing some reading on the grub2 package and how to = configure it. It turns out that I have been using grub-legacy (or just good= old grub) for so long > that configuring the grub2 boot loader is turning out to be my own pe= rsonal kryptonite. >=20 >=20 >=20 > I am still working on getting the hypervisor to load the dom0 kernel = in a Ubuntu system. >=20 >=20 >=20 > I have noticed an example in your wordpress @ [5]Set up Xen 3.4.3-rc2= & Libvirt 0.7.0 Dom0 (with 2.6.31.8 xenified aka Suse kernel) on top of Ub= untu 9.10 Server >=20 >=20 >=20 > With CONFIG_CMDLINE=3D"root=3D/dev/sdb11 ro console=3Dtty0" >=20 >=20 >=20 > menuentry "Xen 3.4 / Ubuntu 9.10 kernel 2.6.31.8 xenified" { > insmod ext2 > set root=3D(hd1,10) > multiboot (hd1,10)/xen-3.4.gz > module (hd1,10)/vmlinuz-2.6.31.8 > module (hd1,10)/initrd-2.6.31.8.img > } >=20 >=20 >=20 > I am sure you wouldn't of mentioned the CONFIG_CMDLINE kernel option = is it was not important for using with grub2 right? I have not have to comp= ile a kernel with the > command line previous as I was just able to supply the arguments on t= he kernel line of my menu.lst (grub configuration file). >=20 > P.S: What is your feeling on using a recent Ubuntu system with grub-l= egacy as the boot loader? I noticed that Squeeze has adopted grub2 package = as well too... >=20 >=20 > -M >=20 > -----------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > Date: Sat, 27 Feb 2010 04:32:59 -0800 > From: bderzhavets@yahoo.com > Subject: Re: FW: [Xen-users] How to: Improve Network Throughput of a = XEN kernel? > To: jbeulich@novell.com; mike.viau@sheridanc.on.ca > CC: xen-devel@lists.xensource.com; xen-users@lists.xensource.com >=20 > You cannot load pvops kernel under xen-hypervisor 3.2.1 ( >=3D3.4.0 = in general) >=20 > Boris. >=20 > --- On Fri, 2/26/10, Mike Viau wrote: >=20 > From: Mike Viau > Subject: FW: [Xen-users] How to: Improve Network Throughput of a XE= N kernel? > To: bderzhavets@yahoo.com, jbeulich@novell.com > Cc: xen-users@lists.xensource.com, xen-devel@lists.xensource.com > Date: Friday, February 26, 2010, 3:19 PM >=20 > Hello again, >=20 > Just a quick update... >=20 > I was really hoping it was just going to be an issue with the kerne= l .config, however I copied exact configuration from the working Debain 2.6= =2E26-2-xen-amd64 kernel > and used a make oldconfig but the end result was the same. >=20 > After booting the xenified (2.6.31.12) or pvops kernel (2.6.31.6) v= ia both xen-hypervisor 3.2.1 or xen-hypervisor 3.4.2 the maximum data trans= fer to/from the Dom0 > (Xen host) is ~1MB/s. >=20 > Next I plan on re-trying my efforts on a Ubuntu system to see if sa= me network bottlenecking is present. >=20 > ---------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > Date: Fri, 26 Feb 2010 09:16:59 -0800 > From: bderzhavets@yahoo.com > Subject: Re: [Xen-users] How to: Improve Network Throughput of a XE= N kernel? > To: xen-users@lists.xensource.com; viaum@sheridanc.on.ca >=20 > Would escalate to xen-devel. 2.6.31.8 (12) aka Suse xenified was = Jan Beulich stuff. > He might have an answer. > Boris. >=20 > --- On Fri, 2/26/10, Mike Viau wrote: >=20 > From: Mike Viau > Subject: [Xen-users] How to: Improve Network Throughput of a XEN = kernel? > To: xen-users@lists.xensource.com > Cc: waldi@debian.org > Date: Friday, February 26, 2010, 12:09 PM >=20 > Hi there, >=20 > I am starting a new thread to try and solve a problem I am having= will slow network throughput to the XEN host (not to a domU). >=20 > I have confirmed my problem occurs with PVOPS 'xen/master' kernel= as well using > [6]http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.6= -pvops-dom0-xen-master-x86_64 as the .config. >=20 > It also occurs with a forward ported xenified kernel (2.6.31.12) = using the attached config-11-xen (.config). >=20 > However when using the xen kernel found in debain lenny (2.6.26-2= ) for the amd64 architecture I get similiar transfer rates to my Xen host a= s I do on a baremetel > kernel. >=20 > Bastian Blank perhaps you could point me in the right direction i= n regards to a debian specific customization or patch I should be aware of? > Maxium transfer rate (to/from hard disk) using the PVOPS 'xen/mas= ter' kernel and forward ported xenified kernel (2.6.31.12) is bottlenecking= at ~1MB/s. On the > debain lenny xen kernel I am getting ~4.5MB/s. >=20 > I was hoping one could suggest options in the .config that might = be modified in order to improve network throughput of a XEN kernel and ther= efor increase the data > transfer rates over the netowork to and from the Xen host. >=20 > Thanks alot for any help! As always I will be willing to accept a= ny suggestions :) >=20 > -M >=20 > ---------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > Not using Hotmail on your phone? Why not? [7]Get it now. >=20 > -----------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > Live connected with Hotmail on your phone. [8]Learn more. > -----Inline Attachment Follows----- >=20 > _______________________________________________ > Xen-devel mailing list > [9]Xen-devel@lists.xensource.com > [10]http://lists.xensource.com/xen-devel >=20 > -------------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------- >=20 > All your Hotmail contacts on your phone. [11]Try it now. > -----Inline Attachment Follows----- >=20 > _______________________________________________ > Xen-users mailing list > [12]Xen-users@lists.xensource.com > [13]http://lists.xensource.com/xen-users >=20 > ----------------------------------------------------------------------= ---- >=20 > IM on the go with Messenger on your phone. [14]Try now. >=20 > References >=20 > Visible links > 1. http://cateee.net/lkddb/web-lkddb/CMDLINE_BOOL.html > 2. http://wiki.xensource.com/xenwiki/XenCommonProblems > 3. http://old.nabble.com/Strange-interaction-from-grub2-and-XEN-td2646= 4067.html > 4. http://wiki.xensource.com/xenwiki/XenParavirtOps > 5. Permanent Link: Set up Xen 3.4.3-rc2 & Libvirt 0.7.0 Dom0 (with 2.6= =2E31.8 xenified aka Suse kernel) on top of Ubuntu 9.10=01Server > http://bderzhavets.wordpress.com/2010/01/04/set-up-xen-3-4-3-rc1-libvirt= -0-7-0-dom0-with-2-6-31-8-xenified-aka-suse-kernel-on-top-of-ubuntu-9-10-se= rver/ > 6. http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.6-pvop= s-dom0-xen-master-x86_64 > 7. http://go.microsoft.com/?linkid=3D9708120 > 8. http://go.microsoft.com/?linkid=3D9708117 > 9. http://mc/compose?to=3DXen-devel@lists.xensource.com > 10. http://lists.xensource.com/xen-devel > 11. http://go.microsoft.com/?linkid=3D9708118 > 12. http://mc/compose?to=3DXen-users@lists.xensource.com > 13. http://lists.xensource.com/xen-users > 14. http://go.microsoft.com/?linkid=3D9712960 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel