From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steven Haigh Subject: Re: 4.2.1: Poor write performance for DomU. Date: Sun, 24 Mar 2013 20:10:59 +1100 Message-ID: <514EC323.3040106@crc.id.au> References: <51243089.8020307@crc.id.au> <512488AC.1020702@citrix.com> <51248E23.7060408@crc.id.au> <51249C11.3050800@crc.id.au> <5139A747.80703@crc.id.au> <20130308204916.GB22146@phenom.dumpdata.com> <513A669E.4060906@crc.id.au> <20130311133017.GJ23018@phenom.dumpdata.com> <513DDE07.2090607@crc.id.au> <20130312130430.GA17901@phenom.dumpdata.com> <514EA337.7030303@crc.id.au> <514EA6B0.8010504@crc.id.au> <514EA741.7050403@crc.id.au> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <514EA741.7050403@crc.id.au> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: konrad.wilk@oracle.com Cc: xen-devel@lists.xen.org, roger.pau@citrix.com List-Id: xen-devel@lists.xenproject.org On 24/03/13 18:12, Steven Haigh wrote: > In fact, I just thought of something else.... I have an eSATA caddy that > connects to the same SATA controller. With this, I can slot any SATA > drive into it - and I should easily be able to pass this to any DomU. > > I'll throw in a 1Tb SATA drive do that I don't have to break the > existing RAID6 array - as the testing on this drive can be destructive > testing - as otherwise the drive is blank. Disk info: Model Family: Seagate Barracuda 7200.12 Device Model: ST31000528AS Serial Number: 9VP3BE9W LU WWN Device Id: 5 000c50 01a238fd0 Firmware Version: CC49 User Capacity: 1,000,203,804,160 bytes [1.00 TB] Sector Size: 512 bytes logical/physical Results... Dom0 (host machine): # dd if=/dev/zero of=/dev/sdi bs=1M count=4096 oflag=direct 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 33.6909 s, 127 MB/s Created an ext4 filesystem on /dev/sdi1... # mkfs.ext4 -j /dev/sdi1 Run bonnie++ on the filesystem: # mount /dev/sdi1 /mnt/esata # cd /mnt/esata/ # bonnie++ -u 0:0 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP xenhost.lan.crc. 2G 433 95 119107 22 36723 7 960 95 145026 12 191.9 4 Latency 33231us 39824us 211ms 31466us 17459us 5073ms Version 1.96 ------Sequential Create------ --------Random Create-------- xenhost.lan.crc.id. -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 285us 642us 315us 217us 349us 127us We get ~145Mb/sec block read, ~119Mb/sec block write. Now, lets pass the whole device through to a DomU. # xm block-attach zeus.vm phy:/dev/sdi xvdc w From the DomU now: Firstly, the same dd as above: # dd if=/dev/zero of=/dev/xvdc bs=1M count=4096 oflag=direct 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB) copied, 33.6708 s, 128 MB/s Create the ext4 filesystem again: # mkfs.ext4 -j /dev/xvdc1 Run bonnie++ on the DomU: Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP zeus.crc.id.au 2G 387 99 121891 24 47759 14 992 98 141103 17 248.9 7 Latency 40518us 126ms 152ms 47174us 30061us 250ms Version 1.96 ------Sequential Create------ --------Random Create-------- zeus.crc.id.au -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 174us 839us 2249us 113us 42us 185us Interesting. We're at almost full speed in the DomU. 121Mb/sec write, 141Mb/sec read. So my wonder is now... Why when put in a RAID6 do we have a 180Mb/sec+ from the Dom0, but only 50Mb/sec from the DomU of the same filesystem... Any further testing that may indicate something? -- Steven Haigh Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299