Re: 4.2.1: Poor write performance for DomU.

[Steven Haigh <netwiz@crc.id.au>]

On 24/03/13 18:12, Steven Haigh wrote:
> In fact, I just thought of something else.... I have an eSATA caddy that
> connects to the same SATA controller. With this, I can slot any SATA
> drive into it - and I should easily be able to pass this to any DomU.
>
> I'll throw in a 1Tb SATA drive do that I don't have to break the
> existing RAID6 array - as the testing on this drive can be destructive
> testing - as otherwise the drive is blank.

Disk info:
Model Family:     Seagate Barracuda 7200.12
Device Model:     ST31000528AS
Serial Number:    9VP3BE9W
LU WWN Device Id: 5 000c50 01a238fd0
Firmware Version: CC49
User Capacity:    1,000,203,804,160 bytes [1.00 TB]
Sector Size:      512 bytes logical/physical

Results...

Dom0 (host machine):
# dd if=/dev/zero of=/dev/sdi bs=1M count=4096 oflag=direct
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB) copied, 33.6909 s, 127 MB/s

Created an ext4 filesystem on /dev/sdi1...
# mkfs.ext4 -j /dev/sdi1

Run bonnie++ on the filesystem:
# mount /dev/sdi1 /mnt/esata
# cd /mnt/esata/
# bonnie++ -u 0:0
Version  1.96       ------Sequential Output------ --Sequential Input- 
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
xenhost.lan.crc. 2G   433  95 119107  22 36723   7   960  95 145026  12 
191.9   4
Latency             33231us   39824us     211ms   31466us   17459us 
5073ms
Version  1.96       ------Sequential Create------ --------Random 
Create--------
xenhost.lan.crc.id. -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ 
+++++ +++
Latency               285us     642us     315us     217us     349us 
127us

We get ~145Mb/sec block read, ~119Mb/sec block write.

Now, lets pass the whole device through to a DomU.
# xm block-attach zeus.vm phy:/dev/sdi xvdc w

 From the DomU now:
Firstly, the same dd as above:
# dd if=/dev/zero of=/dev/xvdc bs=1M count=4096 oflag=direct
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB) copied, 33.6708 s, 128 MB/s

Create the ext4 filesystem again:
# mkfs.ext4 -j /dev/xvdc1

Run bonnie++ on the DomU:
Version  1.96       ------Sequential Output------ --Sequential Input- 
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
zeus.crc.id.au   2G   387  99 121891  24 47759  14   992  98 141103  17 
248.9   7
Latency             40518us     126ms     152ms   47174us   30061us 
250ms
Version  1.96       ------Sequential Create------ --------Random 
Create--------
zeus.crc.id.au      -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ 
+++++ +++
Latency               174us     839us    2249us     113us      42us 
185us

Interesting. We're at almost full speed in the DomU. 121Mb/sec write, 
141Mb/sec read.

So my wonder is now... Why when put in a RAID6 do we have a 180Mb/sec+ 
from the Dom0, but only 50Mb/sec from the DomU of the same filesystem...

Any further testing that may indicate something?

-- 
Steven Haigh

Email: netwiz@crc.id.au
Web: https://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
Fax: (03) 8338 0299