strange xen disk performance

[Jia Rao <rickenrao@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 1761 bytes --]

Hi all,

I observed some strange disk performance under Xen environment.

*The test bed:*
Dom0: 2.6.18.8, one cpu, pinned to a physical core, CFQ disk scheduler.
Xen: 3.3.1
Guest OS: 2.6.18.8, virtual disks are using physical partitions:
phy:/dev/sdbX. one virtual cpu, each pinned to a separate core.
Intel two quad-core xeon CPU

I did some tests on disk I/O performance when two VMs are sharing the
physical host.

both of the VMs are running iozone benchmark, sequential read a 1GB file
with a request size 32K. The read was issued as O_DIRECT IO skipping the
domU's buffer cache.

To get a reference throughput, I issued the read from each VM individually.
It turned out to be a throughput of 56MB/s-61MB/s, which is the limit of the
system for 32K sequential reads.

Then I issued the read from two VMs simultaneously. It turned out to be a
22MB/s-24MB/s for each VM. Combined together is around 45MB/s for the whole
system, which is pretty normal.

The most strange thing was that, in order to make sure that the above
results come from pure disk performance, nothing to to with buffer cache, I
did the same test again but purge the cached data in domU and dom0 before
the test.

sync and echo 3 > /proc/sys/vm/drop_caches were executed both in domU and
dom0. The resulted throughput of each VM increased to 56MB/s-61MB/s, which
generates a system throughput around 120MB/s. I ran the test for 10 times, 9
out of 10 have this strange result. Only one test looks like the original
test.

It seems impossible to me and it must has something to do with caches.

My question is, do Xen or dom0 cache any disk I/O data from the guest OS? It
seems dom0's buffer cache has nothing to do with this because I already
purged everything,

Any ideas?

Thanks a lot.

[-- Attachment #1.2: Type: text/html, Size: 2152 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel