[Qemu-devel] Re: [RFC] Disk integrity in QEMU

[Ryan Harper <ryanh@us.ibm.com>]

* Anthony Liguori <anthony@codemonkey.ws> [2008-10-09 12:00]:
> Read performance should be unaffected by using O_DSYNC.  O_DIRECT will 
> significantly reduce read performance.  I think we should use O_DSYNC by 
> default and I have sent out a patch that contains that.  We will follow 
> up with benchmarks to demonstrate this.


 baremetal baseline (1g dataset):
 ---------------------------+-------+-------+--------------+------------+
 Test scenarios             | bandw | % CPU | ave submit   | ave compl  |
 type, block size, iface    | MB/s  | usage | latency usec | latency ms |
 ---------------------------+-------+-------+--------------+------------+
 write, 16k, lvm, direct=1  | 127.7 |  12   |   11.66      |    9.48    |
 write, 64k, lvm, direct=1  | 178.4 |   5   |   13.65      |   27.15    |
 write, 1M,  lvm, direct=1  | 186.0 |   3   |  163.75      |  416.91    |
 ---------------------------+-------+-------+--------------+------------+
 read , 16k, lvm, direct=1  | 170.4 |  15   |   10.86      |    7.10    |
 read , 64k, lvm, direct=1  | 199.2 |   5   |   12.52      |   24.31    |
 read , 1M,  lvm, direct=1  | 202.0 |   3   |  133.74      |  382.67    |
 ---------------------------+-------+-------+--------------+------------+

 kvm write (1g dataset):
 ---------------------------+-------+-------+--------------+------------+
 Test scenarios             | bandw | % CPU | ave submit   | ave compl  |
 block size,iface,cache,sync| MB/s  | usage | latency usec | latency ms |
 ---------------------------+-------+-------+--------------+------------+
 16k,virtio,off,none        | 135.0 |  94   |    9.1       |    8.71    |
 16k,virtio,on ,none        | 184.0 | 100   |   63.69      |   63.48    |
 16k,virtio,on ,O_DSYNC     | 150.0 |  35   |    6.63      |    8.31    |
 ---------------------------+-------+-------+--------------+------------+
 64k,virtio,off,none        | 169.0 |  51   |   17.10      |   28.00    |
 64k,virtio,on ,none        | 189.0 |  60   |   69.42      |   24.92    |
 64k,virtio,on ,O_DSYNC     | 171.0 |  48   |   18.83      |   27.72    |
 ---------------------------+-------+-------+--------------+------------+
 1M ,virtio,off,none        | 142.0 |  30   |  7176.00     |  523.00    |
 1M ,virtio,on ,none        | 190.0 |  45   |  5332.63     |  392.35    |
 1M ,virtio,on ,O_DSYNC     | 164.0 |  39   |  6444.48     |  471.20    |
 ---------------------------+-------+-------+--------------+------------+

 kvm read (1g dataset):
 ---------------------------+-------+-------+--------------+------------+
 Test scenarios             | bandw | % CPU | ave submit   | ave compl  |
 block size,iface,cache,sync| MB/s  | usage | latency usec | latency ms |
 ---------------------------+-------+-------+--------------+------------+
 16k,virtio,off,none        | 175.0 |  40   |   22.42      |    6.71    |
 16k,virtio,on ,none        | 211.0 | 147   |   59.49      |    5.54    |
 16k,virtio,on ,O_DSYNC     | 212.0 | 145   |   60.45      |    5.47    |
 ---------------------------+-------+-------+--------------+------------+
 64k,virtio,off,none        | 190.0 |  64   |   16.31      |   24.92    |
 64k,virtio,on ,none        | 546.0 | 161   |  111.06      |    8.54    |
 64k,virtio,on ,O_DSYNC     | 520.0 | 151   |  116.66      |    8.97    |
 ---------------------------+-------+-------+--------------+------------+
 1M ,virtio,off,none        | 182.0 |  32   | 5573.44      |  407.21    |
 1M ,virtio,on ,none        | 750.0 | 127   | 1344.65      |   96.42    |
 1M ,virtio,on ,O_DSYNC     | 768.0 | 123   | 1289.05      |   94.25    |
 ---------------------------+-------+-------+--------------+------------+

 --------------------------------------------------------------------------
 exporting file in ext3 filesystem as block device (1g)
 --------------------------------------------------------------------------

 kvm write (1g dataset):
 ---------------------------+-------+-------+--------------+------------+
 Test scenarios             | bandw | % CPU | ave submit   | ave compl  |
 block size,iface,cache,sync| MB/s  | usage | latency usec | latency ms |
 ---------------------------+-------+-------+--------------+------------+
 16k,virtio,off,none        |  12.1 |  15   |    9.1       |    8.71    |
 16k,virtio,on ,none        | 192.0 |  52   |   62.52      |    6.17    |
 16k,virtio,on ,O_DSYNC     | 142.0 |  59   |   18.81      |    8.29    |
 ---------------------------+-------+-------+--------------+------------+
 64k,virtio,off,none        |  15.5 |   8   |   21.10      |  311.00    |
 64k,virtio,on ,none        | 454.0 | 130   |  113.25      |   10.65    |
 64k,virtio,on ,O_DSYNC     | 154.0 |  48   |   20.25      |   30.75    |
 ---------------------------+-------+-------+--------------+------------+
 1M ,virtio,off,none        |  24.7 |   5   | 41736.22     | 3020.08    |
 1M ,virtio,on ,none        | 485.0 | 100   |  2052.09     |  149.81    |
 1M ,virtio,on ,O_DSYNC     | 161.0 |  42   |  6268.84     |  453.84    |
 ---------------------------+-------+-------+--------------+------------+


--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com