Re: [Qemu-devel] [sheepdog] [PATCH] sheepdog: fix overlapping metadata update

[Vasiliy Tolstov <v.tolstov@selfip.ru>]

2015-07-31 15:08 GMT+03:00 Vasiliy Tolstov <v.tolstov@selfip.ru>:
> Please wait to performance comparison. As i see Liu's patch may be
> more slow then Hitoshi.


I'm switch to local cluster driver to test only local ssd and not
network overhead.

But now Liu's qemu does not able to completely install vps:
sheep runs as:
/usr/sbin/sheep --log level=debug format=server dir=/var/log/
dst=default --bindaddr 0.0.0.0 --cluster local --directio --upgrade
--myaddr 192.168.240.134 --pidfile /var/run/sheepdog.pid
/var/lib/sheepdog/sd_meta
/var/lib/sheepdog/vg0-sd_data,/var/lib/sheepdog/vg1-sd_data,/var/lib/sheepdog/vg2-sd_data,/var/lib/sheepdog/vg3-sd_data,/var/lib/sheepdog/vg4-sd_data,/var/lib/sheepdog/vg5-sd_data

qemu runs as:
qemu-system-x86_64 -enable-kvm -name 30681 -S -machine
pc-i440fx-1.7,accel=kvm,usb=off -m 1024 -realtime
mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid
49771611-5e45-70c9-1a78-00000043bf91 -no-user-config -nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/30681.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-scsi-pci,id=scsi0,num_queues=1,bus=pci.0,addr=0x4 -device
virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
file=sheepdog:30681,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,discard=ignore,aio=native
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
-drive if=none,id=drive-scsi0-0-1-0,readonly=on,format=raw -device
scsi-cd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0
-netdev tap,fd=37,id=hostnet0,vhost=on,vhostfd=39 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:42:8d,bus=pci.0,addr=0x3,rombar=0
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/30681.agent,server,nowait
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
-device usb-mouse,id=input0 -device usb-kbd,id=input1 -vnc
[::]:0,password -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object
rng-random,id=objrng0,filename=/dev/random -device
virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=1024,period=2000,bus=pci.0,addr=0x7
-msg timestamp=on

sheep.log:
https://gist.github.com/raw/31201454b0afe1b86d00

dmesg:
[Sun Aug  2 14:48:45 2015] qemu-system-x86[6894]: segfault at 401364
ip 00007f212f060ff8 sp 00007f213efe5f40 error 4 in
qemu-system-x86_64[7f212ecbc000+518000]

fio job file:
[randrw]
blocksize=4k
filename=/dev/sdb
rw=randrw
direct=1
buffered=0
ioengine=libaio
iodepth=32
group_reporting
numjobs=10
runtime=40

For clean test i'm create new vdi and attach it to running vm: qemu
cache=none, io=native, discard=ignore
Hitoshi patch:
randrw: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
fio-2.1.11
Starting 10 processes
Jobs: 10 (f=10): [m(10)] [100.0% done] [13414KB/12291KB/0KB /s]
[3353/3072/0 iops] [eta 00m:00s]
randrw: (groupid=0, jobs=10): err= 0: pid=1318: Sun Aug  2 11:51:25 2015
  read : io=368004KB, bw=9191.9KB/s, iops=2297, runt= 40036msec
    slat (usec): min=1, max=408908, avg=1316.35, stdev=10057.52
    clat (usec): min=229, max=972302, avg=67203.98, stdev=65126.08
     lat (usec): min=294, max=972306, avg=68520.61, stdev=66741.02
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[   16], 10.00th=[   30], 20.00th=[   36],
     | 30.00th=[   40], 40.00th=[   44], 50.00th=[   48], 60.00th=[   52],
     | 70.00th=[   59], 80.00th=[   85], 90.00th=[  128], 95.00th=[  186],
     | 99.00th=[  367], 99.50th=[  441], 99.90th=[  570], 99.95th=[  619],
     | 99.99th=[  799]
    bw (KB  /s): min=    6, max= 1824, per=10.19%, avg=936.59, stdev=434.25
  write: io=366964KB, bw=9165.9KB/s, iops=2291, runt= 40036msec
    slat (usec): min=1, max=338238, avg=1320.61, stdev=9586.81
    clat (usec): min=912, max=972334, avg=69411.58, stdev=64715.54
     lat (usec): min=920, max=972338, avg=70732.51, stdev=66193.96
    clat percentiles (msec):
     |  1.00th=[    8],  5.00th=[   30], 10.00th=[   33], 20.00th=[   38],
     | 30.00th=[   42], 40.00th=[   45], 50.00th=[   49], 60.00th=[   53],
     | 70.00th=[   60], 80.00th=[   86], 90.00th=[  128], 95.00th=[  184],
     | 99.00th=[  379], 99.50th=[  441], 99.90th=[  570], 99.95th=[  635],
     | 99.99th=[  824]
    bw (KB  /s): min=    7, max= 1771, per=10.21%, avg=935.65, stdev=439.55
    lat (usec) : 250=0.01%, 500=0.02%, 750=0.05%, 1000=0.04%
    lat (msec) : 2=0.20%, 4=0.57%, 10=1.53%, 20=1.78%, 50=50.50%
    lat (msec) : 100=28.04%, 250=14.56%, 500=2.47%, 750=0.23%, 1000=0.03%
  cpu          : usr=0.17%, sys=0.26%, ctx=16905, majf=0, minf=78
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.8%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued    : total=r=92001/w=91741/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=368004KB, aggrb=9191KB/s, minb=9191KB/s, maxb=9191KB/s,
mint=40036msec, maxt=40036msec
  WRITE: io=366964KB, aggrb=9165KB/s, minb=9165KB/s, maxb=9165KB/s,
mint=40036msec, maxt=40036msec

Disk stats (read/write):
  sdb: ios=91739/91385, merge=55/0, ticks=2854936/3040936,
in_queue=5901740, util=99.82%


-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru