Re: [Qemu-devel] Windows slow boot: contractor wanted

[Robert Vineyard <vineyard@tuffmail.com>]

Richard,

Not sure if you've tried this, but I noticed massive performance gains 
(easily booting 2-3 times as fast) by converting from RAW disk images to 
direct-mapped raw partitions and making sure that IOMMU support was 
enabled in the BIOS and in the kernel at boot time. The obvious downside 
to using raw partitions is a loss of flexibility and portability across 
physical machines, but in some cases the trade-offs may be worth it.

I never ran any formal benchmarks, but it "felt" like about a 50% 
performance boost going from RAW disk images to raw partitions (don't 
even think about using QCOW2 disk images for Windows, your VM's will 
still be booting next week...). The real gains, which I can't yet fully 
explain, came from passing "iommu=on intel_iommu=on" to the host kernel 
on bootup. I believe the boot option to enable IOMMU support may be 
different on AMD hardware.

Granted, this is on a much smaller VM than you're using (Windows 7 x64 
with two vCPUs and 4gb of vRAM), but might be worth investigating.

Good luck!

-- Robert Vineyard


On 08/17/2012 08:36 AM, Richard Davies wrote:
> Hi Avi,
>
> Thanks to you and several others for offering help. We will work with Avi at
> first, but are grateful for all the other offers of help. We have a number
> of other qemu-related projects which we'd be interested in getting done, and
> will get in touch with these names (and anyone else who comes forward) to
> see if any are of interest to you.
>
>
> This slow boot problem is intermittent and varys in how slow the boots are,
> but I managed to trigger it this morning with medium slow booting (5-10
> minutes) and link to the requested traces below.
>
> The host in question has 128GB RAM and dual AMD Opteron 6128 (16 cores
> total). It is running kernel 3.5.1 and qemu-kvm 1.1.1.
>
> In this morning's test, we have 3 guests, all booting Windows with 40GB RAM
> and 8 cores each (we have seen small VMs go slow as I originally said, but
> it is easier to trigger with big VMs):
>
> pid 15665: qemu-kvm -nodefaults -m 40960 -smp 8 -cpu host,hv_relaxed \
>    -vga cirrus -usbdevice tablet -vnc :99 -monitor stdio -hda test1.raw
> pid 15676: qemu-kvm -nodefaults -m 40960 -smp 8 -cpu host,hv_relaxed \
>    -vga cirrus -usbdevice tablet -vnc :98 -monitor stdio -hda test2.raw
> pid 15653: qemu-kvm -nodefaults -m 40960 -smp 8 -cpu host,hv_relaxed \
>    -vga cirrus -usbdevice tablet -vnc :97 -monitor stdio -hda test3.raw
>
> We are running with hv_relaxed since this was suggested in the previous
> thread, but we see intermittent slow boots with and without this flag.
>
>
> All 3 VMs are booting slowly for most of the attached capture, which I
> started after confirming the slow boots and stopped as soon as the first of
> them (15665) had booted. In terms of visible symptoms, the VMs are showing
> the Windows boot progress bar, which is moving very slowly. In top, the VMs
> are at 400% CPU and their resident state size (RES) memory is slowly
> counting up until it reaches the full VM size, at which point they finish
> booting.
>
>
> Here are the trace files:
>
> http://users.org.uk/slow-win-boot-1/ps.txt (ps auxwwwf as root)
> http://users.org.uk/slow-win-boot-1/top.txt (top with 2 VMs still slow)
> http://users.org.uk/slow-win-boot-1/trace-console.txt (running trace-cmd)
> http://users.org.uk/slow-win-boot-1/trace.dat (the 1.7G trace data file)
> http://users.org.uk/slow-win-boot-1/trace-report.txt (the 4G trace report)
>
>
> Please let me know if there is anything else which I can provide?
>
> Thank you,
>
> Richard.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>