* Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory [not found] ` <20120430180755.30027.70465.malone@soybean.canonical.com> @ 2012-05-01 8:03 ` Gleb Natapov 2012-05-01 11:04 ` Alexandre DERUMIER 0 siblings, 1 reply; 6+ messages in thread From: Gleb Natapov @ 2012-05-01 8:03 UTC (permalink / raw) To: anthony; +Cc: matthewa, qemu-devel On Mon, Apr 30, 2012 at 06:07:55PM -0000, Anthony Liguori wrote: > This should be resolved by using Hyper-V relaxed timers which is in the > latest development version of QEMU. You would need to add -cpu > host,+hv_relaxed to the command line to verify this. > The described scenario still shouldn't happen and it doesn't for me. Is this 32 or 64 bit version of Windows. Are you running 32 bit version of the host may be (but Scientific Linux shouldn't have that)? What is your command line? > ** Changed in: qemu > Status: New => Fix Committed > > -- > You received this bug notification because you are a member of qemu- > devel-ml, which is subscribed to QEMU. > https://bugs.launchpad.net/bugs/992067 > > Title: > Windows 2008R2 very slow cold boot when >4GB memory > > Status in QEMU: > Fix Committed > > Bug description: > I've been having a consistent problem booting 2008R2 guests with > 4096MB of RAM or greater. On the initial boot the KVM process starts > out with a ~200MB memory allocation and will use 100% of all CPU > allocated to it. The RES memory of the KVM process slowly rises by > around 200mb every few minutes until it reaches it's memory allocation > (several hours in some cases). Whilst this is happening the guest will > usually blue screen with the message of - > > A clock interrupt was not received on a secondary processor within the > allocated time interval > > If I let the KVM process continue to run it will eventually allocate > the required memory the guest will run at full speed, usually > restarting after the blue screen and booting into startup repair. From > here you can restart it and it will boot perfectly. Once booted the > guest has no performance issues at all. > > I've tried everything I could think of. Removing PAE, playing with > huge pages, different kernels, different userspaces, different > systems, different backing file systems, different processor feature > set, with or without Virtio etc. My best theory is that the problem is > caused by Windows 2008 zeroing out all the memory on boot and > something is causing this to be held up or slowed to a crawl. The > hosts always have memory free to boot the guest and are not using swap > at all. > > Nothing so far has solved the issue. A few observations I've made about the issue are - > Large memory 2008R2 guests seem to boot fine (or with a small delay) when they are the first to boot on the host after a reboot > Sometimes dropping the disk cache (echo 1 > /proc/sys/vm/drop_caches) will cause them to boot faster > > > The hosts I've tried are - > All Nehalem based (5540, 5620 and 5660) > Host ram of 48GB, 96GB and 192GB > Storage on NFS, Gluster and local (ext4, xfs and zfs) > QED, QCOW and RAW formats > Scientific Linux 6.1 with the standard kernel 2.6.32, 2.6.38 and 3.3.1 > KVM userspaces 0.12, 0.14 and (currently) 0.15.1 > > To manage notifications about this bug go to: > https://bugs.launchpad.net/qemu/+bug/992067/+subscriptions -- Gleb. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory 2012-05-01 8:03 ` [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory Gleb Natapov @ 2012-05-01 11:04 ` Alexandre DERUMIER 2012-05-07 18:47 ` Matthew Anderson 0 siblings, 1 reply; 6+ messages in thread From: Alexandre DERUMIER @ 2012-05-01 11:04 UTC (permalink / raw) To: Gleb Natapov; +Cc: matthewa, qemu-devel, anthony Hi, I was having this problem with qemu 0.15 , (win2008R2 x64 > 4GB). guest with 32gb ram take around 40min to boot. (When windows fill memory at boot, memory grow slowly) but now with last qemu kvm git (proxmox 2.0 distrib), it's working fine. (can tell for qemu 1.0) ----- Mail original ----- De: "Gleb Natapov" <gleb@redhat.com> À: anthony@codemonkey.ws Cc: matthewa@base3.com.au, qemu-devel@nongnu.org Envoyé: Mardi 1 Mai 2012 10:03:44 Objet: Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory On Mon, Apr 30, 2012 at 06:07:55PM -0000, Anthony Liguori wrote: > This should be resolved by using Hyper-V relaxed timers which is in the > latest development version of QEMU. You would need to add -cpu > host,+hv_relaxed to the command line to verify this. > The described scenario still shouldn't happen and it doesn't for me. Is this 32 or 64 bit version of Windows. Are you running 32 bit version of the host may be (but Scientific Linux shouldn't have that)? What is your command line? > ** Changed in: qemu > Status: New => Fix Committed > > -- > You received this bug notification because you are a member of qemu- > devel-ml, which is subscribed to QEMU. > https://bugs.launchpad.net/bugs/992067 > > Title: > Windows 2008R2 very slow cold boot when >4GB memory > > Status in QEMU: > Fix Committed > > Bug description: > I've been having a consistent problem booting 2008R2 guests with > 4096MB of RAM or greater. On the initial boot the KVM process starts > out with a ~200MB memory allocation and will use 100% of all CPU > allocated to it. The RES memory of the KVM process slowly rises by > around 200mb every few minutes until it reaches it's memory allocation > (several hours in some cases). Whilst this is happening the guest will > usually blue screen with the message of - > > A clock interrupt was not received on a secondary processor within the > allocated time interval > > If I let the KVM process continue to run it will eventually allocate > the required memory the guest will run at full speed, usually > restarting after the blue screen and booting into startup repair. From > here you can restart it and it will boot perfectly. Once booted the > guest has no performance issues at all. > > I've tried everything I could think of. Removing PAE, playing with > huge pages, different kernels, different userspaces, different > systems, different backing file systems, different processor feature > set, with or without Virtio etc. My best theory is that the problem is > caused by Windows 2008 zeroing out all the memory on boot and > something is causing this to be held up or slowed to a crawl. The > hosts always have memory free to boot the guest and are not using swap > at all. > > Nothing so far has solved the issue. A few observations I've made about the issue are - > Large memory 2008R2 guests seem to boot fine (or with a small delay) when they are the first to boot on the host after a reboot > Sometimes dropping the disk cache (echo 1 > /proc/sys/vm/drop_caches) will cause them to boot faster > > > The hosts I've tried are - > All Nehalem based (5540, 5620 and 5660) > Host ram of 48GB, 96GB and 192GB > Storage on NFS, Gluster and local (ext4, xfs and zfs) > QED, QCOW and RAW formats > Scientific Linux 6.1 with the standard kernel 2.6.32, 2.6.38 and 3.3.1 > KVM userspaces 0.12, 0.14 and (currently) 0.15.1 > > To manage notifications about this bug go to: > https://bugs.launchpad.net/qemu/+bug/992067/+subscriptions -- Gleb. -- -- Alexandre D erumier Ingénieur Système Fixe : 03 20 68 88 90 Fax : 03 20 68 90 81 45 Bvd du Général Leclerc 59100 Roubaix - France 12 rue Marivaux 75002 Paris - France ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory 2012-05-01 11:04 ` Alexandre DERUMIER @ 2012-05-07 18:47 ` Matthew Anderson 2012-07-23 11:07 ` Owen Tuz 0 siblings, 1 reply; 6+ messages in thread From: Matthew Anderson @ 2012-05-07 18:47 UTC (permalink / raw) To: 'Alexandre DERUMIER', Gleb Natapov Cc: qemu-devel@nongnu.org, anthony@codemonkey.ws Apologies for the late feedback, I've tested with the 1.0 source from sourceforge and also from the Proxmox git, both appeared to behave the same. The proxmox version appeared to be slightly better in the sense that the VM never blue screened on boot and it eventually booted whilst the sourceforge version consistently blue screened. I've recently noticed VM's on another host, with 2GB allocated to the VM, are slow to boot under the 0.15.1 userspace. I don't think the problem is strictly related to the VM having more than 4GB of memory but rather the problem is more easily spotted. The memory fill slowness appears to be linked to the number of VM's currently running on the host with anything over 5 concurrent VM's causing enough of a slow down that the large memory VM's fail to boot. The XML for one of the guests is below. The disabled HPET is due to Windows time keeping problems with it enabled. If I can do anything else to help diagnose the problem please let me know. I should have a free host with 48GB memory to test on shortly which may help further isolate the problem. Thanks -Matt <domain type='kvm'> <name>-Mosaic-TS</name> <uuid>57c86460-28a5-0ab3-314d-b428e19d2c54</uuid> <memory>8388608</memory> <currentMemory>8388608</currentMemory> <vcpu>4</vcpu> <os> <type arch='x86_64' machine='pc-0.14'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qed' cache='writeback'/> <source file='/mnt/ssd/iHost/iHost-Mosaic-TS.qed'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='block' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='hdc' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='1' unit='0'/> </disk> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:52:78:67'/> <source bridge='br0'/> <model type='e1000'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes'/> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </memballoon> </devices> </domain> -----Original Message----- From: Alexandre DERUMIER [mailto:aderumier@odiso.com] Sent: Tuesday, 1 May 2012 7:05 PM To: Gleb Natapov Cc: Matthew Anderson; qemu-devel@nongnu.org; anthony@codemonkey.ws Subject: Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory Hi, I was having this problem with qemu 0.15 , (win2008R2 x64 > 4GB). guest with 32gb ram take around 40min to boot. (When windows fill memory at boot, memory grow slowly) but now with last qemu kvm git (proxmox 2.0 distrib), it's working fine. (can tell for qemu 1.0) ----- Mail original ----- De: "Gleb Natapov" <gleb@redhat.com> À: anthony@codemonkey.ws Cc: matthewa@base3.com.au, qemu-devel@nongnu.org Envoyé: Mardi 1 Mai 2012 10:03:44 Objet: Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory On Mon, Apr 30, 2012 at 06:07:55PM -0000, Anthony Liguori wrote: > This should be resolved by using Hyper-V relaxed timers which is in > the latest development version of QEMU. You would need to add -cpu > host,+hv_relaxed to the command line to verify this. > The described scenario still shouldn't happen and it doesn't for me. Is this 32 or 64 bit version of Windows. Are you running 32 bit version of the host may be (but Scientific Linux shouldn't have that)? What is your command line? > ** Changed in: qemu > Status: New => Fix Committed > > -- > You received this bug notification because you are a member of qemu- > devel-ml, which is subscribed to QEMU. > https://bugs.launchpad.net/bugs/992067 > > Title: > Windows 2008R2 very slow cold boot when >4GB memory > > Status in QEMU: > Fix Committed > > Bug description: > I've been having a consistent problem booting 2008R2 guests with > 4096MB of RAM or greater. On the initial boot the KVM process starts > out with a ~200MB memory allocation and will use 100% of all CPU > allocated to it. The RES memory of the KVM process slowly rises by > around 200mb every few minutes until it reaches it's memory allocation > (several hours in some cases). Whilst this is happening the guest will > usually blue screen with the message of - > > A clock interrupt was not received on a secondary processor within the > allocated time interval > > If I let the KVM process continue to run it will eventually allocate > the required memory the guest will run at full speed, usually > restarting after the blue screen and booting into startup repair. From > here you can restart it and it will boot perfectly. Once booted the > guest has no performance issues at all. > > I've tried everything I could think of. Removing PAE, playing with > huge pages, different kernels, different userspaces, different > systems, different backing file systems, different processor feature > set, with or without Virtio etc. My best theory is that the problem is > caused by Windows 2008 zeroing out all the memory on boot and > something is causing this to be held up or slowed to a crawl. The > hosts always have memory free to boot the guest and are not using swap > at all. > > Nothing so far has solved the issue. A few observations I've made > about the issue are - Large memory 2008R2 guests seem to boot fine (or > with a small delay) when they are the first to boot on the host after > a reboot Sometimes dropping the disk cache (echo 1 > > /proc/sys/vm/drop_caches) will cause them to boot faster > > > The hosts I've tried are - > All Nehalem based (5540, 5620 and 5660) Host ram of 48GB, 96GB and > 192GB Storage on NFS, Gluster and local (ext4, xfs and zfs) QED, QCOW > and RAW formats Scientific Linux 6.1 with the standard kernel 2.6.32, > 2.6.38 and 3.3.1 KVM userspaces 0.12, 0.14 and (currently) 0.15.1 > > To manage notifications about this bug go to: > https://bugs.launchpad.net/qemu/+bug/992067/+subscriptions -- Gleb. -- -- Alexandre D erumier Ingénieur Système Fixe : 03 20 68 88 90 Fax : 03 20 68 90 81 45 Bvd du Général Leclerc 59100 Roubaix - France 12 rue Marivaux 75002 Paris - France ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory 2012-05-07 18:47 ` Matthew Anderson @ 2012-07-23 11:07 ` Owen Tuz 0 siblings, 0 replies; 6+ messages in thread From: Owen Tuz @ 2012-07-23 11:07 UTC (permalink / raw) To: Matthew Anderson Cc: anthony@codemonkey.ws, Gleb Natapov, 'Alexandre DERUMIER', qemu-devel@nongnu.org We have been experiencing this problem for a while now too, using qemu-kvm (currently at 1.1.1). Unfortunately, hv_relaxed doesn't seem to fix it. The following command line produces the issue: qemu-kvm -nodefaults -m 4096 -smp 8 -cpu host,hv_relaxed -vga cirrus -usbdevice tablet -vnc :99 -monitor stdio -hda test.img The hardware consists of dual AMD Opteron 6128 processors (16 cores in total) and 64GB of memory. This command line was tested on kernel 3.1.4. I've also tested with -no-hpet. What I have seen is much as described: the memory fills out slowly, and top on the host will show the process using 100% on all allocated CPU cores. The most extreme case was a machine which took something between 6 and 8 hours to boot. This seems to be related to the assigned memory, as described, but also the number of processor cores (which makes sense if we believe it's a timing issue?). I have seen slow-booting guests improved by switching down to a single or even two cores. Matthew, I agree that this seems to be linked to the number of VMs running - in fact, shutting down other VMs on a dedicated test host caused the machine to start booting at a normal speed (with no reboot required). However, the level of contention is never such that this could be explained by the host simply being overcommitted. If it helps anyone, there's an image of the hard drive I've been using to test at: http://46.20.114.253/ It's 5G of gzip file containing a fairly standard Windows 2008 trial installation. Since it's in the trial period, anyone who wants to use it may have to re-arm the trial: http://support.microsoft.com/kb/948472 Please let me know if I can provide any more information, or test anything. Best wishes, Owen Tuz ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory [not found] <20120430165246.29629.23281.malonedeb@soybean.canonical.com> [not found] ` <20120430180755.30027.70465.malone@soybean.canonical.com> @ 2016-11-24 10:28 ` Thomas Huth 2017-07-21 14:33 ` Thomas Huth 2 siblings, 0 replies; 6+ messages in thread From: Thomas Huth @ 2016-11-24 10:28 UTC (permalink / raw) To: qemu-devel Triaging old bug tickets... QEMU 0.12/0.14/0.15 is pretty outdated nowadays. Can you still reproduce this behavior with the latest version of QEMU? If not, I think we should close this bug... -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/992067 Title: Windows 2008R2 very slow cold boot when >4GB memory Status in QEMU: Fix Committed Bug description: I've been having a consistent problem booting 2008R2 guests with 4096MB of RAM or greater. On the initial boot the KVM process starts out with a ~200MB memory allocation and will use 100% of all CPU allocated to it. The RES memory of the KVM process slowly rises by around 200mb every few minutes until it reaches it's memory allocation (several hours in some cases). Whilst this is happening the guest will usually blue screen with the message of - A clock interrupt was not received on a secondary processor within the allocated time interval If I let the KVM process continue to run it will eventually allocate the required memory the guest will run at full speed, usually restarting after the blue screen and booting into startup repair. From here you can restart it and it will boot perfectly. Once booted the guest has no performance issues at all. I've tried everything I could think of. Removing PAE, playing with huge pages, different kernels, different userspaces, different systems, different backing file systems, different processor feature set, with or without Virtio etc. My best theory is that the problem is caused by Windows 2008 zeroing out all the memory on boot and something is causing this to be held up or slowed to a crawl. The hosts always have memory free to boot the guest and are not using swap at all. Nothing so far has solved the issue. A few observations I've made about the issue are - Large memory 2008R2 guests seem to boot fine (or with a small delay) when they are the first to boot on the host after a reboot Sometimes dropping the disk cache (echo 1 > /proc/sys/vm/drop_caches) will cause them to boot faster The hosts I've tried are - All Nehalem based (5540, 5620 and 5660) Host ram of 48GB, 96GB and 192GB Storage on NFS, Gluster and local (ext4, xfs and zfs) QED, QCOW and RAW formats Scientific Linux 6.1 with the standard kernel 2.6.32, 2.6.38 and 3.3.1 KVM userspaces 0.12, 0.14 and (currently) 0.15.1 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/992067/+subscriptions ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory [not found] <20120430165246.29629.23281.malonedeb@soybean.canonical.com> [not found] ` <20120430180755.30027.70465.malone@soybean.canonical.com> 2016-11-24 10:28 ` Thomas Huth @ 2017-07-21 14:33 ` Thomas Huth 2 siblings, 0 replies; 6+ messages in thread From: Thomas Huth @ 2017-07-21 14:33 UTC (permalink / raw) To: qemu-devel ** Changed in: qemu Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/992067 Title: Windows 2008R2 very slow cold boot when >4GB memory Status in QEMU: Fix Released Bug description: I've been having a consistent problem booting 2008R2 guests with 4096MB of RAM or greater. On the initial boot the KVM process starts out with a ~200MB memory allocation and will use 100% of all CPU allocated to it. The RES memory of the KVM process slowly rises by around 200mb every few minutes until it reaches it's memory allocation (several hours in some cases). Whilst this is happening the guest will usually blue screen with the message of - A clock interrupt was not received on a secondary processor within the allocated time interval If I let the KVM process continue to run it will eventually allocate the required memory the guest will run at full speed, usually restarting after the blue screen and booting into startup repair. From here you can restart it and it will boot perfectly. Once booted the guest has no performance issues at all. I've tried everything I could think of. Removing PAE, playing with huge pages, different kernels, different userspaces, different systems, different backing file systems, different processor feature set, with or without Virtio etc. My best theory is that the problem is caused by Windows 2008 zeroing out all the memory on boot and something is causing this to be held up or slowed to a crawl. The hosts always have memory free to boot the guest and are not using swap at all. Nothing so far has solved the issue. A few observations I've made about the issue are - Large memory 2008R2 guests seem to boot fine (or with a small delay) when they are the first to boot on the host after a reboot Sometimes dropping the disk cache (echo 1 > /proc/sys/vm/drop_caches) will cause them to boot faster The hosts I've tried are - All Nehalem based (5540, 5620 and 5660) Host ram of 48GB, 96GB and 192GB Storage on NFS, Gluster and local (ext4, xfs and zfs) QED, QCOW and RAW formats Scientific Linux 6.1 with the standard kernel 2.6.32, 2.6.38 and 3.3.1 KVM userspaces 0.12, 0.14 and (currently) 0.15.1 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/992067/+subscriptions ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-07-21 14:45 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20120430165246.29629.23281.malonedeb@soybean.canonical.com>
[not found] ` <20120430180755.30027.70465.malone@soybean.canonical.com>
2012-05-01 8:03 ` [Qemu-devel] [Bug 992067] Re: Windows 2008R2 very slow cold boot when >4GB memory Gleb Natapov
2012-05-01 11:04 ` Alexandre DERUMIER
2012-05-07 18:47 ` Matthew Anderson
2012-07-23 11:07 ` Owen Tuz
2016-11-24 10:28 ` Thomas Huth
2017-07-21 14:33 ` Thomas Huth
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).