kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

[Jon Watte]

I'm attempting to set up a Windows XP virtual machine using KVM
running on 8.04 LTS x64 or a Core 2 Duo (6550). I have updated all
installed packages to the latest available as of a week ago. This is
KVM version 0.9.1 (kvm-62) on a kernel version 2.6.24-22-server.

I'm creating a 20 GB hard disk image with qemu-img, (on a file system
that is managed by LVM, and in turn backed by a md0 soft mirror) and
then starting the VM as follows:

#!/bin/sh
cd /var/kvm/wxp_build
kvm -m 1024M \
-hda wxp_build.img \
-usb -usbdevice tablet \
-vnc :1 \
-localtime \
-net nic,model=virtio -net user \
-redir tcp:5555::5555 \
-daemonize \
$*

When I install, I pass "-cdrom ../cdrom/wxp_sp3_installer.iso -boot d"
to boot from the Windows install CD. The reboot as part of the install
does not take down the VM, and does not cause any problems.

I can go through install just fine. I format the 20 GB image as NTFS,
using slow format (not quick format).

I can download and install other things fine, too. However, when I
shut down the virtual machine (using the Windows Shut Down command),
and wait for the VNC connection to end, the (virtual) hard disk seems
to get corrupt. When I boot again, one of three things happen:

1) The boot fails, because some necessary file has gotten corrupted.

2) The boot succeeds, but when I attempt to run something that I
installed that worked fine before the shutdown, it doesn't work after
the shutdown, with missing DLL errors or similar.

3) The boot kicks into CHKDSK, which finds tons and tons of NTFS file
system corruption on the virtual hard disk image.

I've tried with both qcow2 and raw disk images, with the same results.

To my mind, it behaves as if the KVM process decides to exit as soon
as the BIOS power-off signal comes, but the process keeps the hard
disk write buffer in user-mode RAM, rather than immediately having
passed it on to the kernel. Thus, when the process exits, whatever was
buffered will not be committed to disk, and badness ensues. I'm not
sure at all that this is what actually happens, but that's what it
feels like. It could be some other kind of image corruption, too, but
it's clearly related to actually powering off the virtual machine.

Btw: the same corruption happens if I kill the KVM while it's running
(rather than do a "clean" shut-down). It's slightly more
understandable under that situation, but not really -- NTFS is a
journaling file system, and never has those corruption issues on my
"native" Windows machines, even during power fail.

So... does anyone have a clue what could be causing this, and what I
can do to fix it?



--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world.
Nevertheless, whether we get there willingly or not, we shall soon
have lower consumption rates, because our present rates are
unsustainable.

Re: kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

[Yaniv Kaul]

On 7/2/2009 6:50 PM, Jon Watte wrote:
> I'm attempting to set up a Windows XP virtual machine using KVM
> running on 8.04 LTS x64 or a Core 2 Duo (6550). I have updated all
> installed packages to the latest available as of a week ago. This is
> KVM version 0.9.1 (kvm-62) on a kernel version 2.6.24-22-server.
>    
I warmly suggest updating to a later, much newer, KVM version. 
Specifically qcow2 format had corruption bugs similar to what described 
below.
Y.

> I'm creating a 20 GB hard disk image with qemu-img, (on a file system
> that is managed by LVM, and in turn backed by a md0 soft mirror) and
> then starting the VM as follows:
>
> #!/bin/sh
> cd /var/kvm/wxp_build
> kvm -m 1024M \
> -hda wxp_build.img \
> -usb -usbdevice tablet \
> -vnc :1 \
> -localtime \
> -net nic,model=virtio -net user \
> -redir tcp:5555::5555 \
> -daemonize \
> $*
>
> When I install, I pass "-cdrom ../cdrom/wxp_sp3_installer.iso -boot d"
> to boot from the Windows install CD. The reboot as part of the install
> does not take down the VM, and does not cause any problems.
>
> I can go through install just fine. I format the 20 GB image as NTFS,
> using slow format (not quick format).
>
> I can download and install other things fine, too. However, when I
> shut down the virtual machine (using the Windows Shut Down command),
> and wait for the VNC connection to end, the (virtual) hard disk seems
> to get corrupt. When I boot again, one of three things happen:
>
> 1) The boot fails, because some necessary file has gotten corrupted.
>
> 2) The boot succeeds, but when I attempt to run something that I
> installed that worked fine before the shutdown, it doesn't work after
> the shutdown, with missing DLL errors or similar.
>
> 3) The boot kicks into CHKDSK, which finds tons and tons of NTFS file
> system corruption on the virtual hard disk image.
>
> I've tried with both qcow2 and raw disk images, with the same results.
>
> To my mind, it behaves as if the KVM process decides to exit as soon
> as the BIOS power-off signal comes, but the process keeps the hard
> disk write buffer in user-mode RAM, rather than immediately having
> passed it on to the kernel. Thus, when the process exits, whatever was
> buffered will not be committed to disk, and badness ensues. I'm not
> sure at all that this is what actually happens, but that's what it
> feels like. It could be some other kind of image corruption, too, but
> it's clearly related to actually powering off the virtual machine.
>
> Btw: the same corruption happens if I kill the KVM while it's running
> (rather than do a "clean" shut-down). It's slightly more
> understandable under that situation, but not really -- NTFS is a
> journaling file system, and never has those corruption issues on my
> "native" Windows machines, even during power fail.
>
> So... does anyone have a clue what could be causing this, and what I
> can do to fix it?
>
>
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world.
> Nevertheless, whether we get there willingly or not, we shall soon
> have lower consumption rates, because our present rates are
> unsustainable.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

Re: kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

[Dustin Kirkland]

On Thu, Jul 2, 2009 at 8:53 AM, Yaniv Kaul<ykaul@redhat.com> wrote:
> On 7/2/2009 6:50 PM, Jon Watte wrote:
>>
>> I'm attempting to set up a Windows XP virtual machine using KVM
>> running on 8.04 LTS x64 or a Core 2 Duo (6550). I have updated all
>> installed packages to the latest available as of a week ago. This is
>> KVM version 0.9.1 (kvm-62) on a kernel version 2.6.24-22-server.
>>
>
> I warmly suggest updating to a later, much newer, KVM version. Specifically
> qcow2 format had corruption bugs similar to what described below.

I second Yaniv's comment.

I am working on a backport of kvm-84 (kernel and userspace) to Ubuntu
8.04 LTS.  You can try the kvm packages in the following personal
package archive (PPA):
 * https://launchpad.net/~ubuntu-virt/+archive/ppa

Or, you can wait about a week and these packages should be uploaded to
the hardy-backports archive, and you can install from there, rather
than from the PPA.

Please install both kvm and kvm-source to update both user and kernel
space.  (Note that these packages contain cherry picks of the qcow2
corruption patches.)

:-Dustin

Re: kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

[Jon Watte]

Yaniv Kaul wrote:
> On 7/2/2009 6:50 PM, Jon Watte wrote:
>> I'm attempting to set up a Windows XP virtual machine using KVM
>> running on 8.04 LTS x64 or a Core 2 Duo (6550). I have updated all
>> installed packages to the latest available as of a week ago. This is
>> KVM version 0.9.1 (kvm-62) on a kernel version 2.6.24-22-server.
>>    
> I warmly suggest updating to a later, much newer, KVM version. 
> Specifically qcow2 format had corruption bugs similar to what 
> described below.
> Y.


Thanks for the suggestion!

Unfortunately, I'm using a "long term support" version of the OS to 
avoid having to screw with the machine all the time. It serves a real 
purpose with real uptime requirements :-)

That being said, I tried uninstalling the provided kvm package, and 
downloading and building kvm-87 from source (the kvm-87.tar.gz tarball 
on Sourceforge). Unfortunately, after running configure, and make, it 
errors out on the following sub-make:

root@svn:/usr/local/src/kvm-87# make
make -C /lib/modules/2.6.24-22-server/build M=`pwd` \
                LINUXINCLUDE="-I`pwd`/include -Iinclude \
                 -Iarch/x86/include -I`pwd`/include-compat \
                -include include/linux/autoconf.h \
                -include `pwd`/x86/external-module-compat.h " \
                "$@"
make: *** /lib/modules/2.6.24-22-server/build: No such file or 
directory.  Stop.
make[1]: *** [all] Error 2
make: *** [kvm-kmod] Error 2
root@svn:/usr/local/src/kvm-87#

There is no /lib/modules/2.6.24-22-server/build on my machine. Creating 
it doesn't help; it just complains that the sub-make has no target or 
file specified.

I looked in the build documentation on linux-kvm.org, but it does not 
mention anything like this, or any particular requirements. It just says 
"configure, then make," which is what I'm doing. I'm pasting the 
print-out from configure below, if anyone has the opportunity to take a 
look at it I'd appreciate it.

Sincerely,

jw



root@svn:/usr/local/src/kvm-87# ./configure

Error: libpci check failed
Disable KVM Device Assignment capability.

Install prefix    /usr/local
BIOS directory    /usr/local/share/qemu
binary directory  /usr/local/bin
Manual directory  /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /usr/local/src/kvm-87
C compiler        gcc
Host C compiler   gcc
ARCH_CFLAGS       -m64
make              make
install           install
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu
tcg debug enabled no
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
-Werror enabled   no
SDL support       no
curses support    no
curl support      no
mingw32 support   no
Audio drivers     oss
Extra audio cards ac97 es1370 sb16
Mixer emulation   no
VNC TLS support   no
VNC SASL support  no
kqemu support     no
xen support       no
CPU emulation     yes
brlapi support    no
Documentation     no
NPTL support      yes
vde support       no
AIO support       yes
IO thread         no
Install blobs     yes
-e KVM support       yes
KVM trace support no
fdt support       no
preadv support    no
root@svn:/usr/local/src/kvm-87#

Re: kvm on Ubuntu 8.04 running windows xp sp3: corrupts file system on shutdown?

[Jon Watte]

Dustin Kirkland wrote:
>
> I am working on a backport of kvm-84 (kernel and userspace) to Ubuntu
> 8.04 LTS.  You can try the kvm packages in the following personal
> package archive (PPA):
>  * https://launchpad.net/~ubuntu-virt/+archive/ppa
>
> Or, you can wait about a week and these packages should be uploaded to
> the hardy-backports archive, and you can install from there, rather
> than from the PPA.
>
> Please install both kvm and kvm-source to update both user and kernel
> space.  (Note that these packages contain cherry picks of the qcow2
> corruption patches.)
>   

That sounds great!

I'm wondering why I still got corruption with the RAW image format, 
though? I notice there is something called "IO thread" in config, which 
config by default turns off. Is that an asynchronizer of some sort? And, 
if so, is it on by deafult in 8.04?
I'm trying to suss out what the root cause might be. Also, I can't 
believe that I'm the only one with these problems? Although I hear on 
the interwebs that other people are complaining, too, and the usually 
get the reply "that's probably because you yank power" -- which I'm 
definitely not doing.

Sincerely,

jw



-- 

  Revenge is the most pointless and damaging of human desires.