* Qemu disk images on BTRFS suffer checksum errors @ 2013-07-08 8:08 Thomas Kuther 2013-07-08 13:20 ` Josef Bacik 0 siblings, 1 reply; 3+ messages in thread From: Thomas Kuther @ 2013-07-08 8:08 UTC (permalink / raw) To: linux-btrfs Hello, I'm about to migrate from VirtualBox to Qemu+VGA-Passthrough. All my virtual disk images are stored in a BTRFS subvolume on-top of a MDRAID 1. The host runs kernel 3.10, and Qemu 1.5.1. The Testing-VM is a Windows 7 64bit, using a RAW virtio disk with cache=none, same happens for qcow2, though. Using VirtualBox and in the past Vmware workstation I never had issues with corrupted diskimages, but now with Qemu all tries ended up with lots of errors like: [ 4871.863009] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 3817758510 private 402306600 [ 4872.481013] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 3817758510 private 402306600 [ 4904.055514] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 4060166193 private 402306600 [ 4904.748130] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 4060166193 private 402306600 [ 4904.987540] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 3817758510 private 402306600 [ 4905.024700] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 3817758510 private 402306600 [ 4932.497793] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 4060166193 private 402306600 [ 4932.533634] BTRFS info (device md10): csum failed ino 687 off 46213922816 csum 4060166193 private 402306600 Trying to copy the disk image elsewhere causes I/O errors at some point. I found a thread about the issue (http://comments.gmane.org/gmane.comp.file-systems.btrfs/20538) and also a bug report against Qemu from Josef Bacik describing the exact same problem: https://bugzilla.redhat.com/show_bug.cgi?id=693530 - Josef states it should be fixed since quite a while. Is this a regression in BTRFS, a problem with my setup (md raid1 layer below btrfs), or (still) a bug in Qemu? Would cache=writethrough or writeback be an option with BTRFS? Thanks in advance for any input. Best regards, Thomas ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Qemu disk images on BTRFS suffer checksum errors 2013-07-08 8:08 Qemu disk images on BTRFS suffer checksum errors Thomas Kuther @ 2013-07-08 13:20 ` Josef Bacik 2013-07-08 18:45 ` Thomas Kuther 0 siblings, 1 reply; 3+ messages in thread From: Josef Bacik @ 2013-07-08 13:20 UTC (permalink / raw) To: Thomas Kuther; +Cc: linux-btrfs On Mon, Jul 08, 2013 at 10:08:46AM +0200, Thomas Kuther wrote: > Hello, > > I'm about to migrate from VirtualBox to Qemu+VGA-Passthrough. All my virtual > disk images are stored in a BTRFS subvolume on-top of a MDRAID 1. > The host runs kernel 3.10, and Qemu 1.5.1. The Testing-VM is a Windows 7 > 64bit, using a RAW virtio disk with cache=none, same happens for qcow2, > though. > > Using VirtualBox and in the past Vmware workstation I never had issues with > corrupted diskimages, but now with Qemu all tries ended up with lots of > errors like: > > [ 4871.863009] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 3817758510 private 402306600 > [ 4872.481013] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 3817758510 private 402306600 > [ 4904.055514] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 4060166193 private 402306600 > [ 4904.748130] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 4060166193 private 402306600 > [ 4904.987540] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 3817758510 private 402306600 > [ 4905.024700] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 3817758510 private 402306600 > [ 4932.497793] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 4060166193 private 402306600 > [ 4932.533634] BTRFS info (device md10): csum failed ino 687 off 46213922816 > csum 4060166193 private 402306600 > > Trying to copy the disk image elsewhere causes I/O errors at some point. > > I found a thread about the issue > (http://comments.gmane.org/gmane.comp.file-systems.btrfs/20538) and also a > bug report against Qemu from Josef Bacik describing the exact same problem: > https://bugzilla.redhat.com/show_bug.cgi?id=693530 - Josef states it should > be fixed since quite a while. > > Is this a regression in BTRFS, a problem with my setup (md raid1 layer below > btrfs), or (still) a bug in Qemu? > Would cache=writethrough or writeback be an option with BTRFS? > So there were two aspects to that bug, one is the thing I describe where we get the same buffer for two parts of an iovec on reads. That part has been fixed. The second part is where the application will modify the page while it's in flight, and that hasn't been fixed. We have a few options here 1) Always double buffer direct io. Kind of defeats the purpose of direct io. 2) Check the buffer after we've written it to see if it matches the csum we put down, if not double buffer it and send it down again. This makes you checksum the page twice and punishes O_DIRECT users that behave. I opted for #3 and let this sort of thing happen. So you can get around it by doing nodatacow for that particular image which will disable checksumming for just that file, or you can use cache=writethrough/writeback and that will use buffered io. FYI this doesn't happen on _all_ qemu, just on guest OS'es that don't provide stable pages, so Windows or like old RHEL versions that are on ext3. Thanks, Josef ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Qemu disk images on BTRFS suffer checksum errors 2013-07-08 13:20 ` Josef Bacik @ 2013-07-08 18:45 ` Thomas Kuther 0 siblings, 0 replies; 3+ messages in thread From: Thomas Kuther @ 2013-07-08 18:45 UTC (permalink / raw) To: linux-btrfs Am 08.07.2013 15:20, schrieb Josef Bacik: > On Mon, Jul 08, 2013 at 10:08:46AM +0200, Thomas Kuther wrote: >> Hello, >> >> I'm about to migrate from VirtualBox to Qemu+VGA-Passthrough. All my virtual >> disk images are stored in a BTRFS subvolume on-top of a MDRAID 1. >> The host runs kernel 3.10, and Qemu 1.5.1. The Testing-VM is a Windows 7 >> 64bit, using a RAW virtio disk with cache=none, same happens for qcow2, >> though. >> >> Using VirtualBox and in the past Vmware workstation I never had issues with >> corrupted diskimages, but now with Qemu all tries ended up with lots of >> errors like: >> >> [ 4871.863009] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 3817758510 private 402306600 >> [ 4872.481013] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 3817758510 private 402306600 >> [ 4904.055514] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 4060166193 private 402306600 >> [ 4904.748130] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 4060166193 private 402306600 >> [ 4904.987540] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 3817758510 private 402306600 >> [ 4905.024700] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 3817758510 private 402306600 >> [ 4932.497793] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 4060166193 private 402306600 >> [ 4932.533634] BTRFS info (device md10): csum failed ino 687 off 46213922816 >> csum 4060166193 private 402306600 >> >> Trying to copy the disk image elsewhere causes I/O errors at some point. >> >> I found a thread about the issue >> (http://comments.gmane.org/gmane.comp.file-systems.btrfs/20538) and also a >> bug report against Qemu from Josef Bacik describing the exact same problem: >> https://bugzilla.redhat.com/show_bug.cgi?id=693530 - Josef states it should >> be fixed since quite a while. >> >> Is this a regression in BTRFS, a problem with my setup (md raid1 layer below >> btrfs), or (still) a bug in Qemu? >> Would cache=writethrough or writeback be an option with BTRFS? >> > > So there were two aspects to that bug, one is the thing I describe where we get > the same buffer for two parts of an iovec on reads. That part has been fixed. > The second part is where the application will modify the page while it's in > flight, and that hasn't been fixed. We have a few options here > > 1) Always double buffer direct io. Kind of defeats the purpose of direct io. > > 2) Check the buffer after we've written it to see if it matches the csum we put > down, if not double buffer it and send it down again. This makes you checksum > the page twice and punishes O_DIRECT users that behave. > > I opted for #3 and let this sort of thing happen. So you can get around it by > doing nodatacow for that particular image which will disable checksumming for > just that file, or you can use cache=writethrough/writeback and that will use > buffered io. FYI this doesn't happen on _all_ qemu, just on guest OS'es that > don't provide stable pages, so Windows or like old RHEL versions that are on > ext3. Thanks, > > Josef > Thanks very much for the explanation, Josef. I opted for 3), too. Used chattr +C on the directory that is meant for holding the qemu image(s), and re-created the RAW image in there (so it has nodatacow flag set now) So far, no issues. Perfect. Thanks again. ~Tom ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2013-07-08 18:45 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-07-08 8:08 Qemu disk images on BTRFS suffer checksum errors Thomas Kuther 2013-07-08 13:20 ` Josef Bacik 2013-07-08 18:45 ` Thomas Kuther
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).