[Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Sven Oehme]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]

i thought it might be good to post this on qemu-devel as well, as i see 
this as a general qemu / kvm / xen /whatevercomesnext issue .

are there any plans to implement a default in qemu to prevent accessing 
the same image multiple times ? 
i know that in xen they deal with this problem somehow xen specific.
wouldn't it be good to prevent multiple access to the same image by 
default and add a switch to overwrite it ? 

i am sure other people had the same issue as i had (corrupting images) and 
would be great to prevent people in the future messing around with data 
loss :-)

Sven




Sven Oehme/Germany/IBM@IBMDE 
Sent by: et-mgmt-tools-bounces@redhat.com
08/27/2007 09:38 PM
Please respond to
Fedora/Linux Management Tools <et-mgmt-tools@redhat.com>


To
Fedora/Linux Management Tools <et-mgmt-tools@redhat.com>
cc

Subject
[et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm







Hi, 

i corrupted a couple of virtual Linux images last week as i accidentally 
started them two times .. 
i opened a bug against qemu (not sure if this was a good idea) -->  
https://bugzilla.redhat.com/show_bug.cgi?id=253533 

are there any plans to ensure that this can't happen in future releases of 
virt-manager with lock files in the image directory or similar ? 

thanks. Sven 
_______________________________________________
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/et-mgmt-tools

[-- Attachment #2: Type: text/html, Size: 3076 bytes --]

[Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Daniel P. Berrange]

On Mon, Aug 27, 2007 at 10:19:42PM +0200, Sven Oehme wrote:
> i thought it might be good to post this on qemu-devel as well, as i see 
> this as a general qemu / kvm / xen /whatevercomesnext issue .
>
> are there any plans to implement a default in qemu to prevent accessing 
> the same image multiple times ? 

Each QEMU instance is unware of each other so can't directly check this. One
possiblity would be for QEMU to use fcntl() to take a lock on a disk image
when opening it. With raw disk images at least, it is, however, safe to let
multiple QEMU instances use it at once *provided* you have a clustered
filesystem installed, rather than regular FAT or ext2/3, so one wouldn't want
to exclude this use case.

> i know that in xen they deal with this problem somehow xen specific.
> wouldn't it be good to prevent multiple access to the same image by 
> default and add a switch to overwrite it ? 

In the scenario you mention, libvirt should probably do a sanity check for
this before letting you start the guest. libvirt already supports the idea
of 'shared' disk images where two or more guests can be optionally configured 
to have write access - basically assumes the admin requesting sharing knows
what they're doing.

> i corrupted a couple of virtual Linux images last week as i accidentally 
> started them two times .. 
> i opened a bug against qemu (not sure if this was a good idea) -->  
> https://bugzilla.redhat.com/show_bug.cgi?id=253533 
> 
> are there any plans to ensure that this can't happen in future releases of 
> virt-manager with lock files in the image directory or similar ? 

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Avi Kivity]

Daniel P. Berrange wrote:
> On Mon, Aug 27, 2007 at 10:19:42PM +0200, Sven Oehme wrote:
>   
>> i thought it might be good to post this on qemu-devel as well, as i see 
>> this as a general qemu / kvm / xen /whatevercomesnext issue .
>>
>> are there any plans to implement a default in qemu to prevent accessing 
>> the same image multiple times ? 
>>     
>
> Each QEMU instance is unware of each other so can't directly check this. One
> possiblity would be for QEMU to use fcntl() to take a lock on a disk image
> when opening it. With raw disk images at least, it is, however, safe to let
> multiple QEMU instances use it at once *provided* you have a clustered
> filesystem installed, rather than regular FAT or ext2/3, so one wouldn't want
> to exclude this use case.
>   

Yes, taking an advisory lock is a good idea, provided there is an option 
to override it.

I've killed a few images myself...

-- 
Any sufficiently difficult bug is indistinguishable from a feature.

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Anthony Liguori]

On Mon, 2007-08-27 at 21:27 +0100, Daniel P. Berrange wrote:
> On Mon, Aug 27, 2007 at 10:19:42PM +0200, Sven Oehme wrote:
> > i thought it might be good to post this on qemu-devel as well, as i see 
> > this as a general qemu / kvm / xen /whatevercomesnext issue .
> >
> > are there any plans to implement a default in qemu to prevent accessing 
> > the same image multiple times ? 
> 
> Each QEMU instance is unware of each other so can't directly check this. One
> possiblity would be for QEMU to use fcntl() to take a lock on a disk image
> when opening it. With raw disk images at least, it is, however, safe to let
> multiple QEMU instances use it at once *provided* you have a clustered
> filesystem installed, rather than regular FAT or ext2/3, so one wouldn't want
> to exclude this use case.
> 
> > i know that in xen they deal with this problem somehow xen specific.
> > wouldn't it be good to prevent multiple access to the same image by 
> > default and add a switch to overwrite it ? 
> 
> In the scenario you mention, libvirt should probably do a sanity check for
> this before letting you start the guest. libvirt already supports the idea
> of 'shared' disk images where two or more guests can be optionally configured 
> to have write access - basically assumes the admin requesting sharing knows
> what they're doing.

I think this is the right level myself.  Advisory locks work okay but
not all filesystems support them.  It's particularly nasty when you have
a clustered filesystem in the host.  I think it would do more harm than
good to have a feature like that was supposed to provide a safe-guard
but then frequently didn't work.

Regards,

Anthony Liguori

> > i corrupted a couple of virtual Linux images last week as i accidentally 
> > started them two times .. 
> > i opened a bug against qemu (not sure if this was a good idea) -->  
> > https://bugzilla.redhat.com/show_bug.cgi?id=253533 
> > 
> > are there any plans to ensure that this can't happen in future releases of 
> > virt-manager with lock files in the image directory or similar ? 
> 
> Dan.

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Daniel Veillard]

On Mon, Aug 27, 2007 at 04:02:38PM -0500, Anthony Liguori wrote:
> 
> On Mon, 2007-08-27 at 21:27 +0100, Daniel P. Berrange wrote:
> > On Mon, Aug 27, 2007 at 10:19:42PM +0200, Sven Oehme wrote:
> > > i know that in xen they deal with this problem somehow xen specific.
> > > wouldn't it be good to prevent multiple access to the same image by 
> > > default and add a switch to overwrite it ? 
> > 
> > In the scenario you mention, libvirt should probably do a sanity check for
> > this before letting you start the guest. libvirt already supports the idea
> > of 'shared' disk images where two or more guests can be optionally configured 
> > to have write access - basically assumes the admin requesting sharing knows
> > what they're doing.
> 
> I think this is the right level myself.  Advisory locks work okay but
> not all filesystems support them.  It's particularly nasty when you have
> a clustered filesystem in the host.  I think it would do more harm than
> good to have a feature like that was supposed to provide a safe-guard
> but then frequently didn't work.

  Well initially I was really of the position that checking had to be
done at the hypervisor level, but as this thread pointed out we can't
always do this (there is also the potential of conflict between different
hypervisor though that sounds more theorical than practical).

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Avi Kivity]

Anthony Liguori wrote:
>> In the scenario you mention, libvirt should probably do a sanity check for
>> this before letting you start the guest. libvirt already supports the idea
>> of 'shared' disk images where two or more guests can be optionally configured 
>> to have write access - basically assumes the admin requesting sharing knows
>> what they're doing.
>>     
>
> I think this is the right level myself.  Advisory locks work okay but
> not all filesystems support them.  It's particularly nasty when you have
> a clustered filesystem in the host.  I think it would do more harm than
> good to have a feature like that was supposed to provide a safe-guard
> but then frequently didn't work.
>   

There's still the unmanaged use case to worry about.  I think qemu can 
default to advisory locking, and management tools can do their own 
locking and always override qemu.

It's too easy to kill an image by starting up another instance right now.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Sven Oehme]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

qemu-devel-bounces+oehmes=de.ibm.com@nongnu.org wrote on 08/28/2007 
04:13:02 AM:

> Anthony Liguori wrote:
> >> In the scenario you mention, libvirt should probably do a sanity 
check for
> >> this before letting you start the guest. libvirt already supports the 
idea
> >> of 'shared' disk images where two or more guests can be 
> optionally configured 
> >> to have write access - basically assumes the admin requesting sharing 
knows
> >> what they're doing.
> >> 
> >
> > I think this is the right level myself.  Advisory locks work okay but
> > not all filesystems support them.  It's particularly nasty when you 
have
> > a clustered filesystem in the host.  I think it would do more harm 
than
> > good to have a feature like that was supposed to provide a safe-guard
> > but then frequently didn't work.
> > 
> 
> There's still the unmanaged use case to worry about.  I think qemu can 
> default to advisory locking, and management tools can do their own 
> locking and always override qemu.
> 
> It's too easy to kill an image by starting up another instance right 
now.
> 

i agree default should be advisory locking and a switch to disable it ..
would that be hard to implement ?

thanks. Sven

[-- Attachment #2: Type: text/html, Size: 1596 bytes --]

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Paul Jakma]

On Mon, 27 Aug 2007, Anthony Liguori wrote:

> I think this is the right level myself.  Advisory locks work okay but
> not all filesystems support them.  It's particularly nasty when you have
> a clustered filesystem in the host.  I think it would do more harm than
> good to have a feature like that was supposed to provide a safe-guard
> but then frequently didn't work.
                ^^^^^^^^^^^^^^^^^^^^^^

Are you trying to say that any kind of significant portion of QEMU 
users have clustered file-systems?

I think that's unlikely, and it'd be nice if QEMU by default did a 
fcntl() on writeable image files to lock itself from multiple access, 
that'd benefit the vast majority of users.

Let the 0.x% of users who need to run with weird/esoteric fses cope..

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
Fashions have done more harm than revolutions.
     		-- Victor Hugo

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Alexander Graf]

Paul Jakma wrote:
> On Mon, 27 Aug 2007, Anthony Liguori wrote:
>
>> I think this is the right level myself.  Advisory locks work okay but
>> not all filesystems support them.  It's particularly nasty when you have
>> a clustered filesystem in the host.  I think it would do more harm than
>> good to have a feature like that was supposed to provide a safe-guard
>> but then frequently didn't work.
>                ^^^^^^^^^^^^^^^^^^^^^^
>
> Are you trying to say that any kind of significant portion of QEMU
> users have clustered file-systems?
>
> I think that's unlikely, and it'd be nice if QEMU by default did a
> fcntl() on writeable image files to lock itself from multiple access,
> that'd benefit the vast majority of users.
>
> Let the 0.x% of users who need to run with weird/esoteric fses cope..
>
> regards,
I'm usually running one "main" qemu instance that has read/write access
to the disk file and several others for tests that use the -snapshot
option, so I think it's very important to have an easy means of
switching this check off.

just my 2 cents,

Alexander Graf

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Paul Jakma]

On Wed, 29 Aug 2007, Alexander Graf wrote:

> Paul Jakma wrote:

>> I think that's unlikely, and it'd be nice if QEMU by default did a
>> fcntl() on writeable image files to lock itself from multiple access,
            ^^^^^^^^^^^^^^^^^^^^^^^^

> I'm usually running one "main" qemu instance that has read/write 
> access to the disk file and several others for tests that use the 
> -snapshot option, so I think it's very important to have an easy 
> means of switching this check off.

Of course. Though, I said 'writeable' - so you might not need to use 
such an option.. ;)

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
   This report is filled with omissions.

Re: [Qemu-devel] Re: [et-mgmt-tools] Image Corruption Possible with qemu and qemu-kvm

[Chris Wilson]

Hi all,

On Wed, 29 Aug 2007, Alexander Graf wrote:
> Paul Jakma wrote:
>> On Mon, 27 Aug 2007, Anthony Liguori wrote:
>>
>>> I think this is the right level myself.  Advisory locks work okay but 
>>> not all filesystems support them... I think it would do more harm than 
>>> good to have a feature like that was supposed to provide a safe-guard 
>>> but then frequently didn't work.
>>                ^^^^^^^^^^^^^^^^^^^^^^
>>
>> Are you trying to say that any kind of significant portion of QEMU 
>> users have clustered file-systems?
>>
>> I think that's unlikely, and it'd be nice if QEMU by default did a 
>> fcntl() on writeable image files to lock itself from multiple access, 
>> that'd benefit the vast majority of users.
>
> I'm usually running one "main" qemu instance that has read/write access 
> to the disk file and several others for tests that use the -snapshot 
> option, so I think it's very important to have an easy means of 
> switching this check off.

Would it make sense to take a read lock (preventing concurrent writes) on 
read-only snapshot images, and a write lock on all writable files? That 
should allow one to run as many instances as you want off a single 
snapshot in snapshot mode, but still prevent one from (accidentally) 
running a non-snapshot QEMU on the same image, or from (accidentally) 
running two instances writing to the same COW file.

By removing the need to add the lock-override command line switch to qemu 
in this case, it would reduce the risk of accidental image corruption by 
accidentally overriding the lock.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |