opening files in exclusive mode?

opening files in exclusive mode?

[Michael Tokarev]

KVM (or qemu for that matter) does not care about
concurrent access to disk images it opens.  So it's
possible to mount the same guest block device on
more than one guest, resulting in a badly broken
filesystem.

I wonder if the  right way is to open all the
devices in exclusive mode (O_EXCL).  At least
for real block devices.  On linux this ensures
there's no other holders of that device, open()
errors out with -EBUSY if it's not the case.

With an option to stop adding such a flag, in
order to be able to use some cluster-aware
filesystem or read-only device.

Dunno how it applies to files (qcow etc), but
it *seems* here the situation is even worse,
because not only the filesystem (meta)data
might be damaged, but qcow structure too,
and there's no fsck.qcow program available
to repair it.

So I think kvm should ensure exclusive access
to the files it opens (except of cd-rom images
and the like), using either O_EXCL for block
devices or locking for qcow/raw files, and
have an option to stop that (e.g. exclusive=0).

Comments?

Thanks!

/mjt

Re: opening files in exclusive mode?

[Brian Jackson]

This has been discussed thoroughly on this list at least once in the  
past. It was decided against changing the default for various reasons.  
See the original thread for all the reasons, but the ones I remember  
are: it would kill backward compatibility for people that start  
multiple guests from the same source with -snapshot and for people who  
use qcow images with base images. It was also suggested that this  
might be a feature better suited for a mgmt application since it  
likely has a better idea of who/how it will be used. It could also  
disrupt live migration.

--Iggy

Sent from my iPhone

On Jan 24, 2009, at 9:03 AM, Michael Tokarev <mjt@tls.msk.ru> wrote:

> KVM (or qemu for that matter) does not care about
> concurrent access to disk images it opens.  So it's
> possible to mount the same guest block device on
> more than one guest, resulting in a badly broken
> filesystem.
>
> I wonder if the  right way is to open all the
> devices in exclusive mode (O_EXCL).  At least
> for real block devices.  On linux this ensures
> there's no other holders of that device, open()
> errors out with -EBUSY if it's not the case.
>
> With an option to stop adding such a flag, in
> order to be able to use some cluster-aware
> filesystem or read-only device.
>
> Dunno how it applies to files (qcow etc), but
> it *seems* here the situation is even worse,
> because not only the filesystem (meta)data
> might be damaged, but qcow structure too,
> and there's no fsck.qcow program available
> to repair it.
>
> So I think kvm should ensure exclusive access
> to the files it opens (except of cd-rom images
> and the like), using either O_EXCL for block
> devices or locking for qcow/raw files, and
> have an option to stop that (e.g. exclusive=0).
>
> Comments?
>
> Thanks!
>
> /mjt
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: opening files in exclusive mode?

[Michael Tokarev]

Brian Jackson wrote:

> This has been discussed thoroughly on this list at least once in the
> past. It was decided against changing the default for various reasons.

Well, I understand points about changing defaults.  I disagree,
because it really hurts to have no warnings whatsoever when, by
a chance, running off an already used block device (it's even
possible to run a guest off a block device mounted on host!),
but that's another topic (I'll take a look at the archives).

> See the original thread for all the reasons, but the ones I remember
> are: it would kill backward compatibility for people that start multiple
> guests from the same source with -snapshot and for people who use qcow
> images with base images.

Well, qcow with base image is easy: there are shared and exclusive
locks out there.  Base image gets locked in shared mode, while the
qcow itself is in exclusive.

>  It was also suggested that this might be a
> feature better suited for a mgmt application since it likely has a
> better idea of who/how it will be used.

There is, so far, no management application.  I for one either run
kvm manually from command line, or use a shell script with a single
command line in it (because the command line becomes quite long).
And it seems most people who experiment with kvm are either doing
the same or are trying libvirt and falling back to bare command line
when that does not work.  So that "management application" becomes
the human who runs the whole thing, and who needs this whole feature
in the first place... :)

>  It could also disrupt live
> migration.

That's probably true.  Again, with some workarounds, or disabling
locking.. I dunno for now.

But the question stands: how about the whole idea about PROVIDING
such an option, to start with?

Thanks!

/mjt