[Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Stefan Hajnoczi]

Amit and I were discussing the pros and cons of using O_EXCL to open
host CD-ROM devices on IRC but this discussion could benefit from more
input.

Linux block devices (like /dev/sr0 CD-ROMs) can be opened with O_EXCL
and only one userspace process will succeed at a time.  This prevents
programs from interfering with each other.  The polling daemons, hald
and udisks, use O_EXCL and mount does too.

Today QEMU does not use O_EXCL and will therefore access host CD-ROMs
while they are in use by other programs.  This also means that
programs can be started on the host while QEMU is already running that
may interfere with the virtual machine's ability to access the CD-ROM
(for example by ejecting it).

Therefore, it sounds reasonable to switch to O_EXCL to prevent
interfering with other programs and to prevent other programs
interfering with QEMU.

On the downside, it will no longer be possible to share a host CD-ROM
between multiple virtual machines or to mount it on host while passing
it through to a guest.  These scenarios are not safe because on of the
clients could eject the device, spoiling the party for everyone else.
However, it is a handy feature for putting installation media into a
machine and installing several guests at the same time.

The other concern I have about using O_EXCL is that we expose
ourselves to race conditions if there is ever a need to re-open the
device.  When QEMU closes its file descriptor another program may be
scheduled to run and open the device with O_EXCL.  Now QEMU will not
be able to open the CD-ROM anymore.  From the guest perspective this
could be at an odd time and we'd have to start failing requests.
Today we do not re-open host CD-ROMs though so this isn't a pressing
problem.

Any thoughts?  Additional pros/cons I've missed?

Stefan

[Qemu-devel] Re: To O_EXCL or not to O_EXCL open host_cdrom

[Amit Shah]

On (Fri) 08 Apr 2011 [12:33:27], Stefan Hajnoczi wrote:
> Amit and I were discussing the pros and cons of using O_EXCL to open
> host CD-ROM devices on IRC but this discussion could benefit from more
> input.
> 
> Linux block devices (like /dev/sr0 CD-ROMs) can be opened with O_EXCL
> and only one userspace process will succeed at a time.  This prevents
> programs from interfering with each other.  The polling daemons, hald
> and udisks, use O_EXCL and mount does too.
> 
> Today QEMU does not use O_EXCL and will therefore access host CD-ROMs
> while they are in use by other programs.  This also means that
> programs can be started on the host while QEMU is already running that
> may interfere with the virtual machine's ability to access the CD-ROM
> (for example by ejecting it).
> 
> Therefore, it sounds reasonable to switch to O_EXCL to prevent
> interfering with other programs and to prevent other programs
> interfering with QEMU.
> 
> On the downside, it will no longer be possible to share a host CD-ROM
> between multiple virtual machines or to mount it on host while passing
> it through to a guest.  These scenarios are not safe because on of the
> clients could eject the device, spoiling the party for everyone else.
> However, it is a handy feature for putting installation media into a
> machine and installing several guests at the same time.

I'm of the opinion that it's simply wrong to allow such concurrent
access.  The feature isn't too compelling, and it's really a bug IMO.
We should open O_EXCL and document somewhere about this.  Host CDROM
passthrough is such a niche concept that people should be able to
ensure to stop other services opening CDROMs in exclusive mode.

Also, since we're really cheating other programs that open the CDROM
device O_EXCL by bypassing that requirement, any actions the guest
takes is likely to hamper the host programs using CDROMs -- maybe even
causing guests to exploit security holes in other host programs.

> The other concern I have about using O_EXCL is that we expose
> ourselves to race conditions if there is ever a need to re-open the
> device.  When QEMU closes its file descriptor another program may be
> scheduled to run and open the device with O_EXCL.  Now QEMU will not
> be able to open the CD-ROM anymore.

The admins should really be the ones worrying about this, not QEMU.

		Amit

[Qemu-devel] Re: To O_EXCL or not to O_EXCL open host_cdrom

[Stefan Hajnoczi]

On Mon, Apr 11, 2011 at 10:37:32AM +0530, Amit Shah wrote:
> On (Fri) 08 Apr 2011 [12:33:27], Stefan Hajnoczi wrote:
> > The other concern I have about using O_EXCL is that we expose
> > ourselves to race conditions if there is ever a need to re-open the
> > device.  When QEMU closes its file descriptor another program may be
> > scheduled to run and open the device with O_EXCL.  Now QEMU will not
> > be able to open the CD-ROM anymore.
> 
> The admins should really be the ones worrying about this, not QEMU.

Think of a desktop use case.  virt-manager lets me pass through the host
CD-ROM today.  Desktops have hald/udisks and you can't expect users to
disable/reenable those services just for QEMU.

Stefan

Re: [Qemu-devel] Re: To O_EXCL or not to O_EXCL open host_cdrom

[Avi Kivity]

On 04/11/2011 11:31 AM, Stefan Hajnoczi wrote:
> On Mon, Apr 11, 2011 at 10:37:32AM +0530, Amit Shah wrote:
> >  On (Fri) 08 Apr 2011 [12:33:27], Stefan Hajnoczi wrote:
> >  >  The other concern I have about using O_EXCL is that we expose
> >  >  ourselves to race conditions if there is ever a need to re-open the
> >  >  device.  When QEMU closes its file descriptor another program may be
> >  >  scheduled to run and open the device with O_EXCL.  Now QEMU will not
> >  >  be able to open the CD-ROM anymore.
> >
> >  The admins should really be the ones worrying about this, not QEMU.
>
> Think of a desktop use case.  virt-manager lets me pass through the host
> CD-ROM today.  Desktops have hald/udisks and you can't expect users to
> disable/reenable those services just for QEMU.

It should be solved at that level then.  If I insert a disc into an 
assigned cd-rom drive, I shouldn't get a file manager or autorun window 
to pop in the host, just the guest.

So: libvirt should inform the rest of the system that it is taking over 
the cd-rom and as far as they're concerned, it no longer exists.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Avi Kivity]

On 04/08/2011 02:33 PM, Stefan Hajnoczi wrote:
> Amit and I were discussing the pros and cons of using O_EXCL to open
> host CD-ROM devices on IRC but this discussion could benefit from more
> input.
>
> Linux block devices (like /dev/sr0 CD-ROMs) can be opened with O_EXCL
> and only one userspace process will succeed at a time.  This prevents
> programs from interfering with each other.  The polling daemons, hald
> and udisks, use O_EXCL and mount does too.
>
> Today QEMU does not use O_EXCL and will therefore access host CD-ROMs
> while they are in use by other programs.  This also means that
> programs can be started on the host while QEMU is already running that
> may interfere with the virtual machine's ability to access the CD-ROM
> (for example by ejecting it).

Also, performance would likely be ruined if the disk wasn't cached 
(likely with a DVD).  CD-ROMs seek very slowly.  It would be even 
funnier if we supported cd-writers.

> Therefore, it sounds reasonable to switch to O_EXCL to prevent
> interfering with other programs and to prevent other programs
> interfering with QEMU.
>
> On the downside, it will no longer be possible to share a host CD-ROM
> between multiple virtual machines or to mount it on host while passing
> it through to a guest.  These scenarios are not safe because on of the
> clients could eject the device, spoiling the party for everyone else.
> However, it is a handy feature for putting installation media into a
> machine and installing several guests at the same time.

You'd probably benefit a lot more by copying the media to disk.

> The other concern I have about using O_EXCL is that we expose
> ourselves to race conditions if there is ever a need to re-open the
> device.  When QEMU closes its file descriptor another program may be
> scheduled to run and open the device with O_EXCL.  Now QEMU will not
> be able to open the CD-ROM anymore.  From the guest perspective this
> could be at an odd time and we'd have to start failing requests.
> Today we do not re-open host CD-ROMs though so this isn't a pressing
> problem.

We really should avoid re-open whenever possible.

The standard response to such questions is "let's add another option", 
but in this case I don't think we should.  Like Amit says, this is a 
niche use case.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Christoph Hellwig]

On Fri, Apr 08, 2011 at 12:33:27PM +0100, Stefan Hajnoczi wrote:
> Amit and I were discussing the pros and cons of using O_EXCL to open
> host CD-ROM devices on IRC but this discussion could benefit from more
> input.
> 
> Linux block devices (like /dev/sr0 CD-ROMs) can be opened with O_EXCL
> and only one userspace process will succeed at a time.  This prevents
> programs from interfering with each other.  The polling daemons, hald
> and udisks, use O_EXCL and mount does too.
> 
> Today QEMU does not use O_EXCL and will therefore access host CD-ROMs
> while they are in use by other programs.  This also means that
> programs can be started on the host while QEMU is already running that
> may interfere with the virtual machine's ability to access the CD-ROM
> (for example by ejecting it).
> 
> Therefore, it sounds reasonable to switch to O_EXCL to prevent
> interfering with other programs and to prevent other programs
> interfering with QEMU.

This all boils down to whether qemu should allow concurrent access
to image files of all sorts, and there are arguments both ways:

 pro: prevents corruption problems on disk images, notification issues
      on CDROMS, etc
 contra: makes clustering not work

which means we need it configurable.  I'd prefer to have the exclusion
on by default, but people caring for backwards comptibility might argue
the other way around.  Either way it needs to be consistent for both
devices and file images.

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Daniel P. Berrange]

On Fri, Apr 08, 2011 at 12:33:27PM +0100, Stefan Hajnoczi wrote:
> Amit and I were discussing the pros and cons of using O_EXCL to open
> host CD-ROM devices on IRC but this discussion could benefit from more
> input.
> 
> Linux block devices (like /dev/sr0 CD-ROMs) can be opened with O_EXCL
> and only one userspace process will succeed at a time.  This prevents
> programs from interfering with each other.  The polling daemons, hald
> and udisks, use O_EXCL and mount does too.
> 
> Today QEMU does not use O_EXCL and will therefore access host CD-ROMs
> while they are in use by other programs.  This also means that
> programs can be started on the host while QEMU is already running that
> may interfere with the virtual machine's ability to access the CD-ROM
> (for example by ejecting it).
> 
> Therefore, it sounds reasonable to switch to O_EXCL to prevent
> interfering with other programs and to prevent other programs
> interfering with QEMU.
> 
> On the downside, it will no longer be possible to share a host CD-ROM
> between multiple virtual machines or to mount it on host while passing
> it through to a guest.  These scenarios are not safe because on of the
> clients could eject the device, spoiling the party for everyone else.
> However, it is a handy feature for putting installation media into a
> machine and installing several guests at the same time.

Two things strike me here

 - Host CDROM device vs ISO image files have different behaviour
   by default, since they use different internal block drivers.
   If the app wants to share the host CDROM, then perhaps it
   should be force QEMU to access it with the 'raw' block driver
   instead of 'host_cdrom'. Thus "eject" would just result in
   the drive file being closed & filename blanked as for ISOs

 - If the -drive specification has  readonly=on (thus O_RDONLY to
   open(2) call) , I expect QEMU (or the kernel) to forbid the
   "eject" command on the host CDROM. This should prevent two guests
   interfering seriously with each other.

So I think using O_EXCL would be OK, in the case where the block
driver was  host_cdrom and readonly=off.

> The other concern I have about using O_EXCL is that we expose
> ourselves to race conditions if there is ever a need to re-open the
> device.  When QEMU closes its file descriptor another program may be
> scheduled to run and open the device with O_EXCL.  Now QEMU will not
> be able to open the CD-ROM anymore.  From the guest perspective this
> could be at an odd time and we'd have to start failing requests.
> Today we do not re-open host CD-ROMs though so this isn't a pressing
> problem.

Ideally the kernel support would be enhanced such that we don't need to
close & reopen the file, so we can avoid the failure path.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Kevin Wolf]

Am 12.04.2011 09:52, schrieb Daniel P. Berrange:
>  - If the -drive specification has  readonly=on (thus O_RDONLY to
>    open(2) call) , I expect QEMU (or the kernel) to forbid the
>    "eject" command on the host CDROM. This should prevent two guests
>    interfering seriously with each other.
> 
> So I think using O_EXCL would be OK, in the case where the block
> driver was  host_cdrom and readonly=off.

This would overload readonly with a completely unrelated option (should
eject be allowed). Doesn't sound like a great idea.

Kevin

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Daniel P. Berrange]

On Tue, Apr 12, 2011 at 10:10:44AM +0200, Kevin Wolf wrote:
> Am 12.04.2011 09:52, schrieb Daniel P. Berrange:
> >  - If the -drive specification has  readonly=on (thus O_RDONLY to
> >    open(2) call) , I expect QEMU (or the kernel) to forbid the
> >    "eject" command on the host CDROM. This should prevent two guests
> >    interfering seriously with each other.
> > 
> > So I think using O_EXCL would be OK, in the case where the block
> > driver was  host_cdrom and readonly=off.
> 
> This would overload readonly with a completely unrelated option (should
> eject be allowed). Doesn't sound like a great idea.

Use of the "host_cdrom" block driver enables passthrough of
commands to the host device.

Use of "readonly" is what controls whether individual passthrough
commands are actually permitted.

To me, "readonly" means don't allow anything that would impact
another guests view of the file/device. So forbidding 'eject'
is within scope of that IMHO.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] To O_EXCL or not to O_EXCL open host_cdrom

[Stefan Hajnoczi]

On Tue, Apr 12, 2011 at 9:19 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Tue, Apr 12, 2011 at 10:10:44AM +0200, Kevin Wolf wrote:
>> Am 12.04.2011 09:52, schrieb Daniel P. Berrange:
>> >  - If the -drive specification has  readonly=on (thus O_RDONLY to
>> >    open(2) call) , I expect QEMU (or the kernel) to forbid the
>> >    "eject" command on the host CDROM. This should prevent two guests
>> >    interfering seriously with each other.
>> >
>> > So I think using O_EXCL would be OK, in the case where the block
>> > driver was  host_cdrom and readonly=off.
>>
>> This would overload readonly with a completely unrelated option (should
>> eject be allowed). Doesn't sound like a great idea.
>
> Use of the "host_cdrom" block driver enables passthrough of
> commands to the host device.
>
> Use of "readonly" is what controls whether individual passthrough
> commands are actually permitted.
>
> To me, "readonly" means don't allow anything that would impact
> another guests view of the file/device. So forbidding 'eject'
> is within scope of that IMHO.

I agree with Kevin here.  You're overloading the option.

Today I doubt readonly will prevent the eject/lock ioctls but I haven't tested.

I'm becoming more convinced now that we should be using O_EXCL.  Like
you say, forcing raw gives us the option to share the device for
reading without allowing the dangerous ioctls.

Stefan