sharing a (mostly) read-only virtual block device

sharing a (mostly) read-only virtual block device

[Antoine Martin]

Hi,

Is there an easy way that I have missed to share a virtual disk
read-only between many guests whilst still having the ability to update
it occasionally from the host?

I was hoping I could use a shared image file, and occasionally replace
it with an updated version (move old copy to disk_image.bak, copy new
image to disk_image), then the guests could umount/mount the drive and
get access to the new disk image.

Unfortunately qemu opens the virtual disk as soon as the guest boots, so
the file descriptor still points to the old image.
Note: I do not want to use the qemu monitor from the host as I want the
guests to be in charge of when/if they get the new disk image.

Any suggestions / other ways of doing this?

Thanks
Antoine

Re: sharing a (mostly) read-only virtual block device

[Avi Kivity]

On 10/16/2009 07:45 PM, Antoine Martin wrote:
> Hi,
>
> Is there an easy way that I have missed to share a virtual disk
> read-only between many guests whilst still having the ability to update
> it occasionally from the host?
>
>    

That's very fragile, since the guest won't expect the disk to change 
under its feet.  Expect oopses.

> I was hoping I could use a shared image file, and occasionally replace
> it with an updated version (move old copy to disk_image.bak, copy new
> image to disk_image), then the guests could umount/mount the drive and
> get access to the new disk image.
>
> Unfortunately qemu opens the virtual disk as soon as the guest boots, so
> the file descriptor still points to the old image.
> Note: I do not want to use the qemu monitor from the host as I want the
> guests to be in charge of when/if they get the new disk image.
>
>    

I suggest using a monitor, and have the host and guest coordinate the 
change (guest unmounts, host modifies, guest mounts).

Alternatively, export the disk from the host using nfs.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: sharing a (mostly) read-only virtual block device

[Michael Tokarev]

Avi Kivity пишет:
> On 10/16/2009 07:45 PM, Antoine Martin wrote:
>> Hi,
>>
>> Is there an easy way that I have missed to share a virtual disk
>> read-only between many guests whilst still having the ability to update
>> it occasionally from the host?
> 
> That's very fragile, since the guest won't expect the disk to change 
> under its feet.  Expect oopses.

There's another way possible.  Not sure if its feasible here due to
the amount of space it requires.

The idea is to move the original _unmodified_ image out of the way but keep
it.  All guests who have it open now will keep it open and will not see the
changes.  But you now require at least 2x space - for old image and for the
new one. Or more, if you want to keep some guests running for longer so they
still refer to pre-last or pre-pre-last image version.

It can be done by preparing the new file as foo.new and moving it into place
by mv.  The old file gets removed from the directory but not removed physically
from the filesystem, till all the references to it (open by another process)
will be gone.

[]
> I suggest using a monitor, and have the host and guest coordinate the 
> change (guest unmounts, host modifies, guest mounts).

Yes that's the way to go.  Or, simpler, reboot the guest(s).

There's no need to umount the filesystem in the guest if going "my way" above.

> Alternatively, export the disk from the host using nfs.

And yes, that's also a very good idea.

/mjt

Re: sharing a (mostly) read-only virtual block device

[Antoine Martin]

Michael Tokarev wrote:
> Avi Kivity пишет:
>> On 10/16/2009 07:45 PM, Antoine Martin wrote:
>>> Hi,
>>>
>>> Is there an easy way that I have missed to share a virtual disk
>>> read-only between many guests whilst still having the ability to update
>>> it occasionally from the host?
>>
>> That's very fragile, since the guest won't expect the disk to change
>> under its feet.  Expect oopses.
That is not what I meant, sorry if this was not clear enough. See below.

> There's another way possible.  Not sure if its feasible here due to
> the amount of space it requires.
Space is not really my concern here. Ease of management is.
Especially if this allows many guests to share the same virtual disks,
this would more than compensate for the extra copies.

> The idea is to move the original _unmodified_ image out of the way but keep
> it.  All guests who have it open now will keep it open and will not see the
> changes.  But you now require at least 2x space - for old image and for the
> new one. Or more, if you want to keep some guests running for longer so
> they
> still refer to pre-last or pre-pre-last image version.
> 
> It can be done by preparing the new file as foo.new and moving it into
> place
> by mv.  The old file gets removed from the directory but not removed
> physically
> from the filesystem, till all the references to it (open by another
> process)
> will be gone.
This is exactly the solution I suggested in my original post. (which got
snipped out)

The problem with this one is that (quote from original post):
"Unfortunately qemu opens the virtual disk as soon as the guest boots,
so the file descriptor still points to the old image."

Which means that the guest will not see the new file until it is rebooted.

So close... yet so far...

> []
>> I suggest using a monitor, and have the host and guest coordinate the
>> change (guest unmounts, host modifies, guest mounts).
In terms of ease of management, that's far from ideal.
This requires the guests and hosts to co-operate. These may not be
managed by the same people.
Which is why I had said "Note: I do not want to use the qemu monitor..."

> Yes that's the way to go.  Or, simpler, reboot the guest(s).
Again, not ideal. Having to reboot just to get access to a file that is
just there waiting... is frustrating!

> There's no need to umount the filesystem in the guest if going "my way"
> above.
(as above: problem is that the guest will not be getting the new
filesystem until it reboots...)

>> Alternatively, export the disk from the host using nfs.
> 
> And yes, that's also a very good idea.
One I had considered and that I dislike for the same reasons I mentioned
above. The sheer number of processes and ports involved on the host
makes me cringe. When trying to get close to bare-metal on the host,
running network daemons like nfs is just not going to happen.
(not to mention the security considerations)

I would much prefer a solution involving just shared read-only files.

I realize that it is probably quite hard (if not impossible) to tell
qemu to re-open the disk image the next time that the guest unmounts the
existing disk image. That's a shame, because it would do the job nicely:
1) signal qemu
2) guest unmounts/remounts (whenever it wants)
Done.

Thanks
Antoine

Re: sharing a (mostly) read-only virtual block device

[Avi Kivity]

On 10/18/2009 09:25 PM, Antoine Martin wrote:
>> The idea is to move the original _unmodified_ image out of the way but keep
>> it.  All guests who have it open now will keep it open and will not see the
>> changes.  But you now require at least 2x space - for old image and for the
>> new one. Or more, if you want to keep some guests running for longer so
>> they
>> still refer to pre-last or pre-pre-last image version.
>>
>> It can be done by preparing the new file as foo.new and moving it into
>> place
>> by mv.  The old file gets removed from the directory but not removed
>> physically
>> from the filesystem, till all the references to it (open by another
>> process)
>> will be gone.
>>      
> This is exactly the solution I suggested in my original post. (which got
> snipped out)
>
> The problem with this one is that (quote from original post):
> "Unfortunately qemu opens the virtual disk as soon as the guest boots,
> so the file descriptor still points to the old image."
>
> Which means that the guest will not see the new file until it is rebooted.
>
> So close... yet so far...
>    

You can try to use the cdrom support.  Eject the old disk, insert the 
new disk.

>>> I suggest using a monitor, and have the host and guest coordinate the
>>> change (guest unmounts, host modifies, guest mounts).
>>>        
> In terms of ease of management, that's far from ideal.
> This requires the guests and hosts to co-operate. These may not be
> managed by the same people.
>    

Write a bit of software to coordinate.

> Which is why I had said "Note: I do not want to use the qemu monitor..."
>    

The guest and host have to cooperate.  The host has no notion of the 
guest mounting and unmounting the filesystem.

>> Yes that's the way to go.  Or, simpler, reboot the guest(s).
>>      
> Again, not ideal. Having to reboot just to get access to a file that is
> just there waiting... is frustrating!
>    

Rebooting won't actually help, since it won't reopen the file.

>>> Alternatively, export the disk from the host using nfs.
>>>        
>> And yes, that's also a very good idea.
>>      
> One I had considered and that I dislike for the same reasons I mentioned
> above. The sheer number of processes and ports involved on the host
> makes me cringe. When trying to get close to bare-metal on the host,
> running network daemons like nfs is just not going to happen.
>    

Is this measured overhead or assumed overhead?

> (not to mention the security considerations)
>    

What security considerations?

> I would much prefer a solution involving just shared read-only files.
>
> I realize that it is probably quite hard (if not impossible) to tell
> qemu to re-open the disk image the next time that the guest unmounts the
> existing disk image. That's a shame, because it would do the job nicely:
> 1) signal qemu
> 2) guest unmounts/remounts (whenever it wants)
>    

Try ejecting the cdrom, it may work for you.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.