Re: [Qemu-devel] nbd: Possible regression in 2.9 RCs

[Kevin Wolf <kwolf@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4758 bytes --]

Am 31.03.2017 um 19:43 hat Max Reitz geschrieben:
> On 31.03.2017 18:03, Ciprian Barbu wrote:
> > Hello,
> > 
> > Similar to the other thread about possible regression with rbd, there might be a regression with nbd.
> > This time we are launching an instance from an image (not volume) and try to live migrate it:
> > 
> > nova live-migration <test_instance>
> > 
> > The nova-compute service complains with:
> > 
> > 2017-03-31 15:32:56.179 7806 INFO nova.virt.libvirt.driver [req-15d79cbe-5956-4738-92df-3624e6b993ee d795de59fb9a4ea38776a11d20ae8469 cee03e74881f4ccba3b83345fb652b2c - - -] [instance: 6a04508f-5d79-4582-8e2c-4cc368753f6c] Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)
> > 2017-03-31 15:32:58.029 7806 WARNING stevedore.named [req-73bc0113-5555-4dd8-8903-d3540cc61b47 b9fbceeadd2d4d1bab9c90ae104db1f7 7e7db99b32c6467184701e9a0c2f1de7 - - -] Could not load instance_network_info
> > 2017-03-31 15:32:59.038 7806 ERROR nova.virt.libvirt.driver [req-15d79cbe-5956-4738-92df-3624e6b993ee d795de59fb9a4ea38776a11d20ae8469 cee03e74881f4ccba3b83345fb652b2c - - -] [instance: 6a04508f-5d79-4582-8e2c-4cc368753f6c] Live Migration failure: internal error: unable to execute QEMU command 'nbd-server-add': Conflicts with use by drive-virtio-disk0 as 'root', which does not allow 'write' on #block143
> > 2017-03-31 15:32:59.190 7806 ERROR nova.virt.libvirt.driver [req-15d79cbe-5956-4738-92df-3624e6b993ee d795de59fb9a4ea38776a11d20ae8469 cee03e74881f4ccba3b83345fb652b2c - - -] [instance: 6a04508f-5d79-4582-8e2c-4cc368753f6c] Migration operation has aborted
> > 
> > I will try and bisect it myself, but I thought I would paste this here first, just so you know there is this issue too.
> 
> Well, I already know the commit in question. It's
> 8a7ce4f9338c475df1afc12502af704e4300a3e0 ("nbd/server: Use real
> permissions for NBD exports").
> 
> Whether this is a bug depends on the standpoint. I would very much
> consider it a bug fix because as of this commit you can no longer create
> a writable NBD server on a block device that is in use by a guest device
> without the guest device being aware of this.
> 
> The problem is that the functionality to "make" the guest device "aware"
> of it was introduced only a couple of commits before, and it's called
> "share-rw".
> 
> So this doesn't work:
> 
> $ x86_64-softmmu/qemu-system-x86_64 \
>     -blockdev node-name=image,driver=qcow2,\
> file.driver=file,file.filename=foo.qcow2 \
>     -device virtio-blk,drive=image \
>     -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 92, "minor": 8, "major": 2},
> "package": " (v2.8.0-2038-g6604c893d0)"}, "capabilities": []}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {'execute':'nbd-server-start','arguments':{'addr':{'type':'inet','data':{'host':'localhost','port':'10809'}}}}
> {"return": {}}
> {'execute':'nbd-server-add','arguments':{'device':'image','writable':true}}
> {"error": {"class": "GenericError", "desc": "Conflicts with use by
> /machine/peripheral-anon/device[0]/virtio-backend as 'root', which does
> not allow 'write' on image"}
> 
> But this works:
> 
> $ x86_64-softmmu/qemu-system-x86_64 \
>     -blockdev node-name=image,driver=qcow2,\
> file.driver=file,file.filename=foo.qcow2 \
>     -device virtio-blk,drive=image,share-rw=on \
>     -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 92, "minor": 8, "major": 2},
> "package": " (v2.8.0-2038-g6604c893d0)"}, "capabilities": []}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {'execute':'nbd-server-start','arguments':{'addr':{'type':'inet','data':{'host':'localhost','port':'10809'}}}}
> {"return": {}}
> {'execute':'nbd-server-add','arguments':{'device':'image','writable':true}}
> {"return": {}}
> 
> (The difference is the share-rw=on in the -device parameter.)
> 
> So in theory all that's necessary is to set share-rw=on for the device
> in the management layer. But I'm not sure whether that's practical.

Yes, libvirt needs to provide this option if the guest supports sharing.
If it doesn't support sharing, rejecting a read-write NBD client seems
correct to me.

Peter, Eric, what is the status on the libvirt side here?

> As for just allowing the NBD server write access to the device... To me
> that appears pretty difficult from an implementation perspective. We
> assert that nobody can write without having requested write access and
> we make sure that nobody can request write access without it being
> allowed. Making an exception for NBD seems very difficult and would
> probably mean we'd have to drop the assertion for write accesses altogether.

Making an exception would simply be wrong.

Kevin

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]