qdisks and stubdomains

qdisks and stubdomains

[Simon Weald]

Hello

I don't know whether this classifies as a bug or missing functionality,
but I'm looking to attach Ceph RBD volumes directly to a guest (as
opposed to mapping them on the host and then passing the block device to
the guest). The rationale for this is that the kernel RBD client's
functionality lags behind that of librbd, meaning the kernel cannot map
RBD volumes created with the newer features.

For a little bit of background, these are PVHVM guests, I'm running
4.7.1 and 4.8.0 on Jessie, and the Ceph cluster is 10.2.5 (latest Jewel
release).

When running the guest outside of a stubdomain, block-attach exits
successfully:

xl block-attach domU format=raw, vdev=xvdb, access=rw,
backendtype=qdisk, target=rbd:cinder/test:id=xenhosts

root@xen:~# xl block-list domU
Vdev  BE  handle state evt-ch ring-ref BE-path
768   0   6      4     12     8        /local/domain/0/backend/vbd/6/768
5632  0   6      6     -1     -1       /local/domain/0/backend/qdisk/6/5632
832   0   6      3     22     897      /local/domain/0/backend/qdisk/6/832

root@mshx-rd-6:~# xenstore-ls /local/domain/0/backend/qdisk/6/832 -f
/local/domain/0/backend/qdisk/6/832/frontend =
"/local/domain/6/device/vbd/832"
/local/domain/0/backend/qdisk/6/832/params =
"aio:rbd:cinder/test:id=xenhosts"
/local/domain/0/backend/qdisk/6/832/frontend-id = "6"
/local/domain/0/backend/qdisk/6/832/online = "1"
/local/domain/0/backend/qdisk/6/832/removable = "0"
/local/domain/0/backend/qdisk/6/832/bootable = "1"
/local/domain/0/backend/qdisk/6/832/state = "2"
/local/domain/0/backend/qdisk/6/832/dev = "hdb"
/local/domain/0/backend/qdisk/6/832/type = "qdisk"
/local/domain/0/backend/qdisk/6/832/mode = "w"
/local/domain/0/backend/qdisk/6/832/device-type = "disk"
/local/domain/0/backend/qdisk/6/832/discard-enable = "1"
/local/domain/0/backend/qdisk/6/832/feature-flush-cache = "1"
/local/domain/0/backend/qdisk/6/832/feature-persistent = "1"
/local/domain/0/backend/qdisk/6/832/info = "0"
/local/domain/0/backend/qdisk/6/832/feature-discard = "1"
/local/domain/0/backend/qdisk/6/832/hotplug-status = "connected"

This all looks good at this point, however the device doesn't actually
appear available to the guest (no device nodes, nothing in dmesg). This
however is academic, as the ultimate goal is to attach the volumes to
stubdomained guests (without relying on the problematic kernel client
and block scripts). When I try a block-attach to the stubdomained guest,
I get the following:

libxl: error: libxl_dm.c:2423:libxl__dm_check_start: device model
required but not running
libxl: error: libxl.c:2012:device_addrm_aocomplete: unable to add device
libxl_device_disk_add failed.

Is this even possible?

Thanks in advance!

Simon


-- 

Simon Weald
Systems Administrator

PGP: http://www.simonweald.com/SimonWeald.asc
     https://pgp.mit.edu/pks/lookup?op=get&search=0x988E9858747ABE88

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: qdisks and stubdomains

[Gémes Géza]

2017-03-02 16:29 keltezéssel, Simon Weald írta:
> Hello
>
> I don't know whether this classifies as a bug or missing functionality,
> but I'm looking to attach Ceph RBD volumes directly to a guest (as
> opposed to mapping them on the host and then passing the block device to
> the guest). The rationale for this is that the kernel RBD client's
> functionality lags behind that of librbd, meaning the kernel cannot map
> RBD volumes created with the newer features.
>
> For a little bit of background, these are PVHVM guests, I'm running
> 4.7.1 and 4.8.0 on Jessie, and the Ceph cluster is 10.2.5 (latest Jewel
> release).
>
> When running the guest outside of a stubdomain, block-attach exits
> successfully:
>
> xl block-attach domU format=raw, vdev=xvdb, access=rw,
> backendtype=qdisk, target=rbd:cinder/test:id=xenhosts
>
> root@xen:~# xl block-list domU
> Vdev  BE  handle state evt-ch ring-ref BE-path
> 768   0   6      4     12     8        /local/domain/0/backend/vbd/6/768
> 5632  0   6      6     -1     -1       /local/domain/0/backend/qdisk/6/5632
> 832   0   6      3     22     897      /local/domain/0/backend/qdisk/6/832
>
> root@mshx-rd-6:~# xenstore-ls /local/domain/0/backend/qdisk/6/832 -f
> /local/domain/0/backend/qdisk/6/832/frontend =
> "/local/domain/6/device/vbd/832"
> /local/domain/0/backend/qdisk/6/832/params =
> "aio:rbd:cinder/test:id=xenhosts"
> /local/domain/0/backend/qdisk/6/832/frontend-id = "6"
> /local/domain/0/backend/qdisk/6/832/online = "1"
> /local/domain/0/backend/qdisk/6/832/removable = "0"
> /local/domain/0/backend/qdisk/6/832/bootable = "1"
> /local/domain/0/backend/qdisk/6/832/state = "2"
> /local/domain/0/backend/qdisk/6/832/dev = "hdb"
> /local/domain/0/backend/qdisk/6/832/type = "qdisk"
> /local/domain/0/backend/qdisk/6/832/mode = "w"
> /local/domain/0/backend/qdisk/6/832/device-type = "disk"
> /local/domain/0/backend/qdisk/6/832/discard-enable = "1"
> /local/domain/0/backend/qdisk/6/832/feature-flush-cache = "1"
> /local/domain/0/backend/qdisk/6/832/feature-persistent = "1"
> /local/domain/0/backend/qdisk/6/832/info = "0"
> /local/domain/0/backend/qdisk/6/832/feature-discard = "1"
> /local/domain/0/backend/qdisk/6/832/hotplug-status = "connected"
>
> This all looks good at this point, however the device doesn't actually
> appear available to the guest (no device nodes, nothing in dmesg). This
> however is academic, as the ultimate goal is to attach the volumes to
> stubdomained guests (without relying on the problematic kernel client
> and block scripts). When I try a block-attach to the stubdomained guest,
> I get the following:
>
> libxl: error: libxl_dm.c:2423:libxl__dm_check_start: device model
> required but not running
> libxl: error: libxl.c:2012:device_addrm_aocomplete: unable to add device
> libxl_device_disk_add failed.
>
> Is this even possible?
>
> Thanks in advance!
>
> Simon
>
>
Hi Simon,

If you are using the minios based stubdom, then the qemu in the stubdom 
is qemu-traditional, which is an old fork of the upstream qemu, which 
clearly lacks support for ceph. Unfortunately there is no support for 
qemu-xen yet in the stubdom implementations in the xen tree.

Cheers,

Geza


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel