Re: handling local attach of phy disks for pygrub (Was: Xen 4.3 xl migrate " htree_dirblock_to_tree" on second host)

["Roger Pau Monné" <roger.pau@citrix.com>]

On 13/02/14 09:17, Roger Pau Monné wrote:
> On 13/02/14 02:29, Miguel Clara wrote:
>> I tried the patch provided by roger, I get a different error now:
>>
>> Parsing config from test.cfg
>> libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend
>> /local/domain/0/backend/vbd/0/51712 not ready
>> libxl: error: libxl_bootloader.c:405:bootloader_disk_attached_cb:
>> failed to attach local disk for bootloader execution
>> libxl: error: libxl_bootloader.c:276:bootloader_local_detached_cb:
>> unable to detach locally attached disk
>> libxl: error: libxl_create.c:900:domcreate_rebuild_done: cannot
>> (re-)build domain: -1
>>
>>
>> with -vvv
>>
>> # xl -vvv create test.cfg
>> Parsing config from test.cfg
>> libxl: debug: libxl_create.c:1230:do_domain_create: ao 0x12548c0:
>> create: how=(nil) callback=(nil) poller=0x1254980
>> libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk
>> vdev=xvda spec.backend=unknown
>> libxl: debug: libxl_device.c:188:disk_try_backend: Disk vdev=xvda,
>> uses script=... assuming phy backend
>> libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk
>> vdev=xvda, using backend phy
>> libxl: debug: libxl_create.c:675:initiate_domain_create: running bootloader
>> libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk
>> vdev=(null) spec.backend=phy
>> libxl: debug: libxl_device.c:188:disk_try_backend: Disk vdev=(null),
>> uses script=... assuming phy backend
>> libxl: debug: libxl.c:2605:libxl__device_disk_local_initiate_attach:
>> trying to locally attach PHY device drbd-remus-test with script
>> block-drbd
>> libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk
>> vdev=xvdf spec.backend=phy
>> libxl: debug: libxl_device.c:188:disk_try_backend: Disk vdev=xvdf,
>> uses script=... assuming phy backend
>> libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch
>> w=0x124a300 wpath=/local/domain/0/backend/vbd/0/51792/state token=3/0:
>> register slotnum=3
>> libxl: debug: libxl_create.c:1243:do_domain_create: ao 0x12548c0:
>> inprogress: poller=0x1254980, flags=i
>> libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x124a300
>> wpath=/local/domain/0/backend/vbd/0/51792/state token=3/0: event
>> epath=/local/domain/0/backend/vbd/0/51792/state
>> libxl: debug: libxl_event.c:647:devstate_watch_callback: backend
>> /local/domain/0/backend/vbd/0/51792/state wanted state 2 still waiting
>> state 1
>> libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x124a300
>> wpath=/local/domain/0/backend/vbd/0/51792/state token=3/0: event
>> epath=/local/domain/0/backend/vbd/0/51792/state
>> libxl: debug: libxl_event.c:643:devstate_watch_callback: backend
>> /local/domain/0/backend/vbd/0/51792/state wanted state 2 ok
>> libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch
>> w=0x124a300 wpath=/local/domain/0/backend/vbd/0/51792/state token=3/0:
>> deregister slotnum=3
>> libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch
>> w=0x124a300: deregister unregistered
>> libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug
>> script: /etc/xen/scripts/block-drbd add
>> libxl: debug: libxl.c:2692:local_device_attach_cb: locally attached
>> disk /dev/xvdf
>> libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend
>> /local/domain/0/backend/vbd/0/51792 not ready
> 
> So the local attach seems to DTRT, but the device never gets to state 4
> (connected). Does the block-drbd script work with guests that are not
> using pygrub? (extract the kernel from the DomU and use it directly on
> the config file).

I've been looking into this, and found out that the block-drbd script 
is outdated, it expects that the type node in xenstore will be "drbd" 
(which probably was what xend was setting), but libxl sets it to "phy", 
so the following patch to block-drbd is needed. If it solves your 
problems I will upstream it to drbd for inclusion on new releases.

(The patch is against git://git.drbd.org/drbd-9.0.git)

---
commit 4a4806421d81b30762ed6a0b111e491b77e78a08
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Thu Feb 13 11:01:48 2014 +0100

    block-drbd: type is "phy" for drbd backends
    
    The type written to xenstore by libxl when attaching a drbd backend is
    "phy", not "drbd", so handle this case also.
    
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

diff --git a/scripts/block-drbd b/scripts/block-drbd
index 5563ccb..975802b 100755
--- a/scripts/block-drbd
+++ b/scripts/block-drbd
@@ -250,7 +250,7 @@ case "$command" in
     fi
 
     case $t in 
-      drbd)
+      drbd|phy)
         drbd_resource=$p
         drbd_role="$(drbdadm role $drbd_resource)"
         drbd_lrole="${drbd_role%%/*}"
@@ -278,7 +278,7 @@ case "$command" in
 
   remove)
     case $t in 
-      drbd)
+      drbd|phy)
         p=$(xenstore_read "$XENBUS_PATH/params")
         drbd_resource=$p
         drbd_role="$(drbdadm role $drbd_resource)"