[BUG,PATCH] race in xen-block-hotplug

[Philipp Hahn <hahn@univention.de>]

[-- Attachment #1: Type: text/plain, Size: 1791 bytes --]

Hello,

we encountered a strange race condition in tools/hotplug/Linux/block,
which only shows very rarely and only once for the first domain started
after the host server is started. The complete details are in our
Bugzilla at <https://forge.univention.org/bugzilla/show_bug.cgi?id=20481>.

In our case its Xen-4.1.3 (yes I know it's ancient, but the code is
still the same in current Xen-git) and it only happens for file-backed
files (in our case a ISO-image as CD-ROM).

>        # Avoid a race with the remove if the path has been deleted, or
> »···# otherwise changed from "InitWait" state e.g. due to a timeout
>         xenbus_state=$(xenstore_read_default "$XENBUS_PATH/state" 'unknown')
>         if [ "$xenbus_state" != '2' ]
>         then
>           release_lock "block"
>           fatal "Path closed or removed during hotplug add: $XENBUS_PATH state: $xenbus_state"
>         fi

The problem is that sometimes the other end (kernel/qemu?) is too slow
and the device is still in in state 1=Initializing. If that happens, the
domU stat is aborted and destroyed.
If the same VM is then started again, it works flawlessly.

That code block was added in
<http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=92e6cb5673b37bd883bdef0d0e83faf000edf61d>
by Keir Fraser.

My work-around is to delay for one more second and retry again if the
state is 1=Initializing. The printout confirmed that that case actually
happened.

Signed-off-by: Philipp Hahn <hahn@univention.de>

BYtE
Philipp Hahn
-- 
Philipp Hahn
Open Source Software Engineer

Univention GmbH
be open.
Mary-Somerville-Str. 1
D-28359 Bremen
Tel.: +49 421 22232-0
Fax : +49 421 22232-99
hahn@univention.de

http://www.univention.de/
Geschäftsführer: Peter H. Ganten
HRB 20755 Amtsgericht Bremen
Steuer-Nr.: 71-597-02876

[-- Attachment #2: fix_hotplug_race.patch --]
[-- Type: text/x-patch, Size: 1242 bytes --]

# Bug #20481: Fix race during device hotplug
# Sometimes qemu/the kernel takes too long for the first domain to start. The
# hotplug script then finds the device still in state 1=Initialising and aborts.
# Add an artifical delay of 1 seconds and try again.
diff --git a/tools/hotplug/Linux/block b/tools/hotplug/Linux/block
index 06de5c9..cbf2af3 100644
--- a/tools/hotplug/Linux/block
+++ b/tools/hotplug/Linux/block
@@ -255,12 +255,16 @@ case "$command" in
 
         # Avoid a race with the remove if the path has been deleted, or
 	# otherwise changed from "InitWait" state e.g. due to a timeout
-        xenbus_state=$(xenstore_read_default "$XENBUS_PATH/state" 'unknown')
-        if [ "$xenbus_state" != '2' ]
-        then
+        while true
+        do
+          xenbus_state=$(xenstore_read_default "$XENBUS_PATH/state" 'unknown')
+          case "$xenbus_state" in
+          1) log notice "Path still initializing: $XENBUS_PATH" ; sleep 1 ; continue ;;
+          2) break ;;
+          esac
           release_lock "block"
           fatal "Path closed or removed during hotplug add: $XENBUS_PATH state: $xenbus_state"
-        fi
+        done
 
         if [ "$mode" = 'w' ] && ! stat "$file" -c %A | grep -q w
         then

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel