[RFC][PATCH] Increase hotplug timeout

[RFC][PATCH] Increase hotplug timeout

[Jiri Denemark]

[-- Attachment #1: Type: text/plain, Size: 666 bytes --]

Hi.

When a process in dom0 is writing huge amount of data to a disk in parallel to
creating new domain, it may take quite a lot of time until all hotplug scripts
finish. For example, on my system it takes almost 5 minutes to start all
hotplug scripts.

So it seems 100 seconds is not enough for such scenarios as xend would give up
waiting for hotplug scripts too early. What do you think about the attached
patch, which increases the timeout to 10 minutes? Of course, the dark side of
this change is that it would take ages to recognize hotplug scripts are not
really working.

Thanks for your opinions on this.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>

[-- Attachment #2: hoplug-timeout.patch --]
[-- Type: text/plain, Size: 511 bytes --]

diff --git a/tools/python/xen/xend/server/DevConstants.py b/tools/python/xen/xend/server/DevConstants.py
--- a/tools/python/xen/xend/server/DevConstants.py
+++ b/tools/python/xen/xend/server/DevConstants.py
@@ -16,7 +16,7 @@
 # Copyright (C) 2005 XenSource Ltd
 #============================================================================
 
-DEVICE_CREATE_TIMEOUT  = 100
+DEVICE_CREATE_TIMEOUT  = 600
 DEVICE_DESTROY_TIMEOUT = 100
 HOTPLUG_STATUS_NODE = "hotplug-status"
 HOTPLUG_ERROR_NODE  = "hotplug-error"

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC][PATCH] Increase hotplug timeout

[Ian Jackson]

Jiri Denemark writes ("[Xen-devel] [RFC][PATCH] Increase hotplug timeout"):
> When a process in dom0 is writing huge amount of data to a disk in
> parallel to creating new domain, it may take quite a lot of time
> until all hotplug scripts finish. For example, on my system it takes
> almost 5 minutes to start all hotplug scripts.

That's a very long time.  All of that for just one domain ?

> So it seems 100 seconds is not enough for such scenarios as xend
> would give up waiting for hotplug scripts too early. What do you
> think about the attached patch, which increases the timeout to 10
> minutes? Of course, the dark side of this change is that it would
> take ages to recognize hotplug scripts are not really working.

I think very few people have configurations like yours.

Perhaps it would be best to make the timeout configurable, but leave
the default unchanged ?

Ian.

Re: [RFC][PATCH] Increase hotplug timeout

[Daniel P. Berrange]

On Thu, Apr 16, 2009 at 11:48:58AM +0100, Ian Jackson wrote:
> Jiri Denemark writes ("[Xen-devel] [RFC][PATCH] Increase hotplug timeout"):
> > When a process in dom0 is writing huge amount of data to a disk in
> > parallel to creating new domain, it may take quite a lot of time
> > until all hotplug scripts finish. For example, on my system it takes
> > almost 5 minutes to start all hotplug scripts.
> 
> That's a very long time.  All of that for just one domain ?
> 
> > So it seems 100 seconds is not enough for such scenarios as xend
> > would give up waiting for hotplug scripts too early. What do you
> > think about the attached patch, which increases the timeout to 10
> > minutes? Of course, the dark side of this change is that it would
> > take ages to recognize hotplug scripts are not really working.
> 
> I think very few people have configurations like yours.

It is a problem that gets more common as you add more devices. eg if
you have lots of disks or nics then it becomes much more likely to
occur. Add in a 2nd guest which is doing alot of I/O at the same time
you are trying to start this guest and it gets worse again, because
xenstored I/O performance starts to hurt. Having xenstored on tmpfs 
should make it happen less. Having a fully in memory xenstored without
any need for tmpfs at all, would be even better.


Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [RFC][PATCH] Increase hotplug timeout

[Jiri Denemark]

> > When a process in dom0 is writing huge amount of data to a disk in
> > parallel to creating new domain, it may take quite a lot of time
> > until all hotplug scripts finish. For example, on my system it takes
> > almost 5 minutes to start all hotplug scripts.
> 
> That's a very long time.  All of that for just one domain ?

Yes. Just a single domain with one nic and one disk.

> Perhaps it would be best to make the timeout configurable, but leave
> the default unchanged ?

You're right, this might be a better way... I'll provide a new patch.

Jirka