From mboxrd@z Thu Jan 1 00:00:00 1970 From: George Shuklin Subject: bug in xenstored? No notification to subscription on @introduceDomain Date: Fri, 09 Dec 2011 23:49:48 +0400 Message-ID: <4EE2665C.8090602@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com, xen-hosting@googlegroups.com List-Id: xen-devel@lists.xenproject.org Good day. I think I met some strange bug in xenstored. I using XCP for long time and all that time we have some funny bug we was not able to debug enough due product environment and very low chance to appear, now we was able to catch it in testing environment and done some research. We have python application running in dom0 and waiting domain appearance. This implemented this via subscription to @introduceDomain xenstore key. Under some conditions we stops to receive notification on subscription. If we ran application as second instance it will receive that notification, if we restart application it will receive too. I unable to pinpoint exact condition for this, but this a) Happens occasionally but consistently (about once a month in farm of 50 hosts at least at one host) b) Not related to xenstored uptime c) Not related to load on xen or dom0 d) Not related to amount of domains e) Occur at least at XCP 0.5, 1.0 and 1.1 (I don't know how to get version from xenstored) Last time I got that on two hosts in lab at same time (with single guest domain without any high load) and done some experiments - so I can say exactly I wrote above. The pieces from python code we ran: from xen.lowlevel.xs import xs conn = xs.xs() conn.watch("@introduceDomain", "+") conn.watch("@releaseDomain", "-") conn.read_watch()