From mboxrd@z Thu Jan 1 00:00:00 1970 From: Charles Duffy Subject: Re: Timing out waiting for vbd device *only when xend started with trace_start* Date: Mon, 14 Nov 2005 06:43:15 -0600 Message-ID: References: <20051114115108.GA28247@leeni.uk.xensource.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------040503070908090905040800" Return-path: In-Reply-To: <20051114115108.GA28247@leeni.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------040503070908090905040800 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Ewan Mellor wrote: > This is showing up intermittently with our regression tests. It seems to be a > race of some sort, which would be consistent with it going away or coming back > when you fiddle with the logging -- all you are doing is changing the timing > of things so that it appears or does not appear. > > If you export XENSTORED_TRACE=1 before starting Xend and Xenstored, then you > will get a log file in /var/log/xenstored-trace.log. This information is > interesting to me, because it shows the ordering over time of the store > events, which makes it easier to piece together what's going on. It does indeed feel like a race condition. Unfortunetly, setting XENSTORED_TRACE impacts the timing such as to make it nonreproducible. > Also, could you put a line in > /usr/lib/python/xen/xend/server/DevController.py, to log entry into the > hotplugStatus function (the syntax is log.debug("hotplugStatus entered"), and > it goes below the def hotplugStatus(): line). Done; xend.log attached. No xenstored.trace yet for the reason described above. Since the issue appears to be happening when xend is running below full speed but xenstored is not, I'm wondering if some mechanism to artificially slow xend (running it in valgrind?) would be effective in permitting the issue to be reproduced with tracing in place on both processes. I'll look into it, unless you have other suggestions. --------------040503070908090905040800 Content-Type: application/x-bzip2; name="xend.log.bz2" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="xend.log.bz2" QlpoOTFBWSZTWYnxXSoACBDfgEVQcv//8h/nn8q/7//6YArH3qhpYk0fWADNptIoAUAAAIYk hoQJjTUzSepoZBoDQMgBoaaaAOGmmRiMJpgIYBNMIwTEyGmRoaA4aaZGIwmmAhgE0wjBMTIa ZGhoBJqRI1Mpp4oaNHlNGhoyDIAaAAAARU0pp6aqe0RsqfqmIaPUHqPU0AHqZDQ0B6gARKEA IEyAE1TeU0TJo0aNGTQNBpobU6BbBeCEEAgs/bozrMzaBhmSJvBWmDS+XjW/lpisxwrAgXhO QcOxbeKrUjD04k18bAk0Aj2nE1zRFS8Ex0zyU9XpGoJBgQiERIBCEEJIoEXOwoSIJCLN+1Hi gHp9MqClJ3FaL/3kW5ww6b/9eg79uHTiuQtPd9X0AJa7RmJHxf05rVWHHa0UzLqqa9vTbNwQ A3ugTNjcSFY67wqB+F6cfbSoNRG6Ipyo20z/91lsaN1L8Lishh93Jr4amA9KD1Yl47hMdNMN sS8TbIhpnnZ7qm1aZGOvYnPa2G+GNPjHas1lPwzrgGpqP1vfAhEO8iSlFpRpduLoTCh9HA17 BKCesm/3djvcxwNJ1jBYay4rrVAmEYdmXCo1qpG4DUEIUMr5BhlsGIRB/ExMvs33lYUvQ9GH B+g4LUkKw7RUmdhuhWS6h0htfY4YVcC/oYFTyZiczKpxyJTLxxx0pYlEMj9UKfWkw/dZ8ZWw ZixRoMIY0IIMKBXGQwmk/hr3ZQUdOH6iT/NRZVQWQKIiRlDIFCMBkRiAxERESFIBUbMsqFiI k3yiiIyrFFpDPtb+UU6GBkTqTRe7AAALiGWBiBlQnTp6XF9mpbbF+wCnHm7576a/vlChr1Do Cx4MEY4mETV7smMh7RRyyCK1bmyCrsiByupWiSK6TddFtViab0KApBcdNrZVQ2YVa9SRcb/a 0MFbJDQcWUAvJC4CAjAQiRlculNOdNJz2E8tvtO8feAZAJkAnwT4HzO89DM6pO+OIaIUX5NR z1V8GfJZMcYZyG9chpMpZKKE4O2bdjA4dlGcfBiTCxmCxHARG+0c1sHNk39kdwDxWDaJ0Qlc kNqQ1WxVVVaM5BM9z5ewJdMLA70iC2uKV78hLJLmkQUNIlpt4rc4M8tppRmS3yXCuG4cDORk mHWvmxxXI1JYhf50E+8bX0Li5BLz9on32vATXIqmBnRqglT6MUWoOHyEO6OHmldjkVoUhGRi UaBcXX7qsqAl4V3g5hu11zLQL6UKVwAS1Zj1JUqWASWgFYOfGlbYAJkt9ts41LS+/gUTNJjy AToXkInDUoQHfnjuLuRXBb788ZFuiraGh0NyrcAmQbsjOts8q6Wx7amGFybc7oGa6VATtL8N V1l8hqcEy0jZJrzExys5Mb6JQKQM43FLpjWhxmUq4tMNrivPoGd2LS1gphPQwtdoe2W1L8ds ytSpomO1LskXFvjlGk4HPIvNM87z1dhV8IB4eJRT31kHzPJNUyFeuT4y/Xnx2xS+pIkKl6lL gEkWFHBI2RgnaHwHQbOJMPHfs7aRd9kA2EBJB/FRAg1ED50C/knGeQxrpkhFt50AsNBm/KTE oA+WADwtYVC+sJNf8zwwgfmHzPmJuFzVubkX2Sz/g2T/aAcbFuwpSAYRxCUE6hKIIUlwIPKb pA7ICAsWpxPB5O7JLEYERBAQOsec1BoNhsL7xy5IFW+GTC3hMoGiE44XJzdJLjarocNbjlR4 3bnJiDXoE/o5Zy0nIDSHyW7FI/OHWlHIM8bOnnzfxLFkPUyTv7Hq7gJlDdrJSJ0qlMvzNe+m 469dynwIL1oPTEMzXVZjJoFEjhSsBaUSGk9713JqByHZJilpqUUtgWL4jKpkz88aG9hBeQUt kcXF91UOEmvcfceoFhsBcwpZ3XVW4pCglAIAfYcz+KZWUX8yO+zI5DR7IxhA3UDqIBNJDUlA 8F+Ble4XjxaGWStIYagy4CjFtvHpErUCR8u3pfCtdi6D7hvTAZiULPmWpK21MBr8x2536Qlw w5n25gXy6hPGX4Z0D2sUX6fS43WSZeXxZ2bkzwDBeIdQB4tE+ZE+Bo5sCNRCgxqiwAN5d4Jr w3nbrThIerBkQKGw924OQcX8t6nX17wNgmNsJY8uV6VR+cFiEiySSBA9Dt+LvPQPZoQ9xU61 gZHf/MwxDzzC6IsXQ2/v6jcJp0oJQomHtJlsEZEJflbDCo06B4Gb29aUAtYbEU1TsseAGXNT ZPJAKer3x385gOTgJ1vdOjf9YlPSHDqD3gdXC4uJGRIKlEfPddID/Pqqd2OAkvs8Crl39KzE tkNBieHCYkhd7fYpjUqQ9UmrL+aDqE1N75BTERxXxUyl+InA3ffveSXaZGojz0B7bjff4jy+ nLseVJWlSUKlNyZI0EHaoTPPpFL/kbx93bldqmgBeMA5lcn33kib8ZRhSgUjT3+lUciwkQoU IC0SgFoGZvGNhEcm+6KQZ2mQVT7UWwJcN1MbgnJuahiSF0oMSCNqqadg5yB2gFQXwF8uFhxv CwQR8QnvoSMgMFZnfdgJQ4g/cB3e4lSLmNRGeUYDJ27G9bPqBQ4odtMvM2TGHPtJIgflxTA4 4hzNwdqYunUP1UNpX2KUA4j8R7g7vKHBE9hLhPMNlhIB4ib/BLFgGAQh8D5a773c6Jtiu6rA vNsU5/KQtxAmqXar5bx1OJs+Q1N6YhmkDgI+S+e5u7m4Puct4widhbC/9A6Bgkdff8Mqkxol 2VKNqDezxIWPatjHxx+Mx0yPZrrx691uJxKXB1Hdg54JkbnPw1C7QSXuxjUgGvlcYFflbgrM gmOMbHDVqHCMYfFTDdSx7UDHA8doXpjSlMUL4aAVOOKHclflOovNd2roOkS4DH1E51uBm7eu 7dEhUaOQjE0NTRMUy5ly9hf1f8g4cT02NdtB35nhqmzEiljToJ1mJkJp3ckdSIMDFr335YgV zqBU6VsSq1lMQyQCoX2gUSL62KjasISX04BfXUbLyOkd16nenz6BXASCQVhc0oU5CbjkaFOq UJUfaxcJqmScoF9wHo/4u5IpwoSET4rpUA== --------------040503070908090905040800 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------040503070908090905040800--