From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: domU and dom0 hung with Xen console interrupt binding showing in-flight=1, (---M) Date: Thu, 19 Aug 2010 16:48:45 +0100 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="B_3365081331_8287183" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Bruce Edge Cc: Xen-devel , Jan Beulich List-Id: xen-devel@lists.xenproject.org --B_3365081331_8287183 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable On 19/08/2010 14:42, "Bruce Edge" wrote: > Is there any more information that I can provide that would be helpful in > diagnosing the direct cause and the appropriate fix? > Possibly adding instrumentation or trace code to detect the trigger > conditions? > This is very=A0repeatable=A0on our target systems after a few hours of load. You can try the attached Xen patch which should clear out the in-flight interrupt after a short timeout. It might kick things back into action, and is probably a fix we should have in the tree anyhow. -- Keir --B_3365081331_8287183 Content-Type: application/octet-stream; name="00-irq-eoi-timer" Content-Disposition: attachment; filename="00-irq-eoi-timer" Content-Transfer-Encoding: base64 ZGlmZiAtciA3YjhiOTc2ZjUzNGUgeGVuL2FyY2gveDg2L2lycS5jCi0tLSBhL3hlbi9hcmNo L3g4Ni9pcnEuYwlUdWUgQXVnIDE3IDE5OjI3OjIwIDIwMTAgKzAxMDAKKysrIGIveGVuL2Fy Y2gveDg2L2lycS5jCVRodSBBdWcgMTkgMTY6NDU6NDcgMjAxMCArMDEwMApAQCAtNzg0LDEz ICs3ODQsNTUgQEAKICAgICBkZXNjLT5oYW5kbGVyLT5lbmFibGUoaXJxKTsKIH0KIAorc3Rh dGljIHZvaWQgc2V0X2VvaV9yZWFkeSh2b2lkICpkYXRhKTsKKwogc3RhdGljIHZvaWQgaXJx X2d1ZXN0X2VvaV90aW1lcl9mbih2b2lkICpkYXRhKQogewogICAgIHN0cnVjdCBpcnFfZGVz YyAqZGVzYyA9IGRhdGE7CisgICAgdW5zaWduZWQgaW50IGlycSA9IGRlc2MgLSBpcnFfZGVz YzsKKyAgICBpcnFfZ3Vlc3RfYWN0aW9uX3QgKmFjdGlvbjsKKyAgICBjcHVtYXNrX3QgY3B1 X2VvaV9tYXA7CiAgICAgdW5zaWduZWQgbG9uZyBmbGFnczsKIAogICAgIHNwaW5fbG9ja19p cnFzYXZlKCZkZXNjLT5sb2NrLCBmbGFncyk7Ci0gICAgX2lycV9ndWVzdF9lb2koZGVzYyk7 CisgICAgCisgICAgaWYgKCAhKGRlc2MtPnN0YXR1cyAmIElSUV9HVUVTVCkgKQorICAgICAg ICBnb3RvIG91dDsKKworICAgIGFjdGlvbiA9IChpcnFfZ3Vlc3RfYWN0aW9uX3QgKilkZXNj LT5hY3Rpb247CisKKyAgICBpZiAoIGFjdGlvbi0+YWNrX3R5cGUgIT0gQUNLVFlQRV9OT05F ICkKKyAgICB7CisgICAgICAgIHVuc2lnbmVkIGludCBpOworICAgICAgICBmb3IgKCBpID0g MDsgaSA8IGFjdGlvbi0+bnJfZ3Vlc3RzOyBpKysgKQorICAgICAgICB7CisgICAgICAgICAg ICBzdHJ1Y3QgZG9tYWluICpkID0gYWN0aW9uLT5ndWVzdFtpXTsKKyAgICAgICAgICAgIHVu c2lnbmVkIGludCBwaXJxID0gZG9tYWluX2lycV90b19waXJxKGQsIGlycSk7CisgICAgICAg ICAgICBpZiAoIHRlc3RfYW5kX2NsZWFyX2JpdChwaXJxLCBkLT5waXJxX21hc2spICkKKyAg ICAgICAgICAgICAgICBhY3Rpb24tPmluX2ZsaWdodC0tOworICAgICAgICB9CisgICAgfQor CisgICAgaWYgKCBhY3Rpb24tPmluX2ZsaWdodCAhPSAwICkKKyAgICAgICAgZ290byBvdXQ7 CisKKyAgICBzd2l0Y2ggKCBhY3Rpb24tPmFja190eXBlICkKKyAgICB7CisgICAgY2FzZSBB Q0tUWVBFX1VOTUFTSzoKKyAgICAgICAgZGVzYy0+aGFuZGxlci0+ZW5kKGlycSk7CisgICAg ICAgIGJyZWFrOworICAgIGNhc2UgQUNLVFlQRV9FT0k6CisgICAgICAgIGNwdV9lb2lfbWFw ID0gYWN0aW9uLT5jcHVfZW9pX21hcDsKKyAgICAgICAgc3Bpbl91bmxvY2tfaXJxKCZkZXNj LT5sb2NrKTsKKyAgICAgICAgb25fc2VsZWN0ZWRfY3B1cygmY3B1X2VvaV9tYXAsIHNldF9l b2lfcmVhZHksIGRlc2MsIDApOworICAgICAgICBzcGluX2xvY2tfaXJxKCZkZXNjLT5sb2Nr KTsKKyAgICAgICAgYnJlYWs7CisgICAgY2FzZSBBQ0tUWVBFX05PTkU6CisgICAgICAgIF9p cnFfZ3Vlc3RfZW9pKGRlc2MpOworICAgICAgICBicmVhazsKKyAgICB9CisKKyBvdXQ6CiAg ICAgc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmZGVzYy0+bG9jaywgZmxhZ3MpOwogfQogCkBA IC04NDcsOSArODg5LDExIEBACiAgICAgICAgIH0KICAgICB9CiAKLSAgICBpZiAoIGFscmVh ZHlfcGVuZGluZyA9PSBhY3Rpb24tPm5yX2d1ZXN0cyApCisgICAgc3RvcF90aW1lcigmaXJx X2d1ZXN0X2VvaV90aW1lcltpcnFdKTsKKworICAgIGlmICggKGFjdGlvbi0+YWNrX3R5cGUg PT0gQUNLVFlQRV9OT05FKSAmJgorICAgICAgICAgKGFscmVhZHlfcGVuZGluZyA9PSBhY3Rp b24tPm5yX2d1ZXN0cykgKQogICAgIHsKLSAgICAgICAgc3RvcF90aW1lcigmaXJxX2d1ZXN0 X2VvaV90aW1lcltpcnFdKTsKICAgICAgICAgZGVzYy0+aGFuZGxlci0+ZGlzYWJsZShpcnEp OwogICAgICAgICBkZXNjLT5zdGF0dXMgfD0gSVJRX0dVRVNUX0VPSV9QRU5ESU5HOwogICAg ICAgICBmb3IgKCBpID0gMDsgaSA8IGFscmVhZHlfcGVuZGluZzsgKytpICkKQEAgLTg2NSwx MCArOTA5LDExIEBACiAgICAgICAgICAgICAgKiAtIHNraXAgdGhlIHRpbWVyIHNldHVwIGJl bG93LgogICAgICAgICAgICAgICovCiAgICAgICAgIH0KLSAgICAgICAgaW5pdF90aW1lcigm aXJxX2d1ZXN0X2VvaV90aW1lcltpcnFdLAotICAgICAgICAgICAgICAgICAgIGlycV9ndWVz dF9lb2lfdGltZXJfZm4sIGRlc2MsIHNtcF9wcm9jZXNzb3JfaWQoKSk7Ci0gICAgICAgIHNl dF90aW1lcigmaXJxX2d1ZXN0X2VvaV90aW1lcltpcnFdLCBOT1coKSArIE1JTExJU0VDUygx KSk7CiAgICAgfQorCisgICAgaW5pdF90aW1lcigmaXJxX2d1ZXN0X2VvaV90aW1lcltpcnFd LAorICAgICAgICAgICAgICAgaXJxX2d1ZXN0X2VvaV90aW1lcl9mbiwgZGVzYywgc21wX3By b2Nlc3Nvcl9pZCgpKTsKKyAgICBzZXRfdGltZXIoJmlycV9ndWVzdF9lb2lfdGltZXJbaXJx XSwgTk9XKCkgKyBNSUxMSVNFQ1MoMSkpOwogfQogCiAvKgpAQCAtMTMwNiw5ICsxMzUxLDcg QEAKICAgICBCVUdfT04oIWNwdXNfZW1wdHkoYWN0aW9uLT5jcHVfZW9pX21hcCkpOwogCiAg ICAgZGVzYy0+YWN0aW9uID0gTlVMTDsKLSAgICBkZXNjLT5zdGF0dXMgJj0gfklSUV9HVUVT VDsKLSAgICBkZXNjLT5zdGF0dXMgJj0gfklSUV9JTlBST0dSRVNTOwotICAgIGtpbGxfdGlt ZXIoJmlycV9ndWVzdF9lb2lfdGltZXJbaXJxXSk7CisgICAgZGVzYy0+c3RhdHVzICY9IH4o SVJRX0dVRVNUfElSUV9HVUVTVF9FT0lfUEVORElOR3xJUlFfSU5QUk9HUkVTUyk7CiAgICAg ZGVzYy0+aGFuZGxlci0+c2h1dGRvd24oaXJxKTsKIAogICAgIC8qIENhbGxlciBmcmVlcyB0 aGUgb2xkIGd1ZXN0IGRlc2NyaXB0b3IgYmxvY2suICovCkBAIC0xMzQyLDcgKzEzODUsMTAg QEAKICAgICBzcGluX3VubG9ja19pcnEoJmRlc2MtPmxvY2spOwogCiAgICAgaWYgKCBvbGRh Y3Rpb24gIT0gTlVMTCApCisgICAgeworICAgICAgICBraWxsX3RpbWVyKCZpcnFfZ3Vlc3Rf ZW9pX3RpbWVyW2Rlc2MtaXJxX2Rlc2NdKTsKICAgICAgICAgeGZyZWUob2xkYWN0aW9uKTsK KyAgICB9CiB9CiAKIHN0YXRpYyBpbnQgcGlycV9ndWVzdF9mb3JjZV91bmJpbmQoc3RydWN0 IGRvbWFpbiAqZCwgaW50IGlycSkKQEAgLTEzODAsNyArMTQyNiwxMCBAQAogICAgIHNwaW5f dW5sb2NrX2lycSgmZGVzYy0+bG9jayk7CiAKICAgICBpZiAoIG9sZGFjdGlvbiAhPSBOVUxM ICkKKyAgICB7CisgICAgICAgIGtpbGxfdGltZXIoJmlycV9ndWVzdF9lb2lfdGltZXJbZGVz Yy1pcnFfZGVzY10pOwogICAgICAgICB4ZnJlZShvbGRhY3Rpb24pOworICAgIH0KIAogICAg IHJldHVybiBib3VuZDsKIH0K --B_3365081331_8287183 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --B_3365081331_8287183--