From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steven Haigh Subject: Re: HVMs terminating as (null) Date: Sun, 24 Nov 2013 07:09:35 +1100 Message-ID: <52910B7F.6030503@crc.id.au> References: <5290D480.1000804@crc.id.au> <20131123192746.GA21689@aepfle.de> <52910427.5020507@crc.id.au> <52910853.1090708@crc.id.au> <52910A2F.50806@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8314866370928078030==" Return-path: In-Reply-To: <52910A2F.50806@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============8314866370928078030== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="c4jowk1BMcCQ6FNwA8C8fMisEsV72vRHO" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --c4jowk1BMcCQ6FNwA8C8fMisEsV72vRHO Content-Type: multipart/mixed; boundary="------------040800000308020901070809" This is a multi-part message in MIME format. --------------040800000308020901070809 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 24/11/13 07:03, Andrew Cooper wrote: > On 23/11/13 19:56, Steven Haigh wrote: >> On 24/11/13 06:38, Steven Haigh wrote: >>> On 24/11/13 06:27, Olaf Hering wrote: >>>> On Sun, Nov 24, Steven Haigh wrote: >>>> >>>>> Running Xen 4.2.3 with all the current XSA fixes. >>>> >>>> How exactly did you start the guests? >>> >>> The DomUs were started with: xl create /etc/xen/ >>> >>>> Does 'ps faxu' show qemu processes for the listed domain_ids? >>>> What is the 'xenstore-ls -f | sort' output? >>> >>> I'll have to check this when I manage to reproduce it. So far, I have= >>> been unable to get a reliable way to reproduce it. I managed to get a= >>> system to do it every time a HVM DomU was shutdown OR restarted - but= >>> after a reboot of the Dom0 I can't get it into that state again. >>> >>> As soon as I can get a system in this state again, I'll leave it to s= ee >>> what information I can extract. >> >> Ha! As always, as soon as I send this, I notice its happened on a Dom0= =2E >> >> # xl list >> Name ID Mem VCPUs State >> Time(s) >> Domain-0 0 1579 2 r----- >> 2731.3 >> planner.vm 1 1013 1 -b---- >> 189.3 >> (null) 2 0 1 --psrd >> 301.1 >> tracker.vm 3 1013 2 -b---- >> 834.4 >> >> Attached is the output of: >> # xl debug-keys q >> # xl dmesg > xen-dmesg.log >> # gzip xen-dmesg.log >=20 > Ok - from dmesg. >=20 > (XEN) General information for domain 2: > (XEN) refcnt=3D1 dying=3D2 pause_count=3D2 > (XEN) nr_pages=3D2 xenheap_pages=3D0 shared_pages=3D0 paged_pages=3D= 0 > dirty_cpus=3D{} max_pages=3D262400 > (XEN) handle=3Def58ef1a-784d-4e59-8079-42bdee87f219 vm_assist=3D000= 00000 > (XEN) paging assistance: hap refcounts translate external > ... > (XEN) Memory pages belonging to domain 2: > (XEN) DomPage 00000000000866e0: caf=3D00000001, taf=3D0000000000000= 000 > (XEN) DomPage 00000000000866e1: caf=3D00000001, taf=3D0000000000000= 000 > (XEN) PoD entries=3D0 cachesize=3D0 >=20 >=20 > So there are indeed two outstanding pages causing this domain to become= > a zombie. They are normal pages, with 1 outstanding ref. >=20 > Can you collect "xl debug-keys g" as well? Sure - attached. --=20 Steven Haigh Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299 --------------040800000308020901070809 Content-Type: application/gzip; name="debug-keys-g.log.gz" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="debug-keys-g.log.gz" H4sICC0LkVIAA2RlYnVnLWtleXMtZy5sb2cA1Vttb6O4Fv6+v8Kr+2E6q5KxDSEkUldK89JG 27TZJNM7q9GocsC0qAkwQDrtrua/32PzEkJomjLodoeVtmCf8/jYPufxOQzBGB0FPOTBA7fe /3L0aXD5HiEcXyT5i5GSNVEjaTpah2yx5KlOtzcZddB01p8I0SHWCD6GO6oBPEVodDkfXGzL fpr156h3qlM8IIaQbfZAlsSiCPXPm815D8FFcIti3Ebj2XAeP2OibmMN4Y/AIsN+W2INxbha Jax+YhcxsLBLb/fUcrviC4uOCwSoTUJIa8eumZyj0dfkemh4q787GfViu4fxGvR6cr2q2D1L 7R72JBbRBvvsJgmWGu/rNta4NzyTdpPELvXAvYGrXViD88kg3mcyiPdGNfZjdcejRmplYY7d 2fDX2GfiOeKucF+1ZMHSSc6Hs7G4G2pUw4WNHnenMdhpDHb6jGEpltzo9CnFmj2FEV+haXfc QarRVsen6EhtG3pL1e9PU6G+t2KOi+4485HjOpHDlk7IrS1zVPpB1z7FLsMsC4IyRCsnXLHI vAMl6ID9VZC5AG/iGv6QBqdssIR7rUPHvQWcBHYSeCaAeAH6D0Z6pxl72wMPQsdzESW7YuQw Me0wseYesdGVaP+Mv3QQ8x3zxrEQTCAVU+lxtgT40eamnOgxOpuNEFaomoAMXGAgMWU5xsqz eAc2eskicJ+Pci1gzz5cye4w0YnbQ/OOW+slDyBsxsAcAbecCM3SVnRkypZs+3jEzYhbSCVt 2iAqxM7538hPp9pIJyV2FtBXfOUFTyi8YwE8pr2THmzypRcl2yRDzIZ1CvntiruRZFi0WMOM saLaGWTEl+h6rljI8VarNcwoXPu+FwhrfHbLUej8zcMO0u5PG8/p4Ao69FU6M9fzfNTz3Cjw lsiFSXKxN9wqkYVYwCL20ISFYXQXeOvbuxdU/lzzNdgxch8gbiwWCR/ZryFug2DtR2jKV8z3 xYrv15jBbsEYg8kcRUImLJUHb3pwgmgt4je2I5FIj04lnl/sjFO+ZI9Z38amILPJcsK8+iAG g+YgdrjB1Qh9cyIgAE/GCTPvb7ylhTz311Tlsnt6Mbo8g5BSZByMpn+GmTW/Jw4vdLq9P8A1 ozsvHW0CoQIeuEKRswKnd0JEtIZKDOHcgrVTelouPZMJa0zPDb0lR4GEtBHR0R9O5g/X408Q TpnXMOuBuSbc2JxFa4jlzmaNpKHj8egKMVPEUGFRC4LzyVQEk+V923QMHiPuWoA+Ec45j3fs CHbv/UbmOgZVNpw0siDSHNsBnkFH15NRf1caXY5Hm8bxbJpshpJYunAi2LyNxEcXphYFjtyu szXcJ33n12NxYI36YdGLZA+sVsF3ZPM5C6xv4ImoG4ZOGMUTFMt9dN6dvEdWwkRbKt1JMUSP ER2fJjKnMsKAdnykod7kY+occzBVAMN80TdxyFjeLVIURcjAgeHds6eGvCe5e5q7V5P7BO+3 335DF1fdvvDF/tW4O7oE4oHGdKk+cRehex64fNlBuqbAQh6jZbg4Brda+SzKzq04goqSk+4g EffF0QDnAkmT09/Fg2024SH17PO/ZqNe9wKNB+Or6V+oO512L88G48HlvLM1ChO+3eigTc5L 4sMGK79nKW+aBR+1KYZsQS52iCIPLXgMIIIj8yRxCjRQwFYQ2/edTeLcpC1jG7dl27ae2Xw9 ms4/7jf5wmPC5dOlsZPLIKnJm5at5SjaVBTLKWqWZm5sQpO7p1AZM8hAwOlziiBmbSs2NaNt sUxxFrEggrTF9sTiorwY21Fk2kLLRhSeHDNwp6C4KCouWG6Opx7wdRgBQxZHXOyMuFjkFOdX 8+5FJ8vzMjG8s6rNuC1VhK0RG9XvTwezWX5VDc0kPBWTfnbHIJljj85qvYKj9ToXhjMzWC8W IhCHAedxHtmwPJdvZRXAS0vvW5pZAFt+RXB08vBOEHvII8QggXjUhJdK70yVZ5HVALe5XfIH 4TGDIPCA+phrQcgHLoyamiGZKycJicqdpJ0pOLeydFYO+HgpQOr4Ir7hCAk4pGRf104o1a/P uump0cjxxIwHYkqO668jEb/AFxBe0ZPP0bvefHqhsHdyflweTTLWQjgGZSIsVOAZhktHFgtn Idqi96cyu06WKR0wCpgfNswObbZpx8JpMs4iyNt9wbD/nQqax1sXp8gOvBUsatYEYWFouCUG z7XiuPWnGQxCfmew+FJ3BxPybzLYu6/vIL+G8xZwwT+s9SpOl+IxBa2gI9f7doIfT/VWp69B gT1Ue6lDnHGXB9K/RHoT52ki1U7UcUaocAXcNt3oREXWE4xwIuJnHfIb01tDK84JusGNDK0T DWua1kSP3BX1XNKoy3SfW8ljHIabJ8ghoqcb01+HJ/9QRf0uyCCFo22trbdou5kb7A4ibMlP 0nVRSv6XXuhhdcNkspCKWzmg7GTHiD/6kMmEopakGpAGxKWVhv+UuWALjyC/4UvPlRkHbFDZ gokceAIpXojQP1CNERuSDZgS/NE0RYcqTadKG54YVcBVoRWbimnDuQ0HhiL4EX3Po6VJcQho BGo7QGspUNF/Lww5jqlPDGnzhQ0DiLqQwIMVP3DxULgylEQ7PrlfnCK4tjyGIB8VXOMhIS+F ndBfsqecKNCQFM35MKXYbHeQyewTc9u9oayNoLWlbbe+DGeUwpFSOPIyXKteOL02OGwusG3W tnaQzRm8Cpw4oLfYQxx4JuR6C0gvwme5RKjhjkiOKfoMR/7J/AvyveUSCGDtC/UbH4oWdILE y4ykZcXCe9myzRHfEdzcMNsWB9rTyT8QaM18SGyxVPJkL9ltuMVZlx7y4aj1LMeMi7yCsUQa q8bGDqsaq1Y3lhxg7AtsTn6MzeE4a2pagc1V7XA6z3M51amG8S6Pt82W0bKoqeBWy1Y0SLEV Rg2iLFSdMLVl6kYZj+OtJZSEFQuIuroD2L6cr5hbCBR1eyPNEge1G0Jpz4HxgVxdWLqDWJ48 y/LPEvazJH0w75KKvDvx+lA+Q90t98OECoWL8ndrj8voqsUW5dxcjf0Ark5ubjHLrheO18nN tFmjdYRTtcaTw2BcVeuFo/XCkXrhcK1wtL6dlXA1+p2As+qFK88JKsMt6oVj9cLVx3cSrj6+ k3D15aISrl5Goc164bR64erlO1ov39F6+Y7Wy3ekXr4j9fIdqZfvSL18R+rlO1Iv35F6+Y5U 4bvD60dSXj/iHyzJdisy9fUFWXnNcYxUJF8ShznRwwpN+lazogfMihw+qxcqUlpSkZKkIqVb Vubt2lSkhWIU11uLcrtpcJswpWVolqLxZlsxcKutaHRhcW60bEraP1iLVq1A6VtUoLSsAs1z gK7z5OzZjn1cuF6GIa+EeanIPZxmaDnN/Oibn1cEJHn2JVXtNPNmszqEZl4xqxdoRn3jF19Y IYfwDW6ZlOnEUFqUq4pm65pi2IapsKZKbJOQlmZbb/7uS30L5lH/r+++sAF1b3nFUP4y/KXc CODKK4aKcCqu1ToV12gdU3WrzgKEN406C1+NsHK4amsHcOV1dGW48jq6Mlx5HV0Zrk6/A7g6 owLgyuvoynDlblwV7pn31ZXhyuvoynDldXRluNf8W+IBcOV1dGW48jq6Mtxr/g37ALjyOroy XL18Z9XLd1a9fGfVy3dWvXxn1ct3Vr1898yxXQmuTfVn3htWhquR7wRcjXwn4Krw3eFFqVpe lJIfLN/Iv/Pl14++0sP/lmldepFjiyoT3cqPRnEHoyP5SSokr+Ljf6Qdiw9xoecD/Pf+WTVS UBM/GXxZj+wMt1FTyD694niH6dGK49GK46kVx1MPHC/5oYsdMPFlLUaKgmbsIdeSyN26UcQW N+sQeOHGDxw3ugHnRJ/RPX9C726z7zNzHoPE7wfkhZgZOQ9881zojl8yZM8Jxmco6b8g8UH9 EigCrWw3VvOd5EZcWfdt2i+9PLU6YG6kyO/HJdUEUIRHPGEc+ZE3Ro1GA7leaqLUiL84T6vq n21KYuOzHyR+hucvKTIWP42gLROrm49t6WZgnLXKmxSB6LRtbGFgjWtqc4NBihg6ld2kncdo 72Lo+zH0AoZqqEUM2sL7MDTZnWH8NHtIt/YQZo7p9szlC+S9M5fdW6tH8S7G/tXjP+fqqXsj ABu4bb4yAgguYqgW3bv+TVpcf9rUdzHofgz6UgQYmGh77Yi7M4xnePwLEj8y+eV/c32XZwBA AAA= --------------040800000308020901070809-- --c4jowk1BMcCQ6FNwA8C8fMisEsV72vRHO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSkQt/AAoJEEGvNdV6fTHclDUP/iMbqs8ZBmNDDRRs7jZmDIoJ Vq10kFXjFd52h6j7/qPm+IoPALGJdFI/+Cune9m0lgyeOAU+MIBkFsJ+Vqpaq0KI W6RYhaxt8qb0ZuK3LYpqs5ksV7j2cCXOQ959zvYV3RuMymYFzbWAJ6YS8wYtCzqO 3uf8DvB0++00/JDRS1QFh70i9dZArxpizNGNwRbOjsD8UJD1b3H9F8iidWQi0c7K FDAVxMKF6lFhjTbyu2CWuG76PX/kq/lr3OfOUnK8sLpHvd8oCpdMfNQgGgPANglh 8LhxGSD/035c/iONMCW6bAEgGMFR8+Fma5aGWDNrYPAO7k5+owblfdqtDCHU6fuK SHdRPhT6epVSvxS58GMuaGhayACRYW5KPyeA1Zr5diWIuVQ3KXr3siPOHK+CRYrm DxgNfuFiF9G7p3ulOMmy8JDJ4dQpKzWRTSbRVKKauMXQuwXVgUEjflVi59vCzldm VVZDQ32rqftzHHn67yocoExiJzOcWlvG+//VfRREK9YydFiDz0SsC1jLLf0C9kYT RxhnpC3sI1yWBY9x4sqU6Ybh2ZZkuZmc/wdcV1uNfPggZXYDpAJxMtI8d3kme+MM NXSBiyleEatvABR/V0gGtk9EoqcZFOyyhxqelp81KYXWMcm8im7FyJhKXwZJy9Lv jXn0uIFc+d7iPfvY7QDP =/CA2 -----END PGP SIGNATURE----- --c4jowk1BMcCQ6FNwA8C8fMisEsV72vRHO-- --===============8314866370928078030== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============8314866370928078030==--