From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Marczykowski Subject: Re: High CPU temp, suspend problem - xen 4.1.5-pre, linux 3.7.x Date: Thu, 28 Mar 2013 18:44:38 +0100 Message-ID: <51548186.9050301@invisiblethingslab.com> References: <5151D0A9.7070100@invisiblethingslab.com> <5151D49C.2000809@citrix.com> <5151DE1C.1020307@invisiblethingslab.com> <5151E0D5.3050707@citrix.com> <5151E72D.30205@invisiblethingslab.com> <5151EE0B.9030605@citrix.com> <5152C16E02000078000C8CB8@nat28.tlf.novell.com> <515302C3.3000607@invisiblethingslab.com> <5153063C.8020307@citrix.com> <5153079E.9030802@citrix.com> <20130327154755.GC5759@phenom.dumpdata.com> <515324A9.2010406@citrix.com> <51532927.8040908@invisiblethingslab.com> <515480BB.6070309@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============9129446383700323781==" Return-path: In-Reply-To: <515480BB.6070309@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper Cc: "xen-devel@lists.xen.org" , Jan Beulich , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============9129446383700323781== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig82CD6EC27092668A33FBA187" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig82CD6EC27092668A33FBA187 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 28.03.2013 18:41, Andrew Cooper wrote: > On 27/03/2013 17:15, Marek Marczykowski wrote: >> On 27.03.2013 17:56, Andrew Cooper wrote: >>> On 27/03/2013 15:47, Konrad Rzeszutek Wilk wrote: >>>> On Wed, Mar 27, 2013 at 02:52:14PM +0000, Andrew Cooper wrote: >>>>> On 27/03/2013 14:46, Andrew Cooper wrote: >>>>>> On 27/03/2013 14:31, Marek Marczykowski wrote: >>>>>>> On 27.03.2013 09:52, Jan Beulich wrote: >>>>>>>>>>> On 26.03.13 at 19:50, Andrew Cooper wrote: >>>>>>>>> So vector e9 doesn't appear to be programmed in anywhere. >>>>>>>> Quite obviously, as it's the 8259A vector for IRQ 9. The questio= n >>>>>>>> really is why an IRQ appears on that vector in the first place. = The >>>>>>>> 8259A resume code _should_ leave all IRQs masked on a fully >>>>>>>> IO-APIC system (see my question raised yesterday). >>>>>>>> >>>>>>>> And that's also why I suggested, for an experiment, to fiddle wi= th >>>>>>>> the loop exit condition to exclude legacy vectors (which wouldn'= t >>>>>>>> be a final solution, but would at least tell us whether the dire= ction >>>>>>>> is the right one). In the end, besides understanding why an >>>>>>>> interrupt on vector E9 gets raised at all, we may also need to >>>>>>>> tweak the IRQ migration logic to not do anything on legacy IRQs,= >>>>>>>> but that would need to happen earlier than in >>>>>>>> smp_irq_move_cleanup_interrupt(). Considering that 4.3 >>>>>>>> apparently doesn't have this problem, we may need to go hunt for= >>>>>>>> a change that isn't directly connected to this, yet deals with t= he >>>>>>>> problem as a side effect (at least I don't recall any particular= fix >>>>>>>> since 4.2). One aspect here is the double mapping of legacy IRQs= >>>>>>>> (once to their IO-APIC vector, and once to their legacy vector, >>>>>>>> i.e. vector_irq[] having two entries pointing to the same IRQ). >>>>>>> So tried change loop condition to LAST_DYNAMIC_VECTOR and it does= n't hit that >>>>>>> BUG/ASSERT. But still it doesn't work - only CPU0 used by schedul= er, also some >>>>>>> errors from dom0 kernel, and errors about PCI devices used by dom= U(1). >>>>>>> >>>>>>> Messages from resume (different tries): >>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-last-dynamic-vec= tor.log >>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-last-dynamic-vec= tor2.log >>>>>>> >>>>>>> Also one time I've got fatal page fault error, earlier in resume = (it isn't >>>>>>> deterministic): >>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-resume-page-faul= t.log >>>>>>> >>>>>> This pagefault is a Null structure pointer dereference, likely the= >>>>>> scheduling data. At a first glance, it looks related to the asser= tion >>>>>> failures I have been seeing sporadically in testing, but unable to= >>>>>> reproduce reliably. There seems to be something quite dodgy with >>>>>> interaction of vcpu_wake and scheduling loops. >>>>>> >>>>>> The other logs indicate that dom0 appears to have a domain id of 1= , >>>>>> which is sure to cause problems. >>>>> Actually - ignore this >>>>> >>>>> >From the log, >>>>> >>>>> (XEN) physdev.c:153: dom0: can't create irq for msi! >>>>> [ 113.637037] xhci_hcd 0000:03:00.0: xen map irq failed -22 for 32= 752 >>>>> domain >>>>> (XEN) physdev.c:153: dom0: can't create irq for msi! >>>>> [ 113.657911] xhci_hcd 0000:03:00.0: xen map irq failed -22 for 32= 752 >>>>> domain >>>>> >>>>> and later >>>>> >>>>> (XEN) physdev.c:153: dom1: can't create irq for msi! >>>>> [ 121.909814] pciback 0000:00:19.0: xen map irq failed -22 for 1 d= omain >>>>> [ 121.954080] error enable msi for guest 1 status ffffffea >>>>> (XEN) physdev.c:153: dom1: can't create irq for msi! >>>>> [ 122.035355] pciback 0000:00:19.0: xen map irq failed -22 for 1 d= omain >>>>> [ 122.044421] error enable msi for guest 1 status ffffffea >>>>> >>>>> I think that there is a separate bug where mapped irqs are not unma= pped >>>>> on the suspend path. >>>> You thinking this is a Linux (xen irq machinery) issue? Meaning it s= hould >>>> end up calling PHYSDEV_unmap_pirq as part of the suspend process? >>> I am not sure. Without looking at the code, I am only speculating. >>> >>> Beyond that, the main question is about the expected behaviour. Do w= e >>> expect dom0/U to unmap its irqs and remap them after resume? What do= we >>> expect from domains which are unaware of the host sleep action? >> BTW this is the case: domain 1 isn't fully aware of sleep. It have som= e PCI >> devices assigned. The only action taken there before suspend is shutdo= wn >> network interfaces (without this system hanged during suspend). >> >=20 > What do you mean here by shutting down the network interfaces? Are the > devices being assigned back to dom0? =20 No, just simple ip link set eth0 down. Seems to be enough to suspend succ= eed, at least on most hardware... > Ifso, is dom0 assigning them back > to domU before the domU driver tries to set itself up? --=20 Best Regards / Pozdrawiam, Marek Marczykowski Invisible Things Lab --------------enig82CD6EC27092668A33FBA187 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQEcBAEBAgAGBQJRVIGHAAoJENuP0xzK19csiS8H/ieHCqFcfHKh7/EklKFmd5NG x9QzKbinfNbmlypffScSYmNxBUdRf3yxdGUxZM0fdWhn2oIgFbWUrpnugYUb/gfv +uy5YUm/q7iEp30V83aQ6yAJQqwvMeLinvhZHtddZUUqtdMGXyWZunKXe5bQf0YD 5TvvJQFBg7Vb9BUkBpZycotWyOjPbnyA3zLE9JC10msL+q4Fi/HLRYDLHFFOf+zv UBqHCGpuOpEct8GdHXIkjLQ790ndjudkZFquVp6XbXkK6ZvcICf6ckKIrA9oCyTj P7eY5PAbWaB72aiGBQKIZuoaWVLt0ox02FPxzUBUpIMzy1xMj73PbjSRajaMHsQ= =kxyB -----END PGP SIGNATURE----- --------------enig82CD6EC27092668A33FBA187-- --===============9129446383700323781== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============9129446383700323781==--