From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Marczykowski Subject: Re: High CPU temp, suspend problem - xen 4.1.5-pre, linux 3.7.x Date: Fri, 29 Mar 2013 01:26:41 +0100 Message-ID: <5154DFC1.4080607@invisiblethingslab.com> References: <5151D0A9.7070100@invisiblethingslab.com> <5151D49C.2000809@citrix.com> <5151DE1C.1020307@invisiblethingslab.com> <5151E0D5.3050707@citrix.com> <5151E72D.30205@invisiblethingslab.com> <5151EE0B.9030605@citrix.com> <5152C16E02000078000C8CB8@nat28.tlf.novell.com> <515302C3.3000607@invisiblethingslab.com> <5153063C.8020307@citrix.com> <5153079E.9030802@citrix.com> <20130327154755.GC5759@phenom.dumpdata.com> <515324A9.2010406@citrix.com> <51532927.8040908@invisiblethingslab.com> <515480BB.6070309@citrix.com> <51548186.9050301@invisiblethingslab.com> <515482E1.4060008@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1290809415649434967==" Return-path: In-Reply-To: <515482E1.4060008@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper Cc: "xen-devel@lists.xen.org" , Jan Beulich , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============1290809415649434967== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCB58CF515F025B7FD99FA62E" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCB58CF515F025B7FD99FA62E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 28.03.2013 18:50, Andrew Cooper wrote: > On 28/03/2013 17:44, Marek Marczykowski wrote: >> On 28.03.2013 18:41, Andrew Cooper wrote: >>> On 27/03/2013 17:15, Marek Marczykowski wrote: >>>> On 27.03.2013 17:56, Andrew Cooper wrote: >>>>> On 27/03/2013 15:47, Konrad Rzeszutek Wilk wrote: >>>>>> On Wed, Mar 27, 2013 at 02:52:14PM +0000, Andrew Cooper wrote: >>>>>>> On 27/03/2013 14:46, Andrew Cooper wrote: >>>>>>>> On 27/03/2013 14:31, Marek Marczykowski wrote: >>>>>>>>> On 27.03.2013 09:52, Jan Beulich wrote: >>>>>>>>>>>>> On 26.03.13 at 19:50, Andrew Cooper wrote: >>>>>>>>>>> So vector e9 doesn't appear to be programmed in anywhere. >>>>>>>>>> Quite obviously, as it's the 8259A vector for IRQ 9. The quest= ion >>>>>>>>>> really is why an IRQ appears on that vector in the first place= =2E The >>>>>>>>>> 8259A resume code _should_ leave all IRQs masked on a fully >>>>>>>>>> IO-APIC system (see my question raised yesterday). >>>>>>>>>> >>>>>>>>>> And that's also why I suggested, for an experiment, to fiddle = with >>>>>>>>>> the loop exit condition to exclude legacy vectors (which would= n't >>>>>>>>>> be a final solution, but would at least tell us whether the di= rection >>>>>>>>>> is the right one). In the end, besides understanding why an >>>>>>>>>> interrupt on vector E9 gets raised at all, we may also need to= >>>>>>>>>> tweak the IRQ migration logic to not do anything on legacy IRQ= s, >>>>>>>>>> but that would need to happen earlier than in >>>>>>>>>> smp_irq_move_cleanup_interrupt(). Considering that 4.3 >>>>>>>>>> apparently doesn't have this problem, we may need to go hunt f= or >>>>>>>>>> a change that isn't directly connected to this, yet deals with= the >>>>>>>>>> problem as a side effect (at least I don't recall any particul= ar fix >>>>>>>>>> since 4.2). One aspect here is the double mapping of legacy IR= Qs >>>>>>>>>> (once to their IO-APIC vector, and once to their legacy vector= , >>>>>>>>>> i.e. vector_irq[] having two entries pointing to the same IRQ)= =2E >>>>>>>>> So tried change loop condition to LAST_DYNAMIC_VECTOR and it do= esn't hit that >>>>>>>>> BUG/ASSERT. But still it doesn't work - only CPU0 used by sched= uler, also some >>>>>>>>> errors from dom0 kernel, and errors about PCI devices used by d= omU(1). >>>>>>>>> >>>>>>>>> Messages from resume (different tries): >>>>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-last-dynamic-v= ector.log >>>>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-last-dynamic-v= ector2.log >>>>>>>>> >>>>>>>>> Also one time I've got fatal page fault error, earlier in resum= e (it isn't >>>>>>>>> deterministic): >>>>>>>>> http://duch.mimuw.edu.pl/~marmarek/qubes/xen-4.1-resume-page-fa= ult.log >>>>>>>>> >>>>>>>> This pagefault is a Null structure pointer dereference, likely t= he >>>>>>>> scheduling data. At a first glance, it looks related to the ass= ertion >>>>>>>> failures I have been seeing sporadically in testing, but unable = to >>>>>>>> reproduce reliably. There seems to be something quite dodgy wit= h >>>>>>>> interaction of vcpu_wake and scheduling loops. >>>>>>>> >>>>>>>> The other logs indicate that dom0 appears to have a domain id of= 1, >>>>>>>> which is sure to cause problems. >>>>>>> Actually - ignore this >>>>>>> >>>>>>> >From the log, >>>>>>> >>>>>>> (XEN) physdev.c:153: dom0: can't create irq for msi! >>>>>>> [ 113.637037] xhci_hcd 0000:03:00.0: xen map irq failed -22 for = 32752 >>>>>>> domain >>>>>>> (XEN) physdev.c:153: dom0: can't create irq for msi! >>>>>>> [ 113.657911] xhci_hcd 0000:03:00.0: xen map irq failed -22 for = 32752 >>>>>>> domain >>>>>>> >>>>>>> and later >>>>>>> >>>>>>> (XEN) physdev.c:153: dom1: can't create irq for msi! >>>>>>> [ 121.909814] pciback 0000:00:19.0: xen map irq failed -22 for 1= domain >>>>>>> [ 121.954080] error enable msi for guest 1 status ffffffea >>>>>>> (XEN) physdev.c:153: dom1: can't create irq for msi! >>>>>>> [ 122.035355] pciback 0000:00:19.0: xen map irq failed -22 for 1= domain >>>>>>> [ 122.044421] error enable msi for guest 1 status ffffffea >>>>>>> >>>>>>> I think that there is a separate bug where mapped irqs are not un= mapped >>>>>>> on the suspend path. >>>>>> You thinking this is a Linux (xen irq machinery) issue? Meaning it= should >>>>>> end up calling PHYSDEV_unmap_pirq as part of the suspend process? >>>>> I am not sure. Without looking at the code, I am only speculating.= >>>>> >>>>> Beyond that, the main question is about the expected behaviour. Do= we >>>>> expect dom0/U to unmap its irqs and remap them after resume? What = do we >>>>> expect from domains which are unaware of the host sleep action? >>>> BTW this is the case: domain 1 isn't fully aware of sleep. It have s= ome PCI >>>> devices assigned. The only action taken there before suspend is shut= down >>>> network interfaces (without this system hanged during suspend). >>>> >>> What do you mean here by shutting down the network interfaces? Are th= e >>> devices being assigned back to dom0? =20 >> No, just simple ip link set eth0 down. Seems to be enough to suspend s= ucceed, >> at least on most hardware... >=20 > In which case repeat map_pirq hypercalls will fail with -EINVAL because= > the pirq is already set up. It is probably worth putting a printk in > map_pirq and unmap_pirq to see exactly what is happening across the > sleep/resume cycle. No unmap/map is done during sleep/resume cycle regarding that domain (hav= e two mapped pirqs). Even for dom0 I see only one unmap/map during suspend/resu= me. For most devices this doesn't break anything. Few exceptions needs module= reload after resume (e.g. sky2), but not sure about the reason (no additi= onal logs, simply no link detected). --=20 Best Regards / Pozdrawiam, Marek Marczykowski Invisible Things Lab --------------enigCB58CF515F025B7FD99FA62E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQEcBAEBAgAGBQJRVN/CAAoJENuP0xzK19cszTQH/Agb8tTGM2MHRHEFtxiBDqFo ook23kj43FUrox9RetBRqAnxHW32NhGYPi3Jb3I32XOnwMVTpN/YDvuDKua4uSPH FH4otiGDt18ZJQRkCdPCuqanK2SQ0JmN6r/sH1OmCPsn5cNaUmqtZErReyhufYHm 78d49VJwZVfUizZIPyYowOlLf2gCe/EICacID+hgygk8PuPKg9uoV7jJzN831tXO ZpJtpdj2dau1Xhm/+wA0rtuQu7WP53W+XRWyoXeBNIpDtiy267jmQNVMStaoBIdC ebdAAxVl6IrinJPzrrnkHwuSEKxmQTUJnIf37hQSaBsDzDj5maUl8vQU+cpBXpM= =YtDL -----END PGP SIGNATURE----- --------------enigCB58CF515F025B7FD99FA62E-- --===============1290809415649434967== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1290809415649434967==--