From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: High CPU temp, suspend problem - xen 4.1.5-pre, linux 3.7.x Date: Tue, 2 Apr 2013 10:05:14 -0400 Message-ID: <20130402140514.GA1754@phenom.dumpdata.com> References: <5151DE1C.1020307@invisiblethingslab.com> <5151E0D5.3050707@citrix.com> <5151E72D.30205@invisiblethingslab.com> <5151EE0B.9030605@citrix.com> <5152C16E02000078000C8CB8@nat28.tlf.novell.com> <515302C3.3000607@invisiblethingslab.com> <51547A5302000078000C962C@nat28.tlf.novell.com> <515493F0.6030708@invisiblethingslab.com> <515A30D4.90003@invisiblethingslab.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <515A30D4.90003@invisiblethingslab.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Marek Marczykowski Cc: Andrew Cooper , Ben Guthro , Jan Beulich , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Tue, Apr 02, 2013 at 03:13:56AM +0200, Marek Marczykowski wrote: > On 01.04.2013 15:53, Ben Guthro wrote: > > On Thu, Mar 28, 2013 at 3:03 PM, Marek Marczykowski > > wrote: > >> (XEN) Restoring affinity for d2v3 > >> (XEN) Assertion '!cpus_empty(cpus) && cpu_isset(cpu, cpus)' failed at > >> sched_credit.c:481 > > > > > > I think the "fix-suspend-scheduler-*" patches posted here are applicable here: > > http://markmail.org/message/llj3oyhgjzvw3t23 > > > > > > Specifically, I think you need this bit: > > > > diff --git a/xen/common/cpu.c b/xen/common/cpu.c > > index 630881e..e20868c 100644 > > --- a/xen/common/cpu.c > > +++ b/xen/common/cpu.c > > @@ -5,6 +5,7 @@ > > #include > > #include > > #include > > +#include > > > > unsigned int __read_mostly nr_cpu_ids = NR_CPUS; > > #ifndef nr_cpumask_bits > > @@ -212,6 +213,8 @@ void enable_nonboot_cpus(void) > > BUG_ON(error == -EBUSY); > > printk("Error taking CPU%d up: %d\n", cpu, error); > > } > > + if (system_state == SYS_STATE_resume) > > + cpumask_set_cpu(cpu, cpupool0->cpu_valid); > > } > > > > cpumask_clear(&frozen_cpus); > > > > Indeed, this makes things better, but still not ideal. > Now after resume all CPUs are in Pool-0, which is good. But CPU0 is much more > preferred than others (xl vcpu-list). For example if I start 4 busy loops in > dom0, I got (even after some time): > [user@dom0 ~]$ xl vcpu-list > Name ID VCPU CPU State Time(s) CPU Affinity > dom0 0 0 0 r-- 98.5 any cpu > dom0 0 1 0 --- 181.3 any cpu > dom0 0 2 2 r-- 262.4 any cpu > dom0 0 3 3 r-- 230.8 any cpu > netvm 1 0 0 -b- 18.4 any cpu > netvm 1 1 0 -b- 9.1 any cpu > netvm 1 2 0 -b- 7.1 any cpu > netvm 1 3 0 -b- 5.4 any cpu > firewallvm 2 0 0 -b- 10.7 any cpu > firewallvm 2 1 0 -b- 3.0 any cpu > firewallvm 2 2 0 -b- 2.5 any cpu > firewallvm 2 3 3 -b- 3.6 any cpu > > If I remove some CPU from Pool-0 and re-add it, things back to normal for this > particular CPU (so I got two equally used CPUs) - to fully restore system I > must remove all but CPU0 from Pool-0 and add it again. > > Also still only CPU0 have all C-states (C0-C3), all others have only C0-C1. > This probably could be fixed by your "xen: Re-upload processor PM data to > hypervisor after S3 resume" patch (reload of xen-acpi-processor module helps > here). But I don't think it is a right way. It isn't necessary on other > systems (with somehow older hardware). It must be something missing on resume > path. The question is what... The xen-acpi-processor should probably also have the cpu hotplug notification in it to deal with this - so that you don't need to do the reload. > > Perhaps someone need to go through enable_nonboot_cpus() (__cpu_up?) and check > if it restore all things disabled in disable_nonboot_cpus() (__cpu_disable?). > Unfortunately I don't know x86 details so good to follow that code... > > -- > Best Regards / Pozdrawiam, > Marek Marczykowski > Invisible Things Lab > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel