From mboxrd@z Thu Jan 1 00:00:00 1970 From: poma Subject: Re: WARNING: CPU: 1 PID: 0 at kernel/time/tick-broadcast.c:668 tick_broadcast_oneshot_control+0x17d/0x190() Date: Tue, 11 Feb 2014 21:44:44 +0100 Message-ID: <52FA8BBC.5000405@gmail.com> References: <52F84A9B.5020008@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kernel-bounces-TuqUDEhatI4ANWPb/1PvSmm0pvjS0E/A@public.gmane.org Errors-To: kernel-bounces-TuqUDEhatI4ANWPb/1PvSmm0pvjS0E/A@public.gmane.org To: Thomas Gleixner Cc: Olaf Hering , Stanislaw Gruszka , Mailing-List fedora-kernel , Josh Boyer , linux-pm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux Kernel list , "Justin M. Forbes" List-Id: linux-pm@vger.kernel.org T24gMTEuMDIuMjAxNCAxNToyNSwgVGhvbWFzIEdsZWl4bmVyIHdyb3RlOgo+IE9uIE1vbiwgMTAg RmViIDIwMTQsIFRob21hcyBHbGVpeG5lciB3cm90ZToKPj4gT24gTW9uLCAxMCBGZWIgMjAxNCwg cG9tYSB3cm90ZToKPj4KPj4+IFsgICA4My41NTg1NTFdICBbPGZmZmZmZmZmODEwMjViMTc+XSBh bWRfZTQwMF9pZGxlKzB4ODcvMHgxMzAKPj4KPj4gU28gdGhpcyBzZWVtcyB0byBoYXBwZW4gb25s eSBvbiBBTUQgbWFjaGluZXMgd2hpY2ggdXNlIHRoYXQgZTQwMCBpZGxlCj4+IG1vZGUuIEkgaGF2 ZSBubyBpZGVhIGF0IHRoZSBtb21lbnQgd2hhdHMgd3JvbmcgdGhlcmUuIEknbGwgZmluZCBvbmUg b2YKPj4gdGhvc2UgbWFjaGluZXMgYW5kIHRyeSB0byByZXByb2R1Y2UuCj4gCj4gRm91bmQgaXQu IFBhdGNoIGJlbG93Lgo+IAo+IFRoYW5rcywKPiAKPiAJdGdseAo+IC0tLS0KPiBTdWJqZWN0OiB0 aWNrOiBDbGVhciBicm9hZGNhc3QgcGVuZGluZyBiaXQgd2hlbiBzd2l0Y2hpbmcgdG8gb25lc2hv dAo+IEZyb206IFRob21hcyBHbGVpeG5lciA8dGdseEBsaW51dHJvbml4LmRlPgo+IERhdGU6IFR1 ZSwgMTEgRmViIDIwMTQgMTQ6MzU6NDAgKzAxMDAKPiAKPiBBTUQgc3lzdGVtcyB3aGljaCB1c2Ug dGhlIEMxRSB3b3JrYXJvdW5kIGluIHRoZSBhbWRfZTQwMF9pZGxlIHJvdXRpbmUKPiB0cmlnZ2Vy IHRoZSBXQVJOX09OX09OQ0UgaW4gdGhlIGJyb2FkY2FzdCBjb2RlIHdoZW4gb25saW5pbmcgYSBD UFUuCj4gCj4gVGhlIHJlYXNvbiBpcyB0aGF0IHRoZSBpZGxlIHJvdXRpbmUgb2YgdGhvc2UgQU1E IHN5c3RlbXMgc3dpdGNoZXMgdGhlCj4gY3B1IGludG8gZm9yY2VkIGJyb2FkY2FzdCBtb2RlIGVh cmx5IG9uIGJlZm9yZSB0aGUgbmV3bHkgYnJvdWdodCB1cAo+IENQVSBjYW4gc3dpdGNoIG92ZXIg dG8gaGlnaCByZXNvbHV0aW9uIC8gTk9IWiBtb2RlLiBUaGUgdGltZXIgcmVsYXRlZAo+IENQVTEg YnJpbmd1cCBsb29rcyBsaWtlIHRoaXM6Cj4gCj4gICBjbG9ja2V2ZW50X3JlZ2lzdGVyX2Rldmlj ZShsb2NhbF9hcGljKTsKPiAgIHRpY2tfc2V0dXAobG9jYWxfYXBpYyk7Cj4gICAuLi4KPiAgIGlk bGUoKQo+IAl0aWNrX2Jyb2FkY2FzdF9vbl9vZmYoRk9SQ0UpOwo+IAl0aWNrX2Jyb2FkY2FzdF9v bmVzaG90X2NvbnRyb2woRU5URVIpCj4gCSAgY3B1bWFza19zZXQoY3B1LCBicm9hZGNhc3Rfb25l c2hvdF9tYXNrKTsKPiAJaGFsdCgpOwo+IAo+IE5vdyB0aGUgYnJvYWRjYXN0IGludGVycnVwdCBv biBDUFUwIHNldHMgQ1BVMSBpbiB0aGUKPiBicm9hZGNhc3RfcGVuZGluZ19tYXNrIGFuZCB3YWtl cyBDUFUxLiBTbyBDUFUxIGNvbnRpbnVlczoKPiAKPiAJbG9jYWxfYXBpY190aW1lcl9pbnRlcnJ1 cHQoKQo+IAkgICB0aWNrX2hhbmRsZV9wZXJpb2RpYygpOwo+IAkgICBzb2Z0aXJxKCkKPiAJICAg ICB0aWNrX2luaXRfaGlnaHJlcygpOwo+IAkgICAgICAgY3B1bWFza19jbHIoY3B1LCBicm9hZGNh c3Rfb25lc2hvdF9tYXNrKTsKPiAJCj4gCXRpY2tfYnJvYWRjYXN0X29uZXNob3RfY29udHJvbChF TlRFUikKPiAJICAgV0FSTl9PTihjcHVtYXNrX3Rlc3QoY3B1LCBicm9hZGNhc3RfcGVuZGluZ19t YXNrKTsKPiAKPiBTbyB3aGlsZSB3ZSByZW1vdmUgQ1BVMSBmcm9tIHRoZSBicm9hZGNhc3Rfb25l c2hvdF9tYXNrIHdoZW4gd2Ugc3dpdGNoCj4gb3ZlciB0byBoaWdocmVzIG1vZGUsIHdlIGRvIG5v dCBjbGVhciB0aGUgcGVuZGluZyBiaXQsIHdoaWNoIHRoZW4KPiB0cmlnZ2VycyB0aGUgd2Fybmlu ZyB3aGVuIHdlIGdvIGJhY2sgdG8gaWRsZS4KPiAKPiBUaGUgcmVhc29uIHdoeSB0aGlzIGlzIG9u bHkgdmlzaWJsZSBvbiBDMUUgYWZmZWN0ZWQgQU1EIHN5c3RlbXMgaXMKPiB0aGF0IHRoZSBvdGhl ciBtYWNoaW5lcyBlbnRlciB0aGUgZGVlcCBzbGVlcCBzdGF0ZXMgdmlhCj4gYWNwaV9pZGxlL2lu dGVsX2lkbGUgYW5kIGV4aXQgdGhlIGJyb2FkY2FzdCBtb2RlIGJlZm9yZSBleGVjdXRpbmcgdGhl Cj4gcmVtb3RlIHRyaWdnZXJlZCBsb2NhbF9hcGljX3RpbWVyX2ludGVycnVwdC4gU28gdGhlIHBl bmRpbmcgYml0IGlzCj4gYWxyZWFkeSBjbGVhcmVkIHdoZW4gdGhlIHN3aXRjaCBvdmVyIHRvIGhp Z2hyZXMgbW9kZSBpcyBjbGVhcmluZyB0aGUKPiBvbmVzaG90IG1hc2suCj4gCj4gVGhlIHNvbHV0 aW9uIGlzIHNpbXBsZTogQ2xlYXIgdGhlIHBlbmRpbmcgYml0IHRvZ2V0aGVyIHdpdGggdGhlIG1h c2sKPiBiaXQgd2hlbiB3ZSBzd2l0Y2ggb3ZlciB0byBoaWdocmVzIG1vZGUuCj4gCj4gUmVwb3J0 ZWQtYnk6IHBvbWEgPHBvbWlkb3JhYmVsaXNpbWFAZ21haWwuY29tPgo+IENjOiBzdGFibGVAdmdl ci5rZXJuZWwub3JnICMgMy4xMCsKPiBTaWduZWQtb2ZmLWJ5OiBUaG9tYXMgR2xlaXhuZXIgPHRn bHhAbGludXRyb25peC5kZT4KPiAtLS0KPiAga2VybmVsL3RpbWUvdGljay1icm9hZGNhc3QuYyB8 ICAgIDEgKwo+ICAxIGZpbGUgY2hhbmdlZCwgMSBpbnNlcnRpb24oKykKPiAKPiBJbmRleDogbGlu dXgtMi42L2tlcm5lbC90aW1lL3RpY2stYnJvYWRjYXN0LmMKPiA9PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Cj4gLS0tIGxp bnV4LTIuNi5vcmlnL2tlcm5lbC90aW1lL3RpY2stYnJvYWRjYXN0LmMKPiArKysgbGludXgtMi42 L2tlcm5lbC90aW1lL3RpY2stYnJvYWRjYXN0LmMKPiBAQCAtNzU2LDYgKzc1Niw3IEBAIG91dDoK PiAgc3RhdGljIHZvaWQgdGlja19icm9hZGNhc3RfY2xlYXJfb25lc2hvdChpbnQgY3B1KQo+ICB7 Cj4gIAljcHVtYXNrX2NsZWFyX2NwdShjcHUsIHRpY2tfYnJvYWRjYXN0X29uZXNob3RfbWFzayk7 Cj4gKwljcHVtYXNrX2NsZWFyX2NwdShjcHUsIHRpY2tfYnJvYWRjYXN0X3BlbmRpbmdfbWFzayk7 Cj4gIH0KPiAgCj4gIHN0YXRpYyB2b2lkIHRpY2tfYnJvYWRjYXN0X2luaXRfbmV4dF9ldmVudChz dHJ1Y3QgY3B1bWFzayAqbWFzaywKPiAKPiAKClRoYW5rcyEKCgpwb21hCgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18Ka2VybmVsIG1haWxpbmcgbGlzdApr ZXJuZWxAbGlzdHMuZmVkb3JhcHJvamVjdC5vcmcKaHR0cHM6Ly9hZG1pbi5mZWRvcmFwcm9qZWN0 Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2tlcm5lbA== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755012AbaBKUpK (ORCPT ); Tue, 11 Feb 2014 15:45:10 -0500 Received: from mail-ea0-f177.google.com ([209.85.215.177]:51117 "EHLO mail-ea0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754977AbaBKUpE (ORCPT ); Tue, 11 Feb 2014 15:45:04 -0500 Message-ID: <52FA8BBC.5000405@gmail.com> Date: Tue, 11 Feb 2014 21:44:44 +0100 From: poma User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Thomas Gleixner CC: Linux Kernel list , linux-pm@vger.kernel.org, Olaf Hering , Dave Jones , "Justin M. Forbes" , Josh Boyer , Stanislaw Gruszka , Mailing-List fedora-kernel Subject: Re: WARNING: CPU: 1 PID: 0 at kernel/time/tick-broadcast.c:668 tick_broadcast_oneshot_control+0x17d/0x190() References: <52F84A9B.5020008@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11.02.2014 15:25, Thomas Gleixner wrote: > On Mon, 10 Feb 2014, Thomas Gleixner wrote: >> On Mon, 10 Feb 2014, poma wrote: >> >>> [ 83.558551] [] amd_e400_idle+0x87/0x130 >> >> So this seems to happen only on AMD machines which use that e400 idle >> mode. I have no idea at the moment whats wrong there. I'll find one of >> those machines and try to reproduce. > > Found it. Patch below. > > Thanks, > > tglx > ---- > Subject: tick: Clear broadcast pending bit when switching to oneshot > From: Thomas Gleixner > Date: Tue, 11 Feb 2014 14:35:40 +0100 > > AMD systems which use the C1E workaround in the amd_e400_idle routine > trigger the WARN_ON_ONCE in the broadcast code when onlining a CPU. > > The reason is that the idle routine of those AMD systems switches the > cpu into forced broadcast mode early on before the newly brought up > CPU can switch over to high resolution / NOHZ mode. The timer related > CPU1 bringup looks like this: > > clockevent_register_device(local_apic); > tick_setup(local_apic); > ... > idle() > tick_broadcast_on_off(FORCE); > tick_broadcast_oneshot_control(ENTER) > cpumask_set(cpu, broadcast_oneshot_mask); > halt(); > > Now the broadcast interrupt on CPU0 sets CPU1 in the > broadcast_pending_mask and wakes CPU1. So CPU1 continues: > > local_apic_timer_interrupt() > tick_handle_periodic(); > softirq() > tick_init_highres(); > cpumask_clr(cpu, broadcast_oneshot_mask); > > tick_broadcast_oneshot_control(ENTER) > WARN_ON(cpumask_test(cpu, broadcast_pending_mask); > > So while we remove CPU1 from the broadcast_oneshot_mask when we switch > over to highres mode, we do not clear the pending bit, which then > triggers the warning when we go back to idle. > > The reason why this is only visible on C1E affected AMD systems is > that the other machines enter the deep sleep states via > acpi_idle/intel_idle and exit the broadcast mode before executing the > remote triggered local_apic_timer_interrupt. So the pending bit is > already cleared when the switch over to highres mode is clearing the > oneshot mask. > > The solution is simple: Clear the pending bit together with the mask > bit when we switch over to highres mode. > > Reported-by: poma > Cc: stable@vger.kernel.org # 3.10+ > Signed-off-by: Thomas Gleixner > --- > kernel/time/tick-broadcast.c | 1 + > 1 file changed, 1 insertion(+) > > Index: linux-2.6/kernel/time/tick-broadcast.c > =================================================================== > --- linux-2.6.orig/kernel/time/tick-broadcast.c > +++ linux-2.6/kernel/time/tick-broadcast.c > @@ -756,6 +756,7 @@ out: > static void tick_broadcast_clear_oneshot(int cpu) > { > cpumask_clear_cpu(cpu, tick_broadcast_oneshot_mask); > + cpumask_clear_cpu(cpu, tick_broadcast_pending_mask); > } > > static void tick_broadcast_init_next_event(struct cpumask *mask, > > Thanks! poma