From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754013AbcAOWKu (ORCPT ); Fri, 15 Jan 2016 17:10:50 -0500 Received: from e35.co.us.ibm.com ([32.97.110.153]:33014 "EHLO e35.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750942AbcAOWKt (ORCPT ); Fri, 15 Jan 2016 17:10:49 -0500 X-IBM-Helo: d03dlp01.boulder.ibm.com X-IBM-MailFrom: paulmck@linux.vnet.ibm.com X-IBM-RcptTo: linux-kernel@vger.kernel.org Date: Fri, 15 Jan 2016 14:10:45 -0800 From: "Paul E. McKenney" To: Thomas Gleixner Cc: Sasha Levin , LKML , Peter Zijlstra Subject: Re: timers: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected Message-ID: <20160115221045.GA8598@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <56955C0F.1090005@oracle.com> <20160113161608.GN3818@linux.vnet.ibm.com> <20160114181846.GZ3818@linux.vnet.ibm.com> <20160115014232.GQ3818@linux.vnet.ibm.com> <20160115211125.GA3818@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160115211125.GA3818@linux.vnet.ibm.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16011522-0013-0000-0000-00001BEE724F Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jan 15, 2016 at 01:11:25PM -0800, Paul E. McKenney wrote: > On Fri, Jan 15, 2016 at 11:03:24AM +0100, Thomas Gleixner wrote: > > On Thu, 14 Jan 2016, Paul E. McKenney wrote: > > > > Untested patch below. > > > > > > One small fix to make it build below. Started rcutorture, somewhat > > > pointlessly given that the splat doesn't appear on my setup. > > > > Well, at least it tells us whether the change explodes by itself. > > Hmmm... > > So this is a strange one. I have been seeing increasing instability > in mainline over the past couple of releases, with the main symptom > being that the kernel decides that awakening RCU's grace-period kthreads > is an optional activity. The usual situation is that the kthread is > blocked for tens of seconds in an wait_event_interruptible_timeout(), > despite having a three-jiffy timeout. Doing periodic wakeups from > the scheduling-clock interrupt seems to clear things up, but such hacks > should not be necessary. > > Normally, I have to run for for some hours to have a good chance of seeing > this happen. This change triggered in a 30-minute run. Not only that, > but in a .config scenario that is normally very hard to trigger. This > scenario does involve CPU hotplug, and I am re-running with CPU hotplug > disabled. > > That said, I am starting to hear reports of people hitting this without > CPU hotplug operations... And without hotplug operations, instead of dying repeatedly in 30 minutes, it goes four hours with no complaints. Next trying wakeups. Thanx, Paul