From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753095Ab1IYQsM (ORCPT ); Sun, 25 Sep 2011 12:48:12 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:49092 "EHLO e7.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753055Ab1IYQsK (ORCPT ); Sun, 25 Sep 2011 12:48:10 -0400 Date: Sun, 25 Sep 2011 09:48:04 -0700 From: "Paul E. McKenney" To: Frederic Weisbecker Cc: "Kirill A. Shutemov" , linux-kernel@vger.kernel.org, Dipankar Sarma , Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Lai Jiangshan Subject: Re: linux-next-20110923: warning kernel/rcutree.c:1833 Message-ID: <20110925164804.GD2995@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20110925002409.GA24220@shutemov.name> <20110925050826.GC2995@linux.vnet.ibm.com> <20110925112637.GA19298@shutemov.name> <20110925130622.GA9205@somewhere.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110925130622.GA9205@somewhere.redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Sep 25, 2011 at 03:06:25PM +0200, Frederic Weisbecker wrote: > On Sun, Sep 25, 2011 at 02:26:37PM +0300, Kirill A. Shutemov wrote: > > On Sat, Sep 24, 2011 at 10:08:26PM -0700, Paul E. McKenney wrote: > > > On Sun, Sep 25, 2011 at 03:24:09AM +0300, Kirill A. Shutemov wrote: > > > > [ 29.974288] ------------[ cut here ]------------ > > > > [ 29.974308] WARNING: at /home/kas/git/public/linux-next/kernel/rcutree.c:1833 rcu_needs_cpu+0xff > > > > [ 29.974316] Hardware name: HP EliteBook 8440p > > > > [ 29.974321] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iple_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc rfcomm bnep acpi_cpufreq mperfckd fscache auth_rpcgss nfs_acl sunrpc ext2 loop kvm_intel kvm snd_hda_codec_hdmi snd_hda_codec_idtideodev media v4l2_compat_ioctl32 snd_seq bluetooth drm_kms_helper snd_timer tpm_infineon snd_seq_drt tpm_tis hp_accel intel_ips soundcore lis3lv02d tpm rfkill i2c_algo_bit snd_page_alloc i2c_core c16 sha256_generic aesni_intel cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod sg sr_mod sd_mod cd thermal_sys [last unloaded: scsi_wait_scan] > > > > [ 29.974517] Pid: 0, comm: kworker/0:1 Not tainted 3.1.0-rc7-next-20110923 #2 > > > > [ 29.974521] Call Trace: > > > > [ 29.974525] [] warn_slowpath_common+0x7a/0xb0 > > > > [ 29.974540] [] warn_slowpath_null+0x15/0x20 > > > > [ 29.974546] [] rcu_needs_cpu+0xff/0x110 > > > > [ 29.974555] [] tick_nohz_stop_sched_tick+0x13f/0x3d0 > > > > [ 29.974563] [] ? notifier_call_chain+0x70/0x70 > > > > [ 29.974571] [] irq_exit+0xa2/0xd0 > > > > [ 29.974578] [] smp_apic_timer_interrupt+0x85/0x1c0 > > > > [ 29.974585] [] ? notifier_call_chain+0x70/0x70 > > > > [ 29.974592] [] apic_timer_interrupt+0x6e/0x80 > > > > [ 29.974596] [] ? acpi_hw_read+0x4a/0x51 > > > > [ 29.974609] [] ? lock_acquire+0xa7/0x160 > > > > [ 29.974615] [] ? notifier_call_chain+0x70/0x70 > > > > [ 29.974622] [] __atomic_notifier_call_chain+0x56/0xb0 > > > > [ 29.974631] [] ? notifier_call_chain+0x70/0x70 > > > > [ 29.974642] [] ? cpuidle_idle_call+0x106/0x350 > > > > [ 29.974651] [] atomic_notifier_call_chain+0x11/0x20 > > > > [ 29.974661] [] cpu_idle+0xe3/0x120 > > > > [ 29.974672] [] start_secondary+0x1fd/0x204 > > > > [ 29.974681] ---[ end trace 6c1d44095a3bb7c5 ]--- > > > > > > Do the following help? > > > > > > https://lkml.org/lkml/2011/9/17/47 > > > https://lkml.org/lkml/2011/9/17/45 > > > https://lkml.org/lkml/2011/9/17/43 > > > > Yes. Thanks. > > I believe that doesn't really fix the issue. But the warning is not > easy to trigger. You simply haven't hit it by chance after applying > the patches. > > This happens when the idle notifier callchain is called in idle > and is interrupted in the middle. So we have called rcu_read_lock() > but haven't yet released with rcu_read_unlock(), and in the end > of the interrupt we call tick_nohz_stop_sched_tick() -> rcu_needs_cpu() > which is illegal while in an rcu read side critical section. > > No idea how to solve that. Any use of RCU after the tick gets stopped > is concerned here. If it is really required that rcu_needs_cpu() can't > be called in an rcu read side critical sectionn then it's not going > to be easy to fix. > > But I don't really understand that requirement. rcu_needs_cpu() simply > checks if we don't have callbacks to handle. So I don't understand how > read side is concerned. It's rather the write side. > The rule I can imagine instead is: don't call __call_rcu() once the tick is > stopped. > > But I'm certainly missing something. > > Paul? This is required for RCU_FAST_NO_HZ, which checks to see whether the current CPU can accelerate the current grace period so as to enter dyntick-idle mode sooner than it would otherwise. This takes effect in the situation where rcu_needs_cpu() sees that there are callbacks. It then notes a quiescent state (which is illegal in an RCU read-side critical section), calls force_quiescent_state(), and so on. For this to work, the current CPU must be in an RCU read-side critical section. If this cannot be made to work, another option is to call a new RCU function in the case where rcu_needs_cpu() returned false, but after the RCU read-side critical section has exited. This new RCU function could then attempt to rearrange RCU so as to allow the CPU to enter dyntick-idle mode more quickly. It is more important for this to happen when the CPU is going idle than when it is executing a user process. So, is this doable? Thanx, Paul