From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753300AbdHWDWL (ORCPT ); Tue, 22 Aug 2017 23:22:11 -0400 Received: from regular1.263xmail.com ([211.150.99.141]:39254 "EHLO regular1.263xmail.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753057AbdHWDWJ (ORCPT ); Tue, 22 Aug 2017 23:22:09 -0400 X-263anti-spam: KSV:0;BIG:0; X-MAIL-GRAY: 0 X-MAIL-DELIVERY: 1 X-KSVirus-check: 0 X-ADDR-CHECKED4: 1 X-ABS-CHECKED: 1 X-SKE-CHECKED: 1 X-ANTISPAM-LEVEL: 2 X-RL-SENDER: jeffy.chen@rock-chips.com X-FST-TO: npiggin@gmail.com X-SENDER-IP: 103.29.142.67 X-LOGIN-NAME: jeffy.chen@rock-chips.com X-UNIQUE-TAG: X-ATTACHMENT-NUM: 0 X-DNS-TYPE: 0 Message-ID: <599CF4C4.2070703@rock-chips.com> Date: Wed, 23 Aug 2017 11:21:40 +0800 From: jeffy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20130126 Thunderbird/19.0 MIME-Version: 1.0 To: Nicholas Piggin , Thomas Gleixner CC: akpm@linux-foundation.org, John Stultz , Stephen Boyd , Jonathan.Cameron@huawei.com, paulmck@linux.vnet.ibm.com, mpe@ellerman.id.au, dzickus@redhat.com, sfr@canb.auug.org.au, linuxarm@huawei.com, abdhalee@linux.vnet.ibm.com, linux-kernel@vger.kernel.org, torvalds@linux-foundation.org Subject: Re: [v2] timers: Fix excessive granularity of new timers after a nohz idle References: <20170822084348.21436-1-npiggin@gmail.com> In-Reply-To: <20170822084348.21436-1-npiggin@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi guys, I was testing a arm64 based device(chromebook bob), and the kernel hang a lot(during booting or suspending) on for-next(next-20170822) with this commit: 71acb768f5b3 (timers: Fix excessive granularity of new timers after a nohz idle) After revert it(or just add "!base->is_idle" check back), things work well... my dts: arch/arm64/boot/dts/rockchip/rk3399-gru-kevin.dts my cpu info: localhost / # cat /proc/cpuinfo processor : 0 model name : ARMv8 Processor rev 4 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 1 model name : ARMv8 Processor rev 4 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 2 model name : ARMv8 Processor rev 4 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 3 model name : ARMv8 Processor rev 4 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4 processor : 4 model name : ARMv8 Processor rev 2 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd08 CPU revision : 2 processor : 5 model name : ARMv8 Processor rev 2 (v8l) BogoMIPS : 48.00 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpa e evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd08 CPU revision : 2 my hang dmesg: [ 304.797633] EXT4-fs (mmcblk1p3): mounted filesystem without journal. Opts: (null ) [ 304.806070] VFS: Mounted root (ext2 filesystem) on device 179:3. [ 304.815775] devtmpfs: mounted [ 304.823110] Freeing unused kernel memory: 2048K [ 304.891944] audit: type=1404 audit(304.617:2): enforcing=1 old_enforcing=0 auid= 4294967295 ses=4294967295 [ 364.655978] INFO: rcu_preempt self-detected stall on CPU [ 364.661924] 3-...: (23 GPs behind) idle=486/2/0 softirq=39/40 fqs=14991 [ 364.669513] (t=60000 jiffies g=-244 c=-245 q=71) [ 364.674872] Task dump for CPU 3: [ 364.678478] swapper/3 R running task 0 0 1 0x00000022 [ 364.686365] Call trace: [ 364.689095] [] dump_backtrace+0x0/0x220 [ 364.695133] [] show_stack+0x24/0x30 [ 364.700784] [] sched_show_task+0x138/0x168 [ 364.707112] [] dump_cpu_task+0x48/0x54 [ 364.713054] [] rcu_dump_cpu_stacks+0xa0/0xe0 [ 364.719577] [] rcu_check_callbacks+0x328/0x934 [ 364.726295] [] update_process_times+0x34/0x5c [ 364.732919] [] tick_sched_handle.isra.3+0x54/0x68 [ 364.739928] [] tick_sched_timer+0x48/0x84 [ 364.746160] [] __hrtimer_run_queues+0x194/0x31c [ 364.752974] [] hrtimer_interrupt+0xac/0x1bc [ 364.759402] [] arch_timer_handler_phys+0x3c/0x4c [ 364.766316] [] handle_percpu_devid_irq+0xf4/0x270 [ 364.773324] [] generic_handle_irq+0x2c/0x44 [ 364.779749] [] __handle_domain_irq+0x90/0xb8 [ 364.786271] [] gic_handle_irq+0x140/0x1a8 [ 364.792502] Exception stack(0xffffff800801bba0 to 0xffffff800801bce0) [ 364.799696] bba0: 000000000000b7e9 0000000000000003 00000040eee72000 ffffffc0f7d 601f8 [ 364.808456] bbc0: 000000000000003f ffffffc0f7d5efb8 ffffff800801bde8 80000000000 00000 [ 364.817215] bbe0: 0000000000000001 0000000000000000 ffffff8009028338 00000000000 00000 [ 364.825975] bc00: 0000000000000105 0000000000000130 0000000000005dc0 ffffff8008a 58000 [ 364.834734] bc20: 0000000000000007 000000000000000e 000000000000005c ffffffc0002 18000 [ 364.843494] bc40: ffffff8009007000 ffffffc000218058 ffffff80080c1034 ffffffc0002 18000 [ 364.852253] bc60: ffffff800801bde0 ffffffc000218000 000000000000000a 00000000000 00001 [ 364.861013] bc80: ffffff80090e8000 ffffff800801bce0 ffffff8008885c00 ffffff80080 1bce0 [ 364.869773] bca0: ffffff8008885c04 0000000060000145 ffffffc000218058 ffffff80080 c1034 [ 364.878532] bcc0: 0000008000000000 ffffff8008885c00 ffffff800801bce0 ffffff80088 85c04 [ 364.887291] [] el1_irq+0xb0/0x124 [ 364.892747] [] _raw_spin_unlock_irq+0x24/0x50 [ 364.899368] [] idle_worker_timeout+0x6c/0x78 [ 364.905892] [] call_timer_fn+0xb0/0x1c8 [ 364.911929] [] expire_timers+0x10c/0x158 [ 364.918055] [] run_timer_softirq.part.1+0xb0/0xe8 [ 364.925062] [] run_timer_softirq+0xc4/0xec [ 364.931381] [] __do_softirq+0x18c/0x348 [ 364.937419] [] irq_exit+0x7c/0xc4 [ 364.942872] [] __handle_domain_irq+0x94/0xb8 [ 364.949393] [] gic_handle_irq+0x140/0x1a8 [ 364.955623] Exception stack(0xffffff80092ebe10 to 0xffffff80092ebf50) [ 364.962826] be00: 00000040eee72000 ffffff8008e e8000 [ 364.971586] be20: 0000000000000000 ffffff80092ebf60 00000040eee72000 ffffff80090 2f900 [ 364.980345] be40: 00000046de94b940 0000000000000040 ffffffc0f2d35360 ffffff80092 ebed0 [ 364.989105] be60: 0000000000000a80 0000000000000003 0000000000003c93 00000000000 03c93 [ 364.997864] be80: 00000000000c8000 ffffff8008a58000 0000000000000007 00000000000 0000e [ 365.006624] bea0: 000000000000005c ffffff8008ee8000 ffffff80090e9000 ffffff80090 e9160 [ 365.015383] bec0: ffffff8009010a30 0000000000000000 0000000000000000 00000000000 00000 [ 365.024143] bee0: 0000000000000000 0000000000000000 0000000000000000 ffffff80092 ebf50 [ 365.032902] bf00: ffffff80080855e0 ffffff80092ebf50 ffffff80080855e4 00000000600 00145 [ 365.041662] bf20: ffffff8008ee8000 ffffff8009010a30 ffffffffffffffff 00000000000 00000 [ 365.050420] bf40: ffffff80092ebf50 ffffff80080855e4 [ 365.055873] [] el1_irq+0xb0/0x124 [ 365.061326] [] arch_cpu_idle+0x74/0x19c [ 365.067364] [] default_idle_call+0x28/0x30 [ 365.073694] [] do_idle+0xd8/0x198 [ 365.079148] [] cpu_startup_entry+0x28/0x2c [ 365.085478] [] secondary_start_kernel+0x184/0x190 [ 492.786002] INFO: task init:1 blocked for more than 120 seconds. [ 492.792730] Not tainted 4.13.0-rc6-next-20170822 #100 [ 492.799000] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this mes sage. [ 492.807773] init D 0 1 0 0x00400000 [ 492.813922] Call trace: [ 492.816662] [] __switch_to+0x7c/0x88 [ 492.822415] [] __schedule+0x40c/0x6c4 [ 492.828265] [] schedule+0x90/0xb0 [ 492.833726] [] schedule_timeout+0x4c/0x354 [ 492.840062] [] wait_for_common+0x164/0x19c [ 492.846397] [] wait_for_completion+0x28/0x34 [ 492.852928] [] __wait_rcu_gp+0x160/0x18c [ 492.859062] [] synchronize_rcu+0x6c/0x90 [ 492.865205] [] synchronize_net+0x38/0x40 [ 492.871348] [] selinux_netcache_avc_callback+0x30/0x40 [ 492.878851] [] avc_ss_reset+0x44/0xa4 [ 492.884703] [] sel_write_enforce+0x138/0x198 [ 492.891235] [] __vfs_write+0x60/0x16c [ 492.897086] [] vfs_write+0xc4/0x160 [ 492.902740] [] SyS_write+0x70/0xc8 [ 492.908297] Exception stack(0xffffff800804bec0 to 0xffffff800804c000) [ 492.915513] bec0: 0000000000000003 00000000ff7f46a0 0000000000000001 00000000fff fffff [ 492.924285] bee0: 00000000ff7f46a0 0000000000000003 0000000005a417b8 00000000000 00004 [ 492.933075] bf00: 00000000ff7f5708 00000000ffffffff 00000000045f8d0c 00000000000 00000 [ 492.941843] bf20: 0000000000000000 00000000ff7f4694 00000000ea720eaf 00000000000 00000 [ 492.950609] bf40: 0000000000000000 0000000000000000 0000000000000000 00000000000 00000 [ 492.959375] bf60: 0000000000000000 0000000000000000 0000000000000000 00000000000 00000 [ 492.968141] bf80: 0000000000000000 0000000000000000 0000000000000000 00000000000 00000 [ 492.976907] bfa0: 0000000000000000 0000000000000000 0000000000000000 00000000000 00000 [ 492.985674] bfc0: 00000000ea6aba56 0000000060000030 0000000000000003 00000000000 00004 [ 492.994440] bfe0: 0000000000000000 0000000000000000 0000000000000000 00000000000 00000 [ 493.003207] [] el0_svc_naked+0x24/0x28 [ 493.009153] Kernel panic - not syncing: hung_task: blocked tasks [ 493.015872] CPU: 0 PID: 43 Comm: khungtaskd Not tainted 4.13.0-rc6-next-20170822 #100 [ 493.024630] Hardware name: Google Kevin (DT) [ 493.029400] Call trace: [ 493.032132] [] dump_backtrace+0x0/0x220 [ 493.038169] [] show_stack+0x24/0x30 [ 493.043819] [] dump_stack+0x98/0xb8 [ 493.049468] [] panic+0x144/0x2a0 [ 493.054826] [] watchdog+0x348/0x380 [ 493.060475] [] kthread+0x134/0x13c [ 493.066025] [] ret_from_fork+0x10/0x20 [ 493.071966] SMP: stopping secondary CPUs [ 493.076352] Kernel Offset: disabled [ 493.080249] CPU features: 0x00200c [ 493.084048] Memory Limit: none