Greetings, I got the below dmesg and the first bad commit is git://git.linaro.org/people/vireshk/linux timer-cleanup-for-tglx commit 6378cb51af5f4743db0dcb3cbcf862eac5908754 Author: Viresh Kumar AuthorDate: Thu Mar 20 14:29:02 2014 +0530 Commit: Viresh Kumar CommitDate: Wed Apr 2 14:54:57 2014 +0530 timer: don't migrate pinned timers migrate_timer() is called when a CPU goes down and its timers are required to be migrated to some other CPU. Its the responsibility of the users of the timer to remove it before control reaches to migrate_timers(). As these were the pinned timers, the best we can do is: don't migrate these and report to the user as well. That's all this patch does. Signed-off-by: Viresh Kumar =================================================== PARENT COMMIT NOT CLEAN. LOOK OUT FOR WRONG BISECT! =================================================== Attached one more dmesg for the NULL pointer bug in parent commit. +--------------------------------------------------------+------------+------------+------------+ | | 5a8530b7c3 | 6378cb51af | 7caf71f403 | +--------------------------------------------------------+------------+------------+------------+ | boot_successes | 103 | 14 | 10 | | boot_failures | 17 | 18 | 13 | | BUG:unable_to_handle_kernel_NULL_pointer_dereference | 16 | | | | Oops:SMP | 16 | | | | Kernel_panic-not_syncing:Fatal_exception | 16 | | | | backtrace:vfs_read | 16 | | | | backtrace:SyS_read | 16 | | | | BUG:kernel_test_crashed | 1 | | | | WARNING:CPU:PID:at_kernel/timer.c:migrate_timer_list() | 0 | 17 | 12 | | backtrace:vfs_write | 0 | 17 | 12 | | backtrace:SyS_write | 0 | 17 | 12 | | BUG:kernel_early_hang_without_any_printk_output | 0 | 1 | 1 | +--------------------------------------------------------+------------+------------+------------+ [ 74.242293] Unregister pv shared memory for cpu 1 [ 74.273280] smpboot: CPU 1 is now offline [ 74.274685] ------------[ cut here ]------------ [ 74.275524] WARNING: CPU: 0 PID: 1935 at kernel/timer.c:1621 migrate_timer_list+0xd6/0xf0() [ 74.275524] migrate_timer_list: can't migrate pinned timer: ffffffff81f06a60, deactivating it [ 74.275524] Modules linked in: [ 74.275524] CPU: 0 PID: 1935 Comm: 01-cpu-hotplug Not tainted 3.14.0-rc1-00087-g6378cb5 #1 [ 74.275524] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 74.275524] 0000000000000009 ffff88001bcabc38 ffffffff817237bd ffff88001bcabc80 [ 74.275524] ffff88001bcabc70 ffffffff8106a1dd 00000000000000f0 ffffffff81f06a60 [ 74.275524] ffff88001e04d120 ffffffff81e3d4c0 ffff88001e04d030 ffff88001bcabcd0 [ 74.275524] Call Trace: [ 74.275524] [] dump_stack+0x4d/0x66 [ 74.275524] [] warn_slowpath_common+0x7d/0xa0 [ 74.275524] [] warn_slowpath_fmt+0x4c/0x50 [ 74.275524] [] ? __internal_add_timer+0x113/0x130 [ 74.275524] [] migrate_timer_list+0xd6/0xf0 [ 74.275524] [] timer_cpu_notify+0xfc/0x1f0 [ 74.275524] [] notifier_call_chain+0x4c/0x70 [ 74.275524] [] __raw_notifier_call_chain+0xe/0x10 [ 74.275524] [] cpu_notify+0x23/0x50 [ 74.275524] [] cpu_notify_nofail+0xe/0x20 [ 74.275524] [] _cpu_down+0x1ad/0x2e0 [ 74.275524] [] cpu_down+0x34/0x50 [ 74.275524] [] cpu_subsys_offline+0x14/0x20 [ 74.275524] [] device_offline+0x95/0xc0 [ 74.275524] [] online_store+0x40/0x90 [ 74.275524] [] dev_attr_store+0x18/0x30 [ 74.275524] [] sysfs_kf_write+0x3d/0x50 [ 74.275524] [] kernfs_fop_write+0xd2/0x140 [ 74.275524] [] vfs_write+0xba/0x1e0 [ 74.275524] [] SyS_write+0x49/0xa0 [ 74.275524] [] system_call_fastpath+0x16/0x1b [ 74.275524] ---[ end trace 9cf1ed50b307bb1a ]--- [ 74.630194] blk-mq: CPU -> queue map git bisect start 7caf71f403b4758e8e2b2fef1d2e2d7f7a0c5db4 b97f0291a2504291aef850077f98cab68a5a2f33 -- git bisect good fe04ef37d949ea92dbc37164eff116fe3c4a0930 # 01:45 32+ 42 hrtimer: make enqueue_hrtimer() return void git bisect good de1f862abff16ab0a1058181bb541a55b691e64d # 03:35 32+ 3 hrtimer: Use for_each_active_base() to iterate over active clock bases git bisect bad 6378cb51af5f4743db0dcb3cbcf862eac5908754 # 04:02 2- 2 timer: don't migrate pinned timers git bisect good 6b4e6938e2f62e8b7f02453cabff5640f6b4bbdf # 04:10 40+ 2 hrtimer: call switch_hrtimer_base() after setting new expiry time git bisect good e5535a330ccddbe95760a52bba837703ee16926f # 04:36 40+ 1 hrtimer: fix routine names in comments git bisect good 5a8530b7c3b1889861949d4e52b2da82d0aff242 # 05:03 40+ 0 timer: track pinned timers with TIMER_PINNED flag # first bad commit: [6378cb51af5f4743db0dcb3cbcf862eac5908754] timer: don't migrate pinned timers git bisect good 5a8530b7c3b1889861949d4e52b2da82d0aff242 # 05:13 120+ 17 timer: track pinned timers with TIMER_PINNED flag git bisect bad 7caf71f403b4758e8e2b2fef1d2e2d7f7a0c5db4 # 05:13 0- 13 sched: don't queue timers on quiesced CPUs git bisect good 4b22efdd5595f0acb48f02bf664a451ee98f9a2e # 05:29 120+ 1 Add linux-next specific files for 20140403 Thanks, Fengguang