Hi Steven, 0day kernel testing robot got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git ftrace/core commit 87354059881ce9315181604dc17076c535f4d744 Author: Steven Rostedt (Red Hat) AuthorDate: Tue Jul 22 20:41:42 2014 -0400 Commit: Steven Rostedt CommitDate: Tue Sep 9 19:26:06 2014 -0400 ftrace: Add helper function ftrace_ops_get_func() Add the helper function to what the mcount trampoline is to call for a ftrace_ops function. This helper will be used by arch code in the future to set up dynamic trampolines. But as this does the same tests that are performed in choosing what function to call for the default mcount trampoline, might as well use it to clean up the existing code. Signed-off-by: Steven Rostedt +---------------------------------------------------+------------+------------+------------------+ | | f1ff6348b3 | 8735405988 | v3.17-rc4_091007 | +---------------------------------------------------+------------+------------+------------------+ | boot_successes | 60 | 0 | 0 | | boot_failures | 0 | 20 | 13 | | WARNING:at_kernel/trace/trace.c:register_tracer() | 0 | 20 | 13 | | backtrace:register_tracer | 0 | 20 | 13 | | backtrace:init_graph_trace | 0 | 20 | 13 | | backtrace:kernel_init_freeable | 0 | 20 | 13 | +---------------------------------------------------+------------+------------+------------------+ FAILED! [ 0.330606] ------------[ cut here ]------------ [ 0.330606] ------------[ cut here ]------------ [ 0.331418] WARNING: CPU: 0 PID: 1 at kernel/trace/trace.c:1127 register_tracer+0x1e5/0x309() [ 0.331418] WARNING: CPU: 0 PID: 1 at kernel/trace/trace.c:1127 register_tracer+0x1e5/0x309() [ 0.333146] CPU: 0 PID: 1 Comm: swapper Not tainted 3.17.0-rc4-00002-g8735405 #1 [ 0.333146] CPU: 0 PID: 1 Comm: swapper Not tainted 3.17.0-rc4-00002-g8735405 #1 [ 0.334406] 0000000000000000 [ 0.334406] 0000000000000000 ffff88001284fe18 ffff88001284fe18 ffffffff8223c6b4 ffffffff8223c6b4 ffff88001284fe50 ffff88001284fe50 [ 0.335758] ffffffff810ca314 [ 0.335758] ffffffff810ca314 ffffffff81172420 ffffffff81172420 ffffffff83560880 ffffffff83560880 ffffffff83638cc0 ffffffff83638cc0 [ 0.337101] ffffffff82bc6397 [ 0.337101] ffffffff82bc6397 ffffffff8371a910 ffffffff8371a910 ffff88001284fe60 ffff88001284fe60 ffffffff810ca468 ffffffff810ca468 [ 0.338435] Call Trace: [ 0.338435] Call Trace: [ 0.338886] [] dump_stack+0x2e/0x3e [ 0.338886] [] dump_stack+0x2e/0x3e [ 0.339777] [] warn_slowpath_common+0x9d/0xcb [ 0.339777] [] warn_slowpath_common+0x9d/0xcb [ 0.340015] [] ? register_tracer+0x1e5/0x309 [ 0.340015] [] ? register_tracer+0x1e5/0x309 [ 0.341041] [] warn_slowpath_null+0x28/0x31 [ 0.341041] [] warn_slowpath_null+0x28/0x31 [ 0.342054] [] register_tracer+0x1e5/0x309 [ 0.342054] [] register_tracer+0x1e5/0x309 [ 0.343055] [] ? init_graph_debugfs+0x4b/0x4b [ 0.343055] [] ? init_graph_debugfs+0x4b/0x4b [ 0.344098] [] init_graph_trace+0x9c/0xa5 [ 0.344098] [] init_graph_trace+0x9c/0xa5 [ 0.345081] [] do_one_initcall+0x181/0x2b4 [ 0.345081] [] do_one_initcall+0x181/0x2b4 [ 0.346076] [] kernel_init_freeable+0x1d4/0x2ca [ 0.346076] [] kernel_init_freeable+0x1d4/0x2ca [ 0.347145] [] ? rest_init+0x150/0x150 [ 0.347145] [] ? rest_init+0x150/0x150 [ 0.348085] [] kernel_init+0x15/0x16e [ 0.348085] [] kernel_init+0x15/0x16e [ 0.349006] [] ret_from_fork+0x7a/0xb0 [ 0.349006] [] ret_from_fork+0x7a/0xb0 [ 0.350009] [] ? rest_init+0x150/0x150 [ 0.350009] [] ? rest_init+0x150/0x150 [ 0.351004] ---[ end trace 2345852dbca13bd5 ]--- [ 0.351004] ---[ end trace 2345852dbca13bd5 ]--- git bisect start a228a329e35a8738fd65282d38d6fb5bd01fc612 2ce7598c9a453e0acd0e07be7be3f5eb39608ebd -- git bisect bad bb03cddc19085b8e20bbc578f45cb8d964a6e858 # 09:56 0- 3 Merge 'nfs/linux-next' into devel-hourly-2014091007 git bisect good 0dc01152a147470985a0f388aa0f989c17e35936 # 10:07 20+ 0 Merge 'tip/perf/urgent' into devel-hourly-2014091007 git bisect good 0ace2a219b583ac59acebbb711d9aa9ed48e36d9 # 10:17 20+ 0 Merge 'kees/gcc-bug' into devel-hourly-2014091007 git bisect bad 24a777839a576d0b58ede0cfeac54700f91a6aa2 # 10:27 0- 2 Merge 'trace/ftrace/core' into devel-hourly-2014091007 git bisect good 428128377a8d63c026b3b3ae42fcacd2a3b9cf66 # 10:40 20+ 0 Merge 'arm-soc/renesas/soc3' into devel-hourly-2014091007 git bisect bad 4930d280aba6672eb44c7adcc0d47c0af8936c72 # 10:49 0- 20 ftrace: Grab any ops for a rec for enabled_functions output git bisect bad 87354059881ce9315181604dc17076c535f4d744 # 10:58 0- 20 ftrace: Add helper function ftrace_ops_get_func() git bisect good f1ff6348b30b3658d138f05643149706f99078ae # 11:11 20+ 0 ftrace: Add separate function for non recursive callbacks # first bad commit: [87354059881ce9315181604dc17076c535f4d744] ftrace: Add helper function ftrace_ops_get_func() git bisect good f1ff6348b30b3658d138f05643149706f99078ae # 11:16 60+ 0 ftrace: Add separate function for non recursive callbacks git bisect bad a228a329e35a8738fd65282d38d6fb5bd01fc612 # 11:16 0- 13 0day head guard for 'devel-hourly-2014091007' git bisect good 619df5d2870b80696fd32521376560e0a4223e44 # 11:32 60+ 0 Merge tag 'microblaze-3.17-rc5' of git://git.monstr.eu/linux-2.6-microblaze git bisect good ec86f34b250695542dd261d5dddf3e00bae4cc1c # 12:06 60+ 0 Add linux-next specific files for 20140909 This script may reproduce the error. ---------------------------------------------------------------------------- #!/bin/bash kernel=$1 initrd=yocto-minimal-x86_64.cgz wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -initrd $initrd -m 320 -smp 1 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -serial stdio -display none -monitor null ) append=( hung_task_panic=1 earlyprintk=ttyS0,115200 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw drbd.minor_count=8 ) "${kvm[@]}" --append "${append[*]}" ---------------------------------------------------------------------------- Thanks, Fengguang