Christoph, FYI, here is one more back trace triggered by commit a25982c15ae5298520033cd5848a82adf6a97376 Author: Christoph Lameter AuthorDate: Thu Mar 6 11:05:21 2014 +1100 percpu: add preemption checks to __this_cpu ops +--------------------------------------------------------------+------------+------------+---------------+ | | 26d02cfa38 | a25982c15a | next-20140306 | +--------------------------------------------------------------+------------+------------+---------------+ | boot_successes | 73 | 0 | 0 | | boot_failures | 7 | 20 | 19 | | BUG:unable_to_handle_kernel_paging_request | 1 | | | | Oops:PREEMPT_SMP | 1 | | | | EIP_is_at_follow_page_mask | 1 | | | | backtrace:__mm_populate | 1 | | | | backtrace:SyS_mlockall | 1 | | | | WARNING:CPU:PID:at_arch/x86/kernel/cpu/amd.c:init_amd() | 4 | | | | backtrace:init_amd | 4 | | | | backtrace:warn_slowpath_fmt | 4 | | | | BUG:kernel_boot_hang | 2 | | | | BUG:using__this_cpu_add()in_preemptible_code:init | 0 | 20 | 19 | | BUG:using__this_cpu_add()in_preemptible_code:udevd | 0 | 14 | 11 | | backtrace:do_execve | 0 | 20 | 19 | | backtrace:run_init_process | 0 | 20 | 19 | | backtrace:user_path_at | 0 | 20 | 19 | | backtrace:SyS_faccessat | 0 | 20 | 19 | | backtrace:SyS_access | 0 | 20 | 19 | | BUG:using__this_cpu_add()in_preemptible_code:mountall | 0 | 1 | | | backtrace:SyS_execve | 0 | 1 | 3 | | backtrace:do_sys_open | 0 | 8 | 9 | | backtrace:SyS_open | 0 | 8 | 9 | | backtrace:do_group_exit | 0 | 2 | 4 | | backtrace:SyS_exit_group | 0 | 2 | 4 | | BUG:using__this_cpu_add()in_preemptible_code:bootlogd | 0 | 1 | 4 | | BUG:using__this_cpu_add()in_p | 0 | 1 | | | BUG:using__this_cpu_add()in_preemptible_code:udevadm | 0 | 0 | 5 | | BUG:using__this_cpu_add()in_preemptible_code:-trinity | 0 | 0 | 2 | | BUG:using__this_cpu_add()in_preemptible_code:upstart-udev-br | 0 | 0 | 2 | | BUG:using__this_cpu_add()in_preemptible_code:sh | 0 | 0 | 1 | | BUG:using__this_cpu_add()in_preemptible_code:update-modules | 0 | 0 | 2 | | BUG:using__this_cpu_add()in_preemptible_code:trinity | 0 | 0 | 1 | | backtrace:vfs_stat | 0 | 0 | 1 | | backtrace:SyS_stat64 | 0 | 0 | 1 | +--------------------------------------------------------------+------------+------------+---------------+ [ 4.164485] Write protecting the kernel text: 9076k [ 4.166234] Write protecting the kernel read-only data: 2904k [ 4.167832] NX-protecting the kernel data: 5260k [ 4.169996] BUG: using __this_cpu_add() in preemptible [00000000] code: init/1 [ 4.172629] caller is __this_cpu_preempt_check+0x2d/0x40 [ 4.174259] CPU: 1 PID: 1 Comm: init Not tainted 3.14.0-rc5-next-20140306-06952-g0ffb2fe #3 [ 4.177120] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 4.178635] 00000000 00000000 cf477d3c c18d4eeb 00000001 cf477d60 c12e80b9 c1ace2c8 [ 4.182201] cf477d78 00000000 cf4782a4 00000001 cf477d78 cfc455c0 cf477da4 c12e810d [ 4.186749] cf477d78 00000028 c1acdb6d c1b42ef1 68745f5f 635f7369 615f7570 29286464 [ 4.190306] Call Trace: [ 4.191374] [] dump_stack+0x4b/0x75 [ 4.192838] [] check_preemption_disabled+0xc9/0xe0 [ 4.194598] [] __this_cpu_preempt_check+0x2d/0x40 [ 4.196363] [] ? __split_vma.isra.29+0x23/0x150 [ 4.197913] [] __slab_free+0x1e/0x43c [ 4.199341] [] ? __remove_shared_vm_struct+0x35/0x50 [ 4.201173] [] ? free_pgtables+0x80/0xb0 [ 4.202712] [] ? get_parent_ip+0xb/0x40 [ 4.204237] [] ? preempt_count_sub+0x5f/0xe0 [ 4.205809] [] kmem_cache_free+0x138/0x190 [ 4.207394] [] ? remove_vma+0x45/0x50 [ 4.208936] [] ? remove_vma+0x45/0x50 [ 4.210450] [] remove_vma+0x45/0x50 [ 4.212026] [] do_munmap+0x1f5/0x2a0 [ 4.213596] [] vm_munmap+0x32/0x50 [ 4.215039] [] load_elf_binary+0x10c2/0x11d0 [ 4.216750] [] ? preempt_count_sub+0x5f/0xe0 [ 4.218388] [] search_binary_handler+0x62/0x170 [ 4.220182] [] ? __task_pid_nr_ns+0x5e/0xa0 [ 4.221864] [] do_execve+0x418/0x570 [ 4.223313] [] run_init_process+0x1c/0x20 [ 4.224943] [] kernel_init+0x37/0xe0 [ 4.226498] [] ret_from_kernel_thread+0x1b/0x28 [ 4.228310] [] ? rest_init+0x70/0x70 git bisect start 0ffb2fe7b9c30082876fa3a17da018bf0632cf03 09de192242d86ede449146ac423effca53bc1810 -- git bisect bad 439cde1f3d8230f858c0a55902381d24cd7a2df2 # 13:05 0- 4 Merge branch 'akpm/master' git bisect good ce34778eb4d9fa0aaaeae76c1068026ee5255fd7 # 13:11 20+ 0 Merge branch 'akpm-current/current' git bisect good 117314461e82adabb6b78dcb03259da75b50833b # 13:23 20+ 1 mm: don't implictly include linux/mm.h in linux/sched.h git bisect bad 9d58cf7a388c1b4a06cb335980eb1bb1352ba876 # 13:41 0- 1 scheduler: replace __get_cpu_var with this_cpu_ptr git bisect bad b2ff2cfebad92bebfca28bad650045821ed270c4 # 13:49 0- 1 tracing: replace __get_cpu_var uses with this_cpu_ptr git bisect good 6fb77936481538ba05726345d86af4b39128a8a9 # 14:41 20+ 0 arm: move arm_dma_limit to setup_dma_zone git bisect good d1cea07efbfa2723ca1b83d3a69d1f1000896d4d # 15:06 20+ 1 modules: use raw_cpu_write for initialization of per cpu refcount. git bisect bad a25982c15ae5298520033cd5848a82adf6a97376 # 15:16 0- 8 percpu: add preemption checks to __this_cpu ops git bisect good 26d02cfa385067c39c0407b022c01a011c2fbb51 # 15:43 20+ 1 net: replace __this_cpu_inc in route.c with raw_cpu_inc # first bad commit: [a25982c15ae5298520033cd5848a82adf6a97376] percpu: add preemption checks to __this_cpu ops git bisect good 26d02cfa385067c39c0407b022c01a011c2fbb51 # 16:02 60+ 7 net: replace __this_cpu_inc in route.c with raw_cpu_inc git bisect bad 0ffb2fe7b9c30082876fa3a17da018bf0632cf03 # 16:03 0- 19 Add linux-next specific files for 20140306 git bisect good cc0ca6b3d97f5006e536a2601a3569baed920397 # 16:31 60+ 1 Revert "percpu: add preemption checks to __this_cpu ops" git bisect good c3bebc71c4bcdafa24b506adf0c1de3c1f77e2e0 # 17:02 60+ 0 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net git bisect good 1b0a7e3263168a06d3858798e48c5a21d1c78d3c # 18:40 60+ 2 Add linux-next specific files for 20140307 Thanks, Fengguang