Hi Ken, I got the below dmesg and the first bad commit is git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 12e364b9f08aa335dc7716ce74113e834c993765 Author: Ken Cox AuthorDate: Tue Mar 4 07:58:07 2014 -0600 Commit: Greg Kroah-Hartman CommitDate: Tue Mar 4 16:58:21 2014 -0800 staging: visorchipset driver to provide registration and other services The visorchipset module receives device creation and destruction events from the Command service partition of s-Par, as well as controlling registration of shared device drivers with the s-Par driver core. The events received are used to populate other s-Par modules with their assigned shared devices. Visorchipset is required for shared device drivers to function properly. Visorchipset also stores information for handling dump disk device creation during kdump. In operation, the visorchipset module processes device creation and destruction messages sent by s-Par's Command service partition through a channel. These messages result in creation (or destruction) of each virtual bus and virtual device. Each bus and device is also associated with a communication channel, which is used to communicate with one or more IO service partitions to perform device IO on behalf of the guest. Signed-off-by: Ken Cox Cc: Ben Romer Signed-off-by: Greg Kroah-Hartman +-------------------------------------------------+------------+---------------+ | | 12e364b9f0 | next-20140403 | +-------------------------------------------------+------------+---------------+ | boot_successes | 12 | 0 | | boot_failures | 18 | 3 | | invalid_opcode | 2 | | | RIP:visorchipset_init | 10 | 3 | | Kernel_panic-not_syncing:Fatal_exception | 10 | 3 | | backtrace:visorchipset_init | 10 | 3 | | backtrace:kernel_init_freeable | 10 | 3 | | loadedinvalid_opcode | 8 | | | BUG:kernel_early_hang_without_any_printk_output | 8 | | | loadedinvalid_opcode:PREEMPT_SMP | 0 | 2 | | invalid_opcode:PREEMPT_SMP | 0 | 1 | | early-boot-hang | 0 | 4 | +-------------------------------------------------+------------+---------------+ [ 24.135101] FPGA image file name: xlinx_fpga_firmware.bit [ 24.137595] GPIO INIT FAIL!! [ 24.141283] driver version 1.0.0.0 loaded [ 24.142539] chipset driver version 1.0.0.0 loadedinvalid opcode: 0000 [#1] PREEMPT SMP [ 24.144793] Modules linked in: [ 24.145303] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc5-00621-g12e364b #1 [ 24.145303] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 24.145303] task: ffff88001157a010 ti: ffff88001157c000 task.ti: ffff88001157c000 [ 24.145303] RIP: 0010:[] [] visorchipset_init+0x7b/0x8c5 [ 24.145303] RSP: 0000:ffff88001157de58 EFLAGS: 00000286 [ 24.145303] RAX: 000000000000070b RBX: 0000000000000004 RCX: 4000000000000000 [ 24.145303] RDX: a70aba7500000000 RSI: ffff88001157de5c RDI: ffff88001157de58 [ 24.145303] RBP: ffff88001157de90 R08: 0000000000000002 R09: ffff88001157de60 [ 24.145303] R10: ffff88001157de64 R11: 0000000000000000 R12: ffff88001157de5c [ 24.145303] R13: ffff88001157de60 R14: ffff88001157de64 R15: 0000000000000000 [ 24.145303] FS: 0000000000000000(0000) GS:ffff880012600000(0000) knlGS:0000000000000000 [ 24.145303] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 24.145303] CR2: ffff880002992000 CR3: 0000000001c07000 CR4: 00000000003006f0 [ 24.145303] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 24.145303] DR3: 0000000000000000 DR6: 0000000000000000 DR7: 0000000000000000 [ 24.145303] Stack: [ 24.145303] 00000800000306c1 078bfbf982d82203 ffffffff81e3709a 0000000000000000 [ 24.145303] 00000000000001df 0000000000000000 0000000000000000 ffff88001157df08 [ 24.145303] ffffffff810002b2 ffffffff810b2600 ffff88001157df08 ffffffff810b27db [ 24.145303] Call Trace: [ 24.145303] [] ? visorchannel_init+0x1d/0x1d [ 24.145303] [] do_one_initcall+0x8e/0x138 [ 24.145303] [] ? param_array_set+0xef/0xf5 [ 24.145303] [] ? parse_args+0x180/0x248 [ 24.145303] [] kernel_init_freeable+0x108/0x199 [ 24.145303] [] ? do_early_param+0x8a/0x8a [ 24.145303] [] ? rest_init+0xc2/0xc2 [ 24.145303] [] kernel_init+0x9/0xda [ 24.145303] [] ret_from_fork+0x7c/0xb0 [ 24.145303] [] ? rest_init+0xc2/0xc2 [ 24.145303] Code: 8d 65 cc 4c 8d 6d d0 4c 8d 75 d4 79 21 48 ba 00 00 00 00 75 ba 0a a7 48 b9 00 00 00 00 00 00 00 40 bb 04 00 00 00 b8 0b 07 00 00 <0f> 01 c1 8b 35 c2 c4 b4 00 48 c7 c7 f5 93 b4 81 31 c0 e8 3b 21 [ 24.145303] RIP [] visorchipset_init+0x7b/0x8c5 [ 24.145303] RSP [ 24.187247] ---[ end trace 62b5721899a66a6c ]--- [ 24.188157] Kernel panic - not syncing: Fatal exception git bisect start 4b22efdd5595f0acb48f02bf664a451ee98f9a2e v3.14 -- git bisect bad 850ba1df2c6aa754c9b2c8c23eac3161373d5492 # 16:46 0- 2 Merge remote-tracking branch 'samsung/for-next' git bisect good 62ff577fa2fec87edbf26f53e87210ba726d4d44 # 16:51 30+ 1 Merge tag 'edac_for_3.15' of git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp git bisect bad e6d9bfc63813882c896bf7ea6f6b14ca7b50b755 # 16:54 0- 15 Merge branch 'powernv-cpuidle' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc git bisect bad b33ce442993865180292df2a314ee5251ba38b50 # 16:56 0- 1 Merge branch 'for-3.15/drivers' of git://git.kernel.dk/linux-block git bisect bad c12e69c6aaf785fd307d05cb6f36ca0e7577ead7 # 16:59 1- 3 Merge tag 'staging-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging git bisect good c70929147a10fa4538886cb23b934b509c4c0e49 # 17:04 30+ 0 Merge tag 'sound-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound git bisect good 675c354a95d5375153b8bb80a0448cab916c7991 # 17:08 30+ 0 Merge tag 'char-misc-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc git bisect good 158e0d3621683ee0cdfeeba56f0e5ddd97ae984f # 17:12 30+ 0 Merge tag 'driver-core-3.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core git bisect bad 10933a9cfa258048813fd8a287be844d30711731 # 17:16 0- 11 staging: comedi: pcl818: exit interrupt quick when there is nothing to do git bisect good b14ec4bd3f576be2bb7dd4c69764a422d782e06d # 17:19 30+ 3 staging: comedi: pcl816: remove 'last_int_sub' from private data git bisect good f37e756033c918c198bc2df6dada27df25ce5a6f # 17:26 30+ 0 staging: rtl8187se: Convert Stats typedef into a struct git bisect good 99a1b98b32211713d8e544e9737a5fa20b73dce3 # 17:37 30+ 1 staging: comedi: ke_counter: fix ke_counter_insn_write() git bisect bad 9b073ac53eea902712f88388b42efcebce211bec # 17:40 0- 4 staging: dgap: Fix various previously missed checkpatch errors git bisect bad 6c76aab5bdec769ac05bb81dc6bb46cd5a253b4b # 17:43 1- 4 drivers: staging: rtl8187se: refactor/clean signal smoothing git bisect good 5c2f26def8f3bb252c32df4cbe0979140d8face6 # 17:50 30+ 0 staging: octeon-ethernet: add missing include git bisect bad 7b2a2d8383d08793aac3f157fa3f38ea90c5d3c0 # 17:53 0- 2 staging: visorchannelstub driver to provide channel support routines git bisect good 9d9baadd4069c77a97bf530abad9ddb74875fe76 # 18:03 30+ 0 staging: visorutil driver to provide common functionality to other s-Par drivers git bisect bad 12e364b9f08aa335dc7716ce74113e834c993765 # 18:28 7- 10 staging: visorchipset driver to provide registration and other services git bisect good e423812a9e430913e41c6565922142fe22f83ad7 # 18:34 43+ 0 staging: visorchannel module # first bad commit: [12e364b9f08aa335dc7716ce74113e834c993765] staging: visorchipset driver to provide registration and other services git bisect good e423812a9e430913e41c6565922142fe22f83ad7 # 18:39 129+ 0 staging: visorchannel module git bisect bad 4b22efdd5595f0acb48f02bf664a451ee98f9a2e # 18:40 0- 7 Add linux-next specific files for 20140403 git bisect bad 18a1a7a1d862ae0794a0179473d08a414dd49234 # 18:45 0- 1 Merge git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile git bisect bad abfcdfd63f0b68994bf3d0de84ddb7220d73a063 # 18:48 0- 16 Add linux-next specific files for 20140407 This script may reproduce the error. ----------------------------------------------------------------------------- #!/bin/bash kernel=$1 kvm=( qemu-system-x86_64 -cpu kvm64 -enable-kvm -kernel $kernel -smp 2 -m 256M -net nic,vlan=0,macaddr=00:00:00:00:00:00,model=virtio -net user,vlan=0 -net nic,vlan=1,model=e1000 -net user,vlan=1 -boot order=nc -no-reboot -watchdog i6300esb -serial stdio -display none -monitor null ) append=( debug sched_debug apic=debug ignore_loglevel sysrq_always_enabled panic=10 prompt_ramdisk=0 earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 vga=normal root=/dev/ram0 rw ) "${kvm[@]}" --append "${append[*]}" ----------------------------------------------------------------------------- Thanks, Fengguang