Hi Jens, I got the below dmesg and the first bad commit is commit f2298c0403b0dfcaef637eba0c02c4a06d7a25ab Author: Jens Axboe CommitDate: Fri Oct 25 11:56:00 2013 +0100 null_blk: multi queue aware block test driver A driver that simply completes IO it receives, it does no transfers. Written to fascilitate testing of the blk-mq code. It supports various module options to use either bio queueing, rq queueing, or mq mode. Signed-off-by: Jens Axboe [ 18.158432] VFS: could not find a valid V7 on nullb0. [ 18.159280] LogFS: Start mount 1 [ 18.159732] LogFS: Start unmounting [ 18.160368] INFO: trying to register non-static key. [ 18.160986] the code is fine but needs lockdep annotation. [ 18.161675] turning off the locking correctness validator. [ 18.162341] CPU: 0 PID: 634 Comm: mount Not tainted 3.14.0-rc2-wl-01363-gf8a1366 #2 [ 18.163266] 0000000000000000 ffff88000b839c00 ffffffff827922a5 ffff88000b839cb8 [ 18.163516] ffffffff810c0202 ffff88000bac1008 ffff88000b839c98 ffff88000b839c58 [ 18.163516] 0000000000000002 ffff88000bb4f000 ffff880000000000 ffffffff00000001 [ 18.163516] Call Trace: [ 18.163516] [] dump_stack+0x19/0x1b [ 18.163516] [] __lock_acquire+0x252/0x1d80 [ 18.163516] [] ? writeback_inodes_sb_nr+0xc0/0xe0 [ 18.163516] [] ? logfs_get_wblocks+0x76/0x90 [ 18.163516] [] lock_acquire+0x70/0x90 [ 18.163516] [] ? logfs_get_wblocks+0x76/0x90 [ 18.163516] [] mutex_lock_nested+0x61/0x380 [ 18.163516] [] ? logfs_get_wblocks+0x76/0x90 [ 18.163516] [] logfs_get_wblocks+0x76/0x90 [ 18.163516] [] logfs_sync_fs+0x14/0x30 [ 18.163516] [] sync_filesystem+0x59/0xb0 [ 18.163516] [] logfs_kill_sb+0x27/0xf0 [ 18.163516] [] deactivate_locked_super+0x2d/0x50 [ 18.163516] [] logfs_mount+0x763/0x7e0 [ 18.163516] [] mount_fs+0x10/0xe0 [ 18.163516] [] vfs_kern_mount+0x69/0x140 [ 18.163516] [] ? _raw_read_unlock+0x2c/0x50 [ 18.163516] [] do_mount+0x934/0xb90 [ 18.163516] [] ? might_fault+0x52/0xb0 [ 18.163516] [] ? strndup_user+0x35/0x50 [ 18.163516] [] SyS_mount+0x8c/0xd0 [ 18.163516] [] system_call_fastpath+0x1a/0x1f git bisect start v3.13 v3.12 -- git bisect bad 3bad8bb5cd3048a67df43ac6b1e2f191f19d9ff0 # 21:46 1- 2 Merge branch 'for-next' of git://git.samba.org/sfrench/cifs-2.6 git bisect bad 049ffa8ab33a63b3bff672d1a0ee6a35ad253fe8 # 22:16 0- 1 Merge branch 'drm-next' of git://people.freedesktop.org/~airlied/linux git bisect good 7e238a2ecd117e16b154e1b0ed77906596600ff5 # 22:48 40+ 6 Merge tag 'hwmon-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging git bisect bad 0910c0bdf7c291a41bc21e40a97389c9d4c1960d # 23:07 1- 1 Merge branch 'for-3.13/core' of git://git.kernel.dk/linux-block git bisect good a9986464564609dd0962e6023513f7d3d313dc80 # 23:45 40+ 0 Merge branch 'for-3.13' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup git bisect good f0230294271f511b41797305b685365a9e569a09 # 00:20 40+ 4 Merge tag 'dlm-3.13' of git://git.kernel.org/pub/scm/linux/kernel/git/teigland/linux-dlm git bisect good 42a2d923cc349583ebf6fdd52a7d35e1c2f7e6bd # 00:49 40+ 2 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next git bisect good f47671e2d861a2093179cd64dda22016664b2015 # 01:08 40+ 4 Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm git bisect good 2821fe6b00a1e902fd399bb4b7e40bc3041f4d44 # 01:25 40+ 24 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect good 8616ebb16bcef312024b9d28719f9bf5c5c3aafb # 01:48 40+ 0 block: replace IS_ERR and PTR_ERR with PTR_ERR_OR_ZERO git bisect good 1dddc01af0d42b21058e0cb9c1ca9e8d5204d9b0 # 02:16 40+ 2 percpu_ida: add an API to return free tags git bisect bad 3228f48be2d19b2dd90db96ec16a40187a2946f3 # 02:24 13- 3 blk-mq: fix for flush deadlock git bisect bad f2298c0403b0dfcaef637eba0c02c4a06d7a25ab # 02:47 8- 7 null_blk: multi queue aware block test driver git bisect good 320ae51feed5c2f13664aa05a76bec198967e04d # 03:17 63+ 4 blk-mq: new multi-queue block IO queueing mechanism # first bad commit: [f2298c0403b0dfcaef637eba0c02c4a06d7a25ab] null_blk: multi queue aware block test driver git bisect good 320ae51feed5c2f13664aa05a76bec198967e04d # 03:34 189+ 28 blk-mq: new multi-queue block IO queueing mechanism git bisect bad f8a13667ab9da01df61cc915bc4c56bb652f6bc4 # 03:35 0- 13 0day head guard for 'devel-hourly-2014021108' git bisect bad 6792dfe383dd20ed270da198aa0676bac47245b4 # 03:41 8- 3 Merge branch 'akpm' (patches from Andrew Morton) git bisect bad ea2ad567c12336608ae96f244f3dc3932c1fcd1b # 03:54 7- 1 Add linux-next specific files for 20140211 Thanks, Fengguang