Re: kernel crashes with btrfs and busy database IO - how to debug?

[Tomasz Chmielewski <tch@virtall.com>]

Hello,

Do you want me to produce one more crash / hang? I had to restart the 
hanged server (via echo b > /proc/sysrq-trigger).


Tomasz


On 2015-06-15 17:10, Qu Wenruo wrote:
> Now we can get the full backtrace.
> That's a step forward
> 
> [45705.854778] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000008
> [45705.854824] IP: [<ffffffffc0158b8e>]
> btrfs_wait_pending_ordered+0x5e/0x110 [btrfs]
> [45705.855615] Call Trace:
> [45705.855637]  [<ffffffffc015addb>]
> btrfs_commit_transaction+0x40b/0xb60 [btrfs]
> [45705.855671]  [<ffffffff810c0700>] ? 
> prepare_to_wait_event+0x100/0x100
> [45705.855698]  [<ffffffffc0171973>] btrfs_sync_file+0x313/0x380 
> [btrfs]
> [45705.855721]  [<ffffffff81236c26>] vfs_fsync_range+0x46/0xc0
> [45705.855740]  [<ffffffff81236cbc>] vfs_fsync+0x1c/0x20
> [45705.855758]  [<ffffffff81236cf8>] do_fsync+0x38/0x70
> [45705.855777]  [<ffffffff812370d0>] SyS_fsync+0x10/0x20
> [45705.855796]  [<ffffffff8180cbb2>] system_call_fastpath+0x16/0x75
> 
> Also the hang seems to be highly related to the bug,
> would you please send a new mail reporting the hang?
> 
> Thanks,
> Qu
> 
> 在 2015年06月14日 15:58, Tomasz Chmielewski 写道:
>> On 2015-06-14 09:30, Tomasz Chmielewski wrote:
>>> On 2015-06-13 08:23, Tomasz Chmielewski wrote:
>>> 
>>>> I did get it from /var/crash/ though - is it more useful? I don't 
>>>> have
>>>> vmlinux for this kernel though, but have just built 4.1-rc7 with the
>>>> same config, can try to get the crash there.
>>> 
>>> I've uploaded a crash dump and vmlinux here:
>>> 
>>> http://www.virtall.com/files/temp/201506132321/
>>> 
>>> Let me know if it's anything useful or if you need more info.
>> 
>> I've tried running it the same procedure to get one more crash, but it
>> didn't crash this time.
>> 
>> Instead, btrfs is hanged on any writes - any processes trying to write
>> get into D state and never return; there is no write activity when
>> checking for example with iostat. "sync" command does not return.
>> 
>> Reads from this btrfs filesystem are OK.
>> 
>> I've uploaded the output of "echo w > /proc/sysrq-trigger" here:
>> 
>> http://www.virtall.com/files/temp/dmesg.txt
>> 
>> 
>> Tomasz Chmielewski
>> 
Hello,

sorry, not sure what you mean.

Do you want me to produce one more crash / or hang?