BTRFS deadlock when OOM killed processes using the file system

[Leo Unbekandt <leo.unbek@gmail.com>]

Hello everyone,

I've encountered what looks like a nasty bug which occures when the OOM
kills a process and that this process is working with the file system.
I've been able to reproduce this issue using docker, by limiting the
memory limits of process and make them crash when working.

(You can find more information there: my post on Docker bug tracker, but
it seems that it is more a BTRF-related issue
https://github.com/docker/docker/issues/15654#issuecomment-132384763)

Once the processes have been killed, a BTRFS kworker switched in
uninterruptible (state D) IO wait, and the file system is completely
unusable, no way to read or write from it which is pretty critical, in
my case Docker is not usable anymore as btrfs is the used storage backend.

OOM logs: http://pastebin.com/Lb4Gybay (Simple OOM logs)
BTRFS freeze and uninterruptible processes (plus Sysrq-w):
http://pastebin.com/qd1yB8u2

Hoping that I've got enough information, tell me if you need more. Now
I'm able to reproduce quite easily.

Thank you,

--
Leo Unbekandt
@Soulou

~# uname -a
Linux <hostname> 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12
14:09:17 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

~# btrfs --version
Btrfs v3.12

~# btrfs fi show
Label: none  uuid: c36b7b1d-910e-4af7-b671-b6d437c4b8d5
    Total devices 1 FS bytes used 34.03GiB
    devid    1 size 100.00GiB used 44.04GiB path /dev/dm-0

btrBtrfs v3.12

~# btrfs fi df /var/lib/docker
Data, single: total=35.01GiB, used=31.77GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=4.50GiB, used=2.25GiB
Metadata, single: total=8.00MiB, used=0.00
unknown, single: total=512.00MiB, used=0.00

~# dmesg.log

http://pastebin.com/S5DvJjxQ