[Bug 89131] New: Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[Bug 89131] New: Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

            Bug ID: 89131
           Summary: Hangs when checking torrent through libtorrent; kernel
                    BUG at mm/iov_iter.c:219!
           Product: File System
           Version: 2.5
    Kernel Version: 3.17.3
          Hardware: x86-64
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: blocking
          Priority: P1
         Component: ext4
          Assignee: fs_ext4@kernel-bugs.osdl.org
          Reporter: kokoko3k@gmail.com
        Regression: No

Created attachment 159351
  --> https://bugzilla.kernel.org/attachment.cgi?id=159351&action=edit
Backtraces from both the systems

When i use qbittorrent (libtorrent) by explicitely disable the os cache and do
the hash-check on a specific torrent data (it does not happen on all of the
torrents), linux hangs leaving me no choice but to reboot somehow.

I managed to get two backtraces from completely different systems (an e7300
desktop and an ivybridge laptop), they're attached.

As a reference, i started to talk about the issue here:
https://github.com/qbittorrent/qBittorrent/issues/2200#issuecomment-64951598

...seems the issue is given by the use of the O_DIRECT flag

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

Lubos Dolezel <lubosd@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lubosd@gmail.com

--- Comment #1 from Lubos Dolezel <lubosd@gmail.com> ---
Created attachment 160921
  --> https://bugzilla.kernel.org/attachment.cgi?id=160921&action=edit
Kernel BUG dump

I'm seeing the same kind of crash, 5 times just yesterday!

I'm using libtorrent to download a huge torrent (almost 50GB) with many files
inside (BluRay dump). I haven't seen it with other torrents.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #2 from kokoko3k@gmail.com <kokoko3k@gmail.com> ---
Depending on the client you're using, you should be able to enable the kernel
cache so that it doesn't crash anymore.
Just a workaround, of course.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #3 from Lubos Dolezel <lubosd@gmail.com> ---
Yep, this workaround helps.

As a little side note, using direct I/O with ext4 has resulted in many other
crashes on my system including softlockups and various other oopses. ext4's
direct involvement could be seen in ~50% of them.

So this "kernel BUG" is not the only possible outcome.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #4 from kokoko3k@gmail.com <kokoko3k@gmail.com> ---
I wasn't able to reproduce the problem using dd with conv=direct
How did you do that?
It may be handy for developers to have a quicker way to test the issue.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #5 from Lubos Dolezel <lubosd@gmail.com> ---
Sorry for misunderstanding, I was only able to reproduce it with libtorrent.

Torrent checking resulted in aforementioned BUGs, whereas torrent downloading
(i.e. write operations) resulted in much more serious problems - e.g. lockups
in kworker in kthread_data().

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

Theodore Tso <tytso@mit.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tytso@mit.edu

--- Comment #6 from Theodore Tso <tytso@mit.edu> ---
Couple of questions.   First of all, is there any chance you could give a try
with 3.18.0, and see if it fails there?

Also, can you try to get line numbers out of the stack dump?   

Also, how long does it take before you trigger the failure?  I'm in particular
interested in seeing where this comes from:

[ 4244.680376]  [<ffffffff81203313>] __blockdev_direct_IO+0x25e3/0x3630

Part of the problem here is that fs/direct_io.c has a bunch of inline
functions, so it becomes tricky to figure out where the call to advance_iovec
comes from.   This looks like a bug in the core direct I/O code, but we do a
fair amount of regression testing and this is a new one; it's a bit surprising
no once else found this when running xfstests.

To get the line numbers, compile the kernel with

CONFIG_DEBUG_INFO=y

(You can also set CONFIG_DEBUG_INFO_REDUCED to reduce the size of your kernel
build.)

Then use the addr2line program to translate the address to a line number, i.e.:

addr2line -e /path/to/kernel/build/vmlinux -i -a ffffffff81203313

Finally, can you send me an output of dumpe2fs -h of your ext3/ext4 file
system, just so I can understand what file system features are enabled with
your file system.

Thanks!!

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #7 from Lubos Dolezel <lubosd@gmail.com> ---
I'd love to test it with 3.18.0, but I cannot due to #89881. But I'll try
CONFIG_DEBUG_INFO=y.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #8 from Lubos Dolezel <lubosd@gmail.com> ---
The kernel BUG typically happened pretty quickly when doing torrent checking
(reading).

Other symptoms (more fatal crashes) when downloading and uploading happened
within minutes/hours, but that's understandable, because the I/O is lower in
this case.

Note that I haven't had this issue for weeks until I started downloading a
particular torrent.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #9 from Lubos Dolezel <lubosd@gmail.com> ---
Created attachment 161341
  --> https://bugzilla.kernel.org/attachment.cgi?id=161341&action=edit
dumpe2fs

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #10 from kokoko3k@gmail.com <kokoko3k@gmail.com> ---
Unfortunately the crash still happens on 3.18.2

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 89131] Hangs when checking torrent through libtorrent; kernel BUG at mm/iov_iter.c:219!

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=89131

--- Comment #11 from Theodore Tso <tytso@mit.edu> ---
Created attachment 163831
  --> https://bugzilla.kernel.org/attachment.cgi?id=163831&action=edit
debugging patch (should apply for 3.17 and 3.18)

This BUG_ON is happening from the mm part of the direct_io code, which is
supposed to be file system independent, and so I don't know that half of the
DIO code as well.   Can you try applying the attached patch to see if it gets
us any useful information when the bug triggers?

Also, at this point it would be *really* useful if we could get a reproduction
case that doesn't require using rtorrent -- i.e., something that uses just dd,
or maybe a simple C program.   Would it be possible to run the a tty-mode
torrent on a serial console under strace, or some such, so we can get figure
out what the system calls were that apparently triggered the BUG_ON?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.