From: Qian Cai <cai@lca.pw>
To: Ming Lei <tom.leiming@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>,
Al Viro <viro@zeniv.linux.org.uk>,
syzkaller-bugs@googlegroups.com,
syzbot <syzbot+61acc40a49a3e46e25ea@syzkaller.appspotmail.com>,
Linux FS Devel <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Ming Lei <ming.lei@canonical.com>,
paulmck@kernel.org
Subject: Re: splice: infinite busy loop lockup bug
Date: Tue, 1 Sep 2020 10:22:51 -0400 [thread overview]
Message-ID: <20200901142251.GA16065@lca.pw> (raw)
In-Reply-To: <CACVXFVNy+qKeSytvDduCuH3HV02mB7i88P27Ou+h=PC22hqwHw@mail.gmail.com>
On Tue, Sep 01, 2020 at 09:10:05AM +0800, Ming Lei wrote:
> On Tue, Sep 1, 2020 at 8:53 AM Qian Cai <cai@lca.pw> wrote:
> >
> > On Fri, Aug 07, 2020 at 09:34:08PM +0900, Tetsuo Handa wrote:
> > > On 2020/08/07 21:27, Al Viro wrote:
> > > > On Fri, Aug 07, 2020 at 07:35:08PM +0900, Tetsuo Handa wrote:
> > > >> syzbot is reporting hung task at pipe_release() [1], for for_each_bvec() from
> > > >> iterate_bvec() from iterate_all_kinds() from iov_iter_alignment() from
> > > >> ext4_unaligned_io() from ext4_dio_write_iter() from ext4_file_write_iter() from
> > > >> call_write_iter() from do_iter_readv_writev() from do_iter_write() from
> > > >> vfs_iter_write() from iter_file_splice_write() falls into infinite busy loop
> > > >> with pipe->mutex held.
> > > >>
> > > >> The reason of falling into infinite busy loop is that iter_file_splice_write()
> > > >> for some reason generates "struct bio_vec" entry with .bv_len=0 and .bv_offset=0
> > > >> while for_each_bvec() cannot handle .bv_len == 0.
> > > >
> > > > broken in 1bdc76aea115 "iov_iter: use bvec iterator to implement iterate_bvec()",
> > > > unless I'm misreading it...
> >
> > I have been chasing something similar for a while as in,
> >
> > https://lore.kernel.org/linux-fsdevel/89F418A9-EB20-48CB-9AE0-52C700E6BB74@lca.pw/
> >
> > In my case, it seems the endless loop happens in iterate_iovec() instead where
> > I put a debug patch here,
> >
> > --- a/lib/iov_iter.c
> > +++ b/lib/iov_iter.c
> > @@ -33,6 +33,7 @@
> > if (unlikely(!__v.iov_len)) \
> > continue; \
> > __v.iov_base = __p->iov_base; \
> > + printk_ratelimited("ITER_IOVEC left = %zu, n = %zu\n", left, n); \
> > left = (STEP); \
> > __v.iov_len -= left; \
> > skip = __v.iov_len; \
> >
> > and end up seeing overflows ("n" supposes to be less than PAGE_SIZE) before the
> > soft-lockups and a dead system,
> >
> > [ 4300.249180][T470195] ITER_IOVEC left = 0, n = 48566423
> >
> > Thoughts?
>
> Does the following patch make a difference for you?
>
> https://lore.kernel.org/linux-block/20200817100055.2495905-1-ming.lei@redhat.com/
Yes, it does. I could no longer be able to reproduce the soft lockups with the
patch applied.
next prev parent reply other threads:[~2020-09-01 14:24 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-02 18:26 INFO: task hung in pipe_release (2) syzbot
2020-08-07 10:35 ` splice: infinite busy loop lockup bug Tetsuo Handa
2020-08-07 12:27 ` Al Viro
2020-08-07 12:34 ` Tetsuo Handa
2020-09-01 0:51 ` Qian Cai
2020-09-01 1:09 ` Al Viro
2020-09-01 1:47 ` Qian Cai
2020-09-01 3:32 ` Qian Cai
2020-09-01 4:07 ` Al Viro
2020-09-01 1:10 ` Ming Lei
2020-09-01 14:22 ` Qian Cai [this message]
2020-08-07 12:38 ` Al Viro
2020-08-07 13:41 ` Ming Lei
2020-08-07 14:11 ` Matthew Wilcox
2020-08-07 15:11 ` Tetsuo Handa
2020-08-09 2:31 ` Ming Lei
2020-08-09 2:49 ` Ming Lei
2020-08-07 14:17 ` Tetsuo Handa
2020-08-13 3:57 ` INFO: task hung in pipe_release (2) syzbot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200901142251.GA16065@lca.pw \
--to=cai@lca.pw \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ming.lei@canonical.com \
--cc=paulmck@kernel.org \
--cc=penguin-kernel@i-love.sakura.ne.jp \
--cc=syzbot+61acc40a49a3e46e25ea@syzkaller.appspotmail.com \
--cc=syzkaller-bugs@googlegroups.com \
--cc=tom.leiming@gmail.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox