From: Peter Hurley <peter@hurleysoftware.com>
To: Peter Zijlstra <peterz@infradead.org>,
Dmitry Vyukov <dvyukov@google.com>, Jiri Slaby <jslaby@suse.com>,
gnomes@lxorguk.ukuu.org.uk
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
LKML <linux-kernel@vger.kernel.org>,
J Freyensee <james_p_freyensee@linux.intel.com>,
syzkaller <syzkaller@googlegroups.com>,
Kostya Serebryany <kcc@google.com>,
Alexander Potapenko <glider@google.com>,
Sasha Levin <sasha.levin@oracle.com>,
Eric Dumazet <edumazet@google.com>
Subject: Re: tty: deadlock between n_tracerouter_receivebuf and flush_to_ldisc
Date: Wed, 20 Jan 2016 08:08:47 -0800 [thread overview]
Message-ID: <569FB10F.4080205@hurleysoftware.com> (raw)
In-Reply-To: <20160120130201.GA1027@worktop>
On 01/20/2016 05:02 AM, Peter Zijlstra wrote:
> On Wed, Dec 30, 2015 at 11:44:01AM +0100, Dmitry Vyukov wrote:
>> -> #3 (&buf->lock){+.+...}:
>> [<ffffffff813f0acf>] lock_acquire+0x19f/0x3c0 kernel/locking/lockdep.c:3585
>> [< inline >] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:112
>> [<ffffffff85c8e790>] _raw_spin_lock_irqsave+0x50/0x70 kernel/locking/spinlock.c:159
>> [<ffffffff82b8c050>] tty_get_pgrp+0x20/0x80 drivers/tty/tty_io.c:2502
>
> So in any recent code that I look at this function tries to acquire
> tty->ctrl_lock, not buf->lock. Am I missing something ?!
Yes.
The tty locks were annotated with __lockfunc so were being elided from lockdep
stacktraces. Greg has a patch in his queue from me that removes the __lockfunc
annotation ("tty: Remove __lockfunc annotation from tty lock functions").
Unfortunately, I think syzkaller's post-processing stack trace isn't helping
either, giving the impression that the stack is still inside tty_get_pgrp().
It's not.
It's in pty_flush_buffer(), which is taking the 'other tty' buf->lock.
Looks to me like the lock inversion is caused by the tty_driver_flush_buffer()
in n_tracerouter_open()/_close(), but I need to look at this mess a little
closer.
Regards,
Peter Hurley
>> [<ffffffff82b9a09a>] __isig+0x1a/0x50 drivers/tty/n_tty.c:1112
>> [<ffffffff82b9c16e>] isig+0xae/0x2c0 drivers/tty/n_tty.c:1131
>> [<ffffffff82b9ef02>] n_tty_receive_signal_char+0x22/0xf0 drivers/tty/n_tty.c:1243
>> [<ffffffff82ba4958>] n_tty_receive_char_special+0x1278/0x2bf0 drivers/tty/n_tty.c:1289
>> [< inline >] n_tty_receive_buf_fast drivers/tty/n_tty.c:1613
>> [< inline >] __receive_buf drivers/tty/n_tty.c:1647
>> [<ffffffff82ba7ca6>] n_tty_receive_buf_common+0x19d6/0x2450 drivers/tty/n_tty.c:1745
>> [<ffffffff82ba8753>] n_tty_receive_buf2+0x33/0x40 drivers/tty/n_tty.c:1780
>> [< inline >] receive_buf drivers/tty/tty_buffer.c:450
>> [<ffffffff82bafa6f>] flush_to_ldisc+0x3bf/0x7f0 drivers/tty/tty_buffer.c:517
>> [<ffffffff8133833c>] process_one_work+0x76c/0x13e0 kernel/workqueue.c:2030
>> [<ffffffff81339093>] worker_thread+0xe3/0xe90 kernel/workqueue.c:2162
>> [<ffffffff8134b63f>] kthread+0x23f/0x2d0 drivers/block/aoe/aoecmd.c:1303
>> [<ffffffff85c8eeef>] ret_from_fork+0x3f/0x70 arch/x86/entry/entry_64.S:468
next prev parent reply other threads:[~2016-01-20 16:08 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-12-30 10:44 tty: deadlock between n_tracerouter_receivebuf and flush_to_ldisc Dmitry Vyukov
2016-01-15 7:51 ` Dmitry Vyukov
2016-01-15 16:33 ` One Thousand Gnomes
2016-01-15 17:22 ` Dmitry Vyukov
2016-01-20 9:36 ` Dmitry Vyukov
2016-01-20 11:44 ` Peter Zijlstra
2016-01-20 11:54 ` Dmitry Vyukov
2016-01-20 12:07 ` Peter Zijlstra
2016-01-20 14:58 ` One Thousand Gnomes
2016-01-20 15:16 ` Dmitry Vyukov
2016-01-20 16:32 ` Peter Zijlstra
2016-01-20 2:09 ` J Freyensee
2016-01-20 12:47 ` Jiri Slaby
2016-01-20 13:02 ` Peter Zijlstra
2016-01-20 13:07 ` Dmitry Vyukov
2016-01-20 16:08 ` Peter Hurley [this message]
2016-01-20 20:47 ` Peter Hurley
2016-01-21 10:06 ` Dmitry Vyukov
2016-01-21 10:20 ` Peter Zijlstra
2016-01-21 17:51 ` Peter Hurley
2016-01-22 14:10 ` Dmitry Vyukov
2016-01-25 16:56 ` Peter Hurley
2016-01-21 17:43 ` Peter Hurley
2016-02-03 4:24 ` Peter Hurley
2016-02-03 17:32 ` Dmitry Vyukov
2016-02-03 19:09 ` Peter Hurley
2016-02-04 12:39 ` Dmitry Vyukov
2016-02-04 13:17 ` Dmitry Vyukov
2016-02-04 18:46 ` Peter Hurley
2016-02-04 18:48 ` Dmitry Vyukov
2016-02-05 21:22 ` Dmitry Vyukov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=569FB10F.4080205@hurleysoftware.com \
--to=peter@hurleysoftware.com \
--cc=dvyukov@google.com \
--cc=edumazet@google.com \
--cc=glider@google.com \
--cc=gnomes@lxorguk.ukuu.org.uk \
--cc=gregkh@linuxfoundation.org \
--cc=james_p_freyensee@linux.intel.com \
--cc=jslaby@suse.com \
--cc=kcc@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=sasha.levin@oracle.com \
--cc=syzkaller@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.