From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934528AbcATQI4 (ORCPT ); Wed, 20 Jan 2016 11:08:56 -0500 Received: from mail-pf0-f172.google.com ([209.85.192.172]:34691 "EHLO mail-pf0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933838AbcATQIx (ORCPT ); Wed, 20 Jan 2016 11:08:53 -0500 Subject: Re: tty: deadlock between n_tracerouter_receivebuf and flush_to_ldisc To: Peter Zijlstra , Dmitry Vyukov , Jiri Slaby , gnomes@lxorguk.ukuu.org.uk References: <20160120130201.GA1027@worktop> Cc: Greg Kroah-Hartman , LKML , J Freyensee , syzkaller , Kostya Serebryany , Alexander Potapenko , Sasha Levin , Eric Dumazet From: Peter Hurley Message-ID: <569FB10F.4080205@hurleysoftware.com> Date: Wed, 20 Jan 2016 08:08:47 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <20160120130201.GA1027@worktop> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/20/2016 05:02 AM, Peter Zijlstra wrote: > On Wed, Dec 30, 2015 at 11:44:01AM +0100, Dmitry Vyukov wrote: >> -> #3 (&buf->lock){+.+...}: >> [] lock_acquire+0x19f/0x3c0 kernel/locking/lockdep.c:3585 >> [< inline >] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:112 >> [] _raw_spin_lock_irqsave+0x50/0x70 kernel/locking/spinlock.c:159 >> [] tty_get_pgrp+0x20/0x80 drivers/tty/tty_io.c:2502 > > So in any recent code that I look at this function tries to acquire > tty->ctrl_lock, not buf->lock. Am I missing something ?! Yes. The tty locks were annotated with __lockfunc so were being elided from lockdep stacktraces. Greg has a patch in his queue from me that removes the __lockfunc annotation ("tty: Remove __lockfunc annotation from tty lock functions"). Unfortunately, I think syzkaller's post-processing stack trace isn't helping either, giving the impression that the stack is still inside tty_get_pgrp(). It's not. It's in pty_flush_buffer(), which is taking the 'other tty' buf->lock. Looks to me like the lock inversion is caused by the tty_driver_flush_buffer() in n_tracerouter_open()/_close(), but I need to look at this mess a little closer. Regards, Peter Hurley >> [] __isig+0x1a/0x50 drivers/tty/n_tty.c:1112 >> [] isig+0xae/0x2c0 drivers/tty/n_tty.c:1131 >> [] n_tty_receive_signal_char+0x22/0xf0 drivers/tty/n_tty.c:1243 >> [] n_tty_receive_char_special+0x1278/0x2bf0 drivers/tty/n_tty.c:1289 >> [< inline >] n_tty_receive_buf_fast drivers/tty/n_tty.c:1613 >> [< inline >] __receive_buf drivers/tty/n_tty.c:1647 >> [] n_tty_receive_buf_common+0x19d6/0x2450 drivers/tty/n_tty.c:1745 >> [] n_tty_receive_buf2+0x33/0x40 drivers/tty/n_tty.c:1780 >> [< inline >] receive_buf drivers/tty/tty_buffer.c:450 >> [] flush_to_ldisc+0x3bf/0x7f0 drivers/tty/tty_buffer.c:517 >> [] process_one_work+0x76c/0x13e0 kernel/workqueue.c:2030 >> [] worker_thread+0xe3/0xe90 kernel/workqueue.c:2162 >> [] kthread+0x23f/0x2d0 drivers/block/aoe/aoecmd.c:1303 >> [] ret_from_fork+0x3f/0x70 arch/x86/entry/entry_64.S:468