From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756278AbcARSs0 (ORCPT ); Mon, 18 Jan 2016 13:48:26 -0500 Received: from e06smtp13.uk.ibm.com ([195.75.94.109]:38744 "EHLO e06smtp13.uk.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756215AbcARSsX (ORCPT ); Mon, 18 Jan 2016 13:48:23 -0500 X-IBM-Helo: d06dlp01.portsmouth.uk.ibm.com X-IBM-MailFrom: borntraeger@de.ibm.com X-IBM-RcptTo: kvm@vger.kernel.org;linux-kernel@vger.kernel.org;linux-s390@vger.kernel.org Subject: Re: regression 4.4: deadlock in with cgroup percpu_rwsem To: Peter Zijlstra References: <56978452.6010606@de.ibm.com> <20160114195630.GA3520@mtj.duckdns.org> <5698A023.9070703@de.ibm.com> <56990C9E.7020801@de.ibm.com> <20160118183205.GW6357@twins.programming.kicks-ass.net> Cc: Tejun Heo , "linux-kernel@vger.kernel.org >> Linux Kernel Mailing List" , linux-s390 , KVM list , Oleg Nesterov , "Paul E. McKenney" From: Christian Borntraeger Message-ID: <569D3370.6040503@de.ibm.com> Date: Mon, 18 Jan 2016 19:48:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <20160118183205.GW6357@twins.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16011818-0013-0000-0000-00000853AB78 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/18/2016 07:32 PM, Peter Zijlstra wrote: > On Fri, Jan 15, 2016 at 04:13:34PM +0100, Christian Borntraeger wrote: >>> Yes, the deadlock is gone and the system is still running. >>> After some time I had the following WARN in the logs, though. >>> Not sure yet if that is related. >>> >>> [25331.763607] DEBUG_LOCKS_WARN_ON(lock->owner != current) >>> [25331.763630] ------------[ cut here ]------------ >>> [25331.763634] WARNING: at kernel/locking/mutex-debug.c:80 > >> I restarted the test with panic_on_warn. Hopefully I can get a dump to check >> which mutex this was. > > Hard to reproduce warnings like this tend to point towards memory > corruption. Someone stepped on the mutex value and tickles the sanity > check. > > With lockdep and debugging enabled the mutex gets quite a bit bigger, so > it gets more likely to be hit by 'random' corruption. > > The locking in seq_read() seems rather straight forward. I was able to reproduce. The dump shows a mutex that has an owner field, which does not exists as a task so this all looks fishy. The good thing is, that I can reproduce the issue within some hours. (exact same backtrace). Will add some more debug data to get a handle where we come from.