From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1755744AbcHXJVG (ORCPT <rfc822;w@1wt.eu>);
        Wed, 24 Aug 2016 05:21:06 -0400
Received: from mail-wm0-f46.google.com ([74.125.82.46]:33855 "EHLO
        mail-wm0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1752974AbcHXJUE (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 24 Aug 2016 05:20:04 -0400
Date: Wed, 24 Aug 2016 11:19:23 +0200
From: Ingo Molnar <mingo@kernel.org>
To: Vince Weaver <vincent.weaver@maine.edu>
Cc: Peter Zijlstra <peterz@infradead.org>, Huang Rui <ray.huang@amd.com>,
        linux-kernel@vger.kernel.org, Borislav Petkov <bp@suse.de>,
        Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>
Subject: Re: perf: fuzzer crashes immediately on AMD system
Message-ID: <20160824091923.GB4340@gmail.com>
References: <alpine.DEB.2.20.1608181025160.19862@macbook-air>
 <alpine.DEB.2.20.1608181046120.19862@macbook-air>
 <20160819100130.GD10121@twins.programming.kicks-ass.net>
 <20160822111631.GA25844@hr-amur2>
 <alpine.DEB.2.20.1608222057360.29115@macbook-air>
 <alpine.DEB.2.20.1608222250120.29115@macbook-air>
 <20160823084519.GN10153@twins.programming.kicks-ass.net>
 <alpine.DEB.2.20.1608230750310.30245@macbook-air>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <alpine.DEB.2.20.1608230750310.30245@macbook-air>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


* Vince Weaver <vincent.weaver@maine.edu> wrote:

> On Tue, 23 Aug 2016, Peter Zijlstra wrote:
> 
> > On Mon, Aug 22, 2016 at 10:54:32PM -0400, Vince Weaver wrote:
> > > > > > > 
> > > > > > > 	perf stat -a -e amd_nb/config=0x37,config1=0x20/ /bin/ls
> > > > 	amd_uncore_find_online_sibling()
> > > > function is broken. 
> > > 
> > > and that's the problem.  uncore_find_online_sibling() does all kinds of 
> > > wrong things including sticking active uncore structures in 
> > > uncore->free_when_cpu_online
> > > 
> > > Then uncore_online() comes along and frees those structures.
> > > 
> > > Then some other part of the kernel comes and re-uses the free'd data.
> > > 
> > > Then when we try to start an event, all of the fields are invalid because 
> > > the uncore pointer is pointing to re-used data.
> > > 
> > > I don't have a patch because I am not 100% clear on what 
> > > uncore_find_online_sibling() is doing in the first place.
> > 
> > Thanks for doing all that, I'll see if I can make sense of it.
> 
> I should have provided more detail, was just tired after chasing the bug 
> for so long.  I mostly found things by sprinkling printks everywhere.
> Comenting out the call to kfree() in uncore_online() makes the code stop 
> crashing (but perhaps causes a memory leak?)

If there's no progress finding the root cause I'd be happy to exchange a crash for 
a leak ...

> In any case it's odd the problem didn't show up earlier, but maybe the 
> recent changes to CPU hotplugging in that file exposed the issue.

Yeah, we had lots of changes to CPU hotplugging recently.

Thanks,

	Ingo