From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93583C83F2C for ; Mon, 4 Sep 2023 15:40:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231127AbjIDPkc (ORCPT ); Mon, 4 Sep 2023 11:40:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbjIDPkc (ORCPT ); Mon, 4 Sep 2023 11:40:32 -0400 Received: from pandora.armlinux.org.uk (unknown [IPv6:2001:4d48:ad52:32c8:5054:ff:fe00:142]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C57E183; Mon, 4 Sep 2023 08:40:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=uA8feDnfGHOk6NcJ6jrXFKT175jb13DVc+EiyDbwTPA=; b=xuzUWMHsXGqAZRQ52Hk+bYd9vb 8qK4wqmRuiiU0q60skq3F8zPEhiOJ7czhr4HDQ2yRd9me0L4jZ6SARUmzhlGioPmzkS6lul7wUcNU k0zZofi0DMgZ0yoa60hSdW7e7Aree+n1vaDr+SAmi67V2oGrRFkn6FYFU+yu1yMalkdqqpZkzN2wl eqN+xTJ57FQ5N6C3VR+G8e0+CAqvLZdR+2oLdFZXaVdnJXC9SQc7h+VM/qTCRKqmbIkLwSazH3Cm4 v7cxYzxjH1pO0y6CaqahzWUDVKPjGRT+pbicU3B7pFg96uIIPVAGKPy6qC+/xisjPS+OcwLhXeF3C jmu48B2A==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:40242) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1qdBgT-00071H-35; Mon, 04 Sep 2023 16:40:14 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.94.2) (envelope-from ) id 1qdBgR-0002fG-PD; Mon, 04 Sep 2023 16:40:11 +0100 Date: Mon, 4 Sep 2023 16:40:11 +0100 From: "Russell King (Oracle)" To: Waiman Long Cc: =?utf-8?B?UmFmYcWCIE1pxYJlY2tp?= , Peter Zijlstra , Ingo Molnar , Will Deacon , Boqun Feng , Daniel Lezcano , Thomas Gleixner , Florian Fainelli , linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, openwrt-devel@lists.openwrt.org, bcm-kernel-feedback-list@broadcom.com Subject: Re: ARM BCM53573 SoC hangs/lockups caused by locks/clock/random changes Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Sender: Russell King (Oracle) Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org On Mon, Sep 04, 2023 at 11:25:57AM -0400, Waiman Long wrote: > > On 9/4/23 04:33, Rafał Miłecki wrote: > > As those hangs/lockups are related to so many different changes it's > > really hard to debug them. > > > > This bug seems to be specific to the slow arch clock that affects > > stability only when kernel locking code and symbols layout trigger some > > very specific timing. > > > > Enabling CONFIG_PROVE_LOCKING seems to make issue go away but it affects > > so much code it's hard to tell why it actually matters. > > > > Same for disabling CONFIG_SMP. I noticed Broadcom's SDK keeps it > > disabled. I tried it and it improves stability (I had 3 devices with 6 > > days of uptime and counting) indeed. Again it affects a lot of kernel > > parts so it's hard to tell why it helps. > > > > Unless someone comes up with some magic solution I'll probably try > > building BCM53573 images without CONFIG_SMP for my personal needs. > > All the locking operations rely on the fact that the instruction to acquire > or release a lock is atomic. Is it possible that it may not be the case > under certain circumstances for this ARM BCM53573 SoC? Or maybe some Kconfig > options are not set correctly like missing some errata that are needed. > > I don't know enough about the 32-bit arm architecture to say whether this is > the case or not, but that is my best guess. So, BCM53573 is Cortex-A7, which is ARMv7, which has the exclusive load/store instructions. Whether the SoC has the necessary exclusive monitors to support these instructions is another matter, and I suspect someone with documentation would need to check that. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26BD3C71153 for ; Mon, 4 Sep 2023 15:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iJGG88rez3i6OSdI2Nj4iDzzT0dXZ1S/PgkytrJ/GUw=; b=c95mup9tguWmVA sSkGTyucnslqfNHIlzRZ+kW7XXoL+CQoeEWOcbNYhOY0TbUQuT6NClez0MN5Sn394e7fqV9YSueV2 urO7l3uueiRDKPxBhveyHLrQ/oy/se0jSKHZfeT5t7VwTHDoFZAKeBUwMX4dYTjFNMrtlSPDWh6Vc fTqRXviw8uDK31ncC7NHWtJp0oVApFzrRTPWoHUe1GLfnR/jqQ/yxVEqPA3hDFABZGaqnk/BpTbRM X1TeEa0xeNNIkcoRrZKpsubkdCdfbY1DG2tFVwuMtzNxkjIFNOyXa42vKKmwSPDMVEWZnuePDeLD5 E/8HyqncpQ0xSZO+dOVg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qdBil-004OeS-36; Mon, 04 Sep 2023 15:42:35 +0000 Received: from [2001:4d48:ad52:32c8:5054:ff:fe00:142] (helo=pandora.armlinux.org.uk) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qdBii-004OXy-2G for linux-arm-kernel@lists.infradead.org; Mon, 04 Sep 2023 15:42:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=uA8feDnfGHOk6NcJ6jrXFKT175jb13DVc+EiyDbwTPA=; b=xuzUWMHsXGqAZRQ52Hk+bYd9vb 8qK4wqmRuiiU0q60skq3F8zPEhiOJ7czhr4HDQ2yRd9me0L4jZ6SARUmzhlGioPmzkS6lul7wUcNU k0zZofi0DMgZ0yoa60hSdW7e7Aree+n1vaDr+SAmi67V2oGrRFkn6FYFU+yu1yMalkdqqpZkzN2wl eqN+xTJ57FQ5N6C3VR+G8e0+CAqvLZdR+2oLdFZXaVdnJXC9SQc7h+VM/qTCRKqmbIkLwSazH3Cm4 v7cxYzxjH1pO0y6CaqahzWUDVKPjGRT+pbicU3B7pFg96uIIPVAGKPy6qC+/xisjPS+OcwLhXeF3C jmu48B2A==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:40242) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1qdBgT-00071H-35; Mon, 04 Sep 2023 16:40:14 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.94.2) (envelope-from ) id 1qdBgR-0002fG-PD; Mon, 04 Sep 2023 16:40:11 +0100 Date: Mon, 4 Sep 2023 16:40:11 +0100 From: "Russell King (Oracle)" To: Waiman Long Cc: =?utf-8?B?UmFmYcWCIE1pxYJlY2tp?= , Peter Zijlstra , Ingo Molnar , Will Deacon , Boqun Feng , Daniel Lezcano , Thomas Gleixner , Florian Fainelli , linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, openwrt-devel@lists.openwrt.org, bcm-kernel-feedback-list@broadcom.com Subject: Re: ARM BCM53573 SoC hangs/lockups caused by locks/clock/random changes Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230904_084232_741407_33388C90 X-CRM114-Status: GOOD ( 23.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBTZXAgMDQsIDIwMjMgYXQgMTE6MjU6NTdBTSAtMDQwMCwgV2FpbWFuIExvbmcgd3Jv dGU6Cj4gCj4gT24gOS80LzIzIDA0OjMzLCBSYWZhxYIgTWnFgmVja2kgd3JvdGU6Cj4gPiBBcyB0 aG9zZSBoYW5ncy9sb2NrdXBzIGFyZSByZWxhdGVkIHRvIHNvIG1hbnkgZGlmZmVyZW50IGNoYW5n ZXMgaXQncwo+ID4gcmVhbGx5IGhhcmQgdG8gZGVidWcgdGhlbS4KPiA+IAo+ID4gVGhpcyBidWcg c2VlbXMgdG8gYmUgc3BlY2lmaWMgdG8gdGhlIHNsb3cgYXJjaCBjbG9jayB0aGF0IGFmZmVjdHMK PiA+IHN0YWJpbGl0eSBvbmx5IHdoZW4ga2VybmVsIGxvY2tpbmcgY29kZSBhbmQgc3ltYm9scyBs YXlvdXQgdHJpZ2dlciBzb21lCj4gPiB2ZXJ5IHNwZWNpZmljIHRpbWluZy4KPiA+IAo+ID4gRW5h YmxpbmcgQ09ORklHX1BST1ZFX0xPQ0tJTkcgc2VlbXMgdG8gbWFrZSBpc3N1ZSBnbyBhd2F5IGJ1 dCBpdCBhZmZlY3RzCj4gPiBzbyBtdWNoIGNvZGUgaXQncyBoYXJkIHRvIHRlbGwgd2h5IGl0IGFj dHVhbGx5IG1hdHRlcnMuCj4gPiAKPiA+IFNhbWUgZm9yIGRpc2FibGluZyBDT05GSUdfU01QLiBJ IG5vdGljZWQgQnJvYWRjb20ncyBTREsga2VlcHMgaXQKPiA+IGRpc2FibGVkLiBJIHRyaWVkIGl0 IGFuZCBpdCBpbXByb3ZlcyBzdGFiaWxpdHkgKEkgaGFkIDMgZGV2aWNlcyB3aXRoIDYKPiA+IGRh eXMgb2YgdXB0aW1lIGFuZCBjb3VudGluZykgaW5kZWVkLiBBZ2FpbiBpdCBhZmZlY3RzIGEgbG90 IG9mIGtlcm5lbAo+ID4gcGFydHMgc28gaXQncyBoYXJkIHRvIHRlbGwgd2h5IGl0IGhlbHBzLgo+ ID4gCj4gPiBVbmxlc3Mgc29tZW9uZSBjb21lcyB1cCB3aXRoIHNvbWUgbWFnaWMgc29sdXRpb24g SSdsbCBwcm9iYWJseSB0cnkKPiA+IGJ1aWxkaW5nIEJDTTUzNTczIGltYWdlcyB3aXRob3V0IENP TkZJR19TTVAgZm9yIG15IHBlcnNvbmFsIG5lZWRzLgo+IAo+IEFsbCB0aGUgbG9ja2luZyBvcGVy YXRpb25zIHJlbHkgb24gdGhlIGZhY3QgdGhhdCB0aGUgaW5zdHJ1Y3Rpb24gdG8gYWNxdWlyZQo+ IG9yIHJlbGVhc2UgYSBsb2NrIGlzIGF0b21pYy4gSXMgaXQgcG9zc2libGUgdGhhdCBpdCBtYXkg bm90IGJlIHRoZSBjYXNlCj4gdW5kZXIgY2VydGFpbiBjaXJjdW1zdGFuY2VzIGZvciB0aGlzIEFS TSBCQ001MzU3MyBTb0M/IE9yIG1heWJlIHNvbWUgS2NvbmZpZwo+IG9wdGlvbnMgYXJlIG5vdCBz ZXQgY29ycmVjdGx5IGxpa2UgbWlzc2luZyBzb21lIGVycmF0YSB0aGF0IGFyZSBuZWVkZWQuCj4g Cj4gSSBkb24ndCBrbm93IGVub3VnaCBhYm91dCB0aGUgMzItYml0IGFybSBhcmNoaXRlY3R1cmUg dG8gc2F5IHdoZXRoZXIgdGhpcyBpcwo+IHRoZSBjYXNlIG9yIG5vdCwgYnV0IHRoYXQgaXMgbXkg YmVzdCBndWVzcy4KClNvLCBCQ001MzU3MyBpcyBDb3J0ZXgtQTcsIHdoaWNoIGlzIEFSTXY3LCB3 aGljaCBoYXMgdGhlIGV4Y2x1c2l2ZQpsb2FkL3N0b3JlIGluc3RydWN0aW9ucy4gV2hldGhlciB0 aGUgU29DIGhhcyB0aGUgbmVjZXNzYXJ5IGV4Y2x1c2l2ZQptb25pdG9ycyB0byBzdXBwb3J0IHRo ZXNlIGluc3RydWN0aW9ucyBpcyBhbm90aGVyIG1hdHRlciwgYW5kIEkKc3VzcGVjdCBzb21lb25l IHdpdGggZG9jdW1lbnRhdGlvbiB3b3VsZCBuZWVkIHRvIGNoZWNrIHRoYXQuCgotLSAKUk1LJ3Mg UGF0Y2ggc3lzdGVtOiBodHRwczovL3d3dy5hcm1saW51eC5vcmcudWsvZGV2ZWxvcGVyL3BhdGNo ZXMvCkZUVFAgaXMgaGVyZSEgODBNYnBzIGRvd24gMTBNYnBzIHVwLiBEZWNlbnQgY29ubmVjdGl2 aXR5IGF0IGxhc3QhCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3Rz LmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5m by9saW51eC1hcm0ta2VybmVsCg==