From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AF882C001DB for ; Fri, 4 Aug 2023 11:08:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:References:Cc:To :From:Subject:MIME-Version:Date:Message-ID:Content-Type:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mmM20ktcTAFupmfPe5XuX9xlHv5TarMwVlkR9+bYRRA=; b=Yp9ca+YcqyAKtqFVG3kgTeJZnj ubLPfUSOvab8qpHrF5S/BmjXdaBiy8yDwOsd8x7ODpYfUh+U/EViGFLRD/0DGl4k4Z/w0Jl2dMjMx c8zNg9cveiaDYGS5ja8P9qe3FKTaHsNazWARDUibeN+5sHNkIDf+fIhSUUsr/t79vPOk5YBbTLu5p OIEcqddk27cFxsg4l/o/WbRE5rmzYQ5uCnu0F4bz6qO6MfRYxon3nMluTUjP7yFbw41nQVFhbCVZJ f8ixnGkXn/WCD0x0M7BC3Xnf1LSQJCdLO+EbureFicpWR9OhzLIe4dETgw7MbL9uY8okcBNGZ3tZP FevKqXwg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qRseq-00CJRc-1V; Fri, 04 Aug 2023 11:07:48 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qRsen-00CJQQ-2a for linux-arm-kernel@lists.infradead.org; Fri, 04 Aug 2023 11:07:47 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-4fe28f92d8eso3322602e87.1 for ; Fri, 04 Aug 2023 04:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691147264; x=1691752064; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=fEy1+hL9zQHiNkyeCV6qt4c/bSL6uGGiQWAnjgglbbw=; b=MFDuNHIS9vvGBaqlQhiPcM7MS4Nzh9iGbsvcE2M4fzYXhAAiMVll0D5waVMCyYV5Kb 0Im8TWE8LYmEffS4slT1P5QfCoIc+IBbwqI0y7dS2kzad/rGgiKEJC4ZUw2v+52y2Emx GGkS1/A6wP1z4A6ptTQPzYthHwmizWg/rnNfu4FmvK0XIZecKa/kBOXCrYZnCwPbf+gB d49KAoqKIUXE9JvuCLky/hAk3VGjGfJhhENBAFDEW3SxRAgose8AfSFEbSudws0880RV oVCzLFXTKVWk2jEVW5TqcI7iXkhclNj0jegAUotnwFN1NYqw6S1khnJlJ5953UR334yr vfLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691147264; x=1691752064; h=in-reply-to:references:cc:to:from:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fEy1+hL9zQHiNkyeCV6qt4c/bSL6uGGiQWAnjgglbbw=; b=EhXlRsjLKvYTLbwwboPjg08q3fgYlAjxiUDCbmo8Spl6FyUnOC4oCNzbVij8g8+6sI sZl0Z7apfnlc+lwyL5Q+pWr21kjz9aTSfBPD6EzJJ26EIFeCc/ZrIBedY+zLdtuQUw+K SOgvJIq0YwkLKa9GtGT+zpnjPUWS9gg4DzJK8r7zilucpEWX5vYeP7OL++gNQ0/Cp/UN QaTbBRh4A9Gh9LJLQ7/4g93RHaSNvlQ9wTreA0vBArkG8WolAzLQyDoCah7xR+YgWNsp f2mxZm1xxC7k2EooiUOqyZyx6qfsya4hYWqJCqbbphP9GuAFODS4GlQMCqcSaq/BGdbr 6GRA== X-Gm-Message-State: AOJu0YwuXizXSQkNnNfzxM/q7gs5yAv9s29U4FPTGJwg7o4xc/JKlr4g nDysBspykXXS/1sXGjZ5aZs= X-Google-Smtp-Source: AGHT+IHkmshvNSeDT03iNPwTvWI8VbU7T+hWSVUpm7GSCLI6Ul2Q3ncHyw3jKYX7h7yx+hINgsUivw== X-Received: by 2002:a19:9201:0:b0:4fb:9469:d659 with SMTP id u1-20020a199201000000b004fb9469d659mr1010973lfd.23.1691147263725; Fri, 04 Aug 2023 04:07:43 -0700 (PDT) Received: from [192.168.26.149] (031011218106.poznan.vectranet.pl. [31.11.218.106]) by smtp.googlemail.com with ESMTPSA id v26-20020ac2559a000000b004fddb0eb961sm332522lfg.18.2023.08.04.04.07.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 04 Aug 2023 04:07:43 -0700 (PDT) Content-Type: multipart/mixed; boundary="------------atppxto8rPUknDM1CKkUqQJb" Message-ID: Date: Fri, 4 Aug 2023 13:07:41 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: ARM board lockups/hangs triggered by locks and mutexes Content-Language: en-US From: =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long , Boqun Feng , Russell King , Daniel Lezcano , Thomas Gleixner , Florian Fainelli , linux-clk@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , Network Development , Linux Kernel Mailing List Cc: OpenWrt Development List , bcm-kernel-feedback-list References: In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230804_040745_841334_9EE0BC0B X-CRM114-Status: GOOD ( 17.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is a multi-part message in MIME format. --------------atppxto8rPUknDM1CKkUqQJb Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2.08.2023 00:10, Rafał Miłecki wrote: > Unfortunately enabling *any* of following options: > CONFIG_DEBUG_RT_MUTEXES=y > CONFIG_DEBUG_SPINLOCK=y > CONFIG_DEBUG_MUTEXES=y > seems to make locksup/hangs go away. I tried for few hours. I decided to find out why enabling CONFIG_DEBUG_MUTEXES "fixes" kernel / device stability for me. I tried enabling manually code that normally hides behind the #ifdev CONFIG_DEBUG_MUTEXES. Attached to this e-mail is a small patch that is enough to make my kernel stable (mutex-fix-bcm53573.diff). ##### It's not what's the most interesting thought. What really doesn't make sense anymore is that below diff (on top of attached one) brings back hangs/lockups. I triple checked that. Dropping a single unused function breaks kernel / device stability on BCM53573! AFAIK the only thing below diff actually affects is location of symbols (I actually verified that by comparing System.map before and after - over 22'000 of relocated symbols). Can some unfortunate location of symbols cause those hangs/lockups? diff --git a/kernel/locking/mutex-debug.c b/kernel/locking/mutex-debug.c index 4fe40910f..c440222a4 100644 --- a/kernel/locking/mutex-debug.c +++ b/kernel/locking/mutex-debug.c @@ -34,6 +34,8 @@ void debug_mutex_lock_common(struct mutex *lock, struct mutex_waiter *waiter) INIT_LIST_HEAD(&waiter->list); } +/* Dropping below function brings back hangs/lockups & reboots */ +#if 0 void debug_mutex_wake_waiter(struct mutex *lock, struct mutex_waiter *waiter) { lockdep_assert_held(&lock->wait_lock); @@ -41,6 +43,7 @@ void debug_mutex_wake_waiter(struct mutex *lock, struct mutex_waiter *waiter) DEBUG_LOCKS_WARN_ON(waiter->magic != waiter); DEBUG_LOCKS_WARN_ON(list_empty(&waiter->list)); } +#endif void debug_mutex_free_waiter(struct mutex_waiter *waiter) { --------------atppxto8rPUknDM1CKkUqQJb Content-Type: text/x-patch; charset=UTF-8; name="mutex-fix-bcm53573.diff" Content-Disposition: attachment; filename="mutex-fix-bcm53573.diff" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbXV0ZXguaCBiL2luY2x1ZGUvbGludXgvbXV0 ZXguaAppbmRleCA0NzliYzk2YzMuLjE1YmQ0NjkxYiAxMDA2NDQKLS0tIGEvaW5jbHVkZS9s aW51eC9tdXRleC5oCisrKyBiL2luY2x1ZGUvbGludXgvbXV0ZXguaApAQCAtNTcsOSArNTcs NyBAQCBzdHJ1Y3QgbXV0ZXggewogCXN0cnVjdCBvcHRpbWlzdGljX3NwaW5fcXVldWUgb3Nx OyAvKiBTcGlubmVyIE1DUyBsb2NrICovCiAjZW5kaWYKIAlzdHJ1Y3QgbGlzdF9oZWFkCXdh aXRfbGlzdDsKLSNpZmRlZiBDT05GSUdfREVCVUdfTVVURVhFUwogCXZvaWQJCQkqbWFnaWM7 Ci0jZW5kaWYKICNpZmRlZiBDT05GSUdfREVCVUdfTE9DS19BTExPQwogCXN0cnVjdCBsb2Nr ZGVwX21hcAlkZXBfbWFwOwogI2VuZGlmCkBAIC03MywxMiArNzEsMTAgQEAgc3RydWN0IG11 dGV4X3dhaXRlciB7CiAJc3RydWN0IGxpc3RfaGVhZAlsaXN0OwogCXN0cnVjdCB0YXNrX3N0 cnVjdAkqdGFzazsKIAlzdHJ1Y3Qgd3dfYWNxdWlyZV9jdHgJKnd3X2N0eDsKLSNpZmRlZiBD T05GSUdfREVCVUdfTVVURVhFUwogCXZvaWQJCQkqbWFnaWM7Ci0jZW5kaWYKIH07CiAKLSNp ZmRlZiBDT05GSUdfREVCVUdfTVVURVhFUworI2lmIDEgLy9kZWYgQ09ORklHX0RFQlVHX01V VEVYRVMKIAogI2RlZmluZSBfX0RFQlVHX01VVEVYX0lOSVRJQUxJWkVSKGxvY2tuYW1lKQkJ CQlcCiAJLCAubWFnaWMgPSAmbG9ja25hbWUKZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgv c2NoZWQuaCBiL2luY2x1ZGUvbGludXgvc2NoZWQuaAppbmRleCBkMGU2Mzk0OTcuLjhmZWY0 NDg1ZSAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9zY2hlZC5oCisrKyBiL2luY2x1ZGUv bGludXgvc2NoZWQuaApAQCAtOTU4LDEwICs5NTgsOCBAQCBzdHJ1Y3QgdGFza19zdHJ1Y3Qg ewogCXN0cnVjdCBydF9tdXRleF93YWl0ZXIJCSpwaV9ibG9ja2VkX29uOwogI2VuZGlmCiAK LSNpZmRlZiBDT05GSUdfREVCVUdfTVVURVhFUwogCS8qIE11dGV4IGRlYWRsb2NrIGRldGVj dGlvbjogKi8KIAlzdHJ1Y3QgbXV0ZXhfd2FpdGVyCQkqYmxvY2tlZF9vbjsKLSNlbmRpZgog CiAjaWZkZWYgQ09ORklHX0RFQlVHX0FUT01JQ19TTEVFUAogCWludAkJCQlub25fYmxvY2tf Y291bnQ7CmRpZmYgLS1naXQgYS9rZXJuZWwvbG9ja2luZy9NYWtlZmlsZSBiL2tlcm5lbC9s b2NraW5nL01ha2VmaWxlCmluZGV4IDQ1NDUyZmFjZi4uYjIyZTZlY2Q4IDEwMDY0NAotLS0g YS9rZXJuZWwvbG9ja2luZy9NYWtlZmlsZQorKysgYi9rZXJuZWwvbG9ja2luZy9NYWtlZmls ZQpAQCAtMTIsNyArMTIsNyBAQCBDRkxBR1NfUkVNT1ZFX211dGV4LWRlYnVnLm8gPSAkKEND X0ZMQUdTX0ZUUkFDRSkKIENGTEFHU19SRU1PVkVfcnRtdXRleC1kZWJ1Zy5vID0gJChDQ19G TEFHU19GVFJBQ0UpCiBlbmRpZgogCi1vYmotJChDT05GSUdfREVCVUdfTVVURVhFUykgKz0g bXV0ZXgtZGVidWcubworb2JqLXkgKz0gbXV0ZXgtZGVidWcubwogb2JqLSQoQ09ORklHX0xP Q0tERVApICs9IGxvY2tkZXAubwogaWZlcSAoJChDT05GSUdfUFJPQ19GUykseSkKIG9iai0k KENPTkZJR19MT0NLREVQKSArPSBsb2NrZGVwX3Byb2MubwpkaWZmIC0tZ2l0IGEva2VybmVs L2xvY2tpbmcvbXV0ZXguYyBiL2tlcm5lbC9sb2NraW5nL211dGV4LmMKaW5kZXggYjAyZmZm MjgyLi42ZGMzZjgwYTMgMTAwNjQ0Ci0tLSBhL2tlcm5lbC9sb2NraW5nL211dGV4LmMKKysr IGIva2VybmVsL2xvY2tpbmcvbXV0ZXguYwpAQCAtOTQ2LDkgKzk0Niw3IEBAIF9fbXV0ZXhf bG9ja19jb21tb24oc3RydWN0IG11dGV4ICpsb2NrLCBsb25nIHN0YXRlLCB1bnNpZ25lZCBp bnQgc3ViY2xhc3MsCiAKIAltaWdodF9zbGVlcCgpOwogCi0jaWZkZWYgQ09ORklHX0RFQlVH X01VVEVYRVMKIAlERUJVR19MT0NLU19XQVJOX09OKGxvY2stPm1hZ2ljICE9IGxvY2spOwot I2VuZGlmCiAKIAl3dyA9IGNvbnRhaW5lcl9vZihsb2NrLCBzdHJ1Y3Qgd3dfbXV0ZXgsIGJh c2UpOwogCWlmICh3d19jdHgpIHsKQEAgLTE0MTcsOSArMTQxNSw3IEBAIGludCBfX3NjaGVk IG11dGV4X3RyeWxvY2soc3RydWN0IG11dGV4ICpsb2NrKQogewogCWJvb2wgbG9ja2VkOwog Ci0jaWZkZWYgQ09ORklHX0RFQlVHX01VVEVYRVMKIAlERUJVR19MT0NLU19XQVJOX09OKGxv Y2stPm1hZ2ljICE9IGxvY2spOwotI2VuZGlmCiAKIAlsb2NrZWQgPSBfX211dGV4X3RyeWxv Y2sobG9jayk7CiAJaWYgKGxvY2tlZCkK --------------atppxto8rPUknDM1CKkUqQJb Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel --------------atppxto8rPUknDM1CKkUqQJb--