From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFA89C25B75 for ; Wed, 29 May 2024 15:58:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: Message-ID:In-Reply-To:Date:References:Subject:Cc:To:From:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VARqXBFDwEMLKE6qIvB72ivW4EKZdIa/kyIpg1Ejg0s=; b=szUSqdFclmezks83Sx9+/NDC7a s/LnEkxy1vSvO6NYl7V+VPkl5mei1wtEikGCyy5EV5mWjrQgWO9d9f9XzTBWocuBRymc3eL6Yciu+ ivu1Cs8nws+68W3UPuJSAo9ttdIkenmKnQsKVQVJVnG1aunk8iUeNJK9esU94MMqrLsJ/qD0KvjWr ya3BzLfVTwlwJMfmMAWDDMQR/68Ku5FSOXLy5rdGvXOJ5cDBEgeQ0IgVeclsR7tFAu1hlAg5AjlWg rpGrmdLjjRVknI3SwJ4B1W/xRYJ7USYFSvKCAqjK6ANa2TKZmoAn5ak5KPQh8XhLTMutNuWKk6/y9 J6INl67A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLhN-00000004nIY-1jfH; Wed, 29 May 2024 15:58:45 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLhJ-00000004nGp-1huT for ath11k@lists.infradead.org; Wed, 29 May 2024 15:58:43 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 19FCF60DB9; Wed, 29 May 2024 15:58:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 491BCC113CC; Wed, 29 May 2024 15:58:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716998319; bh=ufyQbqT93SwSt4s4JqmoiAB/zHJa/FMuOgckZ/BR4TU=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=fypTA+JW2IV7pVP1qSfV1ol4Ja9TfGeBVVJx8MRhuBAJGIEN1e8OFOne5tbK7RN2a D0LvLXsB5H5+Gy+2++biOK246LPXm1vkQzR9dwUErgCpKivRZRfiAf8HPNiTGKX9pU oWErLgw+ObFGOiZhKOKrIKTl3UdfBp0zLwwvyhBPfMijoZcJ4GqJGbGRVpxLiXLuK3 rlD365rHM/V9afV0i+oKhcXyHY2VXEmAZKWOS9YOv4s3qcmAjzDtAySsurpBtSy2FR KYwVbp/sbn1MbK0yr5b9F++Zx1/dfnPzTxAlo4T5Gd8s0DbfhjRb+ud079yR5zNYSX CxazeMz/4IGdw== From: Kalle Valo To: linux-kernel@vger.kernel.org Cc: linux-wireless@vger.kernel.org, ath11k@lists.infradead.org, regressions@lists.linux.dev, Jeff Johnson , Bjorn Helgaas Subject: Re: [regression] BUG: KASAN: use-after-free in lockdep_register_key+0x755/0x8f0 References: <87v82y6wvi.fsf@kernel.org> Date: Wed, 29 May 2024 18:58:36 +0300 In-Reply-To: <87v82y6wvi.fsf@kernel.org> (Kalle Valo's message of "Tue, 28 May 2024 13:42:25 +0300") Message-ID: <87wmncwqxf.fsf@kernel.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_085841_785112_09C3263E X-CRM114-Status: GOOD ( 18.08 ) X-BeenThere: ath11k@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "ath11k" Errors-To: ath11k-bounces+ath11k=archiver.kernel.org@lists.infradead.org Kalle Valo writes: > Yesterday I run our ath11k regression tests with v6.10-rc1 and our > simple ath11k module reload stress started failing reliably with various > KASAN errors. The test removes and inserts ath11k and other wireless > modules in a loop. Usually I run it at least 100 times, some times even > more, and no issues until yesterday. > > I have verified that the last wireless-next pull request (tag > wireless-next-2024-05-08) works without issues and v6.10-rc1 fails > always, usually within 50 module reload loops. From this I'm _guessing_ > that we have a regression outside wireless, most probably introduced > between v6.9 and v6.10-rc1. But of course I cannot be sure of anything > yet. > > I see different KASAN warnings and lockdep seems to be always visible in > the stack traces. I think I can reproduce the issue within 15 minutes or > so. Before I start bisecting has anyone else seen anything similar? Or > any suggestions how to debug this further? > > I have included some crash logs below, they are retrieved using > netconsole. Here's a summary of the errors: > > [ 159.970765] KASAN: maybe wild-memory-access in range > [0xbbbbbbbbbbbbbbb8-0xbbbbbbbbbbbbbbbf] > [ 700.017632] BUG: KASAN: use-after-free in lockdep_register_key+0x755/0x8f0 > [ 224.695821] BUG: KASAN: slab-out-of-bounds in lockdep_register_key+0x755/0x8f0 > [ 259.666542] BUG: KASAN: slab-use-after-free in lockdep_register_key+0x755/0x8f0 I did a bisect and got this: cf29111d3e4a9ebe1cbe2b431274718506d69f10 is the first bad commit commit cf29111d3e4a9ebe1cbe2b431274718506d69f10 Merge: ed11a28cb709 e6f7d27df5d2 Author: Bjorn Helgaas Date: Thu May 16 18:14:11 2024 -0500 Merge branch 'pci/of' - Check for kcalloc() failure and handle it gracefully (Duoming Zhou) * pci/of: PCI: of_property: Return error for int_map allocation failure drivers/pci/of_property.c | 2 ++ 1 file changed, 2 insertions(+) But that doesn't make any sense to me, I don't even have CONFIG_PCI_DYNAMIC_OF_NODES enabled in my .config. I guess I did a mistake during bisect, I'm now testing the parents (e6f7d27df5d2 and ed11a28cb709) and trying to pinpoint where I did it wrong. Adding Bjorn in case he has any ideas. This might be something PCI related based on my bisect log (but just guessing at this point): git bisect start # status: waiting for both good and bad commits # good: [a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6] Linux 6.9 git bisect good a38297e3fb012ddfa7ce0321a7e5a8daeb1872b6 # status: waiting for bad commit, 1 good commit known # good: [1d60eabb82694e58543e2b6366dae3e7465892a5] wifi: mwl8k: initialize cmd->addr[] properly git bisect good 1d60eabb82694e58543e2b6366dae3e7465892a5 # status: waiting for bad commit, 2 good commits known # bad: [1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0] Linux 6.10-rc1 git bisect bad 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0 # good: [d34672777da3ea919e8adb0670ab91ddadf7dea0] Merge tag 'fbdev-for-6.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/linux-fbdev git bisect good d34672777da3ea919e8adb0670ab91ddadf7dea0 # good: [a90f1cd105c6c5c246f07ca371d873d35b78c7d9] Merge tag 'turbostat-for-Linux-6.10-merge-window' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux git bisect good a90f1cd105c6c5c246f07ca371d873d35b78c7d9 # bad: [29c73fc794c83505066ee6db893b2a83ac5fac63] Merge tag 'perf-tools-for-v6.10-1-2024-05-21' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools git bisect bad 29c73fc794c83505066ee6db893b2a83ac5fac63 # good: [30aec6e1bb617e1349d7fa5498898d7d4351d71e] Merge tag 'vfio-v6.10-rc1' of https://github.com/awilliam/linux-vfio git bisect good 30aec6e1bb617e1349d7fa5498898d7d4351d71e # bad: [8053d2ffc4502bbb50a78c805d964e65a6de1803] Merge tag 'phy-for-6.10' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy git bisect bad 8053d2ffc4502bbb50a78c805d964e65a6de1803 # bad: [f0bae243b2bcf2b160ae547463bf542762beef8f] Merge tag 'pci-v6.10-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci git bisect bad f0bae243b2bcf2b160ae547463bf542762beef8f # good: [38da32ee70b876f5b8bea7c4135eff46339c18f2] Merge tag 'pull-bd_inode-1' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect good 38da32ee70b876f5b8bea7c4135eff46339c18f2 # good: [ed11a28cb709a9ab69c4cd4e0669079a455f9a8d] Merge branch 'pci/msi' git bisect good ed11a28cb709a9ab69c4cd4e0669079a455f9a8d # bad: [102c69699b5b5d4aebfe8d15d5f91bde68dababd] Merge branch 'pci/controller/mt7621' git bisect bad 102c69699b5b5d4aebfe8d15d5f91bde68dababd # bad: [14680b252788675e2007fffde371e76a3a7a9b21] Merge branch 'pci/dt-bindings' git bisect bad 14680b252788675e2007fffde371e76a3a7a9b21 # bad: [12ff1ef539c23cb7563bc3d894de9edd9469ea98] Merge branch 'pci/pm' git bisect bad 12ff1ef539c23cb7563bc3d894de9edd9469ea98 # bad: [cf29111d3e4a9ebe1cbe2b431274718506d69f10] Merge branch 'pci/of' git bisect bad cf29111d3e4a9ebe1cbe2b431274718506d69f10 # good: [e6f7d27df5d208b50cae817a91d128fb434bb12c] PCI: of_property: Return error for int_map allocation failure git bisect good e6f7d27df5d208b50cae817a91d128fb434bb12c # first bad commit: [cf29111d3e4a9ebe1cbe2b431274718506d69f10] Merge branch 'pci/of' -- https://patchwork.kernel.org/project/linux-wireless/list/ https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches