From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3DD9C77B76 for ; Mon, 17 Apr 2023 16:21:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229926AbjDQQVH (ORCPT ); Mon, 17 Apr 2023 12:21:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229892AbjDQQVE (ORCPT ); Mon, 17 Apr 2023 12:21:04 -0400 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DE019034 for ; Mon, 17 Apr 2023 09:20:28 -0700 (PDT) Received: by mail-qv1-f53.google.com with SMTP id me15so5563755qvb.4 for ; Mon, 17 Apr 2023 09:20:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681748427; x=1684340427; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ub7tIRwtkL3uHAk4AECnTw80MqWTDLFc81n+6HzSTcQ=; b=ew6t64OMaXFmViSBhZgAsmHXsYWOWHTROovCtdKldsEGXJyJTy/yZYRDIEYPVnWxKD AmKIbWrfKvC1mgYhI9EoZay7Lcbt1COET3kzd/r2bBMJLrUbGqMUPeqtbb9g8z+JzT5y CKKSsUN2QUUKp1dGFDmhKqkQ5Y/ll1rsa0fSt/MDViobeO87p+aP/aOiXgXH+1Ywu7uJ ymP7SdejqignRSsafaeP89OVcla5NGL4WZCuJ10Pn8jYGrEz6u9DndZck9yd1jorVc8I LZvKrtTkOTXeMZkTQ5ZHY2tFKZQJ89f17+waeQeu5LHIuQOEt+0cN5ZzZSAUBANUufzS ZrUw== X-Gm-Message-State: AAQBX9d9RwCOAe/4Row7ceV7tm3mHiE4FkG6QgmUswiqsJzpH2ORKY4W 19J1uI6IYci8nfsOyX2l4Gmy X-Google-Smtp-Source: AKy350Zif85wmja9YUeBTmaCZVN0DyRBM/DP2PS9ZycOFsNEmNYPlKv/tw8AZBWEGdX0pkmFsSASJg== X-Received: by 2002:a05:6214:c45:b0:5ef:6b9a:6948 with SMTP id r5-20020a0562140c4500b005ef6b9a6948mr6838354qvj.17.1681748427225; Mon, 17 Apr 2023 09:20:27 -0700 (PDT) Received: from localhost (pool-68-160-166-30.bstnma.fios.verizon.net. [68.160.166.30]) by smtp.gmail.com with ESMTPSA id b8-20020ac812c8000000b003e4dab0776esm3391128qtj.40.2023.04.17.09.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Apr 2023 09:20:26 -0700 (PDT) Date: Mon, 17 Apr 2023 12:20:25 -0400 From: Mike Snitzer To: Zheng Zhang Cc: agk@redhat.com, dm-devel@redhat.com, linux-kernel@vger.kernel.org, mpatocka@redhat.com Subject: Re: Possible deadlock detected in Linux 6.2.0 in dm_get_inactive_table (dm-ioctl.c) Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Apr 17 2023 at 1:08P -0400, Zheng Zhang wrote: > Alasdir, Mike, and to whom it may concern: > > Hello! We have found a bug in the Linux kernel version 6.2.0 by syzkaller > with our own templates. The bug causes a possible recursive locking > scenario, resulting in a deadlock. > The key trace is as follows (the complete trace is in the attached report > file): > > down_read+0x9d/0x450 kernel/locking/rwsem.c:1509 > > dm_get_inactive_table+0x2b/0xc0 drivers/md/dm-ioctl.c:773 > > __dev_status+0x4fd/0x7c0 drivers/md/dm-ioctl.c:844 > table_clear+0x197/0x280 drivers/md/dm-ioctl.c:1537 > > In table_clear, it acquires a *write lock* > https://elixir.bootlin.com/linux/v6.2/source/drivers/md/dm-ioctl.c#L1520 > down_write(&_hash_lock); > > Then before the lock is released at L1539, there is a path shown above: > table_clear -> __dev_status -> dm_get_inactive_table -> down_read > https://elixir.bootlin.com/linux/v6.2/source/drivers/md/dm-ioctl.c#L773 > down_read(&_hash_lock); > It tries to acquire* the same read lock* again, resulting in the deadlock > problem > > Attached is the report, log, and reproducers generated by syzkaller > Please let me know if there is any additional information that I can > provide to help debug this issue. > Thanks! Thanks for the report, I've staged this fix: From: Mike Snitzer Subject: [PATCH] dm ioctl: fix nested locking in table_clear() to remove deadlock concern syzkaller found the following problematic rwsem locking (with write lock already held): down_read+0x9d/0x450 kernel/locking/rwsem.c:1509 dm_get_inactive_table+0x2b/0xc0 drivers/md/dm-ioctl.c:773 __dev_status+0x4fd/0x7c0 drivers/md/dm-ioctl.c:844 table_clear+0x197/0x280 drivers/md/dm-ioctl.c:1537 In table_clear, it first acquires a write lock https://elixir.bootlin.com/linux/v6.2/source/drivers/md/dm-ioctl.c#L1520 down_write(&_hash_lock); Then before the lock is released at L1539, there is a path shown above: table_clear -> __dev_status -> dm_get_inactive_table -> down_read https://elixir.bootlin.com/linux/v6.2/source/drivers/md/dm-ioctl.c#L773 down_read(&_hash_lock); It tries to acquire the same read lock again, resulting in the deadlock problem. Fix this by moving table_clear()'s __dev_status() call to after its up_write(&_hash_lock); Cc: stable@vger.kernel.org Reported-by: Zheng Zhang Signed-off-by: Mike Snitzer --- drivers/md/dm-ioctl.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index 50a1259294d1..7d5c9c582ed2 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -1556,11 +1556,12 @@ static int table_clear(struct file *filp, struct dm_ioctl *param, size_t param_s has_new_map = true; } - param->flags &= ~DM_INACTIVE_PRESENT_FLAG; - - __dev_status(hc->md, param); md = hc->md; up_write(&_hash_lock); + + param->flags &= ~DM_INACTIVE_PRESENT_FLAG; + __dev_status(md, param); + if (old_map) { dm_sync_table(md); dm_table_destroy(old_map); -- 2.40.0