public inbox for stable@vger.kernel.org
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Sagi Grimberg <sagi@grimberg.me>, Sasha Levin <sashal@kernel.org>,
	kbusch@kernel.org, axboe@fb.com, linux-nvme@lists.infradead.org
Subject: [PATCH AUTOSEL 5.10 20/27] nvme: fix a possible use-after-free in controller reset during load
Date: Wed,  9 Feb 2022 13:40:56 -0500	[thread overview]
Message-ID: <20220209184103.47635-20-sashal@kernel.org> (raw)
In-Reply-To: <20220209184103.47635-1-sashal@kernel.org>

From: Sagi Grimberg <sagi@grimberg.me>

[ Upstream commit 0fa0f99fc84e41057cbdd2efbfe91c6b2f47dd9d ]

Unlike .queue_rq, in .submit_async_event drivers may not check the ctrl
readiness for AER submission. This may lead to a use-after-free
condition that was observed with nvme-tcp.

The race condition may happen in the following scenario:
1. driver executes its reset_ctrl_work
2. -> nvme_stop_ctrl - flushes ctrl async_event_work
3. ctrl sends AEN which is received by the host, which in turn
   schedules AEN handling
4. teardown admin queue (which releases the queue socket)
5. AEN processed, submits another AER, calling the driver to submit
6. driver attempts to send the cmd
==> use-after-free

In order to fix that, add ctrl state check to validate the ctrl
is actually able to accept the AER submission.

This addresses the above race in controller resets because the driver
during teardown should:
1. change ctrl state to RESETTING
2. flush async_event_work (as well as other async work elements)

So after 1,2, any other AER command will find the
ctrl state to be RESETTING and bail out without submitting the AER.

Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/nvme/host/core.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 99b5152482fe4..71c85c99e86c6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4259,7 +4259,14 @@ static void nvme_async_event_work(struct work_struct *work)
 		container_of(work, struct nvme_ctrl, async_event_work);
 
 	nvme_aen_uevent(ctrl);
-	ctrl->ops->submit_async_event(ctrl);
+
+	/*
+	 * The transport drivers must guarantee AER submission here is safe by
+	 * flushing ctrl async_event_work after changing the controller state
+	 * from LIVE and before freeing the admin queue.
+	*/
+	if (ctrl->state == NVME_CTRL_LIVE)
+		ctrl->ops->submit_async_event(ctrl);
 }
 
 static bool nvme_ctrl_pp_status(struct nvme_ctrl *ctrl)
-- 
2.34.1


  parent reply	other threads:[~2022-02-09 18:45 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-09 18:40 [PATCH AUTOSEL 5.10 01/27] platform/x86: touchscreen_dmi: Add info for the RWC NANOTE P8 AY07J 2-in-1 Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 02/27] platform/x86: ISST: Fix possible circular locking dependency detected Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 03/27] selftests: rtc: Increase test timeout so that all tests run Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 04/27] kselftest: signal all child processes Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 05/27] net: ieee802154: at86rf230: Stop leaking skb's Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 06/27] selftests/zram: Skip max_comp_streams interface on newer kernel Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 07/27] selftests/zram01.sh: Fix compression ratio calculation Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 08/27] selftests/zram: Adapt the situation that /dev/zram0 is being used Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 09/27] selftests: openat2: Print also errno in failure messages Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 10/27] selftests: openat2: Add missing dependency in Makefile Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 11/27] selftests: openat2: Skip testcases that fail with EOPNOTSUPP Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 12/27] selftests: skip mincore.check_file_mmap when fs lacks needed support Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 13/27] ax25: improve the incomplete fix to avoid UAF and NPD bugs Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 14/27] vfs: make freeze_super abort when sync_filesystem returns error Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 15/27] quota: make dquot_quota_sync return errors from ->sync_fs Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 16/27] btrfs: tree-checker: check item_size for dev_item Sasha Levin
2022-02-18 10:36   ` Greg KH
2022-02-18 11:25     ` Su Yue
2022-02-18 13:04       ` Greg KH
2022-02-23 17:20       ` David Sterba
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 17/27] iommu: Fix potential use-after-free during probe Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 18/27] scsi: pm8001: Fix use-after-free for aborted TMF sas_task Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 19/27] scsi: pm8001: Fix use-after-free for aborted SSP/STP sas_task Sasha Levin
2022-02-09 18:40 ` Sasha Levin [this message]
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 21/27] nvme-tcp: fix possible use-after-free in transport error_recovery work Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 22/27] nvme-rdma: " Sasha Levin
2022-02-09 18:40 ` [PATCH AUTOSEL 5.10 23/27] drm/amdgpu: fix logic inversion in check Sasha Levin
2022-02-09 18:41 ` [PATCH AUTOSEL 5.10 24/27] x86/Xen: streamline (and fix) PV CPU enumeration Sasha Levin
2022-02-09 18:41 ` [PATCH AUTOSEL 5.10 25/27] Revert "module, async: async_synchronize_full() on module init iff async is used" Sasha Levin
2022-02-09 18:41 ` [PATCH AUTOSEL 5.10 26/27] gcc-plugins/stackleak: Use noinstr in favor of notrace Sasha Levin
2022-02-09 18:41 ` [PATCH AUTOSEL 5.10 27/27] random: wake up /dev/random writers after zap Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220209184103.47635-20-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=axboe@fb.com \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=sagi@grimberg.me \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox