From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A41F6C433EF for ; Sat, 9 Oct 2021 03:48:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D2B660EE2 for ; Sat, 9 Oct 2021 03:48:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3D2B660EE2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=R4qo2ntrZr4d75sftlEPtE+ktdRYzsMqUyCH6a1T4AA=; b=TO48HFXV5AfEjx jNPaZFEQkuZGYK4gX10iM/6FE98PsTY90mveIJPQfzrVFcDmyg69/3EnnY+kpGSiG6gjNOjUM6Ynp l6sDJwxpI2Dbn4ClalzKgXlCHN9AI5Mm4lCII0OxZt+FtHRU5iGeCxb62EJkKqjPFOsAVX0aXYtEg NtQu/4xJmpVCRnvH7a1kyWQsJqUbPDOIOaAaD5ZP51vTNP20kb8oxdu2MRMy850yp80gSLILGbCHb 10gCl1h0b4pM4ASAkI5GJbybNsjrJuFA00HZ+/tG2fOJLFZSI4gspslveC6cMNfv5gZgm8HpxCMjd FBf+9WtiKu4DOEgT+g2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mZ3Kp-00521H-5a; Sat, 09 Oct 2021 03:47:43 +0000 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mZ3Kl-00520X-JW for linux-nvme@lists.infradead.org; Sat, 09 Oct 2021 03:47:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633751257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=QupAqVSNhmjXK/aXyLiaiaiGdci7w3/gOP38K27m4Y8=; b=b/Utw1HfFUCjBOsbF1fCthdS+KvDy5Qgo0CdP36PLS1y81/KucJWAkHCvDdtZQyhNub+LZ efF0/5JZCWdomBen49rfCWjfsAqb/1GJdxXkYNAASfPNAdLmB+oaiishd2el0t9M/DUx2z kXs9RzqW4G4zq/VYtSLSb1NThKUqOC0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-371-HOLnPP8yMKycadiJlaQNlg-1; Fri, 08 Oct 2021 23:47:31 -0400 X-MC-Unique: HOLnPP8yMKycadiJlaQNlg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B6085127; Sat, 9 Oct 2021 03:47:30 +0000 (UTC) Received: from localhost (ovpn-8-20.pek2.redhat.com [10.72.8.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58868179B3; Sat, 9 Oct 2021 03:47:24 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Chaitanya Kulkarni Cc: Sagi Grimberg , Keith Busch , Bart Van Assche , Ming Lei Subject: [PATCH V3 0/6] blk-mq: support concurrent queue quiescing Date: Sat, 9 Oct 2021 11:47:07 +0800 Message-Id: <20211009034713.1489183-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_204739_952571_0C74D5EE X-CRM114-Status: GOOD ( 13.13 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org Hello, request queue quiescing has been applied in lots of block drivers and block core from different/un-related code paths. So far, both quiesce and unquiesce changes the queue state unconditionally. This way has caused trouble, such as, driver is quiescing queue for its own purpose, however block core's queue quiesce may come because of elevator switch, updating nr_requests or other queue attributes, then un-expected unquiesce may come too early. It has been observed kernel panic when running stress test on dm-mpath suspend and updating nr_requests. Fix the issue by supporting concurrent queue quiescing. But nvme has very complicated uses on quiesce/unquiesce, which two may not be called in pairing, so switch to this way in patch 1~4, and patch 5 provides nested queue quiesce. V3: - add patch 5/6 to clear NVME_CTRL_ADMIN_Q_STOPPED for nvme-loop after reallocating admin queue - take Bart's suggestion to add warning in blk_mq_unquiesce_queue() & update commit log V2: - replace mutex with atomic ops for supporting paring quiesce & unquiesce Ming Lei (6): nvme: add APIs for stopping/starting admin queue nvme: apply nvme API to quiesce/unquiesce admin queue nvme: prepare for pairing quiescing and unquiescing nvme: paring quiesce/unquiesce nvme: loop: clear NVME_CTRL_ADMIN_Q_STOPPED after admin queue is reallocated blk-mq: support concurrent queue quiesce/unquiesce block/blk-mq.c | 21 ++++++++++-- drivers/nvme/host/core.c | 70 ++++++++++++++++++++++++++------------ drivers/nvme/host/fc.c | 8 ++--- drivers/nvme/host/nvme.h | 4 +++ drivers/nvme/host/pci.c | 8 ++--- drivers/nvme/host/rdma.c | 14 ++++---- drivers/nvme/host/tcp.c | 16 ++++----- drivers/nvme/target/loop.c | 6 ++-- include/linux/blkdev.h | 2 ++ 9 files changed, 99 insertions(+), 50 deletions(-) -- 2.31.1 _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme