From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AC98EB64D7 for ; Tue, 13 Jun 2023 13:30:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=y1wWr1tMbudbOK4z/i9FCl7+zcdXRR+g7ms2GZ5RIas=; b=DRhgM5EZoy586+mvA17ZCUKk3k RwKuzecdold0Vwl8+n8lKsLVCx2HtACRaE87m9h2pxGBPG5xTPGIaGWZ3E8J9DtWiRCnhBJGP3VS3 S4pNnN0WkdrjXrh9GcNhZBu5YPiKB0do8AGIu65Ln2QrBvnmlxCb5JlDr0pvssjsV7fZSBORKNLF1 LytNjv9b/2ZWtQgEKoAaOl33ybq/rz7yPhLliAV4wVibwhAHjBSdwpEFFRhP5Kr/cdB5ebm1kBmq4 rTEgysASkzLKIPZ+fhyWQPAXAV22CF85wO0q4UeUbDnOlS2YXGDJRW3Eeyk0xbJd7ABux9Ckxyml1 TF5KnzZw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q946i-008Bfd-2c; Tue, 13 Jun 2023 13:30:48 +0000 Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q946f-008Bdj-0g for linux-nvme@lists.infradead.org; Tue, 13 Jun 2023 13:30:47 +0000 Received: by mail-qv1-xf2e.google.com with SMTP id 6a1803df08f44-62de5392c7bso7091176d6.2 for ; Tue, 13 Jun 2023 06:30:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1686663039; x=1689255039; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=y1wWr1tMbudbOK4z/i9FCl7+zcdXRR+g7ms2GZ5RIas=; b=gFdKAzdsC4Km23ekn6ACrWo5EYSNsh2vy5ZkHcAK7IVtBMlaYRrYIVVlhxpfZSmMg7 WYpvSKLd+daLU69dYdZTnTZwY8a3cmpIHKv0ThNZnJ6fy8fnDSuBy/U9xfvo5q6YcOBR IbQp2aY1QmzuFR8osKO90LaK49Pp3V82j9Y6OCpRjFey5CkcH2d3FMqFkUDQ2jEKSofN KtwlkG48oBUAOAnP40DTZRYICSL5npHro95D4voyG+gM2QVGgRzdctKeBVmLql56Ev6D n8hQoPRtiE3QruGuT+BRpyUxWzpgIpAN4gBNLFmDtHmBUhJG12ZW7gmICwI+AcnllmkN l3oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686663039; x=1689255039; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=y1wWr1tMbudbOK4z/i9FCl7+zcdXRR+g7ms2GZ5RIas=; b=XGLh0EP+UzCzQHvX5GplbNwEEUT4u7e0C2COwI1WA2OKAcZeymSqDVnn8YUQxwtSPM 7eSOQOM/hD3yHxq3WrG3urUM3eWCaBvtXgC9bKPjy39v/nLMqEC9zIH860lLpIkmWd4X I14UJE1zaD6WmmV2ZOe4XugpFGF447VNb00DFEiYbEsA83QwrJhYkXXQcsqvWicUe0p1 /1wmHrKYvF6vdqyxkYMQBAKG8DimOHV6oQbtGAt7flAwu3zWe2Bzhc601GaWNcaa03kg 9U2jw4cCUbz1ntxcEUPnZkglyr79rplFcUcanFuuv6qRYpz/pdc5B99nGr3nLykgldtk cPcg== X-Gm-Message-State: AC+VfDx+mqSLKvfK/P8S00qCiFGsuNTdBBszH2JiFCYCzf9D87OKMAnL zhsO/duCaR1N8zaYRk7pOVVETA== X-Google-Smtp-Source: ACHHUZ4IcAn5/udys+q00pDgo61L01aeH7U8dVDPeDSvd4nHc0VSAtwZb+HE9DprrPHS4pGK0tojOA== X-Received: by 2002:ad4:5fc5:0:b0:62d:f6f1:10bf with SMTP id jq5-20020ad45fc5000000b0062df6f110bfmr2155854qvb.51.1686663038949; Tue, 13 Jun 2023 06:30:38 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id y16-20020a0ce050000000b006210e0365f7sm3876191qvk.69.2023.06.13.06.30.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 06:30:38 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1q946X-004wZr-Kz; Tue, 13 Jun 2023 10:30:37 -0300 Date: Tue, 13 Jun 2023 10:30:37 -0300 From: Jason Gunthorpe To: Shinichiro Kawasaki Cc: "linux-rdma@vger.kernel.org" , Leon Romanovsky , "linux-nvme@lists.infradead.org" , Damien Le Moal Subject: Re: [PATCH v2] RDMA/cma: prevent rdma id destroy during cma_iw_handler Message-ID: References: <20230612054237.1855292-1-shinichiro.kawasaki@wdc.com> <3x4kcccwy5s2yhni5t26brhgejj24kxyk7bnlabp5zw2js26eb@kjwyilm5d4wc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3x4kcccwy5s2yhni5t26brhgejj24kxyk7bnlabp5zw2js26eb@kjwyilm5d4wc> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230613_063045_388005_62740046 X-CRM114-Status: GOOD ( 17.44 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On Tue, Jun 13, 2023 at 01:43:43AM +0000, Shinichiro Kawasaki wrote: > > I think there is likely some much larger issue with the IW CM if the > > cm_id can be destroyed while the iwcm_id is in use? It is weird that > > there are two id memories for this :\ > > My understanding about the call chain to rdma id destroy is as follows. I guess > _destory_id calls iw_destory_cm_id before destroying the rdma id, but not sure > why it does not wait for cm_id deref by cm_work_handler. > > nvme_rdma_teardown_io_queueus > nvme_rdma_stop_io_queues -> chained to cma_iw_handler > nvme_rdma_free_io_queues > nvme_rdma_free_queue > rdma_destroy_id > mutex_lock(&id_priv->handler_mutex) > destroy_id_handler_unlock > mutex_unlock(&id_priv->handler_mutex) > _destory_id > iw_destroy_cm_id > wait_for_completiion(&id_priv->comp) > kfree(id_priv) Once a destroy_cm_id() has returned that layer is no longer permitted to run or be running in its handlers. The iw cm is broken if it allows this, and that is the cause of the bug. Taking more refs within handlers that are already not allowed to be running is just racy. Jason