From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1A00C43381 for ; Wed, 20 Mar 2019 19:57:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A765021841 for ; Wed, 20 Mar 2019 19:57:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1553111852; bh=SWFYrm27nBw22A07wu7S/JN2aynv69vLMQF3rfbjryI=; h=Subject:To:Cc:From:Date:List-ID:From; b=M1JIDy/3tCHUmIodAPxE/J+z0N3jNK9f69H6DHFNWDm5gvSGHyNCbxeZKmjxs9Pd3 S1D/Nqu/if0DeT90f+Crbl4pTaLMmFSFFOEA1uHd6Ox/Lh5HtCMuGbnWmUHdzICkuG WSb2TPpeptWr8Qm4YQC+JHw+ZmusJVknYw1L+VEs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726487AbfCTT5c (ORCPT ); Wed, 20 Mar 2019 15:57:32 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:34149 "EHLO out5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725988AbfCTT5c (ORCPT ); Wed, 20 Mar 2019 15:57:32 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id E8D0C231A5; Wed, 20 Mar 2019 15:57:30 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Wed, 20 Mar 2019 15:57:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=DNhNpS M4zw11vD5b2gV58sMYEaXnsBucAMdwBpgWIsQ=; b=7fpRgDx6FuyDGBYdloFKwR mMzil1/kVPMf6rl20RN9bpVRkRuq39rHIu+3Gyd6WxJSKoyfp1zp/alMjObjpQ7l HnMMSkxHHnBhdUPMVXGDLHRU67fIvHeOO+2mgXzZdJw2m3/RhdaCY+e1S3hOgXTk 8O/tGfdEKBDW5rYwDie8cG1LpBwEOay9EEmcSdESHB54e7t697Vmjwm0gLktVuEH aQvboYZuhLNEe/zFGvmSZ73gqd7VuFxswS04VCtM5VVM+zwC2zKsYnPfqKRafELo 9/o/zmJb8KIPXlmPS4GdYt4w9p3pI+UDpH9l+Ka4W8IU1c4xcvxlY0S6iyjXw9Zw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrieeigddufedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvffhfffkgggtgfesthekredttd dtlfenucfhrhhomhepoehgrhgvghhkhheslhhinhhugihfohhunhgurghtihhonhdrohhr gheqnecukfhppeekfedrkeeirdekledruddtjeenucfrrghrrghmpehmrghilhhfrhhomh epghhrvghgsehkrhhorghhrdgtohhmnecuvehluhhsthgvrhfuihiivgepvd X-ME-Proxy: Received: from localhost (5356596b.cm-6-7b.dynamic.ziggo.nl [83.86.89.107]) by mail.messagingengine.com (Postfix) with ESMTPA id 63089E475B; Wed, 20 Mar 2019 15:57:30 -0400 (EDT) Subject: FAILED: patch "[PATCH] IB/rdmavt: Fix concurrency panics in QP post_send and modify" failed to apply to 4.9-stable tree To: michael.j.ruhl@intel.com, dennis.dalessandro@intel.com, jgg@mellanox.com, mike.marciniszyn@intel.com, stable@vger.kernel.org Cc: From: Date: Wed, 20 Mar 2019 20:57:20 +0100 Message-ID: <15531118404752@kroah.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ANSI_X3.4-1968 Content-Transfer-Encoding: 8bit Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org The patch below does not apply to the 4.9-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to . thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From d757c60eca9b22f4d108929a24401e0fdecda0b1 Mon Sep 17 00:00:00 2001 From: "Michael J. Ruhl" Date: Tue, 26 Feb 2019 08:45:25 -0800 Subject: [PATCH] IB/rdmavt: Fix concurrency panics in QP post_send and modify to error The RC/UC code path can go through a software loopback. In this code path the receive side QP is manipulated. If two threads are working on the QP receive side (i.e. post_send, and modify_qp to an error state), QP information can be corrupted. (post_send via loopback) set r_sge loop update r_sge (modify_qp) take r_lock update r_sge <---- r_sge is now incorrect (post_send) update r_sge <---- crash, etc. ... This can lead to one of the two following crashes: BUG: unable to handle kernel NULL pointer dereference at (null) IP: hfi1_copy_sge+0xf1/0x2e0 [hfi1] PGD 8000001fe6a57067 PUD 1fd9e0c067 PMD 0 Call Trace: ruc_loopback+0x49b/0xbc0 [hfi1] hfi1_do_send+0x38e/0x3e0 [hfi1] _hfi1_do_send+0x1e/0x20 [hfi1] process_one_work+0x17f/0x440 worker_thread+0x126/0x3c0 kthread+0xd1/0xe0 ret_from_fork_nospec_begin+0x21/0x21 or: BUG: unable to handle kernel NULL pointer dereference at 0000000000000048 IP: rvt_clear_mr_refs+0x45/0x370 [rdmavt] PGD 80000006ae5eb067 PUD ef15d0067 PMD 0 Call Trace: rvt_error_qp+0xaa/0x240 [rdmavt] rvt_modify_qp+0x47f/0xaa0 [rdmavt] ib_security_modify_qp+0x8f/0x400 [ib_core] ib_modify_qp_with_udata+0x44/0x70 [ib_core] modify_qp.isra.23+0x1eb/0x2b0 [ib_uverbs] ib_uverbs_modify_qp+0xaa/0xf0 [ib_uverbs] ib_uverbs_write+0x272/0x430 [ib_uverbs] vfs_write+0xc0/0x1f0 SyS_write+0x7f/0xf0 system_call_fastpath+0x1c/0x21 Fix by using the appropriate locking on the receiving QP. Fixes: 15703461533a ("IB/{hfi1, qib, rdmavt}: Move ruc_loopback to rdmavt") Cc: #v4.9+ Reviewed-by: Mike Marciniszyn Signed-off-by: Michael J. Ruhl Signed-off-by: Dennis Dalessandro Signed-off-by: Jason Gunthorpe diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c index 337365806261..a34b9a2a32b6 100644 --- a/drivers/infiniband/sw/rdmavt/qp.c +++ b/drivers/infiniband/sw/rdmavt/qp.c @@ -2790,6 +2790,18 @@ again: } EXPORT_SYMBOL(rvt_copy_sge); +static enum ib_wc_status loopback_qp_drop(struct rvt_ibport *rvp, + struct rvt_qp *sqp) +{ + rvp->n_pkt_drops++; + /* + * For RC, the requester would timeout and retry so + * shortcut the timeouts and just signal too many retries. + */ + return sqp->ibqp.qp_type == IB_QPT_RC ? + IB_WC_RETRY_EXC_ERR : IB_WC_SUCCESS; +} + /** * ruc_loopback - handle UC and RC loopback requests * @sqp: the sending QP @@ -2862,17 +2874,14 @@ again: } spin_unlock_irqrestore(&sqp->s_lock, flags); - if (!qp || !(ib_rvt_state_ops[qp->state] & RVT_PROCESS_RECV_OK) || + if (!qp) { + send_status = loopback_qp_drop(rvp, sqp); + goto serr_no_r_lock; + } + spin_lock_irqsave(&qp->r_lock, flags); + if (!(ib_rvt_state_ops[qp->state] & RVT_PROCESS_RECV_OK) || qp->ibqp.qp_type != sqp->ibqp.qp_type) { - rvp->n_pkt_drops++; - /* - * For RC, the requester would timeout and retry so - * shortcut the timeouts and just signal too many retries. - */ - if (sqp->ibqp.qp_type == IB_QPT_RC) - send_status = IB_WC_RETRY_EXC_ERR; - else - send_status = IB_WC_SUCCESS; + send_status = loopback_qp_drop(rvp, sqp); goto serr; } @@ -3030,6 +3039,7 @@ do_write: wqe->wr.send_flags & IB_SEND_SOLICITED); send_comp: + spin_unlock_irqrestore(&qp->r_lock, flags); spin_lock_irqsave(&sqp->s_lock, flags); rvp->n_loop_pkts++; flush_send: @@ -3056,6 +3066,7 @@ rnr_nak: } if (sqp->s_rnr_retry_cnt < 7) sqp->s_rnr_retry--; + spin_unlock_irqrestore(&qp->r_lock, flags); spin_lock_irqsave(&sqp->s_lock, flags); if (!(ib_rvt_state_ops[sqp->state] & RVT_PROCESS_RECV_OK)) goto clr_busy; @@ -3084,6 +3095,8 @@ err: rvt_rc_error(qp, wc.status); serr: + spin_unlock_irqrestore(&qp->r_lock, flags); +serr_no_r_lock: spin_lock_irqsave(&sqp->s_lock, flags); rvt_send_complete(sqp, wqe, send_status); if (sqp->ibqp.qp_type == IB_QPT_RC) {