All of lore.kernel.org
 help / color / mirror / Atom feed
From: dledford@redhat.com (Doug Ledford)
Subject: [PATCH v2 0/2] iwarp device removal deadlock fixes
Date: Tue, 02 Aug 2016 13:24:16 -0400	[thread overview]
Message-ID: <1470158656.18081.17.camel@redhat.com> (raw)
In-Reply-To: <cover.1469824577.git.swise@opengridcomputing.com>

On Fri, 2016-07-29@13:36 -0700, Steve Wise wrote:
> This series fixes the deadlock issue discovered while testing
> nvmf/rdma
> handling rdma device removal events from the rdma_cm.??For a
> discussion
> of the deadlock that can happen, see
> 
> http://lists.infradead.org/pipermail/linux-nvme/2016-July/005440.html
> .
> 
> For my description of the deadlock itself, see this post in the above
> thread:
> 
> http://lists.infradead.org/pipermail/linux-nvme/2016-July/005465.html
> 
> In a nutshell, iw_cxgb4 and the iw_cm block during qp/cm_id
> destruction
> until all references are removed.??This combined with the iwarp CM
> passing
> disconnect events up to the rdma_cm during disconnect and/or qp/cm_id
> destruction
> leads to a deadlock.
> 
> My proposed solution is to remove the need for iw_cxgb4 and iw_cm to
> block during object destruction for the refnts to reach 0, but rather
> to
> let the freeing of the object memory be deferred when the last deref
> is
> done, which is SOP in the much of the linux kernel. This allows all
> the
> qps/cm_ids to be destroyed without blocking, and all the object
> memory
> freeing ends up happinging when the application's device_remove event
> handler function returns to the rdma_cm.
> 
> This series is needed along with Sagi's fixes from:
> https://www.spinics.net/lists/linux-rdma/msg38715.html
> 
> Hey Faisal, it would be great to get some review/test tags from Intel
> on the iw_cm change.??Thanks!
> 
> Changes since v1:
> 
> - reworded commit text for the iw_cm patch
> 
> - added a iw_cm_id flag to drop pending events when the cm_id has
> been marked for destruction.
> 
> ---
> 
> Steve Wise (2):
> ? iw_cxgb4: don't block in destroy_qp awaiting the last deref
> ? iw_cm: free cm_id resources on the last deref
> 
> ?drivers/infiniband/core/iwcm.c?????????| 54 +++++++++++-------------
> ----------
> ?drivers/infiniband/core/iwcm.h?????????|??2 +-
> ?drivers/infiniband/hw/cxgb4/iw_cxgb4.h |??2 +-
> ?drivers/infiniband/hw/cxgb4/qp.c???????| 21 ++++++++-----
> ?4 files changed, 33 insertions(+), 46 deletions(-)
> 

Series applied, thanks.

-- 
Doug Ledford <dledford at redhat.com>
              GPG KeyID: 0E572FDD
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20160802/ec1255d9/attachment.sig>

WARNING: multiple messages have this Message-ID (diff)
From: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
To: Steve Wise
	<swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>,
	linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org,
	sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	shiraz.saleem-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
Subject: Re: [PATCH v2 0/2] iwarp device removal deadlock fixes
Date: Tue, 02 Aug 2016 13:24:16 -0400	[thread overview]
Message-ID: <1470158656.18081.17.camel@redhat.com> (raw)
In-Reply-To: <cover.1469824577.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

On Fri, 2016-07-29 at 13:36 -0700, Steve Wise wrote:
> This series fixes the deadlock issue discovered while testing
> nvmf/rdma
> handling rdma device removal events from the rdma_cm.  For a
> discussion
> of the deadlock that can happen, see
> 
> http://lists.infradead.org/pipermail/linux-nvme/2016-July/005440.html
> .
> 
> For my description of the deadlock itself, see this post in the above
> thread:
> 
> http://lists.infradead.org/pipermail/linux-nvme/2016-July/005465.html
> 
> In a nutshell, iw_cxgb4 and the iw_cm block during qp/cm_id
> destruction
> until all references are removed.  This combined with the iwarp CM
> passing
> disconnect events up to the rdma_cm during disconnect and/or qp/cm_id
> destruction
> leads to a deadlock.
> 
> My proposed solution is to remove the need for iw_cxgb4 and iw_cm to
> block during object destruction for the refnts to reach 0, but rather
> to
> let the freeing of the object memory be deferred when the last deref
> is
> done, which is SOP in the much of the linux kernel. This allows all
> the
> qps/cm_ids to be destroyed without blocking, and all the object
> memory
> freeing ends up happinging when the application's device_remove event
> handler function returns to the rdma_cm.
> 
> This series is needed along with Sagi's fixes from:
> https://www.spinics.net/lists/linux-rdma/msg38715.html
> 
> Hey Faisal, it would be great to get some review/test tags from Intel
> on the iw_cm change.  Thanks!
> 
> Changes since v1:
> 
> - reworded commit text for the iw_cm patch
> 
> - added a iw_cm_id flag to drop pending events when the cm_id has
> been marked for destruction.
> 
> ---
> 
> Steve Wise (2):
>   iw_cxgb4: don't block in destroy_qp awaiting the last deref
>   iw_cm: free cm_id resources on the last deref
> 
>  drivers/infiniband/core/iwcm.c         | 54 +++++++++++-------------
> ----------
>  drivers/infiniband/core/iwcm.h         |  2 +-
>  drivers/infiniband/hw/cxgb4/iw_cxgb4.h |  2 +-
>  drivers/infiniband/hw/cxgb4/qp.c       | 21 ++++++++-----
>  4 files changed, 33 insertions(+), 46 deletions(-)
> 

Series applied, thanks.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

  parent reply	other threads:[~2016-08-02 17:24 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-29 20:36 [PATCH v2 0/2] iwarp device removal deadlock fixes Steve Wise
2016-07-29 20:36 ` Steve Wise
2016-07-29 18:00 ` [PATCH v2 2/2] iw_cm: free cm_id resources on the last deref Steve Wise
2016-07-29 18:00   ` Steve Wise
2016-07-29 18:00 ` [PATCH v2 1/2] iw_cxgb4: don't block in destroy_qp awaiting " Steve Wise
2016-07-29 18:00   ` Steve Wise
2016-07-31 14:29   ` Sagi Grimberg
2016-07-31 14:29     ` Sagi Grimberg
2016-08-02 17:24 ` Doug Ledford [this message]
2016-08-02 17:24   ` [PATCH v2 0/2] iwarp device removal deadlock fixes Doug Ledford

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1470158656.18081.17.camel@redhat.com \
    --to=dledford@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.