All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Steve Wise" <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
To: 'Sagi Grimberg'
	<sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>,
	'Christoph Hellwig' <hch-jcswGhMUV9g@public.gmane.org>,
	linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org,
	axboe-b10kYP2dOMg@public.gmane.org,
	linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: RE: [PATCH 3/9] IB: add a helper to safely drain a QP
Date: Mon, 16 Nov 2015 13:03:27 -0600	[thread overview]
Message-ID: <003801d120a1$799338d0$6cb9aa70$@opengridcomputing.com> (raw)
In-Reply-To: <564A2270.1040004-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>



> -----Original Message-----
> From: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org] On Behalf Of Sagi Grimberg
> Sent: Monday, November 16, 2015 12:38 PM
> To: Steve Wise; 'Christoph Hellwig'; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org; axboe-b10kYP2dOMg@public.gmane.org; linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Subject: Re: [PATCH 3/9] IB: add a helper to safely drain a QP
> 
> 
> > After looking at the nes driver, I don't see any common way to support drain w/o some serious driver mods.  Since SRP is the
only
> > user, perhaps we can ignore iWARP for this function...
> 
> But iser/isert essentially does it too (and I think xprtrdma will have
> it soon)...
> 
> the modify_qp is invoked from rdma_disconnect() and we do post
> an 'empty' wr to wait for all the flushes to drain (see
> iser_conn_terminate).

That won't work for iWARP.  Is this code new?  I didn't see any errors that would result from this code when I tested iSER over
cxgb4 with the old iwarp support patches.   

Perhaps we need another way to do this?  Like a completion object in the QP that gets triggered when the SQ and RQ become empty
after a transition to ERROR (and CLOSING for iwarp).  Then a core service that just waits until the QP is empty.  Implementation of
this design would hit the providers though since only they know when the flush is completed.

Alternatively, I could enable post-while-in-error support in cxgb4 and ignore the spec in this regard.  But I'd rather not do that.
:)

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WARNING: multiple messages have this Message-ID (diff)
From: "Steve Wise" <swise@opengridcomputing.com>
To: "'Sagi Grimberg'" <sagig@dev.mellanox.co.il>,
	"'Christoph Hellwig'" <hch@lst.de>, <linux-rdma@vger.kernel.org>
Cc: <bart.vanassche@sandisk.com>, <axboe@fb.com>,
	<linux-scsi@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 3/9] IB: add a helper to safely drain a QP
Date: Mon, 16 Nov 2015 13:03:27 -0600	[thread overview]
Message-ID: <003801d120a1$799338d0$6cb9aa70$@opengridcomputing.com> (raw)
In-Reply-To: <564A2270.1040004@dev.mellanox.co.il>



> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-owner@vger.kernel.org] On Behalf Of Sagi Grimberg
> Sent: Monday, November 16, 2015 12:38 PM
> To: Steve Wise; 'Christoph Hellwig'; linux-rdma@vger.kernel.org
> Cc: bart.vanassche@sandisk.com; axboe@fb.com; linux-scsi@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 3/9] IB: add a helper to safely drain a QP
> 
> 
> > After looking at the nes driver, I don't see any common way to support drain w/o some serious driver mods.  Since SRP is the
only
> > user, perhaps we can ignore iWARP for this function...
> 
> But iser/isert essentially does it too (and I think xprtrdma will have
> it soon)...
> 
> the modify_qp is invoked from rdma_disconnect() and we do post
> an 'empty' wr to wait for all the flushes to drain (see
> iser_conn_terminate).

That won't work for iWARP.  Is this code new?  I didn't see any errors that would result from this code when I tested iSER over
cxgb4 with the old iwarp support patches.   

Perhaps we need another way to do this?  Like a completion object in the QP that gets triggered when the SQ and RQ become empty
after a transition to ERROR (and CLOSING for iwarp).  Then a core service that just waits until the QP is empty.  Implementation of
this design would hit the providers though since only they know when the flush is completed.

Alternatively, I could enable post-while-in-error support in cxgb4 and ignore the spec in this regard.  But I'd rather not do that.
:)

Steve.


  parent reply	other threads:[~2015-11-16 19:03 UTC|newest]

Thread overview: 140+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-13 13:46 add a proper completion queue abstraction Christoph Hellwig
2015-11-13 13:46 ` [PATCH 1/9] move blk_iopoll to limit and make it generally available Christoph Hellwig
     [not found]   ` <1447422410-20891-2-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-13 15:23     ` Or Gerlitz
2015-11-13 15:23       ` Or Gerlitz
     [not found]       ` <CAJ3xEMgj2ycv61K38ZOowTRbrri_UhQgBcaKT0ZnnMHiBrmL5A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-14  7:02         ` Christoph Hellwig
2015-11-14  7:02           ` Christoph Hellwig
     [not found]           ` <20151114070200.GA27738-jcswGhMUV9g@public.gmane.org>
2015-11-15  8:48             ` Sagi Grimberg
2015-11-15  8:48               ` Sagi Grimberg
     [not found]               ` <564846E9.9070301-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-15  9:04                 ` Or Gerlitz
2015-11-15  9:04                   ` Or Gerlitz
     [not found]                   ` <CAJ3xEMgvttM1D3bePz0CWhZAZ3gCSQsf_qgmq9Ny4gzK5d0bXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-11-15 13:16                     ` Sagi Grimberg
2015-11-15 13:16                       ` Sagi Grimberg
2015-11-15 12:51                 ` Christoph Hellwig
2015-11-15 12:51                   ` Christoph Hellwig
2015-11-13 19:19     ` Bart Van Assche
2015-11-13 19:19       ` Bart Van Assche
2015-11-14  7:02       ` Christoph Hellwig
2015-11-17 17:16         ` Bart Van Assche
2015-11-17 17:16           ` Bart Van Assche
2015-11-17 17:27           ` Bart Van Assche
2015-11-18 13:58           ` Christoph Hellwig
2015-11-13 13:46 ` [PATCH 2/9] IB: add a proper completion queue abstraction Christoph Hellwig
2015-11-15  9:40   ` Sagi Grimberg
     [not found]     ` <564852F2.5080602-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-15 12:55       ` Christoph Hellwig
2015-11-15 12:55         ` Christoph Hellwig
     [not found]         ` <20151115125501.GB2218-jcswGhMUV9g@public.gmane.org>
2015-11-15 13:21           ` Sagi Grimberg
2015-11-15 13:21             ` Sagi Grimberg
     [not found]   ` <1447422410-20891-3-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-13 18:25     ` Jason Gunthorpe
2015-11-13 18:25       ` Jason Gunthorpe
     [not found]       ` <20151113182513.GB21808-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-13 19:57         ` Bart Van Assche
2015-11-13 19:57           ` Bart Van Assche
2015-11-13 22:06           ` Jason Gunthorpe
     [not found]             ` <20151113220636.GA32133-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-14  7:13               ` Christoph Hellwig
2015-11-14  7:13                 ` Christoph Hellwig
     [not found]                 ` <20151114071344.GE27738-jcswGhMUV9g@public.gmane.org>
2015-11-23 20:37                   ` Jason Gunthorpe
2015-11-23 20:37                     ` Jason Gunthorpe
2015-11-23 21:04                     ` Bart Van Assche
2015-11-23 21:04                       ` Bart Van Assche
2015-11-23 21:28                       ` Jason Gunthorpe
2015-11-23 21:54                         ` Bart Van Assche
2015-11-23 22:18                           ` Jason Gunthorpe
2015-11-23 22:33                             ` Bart Van Assche
2015-11-23 23:06                               ` Jason Gunthorpe
     [not found]                                 ` <B24F4DDE-709A-4D2D-8B26-4E83325DBB1A@asomi.com>
2015-11-24  0:00                                   ` Jason Gunthorpe
     [not found]                                     ` <20151124000011.GA9301-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-24  0:34                                       ` Tom Talpey
2015-11-24  0:34                                         ` Tom Talpey
     [not found]                                         ` <5653B0AD.7090402-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>
2015-11-24  0:40                                           ` Jason Gunthorpe
2015-11-24  0:40                                             ` Jason Gunthorpe
2015-11-24  2:35                                       ` Caitlin Bestler
2015-11-24  2:35                                         ` Caitlin Bestler
     [not found]                                         ` <5653CCF0.7050501-DpaxOq6QOWMAvxtiuMwx3w@public.gmane.org>
2015-11-24  7:03                                           ` Jason Gunthorpe
2015-11-24  7:03                                             ` Jason Gunthorpe
     [not found]                                             ` <20151124070301.GA23597-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-24 12:52                                               ` Tom Talpey
2015-11-24 12:52                                                 ` Tom Talpey
2015-11-14  7:08         ` Christoph Hellwig
2015-11-14  7:08           ` Christoph Hellwig
2015-11-23 20:01           ` Jason Gunthorpe
     [not found]             ` <20151123200136.GA5640-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-11-23 20:57               ` Christoph Hellwig
2015-11-23 20:57                 ` Christoph Hellwig
2015-11-17 17:52     ` Bart Van Assche
2015-11-17 17:52       ` Bart Van Assche
     [not found]       ` <564B697A.2020601-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-11-18  7:55         ` Sagi Grimberg
2015-11-18  7:55           ` Sagi Grimberg
     [not found]           ` <564C2F01.6020407-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-18 18:20             ` Bart Van Assche
2015-11-18 18:20               ` Bart Van Assche
2015-11-20 10:16               ` Christoph Hellwig
2015-11-20 16:50                 ` Bart Van Assche
2015-11-22  9:51                   ` Sagi Grimberg
2015-11-22 10:13                     ` Christoph Hellwig
     [not found]                       ` <20151122101308.GA12189-jcswGhMUV9g@public.gmane.org>
2015-11-22 10:36                         ` Sagi Grimberg
2015-11-22 10:36                           ` Sagi Grimberg
     [not found]                           ` <56519A90.5010502-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-22 13:23                             ` Christoph Hellwig
2015-11-22 13:23                               ` Christoph Hellwig
     [not found]                               ` <20151122132352.GA14154-jcswGhMUV9g@public.gmane.org>
2015-11-22 14:57                                 ` Sagi Grimberg
2015-11-22 14:57                                   ` Sagi Grimberg
2015-11-22 16:55                                   ` Bart Van Assche
2015-11-18 14:00         ` Christoph Hellwig
2015-11-18 14:00           ` Christoph Hellwig
2015-11-13 13:46 ` [PATCH 3/9] IB: add a helper to safely drain a QP Christoph Hellwig
     [not found]   ` <1447422410-20891-4-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-13 16:16     ` Steve Wise
2015-11-13 16:16       ` Steve Wise
     [not found]       ` <56460CC4.3030001-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
2015-11-14  7:05         ` Christoph Hellwig
2015-11-14  7:05           ` Christoph Hellwig
2015-11-18 11:32     ` Sagi Grimberg
2015-11-18 11:32       ` Sagi Grimberg
     [not found]       ` <564C61C3.3050307-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-18 14:06         ` Christoph Hellwig
2015-11-18 14:06           ` Christoph Hellwig
     [not found]           ` <20151118140645.GI18820-jcswGhMUV9g@public.gmane.org>
2015-11-18 15:21             ` Steve Wise
2015-11-18 15:21               ` Steve Wise
2015-11-15  9:34   ` Sagi Grimberg
     [not found]     ` <564851BB.1020004-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-16 16:38       ` Steve Wise
2015-11-16 16:38         ` Steve Wise
     [not found]         ` <564A067B.8030504-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
2015-11-16 18:30           ` Steve Wise
2015-11-16 18:30             ` Steve Wise
2015-11-16 18:37             ` Sagi Grimberg
     [not found]               ` <564A2270.1040004-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-16 19:03                 ` Steve Wise [this message]
2015-11-16 19:03                   ` Steve Wise
2015-11-17  8:54                   ` Sagi Grimberg
2015-11-17  8:54                     ` Sagi Grimberg
2015-11-23 10:28                   ` Sagi Grimberg
2015-11-23 10:28                     ` Sagi Grimberg
2015-11-23 10:35                     ` Sagi Grimberg
2015-11-23 14:33                       ` 'Christoph Hellwig'
     [not found]                       ` <5652EC00.8010705-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-11-23 14:48                         ` Steve Wise
2015-11-23 14:48                           ` Steve Wise
2015-11-23 14:44                     ` Steve Wise
2015-11-23 14:44                       ` Steve Wise
2015-11-17 17:06       ` Bart Van Assche
2015-11-17 17:06         ` Bart Van Assche
     [not found]         ` <564B5E7D.9030309-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-11-18  7:59           ` Sagi Grimberg
2015-11-18  7:59             ` Sagi Grimberg
     [not found] ` <1447422410-20891-1-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-13 13:46   ` [PATCH 4/9] srpt: chain RDMA READ/WRITE requests Christoph Hellwig
2015-11-13 13:46     ` Christoph Hellwig
2015-11-18  1:17     ` Bart Van Assche
2015-11-18  1:17       ` Bart Van Assche
2015-11-18  9:15       ` Sagi Grimberg
2015-11-18 16:32         ` Bart Van Assche
2015-11-18 16:32           ` Bart Van Assche
     [not found]           ` <564CA83B.4060403-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-11-20 10:20             ` Christoph Hellwig
2015-11-20 10:20               ` Christoph Hellwig
     [not found]       ` <564BD1AF.60200-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-11-18 14:06         ` Christoph Hellwig
2015-11-18 14:06           ` Christoph Hellwig
2015-11-13 13:46 ` [PATCH 5/9] srpt: use the new CQ API Christoph Hellwig
     [not found]   ` <1447422410-20891-6-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-17 18:22     ` Bart Van Assche
2015-11-17 18:22       ` Bart Van Assche
2015-11-17 19:38   ` Bart Van Assche
2015-11-17 19:38     ` Bart Van Assche
     [not found]     ` <564B8248.7050407-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-11-18 14:03       ` Christoph Hellwig
2015-11-18 14:03         ` Christoph Hellwig
2015-11-13 13:46 ` [PATCH 6/9] srp: " Christoph Hellwig
     [not found]   ` <1447422410-20891-7-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-17 19:56     ` Bart Van Assche
2015-11-17 19:56       ` Bart Van Assche
2015-11-18 14:03       ` Christoph Hellwig
2015-11-13 13:46 ` [PATCH 7/9] IB/iser: Use a dedicated descriptor for login Christoph Hellwig
     [not found]   ` <1447422410-20891-8-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-15  9:14     ` Or Gerlitz
2015-11-15  9:14       ` Or Gerlitz
2015-11-13 13:46 ` [PATCH 8/9] IB/iser: Use helper for container_of Christoph Hellwig
2015-11-13 13:46 ` [PATCH 9/9] IB/iser: Convert to CQ abstraction Christoph Hellwig
     [not found]   ` <1447422410-20891-10-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-11-15  9:21     ` Or Gerlitz
2015-11-15  9:21       ` Or Gerlitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='003801d120a1$799338d0$6cb9aa70$@opengridcomputing.com' \
    --to=swise-7bpotxp6k4+p2yhjcf5u+vpxobypeauw@public.gmane.org \
    --cc=axboe-b10kYP2dOMg@public.gmane.org \
    --cc=bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org \
    --cc=hch-jcswGhMUV9g@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.