From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steve Wise Subject: Re: [PATCH] IB/core: Don't drain the receive queue for srq attached queue-pair Date: Tue, 26 Apr 2016 12:35:17 -0500 Message-ID: <571FA6D5.2060607@opengridcomputing.com> References: <1461682538-19647-1-git-send-email-sagi@grimberg.me> <571F841D.3010909@opengridcomputing.com> <20160426154328.GA12398@lst.de> <912C9E71-05E3-4ED9-9B41-137E131E3A71@oracle.com> <571F9084.2040506@opengridcomputing.com> <571F917C.40008@sandisk.com> <571F972B.6030904@opengridcomputing.com> <571FA4E5.2030501@sandisk.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <571FA4E5.2030501-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Bart Van Assche , Chuck Lever , Christoph Hellwig Cc: sagi grimberg , "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: linux-rdma@vger.kernel.org On 4/26/2016 12:27 PM, Bart Van Assche wrote: >>>>> Conversely, I would think that a consumer that uses SRQ >>>>> would want a similar drain mechanism to guarantee there >>>>> are no more posted Receives for the associated QP, and >>>>> thus it is safe to destroy it. >>>> >>>> I don't think there is anything now that handles draining an SRQ, nor >>>> ensuring a QP's RQEs are completed/consumed from its SRQ when draining >>>> the QP. Sounds like work is needed here. >>>> >>>> But are there any kernel SRQ consumers at this point? >>> >>> I think the following two: >>> >>> $ git grep -nHE '(ib|rdma)_create_srq' drivers/infiniband/ulp >>> drivers/infiniband/ulp/ipoib/ipoib_cm.c:1521: priv->cm.srq = >>> ib_create_srq(priv->pd, &srq_init_attr); >>> drivers/infiniband/ulp/srpt/ib_srpt.c:2723: sdev->srq = >>> ib_create_srq(sdev->pd, &srq_attr); >> >> If we make the requirement that the ib_drain_rq() caller must consume >> all completions for all QPs attached to an SRQ if they are outstanding, >> then I think we can modify ib_drain_rq() to post the drain recv WR to >> the SRQ. It should work, right? > > At least the ib_srpt driver already guarantees that no further receive > completions will be generated before ib_destroy_qp() is called. But > posting an additional receive WR on the SRQ from inside ib_drain_rq() > shouldn't hurt. Actually, after thinking more about this, posting anything to the SRQ will not force it to complete. The only reason that works for a QP with an RQ is that the QP is in ERR state and thus the RECV WR gets completed with FLUSHED status. So SRQ drain would require some other method... Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html