[PATCH for-rc v2] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event

[PATCH for-rc v2] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event

[Anumula Murali Mohan Reddy]

This patch sends IB_EVENT_QP_LAST_WQE_REACHED event on a QP that is in
error state and associated with an SRQ. This behaviour is incorporated
in flush_qp() which is called when QP transitions to error state.
Supports SRQ drain functionality added by commit 844bc12e6da3 ("IB/core:
add support for draining Shared receive queues")

Fixes: 844bc12e6da3 ("IB/core: add support for draining Shared receive queues")
Signed-off-by: Anumula Murali Mohan Reddy <anumula@chelsio.com>
Signed-off-by: Potnuri Bharat Teja <bharat@chelsio.com>
---
Changes since v1:
Addressed previous review comments
---
 drivers/infiniband/hw/cxgb4/qp.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
index 7b5c4522b426..10f61bc16dd5 100644
--- a/drivers/infiniband/hw/cxgb4/qp.c
+++ b/drivers/infiniband/hw/cxgb4/qp.c
@@ -1599,6 +1599,7 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
 	int count;
 	int rq_flushed = 0, sq_flushed;
 	unsigned long flag;
+	struct ib_event ev;
 
 	pr_debug("qhp %p rchp %p schp %p\n", qhp, rchp, schp);
 
@@ -1607,6 +1608,14 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
 	if (schp != rchp)
 		spin_lock(&schp->lock);
 	spin_lock(&qhp->lock);
+	if (qhp->srq) {
+		if (qhp->attr.state == C4IW_QP_STATE_ERROR && qhp->ibqp.event_handler) {
+			ev.device = qhp->ibqp.device;
+			ev.element.qp = &qhp->ibqp;
+			ev.event = IB_EVENT_QP_LAST_WQE_REACHED;
+			qhp->ibqp.event_handler(&ev, qhp->ibqp.qp_context);
+		}
+	}
 
 	if (qhp->wq.flushed) {
 		spin_unlock(&qhp->lock);
-- 
2.39.3

Re: [PATCH for-rc v2] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event

[Leon Romanovsky]

On Tue, Jan 07, 2025 at 03:20:53PM +0530, Anumula Murali Mohan Reddy wrote:
> This patch sends IB_EVENT_QP_LAST_WQE_REACHED event on a QP that is in
> error state and associated with an SRQ. This behaviour is incorporated
> in flush_qp() which is called when QP transitions to error state.
> Supports SRQ drain functionality added by commit 844bc12e6da3 ("IB/core:
> add support for draining Shared receive queues")
> 
> Fixes: 844bc12e6da3 ("IB/core: add support for draining Shared receive queues")
> Signed-off-by: Anumula Murali Mohan Reddy <anumula@chelsio.com>
> Signed-off-by: Potnuri Bharat Teja <bharat@chelsio.com>
> ---
> Changes since v1:
> Addressed previous review comments

There are two review comments. One is Fixes line, which is not a big deal
as I can add it when applying patch, but what about second comment?

Why is this event limited to QP with SRQ only?
https://lore.kernel.org/all/20250105090622.GA5511@unreal/

Thanks

> ---
>  drivers/infiniband/hw/cxgb4/qp.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
> index 7b5c4522b426..10f61bc16dd5 100644
> --- a/drivers/infiniband/hw/cxgb4/qp.c
> +++ b/drivers/infiniband/hw/cxgb4/qp.c
> @@ -1599,6 +1599,7 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
>  	int count;
>  	int rq_flushed = 0, sq_flushed;
>  	unsigned long flag;
> +	struct ib_event ev;
>  
>  	pr_debug("qhp %p rchp %p schp %p\n", qhp, rchp, schp);
>  
> @@ -1607,6 +1608,14 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
>  	if (schp != rchp)
>  		spin_lock(&schp->lock);
>  	spin_lock(&qhp->lock);
> +	if (qhp->srq) {
> +		if (qhp->attr.state == C4IW_QP_STATE_ERROR && qhp->ibqp.event_handler) {
> +			ev.device = qhp->ibqp.device;
> +			ev.element.qp = &qhp->ibqp;
> +			ev.event = IB_EVENT_QP_LAST_WQE_REACHED;
> +			qhp->ibqp.event_handler(&ev, qhp->ibqp.qp_context);
> +		}
> +	}
>  
>  	if (qhp->wq.flushed) {
>  		spin_unlock(&qhp->lock);
> -- 
> 2.39.3
> 
> 

Re: [PATCH for-rc v2] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event

[Anumula Murali Mohan Reddy]

On Tuesday, January 01/07/25, 2025 at 15:49:11 +0530, Leon Romanovsky wrote:
> On Tue, Jan 07, 2025 at 03:20:53PM +0530, Anumula Murali Mohan Reddy wrote:
> > This patch sends IB_EVENT_QP_LAST_WQE_REACHED event on a QP that is in
> > error state and associated with an SRQ. This behaviour is incorporated
> > in flush_qp() which is called when QP transitions to error state.
> > Supports SRQ drain functionality added by commit 844bc12e6da3 ("IB/core:
> > add support for draining Shared receive queues")
> > 
> > Fixes: 844bc12e6da3 ("IB/core: add support for draining Shared receive queues")
> > Signed-off-by: Anumula Murali Mohan Reddy <anumula@chelsio.com>
> > Signed-off-by: Potnuri Bharat Teja <bharat@chelsio.com>
> > ---
> > Changes since v1:
> > Addressed previous review comments
> 
> There are two review comments. One is Fixes line, which is not a big deal
> as I can add it when applying patch, but what about second comment?
> 
> Why is this event limited to QP with SRQ only?
> https://lore.kernel.org/all/20250105090622.GA5511@unreal/
> 
> Thanks
> 
> > ---
> >  drivers/infiniband/hw/cxgb4/qp.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> > 
> > diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
> > index 7b5c4522b426..10f61bc16dd5 100644
> > --- a/drivers/infiniband/hw/cxgb4/qp.c
> > +++ b/drivers/infiniband/hw/cxgb4/qp.c
> > @@ -1599,6 +1599,7 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
> >  	int count;
> >  	int rq_flushed = 0, sq_flushed;
> >  	unsigned long flag;
> > +	struct ib_event ev;
> >  
> >  	pr_debug("qhp %p rchp %p schp %p\n", qhp, rchp, schp);
> >  
> > @@ -1607,6 +1608,14 @@ static void __flush_qp(struct c4iw_qp *qhp, struct c4iw_cq *rchp,
> >  	if (schp != rchp)
> >  		spin_lock(&schp->lock);
> >  	spin_lock(&qhp->lock);
> > +	if (qhp->srq) {
> > +		if (qhp->attr.state == C4IW_QP_STATE_ERROR && qhp->ibqp.event_handler) {
> > +			ev.device = qhp->ibqp.device;
> > +			ev.element.qp = &qhp->ibqp;
> > +			ev.event = IB_EVENT_QP_LAST_WQE_REACHED;
> > +			qhp->ibqp.event_handler(&ev, qhp->ibqp.qp_context);
> > +		}
> > +	}
> >  
> >  	if (qhp->wq.flushed) {
> >  		spin_unlock(&qhp->lock);
> > -- 
> > 2.39.3
> > 
> >
RDMA stack expects the IB_EVENT_QP_LAST_WQE_REACHED event only for SRQ
, for non-SRQ  it is handled via the drain cqe

Thanks,
Anumula Murali Mohan Reddy

Re: [PATCH for-rc v2] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event

[Leon Romanovsky]

On Tue, 07 Jan 2025 15:20:53 +0530, Anumula Murali Mohan Reddy wrote:
> This patch sends IB_EVENT_QP_LAST_WQE_REACHED event on a QP that is in
> error state and associated with an SRQ. This behaviour is incorporated
> in flush_qp() which is called when QP transitions to error state.
> Supports SRQ drain functionality added by commit 844bc12e6da3 ("IB/core:
> add support for draining Shared receive queues")
> 
> 
> [...]

Applied, thanks!

[1/1] RDMA/cxgb4: notify rdma stack for IB_EVENT_QP_LAST_WQE_REACHED event
      https://git.kernel.org/rdma/rdma/c/42e6ddda4c17fa

Best regards,
-- 
Leon Romanovsky <leon@kernel.org>