[PATCH rdma-rc] RDMA/cm: Fix IRQ restore in ib_send_cm_sidr_rep

[PATCH rdma-rc] RDMA/cm: Fix IRQ restore in ib_send_cm_sidr_rep

[Leon Romanovsky]

From: Saeed Mahameed <saeedm@nvidia.com>

ib_send_cm_sidr_rep() {
	spin_lock_irqsave()
        cm_send_sidr_rep_locked() {
                ...
        	spin_lock_irq()
                ....
                spin_unlock_irq() <--- this will enable interrupts
        }
        spin_unlock_irqrestore()
}

spin_unlock_irqrestore() expects interrupts to be disabled
but the internal spin_unlock_irq() will always enable hard interrupts.

Fix this by replacing the internal spin_{lock,unlock}_irq() with
irqsave/restore variants.

It fixes the following kernel trace:

 ------------[ cut here ]------------
 raw_local_irq_restore() called with IRQs enabled
 WARNING: CPU: 2 PID: 20001 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x1d/0x20

 Call Trace:
  _raw_spin_unlock_irqrestore+0x4e/0x50
  ib_send_cm_sidr_rep+0x3a/0x50 [ib_cm]
  cma_send_sidr_rep+0xa1/0x160 [rdma_cm]
  rdma_accept+0x25e/0x350 [rdma_cm]
  ucma_accept+0x132/0x1cc [rdma_ucm]
  ucma_write+0xbf/0x140 [rdma_ucm]
  vfs_write+0xc1/0x340
  ksys_write+0xb3/0xe0
  do_syscall_64+0x2d/0x40
  entry_SYSCALL_64_after_hwframe+0x44/0xae

Fixes: 87c4c774cbef ("RDMA/cm: Protect access to remote_sidr_table")
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/core/cm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index be996dba040c..3d194bb60840 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -3651,6 +3651,7 @@ static int cm_send_sidr_rep_locked(struct cm_id_private *cm_id_priv,
 				   struct ib_cm_sidr_rep_param *param)
 {
 	struct ib_mad_send_buf *msg;
+	unsigned long flags;
 	int ret;

 	lockdep_assert_held(&cm_id_priv->lock);
@@ -3676,12 +3677,12 @@ static int cm_send_sidr_rep_locked(struct cm_id_private *cm_id_priv,
 		return ret;
 	}
 	cm_id_priv->id.state = IB_CM_IDLE;
-	spin_lock_irq(&cm.lock);
+	spin_lock_irqsave(&cm.lock, flags);
 	if (!RB_EMPTY_NODE(&cm_id_priv->sidr_id_node)) {
 		rb_erase(&cm_id_priv->sidr_id_node, &cm.remote_sidr_table);
 		RB_CLEAR_NODE(&cm_id_priv->sidr_id_node);
 	}
-	spin_unlock_irq(&cm.lock);
+	spin_unlock_irqrestore(&cm.lock, flags);
 	return 0;
 }

--
2.29.2

Re: [PATCH rdma-rc] RDMA/cm: Fix IRQ restore in ib_send_cm_sidr_rep

[Jason Gunthorpe]

On Mon, Mar 01, 2021 at 10:18:44AM +0200, Leon Romanovsky wrote:
> From: Saeed Mahameed <saeedm@nvidia.com>
> 
> ib_send_cm_sidr_rep() {
> 	spin_lock_irqsave()
>         cm_send_sidr_rep_locked() {
>                 ...
>         	spin_lock_irq()
>                 ....
>                 spin_unlock_irq() <--- this will enable interrupts
>         }
>         spin_unlock_irqrestore()
> }
> 
> spin_unlock_irqrestore() expects interrupts to be disabled
> but the internal spin_unlock_irq() will always enable hard interrupts.
> 
> Fix this by replacing the internal spin_{lock,unlock}_irq() with
> irqsave/restore variants.
> 
> It fixes the following kernel trace:
> 
>  ------------[ cut here ]------------
>  raw_local_irq_restore() called with IRQs enabled
>  WARNING: CPU: 2 PID: 20001 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x1d/0x20
> 
>  Call Trace:
>   _raw_spin_unlock_irqrestore+0x4e/0x50
>   ib_send_cm_sidr_rep+0x3a/0x50 [ib_cm]
>   cma_send_sidr_rep+0xa1/0x160 [rdma_cm]
>   rdma_accept+0x25e/0x350 [rdma_cm]
>   ucma_accept+0x132/0x1cc [rdma_ucm]
>   ucma_write+0xbf/0x140 [rdma_ucm]
>   vfs_write+0xc1/0x340
>   ksys_write+0xb3/0xe0
>   do_syscall_64+0x2d/0x40
>   entry_SYSCALL_64_after_hwframe+0x44/0xae
> 
> Fixes: 87c4c774cbef ("RDMA/cm: Protect access to remote_sidr_table")
> Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
>  drivers/infiniband/core/cm.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

Applied to for-rc, thanks

Jason