siw trigger BUG: sleeping function called from invalid context at mm/slab.h:50

siw trigger BUG: sleeping function called from invalid context at mm/slab.h:50

[Bart Van Assche]

Hi Bernard,

If I try to associate the ib_srpt driver with the siw driver the
complaint shown below appears on the console. In iw_cm_listen() I
found the following:

         [ ... ]
	spin_lock_irqsave(&cm_id_priv->lock, flags);
	switch (cm_id_priv->state) {
	case IW_CM_STATE_IDLE:
		cm_id_priv->state = IW_CM_STATE_LISTEN;
		spin_unlock_irqrestore(&cm_id_priv->lock, flags);
		ret = iw_cm_map(cm_id, false);
		if (!ret)
			ret = cm_id->device->ops.iw_create_listen(cm_id,
								  backlog);
		if (ret)
			cm_id_priv->state = IW_CM_STATE_IDLE;
		spin_lock_irqsave(&cm_id_priv->lock, flags);
		break;
	default:
		ret = -EINVAL;
	}
	spin_unlock_irqrestore(&cm_id_priv->lock, flags);
         [ ... ]

So it surprises me that siw_listen_address() calls a function that
can sleep. Do you think this is a correct analysis of the call trace
shown below?

Thanks,

Bart.

Target_Core_ConfigFS: REGISTER -> Allocated Fabric: srpt
iwpm_register_pid: Unable to send a nlmsg (client = 2)
BUG: sleeping function called from invalid context at mm/slab.h:501
in_atomic(): 1, irqs_disabled(): 0, pid: 1097, name: restart-lio-srp
4 locks held by restart-lio-srp/1097:
  #0: 0000000083aba319 (sb_writers#11){.+.+}, at: vfs_write+0x24a/0x2c0
  #1: 00000000dc6d2df5 (&buffer->mutex){+.+.}, at: configfs_write_file+0x4e/0x1d0 [configfs]
  #2: 000000004db62204 (lock#7){+.+.}, at: rdma_listen+0x34c/0x450 [rdma_cm]
  #3: 00000000ca45d89f (&ndev->lock){++--}, at: siw_create_listen+0x1d1/0x8b0 [siw]
Preemption disabled at:
[<ffffffffa0db6071>] siw_create_listen+0x1d1/0x8b0 [siw]
CPU: 0 PID: 1097 Comm: restart-lio-srp Not tainted 5.3.0-rc5-dbg+ #2
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Call Trace:
  dump_stack+0x8a/0xd6
  ___might_sleep.cold+0x128/0x139
  __might_sleep+0x76/0xe0
  kmem_cache_alloc+0x2b4/0x3a0
  sock_alloc_inode+0x20/0xf0
  alloc_inode+0x34/0xe0
  new_inode_pseudo+0x17/0x90
  sock_alloc+0x2f/0x110
  __sock_create+0x61/0x360
  sock_create+0x5f/0x70
  siw_listen_address+0xbe/0x630 [siw]
  siw_create_listen+0x2f5/0x8b0 [siw]
  iw_cm_listen+0xd9/0x110 [iw_cm]
  rdma_listen+0x32e/0x450 [rdma_cm]
  cma_listen_on_dev+0x276/0x290 [rdma_cm]
  rdma_listen+0x3c7/0x450 [rdma_cm]
  srpt_create_rdma_id+0xc8/0x120 [ib_srpt]
  srpt_rdma_cm_port_store+0x1a0/0x1f0 [ib_srpt]
  configfs_write_file+0x15c/0x1d0 [configfs]
  __vfs_write+0x4c/0x90
  vfs_write+0x145/0x2c0
  ksys_write+0xd7/0x180
  __x64_sys_write+0x47/0x50
  do_syscall_64+0x75/0x280
  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[...]
siw: device registration error -23

Re: siw trigger BUG: sleeping function called from invalid context at mm/slab.h:50

[Bart Van Assche]

On 8/23/19 4:02 PM, Bart Van Assche wrote:
> If I try to associate the ib_srpt driver with the siw driver the
> complaint shown below appears on the console.
According to gdb:

(gdb) list *(siw_create_listen+0x2f5)
0x6195 is in siw_create_listen (drivers/infiniband/sw/siw
/siw_cm.c:2011).
2006                            bind_addr.sin6_port =
                                         s_laddr->sin6_port;
2007                            bind_addr.sin6_flowinfo = 0;
2008                            bind_addr.sin6_addr = ifp->addr;
2009                            bind_addr.sin6_scope_id = dev->ifindex;
2010
2011                            rv = siw_listen_address(id, backlog,
2012                                      (struct sockaddr *)&bind_addr,
2013                                            AF_INET6);
2014                            if (!rv)
2015                                      listeners++;

This is the code that causes trouble:

	read_lock_bh(&in6_dev->lock);
	list_for_each_entry(ifp, &in6_dev->addr_list, if_list) {
		struct sockaddr_in6 bind_addr;

		if (ipv6_addr_any(&s_laddr->sin6_addr) ||
		    ipv6_addr_equal(&s_laddr->sin6_addr, &ifp->addr)) {
			bind_addr.sin6_family = AF_INET6;
			bind_addr.sin6_port = s_laddr->sin6_port;
			bind_addr.sin6_flowinfo = 0;
			bind_addr.sin6_addr = ifp->addr;
			bind_addr.sin6_scope_id = dev->ifindex;
				rv = siw_listen_address(id, backlog,
					(struct sockaddr *)&bind_addr,
					AF_INET6);
			if (!rv)
				listeners++;
		}
	}
	read_unlock_bh(&in6_dev->lock);

siw_listen_address() calls sock_create(). I don't think it is allowed to 
call sock_create() from atomic context.

Thanks,

Bart.

Re: siw trigger BUG: sleeping function called from invalid context at mm/slab.h:50

[Bernard Metzler]

-----"Bart Van Assche" <bvanassche@acm.org> wrote: -----

>To: "Bernard Metzler" <bmt@zurich.ibm.com>,
>linux-rdma@vger.kernel.org
>From: "Bart Van Assche" <bvanassche@acm.org>
>Date: 08/24/2019 01:02AM
>Subject: [EXTERNAL] siw trigger BUG: sleeping function called from
>invalid context at mm/slab.h:50
>
>Hi Bernard,
>
>If I try to associate the ib_srpt driver with the siw driver the
>complaint shown below appears on the console. In iw_cm_listen() I
>found the following:
>
>         [ ... ]
>	spin_lock_irqsave(&cm_id_priv->lock, flags);
>	switch (cm_id_priv->state) {
>	case IW_CM_STATE_IDLE:
>		cm_id_priv->state = IW_CM_STATE_LISTEN;
>		spin_unlock_irqrestore(&cm_id_priv->lock, flags);
>		ret = iw_cm_map(cm_id, false);
>		if (!ret)
>			ret = cm_id->device->ops.iw_create_listen(cm_id,
>								  backlog);
>		if (ret)
>			cm_id_priv->state = IW_CM_STATE_IDLE;
>		spin_lock_irqsave(&cm_id_priv->lock, flags);
>		break;
>	default:
>		ret = -EINVAL;
>	}
>	spin_unlock_irqrestore(&cm_id_priv->lock, flags);
>         [ ... ]
>
>So it surprises me that siw_listen_address() calls a function that
>can sleep. Do you think this is a correct analysis of the call trace
>shown below?
>

Hi Bart,

Yes, this is a BUG. Thanks very much for spotting it!
Obviously, there is not similar thing for ipv6 like
'in_dev_for_each_ifa_rtnl()'. Let me think about a
good fix to that and come back asap.

Many thanks!
Bernard.