Re: [PATCH net] net: devmem: fix kernel panic when socket close after module unload

[Stanislav Fomichev <stfomichev@gmail.com>]

On 04/15, Jakub Kicinski wrote:
> On Tue, 15 Apr 2025 11:59:40 -0700 Stanislav Fomichev wrote:
> > > commit 42f342387841 ("net: fix use-after-free in the
> > > netdev_nl_sock_priv_destroy()") and rolling back a few fixes, it's
> > > really introduced by commit 1d22d3060b9b ("net: drop rtnl_lock for
> > > queue_mgmt operations").
> > > 
> > > My first question, does this issue still reproduce if you remove the
> > > per netdev locking and go back to relying on rtnl_locking? Or do we
> > > crash somewhere else in net_devmem_unbind_dmabuf? If so, where?
> > > Looking through the rest of the unbinding code, it's not clear to me
> > > any of it actually uses dev, so it may just be the locking...  
> >  
> > A proper fix, most likely, will involve resetting binding->dev to NULL
> > when the device is going away.
> 
> Right, tho a bit of work and tricky handling will be necessary to get
> that right. We're not holding a ref on binding->dev.
> 
> I think we need to invert the socket mutex vs instance lock ordering.
> Make the priv mutex protect the binding->list and binding->dev.
> For that to work the binding needs to also store a pointer to its
> owning socket?
> 
> Then in both uninstall paths (from socket and from netdev unreg) we can
> take the socket mutex, delete from list, clear the ->dev pointer,
> unlock, release the ref on the binding.
> 
> The socket close path would probably need to lock the socket, look at 
> the first entry, if entry has ->dev call netdev_hold(), release the
> socket, lock the netdev, lock the socket again, look at the ->dev, if
> NULL we raced - done. If not NULL release the socket, call unbind.
> netdev_put(). Restart this paragraph.
> 
> I can't think of an easier way.

An alternative might be to have a new extra lock to just protect
the binding->bound_rxq? And we can move the netdev_lock/unlock inside
the xa_for_each loop in net_devmem_unbind_dmabuf. This will make sure
we don't touch the outdated 'dev'. But I think you're right, the same
lock ordering issue is gonna happen in this case as well.

> > Replacing rtnl with dev lock exposes the fact that we can't assume
> > that the binding->dev is still valid by the time we do unbind.
> 
> Note that binding->dev is never accessed by net_devmem_unbind_dmabuf().
> So if the device was unregistered and its queues flushed, the only thing
> we touch the netdev pointer for is the instance lock :(

I was assuming that bound_rxq is also protected by the instance lock.
But as you were saying earlier, xa has its own lock, so I might
be wrong with that assumption..