Re: [patch net-next RFC 0/2] net: devlink: remove devlink big lock

[Jiri Pirko <jiri@resnulli.us>]

Tue, Jun 28, 2022 at 08:32:49AM CEST, jiri@resnulli.us wrote:
>Mon, Jun 27, 2022 at 07:49:45PM CEST, kuba@kernel.org wrote:
>>On Mon, 27 Jun 2022 18:41:31 +0300 Ido Schimmel wrote:
>>> On Mon, Jun 27, 2022 at 03:54:59PM +0200, Jiri Pirko wrote:
>>> > This is an attempt to remove use of devlink_mutex. This is a global lock
>>> > taken for every user command. That causes that long operations performed
>>> > on one devlink instance (like flash update) are blocking other
>>> > operations on different instances.  
>>> 
>>> This patchset is supposed to prevent one devlink instance from blocking
>>> another? Devlink does not enable "parallel_ops", which means that the
>>> generic netlink mutex is serializing all user space operations. AFAICT,
>>> this series does not enable "parallel_ops", so I'm not sure what
>>> difference the removal of the devlink mutex makes.
>>> 
>>> The devlink mutex (in accordance with the comment above it) serializes
>>> all user space operations and accesses to the devlink devices list. This
>>> resulted in a AA deadlock in the previous submission because we had a
>>> flow where a user space operation (which acquires this mutex) also tries
>>> to register / unregister a nested devlink instance which also tries to
>>> acquire the mutex.
>>> 
>>> As long as devlink does not implement "parallel_ops", it seems that the
>>> devlink mutex can be reduced to only serializing accesses to the devlink
>>> devices list, thereby eliminating the deadlock.
>>
>>I'm unclear on why we can't wait for mlx5 locking rework which will
>
>Sure we can, no rush.
>
>>allow us to move completely to per-instance locks. Do you have extra
>>insights into how that work is progressing? I was hoping that it will
>
>It's under internal review afaik.
>
>>be complete in the next two months. 
>
>What do you mean exactly? Is that that we would be okay just with
>devlink->lock? I don't think so. We need user lock because we can't take
>devlink->lock for port split and reload. devlink_mutex protects that now,

Okay, I take back port split, that is already fixed.
Moshe is taking care of the reset (port_new/del, reporter_*). I will
check out the reload. One we have that, you are correct, we are fine
with devlink->lock instance lock.

Thanks!


>the devlink->cmd_lock I'm introducing here just replaces devlink_mutex.
>If we can do without, that is fine. I just can't see how.
>Also, I don't see the relation to mlx5 work. What is that?