Re: [patch net-next RFC 0/2] net: devlink: remove devlink big lock

[Jiri Pirko <jiri@resnulli.us>]

Tue, Jun 28, 2022 at 09:43:26AM CEST, idosch@nvidia.com wrote:
>On Mon, Jun 27, 2022 at 05:55:06PM +0200, Jiri Pirko wrote:
>> Mon, Jun 27, 2022 at 05:41:31PM CEST, idosch@nvidia.com wrote:
>> >On Mon, Jun 27, 2022 at 03:54:59PM +0200, Jiri Pirko wrote:
>> >> From: Jiri Pirko <jiri@nvidia.com>
>> >> 
>> >> This is an attempt to remove use of devlink_mutex. This is a global lock
>> >> taken for every user command. That causes that long operations performed
>> >> on one devlink instance (like flash update) are blocking other
>> >> operations on different instances.
>> >
>> >This patchset is supposed to prevent one devlink instance from blocking
>> >another? Devlink does not enable "parallel_ops", which means that the
>> >generic netlink mutex is serializing all user space operations. AFAICT,
>> >this series does not enable "parallel_ops", so I'm not sure what
>> >difference the removal of the devlink mutex makes.
>> 
>> You are correct, that is missing. For me, as a side effect this patchset
>> resolved the deadlock for LC auxdev you pointed out. That was my
>> motivation for this patchset :)
>
>Given that devlink does not enable "parallel_ops" and that the generic
>netlink mutex is held throughout all callbacks, what prevents you from
>simply dropping the devlink mutex now? IOW, why can't this series be
>patch #1 and another patch that removes the devlink mutex?

Yep, I think you are correct. We are currently working with Moshe on
conversion of commands that does not late devlink->lock (like health
reporters and reload) to take devlink->lock. Once we have that, we can
enable parallel_ops.

>
>> 
>> 
>> >
>> >The devlink mutex (in accordance with the comment above it) serializes
>> >all user space operations and accesses to the devlink devices list. This
>> >resulted in a AA deadlock in the previous submission because we had a
>> >flow where a user space operation (which acquires this mutex) also tries
>> >to register / unregister a nested devlink instance which also tries to
>> >acquire the mutex.
>> >
>> >As long as devlink does not implement "parallel_ops", it seems that the
>> >devlink mutex can be reduced to only serializing accesses to the devlink
>> >devices list, thereby eliminating the deadlock.
>> >
>> >> 
>> >> The first patch makes sure that the xarray that holds devlink pointers
>> >> is possible to be safely iterated.
>> >> 
>> >> The second patch moves the user command mutex to be per-devlink.
>> >> 
>> >> Jiri Pirko (2):
>> >>   net: devlink: make sure that devlink_try_get() works with valid
>> >>     pointer during xarray iteration
>> >>   net: devlink: replace devlink_mutex by per-devlink lock
>> >> 
>> >>  net/core/devlink.c | 256 ++++++++++++++++++++++++++++-----------------
>> >>  1 file changed, 161 insertions(+), 95 deletions(-)
>> >> 
>> >> -- 
>> >> 2.35.3
>> >>