Re: [PATCH v1 net-next 01/11] af_unix: Define locking order for unix_table_double_lock().

[Kuniyuki Iwashima <kuniyu@amazon.com>]

From: Kent Overstreet <kent.overstreet@linux.dev>
Date: Mon, 10 Jun 2024 18:43:44 -0400
> On Mon, Jun 10, 2024 at 03:34:51PM -0700, Kuniyuki Iwashima wrote:
> > When created, AF_UNIX socket is put into net->unx.table.buckets[],
> > and the hash is stored in sk->sk_hash.
> > 
> >   * unbound socket  : 0 <= sk_hash <= UNIX_HASH_MOD
> > 
> > When bind() is called, the socket could be moved to another bucket.
> > 
> >   * pathname socket : 0 <= sk_hash <= UNIX_HASH_MOD
> >   * abstract socket : UNIX_HASH_MOD + 1 <= sk_hash <= UNIX_HASH_MOD * 2 + 1
> > 
> > Then, we call unix_table_double_lock() which locks a single bucket
> > or two.
> > 
> > Let's define the order as unix_table_lock_cmp_fn() instead of using
> > spin_lock_nested().
> > 
> > The locking is always done in ascending order of sk->sk_hash, which
> > is the index of buckets/locks array allocated by kvmalloc_array().
> > 
> >   sk_hash_A < sk_hash_B
> >   <=> &locks[sk_hash_A].dep_map < &locks[sk_hash_B].dep_map
> > 
> > So, the relation of two sk->sk_hash can be derived from the addresses
> > of dep_map in the array of locks.
> > 
> > Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> > ---
> >  net/unix/af_unix.c | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
> > index 3821f8945b1e..b0a9891c0384 100644
> > --- a/net/unix/af_unix.c
> > +++ b/net/unix/af_unix.c
> > @@ -126,6 +126,13 @@ static spinlock_t bsd_socket_locks[UNIX_HASH_SIZE / 2];
> >   *    hash table is protected with spinlock.
> >   *    each socket state is protected by separate spinlock.
> >   */
> > +#ifdef CONFIG_PROVE_LOCKING
> > +static int unix_table_lock_cmp_fn(const struct lockdep_map *a,
> > +				  const struct lockdep_map *b)
> > +{
> > +	return a < b ? -1 : 0;
> > +}
> > +#endif
> 
> This should be a proper comparison function: -1 for less than, 0 for
> equal, 1 for greater than.
> 
> I've got a cmp_int() macro in bcachefs that does this nicely.

So, should this be :

  a < b ? -1 : 1

?

or

  ((a > b) - (b < a))

?

I think most double_lock functions eliminate the a == b case beforehand,
and even ->cmp_fn() is not called for such a recursive case because
debug_spin_lock_before() triggers BUG() then.

Initially I added the same macro, but checkpatch complains about it,
and I thought the current form is easier to understand because it's
the actual comparison used in the double lock part.

Also, there is a case, where we just want to return an error without
classifying it into 0 or 1.

I rather think we should define something like this on the lockdep side.

  enum lockdep_cmp_result {
    LOCKDEP_CMP_SAFE = -1,
    LOCKDEP_CMP_DEADLOCK,
  };

What do you think ?