Re: [RFC dlm/next 8/9] dlm: likely read lock path for rsb lookup

Alexander Aring
2024-04-11T13:58:38Z

Hi,

On Wed, Apr 10, 2024 at 9:49 AM Alexander Aring <aahringo@redhat.com> wrote:
>
> As the conversion to rhashtable introduced a hash lock per lockspace
> instead of doing a per bucket hash, this patch will change the hash lock
> handling as holding only a read lock in a very likely hot path in DLM.
>
> This hot path is to lookup rsbs and they are in keep state. Keep state
> means that the RSB_TOSS flag isn't set. If this likely read lock path is
> the case we don't need to hold the ls_rsbtbl_lock in write lock at all.
> If it's on toss state holding the ls_rsbtbl_lock in write lock is
> required. If we see that the rsb is in toss state, we hold the
> ls_rsbtbl_lock in write state and relookup (because the rsb could get
> removed into the transition from read to write state) and check if the
> rsb is still in toss state. However this is an unlikely path as toss
> rsbs are cached rsbs. Having the rsb in keep state is very likely.
>
...
> @@ -858,7 +885,9 @@ static int find_rsb_nodir(struct dlm_ls *ls, const void *name, int len,
> */
>
> kref_get(&r->res_ref);
> - goto out_unlock;
> + read_unlock_bh(&ls->ls_rsbtbl_lock);
> +
> + goto out;
>
>
> do_toss:

we need to start to get the write lock here and then do the relookup
and test if the state has been changed (again do_toss is an unlikely
path as the first lookup that can be done under read lock only).
The reason is that there can be a remove and add again and we check on
some fields in the toss path that can be different then, the whole
do_toss path needs to be under write lock.

It makes more sense that the first lookup is in toss state and drops
the read lock, in do_toss we take the write lock and check if the
lookup and the states are still the same under this condition, if so
we go ahead with the toss lookup path under the write lock.

Same for the other lookup functions that look for the rsb being in
toss state and bring it to the keep state (unlikely case), because it
is mostly on the keep list that can be done under read lock.

- Alex