Re: [PATCH 2/4] dma-buf: add dma_resv_get_singleton_rcu (v4)

["Christian König" <christian.koenig@amd.com>]

Am 25.05.21 um 14:38 schrieb Daniel Vetter:
> On Mon, May 24, 2021 at 03:04:35PM -0500, Jason Ekstrand wrote:
>> On Fri, May 21, 2021 at 12:48 PM Daniel Vetter <daniel@ffwll.ch> wrote:
>>> On Thu, May 20, 2021 at 02:00:05PM -0500, Jason Ekstrand wrote:
>>>> Add a helper function to get a single fence representing
>>>> all fences in a dma_resv object.
>>>>
>>>> This fence is either the only one in the object or all not
>>>> signaled fences of the object in a flatted out dma_fence_array.
>>>>
>>>> v2 (Jason Ekstrand):
>>>>   - Take reference of fences both for creating the dma_fence_array and in
>>>>     the case where we return one fence.
>>>>   - Handle the case where dma_resv_get_list() returns NULL
>>>>
>>>> v3 (Jason Ekstrand):
>>>>   - Add an _rcu suffix because it is read-only
>>>>   - Rewrite to use dma_resv_get_fences_rcu so it's RCU-safe
>>>>   - Add an EXPORT_SYMBOL_GPL declaration
>>>>   - Re-author the patch to Jason since very little is left of Christian
>>>>     König's original patch
>>>>   - Remove the extra fence argument
>>>>
>>>> v4 (Jason Ekstrand):
>>>>   - Restore the extra fence argument
>>>>
>>>> Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
>>>>
>>>> get_singleton
>>> Spurious thing here.
>> Fixed.
>>
>>>> ---
>>>>   drivers/dma-buf/dma-resv.c | 122 +++++++++++++++++++++++++++++++++++++
>>>>   include/linux/dma-resv.h   |   3 +
>>>>   2 files changed, 125 insertions(+)
>>>>
>>>> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
>>>> index 6ddbeb5dfbf65..25995fc15c370 100644
>>>> --- a/drivers/dma-buf/dma-resv.c
>>>> +++ b/drivers/dma-buf/dma-resv.c
>>>> @@ -33,6 +33,8 @@
>>>>    */
>>>>
>>>>   #include <linux/dma-resv.h>
>>>> +#include <linux/dma-fence-chain.h>
>>>> +#include <linux/dma-fence-array.h>
>>>>   #include <linux/export.h>
>>>>   #include <linux/mm.h>
>>>>   #include <linux/sched/mm.h>
>>>> @@ -49,6 +51,19 @@
>>>>    * write-side updates.
>>>>    */
>>>>
>>>> +/**
>>>> + * dma_fence_deep_dive_for_each - deep dive into the fence containers
>>>> + * @fence: resulting fence
>>>> + * @chain: variable for a dma_fence_chain
>>>> + * @index: index into a dma_fence_array
>>>> + * @head: starting point
>>>> + *
>>>> + * Helper to deep dive into the fence containers for flattening them.
>>>> + */
>>>> +#define dma_fence_deep_dive_for_each(fence, chain, index, head)      \
>>>> +     dma_fence_chain_for_each(chain, head)                   \
>>>> +             dma_fence_array_for_each(fence, index, chain)
>>> Since this is is just internal helper in the .c file we generally don't
>>> document it. Maybe small comment if you feel it's worth it.
>> Sure, I can write LESS documentation. :-P
>>
>>>> +
>>>>   DEFINE_WD_CLASS(reservation_ww_class);
>>>>   EXPORT_SYMBOL(reservation_ww_class);
>>>>
>>>> @@ -517,6 +532,113 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
>>>>   }
>>>>   EXPORT_SYMBOL_GPL(dma_resv_get_fences_rcu);
>>>>
>>>> +/**
>>>> + * dma_resv_get_singleton - get a single fence for the dma_resv object
>>> Name doesn't match here.
>> Fixed.
>>
>>>> + * @obj: the reservation object
>>>> + * @extra: extra fence to add to the resulting array
>>>> + * @result: resulting dma_fence
>>>> + *
>>>> + * Get a single fence representing all unsignaled fences in the dma_resv object
>>>> + * plus the given extra fence. If we got only one fence return a new
>>>> + * reference to that, otherwise return a dma_fence_array object.
>>>> + *
>>>> + * RETURNS
>>>> + * Returns -NOMEM if allocations fail, zero otherwise.
>>> Kernel often encodes this in ERR_PTR so that you don't have to pass a
>>> pointer to store the result. Would feel more kerenl-y I think that way. So
>>> no result parameter, and on alloc failure you'd return
>> Done.
>>
>>>          return ERR_PTR(-ENOMEM);
>>>
>>>> + */
>>>> +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,
>>> tbh the _rcu here is confusing. I think _unlocked is the better suffix,
>>> maybe we should rename dma_resv_get_fences_rcu too for consistency. The
>>> rcu-ness of the lookup isn't leaked to callers at all, so no point giving
>>> them a panic.
>> I can make that change.  I'll also include a patch in the next re-send
>> that renames all the _rcu helpers to _unlocked for consistency.
> Maybe double-check with Christian König whether he's on board with this
> bikeshed, but the current _rcu postfix we have in some dma_resv functions
> really confuses me.

Well I would say yes please :)

The _rcu postfix was something I was always wondering about as well.

Christian.

> -Daniel
>
>> --Jason
>>
>>>> +                            struct dma_fence **result)
>>>> +{
>>>> +     struct dma_fence **resv_fences, *fence, *chain, **fences;
>>>> +     struct dma_fence_array *array;
>>>> +     unsigned int num_resv_fences, num_fences;
>>>> +     unsigned int ret, i, j;
>>>> +
>>>> +     ret = dma_resv_get_fences_rcu(obj, NULL, &num_resv_fences, &resv_fences);
>>>> +     if (ret)
>>>> +             return ret;
>>>> +
>>>> +     num_fences = 0;
>>>> +     *result = NULL;
>>>> +
>>>> +     if (num_resv_fences == 0 && !extra)
>>>> +             return 0;
>>>> +
>>>> +     for (i = 0; i < num_resv_fences; ++i) {
>>>> +             dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
>>>> +                     if (dma_fence_is_signaled(fence))
>>>> +                             continue;
>>>> +
>>>> +                     *result = fence;
>>>> +                     ++num_fences;
>>>> +             }
>>>> +     }
>>>> +
>>>> +     if (extra) {
>>>> +             dma_fence_deep_dive_for_each(fence, chain, j, extra) {
>>>> +                     if (dma_fence_is_signaled(fence))
>>>> +                             continue;
>>>> +
>>>> +                     *result = fence;
>>>> +                     ++num_fences;
>>>> +             }
>>>> +     }
>>>> +
>>>> +     if (num_fences <= 1) {
>>>> +             *result = dma_fence_get(*result);
>>>> +             goto put_resv_fences;
>>>> +     }
>>>> +
>>>> +     fences = kmalloc_array(num_fences, sizeof(struct dma_fence*),
>>>> +                            GFP_KERNEL);
>>>> +     if (!fences) {
>>>> +             *result = NULL;
>>>> +             ret = -ENOMEM;
>>>> +             goto put_resv_fences;
>>>> +     }
>>>> +
>>>> +     num_fences = 0;
>>>> +     for (i = 0; i < num_resv_fences; ++i) {
>>>> +             dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
>>>> +                     if (!dma_fence_is_signaled(fence))
>>>> +                             fences[num_fences++] = dma_fence_get(fence);
>>>> +             }
>>>> +     }
>>>> +
>>>> +     if (extra) {
>>>> +             dma_fence_deep_dive_for_each(fence, chain, j, extra) {
>>>> +                     if (dma_fence_is_signaled(fence))
>>>> +                             fences[num_fences++] = dma_fence_get(fence);
>>>> +             }
>>>> +     }
>>>> +
>>>> +     if (num_fences <= 1) {
>>>> +             *result = num_fences ? fences[0] : NULL;
>>>> +             kfree(fences);
>>>> +             goto put_resv_fences;
>>>> +     }
>>>> +
>>>> +     array = dma_fence_array_create(num_fences, fences,
>>>> +                                    dma_fence_context_alloc(1),
>>>> +                                    1, false);
>>>> +     if (array) {
>>>> +             *result = &array->base;
>>>> +     } else {
>>>> +             *result = NULL;
>>>> +             while (num_fences--)
>>>> +                     dma_fence_put(fences[num_fences]);
>>>> +             kfree(fences);
>>>> +             ret = -ENOMEM;
>>>> +     }
>>>> +
>>>> +put_resv_fences:
>>>> +     while (num_resv_fences--)
>>>> +             dma_fence_put(resv_fences[num_resv_fences]);
>>>> +     kfree(resv_fences);
>>>> +
>>>> +     return ret;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(dma_resv_get_singleton_rcu);
>>> With the nits addressed:
>>>
>>> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>>>
>>>> +
>>>>   /**
>>>>    * dma_resv_wait_timeout_rcu - Wait on reservation's objects
>>>>    * shared and/or exclusive fences.
>>>> diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
>>>> index d44a77e8a7e34..d49ca263e78b4 100644
>>>> --- a/include/linux/dma-resv.h
>>>> +++ b/include/linux/dma-resv.h
>>>> @@ -285,6 +285,9 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
>>>>
>>>>   int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src);
>>>>
>>>> +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,
>>>> +                            struct dma_fence **result);
>>>> +
>>>>   long dma_resv_wait_timeout_rcu(struct dma_resv *obj, bool wait_all, bool intr,
>>>>                               unsigned long timeout);
>>>>
>>>> --
>>>> 2.31.1
>>>>
>>> --
>>> Daniel Vetter
>>> Software Engineer, Intel Corporation
>>> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.ffwll.ch%2F&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C4bfa8c05efd842e47f3b08d91f7a0121%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637575431134167291%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8y%2BHz5GldUZK6Ey%2FuZUQUMg%2BObKticrSVX2gx3NBDOw%3D&amp;reserved=0