Re: [PATCH 2/4] dma-buf: add dma_resv_get_singleton_rcu (v4)

[Daniel Vetter <daniel@ffwll.ch>]

On Mon, May 24, 2021 at 03:04:35PM -0500, Jason Ekstrand wrote:
> On Fri, May 21, 2021 at 12:48 PM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Thu, May 20, 2021 at 02:00:05PM -0500, Jason Ekstrand wrote:
> > > Add a helper function to get a single fence representing
> > > all fences in a dma_resv object.
> > >
> > > This fence is either the only one in the object or all not
> > > signaled fences of the object in a flatted out dma_fence_array.
> > >
> > > v2 (Jason Ekstrand):
> > >  - Take reference of fences both for creating the dma_fence_array and in
> > >    the case where we return one fence.
> > >  - Handle the case where dma_resv_get_list() returns NULL
> > >
> > > v3 (Jason Ekstrand):
> > >  - Add an _rcu suffix because it is read-only
> > >  - Rewrite to use dma_resv_get_fences_rcu so it's RCU-safe
> > >  - Add an EXPORT_SYMBOL_GPL declaration
> > >  - Re-author the patch to Jason since very little is left of Christian
> > >    König's original patch
> > >  - Remove the extra fence argument
> > >
> > > v4 (Jason Ekstrand):
> > >  - Restore the extra fence argument
> > >
> > > Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
> > >
> > > get_singleton
> >
> > Spurious thing here.
> 
> Fixed.
> 
> > > ---
> > >  drivers/dma-buf/dma-resv.c | 122 +++++++++++++++++++++++++++++++++++++
> > >  include/linux/dma-resv.h   |   3 +
> > >  2 files changed, 125 insertions(+)
> > >
> > > diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> > > index 6ddbeb5dfbf65..25995fc15c370 100644
> > > --- a/drivers/dma-buf/dma-resv.c
> > > +++ b/drivers/dma-buf/dma-resv.c
> > > @@ -33,6 +33,8 @@
> > >   */
> > >
> > >  #include <linux/dma-resv.h>
> > > +#include <linux/dma-fence-chain.h>
> > > +#include <linux/dma-fence-array.h>
> > >  #include <linux/export.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/sched/mm.h>
> > > @@ -49,6 +51,19 @@
> > >   * write-side updates.
> > >   */
> > >
> > > +/**
> > > + * dma_fence_deep_dive_for_each - deep dive into the fence containers
> > > + * @fence: resulting fence
> > > + * @chain: variable for a dma_fence_chain
> > > + * @index: index into a dma_fence_array
> > > + * @head: starting point
> > > + *
> > > + * Helper to deep dive into the fence containers for flattening them.
> > > + */
> > > +#define dma_fence_deep_dive_for_each(fence, chain, index, head)      \
> > > +     dma_fence_chain_for_each(chain, head)                   \
> > > +             dma_fence_array_for_each(fence, index, chain)
> >
> > Since this is is just internal helper in the .c file we generally don't
> > document it. Maybe small comment if you feel it's worth it.
> 
> Sure, I can write LESS documentation. :-P
> 
> > > +
> > >  DEFINE_WD_CLASS(reservation_ww_class);
> > >  EXPORT_SYMBOL(reservation_ww_class);
> > >
> > > @@ -517,6 +532,113 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
> > >  }
> > >  EXPORT_SYMBOL_GPL(dma_resv_get_fences_rcu);
> > >
> > > +/**
> > > + * dma_resv_get_singleton - get a single fence for the dma_resv object
> >
> > Name doesn't match here.
> 
> Fixed.
> 
> > > + * @obj: the reservation object
> > > + * @extra: extra fence to add to the resulting array
> > > + * @result: resulting dma_fence
> > > + *
> > > + * Get a single fence representing all unsignaled fences in the dma_resv object
> > > + * plus the given extra fence. If we got only one fence return a new
> > > + * reference to that, otherwise return a dma_fence_array object.
> > > + *
> > > + * RETURNS
> > > + * Returns -NOMEM if allocations fail, zero otherwise.
> >
> > Kernel often encodes this in ERR_PTR so that you don't have to pass a
> > pointer to store the result. Would feel more kerenl-y I think that way. So
> > no result parameter, and on alloc failure you'd return
> 
> Done.
> 
> >         return ERR_PTR(-ENOMEM);
> >
> > > + */
> > > +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,
> >
> > tbh the _rcu here is confusing. I think _unlocked is the better suffix,
> > maybe we should rename dma_resv_get_fences_rcu too for consistency. The
> > rcu-ness of the lookup isn't leaked to callers at all, so no point giving
> > them a panic.
> 
> I can make that change.  I'll also include a patch in the next re-send
> that renames all the _rcu helpers to _unlocked for consistency.

Maybe double-check with Christian König whether he's on board with this
bikeshed, but the current _rcu postfix we have in some dma_resv functions
really confuses me.
-Daniel

> 
> --Jason
> 
> > > +                            struct dma_fence **result)
> > > +{
> > > +     struct dma_fence **resv_fences, *fence, *chain, **fences;
> > > +     struct dma_fence_array *array;
> > > +     unsigned int num_resv_fences, num_fences;
> > > +     unsigned int ret, i, j;
> > > +
> > > +     ret = dma_resv_get_fences_rcu(obj, NULL, &num_resv_fences, &resv_fences);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     num_fences = 0;
> > > +     *result = NULL;
> > > +
> > > +     if (num_resv_fences == 0 && !extra)
> > > +             return 0;
> > > +
> > > +     for (i = 0; i < num_resv_fences; ++i) {
> > > +             dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
> > > +                     if (dma_fence_is_signaled(fence))
> > > +                             continue;
> > > +
> > > +                     *result = fence;
> > > +                     ++num_fences;
> > > +             }
> > > +     }
> > > +
> > > +     if (extra) {
> > > +             dma_fence_deep_dive_for_each(fence, chain, j, extra) {
> > > +                     if (dma_fence_is_signaled(fence))
> > > +                             continue;
> > > +
> > > +                     *result = fence;
> > > +                     ++num_fences;
> > > +             }
> > > +     }
> > > +
> > > +     if (num_fences <= 1) {
> > > +             *result = dma_fence_get(*result);
> > > +             goto put_resv_fences;
> > > +     }
> > > +
> > > +     fences = kmalloc_array(num_fences, sizeof(struct dma_fence*),
> > > +                            GFP_KERNEL);
> > > +     if (!fences) {
> > > +             *result = NULL;
> > > +             ret = -ENOMEM;
> > > +             goto put_resv_fences;
> > > +     }
> > > +
> > > +     num_fences = 0;
> > > +     for (i = 0; i < num_resv_fences; ++i) {
> > > +             dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
> > > +                     if (!dma_fence_is_signaled(fence))
> > > +                             fences[num_fences++] = dma_fence_get(fence);
> > > +             }
> > > +     }
> > > +
> > > +     if (extra) {
> > > +             dma_fence_deep_dive_for_each(fence, chain, j, extra) {
> > > +                     if (dma_fence_is_signaled(fence))
> > > +                             fences[num_fences++] = dma_fence_get(fence);
> > > +             }
> > > +     }
> > > +
> > > +     if (num_fences <= 1) {
> > > +             *result = num_fences ? fences[0] : NULL;
> > > +             kfree(fences);
> > > +             goto put_resv_fences;
> > > +     }
> > > +
> > > +     array = dma_fence_array_create(num_fences, fences,
> > > +                                    dma_fence_context_alloc(1),
> > > +                                    1, false);
> > > +     if (array) {
> > > +             *result = &array->base;
> > > +     } else {
> > > +             *result = NULL;
> > > +             while (num_fences--)
> > > +                     dma_fence_put(fences[num_fences]);
> > > +             kfree(fences);
> > > +             ret = -ENOMEM;
> > > +     }
> > > +
> > > +put_resv_fences:
> > > +     while (num_resv_fences--)
> > > +             dma_fence_put(resv_fences[num_resv_fences]);
> > > +     kfree(resv_fences);
> > > +
> > > +     return ret;
> > > +}
> > > +EXPORT_SYMBOL_GPL(dma_resv_get_singleton_rcu);
> >
> > With the nits addressed:
> >
> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> >
> > > +
> > >  /**
> > >   * dma_resv_wait_timeout_rcu - Wait on reservation's objects
> > >   * shared and/or exclusive fences.
> > > diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
> > > index d44a77e8a7e34..d49ca263e78b4 100644
> > > --- a/include/linux/dma-resv.h
> > > +++ b/include/linux/dma-resv.h
> > > @@ -285,6 +285,9 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
> > >
> > >  int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src);
> > >
> > > +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,
> > > +                            struct dma_fence **result);
> > > +
> > >  long dma_resv_wait_timeout_rcu(struct dma_resv *obj, bool wait_all, bool intr,
> > >                              unsigned long timeout);
> > >
> > > --
> > > 2.31.1
> > >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch