Re: [PATCH 2/4] dma-buf: add dma_resv_get_singleton_rcu (v4)

[Daniel Vetter <daniel@ffwll.ch>]

On Thu, May 20, 2021 at 02:00:05PM -0500, Jason Ekstrand wrote:
> Add a helper function to get a single fence representing
> all fences in a dma_resv object.
> 
> This fence is either the only one in the object or all not
> signaled fences of the object in a flatted out dma_fence_array.
> 
> v2 (Jason Ekstrand):
>  - Take reference of fences both for creating the dma_fence_array and in
>    the case where we return one fence.
>  - Handle the case where dma_resv_get_list() returns NULL
> 
> v3 (Jason Ekstrand):
>  - Add an _rcu suffix because it is read-only
>  - Rewrite to use dma_resv_get_fences_rcu so it's RCU-safe
>  - Add an EXPORT_SYMBOL_GPL declaration
>  - Re-author the patch to Jason since very little is left of Christian
>    König's original patch
>  - Remove the extra fence argument
> 
> v4 (Jason Ekstrand):
>  - Restore the extra fence argument
> 
> Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
> 
> get_singleton

Spurious thing here.

> ---
>  drivers/dma-buf/dma-resv.c | 122 +++++++++++++++++++++++++++++++++++++
>  include/linux/dma-resv.h   |   3 +
>  2 files changed, 125 insertions(+)
> 
> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> index 6ddbeb5dfbf65..25995fc15c370 100644
> --- a/drivers/dma-buf/dma-resv.c
> +++ b/drivers/dma-buf/dma-resv.c
> @@ -33,6 +33,8 @@
>   */
>  
>  #include <linux/dma-resv.h>
> +#include <linux/dma-fence-chain.h>
> +#include <linux/dma-fence-array.h>
>  #include <linux/export.h>
>  #include <linux/mm.h>
>  #include <linux/sched/mm.h>
> @@ -49,6 +51,19 @@
>   * write-side updates.
>   */
>  
> +/**
> + * dma_fence_deep_dive_for_each - deep dive into the fence containers
> + * @fence: resulting fence
> + * @chain: variable for a dma_fence_chain
> + * @index: index into a dma_fence_array
> + * @head: starting point
> + *
> + * Helper to deep dive into the fence containers for flattening them.
> + */
> +#define dma_fence_deep_dive_for_each(fence, chain, index, head)	\
> +	dma_fence_chain_for_each(chain, head)			\
> +		dma_fence_array_for_each(fence, index, chain)

Since this is is just internal helper in the .c file we generally don't
document it. Maybe small comment if you feel it's worth it.

> +
>  DEFINE_WD_CLASS(reservation_ww_class);
>  EXPORT_SYMBOL(reservation_ww_class);
>  
> @@ -517,6 +532,113 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
>  }
>  EXPORT_SYMBOL_GPL(dma_resv_get_fences_rcu);
>  
> +/**
> + * dma_resv_get_singleton - get a single fence for the dma_resv object

Name doesn't match here.

> + * @obj: the reservation object
> + * @extra: extra fence to add to the resulting array
> + * @result: resulting dma_fence
> + *
> + * Get a single fence representing all unsignaled fences in the dma_resv object
> + * plus the given extra fence. If we got only one fence return a new
> + * reference to that, otherwise return a dma_fence_array object.
> + *
> + * RETURNS
> + * Returns -NOMEM if allocations fail, zero otherwise.

Kernel often encodes this in ERR_PTR so that you don't have to pass a
pointer to store the result. Would feel more kerenl-y I think that way. So
no result parameter, and on alloc failure you'd return

	return ERR_PTR(-ENOMEM);

> + */
> +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,

tbh the _rcu here is confusing. I think _unlocked is the better suffix,
maybe we should rename dma_resv_get_fences_rcu too for consistency. The
rcu-ness of the lookup isn't leaked to callers at all, so no point giving
them a panic.

> +			       struct dma_fence **result)
> +{
> +	struct dma_fence **resv_fences, *fence, *chain, **fences;
> +	struct dma_fence_array *array;
> +	unsigned int num_resv_fences, num_fences;
> +	unsigned int ret, i, j;
> +
> +	ret = dma_resv_get_fences_rcu(obj, NULL, &num_resv_fences, &resv_fences);
> +	if (ret)
> +		return ret;
> +
> +	num_fences = 0;
> +	*result = NULL;
> +
> +	if (num_resv_fences == 0 && !extra)
> +		return 0;
> +
> +	for (i = 0; i < num_resv_fences; ++i) {
> +		dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
> +			if (dma_fence_is_signaled(fence))
> +				continue;
> +
> +			*result = fence;
> +			++num_fences;
> +		}
> +	}
> +
> +	if (extra) {
> +		dma_fence_deep_dive_for_each(fence, chain, j, extra) {
> +			if (dma_fence_is_signaled(fence))
> +				continue;
> +
> +			*result = fence;
> +			++num_fences;
> +		}
> +	}
> +
> +	if (num_fences <= 1) {
> +		*result = dma_fence_get(*result);
> +		goto put_resv_fences;
> +	}
> +
> +	fences = kmalloc_array(num_fences, sizeof(struct dma_fence*),
> +			       GFP_KERNEL);
> +	if (!fences) {
> +		*result = NULL;
> +		ret = -ENOMEM;
> +		goto put_resv_fences;
> +	}
> +
> +	num_fences = 0;
> +	for (i = 0; i < num_resv_fences; ++i) {
> +		dma_fence_deep_dive_for_each(fence, chain, j, resv_fences[i]) {
> +			if (!dma_fence_is_signaled(fence))
> +				fences[num_fences++] = dma_fence_get(fence);
> +		}
> +	}
> +
> +	if (extra) {
> +		dma_fence_deep_dive_for_each(fence, chain, j, extra) {
> +			if (dma_fence_is_signaled(fence))
> +				fences[num_fences++] = dma_fence_get(fence);
> +		}
> +	}
> +
> +	if (num_fences <= 1) {
> +		*result = num_fences ? fences[0] : NULL;
> +		kfree(fences);
> +		goto put_resv_fences;
> +	}
> +
> +	array = dma_fence_array_create(num_fences, fences,
> +				       dma_fence_context_alloc(1),
> +				       1, false);
> +	if (array) {
> +		*result = &array->base;
> +	} else {
> +		*result = NULL;
> +		while (num_fences--)
> +			dma_fence_put(fences[num_fences]);
> +		kfree(fences);
> +		ret = -ENOMEM;
> +	}
> +
> +put_resv_fences:
> +	while (num_resv_fences--)
> +		dma_fence_put(resv_fences[num_resv_fences]);
> +	kfree(resv_fences);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(dma_resv_get_singleton_rcu);

With the nits addressed:

Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

> +
>  /**
>   * dma_resv_wait_timeout_rcu - Wait on reservation's objects
>   * shared and/or exclusive fences.
> diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
> index d44a77e8a7e34..d49ca263e78b4 100644
> --- a/include/linux/dma-resv.h
> +++ b/include/linux/dma-resv.h
> @@ -285,6 +285,9 @@ int dma_resv_get_fences_rcu(struct dma_resv *obj,
>  
>  int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src);
>  
> +int dma_resv_get_singleton_rcu(struct dma_resv *obj, struct dma_fence *extra,
> +			       struct dma_fence **result);
> +
>  long dma_resv_wait_timeout_rcu(struct dma_resv *obj, bool wait_all, bool intr,
>  			       unsigned long timeout);
>  
> -- 
> 2.31.1
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch