Re: [RFC 4/7] drm/amdgpu: Allocate doorbell slot for user queue

[Shashank Sharma <shashank.sharma@amd.com>]

On 29/12/2022 18:50, Alex Deucher wrote:
> On Fri, Dec 23, 2022 at 2:37 PM Shashank Sharma <shashank.sharma@amd.com> wrote:
>> This patch allocates a doorbell slot in the bar, for the usermode queue.
>> We are using the unique queue-id to get this slot from MES.
> We should manage the doorbell BAR just like VRAM.  I had a set of
> patches to convert doorbell memory to GEM objects.  The user should be
> able to allocate doorbell memory via the GEM IOCTL just like VRAM or
> GTT.  Then when the user calls the USERQ IOCTL, we can just look up
> the GEM object from the handle and then calculate the doorbell offset
> based on the offset of the GEM object from the start of the BAR.
>
> Alex

Noted,

- Shashank

>> Cc: Alex Deucher <alexander.deucher@amd.com>
>> Cc: Christian Koenig <christian.koenig@amd.com>
>> Signed-off-by: Shashank Sharma <shashank.sharma@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 28 +++++++++++++++++++
>>   1 file changed, 28 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
>> index a91cc304cb9e..b566ce4cb7f0 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
>> @@ -50,6 +50,25 @@ amdgpu_userqueue_remove_index(struct amdgpu_device *adev, struct amdgpu_usermode
>>       ida_simple_remove(&uqg->ida, queue->queue_id);
>>   }
>>
>> +static int
>> +amdgpu_userqueue_get_doorbell(struct amdgpu_device *adev,
>> +                    struct amdgpu_usermode_queue *queue)
>> +{
>> +    int r;
>> +    unsigned int doorbell_index;
>> +
>> +    r = amdgpu_mes_alloc_process_doorbells(adev, &doorbell_index);
>> +       if (r < 0) {
>> +        DRM_ERROR("Failed to allocate doorbell for user queue\n");
>> +        return r;
>> +    }
>> +
>> +    /* We are using qnique queue-id to get doorbell here */
>> +    queue->doorbell_index = amdgpu_mes_get_doorbell_dw_offset_in_bar(adev,
>> +                           doorbell_index, queue->queue_id);
>> +    return 0;
>> +}
>> +
>>   static void
>>   amdgpu_userqueue_setup_mqd(struct amdgpu_device *adev, struct amdgpu_usermode_queue *queue)
>>   {
>> @@ -257,12 +276,21 @@ int amdgpu_userqueue_create(struct amdgpu_device *adev, struct drm_file *filp,
>>           goto free_queue;
>>       }
>>
>> +    r = amdgpu_userqueue_get_doorbell(adev, queue);
>> +    if (r) {
>> +        DRM_ERROR("Failed to create doorbell for queue\n");
>> +        goto free_mqd;
>> +    }
>> +
>>       ctx->userq = queue;
>>       args->out.q_id = queue->queue_id;
>>       args->out.flags = 0;
>>       mutex_unlock(&adev->userq.userq_mutex);
>>       return 0;
>>
>> +free_mqd:
>> +    amdgpu_userqueue_destroy_mqd(queue);
>> +
>>   free_queue:
>>       amdgpu_userqueue_remove_index(adev, queue);
>>       mutex_unlock(&adev->userq.userq_mutex);
>> --
>> 2.34.1
>>