Re: [PATCH 1/2] nvmet-rdma: avoid circular locking dependency on install_queue()

[Hannes Reinecke <hare@suse.de>]

On 11/1/23 17:21, Jens Axboe wrote:
> On 11/1/23 4:32 AM, Hannes Reinecke wrote:
>> nvmet_rdma_install_queue() is driven from the ->io_work workqueue
>> function, but will call flush_workqueue() which might trigger
>> ->release_work() which in itself calls flush_work on ->io_work.
>>
>> To avoid that check for pending queue in disconnecting status,
>> and return 'controller busy' until all disconnects are completed.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.de>
>> ---
>>   drivers/nvme/target/rdma.c | 14 ++++++++++++--
>>   1 file changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
>> index 4597bca43a6d..eaeb94a9e863 100644
>> --- a/drivers/nvme/target/rdma.c
>> +++ b/drivers/nvme/target/rdma.c
>> @@ -1583,8 +1583,18 @@ static int nvmet_rdma_queue_connect(struct rdma_cm_id *cm_id,
>>   	}
>>   
>>   	if (queue->host_qid == 0) {
>> -		/* Let inflight controller teardown complete */
>> -		flush_workqueue(nvmet_wq);
>> +		struct nvmet_rdma_queue *q;
>> +		int pending = 0;
>> +
>> +		mutex_lock(&nvmet_rdma_queue_mutex);
>> +		list_for_each_entry(q, &nvmet_rdma_queue_list, queue_list) {
>> +			if (q->state == NVMET_RDMA_Q_DISCONNECTING)
>> +				pending++;
>> +		}
>> +		mutex_unlock(&nvmet_rdma_queue_mutex);
>> +		/* Retry for pending controller teardown */
>> +		if (pending)
>> +			return NVME_SC_CONNECT_CTRL_BUSY;
> 
> Not sure if it's worth turning this into a helper since both patches do
> the same thing. Probably not, since you'd need to pass in the mutex and
> state too. In any case, why not just break if you hit
> NVMET_RDMA_Q_DISCONNECTING rather than keep looping? You don't care
> about the exact count, jsut whether it's non-zero or not.
> 
True, that's easier. Will be updating the patches.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman