Re: [bug report] blktests srp/002 hang

[Zhu Yanjun <yanjun.zhu@linux.dev>]

在 2023/9/21 22:39, Bob Pearson 写道:
> On 9/21/23 09:23, Rain River wrote:
>> On Thu, Sep 21, 2023 at 2:53 AM Bob Pearson <rpearsonhpe@gmail.com> wrote:
>>> On 9/20/23 12:22, Bart Van Assche wrote:
>>>> On 9/20/23 10:18, Bob Pearson wrote:
>>>>> But I have also seen the same behavior in the siw driver which is
>>>>> completely independent.
>>>> Hmm ... I haven't seen any hangs yet with the siw driver.
>>> I was on Ubuntu 6-9 months ago. Currently I don't see hangs on either.
>>>>> As mentioned above at the moment Ubuntu is failing rarely. But it used to fail reliably (srp/002 about 75% of the time and srp/011 about 99% of the time.) There haven't been any changes to rxe to explain this.
>>>> I think that Zhu mentioned commit 9b4b7c1f9f54 ("RDMA/rxe: Add workqueue
>>>> support for rxe tasks")?
>>> That change happened well before the failures went away. I was seeing failures at the same rate with tasklets
>>> and wqs. But after updating Ubuntu and the kernel at some point they all went away.
>> I made tests on the latest Ubuntu with the latest kernel without the
>> commit 9b4b7c1f9f54 ("RDMA/rxe: Add workqueue support for rxe tasks").
>> The latest kernel is v6.6-rc2, the commit 9b4b7c1f9f54 ("RDMA/rxe: Add
>> workqueue support for rxe tasks") is reverted.
>> I made blktest tests for about 30 times, this problem does not occur.
>>
>> So I confirm that without this commit, this hang problem does not
>> occur on Ubuntu without the commit 9b4b7c1f9f54 ("RDMA/rxe: Add
>> workqueue support for rxe tasks").
>>
>> Nanthan
>>
>>>> Thanks,
>>>>
>>>> Bart.
>>>
> This commit is very important for several reasons. It is needed for the ODP implementation
> that is in the works from Daisuke Matsuda and also for QP scaling of performance. The work
> queue implementation scales well with increasing qp number while the tasklet implementation
> does not. This is critical for the drivers use in large scale storage applications. So, if
> there is a bug in the work queue implementation it needs to be fixed not reverted.
>
> I am still hoping that someone will diagnose what is causing the ULPs to hang in terms of
> something missing causing it to wait.

Hi, Bob


You submitted this commit 9b4b7c1f9f54 ("RDMA/rxe: Add workqueue support 
for rxe tasks").

You should be very familiar with this commit.

And this commit causes regression.

So you should delved into the source code to find the root cause, then 
fix it.


Jason && Leon, please comment on this.


Best Regards,

Zhu Yanjun

>
> Bob