Re: [PATCH for-next 0/4] RDMA/hns: Introduce delay-destruction mechanism

[Junxian Huang <huangjunxian6@hisilicon.com>]

On 2025/2/20 22:13, Jason Gunthorpe wrote:
> On Thu, Feb 20, 2025 at 07:05:06PM +0800, Junxian Huang wrote:
> 
>> Mailbox carries information of the specific resource (QP/CQ/SRQ/MR)
>> that are being destroyed. It's impossible for FW to predict which
>> QP/CQ/SRQ/MR will be destroyed by driver during reset before the
>> reset starts.
> 
> That doesn't make any sense, the device reset is supposed to clean up
> everything. It doesn't matter what the mailbox was doing, after the
> reset finishes it is no longer necessary because the reset was the
> thing that cleaned it up.

Yes, our current implementation is exactly what you said. FW reset
will disable HW, trigger driver reset and clean up everything.

> 
> You need a way to track the reset completion and cancel all
> outstanding commands with a reset failure so cleanup can
> happen. Combined with disassociate and some other locking you need to
> create a strong fence across the reset where there is no leakage of
> 'before' and 'after' reset objects and kernel state.

This is also what we're trying to do now. Currently we check the
reset status in driver when posting mailbox and fail them during
the reset.

The problem is that there is a time gap between the start of FW
reset and the stop of HW, where driver's mailbox will fail while HW
may still access memory. This gap won't last long but in some extreme
cases it's still possible to cause some errors. We try to address
this with this series by reserving the memory until HW is disabled.

Thanks,
Junxian

> 
> Jason