Re: [PATCH rdma-next 00/10] Enable relaxed ordering for ULPs

Tom Talpey
2021-04-09T17:44:35Z

On 4/9/2021 12:40 PM, Jason Gunthorpe wrote:
> On Fri, Apr 09, 2021 at 10:26:21AM -0400, Tom Talpey wrote:
>
>> My belief is that the biggest risk is from situations where completions
>> are batched, and therefore polling is used to detect them without
>> interrupts (which explicitly).
>
> We don't do this in the kernel.
>
> All kernel ULPs only read data after they observe the CQE. We do not
> have "last data polling" and our interrupt model does not support some
> hacky "interrupt means go and use the data" approach.
>
> ULPs have to be designed this way to use the DMA API properly.

Yep. Totally agree.

My concern was about the data being written as relaxed, and the CQE
racing it. I'll reply in the other fork.

> Fencing a DMA before it is completed by the HW will cause IOMMU
> errors.
>
> Userspace is a different story, but that will remain as-is with
> optional relaxed ordering.
>
> Jason
>