Re: [PATCH WIP 38/43] iser-target: Port to new memory registration API

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Mon, Jul 27, 2015 at 11:57:46AM -0400, Chuck Lever wrote:
> IMO ib_unmap_fmr is a very different animal from LOCAL_INV WR.

Sure, but how many of these properties does NFS actually care about,
now that it is running the API properly?

> ib_unmap_fmr is synchronous, provides no ordering guarantees with
> send queue operations, and does not depend on a connected QP to
> be available. You could emulate asynchronicity with a work queue
> but that still does not provide SQ ordering. There are few if any
> failure modes for ib_unmap_fmr.

I'm having a hard time seeing how SQ ordering is important when the
API is used properly. Once you explicitly order the DMA unmap after
the invalidate completion you no longer need implicit SQ ordering

Is there a way to combine SQ implicit ordering and the Linux DMA API
together correctly?

> flight, the LINV flushes. MRs can be left in a state where the
> MR's rkey is not in sync with the HW, in which case a
> synchronous operation may be required to recover the MR.

The error handling seems like a trivial difference, a
ib_recover_failed_qp_mr(mr); sort of call could resync everything
after a QP blows up..

> The complexion of MR invalidation could be vastly different in
> a few years: handled entirely by the target-side, and only
> verified by the initiator. Verification doesn't need to sleep,
> and the slow path (the target failed to invalidate) can be
> deferred.

The initiator still needs to have the ability to issue the invalidate
if the target doesn't do it, so all the code still exists..

Even ignoring those issues, should we be talking about putting FMR
under the new ib_alloc_mr and ib_map_mr interfaces? Would that help
much even if the post and unmap flows are totally different?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html