RE: [RFC] XRC upstream merge reboot

Hefty, Sean
2011-08-01T15:03:22Z

> Actually I think it is really not so good idea manage reference counter
> across OOB communication.

But this is exactly what the current API *requires* that users of XRC do!!! And I agree, it's not a good idea. :)

> IMHO, I don't see a good reason to redefine existing API.
> I afraid, that such API change will encourage MPI developers to abandon XRC
> support.

I'm suggesting that most apps don't need to do anything special. No OOB connections, no special message exchanges carrying XRC numbers, no reference counting, no new verbs for xrc recv qps that do the same thing as the existing verbs. Using XRC becomes easier! The lifetime of the XRC receive QP is simply tied to the lifetime of the XRC domain, unless the creating process wants to destroy it sooner.

The only reason reference counting was added at all was to support a questionable usage model of sharing an xrc domain across jobs. I'm suggesting that *only those apps* that want that usage model can implement it over the provided APIs. They can continue to use OOB connections, pass XRC numbers, replace ibv_reg_recv_xrv() with atomic_inc(), and let the persistent server destroy the xrc recv qp when those processes are done. For everyone else (and it sounds like this is really everyone at this point), there's simply no need for any of that.

- Sean