From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Brean Subject: Re: strong ordering for data registered memory Date: Wed, 11 Nov 2009 17:11:03 -0500 Message-ID: <4AFB3677.6050603@Sun.COM> References: <4AF9CACE.8070700@Sun.COM> <4AFAFF7A.4090602@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; CHARSET=US-ASCII; format=flowed Content-Transfer-Encoding: 7BIT Return-path: In-reply-to: <4AFAFF7A.4090602-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Richard Frank Cc: Roland Dreier , linux-rdma List-Id: linux-rdma@vger.kernel.org Yes, there are HCAs that provide strong ordering. And an application such as OpenMPI checks the HCA model and if appropriate enables a mechanism called "eager RDMA" that depends on it. -David Richard Frank wrote: > Today apps are forced to assume that all transports can not provide > strong ordering.. > and hence must implement solutions to work around this. > > There are specific optimizations an app might make if it knows the > underpinning > transport can make these guarantees.. > > It would be useful if "strong ordering" were exposed as attribute from > a transport.. > and as well - have the ability to provide a hint to enable "strong > ordering" on either > registration or per operation or at the qp level . > > Are there any HCAs that provide "strong ordering" today ? > > Roland Dreier wrote: >> > Some time ago there was an email sent to this group with the subject >> > "weak ordering for data registered memory". I don't recall any >> action >> > resulting from this thread. So, I have a question. If a bit were >> > defined to specify "strong ordering", perhaps as a "access" flag (see >> > ibv_access_flags) and used with ibv_reg_mr(), would that be >> sufficient >> > for (1) client applications that need a HW "guarantee" of writing the >> > last byte of an RDMA last and (2) platform implementations that need >> > to deliver that feature? >> >> What would happen if an application asked for strong ordering and the >> adapter and/or platform is not capable of that? >> >> Weak ordering is a bit easier to handle -- the app is saying "if you can >> make things go faster, don't worry about ordering here" and a platform >> where it doesn't matter can just ignore it. >> >> - R. >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html