From mboxrd@z Thu Jan 1 00:00:00 1970 From: Doug Ledford Subject: Re: [PATCH for-next V2 0/9] Add completion timestamping support Date: Tue, 02 Jun 2015 10:35:24 -0400 Message-ID: <1433255724.114391.225.camel@redhat.com> References: <1433074457-26437-1-git-send-email-ogerlitz@mellanox.com> <1433098827.114391.179.camel@redhat.com> <1433157904.114391.188.camel@redhat.com> <20150601164322.GA14391@obsidianresearch.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-z6BFsi67go46P/isQKG4" Return-path: In-Reply-To: <20150601164322.GA14391-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Jason Gunthorpe Cc: Matan Barak , Or Gerlitz , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Amir Vadai , Tal Alon List-Id: linux-rdma@vger.kernel.org --=-z6BFsi67go46P/isQKG4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2015-06-01 at 10:43 -0600, Jason Gunthorpe wrote: > On Mon, Jun 01, 2015 at 07:25:04AM -0400, Doug Ledford wrote: >=20 > > attempted abstraction of ibverbs. Passing in the wc struct allows the > > driver to internally allocate a wc struct with extra private elements > > and pass that back to the user, when the user passes it back to > > ibv_get_timestamp the elements are there in the private portion of the > > struct. >=20 > wc structures are allocated by the caller, there is no option for the > driver to create private elements. Well, they *are* using an extended work completion structure. Unlike what I mentioned, where they create a larger one themselves, you have to allocate a struct ibv_wc_ex instead of a struct ibv_wc and then you have to call poll_cq_ex, which expects a struct ibv_wc_ex. So, just so everyone is clear on this point: the current user space implementation of this feature creates an unversioned, newly named ibv_wc_ex struct that is ibv_wc with a 64bit timestamp tacked on at the end (not 64bit aligned either). If we ever wanted to have a different extension to our ibv_wc struct, there is no good way to do that. If, at some point, we had multiple extension and the user was able to select which they wanted to utilize, this structure extension is not flexible enough to deal with that. At a minimum, if we are going to have a one shot extension to the wc struct like this, I would prefer to see it called struct ibv_wc_timestamp and there be a ibv_poll_cq_timestamp. At least that way people would not use the generic _ex and assume this is the one and only _ex that we will ever need for work completions. Jason, when the XRC and flow steering extensions were added to libibverbs, you complained loudly that they were not added in the agreed upon format and cited a previous on list discussion. Do you have a link to that discussion? > AFAIK, Christoph's use case is essentially the only meaningful use > case for this feature, generalizing too much may destroy the > performance that is valuable here. There is actually room in a 64byte cacheline for two 64bit timestamps and another 2 bytes of padding or something else. --=20 Doug Ledford GPG KeyID: 0E572FDD --=-z6BFsi67go46P/isQKG4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJVbb8sAAoJELgmozMOVy/dGc4P/1TdqoHP7odrW26BTQUPYQBE rJAA/4SeCCAl/PEIEbQdo64maueCIGMeHhtL4E/lnBtUL5qK3NpCcCLr+u/Lvud1 C4MJT7aA2Ups7NKtCZgA5bqExnO3fW9XjXYf8qK2S8ZP3lS3s1EjiejMGX9umpNz AAlkWkCJ98oL1Qk+6dI7hBTrbAjeQ7rd8OPomOXzGtwqvme9DhDEIE1tOas8qAPz dL2zHi2vtjakFcoXp/rdF03voAEkplh9Crhs2n1vXkJd1zbGatvLruB5SZKYhBsR qF1S+VhSKkhpE+R2qabJo1YwbCCWRykkwIqsKpks2ruii+fB3qX2ZwzAvOo0mecH 5vVaa+HGhIH2WYxvjW/MjQM7iquLz4RumLnB2uaVwQnr3hwaOX8XMBu3Wnm0Rmwi /K0i/0ENk/JoZG4NTENYG9UL4JS9QiSguWhzVY/KjHKuvtD64GGmskDGEcOr+8As 9zqKGSBKuU8d13Yi9Tn4BEmM+KE5CqgR8sTPwQnyIxBeWS4z+5NgQU08IlpVThV7 OjXtKbME0cJQ1a6osjpOHydSWxWfqodAgvAoabB+h1IPGiDSZ1OCwu/dvIalgZmp kcC/C3lDHJrFAQOj5dHoBe/DVbYjyHqY/xXqkv/qXkBgvIkqS7VJ22wbjcbEeX5d S2gC6TjVGWjHGCdfBrfp =JFI4 -----END PGP SIGNATURE----- --=-z6BFsi67go46P/isQKG4-- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html