Re: [PATCH for-next 09/10] IB/mlx4: Add timestamp_mask and hca_core_clock to query_device

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]

On Thu, 2015-05-28 at 12:14 -0500, Christoph Lameter wrote:
> On Thu, 28 May 2015, Jason Gunthorpe wrote:
> 
> > After a quick look through, the biggest question in my mind is what
> > should the timestamp value in the wc be?
> >
> > Right now it is some coded thing in clock cycles.
> 
> This is sufficient since it can be converted to ns or whatever one wants.

It is sufficient for your use.  It is not, however, a good API.

> > Should we require the driver to convert to ns before passing the wc
> > back to the app? (Looks like the socket implementation uniformly uses
> > us or ns)
> 
> But that requires additional processing.

Yes.

> > Should the app open code the conversion from clock cycles to ns, or
> > vfunc down to the driver?
> 
> The API provides the abilty to retrieve the clock freq which is
> sufficient for the user to convert the value to meaningful time values.

I would prefer if the access to the timestamp were implemented via a
function in libibverbs (I haven't looked at the git repo, too little
time, I'll get to it).  Something like ibv_get_cqe_timestamp().  That
function should be general and return a suitable, normalized value (ns
probably).  If you just want a simple comparator without the overhead of
normalizing to time, and are willing to accept the consequences of that,
then I would expect you to use something like
ibv_get_raw_cqe_timestamp() to get the unadulterated cycle counter.  For
the most part, the user space application should not know details like
"we are using a cycle counter in the HCA processor for timestamping",
that's below the level of abstraction we attempt to maintain at the
verbs level.  Libmlx4 should be the only thing aware of that fact, and
it talks to the mlx4 driver in the kernel to get the details it needs.
And by putting it into a function that libmlx4 implements, if libcxgb4
decides to implement timestamps and does it in a different way, the app
doesn't care, it just uses the same call.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]