Re: [PATCH for-next 09/10] IB/mlx4: Add timestamp_mask and hca_core_clock to query_device

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2239 bytes --]

On Fri, 2015-05-29 at 08:46 -0500, Christoph Lameter wrote:
> On Thu, 28 May 2015, Jason Gunthorpe wrote:
> 
> > My point was exposing raw wrapping counters is a horrible UAPI.
> 
> Well this is a kernel bypass API and a lot of raw hardware issues will
> have to be handled since you do go directly to the device.

No, that's not entirely true, and it *certainly* is not the correct way
to think about verbs extensions.  Is it kernel bypass?  Yes.  Does it go
direct to the hardware?  Not as far as the user application is
concerned.  The direct hardware access is abstracted away in the verbs
library.  Because the verbs library is a hardware abstraction layer, any
extensions to it need to be well thought out.  And by that I mean if it
is of general use, then it should be added in a general, abstract way
that any hardware can implement.  If it is specific to just one vendor's
hardware, then it can be added in a means that is specific to that
vendor's hardware.

Now, as a general rule, I would call timestamps general.  They should be
added in a fashion that anyone can implement.  They should also be well
defined.  Sean's questions raise a very valid point.  Exactly what is
being timestamped, and do we care about different timestamp options?  Is
it completion of message, start of message, transfer from HCA to main
system memory completion, etc.  The 00/10 header to this patch series
was probably answering Sean's question, but just based on the name of
the TIMESTAMP flag to the CQ creation attr struct it isn't clear that
this is the case.

> > > through a complete cycle. But that is the nature of these counters. And
> > > thats what you see when you look into the kernel timer subsystem for
> > > example.
> >
> > Very little in the kernel is exposed to that wrapping, the timer
> > subsystem takes care of it. Certainly, userspace never sees it.
> 
> Right but then we are not at the comfortable sockets API here but at the
> bare metal level.

That's not entirely true.  We still hold to our abstrations, they are
just intentionally kept very thin and high performing.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]