RE: [PATCH for-next 09/10] IB/mlx4: Add timestamp_mask and hca_core_clock to query_device

Christoph Lameter
2015-05-29T13:55:01Z

On Fri, 29 May 2015, Hefty, Sean wrote:

> > Shouldn't the driver software extend smaller counters to 64 bits?
> > That would take a single or and an unlikely branch, so don't say
> > 'performance' :)
>
> It's one thing to time stamp a frame or packet. But this is assigning a ti=
> me stamp to a work completion. I don't even know what that's supposed to m=
> ean when considering 2 GB (or larger) transfers, RDMA read operations, XRC,=
> dynamic connections, out of order retransmissions, shared receive queues, =
> and other exotic features. IMO, this is currently vendor specific function=

What is the issue here? The timestamp is created when the processing is
finished by the nic and the completion entry becomes available.

> ality, and not obviously applicable as a generic feature. It is certainly =
> poorly defined and exposed in a very implementation specific way.

It is exactly defined like any other cycle counters in hardware and it is
exposed using an API that allows multiple vendors to use these cycle
counters without regard to a particular vendors implementation.

I sure wish that Intel would be supporting a feature like this. Please
come up with a better alternative if there is one. This is likely going to
be a differentiator for the vendor used in our industry.

> The use case given by Christoph only speaks of packet level time stamps. O=
> ne could argue that such a use case would place the stamp near the packet (=
> similar to the GRH), rather than embedded into a work completion. This wou=
> ld allow time stamps even in the absence of a work completion.
>
> IMO, vendors already have ways to expose vendor specific features to user s=
> pace. I would mark this as vendor specific and keep it out of the core.

How exactly would that work? How can we attach vendor specific extension
to a completion structure?