Re: [PATCH for-next V2 0/9] Add completion timestamping support

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3012 bytes --]

On Tue, 2015-06-02 at 12:08 -0600, Jason Gunthorpe wrote:
> On Tue, Jun 02, 2015 at 10:35:24AM -0400, Doug Ledford wrote:
> 
> > So, just so everyone is clear on this point: the current user space
> > implementation of this feature creates an unversioned, newly named
> > ibv_wc_ex struct that is ibv_wc with a 64bit timestamp tacked on at the
> > end (not 64bit aligned either).  If we ever wanted to have a different
> > extension to our ibv_wc struct, there is no good way to do that.
> 
> No, if they followed (I didn't check yes) the extension scheme then the
> poll call is
> 
>  struct ibv_wc_ex wcs[num_wcs]
>  ibv_poll_wc_ex(&wcs,num_wcs,sizeof(wcs[0]));
> 
> And the drivers decide what to do based on the 3rd argument, which is
> essentially the ABI version.

Ick.  OK.  I would *much* prefer something done akin to the routines in
packer.c of the kernel, but that's not my call to make, the decision on
the ABI/API extension mechanism was made long ago.  It does, however,
mean that extensions are serial and not modular, and that's a shame.

> > Jason, when the XRC and flow steering extensions were added to
> > libibverbs, you complained loudly that they were not added in the agreed
> > upon format and cited a previous on list discussion.  Do you have a link
> > to that discussion?
> 
> Not off hand, but that was different, that was a misuse of comp_mask,
> IIRC.
> 
> Or, the question in my mind based on looking at the UAPI patches is
> what things should be driver private and what should be general.
> 
> Broadly my thoughts:
>  - Should the frequency and mask be general, or driver private? If the
>    cycles->ns conversion is a function they should be driver private.
>    Even if they are general at libibverbs, they don't *have* to be in
>    the kernel's general query response.
>  - Should frequency even be frequency? Most clocks are expressed
>    accurately as a period in picoseconds. Frequency is more often
>    imprecise. (eg ethernet is 3200 ps or 312.5MHz)
>    However FDR/EDR is fractional for both (4693.33333333 ps vs
>    213.0681818181818 MHz)
>    Precision is very important for time conversions, so a
>    multiply-divide scheme would be ideal.
>    This is suggesting to me these details really are not
>    general.
>  - There should be much better definition on what all this stuff is,
>    units for frequency? When is the timestamp applied?
>  - Should an app even be exposed to mask? This is very difficult
>    to use correctly in the general case. Only cases where an app is
>    restarted more often than a wrap period are trivial to use properly.
> 
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]