UTC timestamps in v4l2 buffers

UTC timestamps in v4l2 buffers

[Philippe De Muyter]

Hello,

I need to have the v4l2 buffers of my camera sensor timestamped with a
precise (1ms) UTC timestamp, in order to be able to match images from cameras
from several computers (that are of course synchronised with NTP, GPS or PTP).

While I had that some years ago and still have in computers running
freescale's 4.1.15 port for imx6q, I have now discovered that 8 years ago
a decision has been taken by the v4l2 maintainers to switch the timestamp
of the v4l2 buffers to CLOCK_MONOTONIC, which is useless when one needs
to synchronise timestamps of images taken by cameras on different computers,
which of course were not booted at the same time.

At that time a new flag "V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC" was introduced
to tell new users that the timestamp was no more the old and not standardized
behaviour for timestamp, but the new CLOCK_MONOTONIC-based timestamp, but
no other flag for UTC or way to choose which kind of timestamp one wants.

Are there since then new standardized or work-in-progess flag to tell users
that the timestamp is UTC, and way to ask the camera-acquisistion driver to
give that UTC timestamp instead of the CLOCK_MONOTONIC one ?

Best regards.

Philippe

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: UTC timestamps in v4l2 buffers

[Nicolas Dufresne]

Le vendredi 08 janvier 2021 à 13:53 +0100, Philippe De Muyter a écrit :
> Hello,
> 
> I need to have the v4l2 buffers of my camera sensor timestamped with a
> precise (1ms) UTC timestamp, in order to be able to match images from cameras
> from several computers (that are of course synchronised with NTP, GPS or PTP).
> 
> While I had that some years ago and still have in computers running
> freescale's 4.1.15 port for imx6q, I have now discovered that 8 years ago
> a decision has been taken by the v4l2 maintainers to switch the timestamp
> of the v4l2 buffers to CLOCK_MONOTONIC, which is useless when one needs
> to synchronise timestamps of images taken by cameras on different computers,
> which of course were not booted at the same time.
> 
> At that time a new flag "V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC" was introduced
> to tell new users that the timestamp was no more the old and not standardized
> behaviour for timestamp, but the new CLOCK_MONOTONIC-based timestamp, but
> no other flag for UTC or way to choose which kind of timestamp one wants.
> 
> Are there since then new standardized or work-in-progess flag to tell users
> that the timestamp is UTC, and way to ask the camera-acquisistion driver to
> give that UTC timestamp instead of the CLOCK_MONOTONIC one ?

Considering the CLOCK_MONOTONIC and CLOCK_REALTIME have the same slope (respond
to adjtime), you can probably just sample both clock in your application in
order to maintain a delta between both clocks. The more often your sample it,
the more precise you can derive. Don't forget to read REALTIME/MONOTONIC and
MONOTONIC/REALTIME orders, so you can averate to the real value. (GStreamer
needs to be improved in this regard).

> 
> Best regards.
> 
> Philippe
> 

Re: UTC timestamps in v4l2 buffers

[Philippe De Muyter]

Hello Nicolas,

On Thu, Jan 28, 2021 at 03:14:03PM -0500, Nicolas Dufresne wrote:
> Le vendredi 08 janvier 2021 à 13:53 +0100, Philippe De Muyter a écrit :
> > Hello,
> > 
> > I need to have the v4l2 buffers of my camera sensor timestamped with a
> > precise (1ms) UTC timestamp, in order to be able to match images from cameras
> > from several computers (that are of course synchronised with NTP, GPS or PTP).
> > 
> > While I had that some years ago and still have in computers running
> > freescale's 4.1.15 port for imx6q, I have now discovered that 8 years ago
> > a decision has been taken by the v4l2 maintainers to switch the timestamp
> > of the v4l2 buffers to CLOCK_MONOTONIC, which is useless when one needs
> > to synchronise timestamps of images taken by cameras on different computers,
> > which of course were not booted at the same time.
> > 
> > At that time a new flag "V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC" was introduced
> > to tell new users that the timestamp was no more the old and not standardized
> > behaviour for timestamp, but the new CLOCK_MONOTONIC-based timestamp, but
> > no other flag for UTC or way to choose which kind of timestamp one wants.
> > 
> > Are there since then new standardized or work-in-progess flag to tell users
> > that the timestamp is UTC, and way to ask the camera-acquisistion driver to
> > give that UTC timestamp instead of the CLOCK_MONOTONIC one ?
> 
> Considering the CLOCK_MONOTONIC and CLOCK_REALTIME have the same slope (respond
> to adjtime), you can probably just sample both clock in your application in
> order to maintain a delta between both clocks. The more often your sample it,
> the more precise you can derive. Don't forget to read REALTIME/MONOTONIC and
> MONOTONIC/REALTIME orders, so you can averate to the real value. (GStreamer
> needs to be improved in this regard).

I am now actually working with nvidia jetson modules, with a 4.9-based
kernel containing added, but not mainstream, nvidia drivers.  Nvidia jetson
modules contain a separate cpu called rtcpu that does only image capture
and timestamps the captured frames with its clock ticking at 32 ns, and
then does exactly what you explain above to convert it to the MONOTONIC clock.
I have modified that code to convert it to the REALTIME clock instead, so
my driver really provides a very precise UTC timestamp with each captured frame.
Unfortunately v4l2src simply discards my UTC timestamp and replaces it by
what seems to be the MONOTONIC clock. It does not matter if I tag my timestamp
with V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC or V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN,
the result stay identical.

This is with gstreamer-1.14.

I still do not understand why v4l2 requires that the timestamp associated to
buffers are based on the MONOTONIC clock, and why v4l2src discards a 
REALTIME clock if v4l2 gives it.  On a well synchronized system, the
REALTIME clock is as monotonic as the MONOTONIC clock, except when a leap
second is sometimes subtracted at the end of June or December, and I can
live with that.  gstreamer may use its own clock for internal purposes,
but why forbid drivers to provide a REALTIME clock based timestamp ?

just my 2 cents

Philippe