Problem with frame time stamping

Problem with frame time stamping

[Antoine Zen-Ruffinen]

Dear all,

I'm writing a network analyzer software using Linux and I need a VERY
precise frame time stamping. Therefor I am planing to add my own time
stamping algorithm on a modified network driver. For test purpose I
did so :

	skb->tstamp.tv64 = 0x00010002;
	netif_rx(skb);

On the user side, I ask for the timestamp that way :

	...
	sock = socket(AF_PACKET, SOCK_RAW, type);
	...
	//bind this socket with the interface using my modified driver.
	...
	recvByteCount = recv(sock, buffer, 1514, 0);
    	ioctl(sock, SIOCGSTAMP, &timeStamp);

I was surprised to see that the var timeStamp was still holding a
count of second since  year 1970.

Investigating a bit into the kernel code, I found that ioctl(sock,
SIOCGSTAMP, ...) was giving just the current kernel time using
ktime_get_real().

Are my investigation wrong ?
Is that a bug in kernel code ?
Is there an other way to access skb.tstamp from user side ?

Thank's to any one that can help me !

Antoine Zen-Ruffinen

Re: Problem with frame time stamping

[Eric Dumazet]

On Mon, 12 Nov 2007 16:42:34 +0100
"Antoine Zen-Ruffinen" <antoine.zen@gmail.com> wrote:

> Dear all,
> 
> I'm writing a network analyzer software using Linux and I need a VERY
> precise frame time stamping. Therefor I am planing to add my own time
> stamping algorithm on a modified network driver. For test purpose I
> did so :
> 
> 	skb->tstamp.tv64 = 0x00010002;
> 	netif_rx(skb);
> 
> On the user side, I ask for the timestamp that way :
> 
> 	...
> 	sock = socket(AF_PACKET, SOCK_RAW, type);
> 	...
> 	//bind this socket with the interface using my modified driver.
> 	...
> 	recvByteCount = recv(sock, buffer, 1514, 0);
>     	ioctl(sock, SIOCGSTAMP, &timeStamp);
> 
> I was surprised to see that the var timeStamp was still holding a
> count of second since  year 1970.
> 
> Investigating a bit into the kernel code, I found that ioctl(sock,
> SIOCGSTAMP, ...) was giving just the current kernel time using
> ktime_get_real().
> 
> Are my investigation wrong ?
> Is that a bug in kernel code ?
> Is there an other way to access skb.tstamp from user side ?
> 
> Thank's to any one that can help me !

If your kernel is recent enough, you can already get nanosecond precision from skb tstamp, using SIOCGSTAMPNS ioctl or SO_TIMESTAMPNS / SCM_TIMESTAMPNS

(changes where done in March/April this year)

http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=b7aa0bf70c4afb9e38be25f5c0922498d0f8684c

http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=ae40eb1ef30ab4120bd3c8b7e3da99ee53d27a23

http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=92f37fd2ee805aa77925c1e64fd56088b46094fc


No need to modify a network driver :)

Re: Problem with frame time stamping

[Antoine Zen-Ruffinen]

What does it bring me to have a nanosecond precision if it is not
related to the actual arrival of frame time ? As it seem I can feel
skb->tstamp with whatever I want, I always become something else using
ioctl(). (I'm using kernel 2.6.23).

2007/11/12, Eric Dumazet <dada1@cosmosbay.com>:
> On Mon, 12 Nov 2007 16:42:34 +0100
> "Antoine Zen-Ruffinen" <antoine.zen@gmail.com> wrote:
>
> > Dear all,
> >
> > I'm writing a network analyzer software using Linux and I need a VERY
> > precise frame time stamping. Therefor I am planing to add my own time
> > stamping algorithm on a modified network driver. For test purpose I
> > did so :
> >
> >       skb->tstamp.tv64 = 0x00010002;
> >       netif_rx(skb);
> >
> > On the user side, I ask for the timestamp that way :
> >
> >       ...
> >       sock = socket(AF_PACKET, SOCK_RAW, type);
> >       ...
> >       //bind this socket with the interface using my modified driver.
> >       ...
> >       recvByteCount = recv(sock, buffer, 1514, 0);
> >       ioctl(sock, SIOCGSTAMP, &timeStamp);
> >
> > I was surprised to see that the var timeStamp was still holding a
> > count of second since  year 1970.
> >
> > Investigating a bit into the kernel code, I found that ioctl(sock,
> > SIOCGSTAMP, ...) was giving just the current kernel time using
> > ktime_get_real().
> >
> > Are my investigation wrong ?
> > Is that a bug in kernel code ?
> > Is there an other way to access skb.tstamp from user side ?
> >
> > Thank's to any one that can help me !
>
> If your kernel is recent enough, you can already get nanosecond precision from skb tstamp, using SIOCGSTAMPNS ioctl or SO_TIMESTAMPNS / SCM_TIMESTAMPNS
>
> (changes where done in March/April this year)
>
> http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=b7aa0bf70c4afb9e38be25f5c0922498d0f8684c
>
> http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=ae40eb1ef30ab4120bd3c8b7e3da99ee53d27a23
>
> http://git2.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=92f37fd2ee805aa77925c1e64fd56088b46094fc
>
>
> No need to modify a network driver :)
>
>
>

Re: Problem with frame time stamping

[Eric Dumazet]

Antoine Zen-Ruffinen a écrit :
> What does it bring me to have a nanosecond precision if it is not
> related to the actual arrival of frame time ? As it seem I can feel
> skb->tstamp with whatever I want, I always become something else using
> ioctl(). (I'm using kernel 2.6.23).
>
>   
I guess you misunderstood kernel source, because it is related to 
arrival time, more exactly when it was processed by network stack.
(Beware  modern NICS can delay the rx interrupt by some us (ethtool -c 
eth0), so that an interrupt can feed more than one packet to the OS)

Check net/core/dev.c function netif_rx()

{
...
   if (!skb->tstamp.tv64)
       net_timestamp(skb);

}

So as soon your system as at least one socket 'asking for tsamps', 
netstamp_needed is not null, and net_timestamp() will call __net_timestamp()
wich does :

skb->tstamp = ktime_get_real();

So you should not 'feed' tstamp.tv64 and let it being 0, so that 
netif_rx() can do its job.

Re: Problem with frame time stamping

[Antoine Zen-Ruffinen]

This is exactly my problem : The driver of the network card I am using
(see rt2x00.serialmonkey.com) do the minimum in the hardware interrupt
(not filling skb->tstamp). Then netif_rx() is called later using a
tasklet (also not filling skb->tstamp). As it seem to me (maybe I am
wrong, if so please tell), the elapse time between the actual frame
arrival and the time where netif_rx() do net_timestamp(skb) is not
predicable !?

Else, I would like to thank you to spend time helping me.



2007/11/13, Eric Dumazet <dada1@cosmosbay.com>:
> Antoine Zen-Ruffinen a écrit :
> > What does it bring me to have a nanosecond precision if it is not
> > related to the actual arrival of frame time ? As it seem I can feel
> > skb->tstamp with whatever I want, I always become something else using
> > ioctl(). (I'm using kernel 2.6.23).
> >
> >
> I guess you misunderstood kernel source, because it is related to
> arrival time, more exactly when it was processed by network stack.
> (Beware  modern NICS can delay the rx interrupt by some us (ethtool -c
> eth0), so that an interrupt can feed more than one packet to the OS)
>
> Check net/core/dev.c function netif_rx()
>
> {
> ...
>    if (!skb->tstamp.tv64)
>        net_timestamp(skb);
>
> }
>
> So as soon your system as at least one socket 'asking for tsamps',
> netstamp_needed is not null, and net_timestamp() will call __net_timestamp()
> wich does :
>
> skb->tstamp = ktime_get_real();
>
> So you should not 'feed' tstamp.tv64 and let it being 0, so that
> netif_rx() can do its job.
>
>
>
>
>
>
-
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Problem with frame time stamping

[Antoine Zen-Ruffinen]

no worries ont that side, In my code timeStamp is a "struct timeval".

2007/11/13, Antoine Zen-Ruffinen <antoine.zen@gmail.com>:
> This is exactly my problem : The driver of the network card I am using
> (see rt2x00.serialmonkey.com) do the minimum in the hardware interrupt
> (not filling skb->tstamp). Then netif_rx() is called later using a
> tasklet (also not filling skb->tstamp). As it seem to me (maybe I am
> wrong, if so please tell), the elapse time between the actual frame
> arrival and the time where netif_rx() do net_timestamp(skb) is not
> predicable !?
>
> Else, I would like to thank you to spend time helping me.
>
>
>
> 2007/11/13, Eric Dumazet <dada1@cosmosbay.com>:
> > Antoine Zen-Ruffinen a écrit :
> > > What does it bring me to have a nanosecond precision if it is not
> > > related to the actual arrival of frame time ? As it seem I can feel
> > > skb->tstamp with whatever I want, I always become something else using
> > > ioctl(). (I'm using kernel 2.6.23).
> > >
> > >
> > I guess you misunderstood kernel source, because it is related to
> > arrival time, more exactly when it was processed by network stack.
> > (Beware  modern NICS can delay the rx interrupt by some us (ethtool -c
> > eth0), so that an interrupt can feed more than one packet to the OS)
> >
> > Check net/core/dev.c function netif_rx()
> >
> > {
> > ...
> >    if (!skb->tstamp.tv64)
> >        net_timestamp(skb);
> >
> > }
> >
> > So as soon your system as at least one socket 'asking for tsamps',
> > netstamp_needed is not null, and net_timestamp() will call __net_timestamp()
> > wich does :
> >
> > skb->tstamp = ktime_get_real();
> >
> > So you should not 'feed' tstamp.tv64 and let it being 0, so that
> > netif_rx() can do its job.
> >
> >
> >
> >
> >
> >
>
-
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Problem with frame time stamping

[Eric Dumazet]

Antoine Zen-Ruffinen a écrit :
> This is exactly my problem : The driver of the network card I am using
> (see rt2x00.serialmonkey.com) do the minimum in the hardware interrupt
> (not filling skb->tstamp). Then netif_rx() is called later using a
> tasklet (also not filling skb->tstamp). As it seem to me (maybe I am
> wrong, if so please tell), the elapse time between the actual frame
> arrival and the time where netif_rx() do net_timestamp(skb) is not
> predicable !?
>
> Else, I would like to thank you to spend time helping me.
>
>
>   
A tasklet could process the skb much later than corresponding IRQ, 
depending on various things
(other tasks/softirqs on system with higher priorities). So yes, it is 
not predictable at all.

Usually it doesnt matter, but if your business depends on precise tstamps,
then just do skb->tstamp = ktime_get_real(); in IRQ handler (but it will 
slow it a bit,
depending on how fast is ktime_get_real() on the target machine)
netif_rx() wont overwrite it.

skb = dev_alloc_skb(desc.size + NET_IP_ALIGN);
if (!skb)
    return;
skb->tstamp = ktime_get_real(); /* do it before other copies */
skb_reserve(skb, NET_IP_ALIGN);
skb_put(skb, desc.size);
memcpy(skb->data, entry->data_addr, desc.size);

...

Re: Problem with frame time stamping

[Antoine Zen-Ruffinen]

THIS is what I did at the beginning. But is seem me to be some thing
wrong. So I put a static value in skb->tstamp  instead of
ktime_get_real() for debug purpose. And I was still becoming the
amount of second, microsecond since 1970. We are back to my initial
mail ! The value of skb->tsamp doesn't seems to be returned by
ioctl().

2007/11/13, Eric Dumazet <dada1@cosmosbay.com>:
> Antoine Zen-Ruffinen a écrit :
> > This is exactly my problem : The driver of the network card I am using
> > (see rt2x00.serialmonkey.com) do the minimum in the hardware interrupt
> > (not filling skb->tstamp). Then netif_rx() is called later using a
> > tasklet (also not filling skb->tstamp). As it seem to me (maybe I am
> > wrong, if so please tell), the elapse time between the actual frame
> > arrival and the time where netif_rx() do net_timestamp(skb) is not
> > predicable !?
> >
> > Else, I would like to thank you to spend time helping me.
> >
> >
> >
> A tasklet could process the skb much later than corresponding IRQ,
> depending on various things
> (other tasks/softirqs on system with higher priorities). So yes, it is
> not predictable at all.
>
> Usually it doesnt matter, but if your business depends on precise tstamps,
> then just do skb->tstamp = ktime_get_real(); in IRQ handler (but it will
> slow it a bit,
> depending on how fast is ktime_get_real() on the target machine)
> netif_rx() wont overwrite it.
>
> skb = dev_alloc_skb(desc.size + NET_IP_ALIGN);
> if (!skb)
>     return;
> skb->tstamp = ktime_get_real(); /* do it before other copies */
> skb_reserve(skb, NET_IP_ALIGN);
> skb_put(skb, desc.size);
> memcpy(skb->data, entry->data_addr, desc.size);
>
> ...
>
>
>
>
>
>

Re: Problem with frame time stamping

[Eric Dumazet]

Antoine Zen-Ruffinen a écrit :
> THIS is what I did at the beginning. But is seem me to be some thing
> wrong. So I put a static value in skb->tstamp  instead of
> ktime_get_real() for debug purpose. And I was still becoming the
> amount of second, microsecond since 1970. We are back to my initial
> mail ! The value of skb->tsamp doesn't seems to be returned by
> ioctl().
>   
OK Antoine.

Yes, 'struct timeval' is as you discovered, ie relative to *something*, 
a given event in the past. (in your case, CLOCK_REALTIME Epoch : January 
1th 1970)

It is even documented (man gettimeofday)

DESCRIPTION
       The  gettimeofday() function shall obtain the current time, 
expressed as seconds and microseconds since the Epoch, and store it in the
       timeval structure pointed to by tp.

For timespec STAMPNS, see clock_gettime() description.

clock_gettime(CLOCK_REALTIME, &ts);

If in your driver you stick in tstamp.tv64 a value like 0x00010002, then 
the result of ioctl(SIOCGSTAMP) will be :
tv.tv_sec = 0   (so yes, January 1th 1970)
tv.tv_usec = 65   (because 0x10002/1000 = 65)

Not counting the fact that .tv64 should not be directly set, unless for 
0 value, because it is not portable.

Re: Problem with frame time stamping

[Antoine Zen-Ruffinen]

ok, I now understood where my error was.  It was a mismatch between my
old kernel headers and my new kernel that I am running !!!! Pretty
stupid !

Tank's you for your time, you learn me some stuff, I appreciate it.

Antoine

2007/11/13, Eric Dumazet <dada1@cosmosbay.com>:
> Antoine Zen-Ruffinen a écrit :
> > THIS is what I did at the beginning. But is seem me to be some thing
> > wrong. So I put a static value in skb->tstamp  instead of
> > ktime_get_real() for debug purpose. And I was still becoming the
> > amount of second, microsecond since 1970. We are back to my initial
> > mail ! The value of skb->tsamp doesn't seems to be returned by
> > ioctl().
> >
> OK Antoine.
>
> Yes, 'struct timeval' is as you discovered, ie relative to *something*,
> a given event in the past. (in your case, CLOCK_REALTIME Epoch : January
> 1th 1970)
>
> It is even documented (man gettimeofday)
>
> DESCRIPTION
>        The  gettimeofday() function shall obtain the current time,
> expressed as seconds and microseconds since the Epoch, and store it in the
>        timeval structure pointed to by tp.
>
> For timespec STAMPNS, see clock_gettime() description.
>
> clock_gettime(CLOCK_REALTIME, &ts);
>
> If in your driver you stick in tstamp.tv64 a value like 0x00010002, then
> the result of ioctl(SIOCGSTAMP) will be :
> tv.tv_sec = 0   (so yes, January 1th 1970)
> tv.tv_usec = 65   (because 0x10002/1000 = 65)
>
> Not counting the fact that .tv64 should not be directly set, unless for
> 0 value, because it is not portable.
>
>
>
>
>
>

Re: Problem with frame time stamping

[Eric Dumazet]

Antoine Zen-Ruffinen a écrit :
> What does it bring me to have a nanosecond precision if it is not
> related to the actual arrival of frame time ? As it seem I can feel
> skb->tstamp with whatever I want, I always become something else using
> ioctl(). (I'm using kernel 2.6.23).
>
> 2007/11/12, Eric Dumazet <dada1@cosmosbay.com>:
>   
>> On Mon, 12 Nov 2007 16:42:34 +0100
>> "Antoine Zen-Ruffinen" <antoine.zen@gmail.com> wrote:
>>
>>     
>>> Dear all,
>>>
>>> I'm writing a network analyzer software using Linux and I need a VERY
>>> precise frame time stamping. Therefor I am planing to add my own time
>>> stamping algorithm on a modified network driver. For test purpose I
>>> did so :
>>>
>>>       skb->tstamp.tv64 = 0x00010002;
>>>       netif_rx(skb);
>>>
>>> On the user side, I ask for the timestamp that way :
>>>
>>>       ...
>>>       sock = socket(AF_PACKET, SOCK_RAW, type);
>>>       ...
>>>       //bind this socket with the interface using my modified driver.
>>>       ...
>>>       recvByteCount = recv(sock, buffer, 1514, 0);
>>>       ioctl(sock, SIOCGSTAMP, &timeStamp);
>>>
>>> I was surprised to see that the var timeStamp was still holding a
>>> count of second since  year 1970.
>>>       
But then, maybe your problem comes from your code : timeStamp should be 
declared as "struct timeval" of course, to get both tv_sec and tv_usec.

struct timeval tv;
ioctl(sock, SIOCSTAMP, &tv);
printf("packet arrived at %ld.%06ld\n", (long)tv.tv_sec, (long)tv.tv_usec);


If you *want* nanosecond resolution instead of microsecond, use :

struct timespec ts;
ioctl(sock, SIOCSTAMPNS, &ts);
printf("packet arrived at %ld.%09ld\n", (long)ts.tv_sec, (long)ts.tv_nsec);




-
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html