[PATCH] fixing hw timestamping in igb

[PATCH] fixing hw timestamping in igb

[Anders Berggren]

Hardware timestamping for Intel 82580 didn't work in either 2.6.36 or 2.6.37. Comparing it to Intel's igb-2.4.12 I found that the timecounter_init clock/counter initialization was done too early. 

Anders Berggren
Halon Security

lab-slang-1:~# diff -u linux-2.6.37/drivers/net/igb/igb_main.c linux/drivers/net/igb/igb_main.c
--- linux-2.6.37/drivers/net/igb/igb_main.c	2011-02-03 10:02:53.000000000 +0100
+++ linux/drivers/net/igb/igb_main.c	2011-02-03 10:12:40.000000000 +0100
@@ -98,6 +98,7 @@
 static void igb_setup_mrqc(struct igb_adapter *);
 static int igb_probe(struct pci_dev *, const struct pci_device_id *);
 static void __devexit igb_remove(struct pci_dev *pdev);
+static void igb_init_hw_timer(struct igb_adapter *adapter);
 static int igb_sw_init(struct igb_adapter *);
 static int igb_open(struct net_device *);
 static int igb_close(struct net_device *);
@@ -1987,6 +1988,10 @@
 	}
 
 #endif
+
+	/* do hw tstamp init after resetting */ 
+	igb_init_hw_timer(adapter);
+
 	dev_info(&pdev->dev, "Intel(R) Gigabit Ethernet Network Connection\n");
 	/* print bus type/speed/width info */
 	dev_info(&pdev->dev, "%s: (PCIe:%s:%s) %pM\n",
@@ -2301,7 +2306,6 @@
 		return -ENOMEM;
 	}
 
-	igb_init_hw_timer(adapter);
 	igb_probe_vfs(adapter);
 
 	/* Explicitly disable IRQ since the NIC can be in any state. */


------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
February 28th, so secure your free ArcSight Logger TODAY! 
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH] fixing hw timestamping in igb

[Jeff Kirsher]

[-- Attachment #1: Type: text/plain, Size: 370 bytes --]

On Thu, 2011-02-03 at 02:39 -0800, Anders Berggren wrote:
> Hardware timestamping for Intel 82580 didn't work in either 2.6.36 or 2.6.37. Comparing it to Intel's igb-2.4.12 I found that the timecounter_init clock/counter initialization was done too early. 
> 
> Anders Berggren
> Halon Security

Thanks Anders, I will add this patch to my queue of igb patches.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [E1000-devel] [PATCH] fixing hw timestamping in igb

[Anders Berggren]

On Feb 4, 2011, at 12:23 AM, Jeff Kirsher wrote:

> Thanks Anders, I will add this patch to my queue of igb patches.

Another question which I asked John Ronciak earlier; is there any work on TX timestamping for IPv6? In 2.6.37 sock_tx_timestamp() is only called in IPv4 UDP (net/ipv4/udp.c) and raw sockets (net/packet/af_packet.c).

Would it be meaningful if I tried to add it to ipv6/ip6_output.c as Marcus D. Leech suggested in http://kerneltrap.org/mailarchive/linux-netdev/2009/11/10/6260604 and http://kerneltrap.org/mailarchive/linux-netdev/2009/11/11/6260643 

Re: [PATCH] fixing hw timestamping in igb

[Ronciak, John]

> -----Original Message-----
> From: Anders Berggren [mailto:anders@halon.se]
> Sent: Tuesday, February 08, 2011 7:25 AM
> To: Kirsher, Jeffrey T
> Cc: Ronciak, John; e1000-devel@lists.sourceforge.net;
> netdev@vger.kernel.org
> Subject: Re: [E1000-devel] [PATCH] fixing hw timestamping in igb
> 
> On Feb 4, 2011, at 12:23 AM, Jeff Kirsher wrote:
> 
> > Thanks Anders, I will add this patch to my queue of igb patches.
> 
> Another question which I asked John Ronciak earlier; is there any work
> on TX timestamping for IPv6? In 2.6.37 sock_tx_timestamp() is only
> called in IPv4 UDP (net/ipv4/udp.c) and raw sockets
> (net/packet/af_packet.c).
> 
> Would it be meaningful if I tried to add it to ipv6/ip6_output.c as
> Marcus D. Leech suggested in http://kerneltrap.org/mailarchive/linux-
> netdev/2009/11/10/6260604 and http://kerneltrap.org/mailarchive/linux-
> netdev/2009/11/11/6260643
Anders,

The question of IPv6 support for TX timestamping is still under discussion.  We are trying to understand the use case for it as well as how it would be used.  We have had no customers asking for this type of support (at least not yet).  If there is kernel work going on regarding it then maybe this needs to be looked at closer.

Other than being a science project, what is your use case for the TX timestamping in IPv6?

Cheers,
John

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired

Re: [E1000-devel] [PATCH] fixing hw timestamping in igb

[Anders Berggren]

On Feb 8, 2011, at 5:27 PM, Ronciak, John wrote:

> The question of IPv6 support for TX timestamping is still under discussion.  We are trying to understand the use case for it as well as how it would be used.  We have had no customers asking for this type of support (at least not yet).  If there is kernel work going on regarding it then maybe this needs to be looked at closer. Other than being a science project, what is your use case for the TX timestamping in IPv6?

We're developing a "hardware/kernel timestamp ping" for Linux in C. It's part of a complete SLA platform (appliance based on Debian with XMLRPC APIs). It's both a science project (Chalmers University, Gothenburg) and a commercial project (Tele2).

Commercial SLA appliances from Cisco and Juniper typically have a accuracy of XXX microseconds. Our hardware ping, using Intel 82580 NICs, have an accuracy of 8 nanoseconds. That is VERY good.

The reason for supporting IPv6 is, well, because it's the future ;) 

Re: [E1000-devel] [PATCH] fixing hw timestamping in igb

[Joe Perches]

On Wed, 2011-02-09 at 09:24 +0100, Anders Berggren wrote:
> Our hardware ping, using Intel 82580 NICs,
> have an accuracy of 8 nanoseconds.

Perhaps you mean 8 nanosecond resolution?
Is documentation available for this claim?

Re: [E1000-devel] [PATCH] fixing hw timestamping in igb

[Anders Berggren]

On Feb 9, 2011, at 9:56 AM, Joe Perches wrote:

> On Wed, 2011-02-09 at 09:24 +0100, Anders Berggren wrote:
>> Our hardware ping, using Intel 82580 NICs,
>> have an accuracy of 8 nanoseconds.
> 
> Perhaps you mean 8 nanosecond resolution?
> Is documentation available for this claim?

Well, both. 8 ns is the accuracy when performing RTT (round-trip time) measurements using our hardware ping tool. We tested it by connecting a 10 m (European meters ;) CAT6 copper and measuring the jitter. The RTT is always 1240 ns or 1248 ns, hence the accuracy of 8 ns.

$ sudo ./probed -c 10.10.10.3 -i eth2 -p 666
SLA-NG probed 0.1
probed: Binding port 666
probed: Using hardware timestamps
probed: client: ::ffff:10.10.10.3: Connecting to port 666
probed: client: ::ffff:10.10.10.3: Connected
Response    1 from 0 in 1240 ns
Response    2 from 0 in 1248 ns
Response    3 from 0 in 1248 ns
Response    4 from 0 in 1248 ns
^C
4 ok, 0 ts err, 0 lost pong, 0 timeout, 0 dup, 0.000000% loss
max: 1248 ns, avg: 1246 ns, min: 1240 ns

This is our tool: 

$ ./probed 
SLA-NG probed 0.1
usage: probed [-saqd] [-c addr] [-t type] [-i iface] [-p port] [-f file]

	          MODES OF OPERATION
	-c addr   Client mode: PING 'addr', fetch UDP timestamps
	-s        Server mode: respond to PING, send UDP timestamps
	-d        Daemon mode: both server and client, output to pipe

	          OPTIONS
	-k        Create timestamps in kernel driver instead of hardware
	-u        Create timestamps in userland instead of hardware
	-i iface  Network interface used for hardware timestamping
	-p port   UDP port, both source and destination
	-w usecs  Client mode wait time between PINGs, in microseconds
	-v        Output more debugging
	-q        Be quiet, log error to syslog only
	-f file   Path to configuration file

Re: [PATCH] fixing hw timestamping in igb

[Anders Berggren]

On Feb 9, 2011, at 9:56 AM, Joe Perches wrote:

> Perhaps you mean 8 nanosecond resolution?
> Is documentation available for this claim?

To clarify; we haven't proven this either mathematically not in a laboratory (looking at actual data transmissions). Also, it's the jitter accuracy we're concerned with. We have simply assumed that the jitter of two independent servers (exchanging timestamps) over a very short cable is an approximation of jitter accuracy.

As you say; if the resolution is 8 ns, the accuracy is of course worse than 8 ns. So far, we have however assumed the accuracy to be within the same order of magnitude.
------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired