way of figuring out total number of retransmitted packets on a TCP socket?

way of figuring out total number of retransmitted packets on a TCP socket?

[Lennert Buytenhek]

Hi,

I'm looking for a way of counting the total number of retransmitted packets
sent on a TCP connection.  I tried querying TCP_INFO:tcpi_retransmits, but
that seems to be not the metric I'm looking for.

Anyone got any ideas?  (Apart from using tcpdump+tcptrace.)


cheers,
Lennert

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Baruch Even]

On Wed, 2004-10-20 at 14:01, Lennert Buytenhek wrote:
> Hi,
> 
> I'm looking for a way of counting the total number of retransmitted packets
> sent on a TCP connection.  I tried querying TCP_INFO:tcpi_retransmits, but
> that seems to be not the metric I'm looking for.
> 
> Anyone got any ideas?  (Apart from using tcpdump+tcptrace.)

The web100 patch gives you that, but it's not part of the basic kernel.
You can find it in web100.org and you also need the user mode utilities
to get the information.

Baruch

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Wed, 20 Oct 2004 15:01:34 +0200
Lennert Buytenhek <buytenh@wantstofly.org> wrote:

> I'm looking for a way of counting the total number of retransmitted packets
> sent on a TCP connection.  I tried querying TCP_INFO:tcpi_retransmits, but
> that seems to be not the metric I'm looking for.
> 
> Anyone got any ideas?  (Apart from using tcpdump+tcptrace.)

tcpi_retransmits only accounts for timeout based retransmits.

Retransmits which occur dynamically due to fast-retransmit
and other non-timeout based decisions are accounted for
in "tcpi_retrans", so that is probably the value you are
looking for.

Unlike another person's response to this thread, you don't
need the huge web100 patch to get at this information :-)

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Wed, 20 Oct 2004 14:25:02 +0100
Baruch Even <baruch@ev-en.org> wrote:

> The web100 patch gives you that, but it's not part of the basic kernel.

You don't need web100 for this simple statistic, see "tcpi_retrans"
We also provide nearly all of the lost packet tracking stats in
the form of "tcpi_unacked", "tcpi_sacked", "tcpi_lost", and
"tcpi_fackets".

In fact, web100 duplicates a lot of functionality provided already
by tcp_diag :-)

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Lennert Buytenhek]

On Wed, Oct 20, 2004 at 03:14:48PM -0700, David S. Miller wrote:

> > I'm looking for a way of counting the total number of retransmitted packets
> > sent on a TCP connection.  I tried querying TCP_INFO:tcpi_retransmits, but
> > that seems to be not the metric I'm looking for.
> > 
> > Anyone got any ideas?  (Apart from using tcpdump+tcptrace.)
> 
> tcpi_retransmits only accounts for timeout based retransmits.

Some places in the TCP stack reset tp->retransmits to zero, or to
one.  Is it supposed to reflect the current number of retransmitted
packets in the network?


> Retransmits which occur dynamically due to fast-retransmit
> and other non-timeout based decisions are accounted for
> in "tcpi_retrans", so that is probably the value you are
> looking for.

tcpi_retrans is taken from tp->retrans_out (looking at 2.6.8 here.)
But that value gets set to zero in a lot of places in the networking
as well, so it's probably not what I'm looking for either.

This is what TCP_INFO looks like after a sample run where 1GB is
pumped into the socket buffer, sleep(1) is done to let the dust settle
and then stats are dumped (2.6.8 SMP sender and 2.6.8 UP receiver,
gigabit ethernet over a cross cable):

tcpi_state              1, ESTABLISHED
tcpi_ca_state           0, CA_Open
tcpi_retransmits        0
tcpi_probes             0
tcpi_backoff            0
tcpi_options            7 [TIMESTAMPS SACK WSCALE ]
tcpi_snd_wscale         7
tcpi_rcv_wscale         7

tcpi_rto                208000
tcpi_ato                0
tcpi_snd_mss            1448
tcpi_rcv_mss            536

tcpi_unacked            0
tcpi_sacked             0
tcpi_lost               0
tcpi_retrans            0
tcpi_fackets            0

tcpi_last_data_sent     999
tcpi_last_ack_sent      0
tcpi_last_data_recv     3040239
tcpi_last_ack_recv      994

tcpi_pmtu               1500
tcpi_rcv_sstresh        5840
tcpi_rtt                8000
tcpi_rttvar             750
tcpi_snd_sstresh        64
tcpi_snd_cwnd           73
tcpi_advmss             1448
tcpi_reordering         3

But I'm sure that packets were retransmitted during this run, because
I had a DROP rule in there for a few seconds:

00:27:09.904020 IP 10.10.10.3.60983 > 10.10.10.4.9999: . 18905849:18907297(1448) ack 1 win 46 <nop,nop,timestamp 449613028 6553106>
00:27:10.327942 IP 10.10.10.3.60983 > 10.10.10.4.9999: . 18905849:18907297(1448) ack 1 win 46 <nop,nop,timestamp 449613452 6553106>
00:27:11.175790 IP 10.10.10.3.60983 > 10.10.10.4.9999: . 18905849:18907297(1448) ack 1 win 46 <nop,nop,timestamp 449614300 6553106>
00:27:12.871486 IP 10.10.10.3.60983 > 10.10.10.4.9999: . 18905849:18907297(1448) ack 1 win 46 <nop,nop,timestamp 449615996 6553106>
00:27:16.262863 IP 10.10.10.3.60983 > 10.10.10.4.9999: . 18905849:18907297(1448) ack 1 win 46 <nop,nop,timestamp 449619388 6553106>


cheers,
Lennert

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Thu, 21 Oct 2004 00:35:47 +0200
Lennert Buytenhek <buytenh@wantstofly.org> wrote:

> Some places in the TCP stack reset tp->retransmits to zero, or to
> one.  Is it supposed to reflect the current number of retransmitted
> packets in the network?

Yes, timeout based retransmits in the network.  It is reset typically
when the connection is healthy again and making forward progress on
the lost packets.

> > Retransmits which occur dynamically due to fast-retransmit
> > and other non-timeout based decisions are accounted for
> > in "tcpi_retrans", so that is probably the value you are
> > looking for.
> 
> tcpi_retrans is taken from tp->retrans_out (looking at 2.6.8 here.)
> But that value gets set to zero in a lot of places in the networking
> as well, so it's probably not what I'm looking for either.

I agree.  We could easily add the thing you are looking for.  It's
just another statistic, zero'd at socket creation time and incremented
right under the line which reads:

		/* Update global TCP statistics. */
		TCP_INC_STATS(TCP_MIB_RETRANSSEGS);

in tcp_retransmit_skb().

I'll add this for you.

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Andi Kleen]

> I agree.  We could easily add the thing you are looking for.  It's
> just another statistic, zero'd at socket creation time and incremented
> right under the line which reads:
> 
> 		/* Update global TCP statistics. */
> 		TCP_INC_STATS(TCP_MIB_RETRANSSEGS);
> 
> in tcp_retransmit_skb().
> 
> I'll add this for you.

Is it really necessary? Our struct sock is already bloated enough 
as is ... If Lennert needs it for some obscure purpose it's probably
better if he just does it locally.

-Andi

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Thu, 21 Oct 2004 01:04:05 +0200
Andi Kleen <ak@suse.de> wrote:

> Is it really necessary? Our struct sock is already bloated enough 
> as is ... If Lennert needs it for some obscure purpose it's probably
> better if he just does it locally.

There are a couple of "__u32" holes before pointers in tcp_opt
which is the space I was going to use for this new item.

tcp_opt will be getting smaller some time soon as well, since
we can consolidate most of the congestion control knobs into
a single set of datums or some kind of union.

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Andi Kleen]

On Wed, Oct 20, 2004 at 04:07:28PM -0700, David S. Miller wrote:
> On Thu, 21 Oct 2004 01:04:05 +0200
> Andi Kleen <ak@suse.de> wrote:
> 
> > Is it really necessary? Our struct sock is already bloated enough 
> > as is ... If Lennert needs it for some obscure purpose it's probably
> > better if he just does it locally.
> 
> There are a couple of "__u32" holes before pointers in tcp_opt
> which is the space I was going to use for this new item.

On 64bit you mean? That doesn't help the 32bit challenged users.

> 
> tcp_opt will be getting smaller some time soon as well, since
> we can consolidate most of the congestion control knobs into
> a single set of datums or some kind of union.

ok. good. I remember the times when you made jokes about the size
of TCBs in some other OSes. I bet we beat them all now in bloat, but it's 
good that things are shrinking again.

-Andi

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Thu, 21 Oct 2004 01:27:19 +0200
Andi Kleen <ak@suse.de> wrote:

> ok. good. I remember the times when you made jokes about the size
> of TCBs in some other OSes. I bet we beat them all now in bloat, but it's 
> good that things are shrinking again.

Nope, Solaris's TCB is still 3 times the size of ours.

BSD's is about the same size, yet they lack most of the
congestion control and ECN functionality we have.

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Lennert Buytenhek]

On Thu, Oct 21, 2004 at 01:04:05AM +0200, Andi Kleen wrote:

> Is it really necessary? Our struct sock is already bloated enough 
> as is ... If Lennert needs it for some obscure purpose it's probably
> better if he just does it locally.

My application is a large-ish streaming video and file download service,
where I want to be able to automatically report on netblocks that are
seeing unusual packet loss, and then use that data to alert our NOC
and subsequently kick my transit providers with.

Determining optimal routing within the internet is a somewhat of random
process, and a lot of ISPs just always use the route with the shortest
AS-path even though a lot of times that turns out not to be the best
path at all (in terms of latency/loss.)


--L

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

On Thu, 21 Oct 2004 01:44:23 +0200
Lennert Buytenhek <buytenh@wantstofly.org> wrote:

> My application is a large-ish streaming video and file download service,
> where I want to be able to automatically report on netblocks that are
> seeing unusual packet loss, and then use that data to alert our NOC
> and subsequently kick my transit providers with.

Isn't this what RSVP is for?  I realize that you may not be in
a position to use RSVP end-to-end as necessary, but watching for
retransmits by hand seems like simply a hackish way to do RSVP.

Furthermore, non-timeout based retransmits are actually normal even
on local subnets when a gigabit switch drops a packet to prevent
internal deadlocks and stuff like that.  I've seen this quite a bit.

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Arnaldo Carvalho de Melo]

David S. Miller wrote:
> On Thu, 21 Oct 2004 01:27:19 +0200
> Andi Kleen <ak@suse.de> wrote:
> 
> 
>>ok. good. I remember the times when you made jokes about the size
>>of TCBs in some other OSes. I bet we beat them all now in bloat, but it's 
>>good that things are shrinking again.
> 
> 
> Nope, Solaris's TCB is still 3 times the size of ours.
> 
> BSD's is about the same size, yet they lack most of the
> congestion control and ECN functionality we have.

<joke>
That doesn't mean we should keep trying to get to their level ;)
</joke>

/me goes back to his pile of sk_buff layer header pointers patches :-)

- Arnaldo

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[David S. Miller]

Please give this 2.6.x patch a spin:

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/10/20 21:21:01-07:00 davem@nuts.davemloft.net 
#   [TCP]: Add total num retransmits accounting.
#   
#   Signed-off-by: David S. Miller <davem@davemloft.net>
# 
# net/ipv4/tcp_output.c
#   2004/10/20 21:20:29-07:00 davem@nuts.davemloft.net +2 -0
#   [TCP]: Add total num retransmits accounting.
# 
# net/ipv4/tcp_diag.c
#   2004/10/20 21:20:29-07:00 davem@nuts.davemloft.net +2 -0
#   [TCP]: Add total num retransmits accounting.
# 
# include/linux/tcp.h
#   2004/10/20 21:20:29-07:00 davem@nuts.davemloft.net +4 -0
#   [TCP]: Add total num retransmits accounting.
# 
diff -Nru a/include/linux/tcp.h b/include/linux/tcp.h
--- a/include/linux/tcp.h	2004-10-20 21:21:21 -07:00
+++ b/include/linux/tcp.h	2004-10-20 21:21:21 -07:00
@@ -186,6 +186,8 @@
 
 	__u32	tcpi_rcv_rtt;
 	__u32	tcpi_rcv_space;
+
+	__u32	tcpi_total_retrans;
 };
 
 #ifdef __KERNEL__
@@ -363,6 +365,8 @@
 	__u8	pending;	/* Scheduled timer event	*/
 	__u8	urg_mode;	/* In urgent mode		*/
 	__u32	snd_up;		/* Urgent pointer		*/
+
+	__u32	total_retrans;	/* Total retransmits for entire connection */
 
 	/* The syn_wait_lock is necessary only to avoid proc interface having
 	 * to grab the main lock sock while browsing the listening hash
diff -Nru a/net/ipv4/tcp_diag.c b/net/ipv4/tcp_diag.c
--- a/net/ipv4/tcp_diag.c	2004-10-20 21:21:21 -07:00
+++ b/net/ipv4/tcp_diag.c	2004-10-20 21:21:21 -07:00
@@ -105,6 +105,8 @@
 
 	info->tcpi_rcv_rtt = jiffies_to_usecs(tp->rcv_rtt_est.rtt)>>3;
 	info->tcpi_rcv_space = tp->rcvq_space.space;
+
+	info->tcpi_total_retrans = tp->total_retrans;
 }
 
 static int tcpdiag_fill(struct sk_buff *skb, struct sock *sk,
diff -Nru a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
--- a/net/ipv4/tcp_output.c	2004-10-20 21:21:21 -07:00
+++ b/net/ipv4/tcp_output.c	2004-10-20 21:21:22 -07:00
@@ -1106,6 +1106,8 @@
 		/* Update global TCP statistics. */
 		TCP_INC_STATS(TCP_MIB_RETRANSSEGS);
 
+		tp->total_retrans++;
+
 #if FASTRETRANS_DEBUG > 0
 		if (TCP_SKB_CB(skb)->sacked&TCPCB_SACKED_RETRANS) {
 			if (net_ratelimit())

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Lennert Buytenhek]

On Wed, Oct 20, 2004 at 04:44:51PM -0700, David S. Miller wrote:

> > My application is a large-ish streaming video and file download service,
> > where I want to be able to automatically report on netblocks that are
> > seeing unusual packet loss, and then use that data to alert our NOC
> > and subsequently kick my transit providers with.
> 
> Isn't this what RSVP is for?  I realize that you may not be in
> a position to use RSVP end-to-end as necessary, but watching for
> retransmits by hand seems like simply a hackish way to do RSVP.

I'm approaching this from the other side.  There is no such thing as
an honest internet transit provider, and an often-seen trick is to
give ICMP packets a higher loss priority so that when you ping it all
looks okay even though say 0.5% of all non-ICMP traffic is actually
not making it to the other side because they're trying to stuff >10gbps
of traffic down a 10gbps pipe and don't want to invest more money in
their infrastructure to keep margins low.  They can then claim that
their 99.9% or 99.99% or 99.999% (depending on the crappiness level
of the provider) packet arrival guarantee is met because "ping from
my workstation doesn't show anything wrong" (actual words of someone
we've done business with.)

Regarding RSVP, at least in Europe people seem to prefer the
'lots of bandwidth'-approach.  In edge networks, people'd rather throw
an extra gig at the problem than to implement QoS.  And transit providers
will never implement it if it will allow people to find out that their
network really is underprovisioned.  I don't know of any provider that
uses or advertises QoS-related stuff such as RSVP.

RSVP is nice for stuff like research networks (Internet2, etc), but in
the actual internet you typically have to cross two or three other
providers to get to your destination and I really don't see each and
every one of those cooperating to provide RSVP end-to-end.  I don't see
end-to-end RSVP happening in the actual internet any time soon to be
honest.  If it ever will, I'm sure someone will find a way to make their
RSVP implementation lie to other providers about the true capacity of
their network (and then say "Sorry, we misconfigured it." if you find
out), at which point it becomes totally useless.


> Furthermore, non-timeout based retransmits are actually normal even
> on local subnets when a gigabit switch drops a packet to prevent
> internal deadlocks and stuff like that.  I've seen this quite a bit.

My crappy gigabit switch at home does this as well, but I've yet to
see this happen on the more 'serious' network gear.

A typical Amsterdam->LA single stream TCP test sees less than one in a
million lost packets over a good provider, and anywhere between ~100
and ~3000 per million over a crappy one.  The corresponding difference
in price is a factor of about 5-10.  (For example, $30-$40 per mbps per
month versus $5 assuming a reasonable traffic volume.)


--L

Re: way of figuring out total number of retransmitted packets on a TCP socket?

[Lennert Buytenhek]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

On Wed, Oct 20, 2004 at 09:37:39PM -0700, David S. Miller wrote:

> Please give this 2.6.x patch a spin:

Works quite nicely, thank you very much.  I've attached the sources
of the tools that we're using in combination with this -- basically
just a discard server and a client.  (You need the libivykis async
I/O lib installed to compile the server.)

A 1GB cross-atlantic transfer (~715kpackets) over a proper provider
typically shows something like this:  (Discard server is on a GbE
connection in LA, the transmitter on FastE in Amsterdam.)

	[...]
	tcpi_total_retrans      2


The same test done over a second-rate provider:

	[...]
	tcpi_total_retrans      3482


--L

[-- Attachment #2: eat.c --]
[-- Type: text/plain, Size: 1731 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <iv.h>
#include <netinet/in.h>
#include <syslog.h>
#include <sys/socket.h>
#include <unistd.h>

struct iv_fd master_fd;

struct connection
{
	struct iv_fd		fd;
};

void connection_handler(void *_conn)
{
	struct connection *conn = (struct connection *)_conn;
	char buf[65536];
	int len;

	len = iv_read(&(conn->fd), buf, sizeof(buf));
	if (len <= 0) {
		if (len == 0 || errno != EAGAIN) {
			iv_unregister_fd(&(conn->fd));
			close(conn->fd.fd);
			free(conn);
		}
		return;
	}
}

void new_connection(void *_dummy)
{
	struct sockaddr_in addr;
	socklen_t addrlen;
	struct connection *conn;
	int fd;

	addrlen = sizeof(addr);
	fd = iv_accept(&master_fd, (struct sockaddr *)&addr, &addrlen);
	if (fd <= 0) {
		if (fd < 0 && errno == EAGAIN)
			return;
		perror("iv_accept");
		exit(-1);
	}

	conn = malloc(sizeof(*conn));
	if (conn == NULL) {
		syslog(LOG_ALERT, "new_connection: memory allocation error, dropping connection");
		close(fd);
		return;
	}

	INIT_IV_FD(&(conn->fd));
	conn->fd.fd = fd;
	conn->fd.cookie = (void *)conn;
	conn->fd.handler_in = connection_handler;
	iv_register_fd(&(conn->fd));
}

int main()
{
	struct sockaddr_in addr;
	int fd;

	fd = socket(AF_INET, SOCK_STREAM, 0);
	if (fd < 0) {
		perror("socket");
		exit(-1);
	}

	addr.sin_family = AF_INET;
	addr.sin_addr.s_addr = htonl(INADDR_ANY);
	addr.sin_port = htons(9);
	if (bind(fd, (struct sockaddr *)&addr, sizeof(addr)) < 0) {
		perror("bind");
		exit(-1);
	}

	if (listen(fd, 1000) < 0) {
		perror("listen");
		exit(-1);
	}

	iv_init();

	INIT_IV_FD(&(master_fd));
	master_fd.fd = fd;
	master_fd.handler_in = new_connection;
	iv_register_fd(&(master_fd));

	iv_main();

	return 0;
}

[-- Attachment #3: feed.c --]
[-- Type: text/plain, Size: 5941 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>

#include <linux/sockios.h>
#include <linux/tcp.h>

#ifndef SOL_TCP
#define SOL_TCP		6
#endif

struct tcp_info2
{
	u_int8_t	tcpi_state;
	u_int8_t	tcpi_ca_state;
	u_int8_t	tcpi_retransmits;
	u_int8_t	tcpi_probes;
	u_int8_t	tcpi_backoff;
	u_int8_t	tcpi_options;
	u_int8_t	tcpi_snd_wscale:4, tcpi_rcv_wscale:4;
 
	u_int32_t	tcpi_rto;
	u_int32_t	tcpi_ato;
	u_int32_t	tcpi_snd_mss;
	u_int32_t	tcpi_rcv_mss;
 
	u_int32_t	tcpi_unacked;
	u_int32_t	tcpi_sacked;
	u_int32_t	tcpi_lost;
	u_int32_t	tcpi_retrans;
	u_int32_t	tcpi_fackets;
 
	/* Times. */
	u_int32_t	tcpi_last_data_sent;
	u_int32_t	tcpi_last_ack_sent;	/* Not remembered, sorry. */
	u_int32_t	tcpi_last_data_recv;
	u_int32_t	tcpi_last_ack_recv;
 
	/* Metrics. */
	u_int32_t	tcpi_pmtu;
	u_int32_t	tcpi_rcv_ssthresh;
	u_int32_t	tcpi_rtt;
	u_int32_t	tcpi_rttvar;
	u_int32_t	tcpi_snd_ssthresh;
	u_int32_t	tcpi_snd_cwnd;
	u_int32_t	tcpi_advmss;
	u_int32_t	tcpi_reordering;
 
	u_int32_t	tcpi_rcv_rtt;
	u_int32_t	tcpi_rcv_space;
 
	u_int32_t	tcpi_total_retrans;
};

static unsigned char zeroes[65536];

static char *tcpi_state(int state)
{
	switch (state) {
	case TCP_ESTABLISHED:		return "ESTABLISHED";
	case TCP_SYN_SENT:		return "SYN_SENT";
	case TCP_SYN_RECV:		return "SYN_RECV";
	case TCP_FIN_WAIT1:		return "FIN_WAIT1";
	case TCP_FIN_WAIT2:		return "FIN_WAIT2";
	case TCP_TIME_WAIT:		return "TIME_WAIT";
	case TCP_CLOSE:			return "CLOSE";
	case TCP_CLOSE_WAIT:		return "CLOSE_WAIT";
	case TCP_LAST_ACK:		return "LAST_ACK";
	case TCP_LISTEN:		return "LISTEN";
	case TCP_CLOSING:		return "CLOSING";
	}

	return "invalid";
}

static char *tcpi_ca_state(int state)
{
	switch (state) {
	case TCP_CA_Open:		return "CA_Open";
	case TCP_CA_Disorder:		return "CA_Disorder";
	case TCP_CA_CWR:		return "CA_CWR";
	case TCP_CA_Recovery:		return "CA_Recovery";
	case TCP_CA_Loss:		return "CA_Loss";
	}

	return "invalid";
}

int main(int argc, char *argv[])
{
	struct sockaddr_in addr;
	struct tcp_info2 tcpi;
	struct hostent *he;
	int bytes;
	int port;
	int ret;
	int fd;

	if (argc <= 1) {
		fprintf(stderr, "syntax: %s <hostname> [port] [megs]\n", argv[0]);
		exit(-1);
	}

	he = gethostbyname(argv[1]);
	if (he == NULL) {
		herror("gethostbyname");
		exit(-1);
	}
	addr.sin_family = he->h_addrtype;
	memcpy(&(addr.sin_addr), he->h_addr_list[0], he->h_length);
	endhostent();

	port = 9;
	if (argc > 2 && sscanf(argv[2], "%d", &port) != 1) {
		fprintf(stderr, "%s: second argument not numeric\n", argv[0]);
		exit(-1);
	}
	addr.sin_port = htons(port);

	bytes = 1024;
	if (argc > 3 && sscanf(argv[3], "%d", &bytes) != 1) {
		fprintf(stderr, "%s: third argument not numeric\n", argv[0]);
		exit(-1);
	}
	bytes <<= 20;

	fd = socket(AF_INET, SOCK_STREAM, 0);
	if (fd < 0) {
		perror("socket");
		return 1;
	}

	ret = connect(fd, (struct sockaddr *)&addr, sizeof(addr));
	if (ret < 0) {
		perror("connect");
		return 1;
	}

	ret = sizeof(tcpi);
	if (getsockopt(fd, SOL_TCP, TCP_INFO, &tcpi, &ret) < 0) {
		perror("getsockopt(TCP_INFO)");
		return 1;
	}

	while (bytes) {
		int tosend;

		tosend = bytes;
		if (tosend > sizeof(zeroes))
			tosend = sizeof(zeroes);

		ret = write(fd, zeroes, tosend);
		if (ret < 0) {
			perror("write");
			break;
		}

		bytes -= ret;
	}

	shutdown(fd, SHUT_RDWR);
	sleep(1);

	tcpi.tcpi_total_retrans = 0xdeadbeef;

	ret = sizeof(tcpi);
	if (getsockopt(fd, SOL_TCP, TCP_INFO, &tcpi, &ret) < 0) {
		perror("getsockopt(TCP_INFO)");
		return 1;
	}

	printf("tcpi_state\t\t%d, %s\n", tcpi.tcpi_state, tcpi_state(tcpi.tcpi_state));
	printf("tcpi_ca_state\t\t%d, %s\n", tcpi.tcpi_ca_state, tcpi_ca_state(tcpi.tcpi_ca_state));
	printf("tcpi_retransmits\t%d\n", tcpi.tcpi_retransmits);
	printf("tcpi_probes\t\t%d\n", tcpi.tcpi_probes);
	printf("tcpi_backoff\t\t%d\n", tcpi.tcpi_backoff);
	printf("tcpi_options\t\t%d", tcpi.tcpi_options);
	if (tcpi.tcpi_options) {
		printf(" [");
		if (tcpi.tcpi_options & TCPI_OPT_TIMESTAMPS)
			printf("TIMESTAMPS ");
		if (tcpi.tcpi_options & TCPI_OPT_SACK)
			printf("SACK ");
		if (tcpi.tcpi_options & TCPI_OPT_WSCALE)
			printf("WSCALE ");
		if (tcpi.tcpi_options & TCPI_OPT_ECN)
			printf("ECN ");
		printf("]");
	}
	printf("\n");
	if (tcpi.tcpi_options & TCPI_OPT_WSCALE) {
		printf("tcpi_snd_wscale\t\t%d\n", tcpi.tcpi_snd_wscale);
		printf("tcpi_rcv_wscale\t\t%d\n", tcpi.tcpi_rcv_wscale);
	}
	printf("\n");

	printf("tcpi_rto\t\t%d\n", tcpi.tcpi_rto);
	printf("tcpi_ato\t\t%d\n", tcpi.tcpi_ato);
	printf("tcpi_snd_mss\t\t%d\n", tcpi.tcpi_snd_mss);
	printf("tcpi_rcv_mss\t\t%d\n", tcpi.tcpi_rcv_mss);
	printf("\n");

	printf("tcpi_unacked\t\t%d\n", tcpi.tcpi_unacked);
	printf("tcpi_sacked\t\t%d\n", tcpi.tcpi_sacked);
	printf("tcpi_lost\t\t%d\n", tcpi.tcpi_lost);
	printf("tcpi_retrans\t\t%d\n", tcpi.tcpi_retrans);
	printf("tcpi_fackets\t\t%d\n", tcpi.tcpi_fackets);
	printf("\n");

	printf("tcpi_last_data_sent\t%d\n", tcpi.tcpi_last_data_sent);
	printf("tcpi_last_ack_sent\t%d\n", tcpi.tcpi_last_ack_sent);
	printf("tcpi_last_data_recv\t%d\n", tcpi.tcpi_last_data_recv);
	printf("tcpi_last_ack_recv\t%d\n", tcpi.tcpi_last_ack_recv);
	printf("\n");

	printf("tcpi_pmtu\t\t%d\n", tcpi.tcpi_pmtu);
	printf("tcpi_rcv_sstresh\t%d\n", tcpi.tcpi_rcv_ssthresh);
	printf("tcpi_rtt\t\t%d\n", tcpi.tcpi_rtt);
	printf("tcpi_rttvar\t\t%d\n", tcpi.tcpi_rttvar);
	printf("tcpi_snd_sstresh\t%d\n", tcpi.tcpi_snd_ssthresh);
	printf("tcpi_snd_cwnd\t\t%d\n", tcpi.tcpi_snd_cwnd);
	printf("tcpi_advmss\t\t%d\n", tcpi.tcpi_advmss);
	printf("tcpi_reordering\t\t%d\n", tcpi.tcpi_reordering);
	printf("\n");

	printf("tcpi_rcv_rtt\t\t%d\n", tcpi.tcpi_rcv_rtt);
	printf("tcpi_rcv_space\t\t%d\n", tcpi.tcpi_rcv_space);
	printf("\n");

	if (tcpi.tcpi_total_retrans != 0xdeadbeef) {
		printf("tcpi_total_retrans\t%d\n", tcpi.tcpi_total_retrans);
		printf("\n");
	}

	close(fd);

	return 0;
}