Not all tcp sockets are found by ss command (iproute2)

Not all tcp sockets are found by ss command (iproute2)

[Eric Dumazet]

Hi all

I sent this message 2 days ago on linux-net@vger but got no answer yet. 
Maybe this list is a better place ?

I have two oddities with ss command (from iproute2 package)

linux kernel 2.6.8.1, SMP on x86_64
ss -V
ss utility, iproute2-ss040823

1) It seems TCP sockets in SYN-RECV are not displayed at all.

ss state syn-recv
Recv-Q Send-Q                                     Local Address:Port 
                                   Peer Address:Port

<nothing>

2) It seems not all tcp sockets are fetched/displayed on a very loaded 
server.

# ss -an|wc -l ; cat /proc/net/sockstat
  565859
sockets: used 611269
TCP: inuse 613140 orphan 2572 tw 4747 alloc 613820 mem 116376
UDP: inuse 7
RAW: inuse 0
FRAG: inuse 1 memory 544

You can see on this sample that ss -an displayed 565859 lines, but 
/proc/net/sockstat correctly said that more sockets were used.

Of course, /proc/net/tcp on such machine is not an option (never 
completes), so I cant list all connections for this machine. I really 
*need* a working ss command.

Are they known problems ?

Thank you

Re: Not all tcp sockets are found by ss command (iproute2)

[Herbert Xu]

Eric Dumazet <dada1@cosmosbay.com> wrote:
>
> Of course, /proc/net/tcp on such machine is not an option (never 
> completes), so I cant list all connections for this machine. I really 
> *need* a working ss command.
> 
> Are they known problems ?

They are known now :) I'm working on a fix to these problems so
stay tuned.
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[1/3] [TCP] Create tcpdiag_dump_sock

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

On Thu, Oct 07, 2004 at 06:19:41PM +1000, Herbert Xu wrote:
> Eric Dumazet <dada1@cosmosbay.com> wrote:
> >
> > Of course, /proc/net/tcp on such machine is not an option (never 
> > completes), so I cant list all connections for this machine. I really 
> > *need* a working ss command.
> > 
> > Are they known problems ?
> 
> They are known now :) I'm working on a fix to these problems so
> stay tuned.

OK, this series of patches should solve your problems.

The first one move tcpdiag_bc_run calls into one place so that we can
change its parameters next.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: p1 --]
[-- Type: text/plain, Size: 2302 bytes --]

===== net/ipv4/tcp_diag.c 1.19 vs 1.20 =====
--- 1.19/net/ipv4/tcp_diag.c	2004-10-06 15:01:14 +10:00
+++ 1.20/net/ipv4/tcp_diag.c	2004-10-14 21:11:20 +10:00
@@ -466,16 +466,26 @@
 	return len == 0 ? 0 : -EINVAL;
 }
 
+static int tcpdiag_dump_sock(struct sk_buff *skb, struct sock *sk,
+			     struct netlink_callback *cb)
+{
+	struct tcpdiagreq *r = NLMSG_DATA(cb->nlh);
+
+	if (cb->nlh->nlmsg_len > 4 + NLMSG_SPACE(sizeof(*r))) {
+		struct rtattr *bc = (struct rtattr *)(r + 1);
+		if (!tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), sk))
+			return 0;
+	}
+
+	return tcpdiag_fill(skb, sk, r->tcpdiag_ext, NETLINK_CB(cb->skb).pid,
+			    cb->nlh->nlmsg_seq);
+}
 
 static int tcpdiag_dump(struct sk_buff *skb, struct netlink_callback *cb)
 {
 	int i, num;
 	int s_i, s_num;
 	struct tcpdiagreq *r = NLMSG_DATA(cb->nlh);
-	struct rtattr *bc = NULL;
-
-	if (cb->nlh->nlmsg_len > 4+NLMSG_SPACE(sizeof(struct tcpdiagreq)))
-		bc = (struct rtattr*)(r+1);
 
 	s_i = cb->args[1];
 	s_num = num = cb->args[2];
@@ -502,11 +512,7 @@
 				if (r->id.tcpdiag_sport != inet->sport &&
 				    r->id.tcpdiag_sport)
 					goto next_listen;
-				if (bc && !tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), sk))
-					goto next_listen;
-				if (tcpdiag_fill(skb, sk, r->tcpdiag_ext,
-						 NETLINK_CB(cb->skb).pid,
-						 cb->nlh->nlmsg_seq) <= 0) {
+				if (tcpdiag_dump_sock(skb, sk, cb) < 0) {
 					tcp_listen_unlock();
 					goto done;
 				}
@@ -546,11 +552,7 @@
 				goto next_normal;
 			if (r->id.tcpdiag_dport != inet->dport && r->id.tcpdiag_dport)
 				goto next_normal;
-			if (bc && !tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), sk))
-				goto next_normal;
-			if (tcpdiag_fill(skb, sk, r->tcpdiag_ext,
-					 NETLINK_CB(cb->skb).pid,
-					 cb->nlh->nlmsg_seq) <= 0) {
+			if (tcpdiag_dump_sock(skb, sk, cb) < 0) {
 				read_unlock_bh(&head->lock);
 				goto done;
 			}
@@ -571,11 +573,7 @@
 				if (r->id.tcpdiag_dport != inet->dport &&
 				    r->id.tcpdiag_dport)
 					goto next_dying;
-				if (bc && !tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), sk))
-					goto next_dying;
-				if (tcpdiag_fill(skb, sk, r->tcpdiag_ext,
-						 NETLINK_CB(cb->skb).pid,
-						 cb->nlh->nlmsg_seq) <= 0) {
+				if (tcpdiag_dump_sock(skb, sk, cb) < 0) {
 					read_unlock_bh(&head->lock);
 					goto done;
 				}

[2/3] [TCP] Make tcpdiag_bc_run take tcpdiag_entry

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

This patch adds a tcpdiag_entry struct and makes tcpdiag_bc_run use it
instead of a struct sock.  This will allow us to use tcpdiag_bc_run on
struct open_request in the next patch.

Please note that I've left the CONFIG_IPV6 defines in as to support
ipv6 modules we'll need to modularise tcpdiag itself.  I'll probably
do that after this is all fixed.
 
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: p2 --]
[-- Type: text/plain, Size: 3189 bytes --]

===== net/ipv4/tcp_diag.c 1.20 vs 1.21 =====
--- 1.20/net/ipv4/tcp_diag.c	2004-10-14 21:11:20 +10:00
+++ 1.21/net/ipv4/tcp_diag.c	2004-10-14 21:15:04 +10:00
@@ -29,6 +29,16 @@
 
 #include <linux/tcp_diag.h>
 
+struct tcpdiag_entry
+{
+	u32 *saddr;
+	u32 *daddr;
+	u16 sport;
+	u16 dport;
+	u16 family;
+	u16 userlocks;
+};
+
 static struct sock *tcpnl;
 
 
@@ -324,11 +334,11 @@
 }
 
 
-static int tcpdiag_bc_run(const void *bc, int len, struct sock *sk)
+static int tcpdiag_bc_run(const void *bc, int len,
+			  const struct tcpdiag_entry *entry)
 {
 	while (len > 0) {
 		int yes = 1;
-		struct inet_opt *inet = inet_sk(sk);
 		const struct tcpdiag_bc_op *op = bc;
 
 		switch (op->code) {
@@ -338,19 +348,19 @@
 			yes = 0;
 			break;
 		case TCPDIAG_BC_S_GE:
-			yes = inet->num >= op[1].no;
+			yes = entry->sport >= op[1].no;
 			break;
 		case TCPDIAG_BC_S_LE:
-			yes = inet->num <= op[1].no;
+			yes = entry->dport <= op[1].no;
 			break;
 		case TCPDIAG_BC_D_GE:
-			yes = ntohs(inet->dport) >= op[1].no;
+			yes = entry->dport >= op[1].no;
 			break;
 		case TCPDIAG_BC_D_LE:
-			yes = ntohs(inet->dport) <= op[1].no;
+			yes = entry->dport <= op[1].no;
 			break;
 		case TCPDIAG_BC_AUTO:
-			yes = !(sk->sk_userlocks & SOCK_BINDPORT_LOCK);
+			yes = !(entry->userlocks & SOCK_BINDPORT_LOCK);
 			break;
 		case TCPDIAG_BC_S_COND:
 		case TCPDIAG_BC_D_COND:
@@ -360,7 +370,7 @@
 
 			if (cond->port != -1 &&
 			    cond->port != (op->code == TCPDIAG_BC_S_COND ?
-					     inet->num : ntohs(inet->dport))) {
+					     entry->sport : entry->dport)) {
 				yes = 0;
 				break;
 			}
@@ -368,26 +378,14 @@
 			if (cond->prefix_len == 0)
 				break;
 
-#ifdef CONFIG_IPV6
-			if (sk->sk_family == AF_INET6) {
-				struct ipv6_pinfo *np = inet6_sk(sk);
-
-				if (op->code == TCPDIAG_BC_S_COND)
-					addr = (u32*)&np->rcv_saddr;
-				else
-					addr = (u32*)&np->daddr;
-			} else
-#endif
-			{
-				if (op->code == TCPDIAG_BC_S_COND)
-					addr = &inet->rcv_saddr;
-				else
-					addr = &inet->daddr;
-			}
+			if (op->code == TCPDIAG_BC_S_COND)
+				addr = entry->saddr;
+			else
+				addr = entry->daddr;
 
 			if (bitstring_match(addr, cond->addr, cond->prefix_len))
 				break;
-			if (sk->sk_family == AF_INET6 &&
+			if (entry->family == AF_INET6 &&
 			    cond->family == AF_INET) {
 				if (addr[0] == 0 && addr[1] == 0 &&
 				    addr[2] == htonl(0xffff) &&
@@ -472,8 +470,28 @@
 	struct tcpdiagreq *r = NLMSG_DATA(cb->nlh);
 
 	if (cb->nlh->nlmsg_len > 4 + NLMSG_SPACE(sizeof(*r))) {
+		struct tcpdiag_entry entry;
 		struct rtattr *bc = (struct rtattr *)(r + 1);
-		if (!tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), sk))
+		struct inet_opt *inet = inet_sk(sk);
+
+		entry.family = sk->sk_family;
+#ifdef CONFIG_IPV6
+		if (entry.family == AF_INET6) {
+			struct ipv6_pinfo *np = inet6_sk(sk);
+
+			entry.saddr = np->rcv_saddr.s6_addr32;
+			entry.daddr = np->daddr.s6_addr32;
+		} else
+#endif
+		{
+			entry.saddr = &inet->rcv_saddr;
+			entry.daddr = &inet->daddr;
+		}
+		entry.sport = inet->num;
+		entry.dport = ntohs(inet->dport);
+		entry.userlocks = sk->sk_userlocks;
+
+		if (!tcpdiag_bc_run(RTA_DATA(bc), RTA_PAYLOAD(bc), &entry))
 			return 0;
 	}
 

[3/3] [TCP] Dump SYN_RECV sockets in tcpdiag

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 410 bytes --]

Finally this patch adds the code to list SYN_RECV sockets.  A future
enhancement would be to do this for the GET operation as well.
  
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: p3 --]
[-- Type: text/plain, Size: 4854 bytes --]

===== net/ipv4/tcp_diag.c 1.21 vs edited =====
--- 1.21/net/ipv4/tcp_diag.c	2004-10-14 21:15:04 +10:00
+++ edited/net/ipv4/tcp_diag.c	2004-10-14 23:09:14 +10:00
@@ -18,6 +18,7 @@
 #include <linux/random.h>
 #include <linux/cache.h>
 #include <linux/init.h>
+#include <linux/time.h>
 
 #include <net/icmp.h>
 #include <net/tcp.h>
@@ -499,6 +500,141 @@
 			    cb->nlh->nlmsg_seq);
 }
 
+static int tcpdiag_fill_req(struct sk_buff *skb, struct sock *sk,
+			    struct open_request *req,
+			    u32 pid, u32 seq)
+{
+	struct inet_opt *inet = inet_sk(sk);
+	unsigned char *b = skb->tail;
+	struct tcpdiagmsg *r;
+	struct nlmsghdr *nlh;
+	long tmo;
+
+	nlh = NLMSG_PUT(skb, pid, seq, TCPDIAG_GETSOCK, sizeof(*r));
+	r = NLMSG_DATA(nlh);
+
+	r->tcpdiag_family = sk->sk_family;
+	r->tcpdiag_state = TCP_SYN_RECV;
+	r->tcpdiag_timer = 1;
+	r->tcpdiag_retrans = req->retrans;
+
+	r->id.tcpdiag_if = sk->sk_bound_dev_if;
+	r->id.tcpdiag_cookie[0] = (u32)(unsigned long)req;
+	r->id.tcpdiag_cookie[1] = (u32)(((unsigned long)req >> 31) >> 1);
+
+	tmo = req->expires - jiffies;
+	if (tmo < 0)
+		tmo = 0;
+
+	r->id.tcpdiag_sport = inet->sport;
+	r->id.tcpdiag_dport = req->rmt_port;
+	r->id.tcpdiag_src[0] = req->af.v4_req.loc_addr;
+	r->id.tcpdiag_dst[0] = req->af.v4_req.rmt_addr;
+	r->tcpdiag_expires = jiffies_to_msecs(tmo),
+	r->tcpdiag_rqueue = 0;
+	r->tcpdiag_wqueue = 0;
+	r->tcpdiag_uid = sock_i_uid(sk);
+	r->tcpdiag_inode = 0;
+#ifdef CONFIG_IPV6
+	if (r->tcpdiag_family == AF_INET6) {
+		ipv6_addr_copy((struct in6_addr *)r->id.tcpdiag_src,
+			       &req->af.v6_req.loc_addr);
+		ipv6_addr_copy((struct in6_addr *)r->id.tcpdiag_dst,
+			       &req->af.v6_req.rmt_addr);
+	}
+#endif
+	nlh->nlmsg_len = skb->tail - b;
+
+	return skb->len;
+
+nlmsg_failure:
+	skb_trim(skb, b - skb->data);
+	return -1;
+}
+
+static int tcpdiag_dump_reqs(struct sk_buff *skb, struct sock *sk,
+			     struct netlink_callback *cb)
+{
+	struct tcpdiag_entry entry;
+	struct tcpdiagreq *r = NLMSG_DATA(cb->nlh);
+	struct tcp_opt *tp = tcp_sk(sk);
+	struct tcp_listen_opt *lopt;
+	struct rtattr *bc = NULL;
+	struct inet_opt *inet = inet_sk(sk);
+	int j, s_j;
+	int reqnum, s_reqnum;
+	int err = 0;
+
+	s_j = cb->args[3];
+	s_reqnum = cb->args[4];
+
+	if (s_j > 0)
+		s_j--;
+
+	entry.family = sk->sk_family;
+
+	read_lock_bh(&tp->syn_wait_lock);
+
+	lopt = tp->listen_opt;
+	if (!lopt || !lopt->qlen)
+		goto out;
+
+	if (cb->nlh->nlmsg_len > 4 + NLMSG_SPACE(sizeof(*r))) {
+		bc = (struct rtattr *)(r + 1);
+		entry.sport = inet->num;
+		entry.userlocks = sk->sk_userlocks;
+	}
+
+	for (j = s_j; j < TCP_SYNQ_HSIZE; j++) {
+		struct open_request *req, *head = lopt->syn_table[j];
+
+		reqnum = 0;
+		for (req = head; req; reqnum++, req = req->dl_next) {
+			if (reqnum < s_reqnum)
+				continue;
+			if (r->id.tcpdiag_dport != req->rmt_port &&
+			    r->id.tcpdiag_dport)
+				continue;
+
+			if (bc) {
+				entry.saddr =
+#ifdef CONFIG_IPV6
+					(entry.family == AF_INET6) ?
+					req->af.v6_req.loc_addr.s6_addr32 :
+#endif
+					&req->af.v4_req.loc_addr;
+				entry.daddr = 
+#ifdef CONFIG_IPV6
+					(entry.family == AF_INET6) ?
+					req->af.v6_req.rmt_addr.s6_addr32 :
+#endif
+					&req->af.v4_req.rmt_addr;
+				entry.dport = ntohs(req->rmt_port);
+
+				if (!tcpdiag_bc_run(RTA_DATA(bc),
+						    RTA_PAYLOAD(bc), &entry))
+					continue;
+			}
+
+			err = tcpdiag_fill_req(skb, sk, req,
+					       NETLINK_CB(cb->skb).pid,
+					       cb->nlh->nlmsg_seq);
+			if (err < 0) {
+				cb->args[3] = j + 1;
+				cb->args[4] = reqnum;
+				goto out;
+			}
+		}
+
+		s_reqnum = 0;
+	}
+
+out:
+	read_unlock_bh(&tp->syn_wait_lock);
+
+	return err;
+}
+
 static int tcpdiag_dump(struct sk_buff *skb, struct netlink_callback *cb)
 {
 	int i, num;
@@ -516,27 +652,47 @@
 			struct sock *sk;
 			struct hlist_node *node;
 
-			if (i > s_i)
-				s_num = 0;
-
 			num = 0;
 			sk_for_each(sk, node, &tcp_listening_hash[i]) {
 				struct inet_opt *inet = inet_sk(sk);
-				if (num < s_num)
-					goto next_listen;
-				if (!(r->tcpdiag_states&TCPF_LISTEN) ||
-				    r->id.tcpdiag_dport)
-					goto next_listen;
+
+				if (num < s_num) {
+					num++;
+					continue;
+				}
+
 				if (r->id.tcpdiag_sport != inet->sport &&
 				    r->id.tcpdiag_sport)
 					goto next_listen;
+
+				if (!(r->tcpdiag_states&TCPF_LISTEN) ||
+				    r->id.tcpdiag_dport ||
+				    cb->args[3] > 0)
+					goto syn_recv;
+
 				if (tcpdiag_dump_sock(skb, sk, cb) < 0) {
 					tcp_listen_unlock();
 					goto done;
 				}
+
+syn_recv:
+				if (!(r->tcpdiag_states&TCPF_SYN_RECV))
+					goto next_listen;
+
+				if (tcpdiag_dump_reqs(skb, sk, cb) < 0) {
+					tcp_listen_unlock();
+					goto done;
+				}
+
 next_listen:
+				cb->args[3] = 0;
+				cb->args[4] = 0;
 				++num;
 			}
+
+			s_num = 0;
+			cb->args[3] = 0;
+			cb->args[4] = 0;
 		}
 		tcp_listen_unlock();
 skip_listen_ht:

Re: [3/3] [TCP] Dump SYN_RECV sockets in tcpdiag

[David S. Miller]

On Fri, 15 Oct 2004 07:35:17 +1000
Herbert Xu <herbert@gondor.apana.org.au> wrote:

> Finally this patch adds the code to list SYN_RECV sockets.  A future
> enhancement would be to do this for the GET operation as well.
>   
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Also applied, thanks for fixing all of this Herbert.

Feel free to make the mentioned GET enhancement :-)

Re: [2/3] [TCP] Make tcpdiag_bc_run take tcpdiag_entry

[David S. Miller]

On Fri, 15 Oct 2004 07:33:47 +1000
Herbert Xu <herbert@gondor.apana.org.au> wrote:

> This patch adds a tcpdiag_entry struct and makes tcpdiag_bc_run use it
> instead of a struct sock.  This will allow us to use tcpdiag_bc_run on
> struct open_request in the next patch.
> 
> Please note that I've left the CONFIG_IPV6 defines in as to support
> ipv6 modules we'll need to modularise tcpdiag itself.  I'll probably
> do that after this is all fixed.
>  
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Also applied, thanks.

Re: [1/3] [TCP] Create tcpdiag_dump_sock

[David S. Miller]

On Fri, 15 Oct 2004 07:30:30 +1000
Herbert Xu <herbert@gondor.apana.org.au> wrote:

> The first one move tcpdiag_bc_run calls into one place so that we can
> change its parameters next.
> 
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Applied, thanks Herbert.