From: Li Yu <raise.sail@gmail.com>
To: Bruce Curtis <brutus@google.com>
Cc: David Miller <davem@davemloft.net>, netdev <netdev@vger.kernel.org>
Subject: Re: v3 for tcp friends?
Date: Wed, 23 Jan 2013 11:14:26 +0800 [thread overview]
Message-ID: <50FF5592.60008@gmail.com> (raw)
In-Reply-To: <CAEkNxbH0MAAU9oiwwrFTbMJP1yzfNdxD5NDdZOqGnvhTLvodoQ@mail.gmail.com>
于 2013年01月23日 05:08, Bruce Curtis 写道:
> Thanks, Li
>
> Started working on friends again, v4, more soon.
>
>
:)
I found another odd bug in TCP friends v3, the clients
may hang at tcp_sendmsg() -> sk_stream_wait_memory() with or
without my refcnt fix patch.
Below shell script can reproduce this bug:
#! /bin/sh -x
sysctl -w net.ipv4.tcp_rmem="4096 1073741824 1073741824"
sysctl -w net.ipv4.tcp_wmem="4096 1073741824 1073741824"
sysctl -w net.ipv4.tcp_friends=1
msg=64K
buf=256M
pkill -9 netserver
netserver
netperf -t TCP_STREAM -l 1 -- -m ${msg} -M ${msg} -s ${buf} -S ${buf} -4
sysctl -w net.ipv4.tcp_friends=0
pkill -9 netserver
netserver
netperf -t TCP_STREAM -l 1 -- -m ${msg} -M ${msg} -s ${buf} -S ${buf} -4
##################SCRIPT END###################
netperf kernel stack is (by cat /proc/$netperf_pid/stack)
[<ffffffff812ce939>] sk_stream_wait_memory+0x2d9/0x2f0
[<ffffffff8131460c>] tcp_sendmsg+0xf6c/0x1240
[<ffffffff8133c117>] inet_sendmsg+0xf7/0x110
[<ffffffff812bedfd>] sock_sendmsg+0x7d/0xa0
[<ffffffff812c0e4d>] sys_sendto+0x13d/0x190
[<ffffffff8138a6c2>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
netserver kernel stack is :
[<ffffffff812c46ae>] sk_wait_data+0x8e/0xe0
[<ffffffff81315993>] tcp_recvmsg+0x5c3/0xbe0
[<ffffffff8133aefd>] inet_recvmsg+0xed/0x110
[<ffffffff812becf4>] sock_recvmsg+0x84/0xb0
[<ffffffff812c0fae>] sys_recvfrom+0xee/0x170
[<ffffffff8138a6c2>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
And, "netstat -tnp" give us below results:
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 0 127.0.0.1:35451 127.0.0.1:12865
ESTABLISHED 2316/netperf
tcp6 0 0 127.0.0.1:12865 127.0.0.1:35451
ESTABLISHED 2317/netserver
(It seems that netperf hangs on the control connection of benchmark)
I also try to fix this ...
Thanks
Yu
> On Mon, Jan 21, 2013 at 12:55 AM, Li Yu <raise.sail@gmail.com
> <mailto:raise.sail@gmail.com>> wrote:
>
> 2013/1/21 Li Yu <raise.sail@gmail.com <mailto:raise.sail@gmail.com>>
>
> 于 2013年01月21日 15:29, Li Yu 写道:
>
> 于 2012年09月05日 00:58, David Miller 写道:
>
> From: Bruce Curtis <brutus@google.com
> <mailto:brutus@google.com>>
> Date: Tue, 4 Sep 2012 08:10:23 -0700
>
> Will do, issues addressed, I'll get the patch out
> later today or
> tomorrow at the latest.
>
>
> Thanks a lot Bruce.
> --
> To unsubscribe from this list: send the line
> "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> <mailto:majordomo@vger.kernel.org>
> More majordomo info at
> http://vger.kernel.org/__majordomo-info.html
> <http://vger.kernel.org/majordomo-info.html>
>
>
>
> Hi, Bruce,
>
> I tested the TCP friends, found a bug here:
>
> [ 106.541372] Pid: 1765, comm: client Not tainted
> 3.7.0-rc1+ #25
> [ 106.543121] Call Trace:
> [ 106.543950] [<ffffffff8133d212>]
> inet_sock_destruct+0x102/0x1f0
> [ 106.545687] [<ffffffff812c38ad>] __sk_free+0x1d/0x110
> [ 106.547209] [<ffffffff812c3a1c>] sk_free+0x1c/0x20
> [ 106.548611] [<ffffffff8131680c>] tcp_close+0x6c/0x3f0
> [ 106.549863] [<ffffffff8133caea>] inet_release+0xda/0xf0
> [ 106.551134] [<ffffffff8133ca30>] ? inet_release+0x20/0xf0
> [ 106.552419] [<ffffffff8137f3de>] ? mutex_unlock+0xe/0x10
> [ 106.553658] [<ffffffff812bf948>] sock_release+0x28/0xa0
> [ 106.557366] [<ffffffff812bfd69>] sock_close+0x29/0x30
> [ 106.558831] [<ffffffff81128972>] __fput+0x122/0x210
> [ 106.560541] [<ffffffff81128a6e>] ____fput+0xe/0x10
> [ 106.562006] [<ffffffff8105354e>] task_work_run+0x9e/0xd0
> [ 106.563285] [<ffffffff810027e1>] do_notify_resume+0x61/0x70
> [ 106.564582] [<ffffffff8138a908>] int_signal+0x12/0x17
>
>
> I also backported and tested it on stable kernel
> 3.7.3/RHEL6
> kernel, this bug still exists. It seem that client may close
> listening
> sockets, may we need to add one reference count for listen
> socket
> before send its address to peer?
>
>
> Sorry, I lost an important line of kernel log before above them:
>
> [ 106.539367] IPv4: Attempt to release TCP socket in state 10
> ffff880074abb5c0
>
> BTW: state 10 = TCP_LISTEN
>
>
> It seem this patch works for me.
>
> diff --git a/net/ipv4/inet_connection_sock.c
> b/net/ipv4/inet_connection_sock.c
> index 9641215..a625c02 100644
> --- a/net/ipv4/inet_connection_sock.c
> +++ b/net/ipv4/inet_connection_sock.c
> @@ -623,8 +623,11 @@ struct sock *inet_csk_clone(struct sock *sk,
> const struct request_sock *req,
> sock_hold(newsk);
> was = xchg(&req->friend->sk_friend, newsk);
> /* If requester already connect()ed, maybe
> sleeping */
> - if (was && !sock_flag(req->friend, SOCK_DEAD))
> - sk->sk_state_change(req->friend);
> + if (was) {
> + if (!sock_flag(req->friend, SOCK_DEAD))
> +
> sk->sk_state_change(req->friend);
> + sock_put(was);
> + }
> }
> newsk->sk_state = TCP_SYN_RECV;
> diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
> index 5917485..7a63245 100644
> --- a/net/ipv4/tcp_output.c
> +++ b/net/ipv4/tcp_output.c
> @@ -2277,8 +2277,10 @@ struct sk_buff *tcp_make_synack(struct sock
> *sk, struct dst_entry *dst,
> memset(&opts, 0, sizeof(opts));
> /* Only try to make friends if enabled */
> - if (sysctl_tcp_friends)
> + if (sysctl_tcp_friends) {
> + sock_hold(sk);
> skb->friend = sk;
> + }
> #ifdef CONFIG_SYN_COOKIES
> if (unlikely(req->cookie_ts))
>
>
> And, our TCP friends v4? :)
>
> Thanks
>
> Yu
>
>
>
next prev parent reply other threads:[~2013-01-23 3:15 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-03 19:48 v3 for tcp friends? David Miller
2012-09-04 15:10 ` Bruce Curtis
2012-09-04 16:58 ` David Miller
2013-01-21 7:29 ` Li Yu
2013-01-21 7:31 ` Li Yu
[not found] ` <CA+WLrf8xuJ3UXK-tZQykuTPMXu67WsNKWNuNzBKk7MBAidW-CQ@mail.gmail.com>
[not found] ` <CAEkNxbH0MAAU9oiwwrFTbMJP1yzfNdxD5NDdZOqGnvhTLvodoQ@mail.gmail.com>
2013-01-23 3:14 ` Li Yu [this message]
2013-01-23 6:12 ` Li Yu
2013-01-23 6:46 ` Eric Dumazet
2013-01-23 7:21 ` Li Yu
2013-01-23 7:58 ` Li Yu
2013-01-23 9:39 ` Li Yu
2013-01-23 9:52 ` Li Yu
2013-01-23 13:16 ` Weiping Pan
2013-01-24 2:04 ` Xiaotian Feng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50FF5592.60008@gmail.com \
--to=raise.sail@gmail.com \
--cc=brutus@google.com \
--cc=davem@davemloft.net \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.