* ssh connections hanging on 4.1rc7
@ 2015-06-11 17:46 Dave Jones
2015-06-11 17:54 ` Dave Jones
2015-06-11 18:24 ` Eric Dumazet
0 siblings, 2 replies; 4+ messages in thread
From: Dave Jones @ 2015-06-11 17:46 UTC (permalink / raw)
To: netdev
Just hit this weird problem where I can ssh into a machine once,
then after logging out, subsequent ssh connections hang.
The client side looks like..
13:39:06.307781 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [S], seq 319726787, win 29200, options [mss 1460,sackOK,TS val 77408596 ecr 0,nop,wscale 7], length 0
13:39:06.308580 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 319726788:319726820, ack 1404309216, win 229, options [nop,nop,TS val 77408597 ecr 19368680], length 32
13:39:06.332551 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 32:2000, ack 33, win 229, options [nop,nop,TS val 77408621 ecr 19368703], length 1968
13:39:06.370042 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2048:2064, ack 1265, win 258, options [nop,nop,TS val 77408658 ecr 19368734], length 16
13:39:06.449511 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options [nop,nop,TS val 77408738 ecr 19368782], length 0
until I ctrl-c which then sends the FIN
13:40:06.914614 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [F.], seq 2108, ack 1309, win 258, options [nop,nop,TS val 77469203 ecr 19368782], length 0
On the sshd side..
13:39:06.311005 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [S], seq 319726787, win 29200, options [mss 1460,sackOK,TS val 77408596 ecr 0,nop,wscale 7], length 0
13:39:06.311254 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [.], ack 1404309216, win 229, options [nop,nop,TS val 77408596 ecr 19368680], length 0
13:39:06.311778 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 0:32, ack 1, win 229, options [nop,nop,TS val 77408597 ecr 19368680], length 32
13:39:06.334546 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [.], ack 33, win 229, options [nop,nop,TS val 77408619 ecr 19368703], length 0
13:39:06.335779 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 32:2000, ack 33, win 229, options [nop,nop,TS val 77408621 ecr 19368703], length 1968
13:39:06.347867 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2000:2048, ack 985, win 243, options [nop,nop,TS val 77408633 ecr 19368711], length 48
13:39:06.373232 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2048:2064, ack 1265, win 258, options [nop,nop,TS val 77408658 ecr 19368734], length 16
13:39:06.413173 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2064:2108, ack 1265, win 258, options [nop,nop,TS val 77408698 ecr 19368782], length 44
13:39:06.452714 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options [nop,nop,TS val 77408738 ecr 19368782], length 0
until the ctrl-c...
13:40:06.917807 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [F.], seq 2108, ack 1309, win 258, options [nop,nop,TS val 77469203 ecr 19368782], length 0
13:40:06.921241 IP wopr.kernelslacker.org.43982 > gelk.kernelslacker.org.ssh: Flags [.], ack 1310, win 258, options [nop,nop,TS val 77469206 ecr 19429290], length 0
Is there anything out of the ordinary here that I'm missing, or is this perhaps
an sshd bug ?
The odd thing is that it seems repeatable with every host I try to ssh to.
4.1rc7 on both sides, haven't tried reproducing with different kernels yet.
Dave
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ssh connections hanging on 4.1rc7
2015-06-11 17:46 ssh connections hanging on 4.1rc7 Dave Jones
@ 2015-06-11 17:54 ` Dave Jones
2015-06-11 18:24 ` Eric Dumazet
1 sibling, 0 replies; 4+ messages in thread
From: Dave Jones @ 2015-06-11 17:54 UTC (permalink / raw)
To: netdev
On Thu, Jun 11, 2015 at 01:46:18PM -0400, Dave Jones wrote:
> Just hit this weird problem where I can ssh into a machine once,
> then after logging out, subsequent ssh connections hang.
>
> The client side looks like..
derp, missed half the tcpdump capture on both sides, and now
I can't reproduce it. I'll resend if it happens again.
Dave
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ssh connections hanging on 4.1rc7
2015-06-11 17:46 ssh connections hanging on 4.1rc7 Dave Jones
2015-06-11 17:54 ` Dave Jones
@ 2015-06-11 18:24 ` Eric Dumazet
2015-06-11 19:39 ` Dave Jones
1 sibling, 1 reply; 4+ messages in thread
From: Eric Dumazet @ 2015-06-11 18:24 UTC (permalink / raw)
To: Dave Jones; +Cc: netdev
On Thu, 2015-06-11 at 13:46 -0400, Dave Jones wrote:
> Just hit this weird problem where I can ssh into a machine once,
> then after logging out, subsequent ssh connections hang.
>
> The client side looks like..
>
> 13:39:06.307781 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [S], seq 319726787, win 29200,
> options [mss 1460,sackOK,TS val 77408596 ecr 0,nop,wscale 7], length 0
> 13:39:06.308580 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 319726788:319726820, ack
> 1404309216, win 229, options [nop,nop,TS val 77408597 ecr 19368680],
> length 32
> 13:39:06.332551 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 32:2000, ack 33, win 229,
> options [nop,nop,TS val 77408621 ecr 19368703], length 1968
> 13:39:06.370042 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 2048:2064, ack 1265, win
> 258, options [nop,nop,TS val 77408658 ecr 19368734], length 16
> 13:39:06.449511 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options
> [nop,nop,TS val 77408738 ecr 19368782], length 0
>
> until I ctrl-c which then sends the FIN
>
> 13:40:06.914614 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [F.], seq 2108, ack 1309, win 258,
> options [nop,nop,TS val 77469203 ecr 19368782], length 0
>
>
> On the sshd side..
>
>
> 13:39:06.311005 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [S], seq 319726787, win 29200,
> options [mss 1460,sackOK,TS val 77408596 ecr 0,nop,wscale 7], length 0
> 13:39:06.311254 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [.], ack 1404309216, win 229,
> options [nop,nop,TS val 77408596 ecr 19368680], length 0
> 13:39:06.311778 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 0:32, ack 1, win 229,
> options [nop,nop,TS val 77408597 ecr 19368680], length 32
> 13:39:06.334546 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [.], ack 33, win 229, options
> [nop,nop,TS val 77408619 ecr 19368703], length 0
> 13:39:06.335779 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 32:2000, ack 33, win 229,
> options [nop,nop,TS val 77408621 ecr 19368703], length 1968
> 13:39:06.347867 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 2000:2048, ack 985, win
> 243, options [nop,nop,TS val 77408633 ecr 19368711], length 48
> 13:39:06.373232 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 2048:2064, ack 1265, win
> 258, options [nop,nop,TS val 77408658 ecr 19368734], length 16
> 13:39:06.413173 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [P.], seq 2064:2108, ack 1265, win
> 258, options [nop,nop,TS val 77408698 ecr 19368782], length 44
> 13:39:06.452714 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options
> [nop,nop,TS val 77408738 ecr 19368782], length 0
>
> until the ctrl-c...
>
> 13:40:06.917807 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [F.], seq 2108, ack 1309, win 258,
> options [nop,nop,TS val 77469203 ecr 19368782], length 0
> 13:40:06.921241 IP wopr.kernelslacker.org.43982 >
> gelk.kernelslacker.org.ssh: Flags [.], ack 1310, win 258, options
> [nop,nop,TS val 77469206 ecr 19429290], length 0
>
>
> Is there anything out of the ordinary here that I'm missing, or is
> this perhaps
> an sshd bug ?
>
> The odd thing is that it seems repeatable with every host I try to ssh
> to.
>
> 4.1rc7 on both sides, haven't tried reproducing with different kernels
> yet.
Your tcpdumps look one way only.
Could this be some routing or mtu problem ?
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ssh connections hanging on 4.1rc7
2015-06-11 18:24 ` Eric Dumazet
@ 2015-06-11 19:39 ` Dave Jones
0 siblings, 0 replies; 4+ messages in thread
From: Dave Jones @ 2015-06-11 19:39 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Thu, Jun 11, 2015 at 11:24:21AM -0700, Eric Dumazet wrote:
> > Just hit this weird problem where I can ssh into a machine once,
> > then after logging out, subsequent ssh connections hang.
>
> Your tcpdumps look one way only.
ok hit it again, so let's try again...
client side:
15:34:10.184012 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1505519347, win 229, options [nop,nop,TS val 84312472 ecr 26272557], length 0
15:34:10.210704 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 33, win 229, options [nop,nop,TS val 84312499 ecr 26272584], length 0
15:34:10.218178 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2000:2048, ack 985, win 243, options [nop,nop,TS val 84312506 ecr 26272587], length 48
15:34:10.248332 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2048:2064, ack 1265, win 258, options [nop,nop,TS val 84312536 ecr 26272614], length 16
15:34:10.288172 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [.], ack 2064, win 257, options [nop,nop,TS val 26272662 ecr 84312536], length 0
15:34:10.328516 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options [nop,nop,TS val 84312617 ecr 26272662], length 0
<ctrl-c>
15:34:27.693304 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1310, win 258, options [nop,nop,TS val 84329981 ecr 26290067], length 0
server side:
15:34:10.188611 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [S], seq 2139969123, win 29200, options [mss 1460,sackOK,TS val 84312472 ecr 0,nop,wscale 7], length 0
15:34:10.188777 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [S.], seq 1505519346, ack 2139969124, win 28960, options [mss 1460,sackOK,TS val 26272557 ecr 84312472,nop,wscale 7], length 0
15:34:10.189033 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1, win 229, options [nop,nop,TS val 84312472 ecr 26272557], length 0
15:34:10.189534 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 1:33, ack 1, win 229, options [nop,nop,TS val 84312473 ecr 26272557], length 32
15:34:10.189636 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [.], ack 33, win 227, options [nop,nop,TS val 26272558 ecr 84312473], length 0
15:34:10.215507 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [P.], seq 1:33, ack 33, win 227, options [nop,nop,TS val 26272584 ecr 84312473], length 32
15:34:10.215700 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 33, win 229, options [nop,nop,TS val 84312499 ecr 26272584], length 0
15:34:10.216834 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 33:2001, ack 33, win 229, options [nop,nop,TS val 84312500 ecr 26272584], length 1968
15:34:10.216938 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [.], ack 2001, win 257, options [nop,nop,TS val 26272585 ecr 84312500], length 0
15:34:10.218280 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [P.], seq 33:985, ack 2001, win 257, options [nop,nop,TS val 26272587 ecr 84312500], length 952
15:34:10.223192 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2001:2049, ack 985, win 243, options [nop,nop,TS val 84312506 ecr 26272587], length 48
15:34:10.245469 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [P.], seq 985:1265, ack 2049, win 257, options [nop,nop,TS val 26272614 ecr 84312506], length 280
15:34:10.253331 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2049:2065, ack 1265, win 258, options [nop,nop,TS val 84312536 ecr 26272614], length 16
15:34:10.293038 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [.], ack 2065, win 257, options [nop,nop,TS val 26272662 ecr 84312536], length 0
15:34:10.293210 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [P.], seq 2065:2109, ack 1265, win 258, options [nop,nop,TS val 84312576 ecr 26272662], length 44
15:34:10.293313 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [.], ack 2109, win 257, options [nop,nop,TS val 26272662 ecr 84312576], length 0
15:34:10.293487 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [P.], seq 1265:1309, ack 2109, win 257, options [nop,nop,TS val 26272662 ecr 84312576], length 44
15:34:10.333528 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1309, win 258, options [nop,nop,TS val 84312617 ecr 26272662], length 0
<ctrl-c>
15:34:27.693322 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [F.], seq 2109, ack 1309, win 258, options [nop,nop,TS val 84329976 ecr 26272662], length 0
15:34:27.698094 IP gelk.kernelslacker.org.ssh > wopr.kernelslacker.org.44268: Flags [F.], seq 1309, ack 2110, win 257, options [nop,nop,TS val 26290067 ecr 84329976], length 0
15:34:27.698293 IP wopr.kernelslacker.org.44268 > gelk.kernelslacker.org.ssh: Flags [.], ack 1310, win 258, options [nop,nop,TS val 84329981 ecr 26290067], length 0
> Could this be some routing or mtu problem ?
Same mtu on both, no fancy routing. Both on same 192.168.42.x subnet
plugged into the same dumb 1Gb switch
The only unusual thing is that the client side is running bonding (bond-mode 0)
I'll take that out of the equation to rule it out when I get home.
Dave
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-06-11 19:39 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-06-11 17:46 ssh connections hanging on 4.1rc7 Dave Jones
2015-06-11 17:54 ` Dave Jones
2015-06-11 18:24 ` Eric Dumazet
2015-06-11 19:39 ` Dave Jones
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).