How I can reset TCP sockets after long suspend/resume cyscle

How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

Hi,

Is there a way to close all TCP sockets before/after suspend to ram?

It is so annoying that after a long suspend/resume cycle all applications
still wait for long closed connections.

Kmail/imap is the worst example here.

I tried to disable network interace, I tried even to remove its module
with no luck.

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

Maxim Levitsky wrote:
> Hi,
> 
> Is there a way to close all TCP sockets before/after suspend to ram?
> 
> It is so annoying that after a long suspend/resume cycle all applications
> still wait for long closed connections.
> 
> Kmail/imap is the worst example here.
> 
> I tried to disable network interace, I tried even to remove its module
> with no luck.
> 
> Best regards,
> 	Maxim Levitsky


Anybody knows about this?

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Adam Langley]

On Wed, Jun 4, 2008 at 8:34 AM, Maxim Levitsky <maximlevitsky@gmail.com> wrote:
>> Is there a way to close all TCP sockets before/after suspend to ram?

As with most things, you should consider how this can be done from
userspace first.

You can find the processes with TCP connections open by walking
/proc/*/fd and readlink()ing the dents therein. Then you can match the
inode numbers up with /proc/net/tcp to see if the given TCP connection
is remote or not.

Now you want to kill those connections somehow. You could imagine
doing it by injecting RST packets back into the kernel, but for that
you would need to know the SEQ/ACK numbers for the connection. Since
that's sensitive information, /proc/net/tcp doesn't carry it. It would
have to be CAP_NET_ADMIN (read: root user) only and changing the
formats of proc files based on the reading user is a no-no. So that
would require another proc file; I've no idea how well that patch
would be received.

Another option would be to close the TCP connections from within the
processes which have them. You could enumerate the processes, ptrace
attach each one, wait() for SIGSTOP, get the current instr pointer and
patch in some code to close the fds then unpatch the process and let
it continue. That would be architecture specific, of couse.

When the process comes to reading/selecting the fds again it would get
a 0 read and act like they had been closed.

I'll admit that neither solution is terribly wonderful.


AGL

-- 
Adam Langley agl@imperialviolet.org http://www.imperialviolet.org

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Stephen Hemminger]

On Wed, 4 Jun 2008 09:09:29 -0700
"Adam Langley" <agl@imperialviolet.org> wrote:

> On Wed, Jun 4, 2008 at 8:34 AM, Maxim Levitsky <maximlevitsky@gmail.com> wrote:
> >> Is there a way to close all TCP sockets before/after suspend to ram?
> 
> As with most things, you should consider how this can be done from
> userspace first.
> 
> You can find the processes with TCP connections open by walking
> /proc/*/fd and readlink()ing the dents therein. Then you can match the
> inode numbers up with /proc/net/tcp to see if the given TCP connection
> is remote or not.
> 
> Now you want to kill those connections somehow. You could imagine
> doing it by injecting RST packets back into the kernel, but for that
> you would need to know the SEQ/ACK numbers for the connection. Since
> that's sensitive information, /proc/net/tcp doesn't carry it. It would
> have to be CAP_NET_ADMIN (read: root user) only and changing the
> formats of proc files based on the reading user is a no-no. So that
> would require another proc file; I've no idea how well that patch
> would be received.
> 
> Another option would be to close the TCP connections from within the
> processes which have them. You could enumerate the processes, ptrace
> attach each one, wait() for SIGSTOP, get the current instr pointer and
> patch in some code to close the fds then unpatch the process and let
> it continue. That would be architecture specific, of couse.
> 
> When the process comes to reading/selecting the fds again it would get
> a 0 read and act like they had been closed.
> 
> I'll admit that neither solution is terribly wonderful.
> 
> 
> AGL

TCP should recover from this anyway. As soon as any activity happens
on the connection the other end will respond with reset.  For an IMAP
session doing the next get-mail should cause a connection reset.
Now it is possible that mail client has other problems.

> 

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

> TCP should recover from this anyway. As soon as any activity happens
> on the connection the other end will respond with reset.  For an IMAP
> session doing the next get-mail should cause a connection reset.
> Now it is possible that mail client has other problems.
> 

I will check this again, but this isn't specific to imap,
I remember that leaving a running download results is same hang behavior.
(I agree that after a bit long timeout (about 1~2 minutes) connection is 
reset, but this is annoying, especially if you turned up your system to 
read mail.)

And exiting kmail won't help, since it does mail i/o via kio ioslave
and thus I have to ether wait 2 minutes or kill this ioslave.

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Rick Jones]

While I know of at least one OS/stack which allows a root user to issue 
a command to abort a TCP connection, (and it may be two given their 
shared history) its use was always strongly discouraged.

> TCP should recover from this anyway. As soon as any activity happens
> on the connection the other end will respond with reset.  For an IMAP
> session doing the next get-mail should cause a connection reset.
> Now it is possible that mail client has other problems.

And one way for there to be activity would be for the application(s) in 
question to have an application-layer keepalive mechanism, or at the 
very least, set SO_KEEPALIVE on their connections.  And if they don't 
want to rely on the sysadmin to have set what they consider a 
"reasonable" value for when to start sending keepalive probes, a 
TCP_KEEPIDLE setsockopt() perhaps.

rick jones

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

Adam Langley wrote:
> On Wed, Jun 4, 2008 at 8:34 AM, Maxim Levitsky <maximlevitsky@gmail.com> wrote:
>>> Is there a way to close all TCP sockets before/after suspend to ram?
> 
> As with most things, you should consider how this can be done from
> userspace first.
> 
> You can find the processes with TCP connections open by walking
> /proc/*/fd and readlink()ing the dents therein. Then you can match the
> inode numbers up with /proc/net/tcp to see if the given TCP connection
> is remote or not.
> 
> Now you want to kill those connections somehow. You could imagine
> doing it by injecting RST packets back into the kernel, but for that
> you would need to know the SEQ/ACK numbers for the connection. Since
> that's sensitive information, /proc/net/tcp doesn't carry it. It would
> have to be CAP_NET_ADMIN (read: root user) only and changing the
> formats of proc files based on the reading user is a no-no. So that
> would require another proc file; I've no idea how well that patch
> would be received.
This isn't a problem, since suspend/resume is done by root.
I was thinking about something like that, and thought that it is 
implemented, so I asked here.

So small question, as a root, I can get SEQ/ACK numbers of a connection?

I am thinking, that maybe such thing can be put in kernel (as optional 
feature)

I can even add a suspend timeout, so if suspend was longer that it, then 
reset the sockets, otherwise not.
setting the timeout to 0 (default) will disable the feature.


> 
> Another option would be to close the TCP connections from within the
> processes which have them. You could enumerate the processes, ptrace
> attach each one, wait() for SIGSTOP, get the current instr pointer and
> patch in some code to close the fds then unpatch the process and let
> it continue. That would be architecture specific, of couse.
> 
> When the process comes to reading/selecting the fds again it would get
> a 0 read and act like they had been closed.
> 
> I'll admit that neither solution is terribly wonderful.
> 
> 
> AGL
> 

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Stephen Hemminger]

On Wed, 04 Jun 2008 23:52:54 +0300
Maxim Levitsky <maximlevitsky@gmail.com> wrote:

> Adam Langley wrote:
> > On Wed, Jun 4, 2008 at 8:34 AM, Maxim Levitsky <maximlevitsky@gmail.com> wrote:
> >>> Is there a way to close all TCP sockets before/after suspend to ram?
> > 
> > As with most things, you should consider how this can be done from
> > userspace first.
> > 
> > You can find the processes with TCP connections open by walking
> > /proc/*/fd and readlink()ing the dents therein. Then you can match the
> > inode numbers up with /proc/net/tcp to see if the given TCP connection
> > is remote or not.
> > 
> > Now you want to kill those connections somehow. You could imagine
> > doing it by injecting RST packets back into the kernel, but for that
> > you would need to know the SEQ/ACK numbers for the connection. Since
> > that's sensitive information, /proc/net/tcp doesn't carry it. It would
> > have to be CAP_NET_ADMIN (read: root user) only and changing the
> > formats of proc files based on the reading user is a no-no. So that
> > would require another proc file; I've no idea how well that patch
> > would be received.
> This isn't a problem, since suspend/resume is done by root.
> I was thinking about something like that, and thought that it is 
> implemented, so I asked here.
> 
> So small question, as a root, I can get SEQ/ACK numbers of a connection?
> 
> I am thinking, that maybe such thing can be put in kernel (as optional 
> feature)
> 
> I can even add a suspend timeout, so if suspend was longer that it, then 
> reset the sockets, otherwise not.
> setting the timeout to 0 (default) will disable the feature.
> 
> 
> > 
> > Another option would be to close the TCP connections from within the
> > processes which have them. You could enumerate the processes, ptrace
> > attach each one, wait() for SIGSTOP, get the current instr pointer and
> > patch in some code to close the fds then unpatch the process and let
> > it continue. That would be architecture specific, of couse.
> > 
> > When the process comes to reading/selecting the fds again it would get
> > a 0 read and act like they had been closed.
> > 
> > I'll admit that neither solution is terribly wonderful.
> > 
> > 
> > AGL
> > 
> 
> Best regards,
> 	Maxim Levitsky

Another idea would be for the kernel to prematurely run the keepalive
timer for all TCP connections after resume. The keepalive code would send
an ACK (out of window) which causes other side to send a RST or ACK.

It is normal behaviour, the change would just make the timer happen sooner.

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Rick Jones]

> Another idea would be for the kernel to prematurely run the keepalive
> timer for all TCP connections after resume. The keepalive code would send
> an ACK (out of window) which causes other side to send a RST or ACK.
> 
> It is normal behaviour, the change would just make the timer happen sooner.

Would you have it done even for those endpoints on which SO_KEEPALIVE 
wasn't set?

rick jones

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

On Thursday, 5 June 2008 00:23:37 Rick Jones wrote:
> > Another idea would be for the kernel to prematurely run the keepalive
> > timer for all TCP connections after resume. The keepalive code would send
> > an ACK (out of window) which causes other side to send a RST or ACK.
> > 
> > It is normal behaviour, the change would just make the timer happen sooner.
> 
> Would you have it done even for those endpoints on which SO_KEEPALIVE 
> wasn't set?

Why not?

This seems the right solution,
if this is possible, why not to check for stuck TCP connections
on resume?

Obviously, we should be able to turn this off, for some reasons, maybe some
weird servers, but generally this can be the default.

> 
> rick jones
> 

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Rick Jones]

Maxim Levitsky wrote:
> On Thursday, 5 June 2008 00:23:37 Rick Jones wrote:
>>Would you have it done even for those endpoints on which SO_KEEPALIVE 
>>wasn't set?
> 
> 
> Why not?

Apart from just general paranoia?  Nothing besides perhaps spite for 
having to cover the backside of an application which on the surface at 
least does not appear to have attempted to make itself suspend/resume 
friendly :)  Although how TCP could tell an application with its own 
keepalive vs one that simply didn't bother I'm not at all sure.

> This seems the right solution, if this is possible, why not to check
> for stuck TCP connections on resume?

The prospect is perhaps far-fetched, but the application could be in 
communication with another laptop, also in a long suspend, and 
initiating keepalive when the application did not request it could 
trigger a connection drop it was not otherwise expecting after the N 
keepalive probes were not answered (because the other end was 
suspended).  Of course, ostensibly such a connection should already be 
dealing with connection drop but still...

rick jones

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Benny Amorsen]

Maxim Levitsky <maximlevitsky@gmail.com> writes:

> This seems the right solution,
> if this is possible, why not to check for stuck TCP connections
> on resume?

Imagine a laptop on wireless going into suspend for a few minutes,
then getting woken up. If you're lucky today, the TCP connections
survive.

If you check for stuck TCP connections right after resume, you could
have problems because the wireless hasn't negotiated encryption keys
at that point. Isn't that a concern?


/Benny

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Nadejda Levitsky]

On Friday, 6 June 2008 11:37:12 Benny Amorsen wrote:
> Maxim Levitsky <maximlevitsky@gmail.com> writes:
> 
> > This seems the right solution,
> > if this is possible, why not to check for stuck TCP connections
> > on resume?
> 
> Imagine a laptop on wireless going into suspend for a few minutes,
> then getting woken up. If you're lucky today, the TCP connections
> survive.
It depends, I for example don't care about TCP connecions survival during suspend
I would even like to close them on suspend. (I usially use web browser, chat client, mail
maybe a download running in background which can be resumed)

Like I said such feature can be user configured.
There are uses like hibernating your server to change power supply, and still save all
TCP connections.



> 
> If you check for stuck TCP connections right after resume, you could
> have problems because the wireless hasn't negotiated encryption keys
> at that point. Isn't that a concern?

Good point, I haven't thought about this.
This means that a in-kernel solution is no-go, but userland solution
can check for such connections _after_ negotiation of wireless keys.

Or, just close all remote TCP connections if the suspend was long enough.



> 
> 
> /Benny
> 

Best regards,
	Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Evgeniy Polyakov]

On Sun, Jun 01, 2008 at 03:15:14PM +0300, Maxim Levitsky (maximlevitsky@gmail.com) wrote:
> Is there a way to close all TCP sockets before/after suspend to ram?

You can do that with frevoke patchset (not in mainline though iirc),
which allows you to close any file descriptor, which you can find via
/proc/$pid/fd

-- 
	Evgeniy Polyakov

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Maxim Levitsky]

On Thursday, 5 June 2008 10:13:40 Evgeniy Polyakov wrote:
> On Sun, Jun 01, 2008 at 03:15:14PM +0300, Maxim Levitsky (maximlevitsky@gmail.com) wrote:
> > Is there a way to close all TCP sockets before/after suspend to ram?
> 
> You can do that with frevoke patchset (not in mainline though iirc),
> which allows you to close any file descriptor, which you can find via
> /proc/$pid/fd
> 

Seems like a great idea.
Do you know the status of this patchset, when to expect it to be
in included in mainline?


Thanks a lot,
	Best regards,
		Maxim Levitsky

Re: How I can reset TCP sockets after long suspend/resume cyscle

[Evgeniy Polyakov]

Hi.

On Fri, Jun 06, 2008 at 12:19:17AM +0300, Maxim Levitsky (maximlevitsky@gmail.com) wrote:
> 
> Seems like a great idea.
> Do you know the status of this patchset, when to expect it to be
> in included in mainline?

It was quite active not that long ago IIRC, but seems to be abandoned
now, at least I did not see it in fsdevel, where it was copied before.
Idea is very interesting indeed, so I think it has to be moved forward,
so you should contact original author, so it got some progress, or push
it upstream yourself otherwise.

-- 
	Evgeniy Polyakov