Finding a hidden bound TCP socket

Finding a hidden bound TCP socket

[G. D. Fuego]

Hello,

I have a question about an odd linux networking behavior, that could
potentially be a local networking DoS.  I'm curious if anyone is
familiar with this behavior.

Essentially I was assisting someone with tracking down a hidden tcp
connection.  Attempts to bind to a particular port were failing,
saying the socket was in use, even though netstat was not reporting
any sort of connection.  They were initially suspecting a root kit,
but after a bit of digging, I came across this page:

http://dcid.me/2007/06/hidden-ports-on-linux/

>From the page:

"Here is the idea. If you get this simple C program, it will attempt
to bind every TCP port from 1025 to 1050, but it will not listen on
them. After it is done, if you do a netstat (or fuser or lsof) nothing
will be shown. However, if you try to use the port, you will get an
error saying that it is already in use."

I tested it out and confirmed that connections opened by their test
program do in fact cause the port to be unavailable for use, and they
are not reported in netstat, lsof, ss, or any other networking tools
that I tried.  I'm unable to to find any references to the ports being
in use anywhere I've looked within /proc.  Are you aware of another
way to figure out which process may be bound to the port?  In our
case, we figured out via trial and error which software was likely
triggering this behavior.

It seems to me that this could be a potentially interesting local
networking DoS.  By binding to all ephemeral ports in this way, you'd
prevent the local system from being able to establish any tcp
connections, and it would be a pain to figure out which process was
causing the problem.

My lame attempts to exploit this failed due to a max file descriptor
limit, but I'm told this could be doable by forking more processes for
doing the binding.

Is this behavior known/expected?

Re: Finding a hidden bound TCP socket

[richard -rw- weinberger]

On Tue, Nov 15, 2011 at 8:21 PM, G. D. Fuego <gdfuego@gmail.com> wrote:
> Hello,
>
> I have a question about an odd linux networking behavior, that could
> potentially be a local networking DoS.  I'm curious if anyone is
> familiar with this behavior.
>
> Essentially I was assisting someone with tracking down a hidden tcp
> connection.  Attempts to bind to a particular port were failing,
> saying the socket was in use, even though netstat was not reporting
> any sort of connection.  They were initially suspecting a root kit,
> but after a bit of digging, I came across this page:
>
> http://dcid.me/2007/06/hidden-ports-on-linux/
>
> From the page:
>
> "Here is the idea. If you get this simple C program, it will attempt
> to bind every TCP port from 1025 to 1050, but it will not listen on
> them. After it is done, if you do a netstat (or fuser or lsof) nothing
> will be shown. However, if you try to use the port, you will get an
> error saying that it is already in use."
>
> I tested it out and confirmed that connections opened by their test
> program do in fact cause the port to be unavailable for use, and they
> are not reported in netstat, lsof, ss, or any other networking tools
> that I tried.  I'm unable to to find any references to the ports being
> in use anywhere I've looked within /proc.  Are you aware of another
> way to figure out which process may be bound to the port?  In our
> case, we figured out via trial and error which software was likely
> triggering this behavior.
>
> It seems to me that this could be a potentially interesting local
> networking DoS.  By binding to all ephemeral ports in this way, you'd
> prevent the local system from being able to establish any tcp
> connections, and it would be a pain to figure out which process was
> causing the problem.
>
> My lame attempts to exploit this failed due to a max file descriptor
> limit, but I'm told this could be doable by forking more processes for
> doing the binding.
>
> Is this behavior known/expected?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

CC'ing netdev

-- 
Thanks,
//richard

Re: Finding a hidden bound TCP socket

[G. D. Fuego]

Any comments?  The behavior seems broken.  At the very least its very
inconsistent with other Unixes.

On Tue, Nov 15, 2011 at 3:23 PM, richard -rw- weinberger
<richard.weinberger@gmail.com> wrote:
> On Tue, Nov 15, 2011 at 8:21 PM, G. D. Fuego <gdfuego@gmail.com> wrote:
>> Hello,
>>
>> I have a question about an odd linux networking behavior, that could
>> potentially be a local networking DoS.  I'm curious if anyone is
>> familiar with this behavior.
>>
>> Essentially I was assisting someone with tracking down a hidden tcp
>> connection.  Attempts to bind to a particular port were failing,
>> saying the socket was in use, even though netstat was not reporting
>> any sort of connection.  They were initially suspecting a root kit,
>> but after a bit of digging, I came across this page:
>>
>> http://dcid.me/2007/06/hidden-ports-on-linux/
>>
>> From the page:
>>
>> "Here is the idea. If you get this simple C program, it will attempt
>> to bind every TCP port from 1025 to 1050, but it will not listen on
>> them. After it is done, if you do a netstat (or fuser or lsof) nothing
>> will be shown. However, if you try to use the port, you will get an
>> error saying that it is already in use."
>>
>> I tested it out and confirmed that connections opened by their test
>> program do in fact cause the port to be unavailable for use, and they
>> are not reported in netstat, lsof, ss, or any other networking tools
>> that I tried.  I'm unable to to find any references to the ports being
>> in use anywhere I've looked within /proc.  Are you aware of another
>> way to figure out which process may be bound to the port?  In our
>> case, we figured out via trial and error which software was likely
>> triggering this behavior.
>>
>> It seems to me that this could be a potentially interesting local
>> networking DoS.  By binding to all ephemeral ports in this way, you'd
>> prevent the local system from being able to establish any tcp
>> connections, and it would be a pain to figure out which process was
>> causing the problem.
>>
>> My lame attempts to exploit this failed due to a max file descriptor
>> limit, but I'm told this could be doable by forking more processes for
>> doing the binding.
>>
>> Is this behavior known/expected?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
>
> CC'ing netdev
>
> --
> Thanks,
> //richard
>

Re: Finding a hidden bound TCP socket

[Eric Dumazet]

Le mercredi 23 novembre 2011 à 15:27 -0500, G. D. Fuego a écrit :
> Any comments?  The behavior seems broken.  At the very least its very
> inconsistent with other Unixes.

Feel free to send a patch ;)

A user/process can consume all ports anyway, the 65000 port range is so
small...

Re: Finding a hidden bound TCP socket

[David Miller]

From: "G. D. Fuego" <gdfuego@gmail.com>
Date: Wed, 23 Nov 2011 15:27:33 -0500

> Any comments?  The behavior seems broken.  At the very least its very
> inconsistent with other Unixes.

Until the socket has a full final tuple it is bound to, there is no
reason to list it.

No UNIX lists a socket which is partially bound and hasn't either
performed a listen() or a connect().

Nobody has any comments because nobody thinks it's worth worrying about.

Re: Finding a hidden bound TCP socket

[Rick Jones]

On 11/23/2011 01:01 PM, David Miller wrote:
> From: "G. D. Fuego"<gdfuego@gmail.com>
> Date: Wed, 23 Nov 2011 15:27:33 -0500
>
>> Any comments?  The behavior seems broken.  At the very least its very
>> inconsistent with other Unixes.
>
> Until the socket has a full final tuple it is bound to, there is no
> reason to list it.
>
> No UNIX lists a socket which is partially bound and hasn't either
> performed a listen() or a connect().

Well....  I took the .c file mentioned previously, and compiled it on a 
Solaris 10 8/11 instance.  The 25-odd sockets it created *were* listed 
in the output of netstat -an -- local address as *.<portnum> remote 
address as *.* and a state of "BOUND."

A FreeBSD  (rev 8 IIRC) netstat -an seems to display them in a state of 
"CLOSED."  I didn't check HP-UX 11i v3 or AIX 6.

rick jones

Re: Finding a hidden bound TCP socket

[richard -rw- weinberger]

On Wed, Nov 23, 2011 at 10:38 PM, Rick Jones <rick.jones2@hp.com> wrote:
> On 11/23/2011 01:01 PM, David Miller wrote:
>>
>> From: "G. D. Fuego"<gdfuego@gmail.com>
>> Date: Wed, 23 Nov 2011 15:27:33 -0500
>>
>>> Any comments?  The behavior seems broken.  At the very least its very
>>> inconsistent with other Unixes.
>>
>> Until the socket has a full final tuple it is bound to, there is no
>> reason to list it.
>>
>> No UNIX lists a socket which is partially bound and hasn't either
>> performed a listen() or a connect().
>
> Well....  I took the .c file mentioned previously, and compiled it on a
> Solaris 10 8/11 instance.  The 25-odd sockets it created *were* listed in
> the output of netstat -an -- local address as *.<portnum> remote address as
> *.* and a state of "BOUND."
>
> A FreeBSD  (rev 8 IIRC) netstat -an seems to display them in a state of
> "CLOSED."  I didn't check HP-UX 11i v3 or AIX 6.
>

IRIX (6.5) shows them as "CLOSED".

-- 
Thanks,
//richard

Re: Finding a hidden bound TCP socket

[Eric Dumazet]

Le jeudi 24 novembre 2011 à 22:31 +0100, richard -rw- weinberger a
écrit :

> IRIX (6.5) shows them as "CLOSED".
> 

As I said, patches are welcome.