FTP Packet Mangling & NAT

FTP Packet Mangling & NAT

[Nadeem Douba]

Hi all,

I have an interesting scenario where I have a transparent proxy that
does some funny things with PASSIVE FTP communications that is making
me think of developing a netfilters module to correct the behavior.
What is going on is the following:

1. Client establishes connection to FTP server s1.s2.s3.s4 via
transparent proxy (where s1, s2, s3, s4 is the first, second, third,
and fourth octet of the IPv4 address for the remote server,
respectively)
2. Client sends PASV command to FTP server indicating that the client
will be using a PASSIVE FTP connection for its data channel
3. Server responds with '227 Entering Passive Mode
(tp1,tp2,tp3,tp4,p1,p2).' (where t1, t2, t3, t4 is the first, second,
third, and fourth octet of the IPv4 address for the transparent proxy,
respectively. Where p1 and p2 are the higher and lower order octets
for the remote TCP port, respectively)

In step three, the transparent proxy modifies the server's original
response to the PASV command by translating what should have been '227
Entering Passive Mode (s1,s2,s3,s4,p1,p2).' to '227 Entering Passive
Mode (tp1,tp2,tp3,tp4,p1,p2).'. Some clients (like IE and Firefox)
don't like this at all and this results in a broken FTP session.

This is where a custom netfilters module would come in handy. I'd like
to be able to "untranslate" the server's response back into it's
original form. However, the transparent proxy expects to broker the
passive data channel at the same time, so simply fixing the response
would break the communications to and from the FTP server. Therefore,
in addition to fixing the server's response, I need to do some NAT'ing
on all data channel packets. I was wondering if anybody could help
point me in the right direction with regards to developing such a
module. The flow would be as follows:

1. module detects PASV response '227 Entering Passive Mode
(tp1,tp2,tp3,tp4,p1,p2).'
2. module records tp1, tp2, tp3, tp4, p1, p2 as well as the server's
remote IP s1, s2, s3, s4 for future NAT'ing.
3. module alters the payload to '227 Entering Passive Mode
(s1,s2,s3,s4,p1,p2).' and forwards packet to client.
4. module detects RELATED data channel being established.
5. module performs destination NAT to change packet destination from
s1,s2,s3,s4 to tp1,tp2,tp3,tp4
6. module performs source NAT to change packet source from
tp1,tp2,tp3,tp4 to s1,s2,s3,s4

I've read over the netfilters hacking HowTo and some of the modules
like ip_nat_ftp.c and ip_conntrack_ftp.c but I'm sure how to tie all
these concepts together to develop my solution. I was wondering if any
of the netfilter gurus could help point me in the right direction with
an overall strategy of how to develop this solution.

Your help would be greatly appreciated!

Thanks,

Nadeem
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FTP Packet Mangling & NAT

[Amos Jeffries]

 On Wed, 8 Jun 2011 16:18:11 -0400, Nadeem Douba wrote:
> Hi all,
>
> I have an interesting scenario where I have a transparent proxy that
> does some funny things with PASSIVE FTP communications that is making
> me think of developing a netfilters module to correct the behavior.
> What is going on is the following:
>
> 1. Client establishes connection to FTP server s1.s2.s3.s4 via
> transparent proxy (where s1, s2, s3, s4 is the first, second, third,
> and fourth octet of the IPv4 address for the remote server,
> respectively)
> 2. Client sends PASV command to FTP server indicating that the client
> will be using a PASSIVE FTP connection for its data channel
> 3. Server responds with '227 Entering Passive Mode
> (tp1,tp2,tp3,tp4,p1,p2).' (where t1, t2, t3, t4 is the first, second,
> third, and fourth octet of the IPv4 address for the transparent 
> proxy,
> respectively. Where p1 and p2 are the higher and lower order octets
> for the remote TCP port, respectively)
>
> In step three, the transparent proxy modifies the server's original
> response to the PASV command by translating what should have been 
> '227
> Entering Passive Mode (s1,s2,s3,s4,p1,p2).' to '227 Entering Passive
> Mode (tp1,tp2,tp3,tp4,p1,p2).'. Some clients (like IE and Firefox)
> don't like this at all and this results in a broken FTP session.

 The proxy completely controls what the client gets given. It is 
 breaking transparency by not reverse-mapping the response properly. 
 Mention it to the authors and get it fixed.

>
> This is where a custom netfilters module would come in handy. I'd 
> like
> to be able to "untranslate" the server's response back into it's
> original form. However, the transparent proxy expects to broker the
> passive data channel at the same time, so simply fixing the response
> would break the communications to and from the FTP server. Therefore,
> in addition to fixing the server's response, I need to do some 
> NAT'ing
> on all data channel packets. I was wondering if anybody could help
> point me in the right direction with regards to developing such a
> module. The flow would be as follows:
>
> 1. module detects PASV response '227 Entering Passive Mode
> (tp1,tp2,tp3,tp4,p1,p2).'
> 2. module records tp1, tp2, tp3, tp4, p1, p2 as well as the server's
> remote IP s1, s2, s3, s4 for future NAT'ing.
> 3. module alters the payload to '227 Entering Passive Mode
> (s1,s2,s3,s4,p1,p2).' and forwards packet to client.
> 4. module detects RELATED data channel being established.
> 5. module performs destination NAT to change packet destination from
> s1,s2,s3,s4 to tp1,tp2,tp3,tp4
> 6. module performs source NAT to change packet source from
> tp1,tp2,tp3,tp4 to s1,s2,s3,s4

 A transparent proxy is equivalent to a NAT module in all respects. 
 Merely operates on the FTP layer in this case.
 Switch "module" for "proxy" and you have a good plan for fixing the 
 bug.

 NOTE: If you do steps 5 and 6 in the IP layer you will be bypassing the 
 proxy data handling and void all your reasons for having it done by a 
 proxy in the first place. Instead of by the conntrack NAT module for FTP 
 for both outgoing and returning traffic.

>
> I've read over the netfilters hacking HowTo and some of the modules
> like ip_nat_ftp.c and ip_conntrack_ftp.c but I'm sure how to tie all
> these concepts together to develop my solution. I was wondering if 
> any
> of the netfilter gurus could help point me in the right direction 
> with
> an overall strategy of how to develop this solution.

 Disclaimer: I'm not one of the NF gurus. Just a proxy guy.

 AYJ
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FTP Packet Mangling & NAT

[Jan Engelhardt]

On Thursday 2011-06-09 05:20, Amos Jeffries wrote:
>
> A transparent proxy is equivalent to a NAT module in all respects.

Not quite. :)  nf_nat does not consume a local TCB - though it
does consume a port - whereas a classical proxy's connections
(e.g. squid) can show up in `ss -p`.

Re: FTP Packet Mangling & NAT

[Nadeem Douba]

>  The proxy completely controls what the client gets given. It is
>  breaking transparency by not reverse-mapping the response properly.
>  Mention it to the authors and get it fixed.

The authors (Websense) are of a different opinion of how this issue is
fixed. They recommended that we enable active FTP (significant
security risk).


>  A transparent proxy is equivalent to a NAT module in all respects.
>  Merely operates on the FTP layer in this case.
>  Switch "module" for "proxy" and you have a good plan for fixing the
>  bug.

Not really, even proxying these requests would result in the same
issue as the child proxy would relay the parent proxy's response.

>  NOTE: If you do steps 5 and 6 in the IP layer you will be bypassing the
>  proxy data handling and void all your reasons for having it done by a
>  proxy in the first place. Instead of by the conntrack NAT module for FTP
>  for both outgoing and returning traffic.

The point is not to get rid of the proxy but to fix the issue that the
proxy is presenting to all FTP communications.

>  Disclaimer: I'm not one of the NF gurus. Just a proxy guy.

Thanks anyways.

Re: FTP Packet Mangling & NAT

[Amos Jeffries]

On 10/06/11 07:06, Nadeem Douba wrote:
>>   The proxy completely controls what the client gets given. It is
>>   breaking transparency by not reverse-mapping the response properly.
>>   Mention it to the authors and get it fixed.
>
> The authors (Websense) are of a different opinion of how this issue is
> fixed. They recommended that we enable active FTP (significant
> security risk).
>

Strange opinion. PORT needs the same types of handing in a relay/proxy 
as PASV does. Only the control channel syntax is different.
  Perhapse they meant websense accepts active FTP 
(client-ACTIVE->proxy-PASV->server) or something like that.


As I see it they have two scenarios to pick from:

a) opening relay data connection themselves and setting that as the PASV 
advertised port. Then "manually" as it were shuffling the bytes back and 
forward there. Being websense and wanting to see inside the data this is 
probably their best fix.

b) sending some port mapping details into netfilter indicating that 
packets from server IP:port are going to be related to client IP:port.

>
>>   A transparent proxy is equivalent to a NAT module in all respects.
>>   Merely operates on the FTP layer in this case.
>>   Switch "module" for "proxy" and you have a good plan for fixing the
>>   bug.
>
> Not really, even proxying these requests would result in the same
> issue as the child proxy would relay the parent proxy's response.

huh? "even proxying these requests". The whole point of your original 
post was that they *are* proxied, and the proxy is failing to NAPT the 
control channel syntax on the reply.

A->B->C->D chaining of relays does not matter. Each relay hop can, and 
must, do its own NAPT.


Just to clarify; what I read you asking about was this scenario:

   When layer-7 software (proxy) does a NAPT mapping of text syntax 
IP1:port1 -> IP2:port2 sent out on IP3:port3 (proxy->server control 
control).
   How to implement a layer-2 software (netfilter) to figure out this 
mapping and to do the reverse IP2:port2 -> IP1:port1 mapping on the text 
data inside the TCP packet body of packets returning on IP4:port4 
(client->proxy control channel).
So the packet flow is:
   control: client--nf_nat (DNAT/REDIRECT)-->proxy->server
   data:    client<--nf_nat (SNAT)----server

Keep in mind that websense proxy terminates a TCP connection and 
generates a completely separate independent TCP connection from itself 
to the source server. Whether that is FTP origin or other proxy.

  So that the client->proxy information is in the INPUT tables.  The 
server->proxy response is in the OUTPUT tables.  And NAT is done in a 
third place the proxy normally does not have write access to.

  So to me those factors all mean layer-2 where netfilter modules 
operate is a very hard place to figure out what the correct mapping 
should be.

(Jan, please correct me if you have some advanced magic voodoo that lets 
INPUT and OUTPUT chain connections link as RELATED state)


Things get particularly nasty when, as you can see from that very first 
comment I made above, the connections between proxy and server can 
differ in their active/passive mode. Things like that happen at layer-7, 
on some developers whim.


>
>>   NOTE: If you do steps 5 and 6 in the IP layer you will be bypassing the
>>   proxy data handling and void all your reasons for having it done by a
>>   proxy in the first place. Instead of by the conntrack NAT module for FTP
>>   for both outgoing and returning traffic.
>
> The point is not to get rid of the proxy but to fix the issue that the
> proxy is presenting to all FTP communications.

Which was the gist of my point.


AYJ

Re: FTP Packet Mangling & NAT

[Jan Engelhardt]

On Friday 2011-06-10 11:26, Amos Jeffries wrote:
>
>Just to clarify; what I read you asking about was this scenario:
>When layer-7 software (proxy) does a NAPT mapping of text syntax
>IP1:port1 -> IP2:port2 sent out on IP3:port3 (proxy->server control
>control).
>  How to implement a layer-2 software (netfilter) to figure out this mapping
>and to do the reverse IP2:port2 -> IP1:port1 mapping on the text data inside
>the TCP packet body of packets returning on IP4:port4 (client->proxy control
>channel).
>So the packet flow is:
> control: client--nf_nat (DNAT/REDIRECT)-->proxy->server
> data:    client<--nf_nat (SNAT)----server
>
>Keep in mind that websense proxy terminates a TCP connection and generates a
>completely separate independent TCP connection from itself to the source
>server. Whether that is FTP origin or other proxy.
>
>So that the client->proxy information is in the INPUT tables.  The
>server->proxy response is in the OUTPUT tables.  And NAT is done in a third
>place the proxy normally does not have write access to.
>
>So to me those factors all mean layer-2 where netfilter modules operate is a
>very hard place to figure out what the correct mapping should be.

tl;dr. FTP has nothing to do with L2.

>(Jan, please correct me if you have some advanced magic voodoo that lets INPUT
>and OUTPUT chain connections link as RELATED state)

Why should it, your proxy seems to be creating a new set of logically
dinstinct connections.