conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

[Frank Bulk]

[-- Attachment #1: Type: text/plain, Size: 2642 bytes --]

I'm working with the tech support of a residential DSL modem vendor on a NAT
issue.  The vendor says they use open source in their products and the logs
show all the typical iptables entries.  

We're using a DSL modem for OOB management of our DSLAMs.  We access the
DSLAM from the internet via the DSL modem's public IP which has been
configured with a DMZ to hit the DSLAM on its private IP.  We can access the
DSLAM via telnet and http, there's no problem there.

The problem occurs when we perform a configuration backup of the DSLAM.  The
DSLAM has a built-in (DSLAM vendor written?) FTP client that uploads the
configuration to our FTP server.  What's unique is that the DSLAM vendor's
FTP client issues each FTP command separately from the closing CRLF
("\r\n").  For example, when the DSLAM logs in it first issues "LOGIN
username" in one packet, and then "\r\n" in the next packet.  Attached to
this post is a trace (I removed the login section to protect the user
credentials) where you can see this happening in packets #1 and #3.
According to the DSLAM vendor it shouldn't matter if the FTP client sends
the command character-by-character in separate packets or in one packet.
That's no problem for non-NATed DSLAMs because the FTP server is supposed to
wait for the closing CRLF, but it appears that iptables doesn't handle the
situation well.

The issue crops up for me when the PORT command is sent.  The PORT (and
PASV) command require an ALG to accomplish the NAT function because the
internal IP in the payload needs to be changed to the public IP.  The ALG
doesn't (and shouldn't) send the PORT command out the WAN interface until
the "\r\n" is received from the FTP client.  The LAN interface facing the
FTP client should be ACKing every TCP packet until the "\n" is received and
then process the PORT command and send out complete command out the WAN
interface.  But it appears iptables doesn't handle this scenario.  An older
post (http://www.kfki.hu/~kadlec/sw/netfilter/ftp_security2.html) suggests
that this is a known issue.

As you can see the attached trace, in packet #7 the PORT command is received
without the CRLF, consistent with this FTP client's behavior.  But iptables
doesn't send an ACK so the FTP client resends the packet (packets #8-15).

Can anyone confirm that iptables still behaves this way, and if so, code a
fix so that no matter how many packets a PORT or PASV command are split over
(in other words, no matter how small the client's MTU) that iptables ACKs
each packet received on the LAN side and the ALG properly reassembles the
command and sends it out the WAN interface?

Regards,

Frank

[-- Attachment #2: ftp-session-2008-11-14-LAN-side-ftp-partial.pcap --]
[-- Type: application/octet-stream, Size: 1427 bytes --]

Re: conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

[Jozsef Kadlecsik]

On Sat, 15 Nov 2008, Frank Bulk wrote:

> I'm working with the tech support of a residential DSL modem vendor on a NAT
> issue.  The vendor says they use open source in their products and the logs
> show all the typical iptables entries.  
> 
> We're using a DSL modem for OOB management of our DSLAMs.  We access the
> DSLAM from the internet via the DSL modem's public IP which has been
> configured with a DMZ to hit the DSLAM on its private IP.  We can access the
> DSLAM via telnet and http, there's no problem there.
> 
> The problem occurs when we perform a configuration backup of the DSLAM.  The
> DSLAM has a built-in (DSLAM vendor written?) FTP client that uploads the
> configuration to our FTP server.  What's unique is that the DSLAM vendor's
> FTP client issues each FTP command separately from the closing CRLF
> ("\r\n").  For example, when the DSLAM logs in it first issues "LOGIN
> username" in one packet, and then "\r\n" in the next packet.  Attached to
> this post is a trace (I removed the login section to protect the user
> credentials) where you can see this happening in packets #1 and #3.
> According to the DSLAM vendor it shouldn't matter if the FTP client sends
> the command character-by-character in separate packets or in one packet.
> That's no problem for non-NATed DSLAMs because the FTP server is supposed to
> wait for the closing CRLF, but it appears that iptables doesn't handle the
> situation well.

Yes, the FTP connection tracking module expects to receive the FTP 
commands and the terminating CR character in one packet. If that's not 
true, then FTP connection tracking won't work.
 
> Can anyone confirm that iptables still behaves this way, and if so, code a
> fix so that no matter how many packets a PORT or PASV command are split over
> (in other words, no matter how small the client's MTU) that iptables ACKs
> each packet received on the LAN side and the ALG properly reassembles the
> command and sends it out the WAN interface?

Anybody is free to improve netfilter/iptables. But I strongly doubt that 
this case could be fixed easily.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

[Philip Craig]

Frank Bulk wrote:
> Can anyone confirm that iptables still behaves this way, and if so, code a
> fix so that no matter how many packets a PORT or PASV command are split over
> (in other words, no matter how small the client's MTU) that iptables ACKs
> each packet received on the LAN side and the ALG properly reassembles the
> command and sends it out the WAN interface?

iptables is a packet filter, not an ALG.  You could add more state to the
helper, but it would be hard to get right, and I don't think it is worth
the effort.  Try using a userspace ftp proxy instead.  (eg I've used frox
with no problems.)

RE: conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

[Frank Bulk]

Philip:

Thanks for the pointer to frox.

I'm not sure if the DSL modem vendor would add in this piece of opensource
software to their box, but we'll see.

Regards,

Frank

-----Original Message-----
From: Philip Craig [mailto:philipc@snapgear.com] 
Sent: Monday, November 17, 2008 6:30 PM
To: Frank Bulk
Cc: netfilter-devel@vger.kernel.org
Subject: Re: conntrack ftp fails to handle PORT (and PASV?) command when
split over multiple TCP packets

Frank Bulk wrote:
> Can anyone confirm that iptables still behaves this way, and if so, code a
> fix so that no matter how many packets a PORT or PASV command are split
over
> (in other words, no matter how small the client's MTU) that iptables ACKs
> each packet received on the LAN side and the ALG properly reassembles the
> command and sends it out the WAN interface?

iptables is a packet filter, not an ALG.  You could add more state to the
helper, but it would be hard to get right, and I don't think it is worth
the effort.  Try using a userspace ftp proxy instead.  (eg I've used frox
with no problems.)

Re: conntrack ftp fails to handle PORT (and PASV?) command when split over multiple TCP packets

[Philip Craig]

Frank Bulk wrote:
> Thanks for the pointer to frox.
> 
> I'm not sure if the DSL modem vendor would add in this piece of opensource
> software to their box, but we'll see.

If you can insert the proxy between the DSLAM and the DSL modem, then
that would work too.  The proxy will rewrite the packets so that the
DSL modem doesn't see the split command.

Note that in either case, you'll have to avoid loading the ftp conntrack
helper on the proxy machine for this to work, which may mean you'll need
to use the TransparentData option for frox (but try without it first).