conntrack table filling up due to missing ACKs near the end of a TCP session

[Menno Smits <menno@netboxblue.com>]

Hi all,

I'm seeing a problem on a customer box that acts as a primary MX and is 
  running netfilter. The secondary MXs (run by the ISP) appear to be 
broken and cause the conntrack table on the primary MX to fill up.

What happens is this:

- secondary connects to deliver mail
- connection is established and SMTP conversation takes place
- secondary issues QUIT
- primary sends response "221 Bye"
- primary sends FIN
- secondary sends FIN
- primary ACKs FIN from secondary

The secondary never ACKs the last data packet ("221 Bye") or FIN sent by 
the primary. Later the primary retries the "221 Bye" packets several 
times. Interestingly the secondary also sends a Keep-Alive soon after it 
sends it's FIN.

This happens for every inbound connection from the secondaries and the 
result is 1000's of ESTABLISHED entries with 5 day timeouts in the 
conntrack table. The table eventually fills up causing packet loss. The 
conntrack entries are all of the form:

tcp      6 24826 ESTABLISHED src=pri.ma.ry.ip dst=sec.ond.ary.ip 
sport=25 dport=54334 packets=3 bytes=183 [UNREPLIED] src=sec.ond.ary.ip 
dst=pri.ma.ry.ip sport=54334 dport=25 packets=0 bytes=0 mark=0 use=1

The primary is blocking packets that aren't NEW, ESTALISHED or RELATED. 
This could be contributing to the problem but I haven't looked closely 
at this.

Clearly the secondary mail servers are broken but shouldn't netfilter 
handle this situation better. Shouldn't the timeout of a connection be 
greatly reduced if a FIN has been sent?

I have an isolated packet dump for one connection. I'm happy to send it 
   to a netfilter developer privately but am hesitant to post it to the 
list since it is from a customer site.

Any thoughts or advice on this would be greatly appreciated.

Regards,
Menno Smits

NetBox Blue
http://netboxblue.com/