RFC: NAT's default behavior of forwarding un-NATed packets

RFC: NAT's default behavior of forwarding un-NATed packets

[Xiaozhou Liu]

Hi,

We find that our SNAT sometimes forwards un-NATed packets out as-is. This
behavior confused us for a while until we saw this in
net/netfilter/nf_nat_core.c:

       ct = nf_ct_get(skb, &ctinfo);
       /* Can't track?  It's not due to stress, or conntrack would
        * have dropped it.  Hence it's the user's responsibilty to
        * packet filter it out, or implement conntrack/NAT for that
        * protocol. 8) --RR
        */
       if (!ct)
              return NF_ACCEPT;

The code and comment are very clear. So it is not kernel's responsibility,
at least in RR's point of view. We added filtering shortly afterwards.

But as normal users, we really want a NAT server which can work out of box as
expected from users' point of view. That said, if any packet that is bad (in
our case, orphaned SYN-ACK and FIN packets going out without getting a chance
to setup conntrack), it's better to drop them in kernel by default.

Is there any possibility to change this default behavior?

Any comments on this is appreciated.


Thanks,
Xiaozhou

Re: RFC: NAT's default behavior of forwarding un-NATed packets

[Jan Engelhardt]

On Wednesday 2019-04-17 10:02, Xiaozhou Liu wrote:
>
>       ct = nf_ct_get(skb, &ctinfo);
>       /* Can't track?  It's not due to stress, or conntrack would
>        * have dropped it.  Hence it's the user's responsibilty to
>        * packet filter it out, or implement conntrack/NAT for that
>        * protocol. 8) --RR
>        */
>       if (!ct)
>              return NF_ACCEPT;
>
>But as normal users, we really want a NAT server which can work out of box as
>expected from users' point of view.

That depends on the user. NAT already breaks with expectations by messing with
the end-to-end principle for instance.

>That said, if any packet that is bad (in our case, orphaned SYN-ACK and FIN
>packets going out without getting a chance to setup conntrack), it's better to
>drop them in kernel by default.

I disagree. The ct state may be absent because the firewall has an empty state
(e.g. reboot / `conntrack -F`) — that alone does not make the TCP connection
invalid. Blocking ct-less packets breaks the user expectation to get a timely
response (the more so on connection shutdown).

NF_ACCEPT is the proper action, because it mirrors the default policy when the
firewall is unconfigured. Pretty much all common acts of dropping, directly or
indirectly, are opt-in.

>Is there any possibility to change this default behavior?

iptables -I FORWARD -m conntrack --ctstate UNTRACKED -j DROP

Or better yet, model your ruleset such that
 --ctstate NEW/ESTABLISHED/$OtherwiseGood -j ACCEPT
 -P DROP

Re: [External Email] Re: RFC: NAT's default behavior of forwarding un-NATed packets

[Xiaozhou Liu]

On Wed, Apr 17, 2019 at 10:58:37AM +0200, Jan Engelhardt wrote:

Hi Jan,
> 
> I disagree. The ct state may be absent because the firewall has an empty state
> (e.g. reboot / `conntrack -F`) — that alone does not make the TCP connection
> invalid. Blocking ct-less packets breaks the user expectation to get a timely
> response (the more so on connection shutdown).
> 

Allowing ct-less packets out does not help in this situation either. If for
some reason the NAT's ct gets emptied, and some abnormal packets (orphaned
SYN-ACK/FIN) pass through SNAT without NATed, they'd still carry their inner
source IP (e.g. 192.168.X.X) in the header, which will not be respected outside
of our private network. Hence, although the bad packets are not dropped by NAT,
they can go to the internet but eventually gets dropped somewhere else. So the
user's experience would be the same as the packets are dropped by NAT. This
is what happened here in our network.

On the other hand, we see that normal established-state packets can setup ct
info on-the-fly even if NAT's ct gets emptied, and can be correctly handled
by SNAT later. User's expectation is not broken.

> NF_ACCEPT is the proper action, because it mirrors the default policy when the
> firewall is unconfigured. Pretty much all common acts of dropping, directly or
> indirectly, are opt-in.

Resonable.

But if an admin setup a SNAT, he/she really does not expect any packet with
inner-source-IP showing on the outgoing wire. ;( Not everyone knows that filtering
of such packets should setup explicitly. At least that is the case here until
we dug into kernel source code. It's a little bit counter intuitive.

> 
> iptables -I FORWARD -m conntrack --ctstate UNTRACKED -j DROP
> 
> Or better yet, model your ruleset such that
>  --ctstate NEW/ESTABLISHED/$OtherwiseGood -j ACCEPT
>  -P DROP

Thanks for your reply and advice!


Regards,
Xiaozhou

Re: [External Email] Re: RFC: NAT's default behavior of forwarding un-NATed packets

[Jan Engelhardt]

On Wednesday 2019-04-17 13:54, Xiaozhou Liu wrote:
>On Wed, Apr 17, 2019 at 10:58:37AM +0200, Jan Engelhardt wrote:
>
>> I disagree. The ct state may be absent because the firewall has an empty state
>> (e.g. reboot / `conntrack -F`) — that alone does not make the TCP connection
>> invalid. Blocking ct-less packets breaks the user expectation to get a timely
>> response (the more so on connection shutdown).
>
>Allowing ct-less packets out does not help in this situation either. If for
>some reason the NAT's ct gets emptied, and some abnormal packets (orphaned
>SYN-ACK/FIN) pass through SNAT without NATed, they'd still carry their inner
>source IP (e.g. 192.168.X.X) in the header

Well argued. I also failed to notice that the return NF_ACCEPT; statement was
in nf_nat_core.c (as opposed to somewhere in general connection tracking).

>, which will not be respected outside
>of our private network. Hence, although the bad packets are not dropped by NAT,
>they can go to the internet but eventually gets dropped somewhere else.

Only if SNAT, but SNAT is not the only type of NAT that is possible.
Though I cannot think of a sensible use case involving DNAT where forwarding
to the original destination is meaningful either.

>On the other hand, we see that normal established-state packets can setup ct
>info on-the-fly even if NAT's ct gets emptied,

This mechanism is part of CT (not NAT), and I am not sure that will work
if the packet goes from public to private network (as there is no NAT entry,
there is no knowing where it could possibly go).

Re: RFC: NAT's default behavior of forwarding un-NATed packets

[Pablo Neira Ayuso]

On Wed, Apr 17, 2019 at 04:02:44PM +0800, Xiaozhou Liu wrote:
> Hi,
> 
> We find that our SNAT sometimes forwards un-NATed packets out as-is. This
> behavior confused us for a while until we saw this in
> net/netfilter/nf_nat_core.c:
> 
>        ct = nf_ct_get(skb, &ctinfo);
>        /* Can't track?  It's not due to stress, or conntrack would
>         * have dropped it.  Hence it's the user's responsibilty to
>         * packet filter it out, or implement conntrack/NAT for that
>         * protocol. 8) --RR
>         */
>        if (!ct)
>               return NF_ACCEPT;
> 
> The code and comment are very clear. So it is not kernel's responsibility,
> at least in RR's point of view. We added filtering shortly afterwards.
> 
> But as normal users, we really want a NAT server which can work out of box as
> expected from users' point of view. That said, if any packet that is bad (in
> our case, orphaned SYN-ACK and FIN packets going out without getting a chance
> to setup conntrack), it's better to drop them in kernel by default.
> 
> Is there any possibility to change this default behavior?

You can just drop invalid traffic via policy.