conntrack: confirm existing but do not create new entries

conntrack: confirm existing but do not create new entries

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 1091 bytes --]

Hello,

My use case is to set up a stateful firewall allowing any outgoing connection
from a host, and restricting incoming, which obviously requires conntracking.
The twist is that there exists a rather high probability of DoS-like incoming
traffic, that easily overflows conntrack table with unconfirmed entries, even
though their lifetime is very short.

I was hoping to keep conntracking enabled in the outgoing/raw hook, but in the
prerouting/raw hook disable conntracking _unless_ an entry (previously created
by an outgoing packet) already exists. In other words, *make incoming packets
update existing entries but never create new entries*.

Looking at the code in resolve_normal_ct(), apparently there is no any special
flag or anything for that. But maybe it is possible to have a rule in the
prerouting/raw chain that would conditionally set `notrack` flag depending on a
lookup in the conntrack table?

Or maybe there are other suggestions how to achieve the goal (of not letting
"unexpected" incoming packets fill conntrack table)?

Thank you,

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: conntrack: confirm existing but do not create new entries

[Florian Westphal]

Eugene Crosser <crosser@average.org> wrote:
> My use case is to set up a stateful firewall allowing any outgoing connection
> from a host, and restricting incoming, which obviously requires conntracking.
> The twist is that there exists a rather high probability of DoS-like incoming
> traffic, that easily overflows conntrack table with unconfirmed entries, even
> though their lifetime is very short.

Create a rule that drops NEW packets in prerouting hook. For iptables,
mangle will work (raw is too early).  For nftables, youl need to choose
a hook prioriy of -199 or higher (-198, ... to anything below 2**31).

Such packets will create a new connection entry, but because packet gets
dropped before confirmation the entry will not be committed to the table.

Re: conntrack: confirm existing but do not create new entries

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 915 bytes --]

On 18/08/2021 18:13, Florian Westphal wrote:
> Eugene Crosser <crosser@average.org> wrote:
>> My use case is to set up a stateful firewall allowing any outgoing connection
>> from a host, and restricting incoming, which obviously requires conntracking.
>> The twist is that there exists a rather high probability of DoS-like incoming
>> traffic, that easily overflows conntrack table with unconfirmed entries, even
>> though their lifetime is very short.
> 
> Create a rule that drops NEW packets in prerouting hook. For iptables,
> mangle will work (raw is too early).  For nftables, youl need to choose
> a hook prioriy of -199 or higher (-198, ... to anything below 2**31).
> 
> Such packets will create a new connection entry, but because packet gets
> dropped before confirmation the entry will not be committed to the table.

Oh, cool trick!  I will try that, thank you very much!

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: conntrack: confirm existing but do not create new entries

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 1479 bytes --]

On 18/08/2021 18:13, Florian Westphal wrote:
> Eugene Crosser <crosser@average.org> wrote:
>> My use case is to set up a stateful firewall allowing any outgoing connection
>> from a host, and restricting incoming, which obviously requires conntracking.
>> The twist is that there exists a rather high probability of DoS-like incoming
>> traffic, that easily overflows conntrack table with unconfirmed entries, even
>> though their lifetime is very short.
> 
> Create a rule that drops NEW packets in prerouting hook. For iptables,
> mangle will work (raw is too early).  For nftables, youl need to choose
> a hook prioriy of -199 or higher (-198, ... to anything below 2**31).
> 
> Such packets will create a new connection entry, but because packet gets
> dropped before confirmation the entry will not be committed to the table.

Unfortunately this approach does not work for my case.
I cannot drop packets before I know that they are going to the INPUT path.
The point of the exercise is to route transit traffic without conntracking it
(because tracking will overflow the table), but conntrack traffic to/from the
host itself.

Maybe I'd be able to determine which traffic is transit on prerouting stage (by
IP addresses or interfaces) and set "notrack" selectively...

Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
entries" does not exist. Would not it be useful to have?..

Thanks again in any case!

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: conntrack: confirm existing but do not create new entries

[Florian Westphal]

Eugene Crosser <crosser@average.org> wrote:
> On 18/08/2021 18:13, Florian Westphal wrote:
> > Eugene Crosser <crosser@average.org> wrote:
> >> My use case is to set up a stateful firewall allowing any outgoing connection
> >> from a host, and restricting incoming, which obviously requires conntracking.
> >> The twist is that there exists a rather high probability of DoS-like incoming
> >> traffic, that easily overflows conntrack table with unconfirmed entries, even
> >> though their lifetime is very short.
> > 
> > Create a rule that drops NEW packets in prerouting hook. For iptables,
> > mangle will work (raw is too early).  For nftables, youl need to choose
> > a hook prioriy of -199 or higher (-198, ... to anything below 2**31).
> > 
> > Such packets will create a new connection entry, but because packet gets
> > dropped before confirmation the entry will not be committed to the table.
> 
> Unfortunately this approach does not work for my case.
> I cannot drop packets before I know that they are going to the INPUT path.
> The point of the exercise is to route transit traffic without conntracking it
> (because tracking will overflow the table), but conntrack traffic to/from the
> host itself.

Confirmation  (== commit of NEW to contnrack table) occurs at end of INPUT
(incoming case) or POSTROUTING (for locally originating and forwarded traffic)

> Maybe I'd be able to determine which traffic is transit on prerouting stage (by
> IP addresses or interfaces) and set "notrack" selectively...

You might have to clarify what you are looking for.

From your original post i concluded its this:

 <Your_Host>  ----> <internet>

... and no forwarding at all.  Requirement is to prevent NEW entries
when traffic is incoming (but not a reply).

With above, its really this:

<some_network> <---> <Your_Host> <---> <internet>

And you don't want to track forwarded traffic (but still want to track
locally originating traffic).

Only choice in that case is to NOTRACK in raw/PREROUTING based on incoming
interface.

If you are concerned about flood from <internet> -> <Your_Host>

.. then you can drop traffic coming from public interface in INPUT
if its NEW.

(assuming you still track packets coming in via the 'public' interface,
 else that won't work).

> Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
> entries" does not exist. Would not it be useful to have?..

How would that even work?

To know if a given packet is a reply there needs to be a record in the
state table.

If you are only looking for 'remote' vs 'locally generated' traffic, you might
want to have a look at the socket match, which could be used to detect
if an incoming packet would match a local socket.

There is also the 'addrtype' match, which could be used to detect
(at prerouting) if the routing table says that the daddr is local (or not).

But depending on wheter you are using NAT this might not work.

Re: conntrack: confirm existing but do not create new entries

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 5161 bytes --]

On 19/08/2021 11:09, Florian Westphal wrote:
> Eugene Crosser <crosser@average.org> wrote:
>> On 18/08/2021 18:13, Florian Westphal wrote:
>>> Eugene Crosser <crosser@average.org> wrote:
>>>> My use case is to set up a stateful firewall allowing any outgoing connection
>>>> from a host, and restricting incoming, which obviously requires conntracking.
>>>> The twist is that there exists a rather high probability of DoS-like incoming
>>>> traffic, that easily overflows conntrack table with unconfirmed entries, even
>>>> though their lifetime is very short.
>>>
>>> Create a rule that drops NEW packets in prerouting hook. For iptables,
>>> mangle will work (raw is too early).  For nftables, youl need to choose
>>> a hook prioriy of -199 or higher (-198, ... to anything below 2**31).
>>>
>>> Such packets will create a new connection entry, but because packet gets
>>> dropped before confirmation the entry will not be committed to the table.
>>
>> Unfortunately this approach does not work for my case.
>> I cannot drop packets before I know that they are going to the INPUT path.
>> The point of the exercise is to route transit traffic without conntracking it
>> (because tracking will overflow the table), but conntrack traffic to/from the
>> host itself.
> 
> Confirmation  (== commit of NEW to contnrack table) occurs at end of INPUT
> (incoming case) or POSTROUTING (for locally originating and forwarded traffic)
> 
>> Maybe I'd be able to determine which traffic is transit on prerouting stage (by
>> IP addresses or interfaces) and set "notrack" selectively...
> 
> You might have to clarify what you are looking for.

Yes, I oversimplified our case and omitted the part that turned to be important.
This is a host for multiple VMs. We want VM traffic to go through unfiltered and
_untracked_ because tracking _this_ traffic overflows the table.

Traffic to the host itself (management and configuration) needs to be filtered,
and to do it effectively we need stateful firewall, hence it should be tracked.
Flooding of the table by traffic that ends up in the INPUT path is not really a
concern, and we would drop bad traffic anyway, preventing ct entries from being
committed (as far as I understood your explanation).

So the problem, as far as I understand it, is that decision to track or not
track has to be taken _before_ the decision between FORWARD and INPUT. We want
FORWARD to go (ahem) forward without modifying the table, and INPUT to be
processed against the table to be able to decide if it belongs to an existing
outgoing connection.

That _could_ be achieved if the "track/notrack" switch had a third position:
"try to track, but if there was no existing entry, pretend that it's notrack".

> And you don't want to track forwarded traffic (but still want to track
> locally originating traffic).
> 
> Only choice in that case is to NOTRACK in raw/PREROUTING based on incoming
> interface.

Yeah, I guess we'd have to settle on something like that (or based on the
destination IP address being "in our management network" maybe). That's be more
fragile w.r.t. possible future changes of interfaces / addresses.

>> Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
>> entries" does not exist. Would not it be useful to have?..
> 
> How would that even work?
>
> To know if a given packet is a reply there needs to be a record in the
> state table.

I was guessing this piece of code

=====
        /* look for tuple match */
        zone = nf_ct_zone_tmpl(tmpl, skb, &tmp);
        hash = hash_conntrack_raw(&tuple, state->net);
        h = __nf_conntrack_find_get(state->net, zone, &tuple, hash);
        if (!h) {
                h = init_conntrack(state->net, tmpl, &tuple,
                                   skb, dataoff, hash);
                if (!h)
                        return 0;
                if (IS_ERR(h))
                        return PTR_ERR(h);
        }
        ct = nf_ct_tuplehash_to_ctrack(h);
=====

could _not_ call `init_conntrack()` depending on the (hypothetical) "track only
when entry exists" flag, and return early otherwise?.. But my understanding of
the workings of netfilter is really poor, it is very likely that I am missing
something crucial.

> If you are only looking for 'remote' vs 'locally generated' traffic, you might
> want to have a look at the socket match, which could be used to detect
> if an incoming packet would match a local socket.
> 
> There is also the 'addrtype' match, which could be used to detect
> (at prerouting) if the routing table says that the daddr is local (or not).
> 
> But depending on wheter you are using NAT this might not work.

Oh, this is a very good hint! I'll need to check if that can be used for our
case. We need NAT, but in the paths between specific interfaces, where we can
enable conntrack separately. So it might work!

While I am here, may I ask: is there a "correct" way to re-enable conntrack that
was disabled in someone else's chain that happened to run before mine?

Thanks once again,

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: conntrack: confirm existing but do not create new entries

[Florian Westphal]

Eugene Crosser <crosser@average.org> wrote:
> On 19/08/2021 11:09, Florian Westphal wrote:
> > You might have to clarify what you are looking for.
> 
> Yes, I oversimplified our case and omitted the part that turned to be important.
> This is a host for multiple VMs. We want VM traffic to go through unfiltered and
> _untracked_ because tracking _this_ traffic overflows the table.
> 
> Traffic to the host itself (management and configuration) needs to be filtered,
> and to do it effectively we need stateful firewall, hence it should be tracked.
> Flooding of the table by traffic that ends up in the INPUT path is not really a
> concern, and we would drop bad traffic anyway, preventing ct entries from being
> committed (as far as I understood your explanation).
> 
> So the problem, as far as I understand it, is that decision to track or not
> track has to be taken _before_ the decision between FORWARD and INPUT. We want
> FORWARD to go (ahem) forward without modifying the table, and INPUT to be
> processed against the table to be able to decide if it belongs to an existing
> outgoing connection.

Yes, conntrack hooks at prerouting and output, so by the the input or
forward are reached the conntrack lookup was already done.

Hence, NOTRACK has to be decided before conntrack lookup/create.

> >> Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
> >> entries" does not exist. Would not it be useful to have?..
> > 
> > How would that even work?
> >
> > To know if a given packet is a reply there needs to be a record in the
> > state table.
> 
> I was guessing this piece of code
> 
> =====
>         /* look for tuple match */
>         zone = nf_ct_zone_tmpl(tmpl, skb, &tmp);
>         hash = hash_conntrack_raw(&tuple, state->net);
>         h = __nf_conntrack_find_get(state->net, zone, &tuple, hash);
>         if (!h) {
>                 h = init_conntrack(state->net, tmpl, &tuple,
>                                    skb, dataoff, hash);
>                 if (!h)
>                         return 0;
>                 if (IS_ERR(h))
>                         return PTR_ERR(h);
>         }
>         ct = nf_ct_tuplehash_to_ctrack(h);
> =====
> 
> could _not_ call `init_conntrack()` depending on the (hypothetical) "track only
> when entry exists" flag, and return early otherwise?.. But my understanding of
> the workings of netfilter is really poor, it is very likely that I am missing
> something crucial.

I don't see how it can work, something does need to create entries, else
__nf_conntrack_find_get() never finds an entry, so init_conntrack() is
never called, so table is always empty.

> While I am here, may I ask: is there a "correct" way to re-enable conntrack that
> was disabled in someone else's chain that happened to run before mine?

You mean, 'cancelling' earlier NOTRACK rule? I'm not aware of such a
feature.

Re: conntrack: confirm existing but do not create new entries

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 2935 bytes --]

On 19/08/2021 16:18, Florian Westphal wrote:
> Eugene Crosser <crosser@average.org> wrote:
...
>>>> Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
>>>> entries" does not exist. Would not it be useful to have?..
>>>
>>> How would that even work?
>>>
>>> To know if a given packet is a reply there needs to be a record in the
>>> state table.
>>
>> I was guessing this piece of code
>>
>> =====
>>         /* look for tuple match */
>>         zone = nf_ct_zone_tmpl(tmpl, skb, &tmp);
>>         hash = hash_conntrack_raw(&tuple, state->net);
>>         h = __nf_conntrack_find_get(state->net, zone, &tuple, hash);
>>         if (!h) {
>>                 h = init_conntrack(state->net, tmpl, &tuple,
>>                                    skb, dataoff, hash);
>>                 if (!h)
>>                         return 0;
>>                 if (IS_ERR(h))
>>                         return PTR_ERR(h);
>>         }
>>         ct = nf_ct_tuplehash_to_ctrack(h);
>> =====
>>
>> could _not_ call `init_conntrack()` depending on the (hypothetical) "track only
>> when entry exists" flag, and return early otherwise?.. But my understanding of
>> the workings of netfilter is really poor, it is very likely that I am missing
>> something crucial.
> 
> I don't see how it can work, something does need to create entries, else
> __nf_conntrack_find_get() never finds an entry, so init_conntrack() is
> never called, so table is always empty.

I was (probably incorrectly?) assuming that __nf_conntrack_find_get() looks up
in the "main" table, so there will be entries created by previous packets that
went "in the opposite direction" and did not have the "track only when entry
exists" flag. In my case, packets that have traversed the OUTPUT chain.

By the way, another way to implement such feature would be a settable flag in
the skb "do not commit to the main conntrack table"?..

I need to check how "fib daddr type local" will work in the presence of VRFs
that may be forwarding the same addresses that in the default VRF are going to
INPUT. Basing decision on the input interface is kind of undesirable for us,
because there are so many, and some may be named differently on different hardware.

>> While I am here, may I ask: is there a "correct" way to re-enable conntrack that
>> was disabled in someone else's chain that happened to run before mine?
> 
> You mean, 'cancelling' earlier NOTRACK rule? I'm not aware of such a
> feature.

That's what I thought. That means that different entities that need to control
`notrack` cannot pile up "exceptions on top of exceptions" and have to cooperate
very stringently. If I want to have a chain that ends with unconditional
catch-all "notrack", then everyone who needs to enable tracking for a specific
class of traffic must have rules in the same chain. But we can work with that.

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: conntrack: confirm existing but do not create new entries

[Mathew Heard]

Perhaps see  https://github.com/X4BNet/xt_NOCREATE



On Mon, 23 Aug 2021 at 23:31, Eugene Crosser <crosser@average.org> wrote:
>
> On 19/08/2021 16:18, Florian Westphal wrote:
> > Eugene Crosser <crosser@average.org> wrote:
> ...
> >>>> Yet I am surprised that a flag meaning "conntrack, but only to confirm exiting
> >>>> entries" does not exist. Would not it be useful to have?..
> >>>
> >>> How would that even work?
> >>>
> >>> To know if a given packet is a reply there needs to be a record in the
> >>> state table.
> >>
> >> I was guessing this piece of code
> >>
> >> =====
> >>         /* look for tuple match */
> >>         zone = nf_ct_zone_tmpl(tmpl, skb, &tmp);
> >>         hash = hash_conntrack_raw(&tuple, state->net);
> >>         h = __nf_conntrack_find_get(state->net, zone, &tuple, hash);
> >>         if (!h) {
> >>                 h = init_conntrack(state->net, tmpl, &tuple,
> >>                                    skb, dataoff, hash);
> >>                 if (!h)
> >>                         return 0;
> >>                 if (IS_ERR(h))
> >>                         return PTR_ERR(h);
> >>         }
> >>         ct = nf_ct_tuplehash_to_ctrack(h);
> >> =====
> >>
> >> could _not_ call `init_conntrack()` depending on the (hypothetical) "track only
> >> when entry exists" flag, and return early otherwise?.. But my understanding of
> >> the workings of netfilter is really poor, it is very likely that I am missing
> >> something crucial.
> >
> > I don't see how it can work, something does need to create entries, else
> > __nf_conntrack_find_get() never finds an entry, so init_conntrack() is
> > never called, so table is always empty.
>
> I was (probably incorrectly?) assuming that __nf_conntrack_find_get() looks up
> in the "main" table, so there will be entries created by previous packets that
> went "in the opposite direction" and did not have the "track only when entry
> exists" flag. In my case, packets that have traversed the OUTPUT chain.
>
> By the way, another way to implement such feature would be a settable flag in
> the skb "do not commit to the main conntrack table"?..
>
> I need to check how "fib daddr type local" will work in the presence of VRFs
> that may be forwarding the same addresses that in the default VRF are going to
> INPUT. Basing decision on the input interface is kind of undesirable for us,
> because there are so many, and some may be named differently on different hardware.
>
> >> While I am here, may I ask: is there a "correct" way to re-enable conntrack that
> >> was disabled in someone else's chain that happened to run before mine?
> >
> > You mean, 'cancelling' earlier NOTRACK rule? I'm not aware of such a
> > feature.
>
> That's what I thought. That means that different entities that need to control
> `notrack` cannot pile up "exceptions on top of exceptions" and have to cooperate
> very stringently. If I want to have a chain that ends with unconditional
> catch-all "notrack", then everyone who needs to enable tracking for a specific
> class of traffic must have rules in the same chain. But we can work with that.
>
> Eugene
>