CONNMARK and ip rule fwmark

CONNMARK and ip rule fwmark

[Steffen Heil]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

Hi

I want every reply packet belonging to every connection made through tun1 to
be send back through tun1 (not being the default gateway tough).
(tun1 is P-t-P, so there is no routing nessessary.)

I tried the following:

# iptables -t mangle -A PREROUTING -i tun1 -p tcp --syn -j CONNMARK
--set-mark 71
# iptables -t mangle -A OUTPUT -p tcp -m connmark --mark 71 -j MARK
--set-mark 71
# ip rule add from 10.4.0.1 fwmark 71 table 201
# ip route add default dev tun1 via 10.4.0.2 table 201

However, it did not work.

I have the feeling, that the fwmark does not match.
If I remove "fwmark 71" in the "ip rule add" command, then it is working,
but has side effects with the following:

# iptables -t nat -A PREROUTING -p TCP -d publicip -j DNAT --to 10.4.0.1

My observations seem to tell me, that ip rule is evaluated BEFORE iptables
rules are applied, so the mark is not set yet.
But I assume someone else already used connmark and fwmark in conjuction, so
there must be a way to do it.

Any hints are welcome.

Regards,
  Steffen

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3109 bytes --]

Re: CONNMARK and ip rule fwmark

[Franck JONCOURT]

On Sun, 30 Mar 2008 13:03:09 +0200, "Steffen Heil" <lists@steffen-heil.de>

wrote:

> Hi



Hi,



> I want every reply packet belonging to every connection made through tun1

> to

> be send back through tun1 (not being the default gateway tough).

> (tun1 is P-t-P, so there is no routing nessessary.)

> 

> I tried the following:

> 

> # iptables -t mangle -A PREROUTING -i tun1 -p tcp --syn -j CONNMARK

> --set-mark 71

> # iptables -t mangle -A OUTPUT -p tcp -m connmark --mark 71 -j MARK

> --set-mark 71



What about using the nat table to add your mark on a whole connection 

instead of using the mangle table ?



[...]

No idea about that :p!



---

Franck Joncourt

http://www.debian.org/ - http://smhteam.info/wiki/

Re: CONNMARK and ip rule fwmark

[Jan Engelhardt]

On Sunday 2008-03-30 13:23, Franck JONCOURT wrote:
>> I want every reply packet belonging to every connection made through tun1
>> to
>> be send back through tun1 (not being the default gateway tough).
>> (tun1 is P-t-P, so there is no routing nessessary.)
>>
>> I tried the following:
>>
>> # iptables -t mangle -A PREROUTING -i tun1 -p tcp --syn -j CONNMARK
>> --set-mark 71

You would want to use -t mangle -A PREROUTING -i tun1 -m conntrack
--ctstate NEW -j CONNMARK --set-mark 71.

>> # iptables -t mangle -A OUTPUT -p tcp -m connmark --mark 71 -j MARK
>> --set-mark 71

Suggesting -t mangle -A OUTPUT [-m connmark --mark 71] -j CONNMARK 
--restore-mark


> What about using the nat table to add your mark on a whole connection
> instead of using the mangle table ?

Using the nat table to 'simulate' -m conntrack --ctstate NEW, that's
just a gross hack IMO. Oh and as soon as you start using IPv6,
there is no nat, so do not even think of doing it :p


># iptables -t nat -A PREROUTING -p TCP -d publicip -j DNAT --to 
>10.4.0.1
>
>My observations seem to tell me, that ip rule is evaluated BEFORE 
>iptables rules are applied, so the mark is not set yet.

See http://jengelh.hopto.org/images/nf-packet-flow.png , routing 
decision takes place in the middle. (It's called _PRE_ROUTING_ for a 
reason.)

>But I assume 
>someone else already used connmark and fwmark in conjuction, so there 
>must be a way to do it.

Re: CONNMARK and ip rule fwmark

[Franck JONCOURT]

[...]

>> What about using the nat table to add your mark on a whole connection

>> instead of using the mangle table ?

> 

> Using the nat table to 'simulate' -m conntrack --ctstate NEW, that's

> just a gross hack IMO. Oh and as soon as you start using IPv6,

> there is no nat, so do not even think of doing it :p



Ok, thanks for your point of view I did not think about that.

I am going to change some stuff :)!



---

Franck Joncourt

http://www.debian.org/ - http://smhteam.info/wiki/

Re: CONNMARK and ip rule fwmark

[Pascal Hambourg]

Jan Engelhardt a écrit :
> 
> On Sunday 2008-03-30 13:23, Franck JONCOURT wrote:
> 
>> What about using the nat table to add your mark on a whole connection
>> instead of using the mangle table ?
> 
> Using the nat table to 'simulate' -m conntrack --ctstate NEW, that's
> just a gross hack IMO.

I agree that the use of the nat table for any purpose not related to NAT 
should be avoided. However the advantage of the nat table is that it 
sees only one packet per connection, while "-m conntrack --ctstate NEW" 
or "-m state --state NEW" may match multiple packets per connection, 
e.g. duplicate TCP SYN or all UDP sent packets in the original direction 
before the first packet sent in the return direction.

>>> If I remove "fwmark 71" in the "ip rule add" command, then it is working,
>>> but has side effects with the following:
>>>
>>> # iptables -t nat -A PREROUTING -p TCP -d publicip -j DNAT --to 10.4.0.1
>>>
>>> My observations seem to tell me, that ip rule is evaluated BEFORE 
>>> iptables rules are applied, so the mark is not set yet.
> 
> See http://jengelh.hopto.org/images/nf-packet-flow.png , routing 
> decision takes place in the middle. (It's called _PRE_ROUTING_ for a 
> reason.)

My understanding is that the side effects mentioned by the OP apply to 
the return packets of DNATed connections, which go through the OUTPUT 
chains. The diagram show that a routing decision occurs before the 
OUTPUT chains, but a rerouting may occur because of a mark or 
destination NAT.

RE: CONNMARK and ip rule fwmark

[Steffen Heil]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

Hi

# iptables -t mangle -A PREROUTING -i tun1 -p tcp --syn -j CONNMARK --set-mark 
71
# iptables -t mangle -A OUTPUT -p tcp -m connmark --mark 71 -j MARK --set-mark 
71

> What about using the nat table to add your mark on a whole connection
> instead of using the mangle table ?

I just added:

# iptables -t nat -A PREROUTING -i tun1 -p tcp --syn -j CONNMARK --set-mark 71

Nothing changed.
However I believe that the original rule was used, because the counters 
(iptables -L -v) ARE incremented.

Regards,
  Steffen

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3109 bytes --]

Re: CONNMARK and ip rule fwmark

[Jan Engelhardt]

On Sunday 2008-03-30 18:10, Pascal Hambourg wrote:
>>
>>  Using the nat table to 'simulate' -m conntrack --ctstate NEW, that's
>>  just a gross hack IMO.
>
> I agree that the use of the nat table for any purpose not related
> to NAT should be avoided. However the advantage of the nat table is
> that it sees only one packet per connection, while "-m conntrack
> --ctstate NEW" or "-m state --state NEW" may match multiple packets
> per connection, e.g. duplicate TCP SYN or all UDP sent packets in
> the original direction before the first packet sent in the return
> direction.

That's nonsense -- the nat table sees every packet that is IPCT_NEW:

iptables -t nat -A OUTPUT -d 134.76.13.21 -p tcp --syn -j LOG
--log-prefix "[nat] "
iptables -t filter -A OUTPUT -d 134.76.13.21 -p tcp --syn -j LOG
--log-prefix "[filt] "
iptables -t filter -A OUTPUT -d 134.76.13.21 -p tcp --syn -j DROP

[698843.456383] [nat] IN= OUT=sis0 SRC=10.10.106.161 DST=134.76.13.21
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=4360 DF PROTO=TCP SPT=36903 DPT=9
WINDOW=5840 RES=0x00 SYN URGP=0

[698843.456418] [filt] IN= OUT=sis0 SRC=10.10.106.161
DST=134.76.13.21 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=4360 DF
PROTO=TCP SPT=36903 DPT=9 WINDOW=5840 RES=0x00 SYN URGP=0

[698846.454976] [nat] IN= OUT=sis0 SRC=10.10.106.161 DST=134.76.13.21
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=4361 DF PROTO=TCP SPT=36903 DPT=9
WINDOW=5840 RES=0x00 SYN URGP=0

[698846.455012] [filt] IN= OUT=sis0 SRC=10.10.106.161
DST=134.76.13.21 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=4361 DF
PROTO=TCP SPT=36903 DPT=9 WINDOW=5840 RES=0x00 SYN URGP=0

Re: CONNMARK and ip rule fwmark

[Pascal Hambourg]

Jan Engelhardt a écrit :
> 
> On Sunday 2008-03-30 18:10, Pascal Hambourg wrote:
> 
>> I agree that the use of the nat table for any purpose not related
>> to NAT should be avoided. However the advantage of the nat table is
>> that it sees only one packet per connection, while "-m conntrack
>> --ctstate NEW" or "-m state --state NEW" may match multiple packets
>> per connection, e.g. duplicate TCP SYN or all UDP sent packets in
>> the original direction before the first packet sent in the return
>> direction.
> 
> That's nonsense -- the nat table sees every packet that is IPCT_NEW:

The nonsense would be that the nat table sees packets which don't create 
a new connection, because this would be totally useless.

> iptables -t nat -A OUTPUT -d 134.76.13.21 -p tcp --syn -j LOG
> --log-prefix "[nat] "
> iptables -t filter -A OUTPUT -d 134.76.13.21 -p tcp --syn -j LOG
> --log-prefix "[filt] "
> iptables -t filter -A OUTPUT -d 134.76.13.21 -p tcp --syn -j DROP

This test is bogus because the final DROP deletes the conntrack entry 
(the packet is dropped before the connection is confirmed) so each 
packet appears to create a new connection and goes through the nat table.