Masquerade based on skb->mark ?

Masquerade based on skb->mark ?

[Ben Greear]

I'm now trying to masquerade packets that have been marked
a certain way.  I'm using these commands:

# I'm not sure this is doing the right thing, but it is not giving errors.
iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001

# This appears to work as planned.
iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001

I added a u32 'mark' field to the conn-track tuple, and I now get
different conn-tracking objects for the same source/dest but with
different 'mark'.  However, the ct->status bit does not have the
IPS_SRC_NAT flag set.

I think I need to figure out what code creates the initial conn-track
and make sure it is setting the status bits correctly based on the
skb->mark, but I am not sure where this code exists.

Any ideas on where to start looking?  I've been trying to follow
the code path in the netfilter/nat logic, but it's proving
slow going!

Thanks,
Ben

PS.  If anyone does this sort of work for hire, please contact me off-list.

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Masquerade based on skb->mark ?

[Jan Engelhardt]

On Apr 24 2007 20:06, Ben Greear wrote:
>
> I'm now trying to masquerade packets that have been marked
> a certain way.  I'm using these commands:
>
> # I'm not sure this is doing the right thing, but it is not giving errors.
> iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001

It does what the programmer said:
  Masquerade only packets with a mark of 10001.

> # This appears to work as planned.
> iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
> iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001

And this says:
  mark all packets that come from eth1 and eth2.

So in essence, you have "masquerade everything that came from eth1 and eth2".
A slight bug, but ok. (In most cases, you only want to masquerade on
some interfaces, not all, so the use of -o is usually wanted.)

> I added a u32 'mark' field to the conn-track tuple,

Just why?

> and I now get
> different conn-tracking objects for the same source/dest but with
> different 'mark'.  However, the ct->status bit does not have the
> IPS_SRC_NAT flag set.
>
> I think I need to figure out what code creates the initial conn-track
> and make sure it is setting the status bits correctly based on the
> skb->mark, but I am not sure where this code exists.
>
> Any ideas on where to start looking?  I've been trying to follow
> the code path in the netfilter/nat logic, but it's proving
> slow going!
>
> Thanks,
> Ben
>
> PS.  If anyone does this sort of work for hire, please contact me off-list.


Jan
-- 

Re: Masquerade based on skb->mark ?

[Ben Greear]

Jan Engelhardt wrote:
> On Apr 24 2007 20:06, Ben Greear wrote:
>> I'm now trying to masquerade packets that have been marked
>> a certain way.  I'm using these commands:
>>
>> # I'm not sure this is doing the right thing, but it is not giving errors.
>> iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
> 
> It does what the programmer said:
>   Masquerade only packets with a mark of 10001.
> 
>> # This appears to work as planned.
>> iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
>> iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001
> 
> And this says:
>   mark all packets that come from eth1 and eth2.
> 
> So in essence, you have "masquerade everything that came from eth1 and eth2".
> A slight bug, but ok. (In most cases, you only want to masquerade on
> some interfaces, not all, so the use of -o is usually wanted.)
> 
>> I added a u32 'mark' field to the conn-track tuple,
> 
> Just why?

Because otherwise it seems to me that there is only a single conn-tracking
tuple for src -- dest, and it also seems to me that the conn-track entity
has the should-we-NAT flags (in the 'status' bitfield).

My scenario involves
virtual routers (ie, routing tables with rules so that pkts hit certain routing
tables) and sending packets through (virtual) looped-back ethernet ports, so
the same source-dest tuple will be seen on multiple interfaces.  I need
a different tuple for the flow that should be NATed (so only that flow is NATed),
so that is why I added the MARK rules and the mark field to the conn-track tuple.

As you noticed, my attempt to MASQ based on skb->mark was not really the right
thing to do, so I changed it back to mask with -o [outgoing-device].
This still does not seem to trigger NAT to happen, and I had no luck figuring out why
yesterday...

I found someone who is interested in considering doing this for hire,
so I will be sending details and such to them.  If that works out, hopefully
there will be a patch from him for review sometime soon.

There's also the possibility I have totally mis-understood what is currently
supported in the kernel and maybe I just need to adjust my rules or something
like that....

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Masquerade based on skb->mark ?

[Jan Engelhardt]

On Apr 26 2007 12:20, Ben Greear wrote:

>> > iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
>> > iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
>> > iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001
>> 
> Because otherwise it seems to me that there is only a single
> conn-tracking tuple for src -- dest, and it also seems to me that
> the conn-track entity has the should-we-NAT flags (in the 'status'
> bitfield).

A ct tuple, to my knowledge, constitutes (srcip, srcport, dstip, dstport).
Whether a connection is actually NATed or nat, is for you to decide
(MASQUERADE, SNAT, SAME, you name it.)

> My scenario involves virtual routers (ie, routing tables with rules
> so that pkts hit certain routing tables)

	ip rule add src 192.168.123.0/24 table 7
or	ip rule add fwmark 999 table 666

for example would do (I assume you do that)

> and sending packets through (virtual) looped-back ethernet ports,
> so the same source-dest tuple will be seen on multiple interfaces. 
> I need a different tuple for the flow that should be NATed (so only
> that flow is NATed), so that is why I added the MARK rules and the
> mark field to the conn-track tuple.

Why is a different tuple needed?


Regards,
Jan
-- 

Re: Masquerade based on skb->mark ?

[Ben Greear]

Jan Engelhardt wrote:
> On Apr 26 2007 12:20, Ben Greear wrote:
> 
>>>> iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
>>>> iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
>>>> iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001
>> Because otherwise it seems to me that there is only a single
>> conn-tracking tuple for src -- dest, and it also seems to me that
>> the conn-track entity has the should-we-NAT flags (in the 'status'
>> bitfield).
> 
> A ct tuple, to my knowledge, constitutes (srcip, srcport, dstip, dstport).
> Whether a connection is actually NATed or nat, is for you to decide
> (MASQUERADE, SNAT, SAME, you name it.)

 From looking at this method in net/ipv4/netfilter/nf_nat_core.c,
I assume it stores NAT decision as well:

/* Do packet manipulations according to nf_nat_setup_info. */
unsigned int nf_nat_packet(struct nf_conn *ct,
                            enum ip_conntrack_info ctinfo,
                            unsigned int hooknum,
                            struct sk_buff **pskb)
{
...
         /* Non-atomic: these bits don't change. */
         if (ct->status & statusbit) {
                 struct nf_conntrack_tuple target;

                 /* We are aiming to look like inverse of other direction. */
                 nf_ct_invert_tuplepr(&target, &ct->tuplehash[!dir].tuple);


>> My scenario involves virtual routers (ie, routing tables with rules
>> so that pkts hit certain routing tables)
> 
> 	ip rule add src 192.168.123.0/24 table 7
> or	ip rule add fwmark 999 table 666

Yes, I'm using commands similar to the first line.  I have not tried
using fwmark.

> for example would do (I assume you do that)
> 
>> and sending packets through (virtual) looped-back ethernet ports,
>> so the same source-dest tuple will be seen on multiple interfaces. 
>> I need a different tuple for the flow that should be NATed (so only
>> that flow is NATed), so that is why I added the MARK rules and the
>> mark field to the conn-track tuple.
> 
> Why is a different tuple needed?

Isn't the decision to NAT or not stored in the ct->status bitfield?

If so, then if I want to NAT some packets and not others,
they must belong to different tuples.

If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2,
and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I
only want to NAT pkts leaving virtual router 1, then I think I
have to somehow force different ct tuples based on which virtual
router the pkts are flowing through.  I was trying to do this by
MARKing packets entering a device in a particular virtual router
and using the mark as part of the tuple....

Thanks,
Ben

> 
> 
> Regards,
> Jan


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Masquerade based on skb->mark ?

[Jan Engelhardt]

On Apr 26 2007 13:27, Ben Greear wrote:
>> 
>> Why is a different tuple needed?
>
> Isn't the decision to NAT or not stored in the ct->status bitfield?
>
> If so, then if I want to NAT some packets and not others,
> they must belong to different tuples.

Why do you want to NAT some, and don't NAT others?

> If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2,
> and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I
> only want to NAT pkts leaving virtual router 1, then I think I
> have to somehow force different ct tuples based on which virtual
> router the pkts are flowing through.  I was trying to do this by
> MARKing packets entering a device in a particular virtual router
> and using the mark as part of the tuple....

Ah in that case it might be easiest to write a netfilter target
that does a tupleless NAT.


Jan
-- 

Re: Masquerade based on skb->mark ?

[Pascal Hambourg]

Hello,

Ben Greear a écrit :
>>
>>>>> iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
>>>>> iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark 10001
>>>>> iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark 10001
[...]
> If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2,
> and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I
> only want to NAT pkts leaving virtual router 1, then I think I
> have to somehow force different ct tuples based on which virtual
> router the pkts are flowing through.  I was trying to do this by
> MARKing packets entering a device in a particular virtual router
> and using the mark as part of the tuple....

 From what you wrote I understand you need a sort of separate connection 
tracking for each "virtual router". But I am afraid it cannot be done 
with MARK, because packet classification by the connection tracking 
takes place before the packet reaches the mangle PREROUTING (or OUTPUT) 
chain.