odd problem

odd problem

[Jonathan Disher]

I have an interesting setup (diagrammed below), and I'm having problems
with one of my rewrites.  Basically, I have three machines:

           [A0]      [A1]    [B0]       [B1]    [C0]       [C1]
+---+        +-------+          +-------+          +-------+         +---+
| I |--------| Box A |----------| Box B |----------| Box C |---------| I |
+---+        +-------+          +-------+          +-------+         +---+

[A0] is Box A's eth0, which is hooked up to a DSL circuit, address
1.1.1.1.
[A1] is eth1 (172.16.0.2), which is crossovered to B0 (172.16.0.3).
[B1] is Box B's eth1 (172.17.0.2), crossovered to C0 (172.17.0.3).
[C1] is Box B's eth1, which is connected on our office T1 (2.2.2.2).

Box C also has an alias of 1.1.1.1 (Box A's eth0 IP) on loopback.

Box C's default route is out its gateway on [C1].  Box A's default route
is
out the DSL modem on [A0].

Basically, I am trying to have a packet come into Box A destined for
1.1.1.1, where its destination is rewritten (as a DNAT) to 172.16.0.3.
It then gets forwarded to Box B, where it is again rewritten (as another
DNAT) to 172.17.0.3.  It gets passed to Box C, where it is rewritten back
to 1.1.1.1, and since Box C has that address as an alias, acted upon.
However, I want the response to go out [C1].

Currently, the first two rewrites are working (verified), and I'm fairly
sure the third is working.  Box C is even putting a response out [C1].
However, the source address of that response is 172.17.0.3.  I've tried
putting an SNAT in to rewrite any source address of 172.17.0.3 to 1.1.1.1,
but it isn't working.

Any ideas would be greatly appreciated.  I know it's ugly, but I need
it to work this way.  All three machines are running stock 2.4.22 kernel.

Thanks!

-j

Re: odd problem

[Ramin Dousti]

On Tue, Oct 07, 2003 at 01:22:51PM -0400, Jonathan Disher wrote:

> I have an interesting setup (diagrammed below), and I'm having problems
> with one of my rewrites.  Basically, I have three machines:
> 
>            [A0]      [A1]    [B0]       [B1]    [C0]       [C1]
> +---+        +-------+          +-------+          +-------+         +---+
> | I |--------| Box A |----------| Box B |----------| Box C |---------| I |
> +---+        +-------+          +-------+          +-------+         +---+
> 
> [A0] is Box A's eth0, which is hooked up to a DSL circuit, address
> 1.1.1.1.
> [A1] is eth1 (172.16.0.2), which is crossovered to B0 (172.16.0.3).
> [B1] is Box B's eth1 (172.17.0.2), crossovered to C0 (172.17.0.3).
> [C1] is Box B's eth1, which is connected on our office T1 (2.2.2.2).
> 
> Box C also has an alias of 1.1.1.1 (Box A's eth0 IP) on loopback.
> 
> Box C's default route is out its gateway on [C1].  Box A's default route
> is
> out the DSL modem on [A0].
> 
> Basically, I am trying to have a packet come into Box A destined for
> 1.1.1.1, where its destination is rewritten (as a DNAT) to 172.16.0.3.
> It then gets forwarded to Box B, where it is again rewritten (as another
> DNAT) to 172.17.0.3.  It gets passed to Box C, where it is rewritten back
> to 1.1.1.1, and since Box C has that address as an alias, acted upon.
> However, I want the response to go out [C1].
> 
> Currently, the first two rewrites are working (verified), and I'm fairly
> sure the third is working.  Box C is even putting a response out [C1].
> However, the source address of that response is 172.17.0.3.  I've tried
> putting an SNAT in to rewrite any source address of 172.17.0.3 to 1.1.1.1,
> but it isn't working.

Can you see response packets on C1 being 172.17.0.3 (by tcpdump)?
What you say is strange because the application on C would see
packets like a.b.c.d -> 1.1.1.1 and if the application which is
supposed to reply back is listening on 1.1.1.1 (alias address) then
the response will have 1.1.1.1 with no need to SNAT.

The reason for it not to work might be the asymmetric nature of the routing.
What is happening "could be" that you'd create a conntrak entry for

a.b.c.d <-> 1.1.1.1 on C0 with state (NEW) when the syn comes in.

then you'd send the syn-ack through C1 which will have another (NEW)
state entry for this on C1.

I don't know the bits of the state machine with asymmitric routing but
that might be your problem. Just try to allow everything NEW,ESTABLISHED,
RELATED on both C1 and A0 and see what happens. Once you can determine 
the cause we can work out a more ellegant/secure solution...

Ramin


> 
> Any ideas would be greatly appreciated.  I know it's ugly, but I need
> it to work this way.  All three machines are running stock 2.4.22 kernel.
> 
> Thanks!
> 
> -j

Re: odd problem

[Jonathan Disher]

On Tue, 7 Oct 2003, Ramin Dousti wrote:

> Can you see response packets on C1 being 172.17.0.3 (by tcpdump)?
> What you say is strange because the application on C would see
> packets like a.b.c.d -> 1.1.1.1 and if the application which is
> supposed to reply back is listening on 1.1.1.1 (alias address) then
> the response will have 1.1.1.1 with no need to SNAT.

Box C is sending responses out the correct interface, with the correct
destination address, but the source address is 172.17.0.3.  This is why I
thought I needed an SNAT (although it's probably more likely that I just
need to beat the application and make it only bind to 1.1.1.1, then it
should only use that for replies).

What I'm using to test, right now, is sshd.  However, by default, sshd
listens on *, so, that may be a factor.  I'm going to go find this out.
However, in the meantime, I just wanted to pick the brains of someone more
knowledgeable in iptables/netfilter than I - I've never used iptables,
beyond some very very simple masquerading before.

Thanks!

-j