[LARTC] Bridging two subnets selectively using routing

[LARTC] Bridging two subnets selectively using routing

[Joshua Kwan]

Hi LARTC,

I have two networks that I manage, A and B.  They both have their own
primary gateways for hosts on the network, let's call them AR and BR.
There is another box, A1, which lives on network A but also has a wire
connecting it to a switch on network B. My goal is to let hosts on B
access three particular hosts on A's subnet (192.168.4.0/24) and let
hosts on A access one particular host on B's subnet (10.3.0.0/24).

Suppose the IP addresses of the machines I wish to address on either
network are A2, A3, A4 and B1.

A1 has IP 10.3.0.13 on network B and 192.168.4.10 on network A. Is it
possible to just tell the gateways for each network:

on BR: ip route add {A2, A3, A4} via 10.3.0.13
on AR: ip route add B1 via 192.168.4.10

But when I do that, ping does something like:

PING A2 56(84) bytes of data.
From BR: Redirect Host (new nexthop: 10.3.0.13)

and doesn't actually report success on the ping.

What am I doing wrong? Is this a job for iptables? I feel like I can do
this with just routing table magic.

Thanks,

-- 
Joshua Kwan
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Bridging two subnets selectively using routing

[Corey Hickey]

Joshua Kwan wrote:
> Hi LARTC,
> 
> I have two networks that I manage, A and B.  They both have their own
> primary gateways for hosts on the network, let's call them AR and BR.
> There is another box, A1, which lives on network A but also has a wire
> connecting it to a switch on network B. My goal is to let hosts on B
> access three particular hosts on A's subnet (192.168.4.0/24) and let
> hosts on A access one particular host on B's subnet (10.3.0.0/24).
> 
> Suppose the IP addresses of the machines I wish to address on either
> network are A2, A3, A4 and B1.
> 
> A1 has IP 10.3.0.13 on network B and 192.168.4.10 on network A. Is it
> possible to just tell the gateways for each network:
> 
> on BR: ip route add {A2, A3, A4} via 10.3.0.13
> on AR: ip route add B1 via 192.168.4.10
> 
> But when I do that, ping does something like:
> 
> PING A2 56(84) bytes of data.
> From BR: Redirect Host (new nexthop: 10.3.0.13)

That's to be expected, as far as I understand.

> and doesn't actually report success on the ping.

> What am I doing wrong? Is this a job for iptables? I feel like I can do
> this with just routing table magic.

Well, nobody has given you a better answer yet, so I'll take a shot.

1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it's
configured to route at all.

2. The hosts on network A will receive packets from miscellaneous IP
addresses in B's subnet, and not have any idea what to do with them.
You'll need to either configure routes on the hosts in network A...
# ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0
...or set up box A1 to SNAT packets
# iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 \
-j SNAT --to-source 192.168.4.10
You'll have to do something similar for the reverse direction, too.

I haven't tested what I wrote above; if it doesn't work, run tcpdump in
various places to see what icmp packets are going where.


Honestly, though, I think you're setting yourself up for a decent
headache trying to do it this way. In my opinion, the easy solution is
in the first word of your subject line: just make host A1 a bridge.
That's what I did when I originally set up that box, and it worked fine
the whole time I managed it (actually, I didn't manage it at all, since
the bridging never had any problems).

Put both physical networks on the same IP subnet and enforce any desired
firewalling/segmentation with a few simple iptables rules running on the
bridge. If you're still using the same machine that used to do that job,
you might be able to find the relevant configuration files lying around.
If not, I'm sure I can recall at least most of the details.


Either way, if you need assistance, let me know and I'll be happy to
stop by sometime and lend a hand.

-Corey
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Bridging two subnets selectively using routing

[Grant Taylor]

On 10/31/07 19:50, Joshua Kwan wrote:
> What am I doing wrong? Is this a job for iptables? I feel like I can 
> do this with just routing table magic.

As Corey said, make sure that IP Forwarding is enabled on A1.

You are setting your self up for a very weird scenario.  Probably what 
would be the easiest for long term would be to put a third network 
between your two routers, AR and BR.  This way you end up with the 
following type of scenario:

                +----+               +----+
                | AI |               | BI |
                +-+--+               +--+-+
                  |                     |
                +-+--+               +--+-+
---( Net A )---+ AR +---( Net C )---+ BR +---( Net B )---
        |       +----+               +----+       |
        |                                         |
        |       +----+                            |
        +-------+ A1 +----------------------------+
                +----+

With this type of set up, you can configure both AR and BR that each can 
reach the others network via the other router on network C.  I.e. AR can 
reach Net B via BR's IP on Net C.  Likewise BR can reach Net A via AR's 
IP on Net C.  This way, each and every computer on both networks A and B 
can reach any host they want to via their default gateway AR or BR 
respectively.  This may be a little more work to set up initially, but 
it is very simple routing that will be very easy to maintain for a long 
time.  In this scenario Net C can very simply be a cross over cable 
between routers AR and BR.

Another option is to enable routing on A1 and tell A2, A3, and A4 that 
they can reach B1 via A1 and likewise tell B1 that it can reach a2, A3, 
and A4 via A1.  Thus you have a bi-directional routing set up between 
just the four additional hosts.  The main disadvantage of this set up is 
that each and every host that wants to communicate across networks will 
have to have its own routing entries set up to pick the correct 
direction to route traffic to.  So if you only have two or three hosts 
on each network and you can not do the additional network (C) between 
the two routers, this will work.

There are some other things that you can do to allow the hosts to 
communicate across the networks like bridging and / or proxy ARP. 
However seeing as how you have different subnets on Net A and Net B, you 
will very likely be doing routing or multi-homing on all hosts to 
support bridging and / or proxy ARP.

I think your best bet is to do the additional network (C) between the 
two routers AR and BR.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Bridging two subnets selectively using routing

[Grant Taylor]

On 11/02/07 15:24, Corey Hickey wrote:
> 1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it's 
> configured to route at all.
> 
> 2. The hosts on network A will receive packets from miscellaneous IP 
> addresses in B's subnet, and not have any idea what to do with them. 
> You'll need to either configure routes on the hosts in network A... 
> 
> # ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0
> 
> ...or set up box A1 to SNAT packets
> 
> # iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 -j 
> SNAT --to-source 192.168.4.10
> 
> You'll have to do something similar for the reverse direction, too.

You could do either of the above.

However there should be a route from any box on either network to 
specific boxes on the other network.  Remember that the OP configured 
routes on AR and BR to reach host(s) on Net B and Net A (respectively) 
via A1.  Thus when a host on Net B receives a packet from a host on Net 
A, it will reply via its default gateway, BR.  BR will then redirect or 
route the packet back to A1 which will then send the packet directly on 
to the original host on Net A.  This is all of course presuming that A1 
is forwarding packets like it needs to be and that there are no 
firewalls in place blocking any thing, especially reverse path filters.

> I haven't tested what I wrote above; if it doesn't work, run tcpdump 
> in various places to see what icmp packets are going where.

TCPDump / Etherial is your *FRIEND*!!!

> Honestly, though, I think you're setting yourself up for a decent 
> headache trying to do it this way. In my opinion, the easy solution 
> is in the first word of your subject line: just make host A1 a 
> bridge.  That's what I did when I originally set up that box, and it 
> worked fine the whole time I managed it (actually, I didn't manage it 
> at all, since the bridging never had any problems).

Bridging and / or proxy ARP by them selves will not solve this problem. 
  Net A and Net B have different IP address ranges, thus even with a 
bridge in place the hosts will never communicate.  In fact a bridge with 
out any thing else will just worsen the situation by passing broadcasts 
where they are not needed.

> Put both physical networks on the same IP subnet and enforce any 
> desired firewalling/segmentation with a few simple iptables rules 
> running on the bridge. If you're still using the same machine that 
> used to do that job, you might be able to find the relevant 
> configuration files lying around.  If not, I'm sure I can recall at 
> least most of the details.

Ugh.  I'm going to presume that the networks have different IP address 
ranges and are not presently joined for a reason.  Thus combining the IP 
address ranges and bridging them together will actually be a regression 
in network config / technology / evolution.

Now if you want to bridge the networks together wile still using the 
separate IP address ranges AND multi-home select boxen so that they are 
on both Net A and Net B then bridging and / or proxy ARP will work just 
fine.

> Either way, if you need assistance, let me know and I'll be happy to 
> stop by sometime and lend a hand.

You can drop a line in here too.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Bridging two subnets selectively using routing

[Corey Hickey]

Grant Taylor wrote:
> On 11/02/07 15:24, Corey Hickey wrote:
>> 1. Check /proc/sys/net/ipv4/ip_forward on box A1 to see if it's 
>> configured to route at all.
>>
>> 2. The hosts on network A will receive packets from miscellaneous IP 
>> addresses in B's subnet, and not have any idea what to do with them. 
>> You'll need to either configure routes on the hosts in network A... 
>>
>> # ip route add 10.3.0.0/16 via 192.168.4.10 dev eth0
>>
>> ...or set up box A1 to SNAT packets
>>
>> # iptables -t nat -A POSTROUTING -s 10.3.0.0/16 -d 192.168.4.0/24 -j 
>> SNAT --to-source 192.168.4.10
>>
>> You'll have to do something similar for the reverse direction, too.
> 
> You could do either of the above.

I meant to do both, which I think is necessary in order to make the OPs
proposed scheme work without modification. I'll defer if I'm wrong,
though--I haven't tested it, and, as you said in your other email, it's
"a very weird scenario."

> However there should be a route from any box on either network to 
> specific boxes on the other network.  Remember that the OP configured 
> routes on AR and BR to reach host(s) on Net B and Net A (respectively) 
> via A1.  Thus when a host on Net B receives a packet from a host on Net 
> A, it will reply via its default gateway, BR.  BR will then redirect or 
> route the packet back to A1 which will then send the packet directly on 
> to the original host on Net A.  This is all of course presuming that A1 
> is forwarding packets like it needs to be and that there are no 
> firewalls in place blocking any thing, especially reverse path filters.

I don't think this will work unless BR has a route like:
# ip route add 192.168.4.0/24 via 10.3.0.13 dev eth0
...whereas the OP only specified wanting routes to a few specific
machines rather than the whole networks.

In any case, debating that is probably academic, since I agree with you
in principle. It should be cleaner to set up routes for the whole
networks and use iptables rules on A1 to only allow traffic to/from
specified hosts.

There are certainly different ways to do it, and I furthermore agree
with you that using a separate link between AR and BR (as you suggested
in your earlier message) is cleaner still.

I prefer bridging in this situation mostly because it distributes
traffic and reduces the load on the routers.

>> I haven't tested what I wrote above; if it doesn't work, run tcpdump 
>> in various places to see what icmp packets are going where.
> 
> TCPDump / Etherial is your *FRIEND*!!!

:)

>> Honestly, though, I think you're setting yourself up for a decent 
>> headache trying to do it this way. In my opinion, the easy solution 
>> is in the first word of your subject line: just make host A1 a 
>> bridge.  That's what I did when I originally set up that box, and it 
>> worked fine the whole time I managed it (actually, I didn't manage it 
>> at all, since the bridging never had any problems).
> 
> Bridging and / or proxy ARP by them selves will not solve this problem. 
>   Net A and Net B have different IP address ranges, thus even with a 
> bridge in place the hosts will never communicate.  In fact a bridge with 
> out any thing else will just worsen the situation by passing broadcasts 
> where they are not needed.

Absolutely.

>> Put both physical networks on the same IP subnet and enforce any 
>> desired firewalling/segmentation with a few simple iptables rules 
>> running on the bridge. If you're still using the same machine that 
>> used to do that job, you might be able to find the relevant 
>> configuration files lying around.  If not, I'm sure I can recall at 
>> least most of the details.
> 
> Ugh.  I'm going to presume that the networks have different IP address 
> ranges and are not presently joined for a reason.  Thus combining the IP 
> address ranges and bridging them together will actually be a regression 
> in network config / technology / evolution.

The two networks in question are rather small and occupy adjoining
buildings. Network A had to be rebuilt after getting torn out while the
corresponding building underwent a very intrusive retrofit and
remodeling. Prior to that, the two networks were bridged and shared the
same subnet. I don't know if the OP has a reason to isolate them from
each other now.


I guess I'll go ahead and describe the former setup in a little detail.

Every host in the entire bridged network was given an IP address within
the subnet 10.0.0.0/8. The bridge was configured to drop all DHCP
packets, so there was a DHCP server on network A and another on network B.

Hosts on network A were given addresses in the following ranges[1]:
10.0.0.0/16
10.1.0.0/16
10.2.0.0/16

Hosts on network B were given addresses in the following range:
10.3.0.0/16

...but, regardless of which network a host was on, it still was given
the /8 subnet, so hosts could communicate over the bridge without any
further configuration.

Since each network had its own router to the Internet, the DHCP servers
also specified separate gateways. The bridge was configured to drop
packets with sources or destinations that didn't match the IP ranges
corresponding to the source/destination networks[2].

That's all.


My philosophy was to allow unrestricted communication over the bridge
and gently LART users that caused trouble (always inadvertently; Windows
worms and such). If the OP wants to allow communication only to a few
hosts, that's no more difficult--just write a few rules to accept
desired traffic and then drop/reject the rest.


[1] Given the chance to do it over, I would have allocated addresses to
network A from 10.0.0.0/18 and network B from 10.4.0.0/18 in order to
simplify a little bit. Also, I should mention that the use of several
/16 ranges doesn't mean we had anywhere near that many hosts; the
separation was just for management.

[2] Just in case some users on network B tried to manually set their IP
address and gateway in order to use the better Internet access of
network A. Of course, they could still have tunneled through the bridge
to an accomplice on network A, but they could have also used an
accomplice's wireless router, or CAT-5 strung between rooftops, or RFC
1149, etc. I dealt with such things on a case-by-case basis. :)



-Corey
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Bridging two subnets selectively using routing

[Grant Taylor]

On 11/2/2007 11:35 PM, Corey Hickey wrote:
> I meant to do both, which I think is necessary in order to make the 
> OPs proposed scheme work without modification. I'll defer if I'm 
> wrong, though--I haven't tested it, and, as you said in your other 
> email, it's "a very weird scenario."

As long as there are routes in both directions there should be no need 
for SNATing.

> I don't think this will work unless BR has a route like:
> 
> # ip route add 192.168.4.0/24 via 10.3.0.13 dev eth0
> 
> ...whereas the OP only specified wanting routes to a few specific 
> machines rather than the whole networks.
> 
> In any case, debating that is probably academic, since I agree with 
> you in principle. It should be cleaner to set up routes for the whole 
> networks and use iptables rules on A1 to only allow traffic to/from 
> specified hosts.

Agreed.  I mis read the routes on the two routers AR and BR to be for 
the entire networks.  Though again presuming there are routes, things 
should work.  This is more just a semantical mis-interpretation on the 
scope of what the routes are for.

> There are certainly different ways to do it, and I furthermore agree 
> with you that using a separate link between AR and BR (as you 
> suggested in your earlier message) is cleaner still.
> 
> I prefer bridging in this situation mostly because it distributes 
> traffic and reduces the load on the routers.

I can see how this would reduce load on the routes, but I don't believe 
that load on routers will be much of a concern.  (At least the routers 
that I use (pick any box (less than 10 years old) and install Linux) 
would do just fine.

However I would be concerned about broadcast storms being propagated 
across the bridge unnecessarily.  But if steps are taken to mitigate 
that then it is probably not that big of an issue.

> The two networks in question are rather small and occupy adjoining 
> buildings. Network A had to be rebuilt after getting torn out while 
> the corresponding building underwent a very intrusive retrofit and 
> remodeling. Prior to that, the two networks were bridged and shared 
> the same subnet. I don't know if the OP has a reason to isolate them 
> from each other now.

Ok...  Obviously you are probably in a very unique position knowing the 
history of the network.

> I guess I'll go ahead and describe the former setup in a little 
> detail.
> 
> Every host in the entire bridged network was given an IP address 
> within the subnet 10.0.0.0/8. The bridge was configured to drop all 
> DHCP packets, so there was a DHCP server on network A and another on 
> network B.

Ok...

> Hosts on network A were given addresses in the following ranges[1]:
> 
> 10.0.0.0/16
> 10.1.0.0/16
> 10.2.0.0/16
> 
> Hosts on network B were given addresses in the following range: 
> 
> 10.3.0.0/16
> 
> ...but, regardless of which network a host was on, it still was given 
> the /8 subnet, so hosts could communicate over the bridge without any 
> further configuration.

Ok, you chose to do in bridging what most people do in routing.  Seeing 
as how things were bridged you had to put things in place to stop things 
that would naturally leave the subnet.  Your preference to have and work 
with.

> Since each network had its own router to the Internet, the DHCP 
> servers also specified separate gateways. The bridge was configured 
> to drop packets with sources or destinations that didn't match the IP 
> ranges corresponding to the source/destination networks[2].

Ok...

> That's all.

So let me get this right, you did bridging rather than routing to avoid 
load on the router(s)?  Yet you had to put more load on the bridging 
host to segregate the networks like they would be if they were routed 
while still allowing host to host communications between the two buildings?

<personal opinion> Strange </personal opinion>

> My philosophy was to allow unrestricted communication over the bridge 
> and gently LART users that caused trouble (always inadvertently; 
> Windows worms and such). If the OP wants to allow communication only 
> to a few hosts, that's no more difficult--just write a few rules to 
> accept desired traffic and then drop/reject the rest.

Ok.

> [1] Given the chance to do it over, I would have allocated addresses 
> to network A from 10.0.0.0/18 and network B from 10.4.0.0/18 in order 
> to simplify a little bit. Also, I should mention that the use of 
> several /16 ranges doesn't mean we had anywhere near that many hosts; 
> the separation was just for management.

*nod*

> [2] Just in case some users on network B tried to manually set their 
> IP address and gateway in order to use the better Internet access of 
> network A. Of course, they could still have tunneled through the 
> bridge to an accomplice on network A, but they could have also used 
> an accomplice's wireless router, or CAT-5 strung between rooftops, or 
> RFC 1149, etc. I dealt with such things on a case-by-case basis. :)

That's what a "Clue-by-4" is used for.  ;)

All in all you chose to implement a solution in one way that very like 
did exactly what you needed even if it was a bit different than what the 
industry norm would have been.  Either way, bridging or routing, they 
both would have / do / will work.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc