2 NICs on same subnet

2 NICs on same subnet

[Sven Schuster]

[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]


Hi everybody,

I have a problem which I'm quite sure can be resolved easily, but
at the moment I'm lacking the "how" :-)
I have a machine (RH ES 3) with two Intel e1000 cards attached to
a gigabit switch, each interface is assigned an IP address on the
same subnet (say 1.2.3.4 on eth0 and 1.2.3.5 on eth1).
What's happening is that if a connection to either of these addresses
is made, it's always received via the same interface (eth1 most of the
time), and the replies go out via this interface, too. But what I'd
like is that packets to 1.2.3.4 come in via eth0 and packets to 1.2.3.5
come in via eth1 and that replies from 1.2.3.4 leave via eth0, from
1.2.3.5 via eth1.
I've already tried to resolve this issue with arptables, doing

arptables -N eth0
arptables -A eth0 -d 1.2.3.4 -j ACCEPT
arptables -A eth0 -j DROP
arptables -N eth1
arptables -A eth1 -d 1.2.3.5 -j ACCEPT
arptables -A eth1 -j DROP
arptables -A IN -i eth0 -j eth0
arptables -A IN -i eth1 -j eth1

and additionally limiting traffic in the same way by iptables, but
that doesn't work unfortunately.
Has anybody done this?? Is a solution to this problem known??


Thanks in advance!!

Sven

-- 
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
 13:30:13 up 1 day, 16:58,  1 user,  load average: 0.08, 0.02, 0.01

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2 NICs on same subnet

[Marc Haber]

On Fri, Nov 05, 2004 at 01:38:43PM +0100, Sven Schuster wrote:
> I have a machine (RH ES 3) with two Intel e1000 cards attached to
> a gigabit switch, each interface is assigned an IP address on the
> same subnet (say 1.2.3.4 on eth0 and 1.2.3.5 on eth1).

Why did you do that?

> What's happening is that if a connection to either of these addresses
> is made, it's always received via the same interface (eth1 most of the
> time), and the replies go out via this interface, too. But what I'd
> like is that packets to 1.2.3.4 come in via eth0 and packets to 1.2.3.5
> come in via eth1 and that replies from 1.2.3.4 leave via eth0, from
> 1.2.3.5 via eth1.

Why? Bandwidth issues?

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany  |  lose things."    Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature |  How to make an American Quilt | Fax: *49 621 72739835

Re: 2 NICs on same subnet

[Sven Schuster]

[-- Attachment #1: Type: text/plain, Size: 372 bytes --]


Hi Marc,

On Fri, Nov 05, 2004 at 01:41:07PM +0100, Marc Haber told us:
> Why did you do that?

please don't ask, it _has to be_ this way, not my decision :-/


Sven

> Why? Bandwidth issues?
>
> Greetings
> Marc
>

--
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
 13:50:42 up 1 day, 17:18,  1 user,  load average: 0.14, 0.06, 0.02

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

RE: 2 NICs on same subnet

[Scott Knake]

Well as far as outbound data that should be determined by your routing
table if I'm not mistaken. Maybe route -n will give you a clue but I
worked with someone on this same issue last night and never got to the
bottom of it. Be sure to post what you find :)

-Scott

-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Sven
Schuster
Sent: Friday, November 05, 2004 7:39 AM
To: netfilter@lists.netfilter.org
Subject: 2 NICs on same subnet


Hi everybody,

I have a problem which I'm quite sure can be resolved easily, but
at the moment I'm lacking the "how" :-)
I have a machine (RH ES 3) with two Intel e1000 cards attached to
a gigabit switch, each interface is assigned an IP address on the
same subnet (say 1.2.3.4 on eth0 and 1.2.3.5 on eth1).
What's happening is that if a connection to either of these addresses
is made, it's always received via the same interface (eth1 most of the
time), and the replies go out via this interface, too. But what I'd
like is that packets to 1.2.3.4 come in via eth0 and packets to 1.2.3.5
come in via eth1 and that replies from 1.2.3.4 leave via eth0, from
1.2.3.5 via eth1.
I've already tried to resolve this issue with arptables, doing

arptables -N eth0
arptables -A eth0 -d 1.2.3.4 -j ACCEPT
arptables -A eth0 -j DROP
arptables -N eth1
arptables -A eth1 -d 1.2.3.5 -j ACCEPT
arptables -A eth1 -j DROP
arptables -A IN -i eth0 -j eth0
arptables -A IN -i eth1 -j eth1

and additionally limiting traffic in the same way by iptables, but
that doesn't work unfortunately.
Has anybody done this?? Is a solution to this problem known??


Thanks in advance!!

Sven

-- 
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon
i386 GNU/Linux
 13:30:13 up 1 day, 16:58,  1 user,  load average: 0.08, 0.02, 0.01

Re: 2 NICs on same subnet

[Marc Haber]

On Fri, Nov 05, 2004 at 01:54:21PM +0100, Sven Schuster wrote:
> On Fri, Nov 05, 2004 at 01:41:07PM +0100, Marc Haber told us:
> > Why did you do that?
> 
> please don't ask, it _has to be_ this way, not my decision :-/

What you intend to do does not make sense.

This surely is meant to be a solution to some unmentioned problem. By
asking about the problem, I am just trying to give a solution.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany  |  lose things."    Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature |  How to make an American Quilt | Fax: *49 621 72739835

Re: 2 NICs on same subnet

[Sven Schuster]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]


Hi Scott,

On Fri, Nov 05, 2004 at 08:00:11AM -0500, Scott Knake told us:
> Well as far as outbound data that should be determined by your routing
> table if I'm not mistaken. Maybe route -n will give you a clue but I
> worked with someone on this same issue last night and never got to the
> bottom of it. Be sure to post what you find :)

if I find a solution I will of course post it here :-))


Sven

>
> -Scott
>

-- 
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
 14:20:54 up 1 day, 17:48,  1 user,  load average: 0.04, 0.01, 0.00

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2 NICs on same subnet

[Jason Opperisano]

On Fri, 2004-11-05 at 07:38, Sven Schuster wrote:
> Hi everybody,
> 
> I have a problem which I'm quite sure can be resolved easily,

wrong.

>  but
> at the moment I'm lacking the "how" :-)
> I have a machine (RH ES 3) with two Intel e1000 cards attached to
> a gigabit switch, each interface is assigned an IP address on the
> same subnet (say 1.2.3.4 on eth0 and 1.2.3.5 on eth1).

no. no. no. no.

> What's happening is that if a connection to either of these addresses
> is made, it's always received via the same interface (eth1 most of the
> time), and the replies go out via this interface, too. But what I'd
> like is that packets to 1.2.3.4 come in via eth0 and packets to 1.2.3.5
> come in via eth1 and that replies from 1.2.3.4 leave via eth0, from
> 1.2.3.5 via eth1.
> I've already tried to resolve this issue with arptables, doing
> 
> arptables -N eth0
> arptables -A eth0 -d 1.2.3.4 -j ACCEPT
> arptables -A eth0 -j DROP
> arptables -N eth1
> arptables -A eth1 -d 1.2.3.5 -j ACCEPT
> arptables -A eth1 -j DROP
> arptables -A IN -i eth0 -j eth0
> arptables -A IN -i eth1 -j eth1
> 
> and additionally limiting traffic in the same way by iptables, but
> that doesn't work unfortunately.
> Has anybody done this?? Is a solution to this problem known??

yeah--don't assign two interface on the same box IP's in the same
subnet.

that being said--i've seen this done successfully *once*.  the
"solution" was that every host in the network had to run OSPF.  it was
the only way to get packets to go in and out the same interface for each
IP.

saying "it _has to be_ this way, not my decision" is a cop out.  it will
result in answers like:

well, it can't be done, so too bad.
-or-
if it's not your decision, the it shouldn't be your problem.

-j 

--
"When I grow up, I'm going to Bovine University!"
	--The Simpsons

Re: 2 NICs on same subnet

[Sven Schuster]

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]


Hi Marc,

On Fri, Nov 05, 2004 at 02:04:24PM +0100, Marc Haber told us:
> What you intend to do does not make sense.
>
> This surely is meant to be a solution to some unmentioned problem. By
> asking about the problem, I am just trying to give a solution.

well, I know that it doesn't make much sense. This setup was made to
"balance" the traffic coming in to the machine. I know that there's
not much  difference between one NIC running at, say 400 Mbps, or
two NICs running at 200 Mbps each which will also be 400 Mbps, because
the machine can't handle more traffic (data (backups) is received via
network from other machines and written to disks)...but as I've already
written, this wasn't my decision...


Sven


>
> Greetings
> Marc
>

--
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
 14:21:46 up 1 day, 17:49,  1 user,  load average: 0.02, 0.01, 0.00

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2 NICs on same subnet

[Aleksandar Milivojevic]

Sven Schuster wrote:
> Hi everybody,
> 
> I have a problem which I'm quite sure can be resolved easily, but
> at the moment I'm lacking the "how" :-)
> I have a machine (RH ES 3) with two Intel e1000 cards attached to
> a gigabit switch, each interface is assigned an IP address on the
> same subnet (say 1.2.3.4 on eth0 and 1.2.3.5 on eth1).
> What's happening is that if a connection to either of these addresses
> is made, it's always received via the same interface (eth1 most of the
> time), and the replies go out via this interface, too. But what I'd
> like is that packets to 1.2.3.4 come in via eth0 and packets to 1.2.3.5
> come in via eth1 and that replies from 1.2.3.4 leave via eth0, from
> 1.2.3.5 via eth1.
> I've already tried to resolve this issue with arptables, doing

Probably not easilly solvable.  The thing is that both interfaces are 
valid for sending responses out.  So the kernel will probably use the 
first where it gets the match (not sure about exact algorithm, so it may 
as well be the last, or random).

Anyhow, said that, there are couple of things to try out.

For incomming packets, I do remember that there's kernel option to 
instruct Linux kernel to send ARP responses from matching interface, 
instead of interface ARP request was made on (ARP request is broadcast, 
so both interfaces will get it, and by default Linux doesn't send from 
"matching" interface).  This way, client will get correct MAC address, 
and switch will send packets down the correct wire.  Might help, but on 
the other hand it might not.

For outgoing packets, mark packets in INPUT chain of mangle table with 
different marks depending if source is 1.2.3.4 or 1.2.3.5.  Than you can 
use fwmark with Linux Advanced Routing (see examples in howto, 
http://www.lartc.org/lartc.txt, chapter 11) to route packets to eth0 or 
eth1 depending on the fwmark.  Should work (tm).

-- 
Aleksandar Milivojevic <amilivojevic@pbl.ca>    Pollard Banknote Limited
Systems Administrator                           1499 Buffalo Place
Tel: (204) 474-2323 ext 276                     Winnipeg, MB  R3T 1L7

Re: 2 NICs on same subnet

[Aleksandar Milivojevic]

Aleksandar Milivojevic wrote:
> For outgoing packets, mark packets in INPUT chain of mangle table with 

Oooops, typo.  Should be OUTPUT chain of mangle table, of course...

-- 
Aleksandar Milivojevic <amilivojevic@pbl.ca>    Pollard Banknote Limited
Systems Administrator                           1499 Buffalo Place
Tel: (204) 474-2323 ext 276                     Winnipeg, MB  R3T 1L7

Re: 2 NICs on same subnet

[Lumberjack]

>well, I know that it doesn't make much sense. This setup was made to
>"balance" the traffic coming in to the machine. I know that there's
>not much  difference between one NIC running at, say 400 Mbps, or
>two NICs running at 200 Mbps each which will also be 400 Mbps, because
>the machine can't handle more traffic (data (backups) is received via
>network from other machines and written to disks)...but as I've already
>written, this wasn't my decision...

Your decision or not if you are the one to "implement the fix" then you 
should
suggest something that will work and not try to get something else to work 
that
won't work.  At that data rate you would probably have more issues with disk
I/O anyway so a load balancer sounds like a better solution.

Good luck. 

Re: 2 NICs on same subnet

[Jason Opperisano]

On Fri, Nov 05, 2004 at 03:22:19PM +0100, Sven Schuster wrote:
> 
> Hi Marc,
> 
> On Fri, Nov 05, 2004 at 02:04:24PM +0100, Marc Haber told us:
> > What you intend to do does not make sense.
> >
> > This surely is meant to be a solution to some unmentioned problem. By
> > asking about the problem, I am just trying to give a solution.
> 
> well, I know that it doesn't make much sense. This setup was made to
> "balance" the traffic coming in to the machine. I know that there's
> not much  difference between one NIC running at, say 400 Mbps, or
> two NICs running at 200 Mbps each which will also be 400 Mbps, because
> the machine can't handle more traffic (data (backups) is received via
> network from other machines and written to disks)...but as I've already
> written, this wasn't my decision...

see--telling us what the actual goal is will help people make
suggestions that work that will meet that goal, rather than trying to
pigeon-hole us into trying implement your thought for the "only" way to
solve this.

have you looked into the bonding driver?  two ethernet connections, link
aggregation (802.3ad), high availability, one IP address:

  http://linux-ip.net/html/ether-bonding.html

-j

--
"Son, when you participate in sporting events, it's not whether you
 win or lose: it's how drunk you get."
        --The Simpsons

Re: 2 NICs on same subnet

[John A. Sullivan III]

On Fri, 2004-11-05 at 09:22, Sven Schuster wrote:
> Hi Marc,
> 
> On Fri, Nov 05, 2004 at 02:04:24PM +0100, Marc Haber told us:
> > What you intend to do does not make sense.
> >
> > This surely is meant to be a solution to some unmentioned problem. By
> > asking about the problem, I am just trying to give a solution.
> 
> well, I know that it doesn't make much sense. This setup was made to
> "balance" the traffic coming in to the machine. I know that there's
> not much  difference between one NIC running at, say 400 Mbps, or
> two NICs running at 200 Mbps each which will also be 400 Mbps, because
> the machine can't handle more traffic (data (backups) is received via
> network from other machines and written to disks)...but as I've already
> written, this wasn't my decision...
<snip>
Unfortunately, I don't have the time right now to think this challenging
issue all the way through but I'll share my first thoughts.  As they are
first thoughts, they may be completely worthless :-)

Like Alexander, I would think iproute2 could be your friend.  I am under
the impression that some of its features are explicitly to load balance
across multiple NICs on the same interface but I don't recall what gave
me that impression.

You may find some other options outside of routing.  If you have
specific services, you may be able to bind those services to a
particular address.  For lack of a better example, let's assume we bind
Apache to 1.2.3.4 and an Asterisk IP PBX to 1.2.3.5.  You could set up
two different DNS entries for the two sets of services:
web.mycompany.com is 1.2.3.4
pbx.mycompany.com is 1.2.3.5

I'm not sure if that will confuse anything.  Just some raw thoughts.  I
hope they help - John


-- 
John A. Sullivan III
Chief Technology Officer
Nexus Management
+1 207-985-7880
john.sullivan@nexusmgmt.com
---
If you are interested in helping to develop a GPL enterprise class
VPN/Firewall/Security device management console, please visit
http://iscs.sourceforge.net 

Re: 2 NICs on same subnet

[a.ledvinka]

netfilter-bounces@lists.netfilter.org wrote on 05.11.2004 15:47:14:

> Probably not easilly solvable.  The thing is that both interfaces are 
> valid for sending responses out.  So the kernel will probably use the 
> first where it gets the match (not sure about exact algorithm, so it may 

> as well be the last, or random).

check the iproute2 package (program called "ip") and look at the grammar 
in "ip route help" output. there is way to assign "source" ip per route. 
so if you use the system for backups (initiated by the host in question) 
then it should work if you can't use binding.

btw is not this thread a little offtopic on this list then? ;-)

example:
here is what i used some time ago for nat box to set it's ip traffic the 
only allowed source address on firewall short after booting.
ip route change default via 212.96.166.17 dev eth1 src 212.96.166.18
you just substitute the "default" which is alias for 0.0.0.0 and also 
"eth1" for wahtever you want.

you possibly need 2.2+ kernel (which is true with iptables) and not sure 
if advanced routing enabled in kernel (i guess not because this is 
somewhat trivial function)

RE: 2 NICs on same subnet

[Daniel Chemko]

Ok, here's my two cents. There IS a solution, I can feel it. I'm just
going to work through it through talking, and hope for the best. (Note,
Although not totally in topic, the solution is in fact)

1. You're getting inbound responses on the wrong NIC because you're
sending IP requests from the wrong device. The Switch thinks that NIC 1
does have IP #2 (AKA: Accidental ARP poison). 
2. I'm assuming you're using this for either an extra security mechanism
or for a backup line
3. I'm assuming that the machine is standalone and is not hosting
services behind it. Lets just throw that complexity out the window.

So, the real thing you need to do is FORCE packets to/from IP1 in/out of
IF1 and packets to/from IP2 in/out of IF2.


Solution Overview

1. Application Layer
When you start your application, you must bind inbound/outbound traffic
to a specific port. If your software doesn't support this, you could
setup a local proxy server to do it.

2. Firewall Layer
When packets sourced as either IP1 or IP2 come into the OUTPUT chain,
mark it accordingly.

3. Routing Layer
Read the mark on the packets coming down the pipe (Firewall OUTPUT) and
direct the packets out the correct interface

4. Datalink Layer 
We must flush out all ARP caches on any devices that may have been
poisoned after everything else has been complete. This usually means
turning off/on the network switch.


Implementation details

Step 1. Routing Tables
Your routing tables should probably look like the following if it was
setup default.

192.168.1.0/24 dev eth0  scope link 
192.168.1.0/24 dev eth1  scope link 
127.0.0.0/8 dev lo  scope link 
default via 192.168.1.254 dev eth0 
default via 192.168.1.254 dev eth1

As you can see here, there's nothing FORCING the packets to go out
either of the interfaces. Lets create a couple tables that will FORCE
the traffic to one interface or the other.

# Create a couple tables that will match against firewall MARK rules and
direct them to new tables
   ip rule add table 1 fwmark 1
   ip rule add table 2 fwmark 2

# Create the new tables to have 1 single route (I'm assuming there are
no other routes!)
   ip route add default dev ${IF_1} table 1 src ${SRC_IP_1}
   ip route add default dev ${IF_2} table 2 src ${SRC_IP_2}

All done. That takes care of the routing. Now to activate the routing
code: Firewalls

Step 2. Firewall Layer

In order for all this to work, we need to mark packets that come in/out
of the system on the correct ports. Since we don't have any other
routes, I'm going to make this simple. If you want to be totally
correct, I think the CONNMARK rule would be better, but worry about it
after everything already works.

# Mark packets from a source address to a specific MARK
   iptables -t mangle -A OUTPUT --source ${SRC_IP_1} -j MARK 1
   iptables -t mangle -A OUTPUT --source ${SRC_IP_2} -j MARK 2
# If we get here without hitting one of the above rules, I'm going to
take a default route from the 'main' routing table

# Make sure that no matter what don't send packets with the wrong
routing out unfettered
   iptables -t nat -A POSTROUTING -o ${IF_1} -j SNAT --to ${IP_1}
   iptables -t nat -A POSTROUTING -o ${IF_2} -j SNAT --to ${IP_2}


Step 3. Application Layer

Notice that in the firewall layer it used the --source to determine what
MARK to assign. Any normal connection coming from the socket layer
generally doesn't set the internet address source of the connection. If
you know socket coding, you usually call socket then connect when a
client connects to a server. Neither of these commands specify the
IP/Port that you are connecting FROM. If your tool doesn't allow you to
specify a src ip/port to connect from, you have issues.

One workaround that I came across was Squid Proxy. It does allow you to
hard code the source address of the external connection by specifying
tcp_outgoing_address. If your tool allows you to specify a proxy server,
this could be your solution. If your tool doesn't allow you to specify
the proxy / src ip of the connection, you may need to explore other
alternatives.

Re: 2 NICs on same subnet

[Sven Schuster]

[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]


Hi Daniel,

On Fri, Nov 05, 2004 at 10:45:45AM -0800, Daniel Chemko told us:
> Ok, here's my two cents. There IS a solution, I can feel it. I'm just
> going to work through it through talking, and hope for the best. (Note,
> Although not totally in topic, the solution is in fact)
> 
> 1. You're getting inbound responses on the wrong NIC because you're
> sending IP requests from the wrong device. The Switch thinks that NIC 1
> does have IP #2 (AKA: Accidental ARP poison). 
> 2. I'm assuming you're using this for either an extra security mechanism
> or for a backup line
> 3. I'm assuming that the machine is standalone and is not hosting
> services behind it. Lets just throw that complexity out the window.
> 
> So, the real thing you need to do is FORCE packets to/from IP1 in/out of
> IF1 and packets to/from IP2 in/out of IF2.
> 

thanks for your and everybody's else's tipps and discussion on this topic,
I already wanted to try the iproute2 based solution today but didn't have
enough time to do this. I'll try tomorrow and come back here and tell you
about my success (or not :-).
But I also think about going to my boss and tell him that this whole idea
is crap as we won't be able to handle more input no matter if it comes in
from one interface or two...I think the best solution would be to let the
second NIC stay down until the moment the first one won't work any more.


Once again, thanks to everybody!!!

Sven

-- 
Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
 20:09:17 up 1 day, 23:37,  1 user,  load average: 0.03, 0.05, 0.01

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2 NICs on same subnet

[Jorge Nerín]

I think your solution it's in
http://lartc.org/howto/lartc.rpdb.multiple-links.html, let me
copy&paste.

---8<------------------
                                                                 ________
                                          +------------+        /
                                          |            |       |
                            +-------------+ Provider 1 +-------
        __                  |             |            |     /
    ___/  \_         +------+-------+     +------------+    |
  _/        \__      |     if1      |                      /
 /             \     |              |                      |
| Local network -----+ Linux router |                      |     Internet
 \_           __/    |              |                      |
   \__     __/       |     if2      |                      \
      \___/          +------+-------+     +------------+    |
                            |             |            |     \
                            +-------------+ Provider 2 +-------
                                          |            |       |
                                          +------------+        \________


4.2.1. Split access

The first is how to route answers to packets coming in over a
particular provider, say Provider 1, back out again over that same
provider.

Let us first set some symbolical names. Let $IF1 be the name of the
first interface (if1 in the picture above) and $IF2 the name of the
second interface. Then let $IP1 be the IP address associated with $IF1
and $IP2 the IP address associated with $IF2. Next, let $P1 be the IP
address of the gateway at Provider 1, and $P2 the IP address of the
gateway at provider 2. Finally, let $P1_NET be the IP network $P1 is
in, and $P2_NET the IP network $P2 is in.

One creates two additional routing tables, say T1 and T2. These are
added in /etc/iproute2/rt_tables. Then you set up routing in these
tables as follows:

	  ip route add $P1_NET dev $IF1 src $IP1 table T1
	  ip route add default via $P1 table T1
	  ip route add $P2_NET dev $IF2 src $IP2 table T2
	  ip route add default via $P2 table T2
	

Nothing spectacular, just build a route to the gateway and build a
default route via that gateway, as you would do in the case of a
single upstream provider, but put the routes in a separate table per
provider. Note that the network route suffices, as it tells you how to
find any host in that network, which includes the gateway, as
specified above.

Next you set up the main routing table. It is a good idea to route
things to the direct neighbour through the interface connected to that
neighbour. Note the `src' arguments, they make sure the right outgoing
IP address is chosen.

	    ip route add $P1_NET dev $IF1 src $IP1
	    ip route add $P2_NET dev $IF2 src $IP2
	  

Then, your preference for default route:

	    ip route add default via $P1
	  

Next, you set up the routing rules. These actually choose what routing
table to route with. You want to make sure that you route out a given
interface if you already have the corresponding source address:

	    ip rule add from $IP1 table T1
	    ip rule add from $IP2 table T2
	  

This set of commands makes sure all answers to traffic coming in on a
particular interface get answered from that interface.

---->8-----------------

That's all, I have tried it in exactly the same way it's depiected,
with two internet providers, and it woks ok, you have a default router
for your generated trafic, but you answer to external petitions using
the incoming interface.

I think it can be massaged to suit your case, good luck.

On Fri, 5 Nov 2004 20:16:21 +0100, Sven Schuster <schuster.sven@gmx.de> wrote:
> 
> Hi Daniel,
> 
> On Fri, Nov 05, 2004 at 10:45:45AM -0800, Daniel Chemko told us:
> 
> 
> > Ok, here's my two cents. There IS a solution, I can feel it. I'm just
> > going to work through it through talking, and hope for the best. (Note,
> > Although not totally in topic, the solution is in fact)
> >
> > 1. You're getting inbound responses on the wrong NIC because you're
> > sending IP requests from the wrong device. The Switch thinks that NIC 1
> > does have IP #2 (AKA: Accidental ARP poison).
> > 2. I'm assuming you're using this for either an extra security mechanism
> > or for a backup line
> > 3. I'm assuming that the machine is standalone and is not hosting
> > services behind it. Lets just throw that complexity out the window.
> >
> > So, the real thing you need to do is FORCE packets to/from IP1 in/out of
> > IF1 and packets to/from IP2 in/out of IF2.
> >
> 
> thanks for your and everybody's else's tipps and discussion on this topic,
> I already wanted to try the iproute2 based solution today but didn't have
> enough time to do this. I'll try tomorrow and come back here and tell you
> about my success (or not :-).
> But I also think about going to my boss and tell him that this whole idea
> is crap as we won't be able to handle more input no matter if it comes in
> from one interface or two...I think the best solution would be to let the
> second NIC stay down until the moment the first one won't work any more.
> 
> Once again, thanks to everybody!!!
> 
> Sven
> 
> --
> Linux zion 2.6.9-rc1-mm4 #1 Tue Sep 7 12:57:19 CEST 2004 i686 athlon i386 GNU/Linux
>  20:09:17 up 1 day, 23:37,  1 user,  load average: 0.03, 0.05, 0.01
> 
> 
> 


-- 
Jorge Nerín
<jnerin@gmail.com>

Re: 2 NICs on same subnet

[George Alexandru Dragoi]

     I think the best solution is to use bonding, if the switch
supports it (Yea, the switch MUST have this). This way you-ll have a
single mac, and the balance for inbound will be made by the switch. I
think everybody knows you can't control what is comming, but the sitch
will be able to controll the outgoing. If the switch knows bonding,
then a single ip should be enough.