ARP problem?

ARP problem?

[Thompson, Ian]

Hi,

I'm seeing some odd behavior in RedHat 7.3 when handling ARP packets.  I
have two Intel NIC cards, eth0 and eth1, in one machine, connected to the
same switch.  eth0 is set to IP0 and has MAC addr M0, and eth1 is at IP1 and
MAC M1.  Now, if another machine connected to the switch sends an ARP
broadcast asking who is at IP0, I see two replies on the wire -- IP0 is at
M0, and IP0 is at M1.  This result seems contradictory to me; could it be
some sort of feature that I'm not aware of?  If so, can I disable it?

I am trying to devlop some code to support an active failover case, so I
want two seperate devices on the same physical network.  I have seen the
same result even if IP0 and IP1 are on different subnets, or even if one is
a class A and the other is a class C address.  

I'm sorry if this has already been discussed -- I haven't seen much relating
to it in the archives.

TIA,
-ian

---
Ian Thompson               Firmware Engineer
Adaptec, Inc               Storage Networking Group
408.957.4909               408.957.6800 (fax)
ian_thompson@adaptec.com

Re: ARP problem?

[Ben Greear]

Thompson, Ian wrote:
> Hi,
> 
> I'm seeing some odd behavior in RedHat 7.3 when handling ARP packets.  I
> have two Intel NIC cards, eth0 and eth1, in one machine, connected to the
> same switch.  eth0 is set to IP0 and has MAC addr M0, and eth1 is at IP1 and
> MAC M1.  Now, if another machine connected to the switch sends an ARP
> broadcast asking who is at IP0, I see two replies on the wire -- IP0 is at
> M0, and IP0 is at M1.  This result seems contradictory to me; could it be
> some sort of feature that I'm not aware of?  If so, can I disable it?
> 
> I am trying to devlop some code to support an active failover case, so I
> want two seperate devices on the same physical network.  I have seen the
> same result even if IP0 and IP1 are on different subnets, or even if one is
> a class A and the other is a class C address.  
> 
> I'm sorry if this has already been discussed -- I haven't seen much relating
> to it in the archives.

You need arp-filtering:

     # Set up arp-filter magic.  This, with source-based routing allows us
     # to have multiple NICs on the same subnet, on the same machine, connected
     # to the same switch...
     if [ -f  /proc/sys/net/ipv4/conf/all/arp_filter ];
     then
	echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
     else
	echo "ERROR: kernel does not support arp_filter.  Don't put more than"
	echo "       one interface on the same subnet on the same machine."
	echo ""
     fi


> 
> TIA,
> -ian
> 
> ---
> Ian Thompson               Firmware Engineer
> Adaptec, Inc               Storage Networking Group
> 408.957.4909               408.957.6800 (fax)
> ian_thompson@adaptec.com
> 
> 


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: ARP problem?

[Julian Anastasov]

	Hello,

On Wed, 16 Oct 2002, Thompson, Ian wrote:

> I am trying to devlop some code to support an active failover case, so I
> want two seperate devices on the same physical network.  I have seen the

	May be you need support for alternative non-default routes:

http://www.ssi.bg/~ja/#routes

	These patches will make the routing to use the two link
routes for your subnet (not only the first one), I assume you are
worrying about problems with the two links to the hub.

> TIA,
> -ian

Regards

--
Julian Anastasov <ja@ssi.bg>

RE: ARP problem?

[Thompson, Ian]

> 
> You need arp-filtering:
> 
>      # Set up arp-filter magic.  This, with source-based 
> routing allows us
>      # to have multiple NICs on the same subnet, on the same 
> machine, connected
>      # to the same switch...
>      if [ -f  /proc/sys/net/ipv4/conf/all/arp_filter ];
>      then
> 	echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
>      else
> 	echo "ERROR: kernel does not support arp_filter.  Don't 
> put more than"
> 	echo "       one interface on the same subnet on the 
> same machine."
> 	echo ""
>      fi
> 

I tried this, and now I'm getting only one ARP response.  However, I get the
same MAC address for ARP broadcasts for either IP address.  Does ARP
filtering turn off all but the first interface when processing ARP packets?
Can I get each interface to answer ARP packets only for its specific IP
address?

Thanks,
-ian

Re: ARP problem?

[Ben Greear]

Thompson, Ian wrote:
>>You need arp-filtering:
>>
>>     # Set up arp-filter magic.  This, with source-based 
>>routing allows us
>>     # to have multiple NICs on the same subnet, on the same 
>>machine, connected
>>     # to the same switch...
>>     if [ -f  /proc/sys/net/ipv4/conf/all/arp_filter ];
>>     then
>>	echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
>>     else
>>	echo "ERROR: kernel does not support arp_filter.  Don't 
>>put more than"
>>	echo "       one interface on the same subnet on the 
>>same machine."
>>	echo ""
>>     fi
>>
> 
> 
> I tried this, and now I'm getting only one ARP response.  However, I get the
> same MAC address for ARP broadcasts for either IP address.  Does ARP
> filtering turn off all but the first interface when processing ARP packets?
> Can I get each interface to answer ARP packets only for its specific IP
> address?

Try setting up source-based routing.  Here is a snippet of perl code that
does that, but it will be difficult for you to decipher out of context:

e_if is a list of interfaces (ie eth2)
e_ip is the IP for this interface
sigb is the significant bits, ie the 24 in 192.168.2.0/24
e_tbl is the table name, you need a table for each interface.


       print "# Setup for device: $e_if[$i]  IP:  $e_ip[$i]  sig-bits: $e_sigb[$i]\n";
       printAndExec("ip link set $e_if[$i] down");
       printAndExec("ip link set $e_if[$i] up");
       printAndExec("ip addr flush dev $e_if[$i]");
       if ($e_ip[$i] ne "0.0.0.0") {
	printAndExec("ip address add $e_ip[$i]/$e_sigb[$i] broadcast $e_bcast[$i] dev $e_if[$i]");
       }
       printAndExec("ip link set dev $e_if[$i] up");
       if ($e_ip[$i] ne "0.0.0.0") {
	printAndExec("ip ru add from $e_ip[$i]/32 table $e_tbl[$i]");
	printAndExec("ip route add $e_sub[$i]/$e_sigb[$i] via $e_ip[$i] table $e_tbl[$i]");
       }
       if ($e_gw[$i] ne "0.0.0.0") {
	printAndExec("ip route add 0/0 via $e_gw[$i] dev $e_if[$i] table $e_tbl[$i]");
       }

You can use this to give you ideas of what to look for as you read one of the
advanced-routing HOWTOs.  With source-based routing and arp-filtering, I have
gotten many interfaces on the same subnet to work as you would expect, so it
can be done :)

Ben


> 
> Thanks,
> -ian
> 


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear