Re: arp corrupted

[Doug Knight <doug.knight@uaf.edu>]

Bobby,

A long question merits a long response.

I suspect that your client's gateway is 192.168.0.254, so your PREROUTING
rules will be screwing up any traffic that uses tcp ports 80, 25, 21, or
110.  You should specify the destination address ( -d 192.168.0.254 ) in
the reroute rules so that you are not changin ports in traffic destined
for the outside world.

Of course, I could be way off.  If you're still having problems, try the
following:

# List your current (not working) ARP cache with
ip neigh show

# Clear your arp cache
ip neigh flush all

# Try to find 192.168.1.254 on eth0
arping -c 1 -I eth0 192.168.1.254

# Try to find 192.168.0.11 on eth1
arping -c 1 -I eth1 192.168.0.11

# List your ARP cache again
ip neigh show

# Clear your arp cache again
ip neigh flush all

# Try to ping 192.168.1.254
ping -c 1 192.168.1.254

# Try to ping 192.168.0.11
ping -c 1 192.168.0.11

# List your ARP cache again
ip neigh show

As long as it keeps finding 192.168.0.11 on dev eth1 and
192.168.1.254 on eth0, there is nothing wrong with ARP on
your network.

Otherwise, do an `ip rule show', then `ip route show
table <table-name/number>' for local, main, default, and any
other routing tables listed.  This will provide more detailed
information than you are getting with netstat.  Post your
results, with commands and the information above, answer the
following questions, and we'll see if we can figure anything
else for you.

1) What distribution of linux are you running?

2) What version of the kernel are you running?

3) What were the commands you used to obtain the information you
   listed in your original post?

4) What were you doing when you generated the arp who-has request?

5) What are the MAC addresses for each of the interfaces?

6) What do corp.domain.com and dell11.domain.com resolve to?
   (simply not reversing addresses would be easier to work with)

Doug Knight

Bobby wrote:
> Hi,
> 
> I've been trying to get a couple of routers up after h/w failure.
> 
> The border router is an OpenBSD firewall running NAT between the Internet and 
> a DMZ like subnet, and in that a Linux antivirus server is running NAT to the 
> LAN. 
> 
> When the client does a DNS query it reaches to the f/w where dns is running 
> and is returned into the A/V server but never hits the 0.254 interface. 
> (Shown by running tcpdump on each interface.)
> 
> (Internet)
> 	|
> OpenBSD Firewall 
> NAT 192.168.1.254
> 	|
> 192.168.1.253 eth0
> Anti-Virus in "DMZ"
> 192.168.0.254 eth1
> 	|
> 192.168.0.11
> Client on LAN
> 
> The routing table on the A/V server is:
>  192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0
>  192.168.0.0     192.168.0.254   255.255.255.0   UG        0 0          0 eth1
>  192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
>  169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth1
>  0.0.0.0         192.168.1.254   0.0.0.0         UG        0 0          0 eth0
> 
> Sitting on the A/V server one can reach the client without problem.
> 
> Arp shows:
>  Address             HWtype  HWaddress           Flags Mask            Iface
>  192.168.1.254        ether   00:20:78:0F:AC:31   C                     eth0
> 
> Unless I try to reach the client web server from the A/Vserver, then it fails 
> and arp says:
> 
>  Address              HWtype  HWaddress           Flags Mask            Iface
>  corp.domain.com          (incomplete)                              eth0
>  dell11.domain.com  ether   00:06:29:AF:A3:67   C                 eth1
>  192.168.1.254            ether   00:20:78:0F:AC:31   C                   eth0
> 
> One can also see the arp requests go out on eth 0 rather than eth1:
> 
>  arp who-has 192.168.0.10 tell 192.168.0.254
> 
> Pinging works well:
> 
>  PING 192.168.0.11 (192.168.0.11) 56(84) bytes of data.
>  64 bytes from 192.168.0.11: icmp_seq=0 ttl=64 time=0.277 ms
> 
> Iptables on the A/V server says:
> 
> *nat
> :PREROUTING ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j REDIRECT --to-port 9080
> -A PREROUTING -i eth1 -p tcp -m tcp --dport 110 -j REDIRECT --to-port 9110
> -A PREROUTING -i eth1 -p tcp -m tcp --dport 25 -j REDIRECT --to-port 9025
> -A PREROUTING -i eth1 -p tcp -m tcp --dport 21 -j REDIRECT --to-port 9021
> COMMIT
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :Firewall-INPUT - [0:0]
> -A INPUT -i lo -j ACCEPT
> -A INPUT -i eth0 -j ACCEPT
> COMMIT
> 
> It is supposed to route all outbound traffic through the various ports where 
> the A/V s/w is listening.
> 
> ifconfig shows:
> 
> eth0      Link encap:Ethernet  HWaddr 00:0D:88:39:6A:F1
>           inet addr:192.168.1.253  Bcast:192.168.1.255  Mask:255.255.255.0
>           inet6 addr: fe80::20d:88ff:fe39:6af1/64 Scope:Link
>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>           RX packets:77670 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:97635 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:523 txqueuelen:1000
>           RX bytes:22858238 (21.7 MiB)  TX bytes:21513745 (20.5 MiB)
>           Interrupt:11 Base address:0x2400
> 
> eth1      Link encap:Ethernet  HWaddr 00:50:FC:AC:52:4B
>           inet addr:192.168.0.254  Bcast:192.168.0.255  Mask:255.255.255.0
>           inet6 addr: fe80::250:fcff:feac:524b/64 Scope:Link
>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>           RX packets:26676 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:20424 errors:0 dropped:0 overruns:1 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:2747567 (2.6 MiB)  TX bytes:26324006 (25.1 MiB)
>           Interrupt:7 Base address:0x2800
> 
> If NAT did not work I could see it having a problem. It appears to be happy 
> routing icmp but not tcp. cat /proc/sys/net/ipv4/ip_forward shows 1 which it 
> would have to anyway since the request goes out from the LAN.
> 
>