hiding arp for server farms

hiding arp for server farms

[Adrian Daminato]

Okay, I've seen similar posts to this, but none of them provide a solution that
I can use.

I'm running several 2.2 machines behind a Radware load balancer, which uses
something called "local triangulation".  Basically the Radware responds to ARP
requests for the IP of the farm, passes the packet to one of the servers, and
the server responds directly to the client.  Each server has an aliased
interface on the loopback for the IP of the farm, and
/proc/sys/net/ipv4/conf/all/hidden and lo/hidden are set to 1.  That works,
great, no problems.

Now, introduce an unpatched 2.4.x kernel.  The hidden option no longer exists,
and for ease of operating a production environment, we prefer to use stock
kernels straight from kernel.org, no patches at all.  I've tried many different
suggestion from the list:

1) ifconfig eth0 -arp
    We have over 60 servers on the subnet these farms are on, and they need to
be able to communicate with each other.  When I do this, I can't talk to other
servers on the network, and keeping an /etc/ethers file up to date is a daunting
task, and not practical.

2) arp_filter
    I tried using it in a couple of ways, but there doesn't appear to be very
good documentation.  I was hoping this would provide the same functionality of
the hidden in the 2.2 kernels for our current setup, but it doesn't appear to

3) I even tried adding the 'hidden' patch available, to put the hidden
functionality back in the 2.4.x kernel (currently I'm testing using a 2.4.9
kernel).  It doesn't appear to work properly either, hosts on the local network
can't ping the server farm, and hosts outside the network although able to ping
the server farm, cannot ping the real IP of the host.  It's kind of a weird
problem.

Is there any way to have this work on an unpatched 2.4.x kernel?  Any
documentation/examples for arp_filter, how it works, how it can be implemented
for this?

Any help would be appreciated.  Thanks.
-- 
Adrian Daminato 
Tucows International Corp.
http://www.tucows.com
Tel: (416) 535-0123
Fax: (416) 531-5584

Beauty awakens the soul to act.
                 - Dante

Re: hiding arp for server farms

[Andi Kleen]

Adrian Daminato <adrian@tucows.com> writes:

> Now, introduce an unpatched 2.4.x kernel.  The hidden option no longer exists,
> and for ease of operating a production environment, we prefer to use stock
> kernels straight from kernel.org, no patches at all.  I've tried many different
> suggestion from the list:
> 
> 1) ifconfig eth0 -arp

It'll make the stack no put any mac addresses into your packets,
which is likely not what you want.

> 
> Is there any way to have this work on an unpatched 2.4.x kernel?  Any
> documentation/examples for arp_filter, how it works, how it can be implemented
> for this?

arp_filter was not really designed to fix such a br^wweird setup.

It is possible to do it but a bit ugly. Basically you have to express a policy
filter rule/route that matches the outgoing ARP, but not the data and make the arp
route a blackhole route. The kernel unfortunately has no special key to select
ARP, so it has to be expressed in some other way (e.g. mark rules etc.), which is 
usually possible, but ugly. 

Your problems in (3) is that you asked for ARP to be turned off which
obviously breaks things if noone else (like your load balancing monstrosity) does 
the ARP for you. IIRC the hidden guys usually work around this by using a 
separate hidden virtual interface and only use that for load balancing purposes.
In the end it gets similarly ugly as the arp_filter setup.

-Andi

P.S.: I would not recommend 2.4.9 unpatched for any production setup. 

Re: hiding arp for server farms

[ZipKid]

Adrian Daminato wrote:

>Okay, I've seen similar posts to this, but none of them provide a solution that
>I can use.
>
>I'm running several 2.2 machines behind a Radware load balancer, which uses
>something called "local triangulation".  Basically the Radware responds to ARP
>requests for the IP of the farm, passes the packet to one of the servers, and
>the server responds directly to the client.  Each server has an aliased
>interface on the loopback for the IP of the farm, and
>/proc/sys/net/ipv4/conf/all/hidden and lo/hidden are set to 1.  That works,
>great, no problems.
>
>Now, introduce an unpatched 2.4.x kernel.  The hidden option no longer exists,
>and for ease of operating a production environment, we prefer to use stock
>kernels straight from kernel.org, no patches at all.  I've tried many different
>suggestion from the list:
>
>1) ifconfig eth0 -arp
>    We have over 60 servers on the subnet these farms are on, and they need to
>be able to communicate with each other.  When I do this, I can't talk to other
>servers on the network, and keeping an /etc/ethers file up to date is a daunting
>task, and not practical.
>
>2) arp_filter
>    I tried using it in a couple of ways, but there doesn't appear to be very
>

I have tested this setup and had the same problems on a 2.4.12 kernel. I 
tried out a few things
and could not resolve this issue. Fortunately for me the client is 
running solaris and that does
not have this bug.
Sorry but I have no solution...

Stefan - ZipKid - Goethals

Re: hiding arp for server farms

[Julian Anastasov]

	Hello,

Adrian Daminato wrote:

> 3) I even tried adding the 'hidden' patch available, to put the hidden
> functionality back in the 2.4.x kernel (currently I'm testing using a 2.4.9
> kernel).  It doesn't appear to work properly either, hosts on the local network
> can't ping the server farm, and hosts outside the network although able to ping
> the server farm, cannot ping the real IP of the host.  It's kind of a weird
> problem.

	As this thread becomes too large I'm appending some
URLs. I don't know what patches you are using and what settings
you have. If you still have problems feel free to contact me directly.
The usage is simple: mark device "lo" as hidden and put there only
local addresses that must not advertised.

The "hidden" device's home page:
http://www.linuxvirtualserver.org/~julian/

Doc files:
http://www.linuxvirtualserver.org/~julian/hidden.txt
http://www.linuxvirtualserver.org/docs/arp.html

Regards

--
Julian Anastasov <ja@ssi.bg>