[LARTC] Proxy ARP and UDP

[LARTC] Proxy ARP and UDP

[Greg Scott]

Woops - my fat fingers hit the send key before I could put in a subject
a minute ago.  

Hello - 

I am using kernel 2.4.27 and running into behavior I don't know how to
explain.  

I have 2 relevant interfaces. eth0 is external, eth1 is internal.  My
internal LAN is 10.10.10.0/24.  My External range is 1.2.3.0/27 (dummied
up).  I have an H.323 videoconference device inside my internal LAN, but
at IP Address 1.2.3.11/27.  (IP Address dummied up.)  I want to proxy
ARP this device.  

Both eth0 and eth1 on my firewall have IP Addresses 1.2.3.2/27.  eth1
also has IP Address 10.10.10.1/24 and is the default gateway for all my
internal hosts.  The router outside my firewall is 1.2.3.1.  

So the network looks like this (apologies if email butchers my ASCII
art): 

10.10.10.0/27               1.2.3.0/27 
       10.10.10.n 
     internal hosts 
           | 
<----+-----+--------+    +-------+------>to the Internet 
     |              |    |       | 
  Proxied           |    |       | 
H.323 device       Firewall      Router 
                  eth1   eth0 
1.2.3.11    10.10.10.1  1.2.3.2  1.2.3.1 
             1.2.3.2 

/proc/sys/net/ipv4/conf/eth0/proxy_arp is 1.  
/proc/sys/net/ipv4/conf/eth1/proxy_arp is 1. 

My firewall has a route to 1.2.3.11 dev eth1. 

The host at 1.2.3.11 has a default GW of 1.2.3.1. 

This is where it gets weird.  The H.323 device should exchange a few TCP
packets with the far end and then thousands of UDP packets.  And I
should see this stream on the firewall watching both interfaces.  

I run tcpdump in two different windows on the firewall - one for eth1,
the other for eth0.  When I initiate an outbound H.323 call from the
device at .11, tcpdump on the firewall shows TCP packets flying on eth1,
but nothing on eth0 - almost all the time.  Calls don't complete most of
the time, although one call kind of completed.  Watching on the
firewall, I saw a TCP conversation on eth1, but nothing on eth0.  Very
strange!  One time a call completed all the way and UDP started flying -
as it should.  I saw a few UDP packets on eth0 and lots (thousands) of
UDP packets on eth1.  For the call that really completed, I would expect
to see thousasnds of UDP packets on both eth0 and eth1 - but instead saw
only a few on eth0.  

This behavior happens even with no firewall filtering rules in place. 

My NATed 10.10.10.nn internal hosts work fine - in fact, my email server
posting this item to the list is one of those hosts.  

The obvious question - why such an old kernel?  Because it's worked for
everything I need so far and every 2.6.nn I try has other bugs with one
module or another.  

My questions - was proxy ARP broken in the 2.4.27 days?  Why doen't
tcpdump show me packets on both interfaces of the firewall?  Am I
missing a setup ingredient someplace?  Should the default GW on that
H.323 device be .2 (the firewall) or .1 (the Internet router)?  Does
mixing NAT and proxy ARP create problems?  Should I put the H.323 device
in its own little DMZ?

Thanks 

- Greg Scott 



_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Proxy ARP and UDP

[gypsy]

Greg Scott wrote:
> I have 2 relevant interfaces. eth0 is external, eth1 is internal.  My
> internal LAN is 10.10.10.0/24.  My External range is 1.2.3.0/27 (dummied
> up).  I have an H.323 videoconference device inside my internal LAN, but
> at IP Address 1.2.3.11/27.  (IP Address dummied up.)  I want to proxy
> ARP this device.
> 
> My questions - was proxy ARP broken in the 2.4.27 days?  Why doen't
> tcpdump show me packets on both interfaces of the firewall?  Am I
> missing a setup ingredient someplace?  Should the default GW on that
> H.323 device be .2 (the firewall) or .1 (the Internet router)?  Does
> mixing NAT and proxy ARP create problems?  Should I put the H.323 device
> in its own little DMZ?
> 
> Thanks
> 
> - Greg Scott

No, not broken; proxy ARP works fine in 2.4.25 - .32.

You should have a look at Martin Brown's proxy ARP script
http://yesican.chsoft.biz/lartc/proxy-arp.sh
and its config file
http://yesican.chsoft.biz/lartc/proxy-arp.conf

but I bet the problem is rp_filter.
--
gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Proxy ARP and UDP

[Greg Scott]

Hmmmm - 

I turned off rp_filter (echo 0 > /proc/sys/net/ipv4/eth0/rp_filter - and
eth1) and ran several test calls.  It all worked.  But I still don't
understand why I see less than 1 percent of the packets on the eth0
interface with tcpdump.  

- Greg


> but I bet the problem is rp_filter.
> --
> gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Proxy ARP and UDP

[Greg Scott]

As it turns out, not seeing proxy ARP traffic on the outside interface
has other consequences.  I do some traffic shaping and noticed in my
testing that the outbound traffic isn't being shaped.  This drove me
crazy until it suddenly dawned on me - tcpdump shows almost no traffic
on the outside interface even though a full H.323 UDP stream is flying
across the Internet to and from my proxy ARP'd device behind my
firewall.  I know lots of data is flying across both interfaces because
I can see the results.  Yet as far as any software is concerned, almost
nothing is going in or out of my outside interface.  

Is this a normal proxy ARP behavior?  Traffic is definitely flying
across both interfaces.  Why doesn't any software see traffic in and out
of the outside interface?  Should I try a newer kernel than 2.4.27?  

I guess I could shape the internal interface for anything routing across
to the Internet but it just makes more sense to shape the interface at
the boundary.  

Here is the network layout again:

10.10.10.0/27               1.2.3.0/27 
       10.10.10.n          (fictional public IP range)
     internal hosts 
           | 
<----+-----+--------+    +-------+------>to the Internet 
     |              |    |       | 
  Proxied           |    |       | 
H.323 device       Firewall      Router 
                  eth1   eth0 
1.2.3.11    10.10.10.1  1.2.3.2  1.2.3.1 
             1.2.3.2 

/proc/sys/net/ipv4/conf/eth0/proxy_arp is 1.  
/proc/sys/net/ipv4/conf/eth1/proxy_arp is 1. 

/proc/sys/net/ipv4/conf/eth0/rp_filter is 0.  
/proc/sys/net/ipv4/conf/eth1/rp_filter is 0.  

/proc/sys/net/ipv4/conf/ip_forward is 1.  

My firewall has a route to 1.2.3.11 dev eth1. 


- Greg Scott


-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Greg Scott
Sent: Monday, February 20, 2006 8:52 PM
To: gypsy; lartc@mailman.ds9a.nl
Subject: RE: [LARTC] Proxy ARP and UDP


Hmmmm - 

I turned off rp_filter (echo 0 > /proc/sys/net/ipv4/eth0/rp_filter - and
eth1) and ran several test calls.  It all worked.  But I still don't
understand why I see less than 1 percent of the packets on the eth0
interface with tcpdump.  

- Greg


> but I bet the problem is rp_filter.
> --
> gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Proxy ARP and UDP

[gypsy]

Greg Scott wrote:
> 
> As it turns out, not seeing proxy ARP traffic on the outside interface
> has other consequences.  I do some traffic shaping and noticed in my
> testing that the outbound traffic isn't being shaped.  This drove me
> crazy until it suddenly dawned on me - tcpdump shows almost no traffic
> on the outside interface even though a full H.323 UDP stream is flying
> across the Internet to and from my proxy ARP'd device behind my
> firewall.  I know lots of data is flying across both interfaces because
> I can see the results.  Yet as far as any software is concerned, almost
> nothing is going in or out of my outside interface.
> 
> Is this a normal proxy ARP behavior?  Traffic is definitely flying
> across both interfaces.  Why doesn't any software see traffic in and out
> of the outside interface?  Should I try a newer kernel than 2.4.27?

Greg,

Please, if you want answers, provide enough information for us to help.

In the absence of any shaping configuration script, it is useless to
speculate about why you see nothing being shaped.  I will say that UDP
is not "protocol ip".  Neither is ARP nor ICMP.

In the absence of the parameters you are passing to tcpdump, nothing can
be said about why you are not seeing the expected traffic on the
external IF.

Run 'cat /proc/net/ip_conntrack | grep udp'

There is nothing wrong with your .27 kernel!  I have done something
similar to what you seem to be trying to do for years running kernels
from 2.4.25 through .32 and never had any problem at all with proxy ARP
(except for the mental part ;)
--
gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Proxy ARP and UDP

[Greg Scott]

OK - 

Here is how I am running tcpdump.  Not really much to tell.

In one window:
tcpdump -i eth1 -n

And then in another window:
tcpdump -i eth0 -n

If I were filtering anything with tcpdump I would be pretty embarrassed.
:)

eth0 is the interface pointing to the Internet.  eth1 is inside.  For
every several thousand packets that tcpdump shows me on eth1, I see
maybe one or two on eth0 when running any traffic at all between the
Internet and my proxy ARP'd device.  

Since these are conversations with a host on the other side of the
Internet I should see packets flying across both interfaces.  But I
don't.  I only see packets flying across the inside interface, except
for a very small subset that I see on the outside interface.  

This behavior makes no sense.  How is it possible that any connection
between my proxy ARP'd host and the Internet works if virtually no
traffic is moving across the outside interface????  The obvious answer -
it isn't.  Traffic must in fact be moving across the outside interface,
otherwise my proxy ARP'd device would never see it.  So the only
possible conclusion is, the traffic must he happening someplace where
tcpdump and evidently also the traffic shaping code does not see it.  

Don't believe me?  Try it yourself.  Send a bunch of pings from
somewhere across the Internet to your proxy ARP'd device and watch your
outside interface.  I'll bet you don't see them.  But your proxy ARP'd
device will see them, assuming you have some firewall rules that allow
this.  It will reply and the requesting host outside the Internet will
see the echo reply packets coming back.  But your outside firewall
interface will look dead even though the echo request/reply packets are
clearly flying across it.  Look for yourself if you don't believe me.  

Here is my traffic shaping script.  Again, pretty basic stuff - nothing
fancy.  And it isn't relevant to my issue.  


TC="/sbin/tc"

$TC qdisc del dev $INET_IFACE root
$TC qdisc del dev $TRUSTED1_IFACE root
$TC qdisc del dev $DMZ_IFACE root

$TC qdisc add dev $INET_IFACE root handle 1: prio 
# This *instantly* creates classes 1:1, 1:2, 1:3
$TC qdisc add dev $TRUSTED1_IFACE root handle 2: prio
# This *instantly* creates classes 2:1, 2:2, 2:3

$TC qdisc add dev $INET_IFACE parent 1:1 handle 11: pfifo
$TC qdisc add dev $INET_IFACE parent 1:2 handle 12: pfifo
$TC qdisc add dev $INET_IFACE parent 1:3 handle 13: pfifo

$TC qdisc add dev $TRUSTED1_IFACE parent 2:1 handle 21: pfifo
$TC qdisc add dev $TRUSTED1_IFACE parent 2:2 handle 22: pfifo
$TC qdisc add dev $TRUSTED1_IFACE parent 2:3 handle 23: pfifo

           

#
# This assigns traffic to/from $PUBLIC_VTC1_IP and $PRIVATE_VTC1_IP
# to the highest priority band of the queue for the appropriate
# interface, and the rest to the next-highest proirity band.
#
# VTC1
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip dst $PUBLIC_VTC1_IP flowid 1:1
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip src $PUBLIC_VTC1_IP flowid 1:1

# VTC2
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip dst $PUBLIC_VTC2_IP flowid 1:1
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip src $PUBLIC_VTC2_IP flowid 1:1

# Everyone else
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 2 u32 \
  match ip src 0.0.0.0/0 flowid 1:2
$TC filter add dev $TRUSTED1_IFACE parent 2:0 protocol ip prio 2 u32 \
  match ip src 0.0.0.0/0 flowid 2:2

exit



> Greg,
>
>Please, if you want answers, provide enough information for us to help.
>
>In the absence of any shaping configuration script, it is useless to 
>speculate about why you see nothing being shaped.  I will say that UDP 
>is not "protocol ip".  Neither is ARP nor ICMP.
>
>In the absence of the parameters you are passing to tcpdump, nothing
can 
>be said about why you are not seeing the expected traffic on the
external IF.
>
>Run 'cat /proc/net/ip_conntrack | grep udp'
>
>There is nothing wrong with your .27 kernel!  I have done something 
>similar to what you seem to be trying to do for years running kernels 
>from 2.4.25 through .32 and never had any problem at all with proxy ARP

>(except for the mental part ;)
>--
>gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Proxy ARP and UDP

[Greg Scott]

I found the problem!  It was me and it was dumb...

This was the network layout:

10.10.10.0/24               1.2.3.0/27 
       10.10.10.n 
     internal hosts 
           | 
<----+-----+--------+    +-------+------>to the Internet 
     |              |    |       | 
  Proxied           |    |       | 
H.323 device       Firewall      Router 
                  eth1   eth0 
1.2.3.11    10.10.10.1  1.2.3.2  1.2.3.1 
             1.2.3.2 

The problem was, before doing proxy ARP, my h.323 device was set up with
NAT and it had a 10.nnn IP Address.  The outside interface on my
firewall had 1.2.3.11 as a secondary address and NATed the appropriate
stuff to the H.323 device.  I changed all that to use proxy ARP - but I
forgot to get rid of the secondary IP Address on eth0 of the firewall.
I changed all my scripts but forgot to change that IP Address in the
live, running system.  Woops!

- Greg Scott


-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Greg Scott
Sent: Monday, February 27, 2006 6:28 AM
To: gypsy; lartc@mailman.ds9a.nl
Subject: RE: [LARTC] Proxy ARP and UDP

OK - 

Here is how I am running tcpdump.  Not really much to tell.

In one window:
tcpdump -i eth1 -n

And then in another window:
tcpdump -i eth0 -n

If I were filtering anything with tcpdump I would be pretty embarrassed.
:)

eth0 is the interface pointing to the Internet.  eth1 is inside.  For
every several thousand packets that tcpdump shows me on eth1, I see
maybe one or two on eth0 when running any traffic at all between the
Internet and my proxy ARP'd device.  

Since these are conversations with a host on the other side of the
Internet I should see packets flying across both interfaces.  But I
don't.  I only see packets flying across the inside interface, except
for a very small subset that I see on the outside interface.  

This behavior makes no sense.  How is it possible that any connection
between my proxy ARP'd host and the Internet works if virtually no
traffic is moving across the outside interface????  The obvious answer -
it isn't.  Traffic must in fact be moving across the outside interface,
otherwise my proxy ARP'd device would never see it.  So the only
possible conclusion is, the traffic must he happening someplace where
tcpdump and evidently also the traffic shaping code does not see it.  

Don't believe me?  Try it yourself.  Send a bunch of pings from
somewhere across the Internet to your proxy ARP'd device and watch your
outside interface.  I'll bet you don't see them.  But your proxy ARP'd
device will see them, assuming you have some firewall rules that allow
this.  It will reply and the requesting host outside the Internet will
see the echo reply packets coming back.  But your outside firewall
interface will look dead even though the echo request/reply packets are
clearly flying across it.  Look for yourself if you don't believe me.  

Here is my traffic shaping script.  Again, pretty basic stuff - nothing
fancy.  And it isn't relevant to my issue.  


TC="/sbin/tc"

$TC qdisc del dev $INET_IFACE root
$TC qdisc del dev $TRUSTED1_IFACE root
$TC qdisc del dev $DMZ_IFACE root

$TC qdisc add dev $INET_IFACE root handle 1: prio # This *instantly*
creates classes 1:1, 1:2, 1:3 $TC qdisc add dev $TRUSTED1_IFACE root
handle 2: prio # This *instantly* creates classes 2:1, 2:2, 2:3

$TC qdisc add dev $INET_IFACE parent 1:1 handle 11: pfifo $TC qdisc add
dev $INET_IFACE parent 1:2 handle 12: pfifo $TC qdisc add dev
$INET_IFACE parent 1:3 handle 13: pfifo

$TC qdisc add dev $TRUSTED1_IFACE parent 2:1 handle 21: pfifo $TC qdisc
add dev $TRUSTED1_IFACE parent 2:2 handle 22: pfifo $TC qdisc add dev
$TRUSTED1_IFACE parent 2:3 handle 23: pfifo

           

#
# This assigns traffic to/from $PUBLIC_VTC1_IP and $PRIVATE_VTC1_IP # to
the highest priority band of the queue for the appropriate # interface,
and the rest to the next-highest proirity band.
#
# VTC1
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip dst $PUBLIC_VTC1_IP flowid 1:1 $TC filter add dev $INET_IFACE
parent 1:0 protocol ip prio 1 u32 \
  match ip src $PUBLIC_VTC1_IP flowid 1:1

# VTC2
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \
  match ip dst $PUBLIC_VTC2_IP flowid 1:1 $TC filter add dev $INET_IFACE
parent 1:0 protocol ip prio 1 u32 \
  match ip src $PUBLIC_VTC2_IP flowid 1:1

# Everyone else
$TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 2 u32 \
  match ip src 0.0.0.0/0 flowid 1:2
$TC filter add dev $TRUSTED1_IFACE parent 2:0 protocol ip prio 2 u32 \
  match ip src 0.0.0.0/0 flowid 2:2

exit



> Greg,
>
>Please, if you want answers, provide enough information for us to help.
>
>In the absence of any shaping configuration script, it is useless to 
>speculate about why you see nothing being shaped.  I will say that UDP 
>is not "protocol ip".  Neither is ARP nor ICMP.
>
>In the absence of the parameters you are passing to tcpdump, nothing
can 
>be said about why you are not seeing the expected traffic on the
external IF.
>
>Run 'cat /proc/net/ip_conntrack | grep udp'
>
>There is nothing wrong with your .27 kernel!  I have done something 
>similar to what you seem to be trying to do for years running kernels 
>from 2.4.25 through .32 and never had any problem at all with proxy ARP

>(except for the mental part ;)
>--
>gypsy
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc