"Proper" way to transparent proxy?

"Proper" way to transparent proxy?

[Kevin White]

Short story: I need to write a transparent, application level proxy. 
I've searched through the mailing lists for information on how to do it, 
and find conflicting information about things like "stateless 
redirection" patches by Bazsi, and comments that they would no longer be 
maintained because something better was coming.  But I can't tell if 
that something better happened, or what it is.

If there's something I should be looking at, I'd be greatful for pointers.

Long story: we have an OpenBSD based firewall that uses this particular 
application level proxy, using IPFilter's transparent proxy support.  We 
are investigating rehosting the whole thing on Linux (for the record, MY 
first choice anyways :) and this is the hardest problem so far:

I need to find the address and port the outgoing connecion was 
originally heading to.  I'm assuming this is what's meant by the 
stateless redirection patches and such, but those (admittedly) didn't 
even come with documentation.

I'm starting from scratch here: I've never programmed to Netfilter 
before.  What I need is fairly simple, I think, but I don't know how to 
get started.

What we had on IPFilter was a "redirect" rule that redirected all 
connections to external hosts on this specific port to a specific port 
on the firewall machine, where a proxy program was waiting for it.  We 
then used IPFilter system calls to retreive where the connection was 
heading.

I expect that I'll be able to reuse most of my code: just the lookup 
part needs to change.

Thanks in advance for any help/pointers.  I'll attempt to collect what 
I've learned and post it, so that it ends up in the archives and more 
people like me don't have to ask.  :)

Kevin

Re: "Proper" way to transparent proxy?

[Peter Surda]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On Wed, Oct 09, 2002 at 04:11:05PM -0400, Kevin White wrote:
> What we had on IPFilter was a "redirect" rule that redirected all 
> connections to external hosts on this specific port to a specific port 
> on the firewall machine, where a proxy program was waiting for it.  We 
> then used IPFilter system calls to retreive where the connection was 
> heading.
This is exactly how it works on linux, you redirect (or reroute+redirect if on
different machines), accept the connection and call a sysctl to find out real
destination. As for examples, there are several open source transparent
proxies:
- squid
- tircproxy
- jftpgw

> Kevin
Bye,

Peter Surda (Shurdeek) <shurdeek@panorama.sth.ac.at>, ICQ 10236103, +436505122023

--
                    "Where do you want to go to die?"

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: "Proper" way to transparent proxy?

[Patrick Schaaf]

> I need to find the address and port the outgoing connecion was 
> originally heading to.

Use standard iptables, with a commandline like this:

	iptables -t nat -p tcp --dport 80 -j REDIRECT --to-port 1234

> I expect that I'll be able to reuse most of my code: just the lookup 
> part needs to change.

Once upon at time, in Linux 2.0.x, with libc5, you had getsockname()
return TheRightThing - in this particular situation. This is no longer
the case. Nowadays, with iptables and a rule like the one above, you
call getsockopt(), querying for SO_ORIGINAL_DST.

google tells me, giving it that hint, that no better documentation
than some ooooold mail threads exists on SO_ORIGINAL_DST. My bad.
I hate it when mailing list traffic by me is first hit :)

For the benefit of the eternal (hope hope hope) google, I am using
the exact code below, compiled like this:

	gcc -c preload_getsockname.c
	ld -o preload_getsockname.so -shared preload_getsockname.o -ldl

as an LD_PRELOAD file for a very ancient squid binary. In production
on some handful on boxes. You probably only need the 'getsockopt' line.

It works(tm).

best regards
  Patrick

#include <sys/socket.h>
#include <dlfcn.h>

#ifndef SO_ORIGINAL_DST
#include <linux/netfilter_ipv4.h>
#ifndef SO_ORIGINAL_DST
#error no SO_ORIGINAL_DST defined
#endif
#endif

int getsockname(int fd, struct sockaddr *sa, int *salen)
{
        static int (*libc_getsockname)(int, struct sockaddr *, int *);
        int ret;

        if (!libc_getsockname) {
                libc_getsockname = (int (*)(int, struct sockaddr *, int
*))
                        dlsym(RTLD_NEXT, "getsockname");
        }
        ret = getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, salen);
        if (ret < 0 && libc_getsockname)
                ret = libc_getsockname(fd, sa, salen);
        return ret;
}

how to flush ip conntrack entries ?

[marian stagarescu]

i run into the problem described here and i wondered if someone knows of
a solution:

       +----------+
       |  SNAT    |
private|--------> |(eth1: public ip)
  ip   +----------+

1) setup masquerading 
(iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADING)

and ping continuously from private to public. all ok.

2) flush the NAT

iptables -t nat -F

(ping will stop working obviously but i don't kill the ping process,
just keep sending echo, no reply)

3) put back nat:
(iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADING)

!!! ping does not come back !!!

4) I have to stop the pings and 

5) restart them to make it work.

looking at the ip_conntrack proc entry it was noticed that:

after flushing (step 2) an UNREPLIED entry for icmp is there
(no reply hence unreplied) but its ttl does not decrement.
(ping echos are still hitting the nat box from private side)

stoping the ping (step 4) allows the ttl timer of the conntrack entry 
to start decrementing (30 sec)

restaring the pings (i don't have to wait till ttl goes to zero ?!?)
(step 5) but now with nat back on (step 3) I don;t get the icmp entry on
conntrack but all is ok (pings goes thru).

question is: is there a way to achieve this (looks like start
decrementing that ttl or reseting it to zero in conntrack) in the nat
box without having to stop the pings on the host side ?

thanks,
marian

Re: how to flush ip conntrack entries ?

[Philip Craig]

marian stagarescu wrote:
> looking at the ip_conntrack proc entry it was noticed that:
> 
> after flushing (step 2) an UNREPLIED entry for icmp is there
> (no reply hence unreplied) but its ttl does not decrement.
> (ping echos are still hitting the nat box from private side)

Normally icmp conntrack entries are deleted as soon as there
is a reply.  Once you remove the MASQ rule though, there is no
reply, so the conntrack stays around.  Any subsequent packets
from the same ping process will match this conntrack, and thus
be NATed exactly the same way.  Adding the MASQ rule back in
does not affect the existing conntrack.

> stoping the ping (step 4) allows the ttl timer of the conntrack entry 
> to start decrementing (30 sec)

The timer doesn't seem to decrement while the ping is still
going because the ping packets are matching the conntrack and
refreshing the timer back to 30 seconds.  Stop the ping and the
timer is no longer refreshed.

> restaring the pings (i don't have to wait till ttl goes to zero ?!?)
> (step 5) but now with nat back on (step 3) I don;t get the icmp entry on
> conntrack but all is ok (pings goes thru).

When you start a new ping process, the ping packets have a new
id, and so they don't match the old conntrack.  A new conntrack
is created, and the new MASQ rule is used to masquerade them
correctly.  You get a ping reply now, and so the conntrack is
immediately deleted, which is why you don't see it.

> question is: is there a way to achieve this (looks like start
> decrementing that ttl or reseting it to zero in conntrack) in the nat
> box without having to stop the pings on the host side ?

You can flush conntrack entries for masqueraded connections
by doing either a down/up or ip addr add/del on the associated
interface.  This probably won't help you in this case though
because the problem conntrack is not masqueraded.  I don't
know of any other ways of flushing conntracks.

Regards
-- 
Philip Craig     Software Engineer     http://www.SnapGear.com
philipc@snapgear.com  Ph: +61 7 3435 2821  Fx: +61 7 3891 3630
SnapGear  -  Custom Embedded Solutions and Security Appliances

Re: how to flush ip conntrack entries ?

[marian stagarescu]

hi philip,
thanks for you answer.

yes ifconfig up/down works but i wanted to make it work without doing
this.

i noticed that the ipsysctl tutorial 1.0.0 (posted to the list by askar)
documentation points to a /proc/sys/net/ipv4/netfilter entry with an 
ip_ct_generic_timeout 
and ip_ct_icmp_timeout
i don't have those into my kernel (2.4.17 patched with newnat13 and
userland iptables 1.2.6a), maybe someone on the list can tell me which
patch installs these entries.
having those i can try to set to zero the above
ip_ct_icmp_timeout

marian



On Sun, 2002-10-13 at 20:59, Philip Craig wrote:
> marian stagarescu wrote:
> > looking at the ip_conntrack proc entry it was noticed that:
> > 
> > after flushing (step 2) an UNREPLIED entry for icmp is there
> > (no reply hence unreplied) but its ttl does not decrement.
> > (ping echos are still hitting the nat box from private side)
> 
> Normally icmp conntrack entries are deleted as soon as there
> is a reply.  Once you remove the MASQ rule though, there is no
> reply, so the conntrack stays around.  Any subsequent packets
> from the same ping process will match this conntrack, and thus
> be NATed exactly the same way.  Adding the MASQ rule back in
> does not affect the existing conntrack.
> 
> > stoping the ping (step 4) allows the ttl timer of the conntrack entry 
> > to start decrementing (30 sec)
> 
> The timer doesn't seem to decrement while the ping is still
> going because the ping packets are matching the conntrack and
> refreshing the timer back to 30 seconds.  Stop the ping and the
> timer is no longer refreshed.
> 
> > restaring the pings (i don't have to wait till ttl goes to zero ?!?)
> > (step 5) but now with nat back on (step 3) I don;t get the icmp entry on
> > conntrack but all is ok (pings goes thru).
> 
> When you start a new ping process, the ping packets have a new
> id, and so they don't match the old conntrack.  A new conntrack
> is created, and the new MASQ rule is used to masquerade them
> correctly.  You get a ping reply now, and so the conntrack is
> immediately deleted, which is why you don't see it.
> 
> > question is: is there a way to achieve this (looks like start
> > decrementing that ttl or reseting it to zero in conntrack) in the nat
> > box without having to stop the pings on the host side ?
> 
> You can flush conntrack entries for masqueraded connections
> by doing either a down/up or ip addr add/del on the associated
> interface.  This probably won't help you in this case though
> because the problem conntrack is not masqueraded.  I don't
> know of any other ways of flushing conntracks.
> 
> Regards
> -- 
> Philip Craig     Software Engineer     http://www.SnapGear.com
> philipc@snapgear.com  Ph: +61 7 3435 2821  Fx: +61 7 3891 3630
> SnapGear  -  Custom Embedded Solutions and Security Appliances

Re: how to flush ip conntrack entries ?

[marian stagarescu]

i see these entries under tcp window tracking patch
i'll give it a try.

On Mon, 2002-10-14 at 15:20, marian stagarescu wrote:
> hi philip,
> thanks for you answer.
> 
> yes ifconfig up/down works but i wanted to make it work without doing
> this.
> 
> i noticed that the ipsysctl tutorial 1.0.0 (posted to the list by askar)
> documentation points to a /proc/sys/net/ipv4/netfilter entry with an 
> ip_ct_generic_timeout 
> and ip_ct_icmp_timeout
> i don't have those into my kernel (2.4.17 patched with newnat13 and
> userland iptables 1.2.6a), maybe someone on the list can tell me which
> patch installs these entries.
> having those i can try to set to zero the above
> ip_ct_icmp_timeout
> 
> marian
> 
> 
> 
> On Sun, 2002-10-13 at 20:59, Philip Craig wrote:
> > marian stagarescu wrote:
> > > looking at the ip_conntrack proc entry it was noticed that:
> > > 
> > > after flushing (step 2) an UNREPLIED entry for icmp is there
> > > (no reply hence unreplied) but its ttl does not decrement.
> > > (ping echos are still hitting the nat box from private side)
> > 
> > Normally icmp conntrack entries are deleted as soon as there
> > is a reply.  Once you remove the MASQ rule though, there is no
> > reply, so the conntrack stays around.  Any subsequent packets
> > from the same ping process will match this conntrack, and thus
> > be NATed exactly the same way.  Adding the MASQ rule back in
> > does not affect the existing conntrack.
> > 
> > > stoping the ping (step 4) allows the ttl timer of the conntrack entry 
> > > to start decrementing (30 sec)
> > 
> > The timer doesn't seem to decrement while the ping is still
> > going because the ping packets are matching the conntrack and
> > refreshing the timer back to 30 seconds.  Stop the ping and the
> > timer is no longer refreshed.
> > 
> > > restaring the pings (i don't have to wait till ttl goes to zero ?!?)
> > > (step 5) but now with nat back on (step 3) I don;t get the icmp entry on
> > > conntrack but all is ok (pings goes thru).
> > 
> > When you start a new ping process, the ping packets have a new
> > id, and so they don't match the old conntrack.  A new conntrack
> > is created, and the new MASQ rule is used to masquerade them
> > correctly.  You get a ping reply now, and so the conntrack is
> > immediately deleted, which is why you don't see it.
> > 
> > > question is: is there a way to achieve this (looks like start
> > > decrementing that ttl or reseting it to zero in conntrack) in the nat
> > > box without having to stop the pings on the host side ?
> > 
> > You can flush conntrack entries for masqueraded connections
> > by doing either a down/up or ip addr add/del on the associated
> > interface.  This probably won't help you in this case though
> > because the problem conntrack is not masqueraded.  I don't
> > know of any other ways of flushing conntracks.
> > 
> > Regards
> > -- 
> > Philip Craig     Software Engineer     http://www.SnapGear.com
> > philipc@snapgear.com  Ph: +61 7 3435 2821  Fx: +61 7 3891 3630
> > SnapGear  -  Custom Embedded Solutions and Security Appliances
> 

Re: "Proper" way to transparent proxy?

[Kevin White]

> For the benefit of the eternal (hope hope hope) google, I am using
> the exact code below, compiled like this:
> 
> 	gcc -c preload_getsockname.c
> 	ld -o preload_getsockname.so -shared preload_getsockname.o -ldl
> 
> as an LD_PRELOAD file for a very ancient squid binary. In production
> on some handful on boxes. You probably only need the 'getsockopt' line.

Great...thanks tons.  I'll put it into action...

Thanks again!

Kevin