DNATing packets sent to the NATing box

DNATing packets sent to the NATing box

[horape]

I've a system that at its core has an UDP proxy that's the performance
bottleneck. I wanted to use the DNAT kernel facilities to replace my
code with the very tuned one on netfilter.

I'm adding a rule that says something like this:

/sbin/iptables -t nat -A PREROUTING -d myip -p udp -m udp --dport 5000 -j DNAT
		--to-destination otherip:18918

but the rule never see the packets (they never got to the chain)

I assume that it's because I've a socket listening on udp:5000, and it seems
reasonable what's happening... I'd like to add a PREPREROUTING chain that
is processed before deciding if the packet is for a local socket, can somebody
give me a hint on where to look for it?

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[Joel Newkirk]

On Sat, 2003-12-13 at 19:52, horape@tinuviel.compendium.com.ar wrote:
> I've a system that at its core has an UDP proxy that's the performance
> bottleneck. I wanted to use the DNAT kernel facilities to replace my
> code with the very tuned one on netfilter.
> 
> I'm adding a rule that says something like this:
> 
> /sbin/iptables -t nat -A PREROUTING -d myip -p udp -m udp --dport 5000 -j DNAT
> 		--to-destination otherip:18918

Try:

/sbin/iptables -t nat -A PREROUTING -d myip -p udp --dport 5000 -j DNAT
--to otherip:18918

And ensure you are letting traffic for -d otherip -p udp --dport 18918
through FORWARD chain.

> but the rule never see the packets (they never got to the chain)
> 
> I assume that it's because I've a socket listening on udp:5000, and it seems

Nope.

> reasonable what's happening... I'd like to add a PREPREROUTING chain that
> is processed before deciding if the packet is for a local socket, can somebody
> give me a hint on where to look for it?

If it's for local, it can be seen in Mangle Prerouting, then Nat
Prerouting, then Mangle Input, then Filter Input.

> Saludos,
> 					HoraPe
> ---
> Horacio J. Pea
> horape@compendium.com.ar
horape@uninet.edu

j

Re: DNATing packets sent to the NATing box

[Antony Stone]

On Sunday 14 December 2003 12:52 am, horape@tinuviel.compendium.com.ar wrote:

> I've a system that at its core has an UDP proxy that's the performance
> bottleneck. I wanted to use the DNAT kernel facilities to replace my
> code with the very tuned one on netfilter.
>
> I'm adding a rule that says something like this:
>
> /sbin/iptables -t nat -A PREROUTING -d myip -p udp -m udp --dport 5000 -j
> DNAT --to-destination otherip:18918
>
> but the rule never see the packets (they never got to the chain)
>
> I assume that it's because I've a socket listening on udp:5000, and it
> seems reasonable what's happening... I'd like to add a PREPREROUTING chain
> that is processed before deciding if the packet is for a local socket, can
> somebody give me a hint on where to look for it?

PREROUTING works exactly as the name suggests - it is applied to packets 
before the routing decision is made about whether they are local, or being 
routed through the box.   Therefore you *can* use the PREROUTING chain to 
divert packets which would otherwise be accepted locally, so that they go to 
another machine, or else divert packets which would have gone somewhere else, 
so that they are accepted locally.

You say the rule never sees the packets...  how do you know this?   Are you 
looking at the packet / byte counters, and they stay at zero all the time?

You also say you have a local process listening on port 5000 - is that getting 
any packets and responding to them, even with above rule in place?

Antony.

-- 
The first fifty percent of an engineering project takes ninety percent of the 
time, and the remaining fifty percent takes another ninety percent of the 
time.

                                                     Please reply to the list;
                                                           please don't CC me.

Re: DNATing packets sent to the NATing box

[horape]

> > I've a system that at its core has an UDP proxy that's the performance
> > bottleneck. I wanted to use the DNAT kernel facilities to replace my
> > code with the very tuned one on netfilter.

> > I'm adding a rule that says something like this:

> > /sbin/iptables -t nat -A PREROUTING -d myip -p udp -m udp --dport 5000 -j
> > DNAT --to-destination otherip:18918

> > but the rule never see the packets (they never got to the chain)

> > I assume that it's because I've a socket listening on udp:5000, and it
> > seems reasonable what's happening... I'd like to add a PREPREROUTING chain
> > that is processed before deciding if the packet is for a local socket, can
> > somebody give me a hint on where to look for it?

> PREROUTING works exactly as the name suggests - it is applied to packets 
> before the routing decision is made about whether they are local, or being 
> routed through the box.   Therefore you *can* use the PREROUTING chain to 
> divert packets which would otherwise be accepted locally, so that they go to 
> another machine, or else divert packets which would have gone somewhere else, 
> so that they are accepted locally.

> You say the rule never sees the packets...  how do you know this?   Are you 
> looking at the packet / byte counters, and they stay at zero all the time?

Yes, and I've added a rule like this:

/sbin/iptables -t nat -A PREROUTING -j LOG and don't see the packets.

> You also say you have a local process listening on port 5000 - is that getting 
> any packets and responding to them, even with above rule in place?

Yes, it continues getting the packets.

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[horape]

> > Yes, and I've added a rule like this:
> > /sbin/iptables -t nat -A PREROUTING -j LOG and don't see the packets.

> 	Ummmm
>     if you ADD the rule above after the rule that is re-routing the packet, 
> 	no ... you wont see the packets.  Try 
> 	iptables -t nat -I PREROUTING (line number)
> 	where (line number) is less than the line on which your DNAT line occurrs.
> 	(see iptables -t nat --line-numbers -v )

I'm really replacing the DNAT rule with the LOG one (only rule in the chain is the
LOG one)

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[Jeffrey Laramie]

On Sunday 14 December 2003 08:52, horape@tinuviel.compendium.com.ar wrote:
> > > Yes, and I've added a rule like this:
> > > /sbin/iptables -t nat -A PREROUTING -j LOG and don't see the packets.
> >
> > 	Ummmm
> >     if you ADD the rule above after the rule that is re-routing the
> > packet, no ... you wont see the packets.  Try
> > 	iptables -t nat -I PREROUTING (line number)
> > 	where (line number) is less than the line on which your DNAT line
> > occurrs. (see iptables -t nat --line-numbers -v )
>
> I'm really replacing the DNAT rule with the LOG one (only rule in the chain
> is the LOG one)
>
If there are no other rules in the nat PREROUTING chain and assuming that you 
don't have and mangle PREROUTING rules (you don't, do you?) then that log 
rule will see every single packet that comes in off the wire. Try this:

iptables -A PREROUTING -p all -j LOG --log-prefix "PREROUTING: "

You should see tons output in /var/log/messages including the packets you're 
looking for. If the packets aren't there, then they aren't making it to the 
box.

Jeff

Re: DNATing packets sent to the NATing box

[horape]

> > I'm really replacing the DNAT rule with the LOG one (only rule in the chain
> > is the LOG one)

> If there are no other rules in the nat PREROUTING chain and assuming that you 
> don't have and mangle PREROUTING rules (you don't, do you?) then that log 
> rule will see every single packet that comes in off the wire. Try this:

> iptables -A PREROUTING -p all -j LOG --log-prefix "PREROUTING: "

I do:

iptables -t nat -A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "
iptables -t filter -A INPUT -p udp -j LOG --log-prefix "INPUT: "

(I'm logged in that box via ssh, so I prefer not to do -p all)

There are no more rules than these ones.

In the log I see the INPUT ones but not the PREROUTING (only see on
PREROUTING packets to port 137, maybe some worm...)

> You should see tons output in /var/log/messages including the packets you're 
> looking for. If the packets aren't there, then they aren't making it to the 
> box.

But the packets got to the INPUT rule, that should be after PREROUTING. And
got to my socket.

> Jeff

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[horape]

> > > > Yes, and I've added a rule like this:
> > > > /sbin/iptables -t nat -A PREROUTING -j LOG and don't see the packets.
> > > 	Ummmm
> > >     if you ADD the rule above after the rule that is re-routing the
> > > packet, no ... you wont see the packets.  Try
> > > 	iptables -t nat -I PREROUTING (line number)
> > > 	where (line number) is less than the line on which your DNAT line
> > > occurrs. (see iptables -t nat --line-numbers -v )
> > I'm really replacing the DNAT rule with the LOG one (only rule in the chain
> > is the LOG one)
> 	This sounds *very* strange.  Perhaps more detail would give those on the list 
> a chance to determine what your problem is. 

Thanks for your help.

> 	More detail of what the proxy is and more detail about your system setup 
> would likely help us determine what course of action you need to take.

I'm running 2.4.23, I have an udp socket listening on port 5000, and a rule
that says:

iptables -t nat -A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "

(that's not the rule that I want, but it seems clearer with that one)

And when the box receives packets for udp 5000, the socket gets it, but
the rule doesn't see it.

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[Jeffrey Laramie]

On Sunday 14 December 2003 10:26, horape@tinuviel.compendium.com.ar wrote:
> > > I'm really replacing the DNAT rule with the LOG one (only rule in the
> > > chain is the LOG one)
> >
> > If there are no other rules in the nat PREROUTING chain and assuming that
> > you don't have and mangle PREROUTING rules (you don't, do you?) then that
> > log rule will see every single packet that comes in off the wire. Try
> > this:
> >
> > iptables -A PREROUTING -p all -j LOG --log-prefix "PREROUTING: "
>
> I do:
>
> iptables -t nat -A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "
> iptables -t filter -A INPUT -p udp -j LOG --log-prefix "INPUT: "
>
> (I'm logged in that box via ssh, so I prefer not to do -p all)
>
> There are no more rules than these ones.
>
> In the log I see the INPUT ones but not the PREROUTING (only see on
> PREROUTING packets to port 137, maybe some worm...)
>

That's very strange. You said in your first post that you had a udp proxy 
running. Is it on this box? I'm not sure at what level a proxy hooks into the 
box, but I know that some programs read the data stream before netfilter sees 
it. If the proxy is in front of netfilter then netfilter would never see 
those packets. The port 137 packets are prolly just internet garbage.

> > You should see tons output in /var/log/messages including the packets
> > you're looking for. If the packets aren't there, then they aren't making
> > it to the box.
>
> But the packets got to the INPUT rule, that should be after PREROUTING. And
> got to my socket.
>

Can you show these log entries? I don't even know how that could happen. Maybe 
someone else has some ideas.

Jeff

Re: DNATing packets sent to the NATing box

[horape]

> > I do:

> > iptables -t nat -A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "
> > iptables -t filter -A INPUT -p udp -j LOG --log-prefix "INPUT: "

> > (I'm logged in that box via ssh, so I prefer not to do -p all)

> > There are no more rules than these ones.

> > In the log I see the INPUT ones but not the PREROUTING (only see on
> > PREROUTING packets to port 137, maybe some worm...)
> That's very strange. You said in your first post that you had a udp proxy 
> running. Is it on this box? I'm not sure at what level a proxy hooks into the 
> box, but I know that some programs read the data stream before netfilter sees 
> it. If the proxy is in front of netfilter then netfilter would never see 
> those packets. The port 137 packets are prolly just internet garbage.

The proxy has a udp socket bound to port 5000, and another udp socket bound to
some other port, it reads from the first socket and does a sendto using the
other socket (poll + read + write, no more)

> > > You should see tons output in /var/log/messages including the packets
> > > you're looking for. If the packets aren't there, then they aren't making
> > > it to the box.
> > But the packets got to the INPUT rule, that should be after PREROUTING. And
> > got to my socket.
> Can you show these log entries? I don't even know how that could happen. Maybe 
> someone else has some ideas.

Dec 14 15:57:22 ivrip kernel: INPUT: IN=eth0 OUT= MAC=00:0a:e6:fa:c6:09:00:09:12:2b:5e:fd:08:00 SRC
=200.61.169.146 DST=200.68.94.100 LEN=60 TOS=0x00 PREC=0xA0 TTL=251 ID=17815 PROTO=UDP SPT=17314 DP
T=5000 LEN=40
Dec 14 15:57:23 ivrip last message repeated 47 times
Dec 14 15:57:23 ivrip kernel: INPUT: IN=eth0 OUT= MAC=00:0a:e6:fa:c6:09:00:09:12:2b:5e:fd:08:00 SRC
=200.61.169.146 DST=200.68.94.100 LEN=41 TOS=0x00 PREC=0xA0 TTL=251 ID=17855 PROTO=UDP SPT=17314 DP
T=5000 LEN=21

Lots of those. Only very sporadic, non related PREROUTING ones, like:

Dec 14 15:57:58 ivrip kernel: PREROUTING: IN=eth0 OUT= MAC=00:0a:e6:fa:c6:09:00:09:12:2b:5e:fd:08:0
0 SRC=217.126.141.13 DST=200.68.94.100 LEN=78 TOS=0x00 PREC=0x00 TTL=108 ID=31313 PROTO=UDP SPT=566
66 DPT=137 LEN=58

iptables-save says:

# Generated by iptables-save v1.2.9 on Sun Dec 14 15:59:05 2003
*nat
:PREROUTING ACCEPT [1474:107026]
:POSTROUTING ACCEPT [19:2376]
:OUTPUT ACCEPT [19:2376]
-A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "
COMMIT
# Completed on Sun Dec 14 15:59:05 2003
# Generated by iptables-save v1.2.9 on Sun Dec 14 15:59:05 2003
*filter
:INPUT ACCEPT [28639:3847013]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [26167:3457193]
-A INPUT -p udp -j LOG --log-prefix "INPUT: "
COMMIT
# Completed on Sun Dec 14 15:59:05 2003

(Note that filter INPUT saw 28639 packets and PREROUTING just 1474 since last boot)

> Jeff

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[Jeffrey Laramie]

On Sunday 14 December 2003 11:04, horape@tinuviel.compendium.com.ar wrote:
> > > I do:
> > >
> > > iptables -t nat -A PREROUTING -p udp -j LOG --log-prefix "PREROUTING: "
> > > iptables -t filter -A INPUT -p udp -j LOG --log-prefix "INPUT: "
> > >
> > > (I'm logged in that box via ssh, so I prefer not to do -p all)
> > >
> > > There are no more rules than these ones.
> > >
> > > In the log I see the INPUT ones but not the PREROUTING (only see on
> > > PREROUTING packets to port 137, maybe some worm...)
> >
> > That's very strange. You said in your first post that you had a udp proxy
> > running. Is it on this box? I'm not sure at what level a proxy hooks into
> > the box, but I know that some programs read the data stream before
> > netfilter sees it. If the proxy is in front of netfilter then netfilter
> > would never see those packets. The port 137 packets are prolly just
> > internet garbage.
>
> The proxy has a udp socket bound to port 5000, and another udp socket bound
> to some other port, it reads from the first socket and does a sendto using
> the other socket (poll + read + write, no more)
>

It looks like the proxy is grabbing the packets first and then dropping them 
directly onto the INPUT chain. Try disabling the proxy and release the bound 
ports then try it again. Once the packets reach PREROUTING you can DNAT them 
to another port.

Jeff

Re: DNATing packets sent to the NATing box

[Antony Stone]

On Sunday 14 December 2003 4:55 pm, Jeffrey Laramie wrote:

> On Sunday 14 December 2003 11:04, horape@tinuviel.compendium.com.ar wrote:
>
> > The proxy has a udp socket bound to port 5000, and another udp socket
> > bound to some other port, it reads from the first socket and does a
> > sendto using the other socket (poll + read + write, no more)
>
> It looks like the proxy is grabbing the packets first and then dropping
> them directly onto the INPUT chain. Try disabling the proxy and release the
> bound ports then try it again. Once the packets reach PREROUTING you can
> DNAT them to another port.

I could ubnderstand the proxy code managing to grab the packet off the wire 
before netfilter (PREROUTING) sees it, but I don't see how it would then get 
seen by the INPUT chain - as far as I know, it's not possible for a packet to 
reach netfilter's INPUT chain without first going through the PREROUTING 
chain.   If a packet bypasses one of these, it will bypass both.

However, the idea of disabling the proxy, and then seeing if the LOG rules 
show anything different, is a good one.

How is the proxy connected to the socket?   Anything unusual?

Antony

-- 
Having been asked for a reference for this man,
I can confirm that you will be very lucky indeed if you can get him to work 
for you.

                                                     Please reply to the list;
                                                           please don't CC me.

Re: DNATing packets sent to the NATing box

[Jeffrey Laramie]

On Sunday 14 December 2003 12:16, Antony Stone wrote:
> On Sunday 14 December 2003 4:55 pm, Jeffrey Laramie wrote:
> > On Sunday 14 December 2003 11:04, horape@tinuviel.compendium.com.ar wrote:
> > > The proxy has a udp socket bound to port 5000, and another udp socket
> > > bound to some other port, it reads from the first socket and does a
> > > sendto using the other socket (poll + read + write, no more)
> >
> > It looks like the proxy is grabbing the packets first and then dropping
> > them directly onto the INPUT chain. Try disabling the proxy and release
> > the bound ports then try it again. Once the packets reach PREROUTING you
> > can DNAT them to another port.
>
> I could ubnderstand the proxy code managing to grab the packet off the wire
> before netfilter (PREROUTING) sees it, but I don't see how it would then
> get seen by the INPUT chain - as far as I know, it's not possible for a
> packet to reach netfilter's INPUT chain without first going through the
> PREROUTING chain.   If a packet bypasses one of these, it will bypass both.

Well that's what I thought but I can't explain his results any other way. What 
are we missing here?

Jeff

Re: DNATing packets sent to the NATing box

[horape]

> > It looks like the proxy is grabbing the packets first and then dropping
> > them directly onto the INPUT chain. Try disabling the proxy and release the
> > bound ports then try it again. Once the packets reach PREROUTING you can
> > DNAT them to another port.
> I could ubnderstand the proxy code managing to grab the packet off the wire 
> before netfilter (PREROUTING) sees it, but I don't see how it would then get 
> seen by the INPUT chain - as far as I know, it's not possible for a packet to 
> reach netfilter's INPUT chain without first going through the PREROUTING 
> chain.   If a packet bypasses one of these, it will bypass both.

> However, the idea of disabling the proxy, and then seeing if the LOG rules 
> show anything different, is a good one.

> How is the proxy connected to the socket?   Anything unusual?

The proxy is just a normal udp socket. (socket, bind, poll, recvfrom...)

What I'm doing is an IVR over H.323, the program receives and send rtp
packets until a conference is made, when the conference is made it starts
working as a proxy (ie, it gets the packet with a recvfrom and sends by
other socket with sendto) Nothing strange there.

> Antony

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[horape]

> > > > It looks like the proxy is grabbing the packets first and then dropping
> > > > them directly onto the INPUT chain. Try disabling the proxy and release
> > > > the bound ports then try it again. Once the packets reach PREROUTING
> > > > you can DNAT them to another port.
> > > I could ubnderstand the proxy code managing to grab the packet off the
> > > wire before netfilter (PREROUTING) sees it, but I don't see how it would
> > > then get seen by the INPUT chain - as far as I know, it's not possible
> > > for a packet to reach netfilter's INPUT chain without first going through
> > > the PREROUTING chain.   If a packet bypasses one of these, it will bypass
> > > both.
> > Well that's what I thought but I can't explain his results any other way.
> > What are we missing here?

> 	My question exactly ... 
> 	a silly question is :
> 	what if anything is in /proc/net/ip_conntrack for these connections?

udp      17 179 src=200.68.94.100 dst=200.61.169.146 sport=5000 dport=18416 src=200.61.169.146 dst=
200.68.94.100 sport=18416 dport=5000 [ASSURED] use=2

(200.68.94.100 is my ip)

> 	Alistair Tonner

Saludos y gracias,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu

Re: DNATing packets sent to the NATing box

[horape]

> I've a system that at its core has an UDP proxy that's the performance
> bottleneck. I wanted to use the DNAT kernel facilities to replace my
> code with the very tuned one on netfilter.

> I'm adding a rule that says something like this:
> /sbin/iptables -t nat -A PREROUTING -d myip -p udp -m udp --dport 5000 -j DNAT
> 		--to-destination otherip:18918
> but the rule never see the packets (they never got to the chain)

> I assume that it's because I've a socket listening on udp:5000, and it seems
> reasonable what's happening... I'd like to add a PREPREROUTING chain that
> is processed before deciding if the packet is for a local socket, can somebody
> give me a hint on where to look for it?

Thanks to everybody who helped. I've just wrote a new mangle target that
does the translation and it's working great.

Saludos,
					HoraPe
---
Horacio J. Peña
horape@compendium.com.ar
horape@uninet.edu