Re: can't port forward on multihome

[sean darcy <seandarcy2@gmail.com>]

Pascal Hambourg wrote:
> Hello,
> 
> sean darcy a écrit :
>> I have a multihomed server: eth0 is a static T1, and eth3 is a Verizon 
>> dsl line. I want eth3 as the default for general traffic, and eth0 for 
>> VOIP traffic.
>>
>> eth1 is the internal interface. eth3 works fine as the masquerade out 
>> for NAT'd lan.
>>
>> I've used ip to set up eth0 so I can ssh into it:
>>
>> ## eth0 is static
>> ETH0_IP_ADDR=www.xxx.yyy.zzz
>> ip rule add from $ETH0_IP_ADDR/32 table 128 priority 128
>> ## this is the route through the gateway ip
>> ip route add default via <eth0 gateway ip> table 128
>>
>> and that works. Which is important since that's the static address; 
>> the Verizon dsl address is dynamic.
>>
>> The VOIP server ( asterisk ) is on the lan. I've tried to port forward 
>> ssh to the voip server:
>>
>> $IPT -t nat -A PREROUTING  -p tcp --dport 2280 -j DNAT --to 
>> 10.10.10.180:22
>> $IPT -A FORWARD -p tcp --dport 22 -m state --state NEW -d 10.10.10.180 
>> -j ACCEPT
>>
>> This works if I ssh to the eth3, the dynamic dsl interface:
>>
>> ssh -p 2280 voip@<dsl ip address>
>>
>> I get an ssh session on the voip server.
>>
>> But:
>>
>> ssh -p 2280 voip@<static ip address>
>>
>> doesn't work. But I need to have others access the voip server using a 
>> static ip, but not give them access to the multihomed server.
> 
> 
> The ip rule won't work for reply packets sent by the server, because . 
> source address mangling occurs after the routing decision so the source 
> address is 10.10.10.180, not (yet) eth0's address. If Verizon drops 
> packets sent with a source address other than the one assigned to eth3, 
> then the client won't receive any reply and the connection will fail.
> 
> In order to route the reply packets using table 128, you need to 
> identify them. I guess that 10.10.10.180:22 as the source address:port 
> is not discriminant enough, as it matches connections forwarded from 
> eth3 too.
> 
> You can use the CONNMARK target to mark the incoming connection on eth0 
> and copy the connection mark to the reply packets on eth1. Then you can 
> use the packet mark in an ip rule.
> 
> iptables -t mangle -A PREROUTING -i eth0 -m state --state NEW \
>   -j CONNMARK --set-mark 0x1
> iptables -t mangle -A PREROUTING -i eth1 -j CONNMARK --restore-mark
> ip rule add fwmark 0x1 table 128 prio 127
> 
> As you used DNAT, you may use the --ctorigdst option of the 'conntrack' 
> match and mark reply packets based on the original destination address 
> of the connection.
> 
> iptables -t mangle -A PREROUTING -i eth1 \
>   -m connmark --ctorigdst $ETH0_IP_ADDR -j MARK --set-mark 0x1
> ip rule add fwmark 0x1 table 128 prio 127


Thanks for the quick response. I, on the other hand, took some time off 
around xmas.

As always, it takes time for me to think through these ip/iptables 
problems. But I realized you'd pointed me in the direction of how to 
solve my general problem, not just ssh. I realized I could have all 
packets from the voip server go out the T1 interface quite simply:

#!/sh/bin
## eth0 is static to broadview
ETH0_IP_ADDR=www.xxx.yyy.zzz
ip rule add from $ETH0_IP_ADDR/32 table 128 priority 128
## this is the route through broadview gateway ip
ip route add default via <eth0 gateway ip> table 128

## this should make all packets from the * server go out over broadview
iptables -t mangle -A PREROUTING -i eth1 \
   -s 10.10.10.180 -j MARK --set-mark 0x1
## this is supposed to make all packets replying to eth0
## go out eth0
iptables -t mangle -A PREROUTING -i eth0 -m state --state NEW \
    -j CONNMARK --set-mark 0x1
iptables -t mangle -A PREROUTING -i eth1 -j CONNMARK --restore-mark
## this won't work on iptables 1.4.1
# iptables -t mangle -A PREROUTING -i eth1 \
#  -m connmark --ctorigdst $ETH0_IP_ADDR -j MARK --set-mark 0x1
ip rule add fwmark 0x1 table 128 prio 127

so now all traffic, including the voip packets, from the voip server go 
out over the T1, but only the reply traffic from the rest of the lan 
goes out the T1.

As you can see ctorigdst didn't work with iptables in fedora 9:

iptables v1.4.1.1: Unknown arg `--ctorigdst`

which is puzzling, but...

Also, I see you set the new ip rule with priority 127. Am I right that 
higher priority numbers override lower priority number in case of a 
conflict? Or does it determine the order in which rules are applied, 
smaller numbers first?

Thanks for the help.

sean