RE: How to reset everything

["Barry A Rich" <barich@trisectrix.com>]

> -----Original Message-----
> From: netfilter-owner@vger.kernel.org
[mailto:netfilter-owner@vger.kernel.org]
> On Behalf Of ??????????? ??????
> Sent: Thursday, May 07, 2009 11:52 AM
> To: barich@trisectrix.com
> Cc: netfilter@vger.kernel.org
> Subject: RE: How to reset everything
> 
> В Чтв, 07/05/2009 в 09:49 -0400, Barry A Rich пишет:
> > > -----Original Message-----
> > > From: Покотиленко Костик [mailto:casper@meteor.dp.ua]
> > > Sent: Thursday, May 07, 2009 6:04 AM
> > > To: barich@trisectrix.com
> > > Cc: netfilter@vger.kernel.org
> > > Subject: Re: How to reset everything
> > >
> > > В Срд, 06/05/2009 в 18:03 -0400, Barry A Rich пишет:
> > > > We use Netfilter to load balance UDP packets across multiple uplinks
> > (ppp0,
> > > > ppp1, ppp2, ppp3). Uplinks can be added or removed on the fly. When
this
> > > > happens, we reset everything and run the firewall/routing script
that
> > > > matches the new uplink configuration. The reset looks like this:
> > > >
> > > > ######################### Begin reset #########################
> > > >
> > > > iptables -F INPUT
> > > > iptables -P INPUT DROP
> > > > iptables -F OUTPUT
> > > > iptables -P OUTPUT DROP
> > > > iptables -F FORWARD
> > > > iptables -P FORWARD DROP
> > > > iptables -F -t raw
> > > > iptables -F -t nat
> > > > iptables -F -t mangle
> > > >
> > > > ip route del default
> > > > ip route flush table uplink1
> > > > ip route flush table uplink2
> > > > ip route flush table uplink3
> > > > ip route flush table uplink4
> > > > ip route flush dev ppp0
> > > > ip route flush dev ppp1
> > > > ip route flush dev ppp2
> > > > ip route flush dev ppp3
> > > >
> > > > tc qdisc del dev ppp0 root
> > > > tc qdisc del dev ppp1 root
> > > > tc qdisc del dev ppp2 root
> > > > tc qdisc del dev ppp3 root
> > > >
> > > > ip route flush cache
> > > >
> > > > ######################### End reset #########################
> > > >
> > > > For two uplinks, the setup looks like this:
> > > >
> > > > ######################### Begin setup #########################
> > > >
> > > > iptables -t raw -A PREROUTING -i eth0 -p udp --sport 6970 -j NOTRACK
> > > >
> > > > iptables -t mangle -A PREROUTING -p udp --sport 6970 -m statistic
--mode
> > nth
> > > > --every 2 --packet 0 -j MARK --set-mark 1
> > > >
> > > > iptables -t mangle -A PREROUTING -p udp --sport 6970 -m statistic
--mode
> > nth
> > > > --every 2 --packet 1 -j MARK --set-mark 2
> > > >
> > > > tc qdisc add dev ppp0 root handle 1: prio
> > > >
> > > > tc qdisc add dev ppp1 root handle 1: prio
> > > >
> > > > tc filter add dev ppp0 parent 1:0 protocol ip prio 1 \
> > > >     handle 1 fw flowid 1:1 action nat egress x.x.x.x/32 y.y.y.y
> > > >
> > > > tc filter add dev ppp1 parent 1:0 protocol ip prio 1 \
> > > >     handle 2 fw flowid 1:1 action nat egress x.x.x.x/32 z.z.z.z
> > > >
> > > > ######################### End setup #########################
> > > >
> > > > The UDP stream continues to be received on the LAN interface during
the
> > > > reset/setup. The reset/setup works most of the time, but
occasionally
> > the
> > > > packets going out ppp0 do not get NAT'd after a reset/setup.
Repeating
> > the
> > > > setup/reset sequence a second time seems to make it work, but I'd
rather
> > > > understand what's wrong and fix it.
> > > >
> > > > All help is appreciated.
> > > >
> > > > Thanks.
> > >
> > > Try adding to your reset script:
> > >
> > > conntrack -F conntrack
> > > conntrack -F expect
> > >
> > > --
> > > Покотиленко Костик <casper@meteor.dp.ua>
> >
> > That did not solve the problem. Could it be packets that get queued up
while
> > the reset is in progress?
> 
> If this is the case the delay shouldn't be long. Also, if this is the
> case next packets should behave as you expected and you wouldn't be
> reseting second time.
> 
> >  Should the uplink queues get flushed? If so, how
> > is that done?
> 
> tc qdics del && tc qdisc add(???)
> 
> --
> Покотиленко Костик <casper@meteor.dp.ua>

When this problem occurs after a reset/setup, the UDP packets are still
routed to the both ppp0 and ppp1. They just are not NAT'd on ppp0.

This seems to indicate:

- The statistic module is correctly marking the packets, and
- The packets are being routed correctly, and
- The traffic control filter is not NAT'ing the packets.

Is there a way to log the packets (with the marks) as they exit traffic
control to verify this?