* Does policy routing based on owner-gid really work?
@ 2008-08-22 23:38 jon hale
2008-09-01 13:14 ` Patrick McHardy
0 siblings, 1 reply; 6+ messages in thread
From: jon hale @ 2008-08-22 23:38 UTC (permalink / raw)
To: netfilter-devel
I have a problem when I combine owner-gid, fwmark, and iproute2.
I am starting to wonder if I can really get there from here.
Synopsis:
I have been trying to set up policy routing based upon the group id
of the process sending the packets.
It works for most packets, but there is some scenario that happens at
the end of every ftp upload, where the packet goes out the wrong
interface and gums up the works.
I am doing this using iptables and iproute2 features. When I change
my iptables fwmark rule from marking gid-owner packets to marking
packets based on their destination address it all works fine.
Details:
in /etc/iproute2/rt_table I added a table p0
I added a rule (with ip ru add) "from all fwmark 0x1 lookup p0"
I bring up my ppp daemon and I get the following address
(from ip addr sh ppp0):
inet 70.223.3.172 peer 66.174.168.192/32 scope global ppp0
I create the entries for table p0:
TARGETIP dev ppp0 scope link
default via 66.174.168.192 dev ppp0 src 70.223.3.172
I take the ppp0 default rule out of table main.
I added some iptables rules:
iptables -t mangle -A OUTPUT -m mark ! --mark 0 -j ACCEPT
iptables -t mangle -A OUTPUT -m mark \
--mark 0 -m owner --gid-owner $GROUP_0 -j MARK \
--set-mark 0x1
iptables -t mangle -A OUTPUT -j CONNMARK --save-mark
iptables -t nat -A POSTROUTING -o ppp0 -j SNAT \
--to-source 70.223.3.172
I then flush the route cache (ip route flush cache)
Now I run ftp and upload a file. Ouch! A few of the packets go out
my eth0 interface.
Interesting facts:
I can successfully upload a file up to size 4344 bytes. When I try using
a file of size 4345 bytes. Things go bad. 4344 is interesting because
this is three times the tcp payload of one packet.
If I change the --set-mark rule above to :
iptables -t mangle -A OUTPUT -d $TARGET -j MARK --set-mark 0x1
Then everything works fine. But this will not accomplish my larger goal.
My larger goal is having two processes owned by the same user, but in
different groups to be able to have packets to $TARGET take different
routes.
If I change the --set-mark rule above to match on the uid instead of
the gid, I still have the same problem.
Versions of things:
#ip -V
ip utility, iproute2-ss070313
#iptables -V
iptables v1.3.7
#linux kernel 2.6.22.9 WITHOUT SMP
I need a clue.
Or maybe just help understanding the clues that I do have.
Thank you,
-jon
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Does policy routing based on owner-gid really work? 2008-08-22 23:38 Does policy routing based on owner-gid really work? jon hale @ 2008-09-01 13:14 ` Patrick McHardy 2008-09-02 18:44 ` jon hale 0 siblings, 1 reply; 6+ messages in thread From: Patrick McHardy @ 2008-09-01 13:14 UTC (permalink / raw) To: jon hale; +Cc: netfilter-devel jon hale wrote: > I have a problem when I combine owner-gid, fwmark, and iproute2. > I am starting to wonder if I can really get there from here. > > Synopsis: > I have been trying to set up policy routing based upon the group id > of the process sending the packets. > > It works for most packets, but there is some scenario that happens at > the end of every ftp upload, where the packet goes out the wrong > interface and gums up the works. The stack may send packets that don't belong to the original socket. You need to use CONNMARK to make sure all packets of a connection are marked similar. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Does policy routing based on owner-gid really work? 2008-09-01 13:14 ` Patrick McHardy @ 2008-09-02 18:44 ` jon hale 2008-09-03 14:22 ` Eric Leblond 0 siblings, 1 reply; 6+ messages in thread From: jon hale @ 2008-09-02 18:44 UTC (permalink / raw) To: Patrick McHardy; +Cc: netfilter-devel On Mon, Sep 1, 2008 at 6:14 AM, Patrick McHardy <kaber@trash.net> wrote: > jon hale wrote: >> >> I have a problem when I combine owner-gid, fwmark, and iproute2. >> I am starting to wonder if I can really get there from here. >> >> Synopsis: >> I have been trying to set up policy routing based upon the group id >> of the process sending the packets. >> >> It works for most packets, but there is some scenario that happens >> at >> the end of every ftp upload, where the packet goes out the wrong >> interface and gums up the works. > > The stack may send packets that don't belong to the original socket. > You need to use CONNMARK to make sure all packets of a connection > are marked similar. > Hmm, I thought I was using CONNMARK I do have the iptables command: iptables -t mangle -A OUTPUT -j CONNMARK --save-mark Is there something else I need as well? Thank you for responding, -jon ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Does policy routing based on owner-gid really work? 2008-09-02 18:44 ` jon hale @ 2008-09-03 14:22 ` Eric Leblond 2008-09-03 22:52 ` jon hale 0 siblings, 1 reply; 6+ messages in thread From: Eric Leblond @ 2008-09-03 14:22 UTC (permalink / raw) To: jon hale; +Cc: Patrick McHardy, netfilter-devel Hello, On Tuesday, 2008 September 2 at 11:44:18 -0700, jon hale wrote: > On Mon, Sep 1, 2008 at 6:14 AM, Patrick McHardy <kaber@trash.net> wrote: > > jon hale wrote: > >> > >> I have a problem when I combine owner-gid, fwmark, and iproute2. > >> I am starting to wonder if I can really get there from here. > >> > >> Synopsis: > >> I have been trying to set up policy routing based upon the group id > >> of the process sending the packets. > >> > >> It works for most packets, but there is some scenario that happens > >> at > >> the end of every ftp upload, where the packet goes out the wrong > >> interface and gums up the works. > > > > The stack may send packets that don't belong to the original socket. > > You need to use CONNMARK to make sure all packets of a connection > > are marked similar. > > > Hmm, I thought I was using CONNMARK > I do have the iptables command: > iptables -t mangle -A OUTPUT -j CONNMARK --save-mark > > Is there something else I need as well? maybe: iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark BR, -- Eric Leblond INL: http://www.inl.fr/ NuFW: http://www.nufw.org/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Does policy routing based on owner-gid really work? 2008-09-03 14:22 ` Eric Leblond @ 2008-09-03 22:52 ` jon hale 2008-09-04 6:59 ` Eric Leblond 0 siblings, 1 reply; 6+ messages in thread From: jon hale @ 2008-09-03 22:52 UTC (permalink / raw) To: Eric Leblond, jon hale, Patrick McHardy, netfilter-devel Hi, Well I tried adding the CONNMARK line that you suggest. I then try a simple ftp from here to there of a 2 byte file. I run tcpdumps on both the ppp0 and eth0 interfaces catching all of the packets that are going out to "there". What I see is that when my machine (here) is responding to the Fin packet sent by the server that this goes out eth0. When I thought that I had configured it otherwise. All other packets until that point seem happy to go out ppp0. Here is the end of my tcpdump for ppp0 (sorry about the newlines): 15:19:55.751793 IP here.56122 > there.31436: P 1:3(2) ack 1 win 183 <nop,nop,timestamp 5883109 3768519952> 15:19:55.751818 IP here.56122 > there.31436: F 3:3(0) ack 1 win 183 <nop,nop,timestamp 5883109 3768519952> 15:19:55.937661 IP there.31436 > here.56122: . ack 3 win 49232 <nop,nop,timestamp 3768519990 5883109> 15:19:55.938657 IP there.31436 > here.56122: . ack 4 win 49232 <nop,nop,timestamp 3768519990 5883109> 15:19:55.939657 IP there.31436 > here.56122: F 1:1(0) ack 4 win 49232 <nop,nop,timestamp 3768519991 5883109> ************************1 15:19:55.944660 IP there.ftp > here.49902: P 99:123(24) ack 14 win 49232 <nop,nop,timestamp 3768519991 5883109> 15:19:55.944700 IP here.49902 > there.ftp: . ack 123 win 216 <nop,nop,timestamp 5883302 3768519991> 15:19:59.324588 IP there.31436 > here.56122: F 1:1(0) ack 4 win 49232 <nop,nop,timestamp 3768520328 5883109> *************************2 15:20:05.364448 IP there.31436 > here.56122: R 3806895453:3806895453(0) win 0 While on eth0 the following two packets went out: 15:19:55.939675 IP here.56122 > there.31436: . ack 3806895454 win 183 <nop,nop,timestamp 5883297 3768519991> 15:19:59.324628 IP here.56122 > there.31436: . ack 1 win 183 <nop,nop,timestamp 5886681 3768519991> Looks like they are both acking the FIN sent by the server. Any ideas on why this packet would go out the "wrong" interface? Suggestions on how I may continue to debug this? Thank you for your time, -jon On Wed, Sep 3, 2008 at 7:22 AM, Eric Leblond <eric@inl.fr> wrote: > Hello, > > On Tuesday, 2008 September 2 at 11:44:18 -0700, jon hale wrote: >> On Mon, Sep 1, 2008 at 6:14 AM, Patrick McHardy <kaber@trash.net> wrote: >> > jon hale wrote: >> >> >> >> I have a problem when I combine owner-gid, fwmark, and iproute2. >> >> I am starting to wonder if I can really get there from here. >> >> >> >> Synopsis: >> >> I have been trying to set up policy routing based upon the group id >> >> of the process sending the packets. >> >> >> >> It works for most packets, but there is some scenario that happens >> >> at >> >> the end of every ftp upload, where the packet goes out the wrong >> >> interface and gums up the works. >> > >> > The stack may send packets that don't belong to the original socket. >> > You need to use CONNMARK to make sure all packets of a connection >> > are marked similar. >> > >> Hmm, I thought I was using CONNMARK >> I do have the iptables command: >> iptables -t mangle -A OUTPUT -j CONNMARK --save-mark >> >> Is there something else I need as well? > > maybe: > iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark > > BR, > -- > Eric Leblond > INL: http://www.inl.fr/ > NuFW: http://www.nufw.org/ > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Does policy routing based on owner-gid really work? 2008-09-03 22:52 ` jon hale @ 2008-09-04 6:59 ` Eric Leblond 0 siblings, 0 replies; 6+ messages in thread From: Eric Leblond @ 2008-09-04 6:59 UTC (permalink / raw) To: jon hale; +Cc: Patrick McHardy, netfilter-devel Hello, On Wednesday, 2008 September 3 at 15:52:37 -0700, jon hale wrote: > Hi, > Well I tried adding the CONNMARK line that you suggest. > I then try a simple ftp from here to there of a 2 byte file. > > I run tcpdumps on both the ppp0 and eth0 interfaces catching all of > the packets that are going out to "there". > > What I see is that when my machine (here) is responding to the Fin > packet sent by the server that this goes out eth0. When I thought that > I had configured it otherwise. > All other packets until that point seem happy to go out ppp0. > > Any ideas on why this packet would go out the "wrong" interface? > Suggestions on how I may continue to debug this? You may try to log every packet marked 0x1 in INPUT and OUTPUT filter chain to see it the mark is correclty propagated through the connection. By doing this, you will be able to check if your conditionnal routing can occur correctly. Just to be sure: Is nf_conntrack_ftp loaded ? It is necessary to have RELATED connection detected. BR, -- Eric Leblond INL: http://www.inl.fr/ NuFW: http://www.nufw.org/ ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2008-09-04 6:59 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-08-22 23:38 Does policy routing based on owner-gid really work? jon hale 2008-09-01 13:14 ` Patrick McHardy 2008-09-02 18:44 ` jon hale 2008-09-03 14:22 ` Eric Leblond 2008-09-03 22:52 ` jon hale 2008-09-04 6:59 ` Eric Leblond
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.