Packets marked by iptables only sent to the correct routing table sometimes

All of lore.kernel.org
 help / color / mirror / Atom feed

From: Jeff Cook <jeff@deserettechnology.com>
To: netfilter@vger.kernel.org
Cc: pablo@netfilter.org, kaber@trash.net
Subject: Packets marked by iptables only sent to the correct routing table sometimes
Date: Tue, 30 Oct 2012 11:21:01 -0600	[thread overview]
Message-ID: <50900C7D.2010300@deserettechnology.com> (raw)

Hello.

I am trying to route packets generated by a specific user out over a
VPN. I have this configuration:

    $ sudo iptables -S -t nat
    -P PREROUTING ACCEPT
    -P OUTPUT ACCEPT
    -P POSTROUTING ACCEPT
    -A POSTROUTING -o tun0 -j MASQUERADE

    $ sudo iptables -S -t mangle
    -P PREROUTING ACCEPT
    -P INPUT ACCEPT
    -P FORWARD ACCEPT
    -P OUTPUT ACCEPT
    -P POSTROUTING ACCEPT
    -A OUTPUT -m owner --uid-owner guy -j MARK --set-xmark 0xb/0xffffffff

    $ sudo ip rule show
    0:      from all lookup local
    32765:  from all fwmark 0xb lookup 11
    32766:  from all lookup main
    32767:  from all lookup default

    $ sudo ip route show table 11
    10.8.0.5 dev tun0  proto kernel  scope link  src 10.8.0.6
    10.8.0.6 dev tun0  scope link
    10.8.0.1 via 10.8.0.5 dev tun0
    0.0.0.0/1 via 10.8.0.5 dev tun0

    $ sudo iptables -S -t raw
    -P PREROUTING ACCEPT
    -P OUTPUT ACCEPT
    -A OUTPUT -m owner --uid-owner guy -j TRACE
    -A OUTPUT -p tcp -m tcp --dport 80 -j TRACE

It seems that some sites work fine and use the VPN, but others don't and
fall back to the normal interface. This is bad. This is a packet trace
that used VPN:

    Oct 27 00:24:28 agent kernel: [612979.976052] TRACE:
raw:OUTPUT:rule:2 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999
    Oct 27 00:24:28 agent kernel: [612979.976105] TRACE:
raw:OUTPUT:policy:3 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999
    Oct 27 00:24:28 agent kernel: [612979.976164] TRACE:
mangle:OUTPUT:rule:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999
    Oct 27 00:24:28 agent kernel: [612979.976210] TRACE:
mangle:OUTPUT:policy:2 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:24:28 agent kernel: [612979.976269] TRACE:
nat:OUTPUT:policy:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:24:28 agent kernel: [612979.976320] TRACE:
filter:OUTPUT:policy:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:24:28 agent kernel: [612979.976367] TRACE:
mangle:POSTROUTING:policy:1 IN= OUT=tun0 SRC=XXX.YYY.ZZZ.AAA
DST=23.1.17.194 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP
SPT=57502 DPT=80 SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0
OPT (020405B40402080A03A6E01D0000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:24:28 agent kernel: [612979.976414] TRACE:
nat:POSTROUTING:rule:1 IN= OUT=tun0 SRC=XXX.YYY.ZZZ.AAA DST=23.1.17.194
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14494 DF PROTO=TCP SPT=57502 DPT=80
SEQ=2294732931 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6E01D0000000001030307) UID=999 GID=999 MARK=0xb

and this is one that didn't:

    Oct 27 00:22:41 agent kernel: [612873.662559] TRACE:
raw:OUTPUT:rule:2 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999
    Oct 27 00:22:41 agent kernel: [612873.662609] TRACE:
raw:OUTPUT:policy:3 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999
    Oct 27 00:22:41 agent kernel: [612873.662664] TRACE:
mangle:OUTPUT:rule:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999
    Oct 27 00:22:41 agent kernel: [612873.662709] TRACE:
mangle:OUTPUT:policy:2 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:22:41 agent kernel: [612873.662761] TRACE:
nat:OUTPUT:policy:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:22:41 agent kernel: [612873.662808] TRACE:
filter:OUTPUT:policy:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA DST=209.68.27.16
LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP SPT=45305 DPT=80
SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999 MARK=0xb
    Oct 27 00:22:41 agent kernel: [612873.662855] TRACE:
mangle:POSTROUTING:policy:1 IN= OUT=eth0 SRC=XXX.YYY.ZZZ.AAA
DST=209.68.27.16 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=40425 DF PROTO=TCP
SPT=45305 DPT=80 SEQ=604973951 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT
(020405B40402080A03A6B6960000000001030307) UID=999 GID=999 MARK=0xb

I have already tried "ip route flush cache", to no avail. I do not know
why the first packet goes through the correct routing table, and the
second doesn't. Both are marked.

Once again, I do not want ALL packets system-wide to go through the VPN,
I only want packets from a specific user (UID=999) to go through the
VPN. I am testing ipchicken.com and walmart.com via `links`, from the
same user, same shell. walmart.com appears to use the VPN; ipchicken.com
does not.

I have tried iptables -t raw -A OUTPUT -j NOTRACK to circumvent
conntrack interference, but this hasn't worked either.

Any help appreciated; need this resolved ASAP. If this is something that
can't be resolved by volunteers on a mailing list and someone is
available as a consultant and can look into this further, would
appreciate it; email me privately with rate information and credentials.

Thanks
Jeff

next             reply	other threads:[~2012-10-30 17:21 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-10-30 17:21 Jeff Cook [this message]
2012-10-30 19:10 ` Packets marked by iptables only sent to the correct routing table sometimes Pablo Neira Ayuso
2012-10-30 19:16   ` Pablo Neira Ayuso
2012-10-30 23:25     ` Jeff Cook
2012-10-30 23:45       ` Ed W
2012-10-31  0:08       ` Pablo Neira Ayuso

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=50900C7D.2010300@deserettechnology.com \
    --to=jeff@deserettechnology.com \
    --cc=kaber@trash.net \
    --cc=netfilter@vger.kernel.org \
    --cc=pablo@netfilter.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Be sure your reply has a Subject: header at the top and a blank line before the message body.

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.