netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Pekka Pietikäinen]

After a kernel update to 3.11 (feat. commit 
681f130f39e10087475383e6771b9366e26bab0c) my "generate fake tcp 
connections from random ip addresses" app broke, test case as simple as 
running (on default gw of victim):

s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_IP, socket.IP_TRANSPARENT, 1)

s.bind(("5.5.5.5",6666))
s.connect(("192.168.122.46",22))

---

ip route flush table 100
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

---

*mangle
-N DIVERT
-A DIVERT -j MARK --set-mark 1
-A DIVERT -j ACCEPT
-A PREROUTING -p tcp -m socket -j DIVERT

and the 3-way handshake never finishes. Without -m socket (or with the 
new --nowildcard) it does.

Bug, feature or end-user cluelessness? (no problem fixing my ruleset, 
but it's still a behaviourial change :P )

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Jari Turkia]

On 22.10.2013 16:33, Pekka Pietikäinen wrote:
> After a kernel update to 3.11 (feat. commit
> ...
> and the 3-way handshake never finishes. Without -m socket (or with the 
> new --nowildcard) it does.
>
> Bug, feature or end-user cluelessness? (no problem fixing my ruleset, 
> but it's still a behaviourial change :P )
I have to say, that there is something fishy in 3.11 netfilter. 
nat-table MASQUERADE used to work, but doesn't anymore. I don't know if 
it is generic to kernel, or Fedora Linux -specific, but I'd appreciate 
if somebody could confirm that I'm right or wrong. On my box, I'm 
running KVM and have bridged interfaces for KVM, but I did disable both 
of them and MASQUERADE still fails.

My guess is that the problems you mention and I can observe in my box 
are not common enough for people to notice.

Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1016739

Regards,
Jari Turkia
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Pablo Neira Ayuso]

Hi Pekka,

On Tue, Oct 22, 2013 at 04:33:48PM +0300, Pekka Pietikäinen wrote:
> After a kernel update to 3.11 (feat. commit
> 681f130f39e10087475383e6771b9366e26bab0c) my "generate fake tcp
> connections from random ip addresses" app broke.

Did you give a try to revert it and things were working back fine? I
think the root cause for this behaviour change is not in that patch.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Eric Dumazet]

On Thu, 2013-10-24 at 11:52 +0200, Pablo Neira Ayuso wrote:
> Hi Pekka,
> 
> On Tue, Oct 22, 2013 at 04:33:48PM +0300, Pekka Pietikäinen wrote:
> > After a kernel update to 3.11 (feat. commit
> > 681f130f39e10087475383e6771b9366e26bab0c) my "generate fake tcp
> > connections from random ip addresses" app broke.
> 
> Did you give a try to revert it and things were working back fine? I
> think the root cause for this behaviour change is not in that patch.

Yes, given that the option is off by default, I do not really understand
the issue.

Its true that the option is currently a bit flawed, but my refactoring
of TCP listener should solve the problem soon. I do not feel necessary
to 'fix' xt_socket --nowildcard right now.

Oh, if the binary is too old, make sure commit
681f130f39e100 is there...
("netfilter: xt_socket: fix broken v0 support")

Thanks


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Pekka Pietikäinen]

On 24/10/13 13:15, Eric Dumazet wrote:
> On Thu, 2013-10-24 at 11:52 +0200, Pablo Neira Ayuso wrote:
>> Hi Pekka,
>>
>> On Tue, Oct 22, 2013 at 04:33:48PM +0300, Pekka Pietikäinen wrote:
>>> After a kernel update to 3.11 (feat. commit
>>> 681f130f39e10087475383e6771b9366e26bab0c) my "generate fake tcp
>>> connections from random ip addresses" app broke.
>> Did you give a try to revert it and things were working back fine? I
>> think the root cause for this behaviour change is not in that patch.
> Yes, given that the option is off by default, I do not really understand
> the issue.
>
> Its true that the option is currently a bit flawed, but my refactoring
> of TCP listener should solve the problem soon. I do not feel necessary
> to 'fix' xt_socket --nowildcard right now.
>
Okie, did some poking,  Going to before "use IP early demux" seems to 
have found the real cause:

Old:

[ 1700.684685] sk->sk_state: 2
[ 1700.684688] wildcard: 0 transparent: 1, sk != skb->sk 1
[ 1700.684691] proto 6 192.168.122.93:22 -> 5.5.5.5:45856 (orig 
5.5.5.5:45856) sock ffff8803fb7b1500
[ 1700.685583] sk->sk_state: 4
[ 1700.685585] wildcard: 0 transparent: 1, sk != skb->sk 1
[ 1700.685587] proto 6 192.168.122.93:22 -> 5.5.5.5:45856 (orig 
5.5.5.5:45856) sock ffff8803fb7b1500
[ 1700.688443] sk->sk_state: 6
[ 1700.688445] wildcard: 0 transparent: 1, sk != skb->sk 1

New:

[ 1613.960054] sk->sk_state: 7
[ 1613.960057] wildcard: 1 transparent: 1, sk != skb->sk 0
[ 1613.960060] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
5.5.5.5:43540) sock           (null)
[ 1615.511751] sk->sk_state: 7
[ 1615.511754] wildcard: 1 transparent: 1, sk != skb->sk 0
[ 1615.511756] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
5.5.5.5:43540) sock           (null)
[ 1615.963020] sk->sk_state: 7
[ 1615.963022] wildcard: 1 transparent: 1, sk != skb->sk 0
[ 1615.963024] proto 6 192.168.122.93:22 -> 5.5.5.5:34950 (orig 
5.5.5.5:34950) sock           (null)
[ 1615.963036] sk->sk_state: 7
[ 1615.963037] wildcard: 1 transparent: 1, sk != skb->sk 0
[ 1615.963038] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
5.5.5.5:43540) sock           (null)

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Eric Dumazet]

On Thu, 2013-10-24 at 14:21 +0300, Pekka Pietikäinen wrote:
> On 24/10/13 13:15, Eric Dumazet wrote:
> > On Thu, 2013-10-24 at 11:52 +0200, Pablo Neira Ayuso wrote:
> >> Hi Pekka,
> >>
> >> On Tue, Oct 22, 2013 at 04:33:48PM +0300, Pekka Pietikäinen wrote:
> >>> After a kernel update to 3.11 (feat. commit
> >>> 681f130f39e10087475383e6771b9366e26bab0c) my "generate fake tcp
> >>> connections from random ip addresses" app broke.
> >> Did you give a try to revert it and things were working back fine? I
> >> think the root cause for this behaviour change is not in that patch.
> > Yes, given that the option is off by default, I do not really understand
> > the issue.
> >
> > Its true that the option is currently a bit flawed, but my refactoring
> > of TCP listener should solve the problem soon. I do not feel necessary
> > to 'fix' xt_socket --nowildcard right now.
> >
> Okie, did some poking,  Going to before "use IP early demux" seems to 
> have found the real cause:
> 
> Old:
> 
> [ 1700.684685] sk->sk_state: 2
> [ 1700.684688] wildcard: 0 transparent: 1, sk != skb->sk 1
> [ 1700.684691] proto 6 192.168.122.93:22 -> 5.5.5.5:45856 (orig 
> 5.5.5.5:45856) sock ffff8803fb7b1500
> [ 1700.685583] sk->sk_state: 4
> [ 1700.685585] wildcard: 0 transparent: 1, sk != skb->sk 1
> [ 1700.685587] proto 6 192.168.122.93:22 -> 5.5.5.5:45856 (orig 
> 5.5.5.5:45856) sock ffff8803fb7b1500
> [ 1700.688443] sk->sk_state: 6
> [ 1700.688445] wildcard: 0 transparent: 1, sk != skb->sk 1
> 
> New:
> 
> [ 1613.960054] sk->sk_state: 7
> [ 1613.960057] wildcard: 1 transparent: 1, sk != skb->sk 0
> [ 1613.960060] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
> 5.5.5.5:43540) sock           (null)
> [ 1615.511751] sk->sk_state: 7
> [ 1615.511754] wildcard: 1 transparent: 1, sk != skb->sk 0
> [ 1615.511756] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
> 5.5.5.5:43540) sock           (null)
> [ 1615.963020] sk->sk_state: 7
> [ 1615.963022] wildcard: 1 transparent: 1, sk != skb->sk 0
> [ 1615.963024] proto 6 192.168.122.93:22 -> 5.5.5.5:34950 (orig 
> 5.5.5.5:34950) sock           (null)
> [ 1615.963036] sk->sk_state: 7
> [ 1615.963037] wildcard: 1 transparent: 1, sk != skb->sk 0
> [ 1615.963038] proto 6 192.168.122.93:22 -> 5.5.5.5:43540 (orig 
> 5.5.5.5:43540) sock           (null)
> 

sk_state 7 means TCP_CLOSE

I do not see how a TCP_CLOSE socket can be matched...



--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Pekka Pietikäinen]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

On 24/10/13 15:05, Eric Dumazet wrote:
> sk_state 7 means TCP_CLOSE
>
> I do not see how a TCP_CLOSE socket can be matched...
>
Yep, TCP_CLOSE can't be right, sk_state isn't correct with early demux 
perhaps?

Finding 
https://android.googlesource.com/kernel/common/+/experimental/android-3.8%5E!/ 
I managed to get  the old behaviour with the attached patch, but I'm 
having a hard time following what's really happening.




[-- Attachment #2: xt_socket.diff --]
[-- Type: text/plain, Size: 442 bytes --]

--- /usr/src/debug/kernel-3.11.fc19/linux-3.11.6-200.fc19.x86_64/net/netfilter/xt_socket.c	2013-09-02 23:46:10.000000000 +0300
+++ xt_socket.c	2013-10-24 15:07:59.592607433 +0300
@@ -115,6 +115,8 @@
 	struct nf_conn const *ct;
 	enum ip_conntrack_info ctinfo;
 #endif
+	if (sk && sk->sk_state == TCP_CLOSE)
+	  sk = NULL;
 
 	if (iph->protocol == IPPROTO_UDP || iph->protocol == IPPROTO_TCP) {
 		hp = skb_header_pointer(skb, ip_hdrlen(skb),

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Florian Westphal]

Pekka Pietikäinen <pp@ee.oulu.fi> wrote:
> On 24/10/13 15:05, Eric Dumazet wrote:
> >sk_state 7 means TCP_CLOSE
> >
> >I do not see how a TCP_CLOSE socket can be matched...
> >
> Yep, TCP_CLOSE can't be right, sk_state isn't correct with early
> demux perhaps?

What is weird is that early_demux should NOT influence xt_socket
because from the rules you posted you are using this in PREROUTING,
which is before tcp early demux magic.

Do you have any other netfilter rules (-j TPROXY perhaps?) that could
explain why the skb has a socket attached in the first place by
the time it ends up in the netfilter socket match?

[ ip_rcv() orphans the skb before netfilter prerouting, so skb->sk
  should be NULL ]
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: netfilter: xt_socket: add XT_SOCKET_NOWILDCARD flag causes behavioural change in userspace?

[Florian Westphal]

Pekka Pietikäinen <pp@ee.oulu.fi> wrote:
> On 24/10/13 16:29, Florian Westphal wrote:

[ restored nf-devel cc ]

> >I don't understand. The packet seen by xt_socket is arriving from
> >a remote machine, right?
> Virtual machine that's on virbr0 yep, and packet that used to match
> should be a synack to a connection initiated from the host.

Ah.  Now the picture is clear.  Underlying tun device in bridge,
which uses a socket internally (this is why skb->sk->sk_state is
TCP_CLOSE).

> bridge-nf-call-iptables = 0 and your patch + bridge-nf-call-iptable
> = 1 both made my test work with unmodified xt_socket (and adding
> debug prints shows sk_state is now sane)
> 
> Oh. Adding some more debug prints it's the
> "inet_sk(sk)->inet_rcv_saddr == 0" part that triggered.
> 
> Before 3.11 sk=nf_tproxy_get_sock_v4() was always done, so I suppose
> it was always working on sane data.

Yep.  When called from bridge netfilter the skb is not orphaned
properly, so the tun sk is still around by the time xt_socket is
consulted.

Thanks for reporting!
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html