invalid socket structure with ip_early_demux

invalid socket structure with ip_early_demux

[Steve Muckle]

Recently I've struggled with crashes in the xt_qtaguid netfilter module.
This module is written by Google and used with Android. The match
function in xt_qtaguid eventually tries to access

  skb->sk->sk_socket->file

What I find is that the sk->sk_socket pointer is sometimes 0xAAAAAAAA,
or PAGE_POISON. In fact everything after the first 16 bytes of the
struct sock sk is PAGE_POISON. I've confirmed that if I change
PAGE_POISON, the values I see in the sk structure change as well.

I was curious how this structure was being allocated/initialized and
instrumented the sk_alloc, sk_free, and sk_clone_lock functions. When
xt_qtaguid encounters a bad struct sock, that sock does not show up as
ever having been allocated (or freed).

The struct sock is being assigned to the skb in tcb_v4_early_demux(). I
modified that function immediately after the sk is assigned from
__inet_lookup_established() to panic if the sk has a sk_socket pointer
of PAGE_POISON. I can reproduce that condition on my target by simply
attempting to mount an NFS volume. Initiating *and* aborting wget
operations also reproduces the issue - simply initiating a bunch of
wgets is not enough to trigger it.

I have not yet been able to reproduce the bad condition when disabling
ip_early_demux via the sysctl. Any possibility this is an actual issue
with that feature? My target is an MSM using the ks8851 ethernet module.

thanks,
Steve

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: invalid socket structure with ip_early_demux

[Eric Dumazet]

On Fri, 2013-02-01 at 18:08 -0800, Steve Muckle wrote:
> Recently I've struggled with crashes in the xt_qtaguid netfilter module.
> This module is written by Google and used with Android. The match
> function in xt_qtaguid eventually tries to access
> 
>   skb->sk->sk_socket->file
> 
> What I find is that the sk->sk_socket pointer is sometimes 0xAAAAAAAA,
> or PAGE_POISON. In fact everything after the first 16 bytes of the
> struct sock sk is PAGE_POISON. I've confirmed that if I change
> PAGE_POISON, the values I see in the sk structure change as well.
> 
> I was curious how this structure was being allocated/initialized and
> instrumented the sk_alloc, sk_free, and sk_clone_lock functions. When
> xt_qtaguid encounters a bad struct sock, that sock does not show up as
> ever having been allocated (or freed).
> 
> The struct sock is being assigned to the skb in tcb_v4_early_demux(). I
> modified that function immediately after the sk is assigned from
> __inet_lookup_established() to panic if the sk has a sk_socket pointer
> of PAGE_POISON. I can reproduce that condition on my target by simply
> attempting to mount an NFS volume. Initiating *and* aborting wget
> operations also reproduces the issue - simply initiating a bunch of
> wgets is not enough to trigger it.
> 
> I have not yet been able to reproduce the bad condition when disabling
> ip_early_demux via the sysctl. Any possibility this is an actual issue
> with that feature? My target is an MSM using the ks8851 ethernet module.


skb->sk might be a TIMEWAIT socket.

Therefore, you cant blindly use skb->sk->sk_socket

Re: invalid socket structure with ip_early_demux

[Eric Dumazet]

On Fri, 2013-02-01 at 18:16 -0800, Eric Dumazet wrote:

> skb->sk might be a TIMEWAIT socket.
> 
> Therefore, you cant blindly use skb->sk->sk_socket
> 

A fix would be to add after :

sk = skb->sk;

the following code :

if (sk && sk->sk_state == TCP_TIME_WAIT)
	sk = NULL;

Re: invalid socket structure with ip_early_demux

[Steve Muckle]

On 02/01/13 18:25, Eric Dumazet wrote:
> A fix would be to add after :
> 
> sk = skb->sk;
> 
> the following code :
> 
> if (sk && sk->sk_state == TCP_TIME_WAIT)
> 	sk = NULL;

Thanks. I assumed the xt_qtaguid code was ok because xt_owner does a
similar thing - does that also need this fix?

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: invalid socket structure with ip_early_demux

[Eric Dumazet]

On Fri, 2013-02-01 at 18:27 -0800, Steve Muckle wrote:
> On 02/01/13 18:25, Eric Dumazet wrote:
> > A fix would be to add after :
> > 
> > sk = skb->sk;
> > 
> > the following code :
> > 
> > if (sk && sk->sk_state == TCP_TIME_WAIT)
> > 	sk = NULL;
> 
> Thanks. I assumed the xt_qtaguid code was ok because xt_owner does a
> similar thing - does that also need this fix?
> 

I believe xt_owner only is valid on the OUT path
(LOCAL_OUT and POST_ROUTING), so it shouldnt be an issue.

I fixed netfilter modules in 

commit 0626af3139572610b56376580d11eb65d45d9dd7
Author: Eric Dumazet <edumazet@google.com>
Date:   Tue Sep 4 07:49:03 2012 +0000

    netfilter: take care of timewait sockets
    
    Sami Farin reported crashes in xt_LOG because it assumes skb->sk is a
    full blown socket.
    
    Since (41063e9 ipv4: Early TCP socket demux), we can have skb->sk
    pointing to a timewait socket.
    
    Same fix is needed in nfnetlink_log.
    
    Diagnosed-by: Florian Westphal <fw@strlen.de>
    Reported-by: Sami Farin <hvtaifwkbgefbaei@gmail.com>
    Signed-off-by: Eric Dumazet <edumazet@google.com>
    Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>