RE: [PATCH] Fix for IPsec leakage with SELinux enabled

RE: [PATCH] Fix for IPsec leakage with SELinux enabled

[Venkat Yekkirala]

> On Sun, 1 Oct 2006, Venkat Yekkirala wrote:
> 
> > > The way I was seeing the problem was when connecting via 
> IPsec to a 
> > > confined service on an SELinux box (vsftpd), which did 
> not have the 
> > > appropriate SELinux policy permissions to send packets via IPsec.
> > > 
> > > The first SYNACK would be blocked,
> > 
> > Given that the resolver fails to find a policy here, I am trying to
> > understand what exactly is blocking it (the first SYNACK) from
> > proceeding without IPSec.
> 
> You're right.  My explanation there was for the case where 
> I'd modified 
> the code to propagate the SELinux denial back to 
> xfrm_lookup(), and not 
> for the original issue (sorry, it's been a long week).
> 
> In the original case, all packets originating from a confined 
> domain will 
> bypass ipsec.  During xfrm_lookup, flow_cache_lookup() returns NULL 
> policy:
> 
> xfrm_lookup()
> {
> 	...
> 
> 	if (!policy) {
>                 /* To accelerate a bit...  */
>                 if ((dst_orig->flags & DST_NOXFRM) ||
>                     !xfrm_policy_count[XFRM_POLICY_OUT])
>                         return 0;
> 
>                 policy = flow_cache_lookup(fl, dst_orig->ops->family,
>                                            dir, xfrm_policy_lookup);
>         }
> 
>         if (!policy)
>                 return 0;  <- bad
> 	...
> }
> 
> The call chain is:
> 
> flow_cache_lookup()
>   xfrm_policy_lookup()
>    xfrm_policy_lookup_bytype()
>      xfrm_policy_match() {
>        xfrm_selector_match => returns true, then
>          security_xfrm_policy_lookup => returns -EACCESS
>      }
> 
> then
>      
> xfrm_policy_match() => returns 0
> xfrm_policy_lookup_bytype => returns 0
> xfrm_policy_lookup() => sets obj to NULL w/ void return
> flow_cache_lookup() => returns NULL
> xfrm_lookup => returns 0
> 
> and the packet proceeds without error and with no transform 
> applied (i.e. 
> leaked as cleartext).

This is indeed the "designed" and expected (for me) behavior. But I am
beginning to see where this is perhaps NOT in line with the user
expectation when the users have an IPSec policy rule that does NOT use
labels.

IOW, if they have their policy like (NO LABELS):

spdadd 192.168.4.79 192.168.4.78 any -P out ipsec
        esp/transport//require;

spdadd 192.168.4.78 192.168.4.79 any -P in ipsec
        esp/transport//require;

currently, EACH flow needing to use this rule MUST
have SELinux policy "polmatch"ing the flow context (ftpd_t)
to unlabeled_t (the implied in the absence of an explicit
context on the IPSec policy rule) or the traffic would flow
in clear text ("leaks" in user perception).

What I propose we do is to do the polmatch check ONLY when
there's an explicit label associated with the spd rule. Does
this sound reasonable and correct in the larger SELinux context?

In cases where there's an explicit label on an spd rule like:

spdadd 192.168.4.79 192.168.4.78 any -ctx 1 1
"system_u:object_r:labeled_ipsec_t:s2-s4"
	-P out ipsec
        esp/transport//require;

spdadd 192.168.4.78 192.168.4.79 any -ctx 1 1
"system_u:object_r:labeled_ipsec_t:s2-s4"
	-P in ipsec
        esp/transport//require;

then the current behavior (prior to this proposed patch) would be the
desired behavior, i.e., a polmatch denial in the SELinux module just
means that the flow isn't expected to undergo IPSec xfrms. IOW, there's
no need to propagate -EACCES all the way back up. We could still propagate
errors other than -EACCES if we like.

> 
> The first component of the fix is to propagate errors back up 
> to callers 
> via the flow cache resolver, and handle them correctly, as 
> this is where 
> we can experience security module denials.

This will cause problems with the case where we may have multiple
"labeled" spd rules, each of which should be attempted to be polmatched
against before letting the flow out in clear text, as would be expected
by the user as well.

> 
> The second component is to then ensure that, on failure,

-EACCES in this case is perceived as a failure since the
designed behavior doesn't seem to be in line with the user
expectation. Otherwise, with the above proposed change to
the design, there won't be a polmatch check in this case and
hence no failure.

It's probably still a good idea to propagate non-EACCES failures
back up the chain though.

> we 
> kill the new 
> flow cache entry created during lookup, so that it is not 
> subsequently 
> looked up with a null policy (i.e. allowing future packets to 
> leak) and to 
> force the security hook to be called again in case the policy 
> has changed.  
> 
> Hope that clarifies.
>     
> 
> - James
> -- 
> James Morris
> <jmorris@namei.org>
> 

RE: [PATCH] Fix for IPsec leakage with SELinux enabled

[James Morris]

On Mon, 2 Oct 2006, Venkat Yekkirala wrote:

> This is indeed the "designed" and expected (for me) behavior.

This is a security hole.  SELinux denies all access by default, so the 
default behavior of this code is to allow all traffic to bypass IPsec.

You should not need to add a rule to 'allow' increased security.

> But I am
> beginning to see where this is perhaps NOT in line with the user
> expectation when the users have an IPSec policy rule that does NOT use
> labels.
> currently, EACH flow needing to use this rule MUST
> have SELinux policy "polmatch"ing the flow context (ftpd_t)
> to unlabeled_t (the implied in the absence of an explicit
> context on the IPSec policy rule) or the traffic would flow
> in clear text ("leaks" in user perception).

Plaintext leak is not a user perception, it's an absolute.

> What I propose we do is to do the polmatch check ONLY when
> there's an explicit label associated with the spd rule. Does
> this sound reasonable and correct in the larger SELinux context?

I think so.

> In cases where there's an explicit label on an spd rule like:
> 
> spdadd 192.168.4.79 192.168.4.78 any -ctx 1 1
> "system_u:object_r:labeled_ipsec_t:s2-s4"
> 	-P out ipsec
>         esp/transport//require;
> 
> spdadd 192.168.4.78 192.168.4.79 any -ctx 1 1
> "system_u:object_r:labeled_ipsec_t:s2-s4"
> 	-P in ipsec
>         esp/transport//require;
> 
> then the current behavior (prior to this proposed patch) would be the
> desired behavior, i.e., a polmatch denial in the SELinux module just
> means that the flow isn't expected to undergo IPSec xfrms. IOW, there's
> no need to propagate -EACCES all the way back up. We could still propagate
> errors other than -EACCES if we like.

This needs to be handled within SELinux as far as possible, and errors 
will generally need to be propagated back to the callers, as we don't know 
what other LSMs might do, and errors unrelated to access control can be 
returned.


- James
-- 
James Morris
<jmorris@namei.org>

RE: [PATCH] Fix for IPsec leakage with SELinux enabled

[Venkat Yekkirala]

> > This is indeed the "designed" and expected (for me) behavior.
> 
> This is a security hole.  SELinux denies all access by 
> default, so the 
> default behavior of this code is to allow all traffic to bypass IPsec.
> 
> You should not need to add a rule to 'allow' increased security.

You are right. Currently working on a patch (should be out
tonight/tomorrow).

<snip>

> This needs to be handled within SELinux as far as possible, 
> and errors 
> will generally need to be propagated back to the callers, as 

Agreed here as well. I have yet to review your patch in depth,
but it definitely makes sense to do what you say here. Thanks.

> we don't know 
> what other LSMs might do, and errors unrelated to access 
> control can be 
> returned.
> 
> 
> - James
> -- 
> James Morris
> <jmorris@namei.org>
> 

RE: [PATCH] Fix for IPsec leakage with SELinux enabled

[Venkat Yekkirala]

> The way I was seeing the problem was when connecting via IPsec to a 
> confined service on an SELinux box (vsftpd), which did not have the 
> appropriate SELinux policy permissions to send packets via IPsec.
> 
> The first SYNACK would be blocked,

Given that the resolver fails to find a policy here, I am trying to
understand what exactly is blocking it (the first SYNACK) from
proceeding without IPSec.

> because of an uncached lookup via 
> flow_cache_lookup(), which would fail to resolve an xfrm 
> policy because 
> the SELinux policy is checked at that point via the resolver.
> 
> However, retransmitted SYNACKs would then find a cached flow 
> entry when 
> calling into flow_cache_lookup() with a null xfrm policy, which is 
> interpreted by xfrm_lookup() as the packet not having any associated 
> policy and similarly to the first case, allowing it to pass without 
> transformation.

RE: [PATCH] Fix for IPsec leakage with SELinux enabled

[James Morris]

On Sun, 1 Oct 2006, Venkat Yekkirala wrote:

> > The way I was seeing the problem was when connecting via IPsec to a 
> > confined service on an SELinux box (vsftpd), which did not have the 
> > appropriate SELinux policy permissions to send packets via IPsec.
> > 
> > The first SYNACK would be blocked,
> 
> Given that the resolver fails to find a policy here, I am trying to
> understand what exactly is blocking it (the first SYNACK) from
> proceeding without IPSec.

You're right.  My explanation there was for the case where I'd modified 
the code to propagate the SELinux denial back to xfrm_lookup(), and not 
for the original issue (sorry, it's been a long week).

In the original case, all packets originating from a confined domain will 
bypass ipsec.  During xfrm_lookup, flow_cache_lookup() returns NULL 
policy:

xfrm_lookup()
{
	...

	if (!policy) {
                /* To accelerate a bit...  */
                if ((dst_orig->flags & DST_NOXFRM) ||
                    !xfrm_policy_count[XFRM_POLICY_OUT])
                        return 0;

                policy = flow_cache_lookup(fl, dst_orig->ops->family,
                                           dir, xfrm_policy_lookup);
        }

        if (!policy)
                return 0;  <- bad
	...
}

The call chain is:

flow_cache_lookup()
  xfrm_policy_lookup()
   xfrm_policy_lookup_bytype()
     xfrm_policy_match() {
       xfrm_selector_match => returns true, then
         security_xfrm_policy_lookup => returns -EACCESS
     }

then
     
xfrm_policy_match() => returns 0
xfrm_policy_lookup_bytype => returns 0
xfrm_policy_lookup() => sets obj to NULL w/ void return
flow_cache_lookup() => returns NULL
xfrm_lookup => returns 0

and the packet proceeds without error and with no transform applied (i.e. 
leaked as cleartext).

The first component of the fix is to propagate errors back up to callers 
via the flow cache resolver, and handle them correctly, as this is where 
we can experience security module denials.

The second component is to then ensure that, on failure, we kill the new 
flow cache entry created during lookup, so that it is not subsequently 
looked up with a null policy (i.e. allowing future packets to leak) and to 
force the security hook to be called again in case the policy has changed.  

Hope that clarifies.
    

- James
-- 
James Morris
<jmorris@namei.org>

Re: Is TCP over IPsec broken in 2.6.18?

[James Morris]

On Sun, 24 Sep 2006, Patrick McHardy wrote:

> James Morris wrote:
> > On Sat, 23 Sep 2006, Evgeniy Polyakov wrote:
> > 
> > 
> >>I never saw unencrypted packets before.
> > 
> > 
> > It's normal and expected, perhaps you didn't notice or had tcpdump 
> > filtering them.
> 
> He's talking about transport mode, unencrypted packet should
> only be visible in tunnel mode.

Ok.

I've done some more testing with local tcpdumps and not seeing any issues.

Evgeniy: if you update to the latest racoon (and kernel), and still see 
it, please send complete logs of 'racoon -dddd' from each side, and also 
'setkey -x', so we can see if the policy entries are being being modified.


- James
-- 
James Morris
<jmorris@namei.org>

Re: Is TCP over IPsec broken in 2.6.18?

[Herbert Xu]

James Morris <jmorris@namei.org> wrote:
>
> Evgeniy: if you update to the latest racoon (and kernel), and still see 
> it, please send complete logs of 'racoon -dddd' from each side, and also 
> 'setkey -x', so we can see if the policy entries are being being modified.

Please also do 'ip -s x s' on both sides to check the various counters.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Is TCP over IPsec broken in 2.6.18?

[Herbert Xu]

On Mon, Sep 25, 2006 at 02:38:36PM +0400, Evgeniy Polyakov wrote:
> 
> I ran two times the same 'telnet 192.168.4.79 22' and got unencrypted
> packets and very long timeout. After some magic initial process things
> started to work as expected - only ESP encrypted packets can be found in
> tcpdump, until next connection is started, which becames to work not
> correctly and then again starts to work as expected.

Perhaps something's screwed up with the policies.  Unfortunately
the racoon logs draw a blank around the interesting interval as
shown by the tcpdump.

Please run ip x p once every second and the post what it shows
before, during and after the period when unecrypted packets show
up on the wire.

You only have to do it on the 79 machine since it's the one sending
unencrypted data.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Is TCP over IPsec broken in 2.6.18?

[Evgeniy Polyakov]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On Mon, Sep 25, 2006 at 09:27:54PM +1000, Herbert Xu (herbert@gondor.apana.org.au) wrote:
> On Mon, Sep 25, 2006 at 02:38:36PM +0400, Evgeniy Polyakov wrote:
> > 
> > I ran two times the same 'telnet 192.168.4.79 22' and got unencrypted
> > packets and very long timeout. After some magic initial process things
> > started to work as expected - only ESP encrypted packets can be found in
> > tcpdump, until next connection is started, which becames to work not
> > correctly and then again starts to work as expected.
> 
> Perhaps something's screwed up with the policies.  Unfortunately
> the racoon logs draw a blank around the interesting interval as
> shown by the tcpdump.

I insrted blank lines specially to show where things started to work
correctly (first blank lines), second one shows where I started second 
telnet. I think you've noticed that time difference on machines
is about 30 minutes.

> Please run ip x p once every second and the post what it shows
> before, during and after the period when unecrypted packets show
> up on the wire.
> 
> You only have to do it on the 79 machine since it's the one sending
> unencrypted data.

Attached three files - before, while and after connection establishment.

-- 
	Evgeniy Polyakov

[-- Attachment #2: after --]
[-- Type: text/plain, Size: 3052 bytes --]

src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 

[-- Attachment #3: before --]
[-- Type: text/plain, Size: 4578 bytes --]

src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 

[-- Attachment #4: while --]
[-- Type: text/plain, Size: 9156 bytes --]

src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir in priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.79/32 dst 192.168.4.78/32 
	dir out priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src 192.168.4.78/32 dst 192.168.4.79/32 
	dir fwd priority 2147483648 
	tmpl src 0.0.0.0 dst 0.0.0.0
		proto esp reqid 0 mode transport
src ::/0 dst ::/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir in priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src ::/0 dst ::/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
	dir out priority 0 

Re: Is TCP over IPsec broken in 2.6.18?

[James Morris]

I've just seen something similar and can recreate it with static keying 
via setkey.

The symptom was that ping was only working in one direction, and I 
quintuple-checked the configs and that they have the same kernels etc., 
then ran a bunch of tcpdumps on each and and a router in the middle with 
various protocols.

Some protocols work (ssh) but others (ftp) doesn't.  Also verified the 
problem via simple telnet to these ports.

In one case, the ftp server receives a SYN from the client over ipsec just 
fine but the synack goes out in cleartext (also verified the server is in 
SYN_RECV), and the client drops these.

tcpdump on intermediate router:

[ SYN packet from client, ESP encapsulated as expected ]

IP (tos 0x10, ttl  63, id 4588, offset 0, flags [DF], proto: ESP (50), 
length: 104) 10.1.2.2 > 10.1.3.2: ESP(spi=0x00055555,seq=0x4), length 84

[ SYNACK from server, in the clear, not expected ]

IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto: TCP (6), length: 
60) 10.1.3.2.ftp > 10.1.2.2.53123: S, cksum 0x314c (correct), 
2413701424:2413701424(0) ack 2343803769 win 11844 <mss 
3960,sackOK,timestamp 1018642 690836,nop,wscale 9>

Again note that another protocol such as SSH works as expected.


Connecting from the other side, it looks fine:

IP (tos 0x10, ttl  64, id 33701, offset 0, flags [DF], proto: ESP (50), 
length: 104) 10.1.3.2 > 10.1.2.2: ESP(spi=0x00066666,seq=0x1), length 84

IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto: ESP (50), length: 
104) 10.1.2.2 > 10.1.3.2: ESP(spi=0x00055555,seq=0x1), length 84



Another odd thing I noticed was a ping client in the non-working direction 
segfaulted under strace (once).

These are both x86 machines running the kernel:
2.6.18-1.2699.fc6  (note that this has xen patches but is running bare 
metal).

I ran tcpdump for the above ftp and ssh cases to see if there was anything 
different about the packets (e.g. TOS or TCP opts) but found nothing -- it 
all looks fine, as well as using nc to make sure they're selecting source 
ports of a similar value etc.

Will try with an upstream kernel and see what happens.

-- 
James Morris
<jmorris@namei.org>

Re: Is TCP over IPsec broken in 2.6.18?

[James Morris]

On Sat, 30 Sep 2006, James Morris wrote:

> I've just seen something similar and can recreate it with static keying 
> via setkey.

It's SELinux related.  Things work when the one system in this setup with 
SELinux enabled is changed to permissive mode.

No audit messages or AVCs, and it's not the /selinux/compat_net setting.


- James
-- 
James Morris
<jmorris@namei.org>

Re: Is TCP over IPsec broken in 2.6.18?

[Evgeniy Polyakov]

On Sat, Sep 30, 2006 at 01:14:27AM -0400, James Morris (jmorris@namei.org) wrote:
> On Sat, 30 Sep 2006, James Morris wrote:
> 
> > I've just seen something similar and can recreate it with static keying 
> > via setkey.
> 
> It's SELinux related.  Things work when the one system in this setup with 
> SELinux enabled is changed to permissive mode.
> 
> No audit messages or AVCs, and it's not the /selinux/compat_net setting.

I need to cofirm that broken system in my setup does have selinux enabled 
with enforcing mode.
I've changed it to permissive mode and it fixed setup (I do not see any 
warnings in dmesg).
 
> - James
> -- 
> James Morris
> <jmorris@namei.org>

-- 
	Evgeniy Polyakov

Re: Is TCP over IPsec broken in 2.6.18?

[James Morris]

On Sat, 30 Sep 2006, Evgeniy Polyakov wrote:

> I need to cofirm that broken system in my setup does have selinux enabled 
> with enforcing mode.
> I've changed it to permissive mode and it fixed setup (I do not see any 
> warnings in dmesg).

Something better in your case would likely be to rebuild the kernel with 
CONFIG_SECURITY_NETWORK_XFRM=n until it's fixed.


- James
-- 
James Morris
<jmorris@namei.org>

Re: Is TCP over IPsec broken in 2.6.18?

[Evgeniy Polyakov]

On Sat, Sep 30, 2006 at 10:36:29AM -0400, James Morris (jmorris@namei.org) wrote:
> On Sat, 30 Sep 2006, Evgeniy Polyakov wrote:
> 
> > I need to cofirm that broken system in my setup does have selinux enabled 
> > with enforcing mode.
> > I've changed it to permissive mode and it fixed setup (I do not see any 
> > warnings in dmesg).
> 
> Something better in your case would likely be to rebuild the kernel with 
> CONFIG_SECURITY_NETWORK_XFRM=n until it's fixed.

Well, it is acrypto test machine and I do not care about security there,
so I can even disable selinux completely, but it will not help to resolve
the issue, right?

So if you have some patches I'm more than happy to test them.
 
> - James
> -- 
> James Morris
> <jmorris@namei.org>

-- 
	Evgeniy Polyakov

Re: Is TCP over IPsec broken in 2.6.18?

[James Morris]

On Sat, 30 Sep 2006, Evgeniy Polyakov wrote:

> On Sat, Sep 30, 2006 at 10:36:29AM -0400, James Morris (jmorris@namei.org) wrote:
> > On Sat, 30 Sep 2006, Evgeniy Polyakov wrote:
> > 
> > > I need to cofirm that broken system in my setup does have selinux enabled 
> > > with enforcing mode.
> > > I've changed it to permissive mode and it fixed setup (I do not see any 
> > > warnings in dmesg).
> > 
> > Something better in your case would likely be to rebuild the kernel with 
> > CONFIG_SECURITY_NETWORK_XFRM=n until it's fixed.
> 
> Well, it is acrypto test machine and I do not care about security there,
> so I can even disable selinux completely, but it will not help to resolve
> the issue, right?

Yes, it is a workaround.

> 
> So if you have some patches I'm more than happy to test them.

Ok, coming soon.


-- 
James Morris
<jmorris@namei.org>

[PATCH] Fix for IPsec leakage with SELinux enabled

[James Morris]

Please review this patch carefully.  It addresses a couple of issues.

When a security module is loaded (in this case, SELinux), the 
security_xfrm_policy_lookup() hook can return an access denied permission 
(or other error).  We were not handling that correctly, and in fact 
inverting the return logic and propagating a false "ok" back up to 
xfrm_lookup(), which then allowed packets to pass as if they were not 
associated with an xfrm policy.

The way I was seeing the problem was when connecting via IPsec to a 
confined service on an SELinux box (vsftpd), which did not have the 
appropriate SELinux policy permissions to send packets via IPsec.

The first SYNACK would be blocked, because of an uncached lookup via 
flow_cache_lookup(), which would fail to resolve an xfrm policy because 
the SELinux policy is checked at that point via the resolver.

However, retransmitted SYNACKs would then find a cached flow entry when 
calling into flow_cache_lookup() with a null xfrm policy, which is 
interpreted by xfrm_lookup() as the packet not having any associated 
policy and similarly to the first case, allowing it to pass without 
transformation.

The solution presented here is to first ensure that errno values are 
correctly propagated all the way back up through the various call chains 
from security_xfrm_policy_lookup(), and handled correctly.

Then, flow_cache_lookup() is modified, so that if the policy resolver 
fails (typically a permission denied via the security module), the flow 
cache entry is killed rather than having a null policy assigned (which 
indicates that the packet can pass freely).  This also forces any future 
lookups for the same flow to consult the security module (e.g. SELinux) 
for current security policy (rather than, say, caching the error on the 
flow cache entry).

I've done quite a bit of testing and not seen any problems, although the 
patch could certainly do with further review.

Evgeniy, please let me know if this fixes your problem.


Signed-off-by: James Morris <jmorris@namei.org>


---

 include/net/flow.h     |    2 -
 net/core/flow.c        |   42 +++++++++++++++++++----------
 net/xfrm/xfrm_policy.c |   70 ++++++++++++++++++++++++++++++++++++++-----------
 3 files changed, 84 insertions(+), 30 deletions(-)


diff -purN -X dontdiff net-2.6.o/include/net/flow.h net-2.6.w/include/net/flow.h
--- net-2.6.o/include/net/flow.h	2006-09-29 11:33:58.000000000 -0400
+++ net-2.6.w/include/net/flow.h	2006-09-30 23:50:32.000000000 -0400
@@ -97,7 +97,7 @@ struct flowi {
 #define FLOW_DIR_FWD	2
 
 struct sock;
-typedef void (*flow_resolve_t)(struct flowi *key, u16 family, u8 dir,
+typedef int (*flow_resolve_t)(struct flowi *key, u16 family, u8 dir,
 			       void **objp, atomic_t **obj_refp);
 
 extern void *flow_cache_lookup(struct flowi *key, u16 family, u8 dir,
diff -purN -X dontdiff net-2.6.o/net/core/flow.c net-2.6.w/net/core/flow.c
--- net-2.6.o/net/core/flow.c	2006-09-29 11:33:59.000000000 -0400
+++ net-2.6.w/net/core/flow.c	2006-10-01 01:07:24.000000000 -0400
@@ -85,6 +85,14 @@ static void flow_cache_new_hashrnd(unsig
 	add_timer(&flow_hash_rnd_timer);
 }
 
+static void flow_entry_kill(int cpu, struct flow_cache_entry *fle)
+{
+	if (fle->object)
+		atomic_dec(fle->object_ref);
+	kmem_cache_free(flow_cachep, fle);
+	flow_count(cpu)--;
+}
+
 static void __flow_cache_shrink(int cpu, int shrink_to)
 {
 	struct flow_cache_entry *fle, **flp;
@@ -100,10 +108,7 @@ static void __flow_cache_shrink(int cpu,
 		}
 		while ((fle = *flp) != NULL) {
 			*flp = fle->next;
-			if (fle->object)
-				atomic_dec(fle->object_ref);
-			kmem_cache_free(flow_cachep, fle);
-			flow_count(cpu)--;
+			flow_entry_kill(cpu, fle);
 		}
 	}
 }
@@ -220,24 +225,33 @@ void *flow_cache_lookup(struct flowi *ke
 
 nocache:
 	{
+		int err;
 		void *obj;
 		atomic_t *obj_ref;
 
-		resolver(key, family, dir, &obj, &obj_ref);
+		err = resolver(key, family, dir, &obj, &obj_ref);
 
 		if (fle) {
-			fle->genid = atomic_read(&flow_cache_genid);
-
-			if (fle->object)
-				atomic_dec(fle->object_ref);
-
-			fle->object = obj;
-			fle->object_ref = obj_ref;
-			if (obj)
-				atomic_inc(fle->object_ref);
+			if (err) {
+				/* Force security policy check on next lookup */
+				*head = fle->next;
+				flow_entry_kill(cpu, fle);
+			} else {
+				fle->genid = atomic_read(&flow_cache_genid);
+				
+				if (fle->object)
+					atomic_dec(fle->object_ref);
+					
+				fle->object = obj;
+				fle->object_ref = obj_ref;
+				if (obj)
+					atomic_inc(fle->object_ref);
+			}
 		}
 		local_bh_enable();
 
+		if (err)
+			obj = ERR_PTR(err);
 		return obj;
 	}
 }
diff -purN -X dontdiff net-2.6.o/net/xfrm/xfrm_policy.c net-2.6.w/net/xfrm/xfrm_policy.c
--- net-2.6.o/net/xfrm/xfrm_policy.c	2006-09-29 11:34:00.000000000 -0400
+++ net-2.6.w/net/xfrm/xfrm_policy.c	2006-10-01 01:53:21.000000000 -0400
@@ -880,30 +880,32 @@ out:
 }
 EXPORT_SYMBOL(xfrm_policy_walk);
 
-/* Find policy to apply to this flow. */
-
+/*
+ * Find policy to apply to this flow.
+ *
+ * Returns 0 if policy found, else an -errno.
+ */
 static int xfrm_policy_match(struct xfrm_policy *pol, struct flowi *fl,
 			     u8 type, u16 family, int dir)
 {
 	struct xfrm_selector *sel = &pol->selector;
-	int match;
+	int match, ret = -ESRCH;
 
 	if (pol->family != family ||
 	    pol->type != type)
-		return 0;
+		return ret;
 
 	match = xfrm_selector_match(sel, fl, family);
-	if (match) {
-		if (!security_xfrm_policy_lookup(pol, fl->secid, dir))
-			return 1;
-	}
+	if (match)
+		ret = security_xfrm_policy_lookup(pol, fl->secid, dir);
 
-	return 0;
+	return ret;
 }
 
 static struct xfrm_policy *xfrm_policy_lookup_bytype(u8 type, struct flowi *fl,
 						     u16 family, u8 dir)
 {
+	int err;
 	struct xfrm_policy *pol, *ret;
 	xfrm_address_t *daddr, *saddr;
 	struct hlist_node *entry;
@@ -919,7 +921,15 @@ static struct xfrm_policy *xfrm_policy_l
 	chain = policy_hash_direct(daddr, saddr, family, dir);
 	ret = NULL;
 	hlist_for_each_entry(pol, entry, chain, bydst) {
-		if (xfrm_policy_match(pol, fl, type, family, dir)) {
+		err = xfrm_policy_match(pol, fl, type, family, dir);
+		if (err) {
+			if (err == -ESRCH)
+				continue;
+			else {
+				ret = ERR_PTR(err);
+				goto fail;
+			}
+		} else {
 			ret = pol;
 			priority = ret->priority;
 			break;
@@ -927,36 +937,53 @@ static struct xfrm_policy *xfrm_policy_l
 	}
 	chain = &xfrm_policy_inexact[dir];
 	hlist_for_each_entry(pol, entry, chain, bydst) {
-		if (xfrm_policy_match(pol, fl, type, family, dir) &&
-		    pol->priority < priority) {
+		err = xfrm_policy_match(pol, fl, type, family, dir);
+		if (err) {
+			if (err == -ESRCH)
+				continue;
+			else {
+				ret = ERR_PTR(err);
+				goto fail;
+			}
+		} else if (pol->priority < priority) {
 			ret = pol;
 			break;
 		}
 	}
 	if (ret)
 		xfrm_pol_hold(ret);
+fail:
 	read_unlock_bh(&xfrm_policy_lock);
 
 	return ret;
 }
 
-static void xfrm_policy_lookup(struct flowi *fl, u16 family, u8 dir,
+static int xfrm_policy_lookup(struct flowi *fl, u16 family, u8 dir,
 			       void **objp, atomic_t **obj_refp)
 {
 	struct xfrm_policy *pol;
+	int err = 0;
 
 #ifdef CONFIG_XFRM_SUB_POLICY
 	pol = xfrm_policy_lookup_bytype(XFRM_POLICY_TYPE_SUB, fl, family, dir);
-	if (pol)
+	if (IS_ERR(pol)) {
+		err = PTR_ERR(pol);
+		pol = NULL;
+	}
+	if (pol || err)
 		goto end;
 #endif
 	pol = xfrm_policy_lookup_bytype(XFRM_POLICY_TYPE_MAIN, fl, family, dir);
-
+	if (IS_ERR(pol)) {
+		err = PTR_ERR(pol);
+		pol = NULL;
+	}
 #ifdef CONFIG_XFRM_SUB_POLICY
 end:
 #endif
 	if ((*objp = (void *) pol) != NULL)
 		*obj_refp = &pol->refcnt;
+	return err;
 }
 
 static inline int policy_to_flow_dir(int dir)
@@ -1294,6 +1321,10 @@ restart:
 
 		policy = flow_cache_lookup(fl, dst_orig->ops->family,
 					   dir, xfrm_policy_lookup);
+		if (IS_ERR(policy)) {
+			err = PTR_ERR(policy);
+			goto error;
+		}
 	}
 
 	if (!policy)
@@ -1340,6 +1371,10 @@ restart:
 							    fl, family,
 							    XFRM_POLICY_OUT);
 			if (pols[1]) {
+				if (IS_ERR(pols[1])) {
+					err = PTR_ERR(pols[1]);
+					goto error;
+				}
 				if (pols[1]->action == XFRM_POLICY_BLOCK) {
 					err = -EPERM;
 					goto error;
@@ -1578,6 +1613,9 @@ int __xfrm_policy_check(struct sock *sk,
 		pol = flow_cache_lookup(&fl, family, fl_dir,
 					xfrm_policy_lookup);
 
+	if (IS_ERR(pol))
+		return 0;
+
 	if (!pol) {
 		if (skb->sp && secpath_has_nontransport(skb->sp, 0, &xerr_idx)) {
 			xfrm_secpath_reject(xerr_idx, skb, &fl);
@@ -1596,6 +1634,8 @@ int __xfrm_policy_check(struct sock *sk,
 						    &fl, family,
 						    XFRM_POLICY_IN);
 		if (pols[1]) {
+			if (IS_ERR(pols[1]))
+				return 0;
 			pols[1]->curlft.use_time = (unsigned long)xtime.tv_sec;
 			npols ++;
 		}

Re: [PATCH] Fix for IPsec leakage with SELinux enabled

[Evgeniy Polyakov]

On Sun, Oct 01, 2006 at 02:27:13AM -0400, James Morris (jmorris@namei.org) wrote:
> Please review this patch carefully.  It addresses a couple of issues.
> 
> When a security module is loaded (in this case, SELinux), the 
> security_xfrm_policy_lookup() hook can return an access denied permission 
> (or other error).  We were not handling that correctly, and in fact 
> inverting the return logic and propagating a false "ok" back up to 
> xfrm_lookup(), which then allowed packets to pass as if they were not 
> associated with an xfrm policy.
> 
> The way I was seeing the problem was when connecting via IPsec to a 
> confined service on an SELinux box (vsftpd), which did not have the 
> appropriate SELinux policy permissions to send packets via IPsec.
> 
> The first SYNACK would be blocked, because of an uncached lookup via 
> flow_cache_lookup(), which would fail to resolve an xfrm policy because 
> the SELinux policy is checked at that point via the resolver.
> 
> However, retransmitted SYNACKs would then find a cached flow entry when 
> calling into flow_cache_lookup() with a null xfrm policy, which is 
> interpreted by xfrm_lookup() as the packet not having any associated 
> policy and similarly to the first case, allowing it to pass without 
> transformation.
> 
> The solution presented here is to first ensure that errno values are 
> correctly propagated all the way back up through the various call chains 
> from security_xfrm_policy_lookup(), and handled correctly.
> 
> Then, flow_cache_lookup() is modified, so that if the policy resolver 
> fails (typically a permission denied via the security module), the flow 
> cache entry is killed rather than having a null policy assigned (which 
> indicates that the packet can pass freely).  This also forces any future 
> lookups for the same flow to consult the security module (e.g. SELinux) 
> for current security policy (rather than, say, caching the error on the 
> flow cache entry).
> 
> I've done quite a bit of testing and not seen any problems, although the 
> patch could certainly do with further review.
> 
> Evgeniy, please let me know if this fixes your problem.

With that patch applied I got kernel panic after some time.
Unfortunately I have not installed serial console, so the most
interesting bits of the stack dump are not visible.
Here is the last ones which are on the screen:
ip_rcv
ip_rcv_finish
packet_rcv_spkt
ip_rcv
netif_receive_skb
sys_accept
skge_poll

and some other uninteresting stuff like hrtimer, softirq and the like...

EIP is at xfrm_lookup+0x43d/0x470

Notice packet socket handler in the trace, may be it can help - I ran
system with tcpdump started.

-- 
	Evgeniy Polyakov

Re: [PATCH] Fix for IPsec leakage with SELinux enabled

[James Morris]

On Mon, 2 Oct 2006, Evgeniy Polyakov wrote:

> > Evgeniy, please let me know if this fixes your problem.
> 
> With that patch applied I got kernel panic after some time.
> Unfortunately I have not installed serial console, so the most
> interesting bits of the stack dump are not visible.
> Here is the last ones which are on the screen:
> ip_rcv
> ip_rcv_finish
> packet_rcv_spkt
> ip_rcv
> netif_receive_skb
> sys_accept
> skge_poll
> 
> and some other uninteresting stuff like hrtimer, softirq and the like...
> 
> EIP is at xfrm_lookup+0x43d/0x470
> 
> Notice packet socket handler in the trace, may be it can help - I ran
> system with tcpdump started.

What kind of traffic was running over the system?  What is the IPsec and 
SELinux configuration?

Can you run gdb on vmlinux, find the start of xfrm_lookup then list what's 
at the EIP offset?

(gdb) p xfrm_lookup
$1 = {int (struct dst_entry **, struct flowi *, struct sock *, int)} 
0xc02cc7e2 <xfrm_lookup>
(gdb) l *(0xc02cc7e2 + 0x043d)


-- 
James Morris
<jmorris@namei.org>

Re: [PATCH] Fix for IPsec leakage with SELinux enabled

[Evgeniy Polyakov]

On Mon, Oct 02, 2006 at 09:31:36AM -0400, James Morris (jmorris@namei.org) wrote:
> What kind of traffic was running over the system?  What is the IPsec and 
> SELinux configuration?

I had login through ssh, started racoon and setup keys.
SElinu is configured by default in FC5 system with enforcing mode.

I switched off to different window and after some time, not immediately,
remote system stopped to answer.

There were no heavy traffic definitely.
It looks like some timeout expired and someone tried to do xfrm_lookup.

> Can you run gdb on vmlinux, find the start of xfrm_lookup then list what's 
> at the EIP offset?
> 
> (gdb) p xfrm_lookup
> $1 = {int (struct dst_entry **, struct flowi *, struct sock *, int)} 
> 0xc02cc7e2 <xfrm_lookup>
> (gdb) l *(0xc02cc7e2 + 0x043d)

(gdb) p xfrm_lookup
$1 = {int (struct dst_entry **, struct flowi *, struct sock *, int)} 0xc0301326 <xfrm_lookup>
(gdb) l *(0xc0301326+0x043d)
0xc0301763 is in xfrm_lookup (include/asm/atomic.h:126).
121      */ 
122     static __inline__ int atomic_dec_and_test(atomic_t *v)
123     {
124             unsigned char c;
125
126             __asm__ __volatile__(
127                     LOCK_PREFIX "decl %0; sete %1"
128                     :"+m" (v->counter), "=qm" (c)
129                     : : "memory");
130             return c != 0;

Probably reference counter is inside freed object...

> -- 
> James Morris
> <jmorris@namei.org>

-- 
	Evgeniy Polyakov

Re: [PATCH] Fix for IPsec leakage with SELinux enabled

[James Morris]

On Mon, 2 Oct 2006, Evgeniy Polyakov wrote:

> Probably reference counter is inside freed object...

I think I know what it is (xfrm_lookup needs to just return on flow cache 
lookup failure, not goto error and free the dst).  Patch forthcoming.




- James
-- 
James Morris
<jmorris@namei.org>