How can modular policy ever have worked? [patch]

How can modular policy ever have worked? [patch]

[Erich Schubert]

[-- Attachment #1: Type: text/plain, Size: 3215 bytes --]

Hi,
After investigating the M4 macros a bit closer (and after being pointed
in the right direction by Method), I wonder how modular policy ever can
have worked.

<Method> right now while building base the optional_policy is turned
into an m4 ifdef, if the module that activates the optional_policy isn't
also in base it essentially disappears

Fact (look at the source):
optional(`whatever',`...')
gets reduced to nothing when building base.pp and whatever as module.
That way base.pp doesn't contain any optional statements, because the
compiler used to not support that yet. (Method says he added this
support)
When building a module, it is expanded to a proper optional {}
statement.

I discovered this after investigating why building mta into base and
policy as module didn't work right - the optional(`postfix',`...') part
gets lost, so /usr/lib/sendmail can't transition into postdrop_t, so it
can't write mails to the postfix queues.
When I build mta as module this worked just fine.

However, this is a chain, and the only "stable" solution is to build a
monolithic policy. Some examples:

policy/modules/kernel/kernel.te:
optional_policy(`init',`

So if you have kernel in base (which is required) you should also put
init, logging, selinuxutil, rpc, portmap, nis, library, hotplug
into base, in order not to lose the optional statements in kernel.
Other modules in base have similar "optional dependencies", of course.

Now let's look at init... you'll find just about any service package
having an optional statement in init.te (which I consider ugly,
actually, having to have
something in the init.te module for a service!)

So basically, unless you build a monolithic policy, certain rules will
be missing from your linked policy that were "optional" in the policy
source. Ouch.

Attached patch will write appropriate optional {} statements to the base
policy.
In my module configuration (I've lots of stuff in base), diffstat for
the base.conf file gives: 36558 insertions(+), no deletions.
(I have filtered any "#line 123" and whitespace-only lines from this
stat)

So (as the attached diff should suggest anyway), this will only generate
additional statement for the policy.

The resulting base.pp file is almost twice as big, however the
"expanded" policy is only half the size and missing like 19k lines...
most of them pretty essential :-( So linking is not yet working
correctly; it doesn't throw any errors but the resulting files are badly
shreddered.

This also seems to turn up a problem in the linux kernel (2.6.15, not
the latest patches yet, sorry):
Even the _working_ (well, except for the missing rules) policy generated
without this patch, will link, but not load any more:
[...]
Committing changes:
/usr/sbin/load_policy:  Can't load policy:  Invalid argument

So it seems the kernel is now rejecting any policy loads, albeit it
still in permissive mode.

best regards,
Erich Schubert
-- 
   erich@(vitavonni.de|debian.org)    --    GPG Key ID: 4B3A135C    (o_
       The best things in life are free: Friendship and Love.       //\
     Glück ist, wenn man dafür geliebt wird, wie man eben ist.      V_/_

[-- Attachment #2: refpolicy-module-optionals.diff --]
[-- Type: text/x-patch, Size: 1797 bytes --]

Index: policy/support/loadable_module.spt
===================================================================
--- policy/support/loadable_module.spt	(Revision 63)
+++ policy/support/loadable_module.spt	(Revision 64)
@@ -24,7 +24,14 @@
 # For use in interfaces, to optionally insert a require block
 #
 define(`gen_require',`
-	ifdef(`self_contained_policy',`',`
+	ifdef(`self_contained_policy',`
+		ifdef(`modular_base_policy',`
+			ifdef(`in_optional_statement',`
+				require {
+					$1
+				}
+			')
+		')',`
 		require {
 			$1
 		}
@@ -87,10 +94,26 @@
 #
 define(`optional_policy',`
 	ifdef(`self_contained_policy',`
-		ifdef(`$1.te',`$2',`$3')
+		ifdef(`$1.te',`$2',`
+			ifdef(`modular_base_policy',`
+				optional {
+					pushdef(`in_optional_statement') dnl
+					$2
+					popdef(`in_optional_statement') dnl
+				ifelse(`$3',`',`',`
+				} else {
+					$3
+				')
+				}
+			',`
+				$3
+			')
+		')
 	',`
 		optional {
+			pushdef(`in_optional_statement') dnl
 			$2
+			popdef(`in_optional_statement') dnl
 		ifelse(`$3',`',`',`
 		} else {
 			$3
Index: Rules.modular
===================================================================
--- Rules.modular	(Revision 63)
+++ Rules.modular	(Revision 64)
@@ -111,7 +111,7 @@
 	@test -d $(TMPDIR) || mkdir -p $(TMPDIR)
 	@test -d $(dir $(BASE_CONF)) || mkdir -p $(dir $(BASE_CONF))
 # checkpolicy can use the #line directives provided by -s for error reporting:
-	$(verbose) m4 -D self_contained_policy $(M4PARAM) -s $^ > $(TMPDIR)/$(@F).tmp
+	$(verbose) m4 -D self_contained_policy -D modular_base_policy $(M4PARAM) -s $^ > $(TMPDIR)/$(@F).tmp
 	$(verbose) sed -e /^portcon/d -e /^nodecon/d -e /^netifcon/d < $(TMPDIR)/$(@F).tmp > $@
 # the ordering of these ocontexts matters:
 	$(verbose) grep ^portcon $(TMPDIR)/$(@F).tmp >> $@ || true

Re: How can modular policy ever have worked? [patch]

[Stephen Smalley]

On Sat, 2006-03-18 at 15:20 +0100, Erich Schubert wrote:
> This also seems to turn up a problem in the linux kernel (2.6.15, not
> the latest patches yet, sorry):
> Even the _working_ (well, except for the missing rules) policy generated
> without this patch, will link, but not load any more:
> [...]
> Committing changes:
> /usr/sbin/load_policy:  Can't load policy:  Invalid argument
> 
> So it seems the kernel is now rejecting any policy loads, albeit it
> still in permissive mode.

Any "security:" messages from the kernel in /var/log/messages upon the
attempted policy load?

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Erich Schubert]

Hello Stephen,
> > Committing changes:
> > /usr/sbin/load_policy:  Can't load policy:  Invalid argument
> Any "security:" messages from the kernel in /var/log/messages upon the
> attempted policy load?

No. A strace shows that load_policy is writing data to the load
interface,
which results in this invalid argument error:
open("/selinux/load", O_RDWR|O_LARGEFILE) = 4
write(4, "\214\377|\371\10\0\0\0SE Linux\24\0\0\0\1\0\0\0\10\0\0"...,
592395) = -1 EINVAL (Invalid argument)

Any progress on the optional{} in base.pp issues?
I have the impression that type attributes are broken; for example
almost all (I can't say for sure, apol doesn't work for me; is there any
other "policy decompiler"?) the restorecon_t relabelto rules are
missing, which are defined for file_type.
So I have the impression that when linking the base policy with
optionals, these attributes are messed up.

best regards,
Erich Schubert
-- 
     erich@(vitavonni.de|debian.org)    --    GPG Key ID: 4B3A135C    (o_
  There is no branch of mathematics, however abstract, which may not  //\
 some day be applied to phenomena of the real world. --- Lobatchevsky V_/_
       Nichts läßt die Erde so geräumig erscheinen, als wenn man
           Freunde in der Ferne hat. --- Henry David Thoreau



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Stephen Smalley]

On Mon, 2006-03-20 at 16:16 +0100, Erich Schubert wrote:
> No. A strace shows that load_policy is writing data to the load
> interface,
> which results in this invalid argument error:
> open("/selinux/load", O_RDWR|O_LARGEFILE) = 4
> write(4, "\214\377|\371\10\0\0\0SE Linux\24\0\0\0\1\0\0\0\10\0\0"...,
> 592395) = -1 EINVAL (Invalid argument)

Odd.  Send me a copy of the generated policy.20 file privately or put it
up on a public site where it can be downloaded, please.

> Any progress on the optional{} in base.pp issues?
> I have the impression that type attributes are broken; for example
> almost all (I can't say for sure, apol doesn't work for me; is there any
> other "policy decompiler"?)

Hmmm...I thought optionals in base was fixed in the current version.
checkpolicy/test has a crude dispol program for dumping a binary policy.
Reported your problem with apol yet?  sediff is useful for comparing two
policies, e.g. a monolithic build against a linked one from a modular
build.

>  the restorecon_t relabelto rules are
> missing, which are defined for file_type.
> So I have the impression that when linking the base policy with
> optionals, these attributes are messed up.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Erich Schubert]

[-- Attachment #1: Type: text/plain, Size: 2757 bytes --]

Hello Stephen,
> > Any progress on the optional{} in base.pp issues?
> Hmmm...I thought optionals in base was fixed in the current version.

No, they are not. Since I added the patch I posted earlier - which is
now in the refpolicy cvs for some time - I only obtain corrupt policies.

Today a user on IRC running Fedora had the very same problem:
<rsc> Since upgrading to latest SELinux stuff from Fedora Core Rawhide,
I'm getting tons of avc denied messages in syslog (serefpolicy targeted)
<rsc> unfortunately, I can't downgrade selinux-policy for testing
purposes
<rsc> nearly every action is commented with avc denied
<rsc> btw. /usr/sbin/load_policy:  Can't load policy:  Invalid argument
<rsc> libsemanage.semanage_reload_policy: load_policy returned error
code 2.
<rsc> while trying to switch one selinux-policy back

Well, this sounds _exactly_ like what I'm seeing. Tons of missing
statements (because type attributes are not working anymore) and no way
to load a different policy despite rebooting.

Anyone running the _current_ CVS refpolicy _successfully_?

> checkpolicy/test has a crude dispol program for dumping a binary policy.

With the dispol tool (menu choice 1) I get the following result:
# grep "restorecon_t .* : file .* relabelto" dispol-result-1
allow restorecon_t policy_config_t : file { relabelto read getattr lock
ioctl };
allow restorecon_t shadow_t : file { relabelto };

Just these two lines. When I do the same on my last working refpolicy
build (without the optionals in base patch), I get 448 rules.

That's why I think the following line is not working properly anymore:
allow restorecon_t { file_type  }:file { getattr relabelfrom
relabelto };

> Reported your problem with apol yet?  sediff is useful for comparing two
> policies, e.g. a monolithic build against a linked one from a modular
> build.

Monolithic builds are broken AFAICT now, by the removal of the module
name from the optional_policy statement... that way, the only optional
policy you could maybe still build is the one containing all modules.
When I downgrade policy/support/loadable_module.spt to the version with
my original patch only, and downgrade to my latest version prior to the
optional_policy $1 removal, I can build a monolithic policy.
The dump 1 then contains 1090 "restorecon_t .* : file .* relabelto"
lines.

So the type attributes are definitely broken somehow.

best regards,
Erich Schubert
-- 
   erich@(vitavonni.de|debian.org)    --    GPG Key ID: 4B3A135C    (o_
    Go away or i'll replace you with a very small shell script.     //\
     Nichts läßt die Erde so geräumig erscheinen, als wenn man      V_/_
         Freunde in der Ferne hat. --- Henry David Thoreau

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: How can modular policy ever have worked? [patch]

[Joshua Brindle]

Erich Schubert wrote:
> Hello Stephen,
>>> Any progress on the optional{} in base.pp issues?
>> Hmmm...I thought optionals in base was fixed in the current version.
> 
> No, they are not. Since I added the patch I posted earlier - which is
> now in the refpolicy cvs for some time - I only obtain corrupt policies.
> 
> Today a user on IRC running Fedora had the very same problem:
> <rsc> Since upgrading to latest SELinux stuff from Fedora Core Rawhide,
> I'm getting tons of avc denied messages in syslog (serefpolicy targeted)
> <rsc> unfortunately, I can't downgrade selinux-policy for testing
> purposes
> <rsc> nearly every action is commented with avc denied
> <rsc> btw. /usr/sbin/load_policy:  Can't load policy:  Invalid argument
> <rsc> libsemanage.semanage_reload_policy: load_policy returned error
> code 2.
> <rsc> while trying to switch one selinux-policy back
> 
> Well, this sounds _exactly_ like what I'm seeing. Tons of missing
> statements (because type attributes are not working anymore) and no way
> to load a different policy despite rebooting.
> 
> Anyone running the _current_ CVS refpolicy _successfully_?
> 
>> checkpolicy/test has a crude dispol program for dumping a binary policy.
> 
> With the dispol tool (menu choice 1) I get the following result:
> # grep "restorecon_t .* : file .* relabelto" dispol-result-1
> allow restorecon_t policy_config_t : file { relabelto read getattr lock
> ioctl };
> allow restorecon_t shadow_t : file { relabelto };
> 
> Just these two lines. When I do the same on my last working refpolicy
> build (without the optionals in base patch), I get 448 rules.
> 
> That's why I think the following line is not working properly anymore:
> allow restorecon_t { file_type  }:file { getattr relabelfrom
> relabelto };
> 
>> Reported your problem with apol yet?  sediff is useful for comparing two
>> policies, e.g. a monolithic build against a linked one from a modular
>> build.
> 
> Monolithic builds are broken AFAICT now, by the removal of the module
> name from the optional_policy statement... that way, the only optional
> policy you could maybe still build is the one containing all modules.
> When I downgrade policy/support/loadable_module.spt to the version with
> my original patch only, and downgrade to my latest version prior to the
> optional_policy $1 removal, I can build a monolithic policy.
> The dump 1 then contains 1090 "restorecon_t .* : file .* relabelto"
> lines.
> 
> So the type attributes are definitely broken somehow.

I can't reproduce this problem using the latest cvs refpolicy and latest 
cvs toolchain

refpolicy]# grep "restorecon_t .* : file .* relabelto" dump  | wc -l
1047

This is a fresh refpolicy checkout:
# cat build.conf | grep ^[A-Z]
TYPE = strict-mcs
NAME = refpolicy
DIRECT_INITRC=n
MONOLITHIC=y
POLY=n
QUIET=n

and I am able to load the resultant binary policy. Is there anything in 
your audit log or dmesg about the policy load failing? is there an MLS 
mismatch between the in kernel policy and the new one you just built?

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Christopher J. PeBenito]

On Sun, 2006-03-26 at 00:57 +0100, Erich Schubert wrote:
> Monolithic builds are broken AFAICT now, by the removal of the module
> name from the optional_policy statement... that way, the only optional
> policy you could maybe still build is the one containing all modules.
> When I downgrade policy/support/loadable_module.spt to the version with
> my original patch only, and downgrade to my latest version prior to the
> optional_policy $1 removal, I can build a monolithic policy.

I committed a fix for Rules.monolithic to fix this; disabled modules
interfaces were not being expanded.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Christopher J. PeBenito]

On Mon, 2006-03-20 at 16:16 +0100, Erich Schubert wrote:
> Any progress on the optional{} in base.pp issues?
> I have the impression that type attributes are broken; for example
> almost all (I can't say for sure, apol doesn't work for me; is there any
> other "policy decompiler"?) the restorecon_t relabelto rules are
> missing, which are defined for file_type.
> So I have the impression that when linking the base policy with
> optionals, these attributes are messed up.

I'm working on it right now.  It requires other changes, as you get
situations like this:

optional {
	require {
		type foo_t;
	}
	...
}

	type foo_t;

which results in an duplicate declaration (if this should be considered
a duplicate declaration is a different discussion).  There is some
sed'ing to move type declarations above, but it doesn't catch
declarations that don't start at the beginning of the line (like types
declared by templates), but if we allow whitespace before the type
declaration, we need a smarter sed script to skip over the ones in
require blocks (already written).  But the gen_require() isn't expanded
until m4 is run (current sed'ing happens before m4), so we have to push
the m4 to before the sed.  So the build has to be changed to run m4 on
the individual modules, which should have the added effect of making
compiler errors useful again.

Sorry for the stream of consciousness :)  Thats where it stands now;
right now I'm working on moving the m4 invocation.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Christopher J. PeBenito]

[-- Attachment #1: Type: text/plain, Size: 1768 bytes --]

On Mon, 2006-03-20 at 11:57 -0500, Christopher J. PeBenito wrote:
> On Mon, 2006-03-20 at 16:16 +0100, Erich Schubert wrote:
> > Any progress on the optional{} in base.pp issues?

> I'm working on it right now.  It requires other changes, as you get
> situations like this:
> 
> optional {
> 	require {
> 		type foo_t;
> 	}
> 	...
> }
> 
> 	type foo_t;
> 
> which results in an duplicate declaration (if this should be considered
> a duplicate declaration is a different discussion).  There is some
> sed'ing to move type declarations above, but it doesn't catch
> declarations that don't start at the beginning of the line (like types
> declared by templates), but if we allow whitespace before the type
> declaration, we need a smarter sed script to skip over the ones in
> require blocks (already written).  But the gen_require() isn't expanded
> until m4 is run (current sed'ing happens before m4), so we have to push
> the m4 to before the sed.  So the build has to be changed to run m4 on
> the individual modules, which should have the added effect of making
> compiler errors useful again.

I've committed all of the prerequisite changes.  The patch to enable
optionals in base is attached; when I apply it, I get:

Compiling refpolicy policy.20
/usr/bin/checkpolicy policy.conf -o policy.20
/usr/bin/checkpolicy:  loading policy configuration from policy.conf
libsepol.expand_module: Error while indexing out symbols
Error while expanding policy
make: *** [policy.20] Error 1

when compiling monolithic.  This is on last week's release versions of
the toolchain.  It doesn't happen with the default base module from make
conf, but if you add enough modules to base, it will also hit this
error.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150


[-- Attachment #2: opt_in_base.diff --]
[-- Type: text/x-patch, Size: 995 bytes --]

Index: policy/support/loadable_module.spt
===================================================================
--- policy/support/loadable_module.spt	(revision 1628)
+++ policy/support/loadable_module.spt	(working copy)
@@ -24,10 +24,16 @@
 # For use in interfaces, to optionally insert a require block
 #
 define(`gen_require',`
-	ifdef(`self_contained_policy',`',`
+	ifdef(`self_contained_policy',`
+		ifdef(`__in_optional_policy',`
+			require {
+				$1
+			} # end require
+		')
+	',`
 		require {
 			$1
-		}
+		} # end require
 	')
 ')
 
@@ -86,17 +92,15 @@
 # Optional policy handling
 #
 define(`optional_policy',`
-	ifdef(`self_contained_policy',`
-		ifdef(`$1.te',`$2',`$3')
-	',`
-		optional {
-			$2
-		ifelse(`$3',`',`',`
-		} else {
-			$3
-		')
-		}
+	optional {
+		pushdef(`__in_optional_policy') dnl
+		$2
+		popdef(`__in_optional_policy') dnl
+	ifelse(`$3',`',`',`
+	} else {
+		$3
 	')
+	}
 ')
 
 ##############################

Re: How can modular policy ever have worked? [patch]

[Stephen Smalley]

On Tue, 2006-03-21 at 11:21 -0500, Christopher J. PeBenito wrote:
> I've committed all of the prerequisite changes.  The patch to enable
> optionals in base is attached; when I apply it, I get:
> 
> Compiling refpolicy policy.20
> /usr/bin/checkpolicy policy.conf -o policy.20
> /usr/bin/checkpolicy:  loading policy configuration from policy.conf
> libsepol.expand_module: Error while indexing out symbols
> Error while expanding policy
> make: *** [policy.20] Error 1
> 
> when compiling monolithic.  This is on last week's release versions of
> the toolchain.  It doesn't happen with the default base module from make
> conf, but if you add enough modules to base, it will also hit this
> error.

FYI, this was fixed in checkpolicy 1.30.1, available in sourceforge cvs.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: How can modular policy ever have worked? [patch]

[Christopher J. PeBenito]

On Sat, 2006-03-18 at 15:20 +0100, Erich Schubert wrote:
> Fact (look at the source):
> optional(`whatever',`...')
> gets reduced to nothing when building base.pp and whatever as module.
> That way base.pp doesn't contain any optional statements, because the
> compiler used to not support that yet. (Method says he added this
> support)
> When building a module, it is expanded to a proper optional {}
> statement.
> 
> I discovered this after investigating why building mta into base and
> policy as module didn't work right - the optional(`postfix',`...') part
> gets lost, so /usr/lib/sendmail can't transition into postdrop_t, so it
> can't write mails to the postfix queues.
> When I build mta as module this worked just fine.

Yes, this type of thing is what prompted the patch that allowed
optionals in base module which Joshua was referring to.

> Attached patch will write appropriate optional {} statements to the base
> policy.

The base module check is not needed.  Since optionals are now allowed in
the base module they also work in monolithic policies (a base module is
basically an unexpanded monolithic policy).

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.