[meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}

All of lore.kernel.org
 help / color / mirror / Atom feed

* [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}
@ 2013-10-21 18:06 Philip Tricca
  2013-10-21 20:15 ` Joe MacDonald
  0 siblings, 1 reply; 6+ messages in thread
From: Philip Tricca @ 2013-10-21 18:06 UTC (permalink / raw)
  To: yocto

The 'semodule' utility can operate on compresed modules so the only
cost of this change is a slower module load time when invoking
'semodule -i' on a running system (increased CPU load due to bzip2).
That said my tests show more than 100M reduction in ext3 image size
of core-image-selinux. This last metric is a bit skewed as the image
includes two policies. Still, a reduction in the size of the refpolicy
package by 1/2 is significant.

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 recipes-security/refpolicy/refpolicy_common.inc |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/recipes-security/refpolicy/refpolicy_common.inc b/recipes-security/refpolicy/refpolicy_common.inc
index 873d887..0ca0b9d 100644
--- a/recipes-security/refpolicy/refpolicy_common.inc
+++ b/recipes-security/refpolicy/refpolicy_common.inc
@@ -12,7 +12,7 @@ SRC_URI += "file://customizable_types \
 S = "${WORKDIR}/refpolicy"
 
 FILES_${PN} = "${sysconfdir}/selinux/${POLICY_NAME}/ \
-	${datadir}/selinux/${POLICY_NAME}/*.pp"
+	${datadir}/selinux/${POLICY_NAME}/*.pp.bz2"
 FILES_${PN}-dev =+ "${datadir}/selinux/${POLICY_NAME}/include/"
 
 DEPENDS += "checkpolicy-native policycoreutils-native m4-native"
@@ -67,11 +67,12 @@ EOF
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files
 	touch ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files/file_contexts.local
-	bzip2 -c ${D}${datadir}/selinux/${POLICY_NAME}/base.pp  > \
-		${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/base.pp
 	for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
+		bzip2 $i
 		if [ "`basename $i`" != "base.pp" ]; then
-			bzip2 -c $i > ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`;
+			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`
+		else
+			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/`basename $i`
 		fi
 	done
 
-- 
1.7.10.4



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}
  2013-10-21 18:06 [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir} Philip Tricca
@ 2013-10-21 20:15 ` Joe MacDonald
  2013-12-03 21:35   ` Joe MacDonald
  0 siblings, 1 reply; 6+ messages in thread
From: Joe MacDonald @ 2013-10-21 20:15 UTC (permalink / raw)
  To: Philip Tricca; +Cc: yocto

[-- Attachment #1: Type: text/plain, Size: 2761 bytes --]

[[meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}] On 13.10.21 (Mon 18:06) Philip Tricca wrote:

> The 'semodule' utility can operate on compresed modules so the only
> cost of this change is a slower module load time when invoking
> 'semodule -i' on a running system (increased CPU load due to bzip2).
> That said my tests show more than 100M reduction in ext3 image size
> of core-image-selinux. This last metric is a bit skewed as the image
> includes two policies. Still, a reduction in the size of the refpolicy
> package by 1/2 is significant.

This is included in the batch of updates I've merged and are currently
staging in my tree.  FWIW, on my build I saw a similar reduction in size
to what you've reported, ~110MB, with a minor hit at load time.  As
expected there's also an increase in memory requirements at load time,
so I'm poking around a bit to see what this does to the lower-end
configurations I've got kicking around.  It'd be really nice if this was
an option rather than an on/off thing.

-J.

> 
> Signed-off-by: Philip Tricca <flihp@twobit.us>
> ---
>  recipes-security/refpolicy/refpolicy_common.inc |    9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/recipes-security/refpolicy/refpolicy_common.inc b/recipes-security/refpolicy/refpolicy_common.inc
> index 873d887..0ca0b9d 100644
> --- a/recipes-security/refpolicy/refpolicy_common.inc
> +++ b/recipes-security/refpolicy/refpolicy_common.inc
> @@ -12,7 +12,7 @@ SRC_URI += "file://customizable_types \
>  S = "${WORKDIR}/refpolicy"
>  
>  FILES_${PN} = "${sysconfdir}/selinux/${POLICY_NAME}/ \
> -	${datadir}/selinux/${POLICY_NAME}/*.pp"
> +	${datadir}/selinux/${POLICY_NAME}/*.pp.bz2"
>  FILES_${PN}-dev =+ "${datadir}/selinux/${POLICY_NAME}/include/"
>  
>  DEPENDS += "checkpolicy-native policycoreutils-native m4-native"
> @@ -67,11 +67,12 @@ EOF
>  	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules
>  	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files
>  	touch ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files/file_contexts.local
> -	bzip2 -c ${D}${datadir}/selinux/${POLICY_NAME}/base.pp  > \
> -		${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/base.pp
>  	for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
> +		bzip2 $i
>  		if [ "`basename $i`" != "base.pp" ]; then
> -			bzip2 -c $i > ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`;
> +			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`
> +		else
> +			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/`basename $i`
>  		fi
>  	done
>  
-- 
-Joe MacDonald.
:wq

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}
  2013-10-21 20:15 ` Joe MacDonald
@ 2013-12-03 21:35   ` Joe MacDonald
  2013-12-03 21:35     ` [meta-selinux][PATCH 1/2] " Joe MacDonald
                       ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Joe MacDonald @ 2013-12-03 21:35 UTC (permalink / raw)
  To: flihp; +Cc: yocto

(resending, this time including the list ...)

[Re: [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}] On
13.10.21 (Mon 16:15) Joe MacDonald wrote:

> [[meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}] On 13.10.21
> (Mon 18:06) Philip Tricca wrote:
>
> > The 'semodule' utility can operate on compresed modules so the only
> > cost of this change is a slower module load time when invoking
> > 'semodule -i' on a running system (increased CPU load due to bzip2).
> > That said my tests show more than 100M reduction in ext3 image size
> > of core-image-selinux. This last metric is a bit skewed as the image
> > includes two policies. Still, a reduction in the size of the refpolicy
> > package by 1/2 is significant.
>
> This is included in the batch of updates I've merged and are currently
> staging in my tree.  FWIW, on my build I saw a similar reduction in size
> to what you've reported, ~110MB, with a minor hit at load time.  As
> expected there's also an increase in memory requirements at load time,
> so I'm poking around a bit to see what this does to the lower-end
> configurations I've got kicking around.  It'd be really nice if this was
> an option rather than an on/off thing.

This took rather longer than I'd hoped.  :-/

Anyway, I tried a bunch of different configurations and didn't find a huge hit
on memory requirements by doing this, though I still think there's an advantage
to making this an option that can be turned off for folks where storage is cheap
and memory and processing power is at a premium.  That, and the discussion on
the SELinux mailing list along the same line where the general feeling was that
smaller policies are better achieved by actually having less policy rather than
compressing it, led me to this idea.

A DISTRO_FEATURE that is on by default and incorporates your patch.  What do you
think, Phil?

-- 
-Joe MacDonald.
:wq



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [meta-selinux][PATCH 1/2] bzip SELinux policy modules in ${datadir}
  2013-12-03 21:35   ` Joe MacDonald
@ 2013-12-03 21:35     ` Joe MacDonald
  2013-12-03 21:35     ` [meta-selinux][PATCH 2/2] policy: Create compressed_policy distro feature Joe MacDonald
  2013-12-05  1:34     ` [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir} Philip Tricca
  2 siblings, 0 replies; 6+ messages in thread
From: Joe MacDonald @ 2013-12-03 21:35 UTC (permalink / raw)
  To: flihp; +Cc: yocto

From: Philip Tricca <flihp@twobit.us>

The 'semodule' utility can operate on compresed modules so the only
cost of this change is a slower module load time when invoking
'semodule -i' on a running system (increased CPU load due to bzip2).
That said my tests show more than 100M reduction in ext3 image size
of core-image-selinux. This last metric is a bit skewed as the image
includes two policies. Still, a reduction in the size of the refpolicy
package by 1/2 is significant.

Signed-off-by: Philip Tricca <flihp@twobit.us>
Signed-off-by: Joe MacDonald <joe@deserted.net>
---
 recipes-security/refpolicy/refpolicy_common.inc |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/recipes-security/refpolicy/refpolicy_common.inc b/recipes-security/refpolicy/refpolicy_common.inc
index 2ee42e1..b46903f 100644
--- a/recipes-security/refpolicy/refpolicy_common.inc
+++ b/recipes-security/refpolicy/refpolicy_common.inc
@@ -12,7 +12,7 @@ SRC_URI += "file://customizable_types \
 S = "${WORKDIR}/refpolicy"
 
 FILES_${PN} = "${sysconfdir}/selinux/${POLICY_NAME}/ \
-	${datadir}/selinux/${POLICY_NAME}/*.pp"
+	${datadir}/selinux/${POLICY_NAME}/*.pp.bz2"
 FILES_${PN}-dev =+ "${datadir}/selinux/${POLICY_NAME}/include/"
 
 DEPENDS += "checkpolicy-native policycoreutils-native m4-native"
@@ -79,11 +79,12 @@ EOF
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files
 	touch ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files/file_contexts.local
-	bzip2 -c ${D}${datadir}/selinux/${POLICY_NAME}/base.pp  > \
-		${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/base.pp
 	for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
+		bzip2 $i
 		if [ "`basename $i`" != "base.pp" ]; then
-			bzip2 -c $i > ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`;
+			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`
+		else
+			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/`basename $i`
 		fi
 	done
 
-- 
1.7.10.4



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [meta-selinux][PATCH 2/2] policy: Create compressed_policy distro feature
  2013-12-03 21:35   ` Joe MacDonald
  2013-12-03 21:35     ` [meta-selinux][PATCH 1/2] " Joe MacDonald
@ 2013-12-03 21:35     ` Joe MacDonald
  2013-12-05  1:34     ` [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir} Philip Tricca
  2 siblings, 0 replies; 6+ messages in thread
From: Joe MacDonald @ 2013-12-03 21:35 UTC (permalink / raw)
  To: flihp; +Cc: yocto

There is a small cost to having compressed policy files on the final
image both in terms of memory requirements and load times.  In nearly all
circumstances this is negligible, but this adds a DISTRO_FEATURE that
can be used to enable it, if desired.

The default selinux distros will enable the feature by default.

Signed-off-by: Joe MacDonald <joe@deserted.net>
---
 conf/distro/oe-selinux.conf                     |    2 +-
 recipes-security/refpolicy/refpolicy_common.inc |   32 ++++++++++++++++-------
 2 files changed, 23 insertions(+), 11 deletions(-)

diff --git a/conf/distro/oe-selinux.conf b/conf/distro/oe-selinux.conf
index 6e55a32..5f4af87 100644
--- a/conf/distro/oe-selinux.conf
+++ b/conf/distro/oe-selinux.conf
@@ -1,4 +1,4 @@
 DISTRO = "oe-selinux"
 DISTROOVERRIDES .= ":selinux"
 
-DISTRO_FEATURES_append = " acl xattr pam selinux"
+DISTRO_FEATURES_append = " acl xattr pam selinux compressed_policy"
diff --git a/recipes-security/refpolicy/refpolicy_common.inc b/recipes-security/refpolicy/refpolicy_common.inc
index b46903f..a71c5dd 100644
--- a/recipes-security/refpolicy/refpolicy_common.inc
+++ b/recipes-security/refpolicy/refpolicy_common.inc
@@ -11,8 +11,10 @@ SRC_URI += "file://customizable_types \
 
 S = "${WORKDIR}/refpolicy"
 
-FILES_${PN} = "${sysconfdir}/selinux/${POLICY_NAME}/ \
-	${datadir}/selinux/${POLICY_NAME}/*.pp.bz2"
+FILES_${PN} = " \
+	${sysconfdir}/selinux/${POLICY_NAME}/ \
+	${@base_contains('DISTRO_FEATURES', 'compressed_policy', '${datadir}/selinux/${POLICY_NAME}/*.pp.bz2', '${datadir}/selinux/${POLICY_NAME}/*.pp', d)} \
+	"
 FILES_${PN}-dev =+ "${datadir}/selinux/${POLICY_NAME}/include/"
 
 DEPENDS += "checkpolicy-native policycoreutils-native m4-native"
@@ -79,14 +81,24 @@ EOF
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules
 	mkdir -p ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files
 	touch ${D}${sysconfdir}/selinux/${POLICY_NAME}/contexts/files/file_contexts.local
-	for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
-		bzip2 $i
-		if [ "`basename $i`" != "base.pp" ]; then
-			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`
-		else
-			cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/`basename $i`
-		fi
-	done
+	if  ${@base_contains('DISTRO_FEATURES','compressed_policy','true','false',d)}; then
+		for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
+			bzip2 $i
+			if [ "`basename $i`" != "base.pp" ]; then
+				cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`
+			else
+				cp ${i}.bz2 ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/`basename $i`
+			fi
+		done
+	else
+		bzip2 -c ${D}${datadir}/selinux/${POLICY_NAME}/base.pp  >\
+			${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/base.pp
+		for i in ${D}${datadir}/selinux/${POLICY_NAME}/*.pp; do
+			if [ "`basename $i`" != "base.pp" ]; then
+				bzip2 -c $i > ${D}${sysconfdir}/selinux/${POLICY_NAME}/modules/active/modules/`basename $i`;
+			fi
+		done
+	fi
 
 	# Create policy store and build the policy
 	semodule -p ${D} -s ${POLICY_NAME} -n -B
-- 
1.7.10.4



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}
  2013-12-03 21:35   ` Joe MacDonald
  2013-12-03 21:35     ` [meta-selinux][PATCH 1/2] " Joe MacDonald
  2013-12-03 21:35     ` [meta-selinux][PATCH 2/2] policy: Create compressed_policy distro feature Joe MacDonald
@ 2013-12-05  1:34     ` Philip Tricca
  2 siblings, 0 replies; 6+ messages in thread
From: Philip Tricca @ 2013-12-05  1:34 UTC (permalink / raw)
  To: Joe MacDonald; +Cc: yocto

On 12/03/2013 04:35 PM, Joe MacDonald wrote:
> (resending, this time including the list ...)
> 
> [Re: [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}] On
> 13.10.21 (Mon 16:15) Joe MacDonald wrote:
> 
>> [[meta-selinux][PATCH] bzip SELinux policy modules in ${datadir}] On 13.10.21
>> (Mon 18:06) Philip Tricca wrote:
>>
>>> The 'semodule' utility can operate on compresed modules so the only
>>> cost of this change is a slower module load time when invoking
>>> 'semodule -i' on a running system (increased CPU load due to bzip2).
>>> That said my tests show more than 100M reduction in ext3 image size
>>> of core-image-selinux. This last metric is a bit skewed as the image
>>> includes two policies. Still, a reduction in the size of the refpolicy
>>> package by 1/2 is significant.
>>
>> This is included in the batch of updates I've merged and are currently
>> staging in my tree.  FWIW, on my build I saw a similar reduction in size
>> to what you've reported, ~110MB, with a minor hit at load time.  As
>> expected there's also an increase in memory requirements at load time,
>> so I'm poking around a bit to see what this does to the lower-end
>> configurations I've got kicking around.  It'd be really nice if this was
>> an option rather than an on/off thing.
> 
> This took rather longer than I'd hoped.  :-/
> 
> Anyway, I tried a bunch of different configurations and didn't find a huge hit
> on memory requirements by doing this, though I still think there's an advantage
> to making this an option that can be turned off for folks where storage is cheap
> and memory and processing power is at a premium.  That, and the discussion on
> the SELinux mailing list along the same line where the general feeling was that
> smaller policies are better achieved by actually having less policy rather than
> compressing it, led me to this idea.
> 
> A DISTRO_FEATURE that is on by default and incorporates your patch.  What do you
> think, Phil?

Sounds good Joe. Thanks for getting this one in.

- Philip



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2013-12-05  1:34 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-10-21 18:06 [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir} Philip Tricca
2013-10-21 20:15 ` Joe MacDonald
2013-12-03 21:35   ` Joe MacDonald
2013-12-03 21:35     ` [meta-selinux][PATCH 1/2] " Joe MacDonald
2013-12-03 21:35     ` [meta-selinux][PATCH 2/2] policy: Create compressed_policy distro feature Joe MacDonald
2013-12-05  1:34     ` [meta-selinux][PATCH] bzip SELinux policy modules in ${datadir} Philip Tricca

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.