regression test of security policy

regression test of security policy

[Kohei KaiGai]

Does anyone have a tool to run regression test when we construct a patch?
(Or, is it available to construct using existing tools?)

Right now, I have to replace a working policy by the modified one whenever
I prepare to submit a patch towards reference policy. However, the default
security policy of Fedora is optimized to Fedora environment, thus, it often
mismatch with the latest upstream policy.
For example, "allow_execmem" is not defined at Fedora, so, I could not
load the staff.pp being constructed based on the upstream policy

So, the solution I'm looking for is a tool that loads a monolithic policy and
checks its access control decision towards a certain pair of subject context
and target context according to catalog files, then it prints the result of
diff commands between the computed one and expected one.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: regression test of security policy

[Stephen Smalley]

On Fri, 2012-05-04 at 15:48 +0200, Kohei KaiGai wrote:
> Does anyone have a tool to run regression test when we construct a patch?
> (Or, is it available to construct using existing tools?)
> 
> Right now, I have to replace a working policy by the modified one whenever
> I prepare to submit a patch towards reference policy. However, the default
> security policy of Fedora is optimized to Fedora environment, thus, it often
> mismatch with the latest upstream policy.
> For example, "allow_execmem" is not defined at Fedora, so, I could not
> load the staff.pp being constructed based on the upstream policy
> 
> So, the solution I'm looking for is a tool that loads a monolithic policy and
> checks its access control decision towards a certain pair of subject context
> and target context according to catalog files, then it prints the result of
> diff commands between the computed one and expected one.

Possibly you could derive such a tool from checkpolicy -d, switching
from a menu-driven interface to a scriptable one.

checkpolicy -Mdb /etc/selinux/targeted/policy/policy.24

setools would be the other option, but sesearch only deals with TE
rules.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: regression test of security policy

[Kohei KaiGai]

2012/5/4 Stephen Smalley <sds@tycho.ncsc.mil>:
> On Fri, 2012-05-04 at 15:48 +0200, Kohei KaiGai wrote:
>> Does anyone have a tool to run regression test when we construct a patch?
>> (Or, is it available to construct using existing tools?)
>>
>> Right now, I have to replace a working policy by the modified one whenever
>> I prepare to submit a patch towards reference policy. However, the default
>> security policy of Fedora is optimized to Fedora environment, thus, it often
>> mismatch with the latest upstream policy.
>> For example, "allow_execmem" is not defined at Fedora, so, I could not
>> load the staff.pp being constructed based on the upstream policy
>>
>> So, the solution I'm looking for is a tool that loads a monolithic policy and
>> checks its access control decision towards a certain pair of subject context
>> and target context according to catalog files, then it prints the result of
>> diff commands between the computed one and expected one.
>
> Possibly you could derive such a tool from checkpolicy -d, switching
> from a menu-driven interface to a scriptable one.
>
> checkpolicy -Mdb /etc/selinux/targeted/policy/policy.24
>
> setools would be the other option, but sesearch only deals with TE
> rules.
>
Thanks for your suggestion. I totally agree with the idea to expand
checkpolicy to accept a script that describes a set of patterns to be
checked. I'll try to investigate it in this weekend.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: regression test of security policy

[Kohei KaiGai]

[-- Attachment #1: Type: text/plain, Size: 3895 bytes --]

2012/5/4 Kohei KaiGai <kaigai@kaigai.gr.jp>:
> 2012/5/4 Stephen Smalley <sds@tycho.ncsc.mil>:
>> On Fri, 2012-05-04 at 15:48 +0200, Kohei KaiGai wrote:
>>> Does anyone have a tool to run regression test when we construct a patch?
>>> (Or, is it available to construct using existing tools?)
>>>
>>> Right now, I have to replace a working policy by the modified one whenever
>>> I prepare to submit a patch towards reference policy. However, the default
>>> security policy of Fedora is optimized to Fedora environment, thus, it often
>>> mismatch with the latest upstream policy.
>>> For example, "allow_execmem" is not defined at Fedora, so, I could not
>>> load the staff.pp being constructed based on the upstream policy
>>>
>>> So, the solution I'm looking for is a tool that loads a monolithic policy and
>>> checks its access control decision towards a certain pair of subject context
>>> and target context according to catalog files, then it prints the result of
>>> diff commands between the computed one and expected one.
>>
>> Possibly you could derive such a tool from checkpolicy -d, switching
>> from a menu-driven interface to a scriptable one.
>>
>> checkpolicy -Mdb /etc/selinux/targeted/policy/policy.24
>>
>> setools would be the other option, but sesearch only deals with TE
>> rules.
>>
> Thanks for your suggestion. I totally agree with the idea to expand
> checkpolicy to accept a script that describes a set of patterns to be
> checked. I'll try to investigate it in this weekend.
>
I tried to implement an infrastructure of regression test to cover
wider test cases, without replacement of existing environment.

It consists of two portions. The first one is a new script support at
checkpolicy command. It allows to give a script to check access
control decision of the loaded policy, and print out its results
towards a pair of security contexts.
The other one is a test case in reference policy. It should contain
a set of test cases described as a script of checkpolicy and expected
result for each policy types (standard, mcs and mls). If and when
someone applied a patch to the reference policy, he can confirm
how it works or unexpected side-effect as differences from the
expected results.

The current revision towards checkpolicy supports the following
commands.

* access <scontext> <tcontext> <tclass>
It prints all the allowed permissions for a pair of contexts.

* create <scontext> <tcontext> <tclass> [<objname>]
It prints a security context to be assigned on a new object
if and when <scontext> creates a new one under the <tcontext>.
The <objname> is optional, but not supported yet, because
libsepol does not support to query named type transition.

* boolean <bool name> [<new value>]
It shows or turns on/off a particular boolean. It should be used
to confirm permissions being allowed conditionally.

At the reference policy side, I create "regtest" directory to store
test cases and expected results. I works towards only monolithic
policy, so you can try the regression test as follows.

  [kaigai@iwashi refpolicy]$ make MONOLITHIC=y TYPE=mcs regtest
      :
  /usr/bin/checkpolicy:  policy configuration loaded
  /usr/bin/checkpolicy:  writing binary representation (version 27) to policy.27
    Regression Test for postgresql ... ok
  Regression Tess Pass

This make target runs the test cases (*.test) stored in the "regress"
directory to the checkpolicy, then it compares its output with
expected results (*.expected).
If regtest-$(TYPE).diff is not empty, it means diff command generate
something different from expected results.

I'd like to have such kind of test in the reference policy, to cover
wider range test cases at security policy side.
It helps to improve the quality and to reduce the burden for testing.
(In fact, I found a few bugs in mcs/mls rules during this development...)

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

[-- Attachment #2: selinux-refpolicy-regtest.20120506.patch.gz --]
[-- Type: application/x-gzip, Size: 12122 bytes --]

[-- Attachment #3: selinux-checkpolicy-regtest.20120506.patch --]
[-- Type: application/octet-stream, Size: 9095 bytes --]

 checkpolicy/checkpolicy.8 |    5 +-
 checkpolicy/checkpolicy.c |  257 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 258 insertions(+), 4 deletions(-)

diff --git a/checkpolicy/checkpolicy.8 b/checkpolicy/checkpolicy.8
index 6826938..c7c0949 100644
--- a/checkpolicy/checkpolicy.8
+++ b/checkpolicy/checkpolicy.8
@@ -3,7 +3,7 @@
 checkpolicy \- SELinux policy compiler
 .SH SYNOPSIS
 .B checkpolicy
-.I "[-b] [-d] [-M] [-c policyvers] [-o output_file] [input_file]"
+.I "[-b] [-d] [-M] [-s <test_script>][-c policyvers] [-o output_file] [input_file]"
 .br
 .SH "DESCRIPTION"
 This manual page describes the
@@ -27,6 +27,9 @@ Enter debug mode after loading the policy.
 .B \-M,\-\-mls
 Enable the MLS policy when checking and compiling the policy.
 .TP
+.B \-s,\-\-script
+Run the given test script to print access control rules in the loaded policy.
+.TP
 .B \-o,\-\-output filename
 Write a binary policy file to the specified filename.
 .TP
diff --git a/checkpolicy/checkpolicy.c b/checkpolicy/checkpolicy.c
index a35a749..bd8393d 100644
--- a/checkpolicy/checkpolicy.c
+++ b/checkpolicy/checkpolicy.c
@@ -101,8 +101,8 @@ void usage(char *progname)
 {
 	printf
 	    ("usage:  %s [-b] [-d] [-U handle_unknown (allow,deny,reject)] [-M]"
-	     "[-c policyvers (%d-%d)] [-o output_file] [-t target_platform (selinux,xen)]"
-	     "[input_file]\n",
+	     " [-s <test script>] [-c policyvers (%d-%d)] [-o output_file]"
+	     " [-t target_platform (selinux,xen)] [input_file]\n",
 	     progname, POLICYDB_VERSION_MIN, POLICYDB_VERSION_MAX);
 	exit(1);
 }
@@ -370,6 +370,250 @@ static int check_level(hashtab_key_t key, hashtab_datum_t datum, void *arg)
 	return 0;
 }
 
+static sepol_security_id_t script_lookup_sid(char *context)
+{
+	sepol_security_id_t	ssid;
+	int	ret;
+
+	ret = sepol_context_to_sid(context, strlen(context), &ssid);
+	if (ret == 0)
+		return ssid;
+
+	if (ret == -EINVAL)
+		printf("Error: context \"%s\" is invalid\n", context);
+	else
+		printf("Error: could not get SID from context \"%s\": %s\n",
+		       context, strerror(-ret));
+
+	return SEPOL_SECSID_NULL;
+}
+
+static void script_cmd_access(int nargs,
+			      char *scontext,
+			      char *tcontext,
+			      char *tclass_name)
+{
+	sepol_security_id_t	ssid;
+	sepol_security_id_t	tsid;
+	class_datum_t	       *cdatum;
+	struct sepol_av_decision avd;
+	int	i, ret;
+
+	if (nargs != 4) {
+		printf("Error: access command should take 3 arguments.\n");
+		return;
+	}
+
+	if ((ssid = script_lookup_sid(scontext)) == SEPOL_SECSID_NULL)
+		return;
+	if ((tsid = script_lookup_sid(tcontext)) == SEPOL_SECSID_NULL)
+		return;
+	cdatum = (class_datum_t *) hashtab_search(policydb.p_classes.table,
+						  tclass_name);
+	if (!cdatum)
+	{
+		printf("Error: \"%s\" is not a valid object class\n",
+		       tclass_name);
+		return;
+	}
+
+	ret = sepol_compute_av(ssid, tsid, cdatum->s.value, 0, &avd);
+	if (ret < 0)
+	{
+		printf("Error: unexpected result at sepol_compute_av: %s\n",
+		       strerror(errno));
+		return;
+	}
+
+	printf("  scontext=%s\n  tcontext=%s\n  tclass=%s\n",
+	       scontext, tcontext, tclass_name);
+	printf("  allowed {");
+	for (i = 0; i < sizeof(avd.allowed) * 8; i++)
+	{
+		struct val_to_name v;
+
+		if ((avd.allowed & (1 << i)) == 0)
+			continue;
+
+		v.val = i+1;
+		ret =  hashtab_map(cdatum->permissions.table, find_perm, &v);
+		if (!ret && cdatum->comdatum) {
+			ret = hashtab_map(cdatum->comdatum->permissions.table,
+					  find_perm, &v);
+		}
+		if (ret)
+			printf(" %s", v.name);
+	}
+	printf(" }\n");
+}
+
+static void script_cmd_create(int nargs,
+			      char *scontext,
+			      char *tcontext,
+			      char *tclass_name,
+			      char *objname)
+{
+	sepol_security_id_t	ssid;
+	sepol_security_id_t	tsid;
+	sepol_security_id_t	nsid;
+	class_datum_t	       *cdatum;
+	sepol_security_context_t ncontext;
+	size_t	ncontext_len;
+	int	ret;
+
+	if (nargs != 4 && nargs != 5) {
+		printf("Error: create command should take 3 or 4 arguments.\n");
+		return;
+	}
+
+	if ((ssid = script_lookup_sid(scontext)) == SEPOL_SECSID_NULL)
+		return;
+	if ((tsid = script_lookup_sid(tcontext)) == SEPOL_SECSID_NULL)
+		return;
+	cdatum = (class_datum_t *) hashtab_search(policydb.p_classes.table,
+						  tclass_name);
+	if (!cdatum)
+	{
+		printf("Error: \"%s\" is not a valid object class\n",
+		       tclass_name);
+		return;
+	}
+
+	/*
+	 * XXX - libsepol should have interface to query type_transition
+	 * rule with object name. however, we have no way for this kind
+	 * of rules.
+	 */
+	if (objname)
+	{
+		printf("Error: named type transition is not supported yet\n");
+		return;
+	}
+
+	ret = sepol_transition_sid(ssid, tsid, cdatum->s.value, &nsid);
+	if (ret < 0)
+	{
+		printf("Error: unexpected result at sepol_transition_sid: %s\n",
+		       strerror(-ret));
+		return;
+	}
+
+	ret = sepol_sid_to_context(nsid, &ncontext, &ncontext_len);
+	if (ret < 0)
+	{
+		printf("Error: unexpected result at sepol_sid_to_context: %s\n",
+		       strerror(-ret));
+		return;
+	}
+
+	printf("  scontext=%s\n  tcontext=%s\n  tclass=%s\n  ncontext=%s\n",
+	       scontext, tcontext, tclass_name, ncontext);
+}
+
+static void script_cmd_boolean(int nargs, char *bool_name, char *value)
+{
+	cond_bool_datum_t      *bool;
+	int	old_state;
+	int	new_state;
+
+	if (nargs != 2 && nargs != 3) {
+		printf("Error: boolean command should take 1 or 2 arguments.\n");
+		return;
+	}
+
+	bool = hashtab_search(policydbp->p_bools.table, bool_name);
+	if (bool == NULL) {
+		printf("Error: could not find boolean \"%s\"\n", bool_name);
+		return;
+	}
+
+	if (value == NULL) {
+		printf("  boolean %s = %s\n", bool_name,
+		       bool->state ? "true" : "false");
+		return;
+	}
+
+	if (strcmp(value, "1") == 0 ||
+	    strcmp(value, "on") == 0 ||
+	    strcmp(value, "true") == 0)
+		new_state = 1;
+	else if (strcmp(value, "0") == 0 ||
+		 strcmp(value, "off") == 0 ||
+		 strcmp(value, "false") == 0)
+		new_state = 0;
+	else {
+		printf("Error: invalid boolean state \"%s\"\n", value);
+		return;
+	}
+	old_state = bool->state;
+	bool->state = new_state;
+	evaluate_conds(policydbp);
+
+	printf("  boolean %s = %s -> %s\n", bool_name,
+	       old_state ? "true" : "false",
+	       new_state ? "true" : "false");
+}
+
+static void exec_script(char *script_path)
+{
+	FILE   *filp;
+	int	lineno = 1;
+	char   *linebuf = NULL;
+	size_t	linelen;
+
+	filp = fopen(script_path, "rb");
+	if (!filp) {
+		fprintf(stderr, "Error: could not open script \"%s\": %m\n",
+			script_path);
+		return;
+	}
+
+	puts("@BEGIN_CHECKPOLICY_SCRIPT");
+
+	for (linebuf = NULL, lineno = 1;
+	     getline(&linebuf, &linelen, filp) > 0;
+	     free(linebuf), linebuf = NULL, lineno++)
+	{
+		char   *pos;
+		char   *argv[6];
+		int	nargs;
+
+		/* write out the line read as is */
+		fwrite(linebuf, strlen(linebuf), 1, stdout);
+
+		/* comment out after the '#' character */
+		pos = strchr(linebuf, '#');
+		if (pos)
+			*pos = '\0';
+
+		memset(argv, 0, sizeof(argv));
+		nargs = sscanf(linebuf, "%ms %ms %ms %ms %ms %ms",
+			       &argv[0], &argv[1], &argv[2],
+			       &argv[3], &argv[4], &argv[5]);
+		if (nargs < 1)
+		{
+			if (errno)
+				printf("Error: could not parse script: %m\n");
+			continue;
+		}
+		if (strcmp(argv[0], "access") == 0)
+			script_cmd_access(nargs, argv[1], argv[2], argv[3]);
+		else if (strcmp(argv[0], "create") == 0)
+			script_cmd_create(nargs, argv[1], argv[2], argv[3], argv[4]);
+		else if (strcmp(argv[0], "boolean") == 0)
+			script_cmd_boolean(nargs, argv[1], argv[2]);
+		else
+			printf("Error: \"%s\" is not a supported command\n",
+			       argv[0]);
+		/* release */
+		while (nargs > 0)
+			free(argv[--nargs]);
+	}
+	puts("@END_CHECKPOLICY_SCRIPT");
+
+	fclose(filp);
+}
+
 int main(int argc, char **argv)
 {
 	sepol_security_class_t tclass;
@@ -382,6 +626,7 @@ int main(int argc, char **argv)
 	unsigned int i;
 	unsigned int protocol, port;
 	unsigned int binary = 0, debug = 0;
+	char *script = NULL;
 	struct val_to_name v;
 	int ret, ch, fd, target = SEPOL_TARGET_SELINUX;
 	unsigned int nel, uret;
@@ -396,6 +641,7 @@ int main(int argc, char **argv)
 		{"output", required_argument, NULL, 'o'},
 		{"target", required_argument, NULL, 't'},
 		{"binary", no_argument, NULL, 'b'},
+		{"script", required_argument, NULL, 's'},
 		{"debug", no_argument, NULL, 'd'},
 		{"version", no_argument, NULL, 'V'},
 		{"handle-unknown", optional_argument, NULL, 'U'},
@@ -404,7 +650,7 @@ int main(int argc, char **argv)
 		{NULL, 0, NULL, 0}
 	};
 
-	while ((ch = getopt_long(argc, argv, "o:t:dbU:MVc:h", long_options, NULL)) != -1) {
+	while ((ch = getopt_long(argc, argv, "o:t:ds:bU:MVc:h", long_options, NULL)) != -1) {
 		switch (ch) {
 		case 'o':
 			outfile = optarg;
@@ -427,6 +673,9 @@ int main(int argc, char **argv)
 		case 'd':
 			debug = 1;
 			break;
+		case 's':
+			script = optarg;
+			break;
 		case 'V':
 			show_version = 1;
 			break;
@@ -615,6 +864,8 @@ int main(int argc, char **argv)
 		}
 		fclose(outfp);
 	}
+	if (script)
+		exec_script(script);
 	if (!debug) {
 		policydb_destroy(&policydb);
 		exit(0);

Re: regression test of security policy

[Christopher J. PeBenito]

On 05/06/12 14:51, Kohei KaiGai wrote:
> I tried to implement an infrastructure of regression test to cover
> wider test cases, without replacement of existing environment.
> 
> It consists of two portions. The first one is a new script support at
> checkpolicy command. It allows to give a script to check access
> control decision of the loaded policy, and print out its results
> towards a pair of security contexts.
> The other one is a test case in reference policy. It should contain
> a set of test cases described as a script of checkpolicy and expected
> result for each policy types (standard, mcs and mls). If and when
> someone applied a patch to the reference policy, he can confirm
> how it works or unexpected side-effect as differences from the
> expected results.
> 
> The current revision towards checkpolicy supports the following
> commands.
> 
> * access <scontext> <tcontext> <tclass>
> It prints all the allowed permissions for a pair of contexts.
> 
> * create <scontext> <tcontext> <tclass> [<objname>]
> It prints a security context to be assigned on a new object
> if and when <scontext> creates a new one under the <tcontext>.
> The <objname> is optional, but not supported yet, because
> libsepol does not support to query named type transition.
> 
> * boolean <bool name> [<new value>]
> It shows or turns on/off a particular boolean. It should be used
> to confirm permissions being allowed conditionally.
> 
> At the reference policy side, I create "regtest" directory to store
> test cases and expected results. I works towards only monolithic
> policy, so you can try the regression test as follows.
> 
>   [kaigai@iwashi refpolicy]$ make MONOLITHIC=y TYPE=mcs regtest
>       :
>   /usr/bin/checkpolicy:  policy configuration loaded
>   /usr/bin/checkpolicy:  writing binary representation (version 27) to policy.27
>     Regression Test for postgresql ... ok
>   Regression Tess Pass
> 
> This make target runs the test cases (*.test) stored in the "regress"
> directory to the checkpolicy, then it compares its output with
> expected results (*.expected).
> If regtest-$(TYPE).diff is not empty, it means diff command generate
> something different from expected results.
> 
> I'd like to have such kind of test in the reference policy, to cover
> wider range test cases at security policy side.
> It helps to improve the quality and to reduce the burden for testing.
> (In fact, I found a few bugs in mcs/mls rules during this development...)

I'm not adverse to this for refpolicy, but what worries me is the size and maintainability of the tests.  What you have in your patch for testing sepostgresql looks several times bigger than the sepostgresql policy itself.  It seems that the tests would be larger than the policy itself so that the constraints can be checked.  Additionally, the community (I'm including myself) isn't exactly good about keeping tests up to date (see tests in the toolchain, for example).

-- 
Chris PeBenito
Tresys Technology, LLC
www.tresys.com | oss.tresys.com

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: regression test of security policy

[Kohei KaiGai]

2012/5/10 Christopher J. PeBenito <cpebenito@tresys.com>:
> On 05/06/12 14:51, Kohei KaiGai wrote:
>> I'd like to have such kind of test in the reference policy, to cover
>> wider range test cases at security policy side.
>> It helps to improve the quality and to reduce the burden for testing.
>> (In fact, I found a few bugs in mcs/mls rules during this development...)
>
> I'm not adverse to this for refpolicy, but what worries me is the size
> and maintainability of the tests.  What you have in your patch for
> testing sepostgresql looks several times bigger than the sepostgresql
> policy itself.  It seems that the tests would be larger than the policy
> itself so that the constraints can be checked.  Additionally, the
> community (I'm including myself) isn't exactly good about keeping
> tests up to date (see tests in the toolchain, for example).
>
I could understand the maintenance burden.

How about your opinion to add Makefile support to run external
test cases? It will help contributors test their own patches being
submitted.

In my idea, it adds a new make target "regtest" with TESTCASE
argument that points to the *.test file.

  $ make TYPE=xxx MONOLITHIC=y TESTCASE=/path/to/testcases regtest

Then, makefile generates a monolithic policy chunk and kicks
checkpolicy with the new -s option that takes processed testcase
by m4.

The reason why I want refpolicy to provide such kind of infrastructure
is utilization of existing macro definitions to generate multiple
testcases from a single source.
Do you think it is reasonable to improve the quality of policy?

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: regression test of security policy

[Christopher J. PeBenito]

On 05/11/12 08:59, Kohei KaiGai wrote:
> 2012/5/10 Christopher J. PeBenito <cpebenito@tresys.com>:
>> On 05/06/12 14:51, Kohei KaiGai wrote:
>>> I'd like to have such kind of test in the reference policy, to cover
>>> wider range test cases at security policy side.
>>> It helps to improve the quality and to reduce the burden for testing.
>>> (In fact, I found a few bugs in mcs/mls rules during this development...)
>>
>> I'm not adverse to this for refpolicy, but what worries me is the size
>> and maintainability of the tests.  What you have in your patch for
>> testing sepostgresql looks several times bigger than the sepostgresql
>> policy itself.  It seems that the tests would be larger than the policy
>> itself so that the constraints can be checked.  Additionally, the
>> community (I'm including myself) isn't exactly good about keeping
>> tests up to date (see tests in the toolchain, for example).
>>
> I could understand the maintenance burden.
> 
> How about your opinion to add Makefile support to run external
> test cases? It will help contributors test their own patches being
> submitted.
> 
> In my idea, it adds a new make target "regtest" with TESTCASE
> argument that points to the *.test file.
> 
>   $ make TYPE=xxx MONOLITHIC=y TESTCASE=/path/to/testcases regtest
> 
> Then, makefile generates a monolithic policy chunk and kicks
> checkpolicy with the new -s option that takes processed testcase
> by m4.
> 
> The reason why I want refpolicy to provide such kind of infrastructure
> is utilization of existing macro definitions to generate multiple
> testcases from a single source.
> Do you think it is reasonable to improve the quality of policy?

It sounds fine.  We can discuss the implementation on the refpolicy mail list.

-- 
Chris PeBenito
Tresys Technology, LLC
www.tresys.com | oss.tresys.com

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.