Fedora refpolicy patches

Fedora refpolicy patches

[David Härdeman]

While working on SELinux-enabling a Debian system, I often Google for 
avc messages that show up in dmesg and 90% of the time it seems that the 
problem has already been solved in Fedora's version of the refpolicy but 
not in the upstream version.

Googling a bit more lead me to these emails:
http://marc.info/?l=selinux&m=121155835630301&w=2
http://marc.info/?l=selinux&m=121622105928866&w=2

The latest Fedora patch:
http://cvs.fedoraproject.org/viewcvs/devel/selinux-policy/policy-20080710.patch?rev=1.2&view=auto
Is 36918 lines totalling over 1.1 Mb.

The latest Debian patch:
http://ftp.de.debian.org/debian/pool/main/r/refpolicy/refpolicy_0.0.20080702-1.diff.gz
Is 8759 lines totalling 258Kb (but that includes the build scripts).

I wrote a quick python script that splits the Fedora patch into 
per-module patches (much like the ones Daniel J Walsh posted, only that 
I get 214 patches) and I'm prepared to start going over these patches 
seeing which ones are relevant and which ones would need some changes to 
work in Debian as well (for instance, lots of *.fc files would need to 
have lines like /etc/rc.d/init.d/something changed to 
/etc/(rc.d/)?init.d/something to work in both RH and Debian).

The question is how to treat the patches after that? Should I post them 
as I go through them (a couple per day for a couple of weeks?) and hope 
that someone at Tresys will apply them?

Also, Daniel, do you think it would be possible to change the Redhat 
build scripts to take a directory of patches instead of the huge patch 
it uses right now? It would make it much much easier to track the 
differences if the changes to each module was tracked in one patch in 
the CVS repo. It would also make it clearer what each change does (not 
at all clear sometimes with the current huge patch...comments and/or 
links to bugzilla entries would have been great) as each change to each 
patch will at least have the commit message to go along with it.

And in the end...does it really help? Someone at Tresys will have to 
review every patch anyway...so should I start looking at patches?

-- 
David Härdeman

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Härdeman wrote:
> While working on SELinux-enabling a Debian system, I often Google for
> avc messages that show up in dmesg and 90% of the time it seems that the
> problem has already been solved in Fedora's version of the refpolicy but
> not in the upstream version.
> 
> Googling a bit more lead me to these emails:
> http://marc.info/?l=selinux&m=121155835630301&w=2
> http://marc.info/?l=selinux&m=121622105928866&w=2
> 
> The latest Fedora patch:
> http://cvs.fedoraproject.org/viewcvs/devel/selinux-policy/policy-20080710.patch?rev=1.2&view=auto
> 
> Is 36918 lines totalling over 1.1 Mb.
> 
> The latest Debian patch:
> http://ftp.de.debian.org/debian/pool/main/r/refpolicy/refpolicy_0.0.20080702-1.diff.gz
> 
> Is 8759 lines totalling 258Kb (but that includes the build scripts).
> 
> I wrote a quick python script that splits the Fedora patch into
> per-module patches (much like the ones Daniel J Walsh posted, only that
> I get 214 patches) and I'm prepared to start going over these patches
> seeing which ones are relevant and which ones would need some changes to
> work in Debian as well (for instance, lots of *.fc files would need to
> have lines like /etc/rc.d/init.d/something changed to
> /etc/(rc.d/)?init.d/something to work in both RH and Debian).
> 
> The question is how to treat the patches after that? Should I post them
> as I go through them (a couple per day for a couple of weeks?) and hope
> that someone at Tresys will apply them?
> 
> Also, Daniel, do you think it would be possible to change the Redhat
> build scripts to take a directory of patches instead of the huge patch
> it uses right now? It would make it much much easier to track the
> differences if the changes to each module was tracked in one patch in
> the CVS repo. It would also make it clearer what each change does (not
> at all clear sometimes with the current huge patch...comments and/or
> links to bugzilla entries would have been great) as each change to each
> patch will at least have the commit message to go along with it.
> 
> And in the end...does it really help? Someone at Tresys will have to
> review every patch anyway...so should I start looking at patches?
> 
My goal is to get more then just TreSys to check in patches.  Currently
we have a bottleneck that is not likely to be fixed.  This is not
anyone's fault, but we are not taking advantage of OpenSource and
allowing many hands/eyes to do this work.  Changing a module to user
auth_use_nsswitch when we realize it calls getpw*, should be easy to get
merged, and should not have a gatekeeper to prevent it.  If multiple
policy writers agree on the patch, it should just go in.  Now more
complicated changes like removing user prefix need to go much more slowly.

As far as my massive fix versus a 250 minor patches, is just a matter of
someone coming up with a better way.  I don't want to have 250 different
patches in the spec file.  Quilt has been suggested, but I am  not
familiar with it and am not sure how it work, nor do I have time to
implement it.

My current work habit is to take AVC messages from everywhere that I get
them. Bugzilla, Mail List, Chat Rooms, email, Personal Testing ...  And
modify my pool.  When I am done I run a big diff against refpolicy and
 update the patch.  So referencing all of the bugzilla's would only get
a snapshot of the fixes and would be very time consuming and dirty up
the patches.  Getting a quilt of different fixes for each day, I do not
believe would be of much use either since they are random.  Me
documenting each fix and sending it immediately upstream would not help,
since this would overwhelm me and upstream.

We are bringing on another dedicated policy writer in September,   who
could maybe help with verifying policy changes and getting them upstream.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkh+LB4ACgkQrlYvE4MpobOHzACeMoNUCo46kxp5eOCaMOmRznMA
Gw4AoNy5Kp6rOcEDzcRDimbuyHlPqjdO
=MrXr
-----END PGP SIGNATURE-----

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[David Härdeman]

On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
>David Härdeman wrote:
>> While working on SELinux-enabling a Debian system, I often Google for
>> avc messages that show up in dmesg and 90% of the time it seems that the
>> problem has already been solved in Fedora's version of the refpolicy but
>> not in the upstream version.
...
>> The question is how to treat the patches after that? Should I post them
>> as I go through them (a couple per day for a couple of weeks?) and hope
>> that someone at Tresys will apply them?
...
>> 
>My goal is to get more then just TreSys to check in patches.  Currently
>we have a bottleneck that is not likely to be fixed.  This is not
>anyone's fault, but we are not taking advantage of OpenSource and
>allowing many hands/eyes to do this work.  Changing a module to user
>auth_use_nsswitch when we realize it calls getpw*, should be easy to get
>merged, and should not have a gatekeeper to prevent it.  If multiple
>policy writers agree on the patch, it should just go in.  Now more
>complicated changes like removing user prefix need to go much more slowly.
>
>As far as my massive fix versus a 250 minor patches, is just a matter of
>someone coming up with a better way.  I don't want to have 250 different
>patches in the spec file.  Quilt has been suggested, but I am  not
>familiar with it and am not sure how it work, nor do I have time to
>implement it.

You obviously already have a script to split the big patch into smaller 
ones. Wouldn't it be ok to have the smaller patches in a directory in 
the CVS repo, do your work against the small patches and then cat all 
patches into a big one before you do a new release? It would avoid 
having to change the .spec file and you'd be able to drop minor patches 
as they are merged upstream easily (and quilt is quite nice btw).

>My current work habit is to take AVC messages from everywhere that I get
>them. Bugzilla, Mail List, Chat Rooms, email, Personal Testing ...  And
>modify my pool.

Is that pool maintained in a repo somewhere? If not, would it perhaps be 
an idea to maintain the pool as a branch in the tresys repo?

> When I am done I run a big diff against refpolicy and
> update the patch.  So referencing all of the bugzilla's would only get
>a snapshot of the fixes and would be very time consuming and dirty up
>the patches. 

"dirty" the patches...but I'm thinking one line comments which explain 
some of the changes. I wouldn't call that dirtying the patches. For 
example, in postgrey.te, the redhat patch gives postgrey_t the ability 
to restart apache...and I can't see why. A quick one-liner which 
explained the background would have been great :)

>Getting a quilt of different fixes for each day, I do not
>believe would be of much use either since they are random.  Me
>documenting each fix and sending it immediately upstream would not help,
>since this would overwhelm me and upstream.

No, but committing one fix at a time to a repo would help, even if the 
"fix documentation" (i.e. commit message) is just "let firefox read config 
file foo".

>We are bringing on another dedicated policy writer in September,   who
>could maybe help with verifying policy changes and getting them upstream.

I guess what I'm really wondering is if I can help you in some way? 

-- 
David Härdeman

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Christopher J. PeBenito]

On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
> >David Härdeman wrote:
> >> While working on SELinux-enabling a Debian system, I often Google for
> >> avc messages that show up in dmesg and 90% of the time it seems that the
> >> problem has already been solved in Fedora's version of the refpolicy but
> >> not in the upstream version.
> ...
> >> The question is how to treat the patches after that? Should I post them
> >> as I go through them (a couple per day for a couple of weeks?) and hope
> >> that someone at Tresys will apply them?
[...]
> I guess what I'm really wondering is if I can help you in some way? 

The main points which would improve upstreaming efficiency from Dan's
set are:

1. description / justification

What this means tends to vary depending on what access is added by a
patch.  A patch that allows reading of usr_t files probably doesn't need
a big description while a patch that allows reading shadow_t does.
"myapp breaks without this rule" isn't a very good explanation,
especially if the access is questionable.  The app may be incorrectly
requesting extra access or it might be a bug in the app.

2. style

The changes need to meet the refpolicy style guidelines.  Dan is pretty
good about this, but with the volume, things still get by.

3. patch composition

A patch should be a changeset.  So if a rule is added to a bunch of
domains for the same reason, then that should be in a single patch by
itself.  If you add an interface so some modules can call it, that
should be a single patch that includes the new interface and the
additions to the callers.  This enables me to review the change as a
whole, and not have distractions from unrelated changes.  A set of small
fixes to a single module can be a single patch.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christopher J. PeBenito wrote:
> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
>>> David Härdeman wrote:
>>>> While working on SELinux-enabling a Debian system, I often Google for
>>>> avc messages that show up in dmesg and 90% of the time it seems that the
>>>> problem has already been solved in Fedora's version of the refpolicy but
>>>> not in the upstream version.
>> ...
>>>> The question is how to treat the patches after that? Should I post them
>>>> as I go through them (a couple per day for a couple of weeks?) and hope
>>>> that someone at Tresys will apply them?
> [...]
>> I guess what I'm really wondering is if I can help you in some way? 
> 
> The main points which would improve upstreaming efficiency from Dan's
> set are:
> 
> 1. description / justification
> 
> What this means tends to vary depending on what access is added by a
> patch.  A patch that allows reading of usr_t files probably doesn't need
> a big description while a patch that allows reading shadow_t does.
> "myapp breaks without this rule" isn't a very good explanation,
> especially if the access is questionable.  The app may be incorrectly
> requesting extra access or it might be a bug in the app.
> 
> 2. style
> 
> The changes need to meet the refpolicy style guidelines.  Dan is pretty
> good about this, but with the volume, things still get by.
> 
> 3. patch composition
> 
> A patch should be a changeset.  So if a rule is added to a bunch of
> domains for the same reason, then that should be in a single patch by
> itself.  If you add an interface so some modules can call it, that
> should be a single patch that includes the new interface and the
> additions to the callers.  This enables me to review the change as a
> whole, and not have distractions from unrelated changes.  A set of small
> fixes to a single module can be a single patch.
> 
And the problem I have with all of these is volume of change.  When a
new release goes out and a whole bunch of new users start using SELinux
the volume of bugzillas generated is use.  My first responsibility is to
get SELinux fixed for these users.  Marking up the policy with lots of
bugzilla's or justifications is both time consuming and I believe just
dirties the policy.  In the more bizarre categories that should be
required.  My goal is to get the non-bizarre changes upstreamed so we
could concentrate on the bizarre ones and either justify or remove them.

Changes like adding

fs_list_inotify should just get into the upstream.

grep fs_list_inotify  policy-20080710.patch
+fs_list_inotifyfs(certwatch_t)
+fs_list_inotifyfs(logrotate_t)
+fs_list_inotifyfs(logwatch_t)
+fs_list_inotifyfs(tmpreaper_t)
+	fs_list_inotifyfs($1_gconfd_t)
+fs_list_inotifyfs(gpg_t)
+fs_list_inotifyfs(gpg_helper_t)
 	fs_list_inotifyfs($1_mozilla_t)
+fs_list_inotifyfs(nsplugin_t)
+fs_list_inotifyfs(nsplugin_config_t)
+fs_list_inotifyfs(locate_t)
+fs_list_inotifyfs(httpd_t)
+	fs_list_inotifyfs($1_dbusd_t)
+fs_list_inotifyfs(system_dbusd_t)
 fs_list_inotifyfs(dovecot_t)
+fs_list_inotifyfs(fail2ban_t)
+fs_list_inotifyfs(gamin_t)
+fs_list_inotifyfs(gnomeclock_t)
+fs_list_inotifyfs(system_mail_t)
+fs_list_inotifyfs(NetworkManager_t)
+fs_list_inotifyfs(nscd_t)
+fs_list_inotifyfs(polkit_t)
+fs_list_inotifyfs(prelude_lml_t)
+fs_list_inotifyfs(nmbd_t)
+fs_list_inotifyfs(swat_t)
+	fs_list_inotifyfs($1_xserver_t)
+fs_search_inotifyfs(xdm_t)
+fs_list_inotifyfs(init_t)
+fs_list_inotifyfs(iptables_t)
+	fs_list_inotifyfs($1)
+fs_list_inotifyfs(load_policy_t)
- -	fs_list_inotifyfs($1_t)
+	fs_list_inotifyfs($1_usertype)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkh+RRwACgkQrlYvE4MpobM+vgCeJSBxEJ6/uErGs+dsn/JGSzAk
DBUAoIVP37Ejpr1b21QMfEfdqLrA+QKe
=sHu2
-----END PGP SIGNATURE-----

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[David Härdeman]

On Wed, Jul 16, 2008 at 02:59:40PM -0400, Daniel J Walsh wrote:
>Christopher J. PeBenito wrote:
>> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
>>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
>>>> David Härdeman wrote:
>>>>> While working on SELinux-enabling a Debian system, I often Google for
>>>>> avc messages that show up in dmesg and 90% of the time it seems that the
>>>>> problem has already been solved in Fedora's version of the refpolicy but
>>>>> not in the upstream version.
>>> ...
>>>>> The question is how to treat the patches after that? Should I post them
>>>>> as I go through them (a couple per day for a couple of weeks?) and hope
>>>>> that someone at Tresys will apply them?
>> [...]
>>> I guess what I'm really wondering is if I can help you in some way? 
>> 
>> The main points which would improve upstreaming efficiency from Dan's
>> set are:
>> 
>> 1. description / justification
...
>> 2. style
...
>> 3. patch composition
...

Basically all three requirements are the same as the general rules that 
apply for patch submissions to the linux-kernel mailing list (or to any 
well-behaved OSS project).

>And the problem I have with all of these is volume of change.  When a
>new release goes out and a whole bunch of new users start using SELinux
>the volume of bugzillas generated is use.  My first responsibility is to
>get SELinux fixed for these users. 

I can't imagine that a one-line commit comment would be an overwhelming 
burden when committing a couple of lines of policy changes? Heck, most 
maintainers should welcome it as it serves as a support for their own 
memory as well...I mean, most of these issues aren't exactly new for 
FOSS projects and SCM repos is the best answer people have come up with 
so far...

>Marking up the policy with lots of
>bugzilla's or justifications is both time consuming and I believe just
>dirties the policy.

Comments about patches are generally carried as commit messages and not 
inline. I don't see how it would dirty any policy. And in-line comments 
for unconventional changes I'd see not as "dirty" but as a great help to 
the person reading through the policy (I've already had a few "huh?" 
moments when reading through the RedHat patch, and that's not because 
they are bad in any way, its because its inevitable without the proper 
context).

>In the more bizarre categories that should be
>required.  My goal is to get the non-bizarre changes upstreamed so we
>could concentrate on the bizarre ones and either justify or remove them.

The question is if it's even possible to hunt down the original 
explanation for the bizarre ones after a few hundred of them have 
accumulated and they have been obscured by the passing of time?

>Changes like adding fs_list_inotify should just get into the upstream.

I realize I'm stepping into a debate which is somewhat over my head 
here, but couldn't Tresys arrange so that you get direct commit access, 
then you could commit trivial patches directly and send more 
unconventional ones to the list for discussion?

Or alternatively...perhaps a shadow branch could be setup where the 
commit rules would be more lax and then the changes could be synced over 
at intervals...

(Or even better, everything could be done via git...but that's a 
pipedream at this stage) :)

-- 
David Härdeman

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Härdeman wrote:
> On Wed, Jul 16, 2008 at 02:59:40PM -0400, Daniel J Walsh wrote:
>> Christopher J. PeBenito wrote:
>>> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
>>>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
>>>>> David Härdeman wrote:
>>>>>> While working on SELinux-enabling a Debian system, I often Google for
>>>>>> avc messages that show up in dmesg and 90% of the time it seems
>>>>>> that the
>>>>>> problem has already been solved in Fedora's version of the
>>>>>> refpolicy but
>>>>>> not in the upstream version.
>>>> ...
>>>>>> The question is how to treat the patches after that? Should I post
>>>>>> them
>>>>>> as I go through them (a couple per day for a couple of weeks?) and
>>>>>> hope
>>>>>> that someone at Tresys will apply them?
>>> [...]
>>>> I guess what I'm really wondering is if I can help you in some way? 
>>>
>>> The main points which would improve upstreaming efficiency from Dan's
>>> set are:
>>>
>>> 1. description / justification
> ...
>>> 2. style
> ...
>>> 3. patch composition
> ...
> 
> Basically all three requirements are the same as the general rules that
> apply for patch submissions to the linux-kernel mailing list (or to any
> well-behaved OSS project).
> 
>> And the problem I have with all of these is volume of change.  When a
>> new release goes out and a whole bunch of new users start using SELinux
>> the volume of bugzillas generated is use.  My first responsibility is to
>> get SELinux fixed for these users. 
> 
> I can't imagine that a one-line commit comment would be an overwhelming
> burden when committing a couple of lines of policy changes? Heck, most
> maintainers should welcome it as it serves as a support for their own
> memory as well...I mean, most of these issues aren't exactly new for
> FOSS projects and SCM repos is the best answer people have come up with
> so far...
> 
>> Marking up the policy with lots of
>> bugzilla's or justifications is both time consuming and I believe just
>> dirties the policy.
> 
> Comments about patches are generally carried as commit messages and not
> inline. I don't see how it would dirty any policy. And in-line comments
> for unconventional changes I'd see not as "dirty" but as a great help to
> the person reading through the policy (I've already had a few "huh?"
> moments when reading through the RedHat patch, and that's not because
> they are bad in any way, its because its inevitable without the proper
> context).
> 
>> In the more bizarre categories that should be
>> required.  My goal is to get the non-bizarre changes upstreamed so we
>> could concentrate on the bizarre ones and either justify or remove them.
> 
> The question is if it's even possible to hunt down the original
> explanation for the bizarre ones after a few hundred of them have
> accumulated and they have been obscured by the passing of time?
> 
>> Changes like adding fs_list_inotify should just get into the upstream.
> 
> I realize I'm stepping into a debate which is somewhat over my head
> here, but couldn't Tresys arrange so that you get direct commit access,
> then you could commit trivial patches directly and send more
> unconventional ones to the list for discussion?
> 
> Or alternatively...perhaps a shadow branch could be setup where the
> commit rules would be more lax and then the changes could be synced over
> at intervals...
> 
> (Or even better, everything could be done via git...but that's a
> pipedream at this stage) :)
> 
All of these suggestions are fine and yes if we had to do it all over
again, every change would be documented with links to bugzilla.emails,
conversations in the hall.  I am looking for help to get it better under
control.  I am not looking for direct commit, or at least a commit via
an ack process.

Patches have been sent up stream in the past that have got lost in the
volume of work that Chris has to do.  Not his fault.  But we have a
system where we have only one person whose primary job is not to check
in policy patches, having to review every patch.  And we have the person
generating most of the policy falling further and further behind.  While
the kernel has teams of engineers working on patches, reviewing them and
applying them.  They also have people who just cherry pick obvious fixes
and apply them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkh+TqwACgkQrlYvE4MpobPhCACg4Mw87YU6lUR5HsuIjmKADWFE
8PAAoMD+5BGOHYlPaGULJ4apbpIVRwPL
=Rz61
-----END PGP SIGNATURE-----

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Brett Lentz]

On Wed, 2008-07-16 at 15:40 -0400, Daniel J Walsh wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> David Härdeman wrote:
> > On Wed, Jul 16, 2008 at 02:59:40PM -0400, Daniel J Walsh wrote:
> >> Christopher J. PeBenito wrote:
> >>> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
> >>>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
> >>>>> David Härdeman wrote:
> >>>>>> While working on SELinux-enabling a Debian system, I often Google for
> >>>>>> avc messages that show up in dmesg and 90% of the time it seems
> >>>>>> that the
> >>>>>> problem has already been solved in Fedora's version of the
> >>>>>> refpolicy but
> >>>>>> not in the upstream version.
> >>>> ...
> >>>>>> The question is how to treat the patches after that? Should I post
> >>>>>> them
> >>>>>> as I go through them (a couple per day for a couple of weeks?) and
> >>>>>> hope
> >>>>>> that someone at Tresys will apply them?
> >>> [...]
> >>>> I guess what I'm really wondering is if I can help you in some way? 
> >>>
> >>> The main points which would improve upstreaming efficiency from Dan's
> >>> set are:
> >>>
> >>> 1. description / justification
> > ...
> >>> 2. style
> > ...
> >>> 3. patch composition
> > ...
> > 
> > Basically all three requirements are the same as the general rules that
> > apply for patch submissions to the linux-kernel mailing list (or to any
> > well-behaved OSS project).
> > 
> >> And the problem I have with all of these is volume of change.  When a
> >> new release goes out and a whole bunch of new users start using SELinux
> >> the volume of bugzillas generated is use.  My first responsibility is to
> >> get SELinux fixed for these users. 
> > 
> > I can't imagine that a one-line commit comment would be an overwhelming
> > burden when committing a couple of lines of policy changes? Heck, most
> > maintainers should welcome it as it serves as a support for their own
> > memory as well...I mean, most of these issues aren't exactly new for
> > FOSS projects and SCM repos is the best answer people have come up with
> > so far...
> > 
> >> Marking up the policy with lots of
> >> bugzilla's or justifications is both time consuming and I believe just
> >> dirties the policy.
> > 
> > Comments about patches are generally carried as commit messages and not
> > inline. I don't see how it would dirty any policy. And in-line comments
> > for unconventional changes I'd see not as "dirty" but as a great help to
> > the person reading through the policy (I've already had a few "huh?"
> > moments when reading through the RedHat patch, and that's not because
> > they are bad in any way, its because its inevitable without the proper
> > context).
> > 
> >> In the more bizarre categories that should be
> >> required.  My goal is to get the non-bizarre changes upstreamed so we
> >> could concentrate on the bizarre ones and either justify or remove them.
> > 
> > The question is if it's even possible to hunt down the original
> > explanation for the bizarre ones after a few hundred of them have
> > accumulated and they have been obscured by the passing of time?
> > 
> >> Changes like adding fs_list_inotify should just get into the upstream.
> > 
> > I realize I'm stepping into a debate which is somewhat over my head
> > here, but couldn't Tresys arrange so that you get direct commit access,
> > then you could commit trivial patches directly and send more
> > unconventional ones to the list for discussion?
> > 
> > Or alternatively...perhaps a shadow branch could be setup where the
> > commit rules would be more lax and then the changes could be synced over
> > at intervals...
> > 
> > (Or even better, everything could be done via git...but that's a
> > pipedream at this stage) :)
> > 
> All of these suggestions are fine and yes if we had to do it all over
> again, every change would be documented with links to bugzilla.emails,
> conversations in the hall.  I am looking for help to get it better under
> control.  I am not looking for direct commit, or at least a commit via
> an ack process.
> 
> Patches have been sent up stream in the past that have got lost in the
> volume of work that Chris has to do.  Not his fault.  But we have a
> system where we have only one person whose primary job is not to check
> in policy patches, having to review every patch.  And we have the person
> generating most of the policy falling further and further behind.  While
> the kernel has teams of engineers working on patches, reviewing them and
> applying them.  They also have people who just cherry pick obvious fixes
> and apply them.
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkh+TqwACgkQrlYvE4MpobPhCACg4Mw87YU6lUR5HsuIjmKADWFE
> 8PAAoMD+5BGOHYlPaGULJ4apbpIVRwPL
> =Rz61
> -----END PGP SIGNATURE-----
> 


What this tells me is that the reference policy has grown beyond the
ability of one person to maintain it.

Perhaps it's time to add another maintainer?

If this means that Tresys isn't the appropriate place to host the
reference policy, perhaps a new host needs to be found as well.

To be honest, from my perspective as an SELinux consumer and long-time
follower of this list, it seems to me that Fedora's policy is very
nearly becoming the de facto reference policy just by virtue of its more
active development.

Perhaps it's time to think about moving the reference policy hosting back 
to sourceforge, or even to fedorahosted?


_______________________________
Brett Lentz | CarDomain Network
System Administrator

blentz@cardomain.com | tel 206.926.2109 | cell 206.851.6669
http://www.cardomain.com/id/wakko666

Weiler's Law:
	Nothing is impossible for the man who doesn't have to do it himself.


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[David Härdeman]

On Wed, Jul 16, 2008 at 03:40:29PM -0400, Daniel J Walsh wrote:
>All of these suggestions are fine and yes if we had to do it all over
>again, every change would be documented with links to bugzilla.emails,
>conversations in the hall.  I am looking for help to get it better under
>control.  I am not looking for direct commit, or at least a commit via
>an ack process.

I'm sorry, but I still haven't understood what *kind* of help you're 
looking for...except for wishing Chris had > 24h per day. :)

>Patches have been sent up stream in the past that have got lost in the
>volume of work that Chris has to do.  Not his fault.  But we have a
>system where we have only one person whose primary job is not to check
>in policy patches, having to review every patch. 

So obviously something is wrong in the refpolicy patch acceptance 
process? As a comparison, every single patch is applied by Linus to the 
kernel (even though they've been filtered by maintainers first) and going 
from 2.6.25 to 2.6.26-rc1 alone was 7555 patches.

>And we have the person
>generating most of the policy falling further and further behind.  While
>the kernel has teams of engineers working on patches, reviewing them and
>applying them.  They also have people who just cherry pick obvious fixes
>and apply them.

Well, I still don't know what should be done? Just splitting the RH patch 
into per-module patches was a great help to me. Out of those 200+ patches, 
about 50% were less than 100 lines and I'm guessing around 50% are of the 
no-brainer kind (3 were 1000+ lines). If those 50% could be identified and 
applied in quick succession by Chris...the RH patch wouldn't shrink by 50% 
in number of lines but it would shrink by 50% in number of modules affected.

-- 
David Härdeman

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Mike Edenfield]

Christopher J. PeBenito wrote:

Having built up a few patches myself, I can say that my employer is (so 
far) willing to let me work on this kind of thing in my down time.  I'd 
be happy to help out, but I'm not sure where to jump in.

> The main points which would improve upstreaming efficiency from Dan's
> set are:
> 
> 1. description / justification
> 
> What this means tends to vary depending on what access is added by a
> patch.  A patch that allows reading of usr_t files probably doesn't need
> a big description while a patch that allows reading shadow_t does.
> "myapp breaks without this rule" isn't a very good explanation,
> especially if the access is questionable.  The app may be incorrectly
> requesting extra access or it might be a bug in the app.

I guess from my perspective there are two different things that I think 
I could help with if I knew how to proceed:

* Clearing the backlog from the current patchset.  For this, would it be 
helpful to go through the patches that Dan already posted and try to 
break them out and figure out what the justification was?  It would be 
time consuming, but for example, some of the samba changes I'm pretty 
sure I know what the motivation was, since I've made similar changes.

* Keeping the backlog from building up.  For this, would it help to have 
more people (e.g. like myself) watching bugzilla for problems and 
working up patches?  My main issue doing this right now is I don't run 
FC (I run Gentoo) but I could probably set up an environment to help 
out.  I'm also particularly aware of the problem Dan mentioned: other 
distributions aren't getting the benefits of his patches, and in some 
cases, even when they are upstreamed they are stuck behind an "if 
redhat" conditional.

> 2. style
> 
> The changes need to meet the refpolicy style guidelines.  Dan is pretty
> good about this, but with the volume, things still get by.

Are these published?  I try to make my changes "look like" existing 
policy but that's just my subjective guessing.

--Mike



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Eric Paris]

On Wed, Jul 16, 2008 at 4:18 PM, David Härdeman <david@hardeman.nu> wrote:
> On Wed, Jul 16, 2008 at 03:40:29PM -0400, Daniel J Walsh wrote:

>> Patches have been sent up stream in the past that have got lost in the
>> volume of work that Chris has to do.  Not his fault.  But we have a
>> system where we have only one person whose primary job is not to check
>> in policy patches, having to review every patch.
>
> So obviously something is wrong in the refpolicy patch acceptance process?
> As a comparison, every single patch is applied by Linus to the kernel (even
> though they've been filtered by maintainers first) and going from 2.6.25 to
> 2.6.26-rc1 alone was 7555 patches.

The difference being that Linus has trusted subsystem maintainers and
anything from those people are pulled into his tree without any
review.  Linus actually looks at (relatively) few patches these day.
As an example look at how Linus doesn't add his 'Signed-off-by' to
SELinux patches.  He didn't review them or even look at them.  He
trusted James and so in they went.

In the policy world Chris looks at every single patch and we don't
have any 'trust' from other people to commit.  The main reason for
that being that we only really have one other person in the community
who really does a lot of active development (Dan) and he is the first
to tell you that his primary motivation is to fix user problems and
move on rather than try to 'do what is right for the technology."  He
doesn't have time to follow strict (unwritten) style guidelines or
verify that every change is 'the right way' for upstream.  Just not
enough man hours in the day.

As an example if my fedora sendmail program starts needing bogus
permissions Dan is going to allow it in Fedora policy and will push
back on the sendmail people to stop it from needing that bogus access.
 This shouldn't (and doesn't usually) get pushed towards upstream.
What Dan doesn't have time for is to go back this quick fixes make
sure they are clean and documented, and send them along. We have time
constraints on both Dan and Chris.

I'm told the Red Hat is trying to hire a new person to work full time
on nothing but policy.  My guess is that as this person becomes
knowledgeable enough direct commit access to the upstream refpolicy
will be given allowing him to bypass Chris.  Chris will probably still
be the overlord and might kick things out that he doesn't like (much
as Linus does in the kernel) after the fact, but until there are
multiple people who both can and want direct commit access this isn't
going to resolve.  (I don't think Dan even wants upstream direct
commit access but I could be wrong)

I don't think there is a solution with the manpower we have today to
fix the issue of the ever growing backlog.  My guess is that if you
start showing Chris and the community that you can decide what from
the monstrosity that is the Fedora patch set is reasonable and ready
for upstream you can get direct commit access.  Tresys is not trying
to be a bottleneck, stranglehold, dictator, or sole proprietor of the
policy they just happen to have the man who cares about the code above
all else and he is rightfully the maintainer.  If more people come
along and want to do what's right for upstream we can start working on
the web of trust rather than the one man show we have today.  (moving
the actual repository from tresys to sourceforge or to fedoraproject
won't magically solve any of these issues.  I haven't seen tresys
reject developers, I just haven't seen nearly enough developers)

The solution?  Take what you find, clean it, send it, get it in
through Chris.  Hopefully RH will have a person to do the same soon!

-Eric


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Christopher J. PeBenito]

On Wed, 2008-07-16 at 16:19 -0400, Mike Edenfield wrote:
> Christopher J. PeBenito wrote:
> 
> Having built up a few patches myself, I can say that my employer is (so 
> far) willing to let me work on this kind of thing in my down time.  I'd 
> be happy to help out, but I'm not sure where to jump in.
> 
> > The main points which would improve upstreaming efficiency from Dan's
> > set are:
> > 
> > 1. description / justification
> > 
> > What this means tends to vary depending on what access is added by a
> > patch.  A patch that allows reading of usr_t files probably doesn't need
> > a big description while a patch that allows reading shadow_t does.
> > "myapp breaks without this rule" isn't a very good explanation,
> > especially if the access is questionable.  The app may be incorrectly
> > requesting extra access or it might be a bug in the app.
> 
> I guess from my perspective there are two different things that I think 
> I could help with if I knew how to proceed:
> 
> * Clearing the backlog from the current patchset.  For this, would it be 
> helpful to go through the patches that Dan already posted and try to 
> break them out and figure out what the justification was?  It would be 
> time consuming, but for example, some of the samba changes I'm pretty 
> sure I know what the motivation was, since I've made similar changes.

Yes.

> * Keeping the backlog from building up.  For this, would it help to have 
> more people (e.g. like myself) watching bugzilla for problems and 
> working up patches?

You could do that, or if Dan has his patch set available from a repo or
even just a ftp/web site, you could just look through that.

>   My main issue doing this right now is I don't run 
> FC (I run Gentoo) but I could probably set up an environment to help 
> out.  I'm also particularly aware of the problem Dan mentioned: other 
> distributions aren't getting the benefits of his patches, and in some 
> cases, even when they are upstreamed they are stuck behind an "if 
> redhat" conditional.

Well we definitely want distro-specific behaviors to be controlled by
build options like distro_redhat, distro_gentoo, etc.  If something
turns out to be general, then theres nothing wrong with pulling it out
of an ifdef block.

> > 2. style
> > 
> > The changes need to meet the refpolicy style guidelines.  Dan is pretty
> > good about this, but with the volume, things still get by.
> 
> Are these published?  I try to make my changes "look like" existing 
> policy but that's just my subjective guessing.

There is an interface naming guide, but not a written up style guide for
the remainder of the policy.  If people are becoming interested in
helping out, then I can write that up.  In short, most things tend to be
in alphabetical order.  In the TE policy, you do the declarations, then
the raw allow rules, then calls to other modules interfaces' sorted
first by layer (bottom up) then by alpha order.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Christopher J. PeBenito]

On Wed, 2008-07-16 at 13:09 -0700, Brett Lentz wrote:
> On Wed, 2008-07-16 at 15:40 -0400, Daniel J Walsh wrote:
> > David Härdeman wrote:
> > > On Wed, Jul 16, 2008 at 02:59:40PM -0400, Daniel J Walsh wrote:
> > >> Christopher J. PeBenito wrote:
> > >>> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
> > >>>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
> > >>>>> David Härdeman wrote:
> > >>>>>> While working on SELinux-enabling a Debian system, I often Google for
> > >>>>>> avc messages that show up in dmesg and 90% of the time it seems
> > >>>>>> that the
> > >>>>>> problem has already been solved in Fedora's version of the
> > >>>>>> refpolicy but
> > >>>>>> not in the upstream version.
[...]
> To be honest, from my perspective as an SELinux consumer and long-time
> follower of this list, it seems to me that Fedora's policy is very
> nearly becoming the de facto reference policy just by virtue of its more
> active development.

What is probably not clear to you is that I focus on large scale
changes/policy architecture, such as the experiment with ubac/rbac
separations, building the enforcing X desktop policies, and the FCGlob
file contexts experiment.  Being a distribution policy person, Dan is on
the front lines handling bugs, while I am somewhat disconnected (Gentoo
doesn't have nearly as many SELinux users).  As mentioned by others, Dan
is working mainly on get things functioning.  Obviously, as upstream, I
want things to work too, but Dan deserves much credit for the many
policy adjustments that are required as software gets updated.  But to
say that the Fedora policy has more active development is dead wrong.

-- 
Chris PeBenito
Tresys Technology, LLC
(410) 290-1411 x150



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Fedora refpolicy patches

[Brett Lentz]

On Fri, 2008-07-18 at 08:32 -0400, Christopher J. PeBenito wrote:
> On Wed, 2008-07-16 at 13:09 -0700, Brett Lentz wrote:
> > On Wed, 2008-07-16 at 15:40 -0400, Daniel J Walsh wrote:
> > > David Härdeman wrote:
> > > > On Wed, Jul 16, 2008 at 02:59:40PM -0400, Daniel J Walsh wrote:
> > > >> Christopher J. PeBenito wrote:
> > > >>> On Wed, 2008-07-16 at 19:44 +0200, David Härdeman wrote:
> > > >>>> On Wed, Jul 16, 2008 at 01:13:03PM -0400, Daniel J Walsh wrote:
> > > >>>>> David Härdeman wrote:
> > > >>>>>> While working on SELinux-enabling a Debian system, I often Google for
> > > >>>>>> avc messages that show up in dmesg and 90% of the time it seems
> > > >>>>>> that the
> > > >>>>>> problem has already been solved in Fedora's version of the
> > > >>>>>> refpolicy but
> > > >>>>>> not in the upstream version.
> [...]
> > To be honest, from my perspective as an SELinux consumer and long-time
> > follower of this list, it seems to me that Fedora's policy is very
> > nearly becoming the de facto reference policy just by virtue of its more
> > active development.
> 
> What is probably not clear to you is that I focus on large scale
> changes/policy architecture, such as the experiment with ubac/rbac
> separations, building the enforcing X desktop policies, and the FCGlob
> file contexts experiment.  Being a distribution policy person, Dan is on
> the front lines handling bugs, while I am somewhat disconnected (Gentoo
> doesn't have nearly as many SELinux users).  As mentioned by others, Dan
> is working mainly on get things functioning.  Obviously, as upstream, I
> want things to work too, but Dan deserves much credit for the many
> policy adjustments that are required as software gets updated.  But to
> say that the Fedora policy has more active development is dead wrong.
> 


I completely agree.  Apologies for my poor wording. I didn't mean to
characterize it quite like that. I've been following the list long
enough to have seen several of your contributions. I believe that, like
Dan and Stephen, you've been very key in selinux development.

What I meant was this. The Fedora policy tends to be very quick to fix
bugs and make adjustments. It's maintainers are generally very
responsive.

When it comes to maintaining the refpolicy, these are qualities that I
would find desirable. I believe that refpolicy's value is diminished if
key fixes begin to lag behind specific distros, because then every other
distro that ships selinux either need to port the changes from Fedora,
or wait until refpolicy gets around to merging these changes.

There's nothing wrong with focusing on developing large features. It's
very necessary work. However, this may just be a sign that the refpolicy
has matured enough that its needs are changing. This seems similar to
the discussions of changing from a monolithic policy to a modular
policy.

Perhaps it needs more than a single point of contact for merging
patches. Alternately, similar to Linus' current role with the kernel, it
may just need someone whose focus is on merging the work of others
rather than developing large features themselves.



 ---Brett.


Having nothing, nothing can he lose.
		-- William Shakespeare, "Henry VI"


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.