New SELinux toolchain build system

New SELinux toolchain build system

[Vikram Ambrose]

[-- Attachment #1: Type: text/plain, Size: 3129 bytes --]

Attached to this email is a tarball snapshot of a new build system
for the SELinux toolchain that I have been working on for the last
couple of months.

This autoconf/automake build system has many advantages over the
current Makefiles.

1) It is capable of correctly cross compiling all libraries and
  tools, including the python wrappers.
2) It allows the distro maintainer to set sysconfdir, prefix,
  pamdir, startupdir etc..
3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
  Autoconf code, formatted to ~80 char line length.
4) It lets RPM spec and .deb control take advantage of the autoconf
  infrastructure already built into these package formats, allowing
  distro maintainers to quickly and effortlessly build distro
  packages.
5) It is completely self dependent, ie. one can build the complete
  toolchain now resolving dependencies within the build, without
  needing to link against a system wide library.

Due to an incompatibility with libtool, related to the current
naming convention of the python wrappers and classes, a patch is
supplied to rename:
  *) _selinux.so to libpyselinux.so
  *) _semanage.so to libpysemanage.so
  *) _audit2why.so to libpyaudit2why.so

Similarily, the update also renames:
  *) selinux.py to __init__.py
  *) semanage.py to __init__.py
and installs these plugins into their own sub site-packages folder.

For Example: libselinux/src/Makefile.am
line 35 # When the upstream tree is patched this should be removed
line 36 # and selinux.py must be renamed to __init__.py
line 37 __init__.py : selinux.py
line 38        cp selinux.py $@

A similar measure is taken in libsemanage/src/Makefile.am for semanage.py.

On a similar note, the code currently uses a macro called SHARED,
which is currently being wrapped with a forced -include libtool_compat.h
using -DPIC (a libtool defined macro). This too can be removed if the code
can be reformatted.

This build system has been thoroughly tested to function correctly.
Compiling natively for localhost, as well as cross compiling for
the following platforms:
  *) PPC32
  *) PPC64
  *) ARM (Versatile family)
  *) Common PC X86_64

It has also been tested on mainstream Linux distributions such
as Ubuntu 8.04 and Fedora 9.

This new build system is a replacement for the existing build system.
They cannot be used concurrently. I leave no guarantee on backward
compatibility after applying this update.

Recently a Ruby wrapper was added to the SELinux trunk, this build
system does not build it. Though with the current infrastructure in
place, it is a very simple procedure for anyone familiar with ruby
to include it.

This contribution comes out of the integration efforts WindRiver has
recently taken to add a SELinux feature to WindRiver Linux. It comes
with no obligation or indemnity from WindRiver or myself. Distributed
as not copyrighted, public domain software, in accordance with
libselinux.

I ask if this update can be merged into the main SELinux trunk.
I will help coordinate the merge if necessary.

Sincerely,

Vikram Ambrose

-- 
Vikram Ambrose | Linux Products Division | WindRiver Corporation


[-- Attachment #2: selinux-trunk-autotools.tar.bz2 --]
[-- Type: application/x-bzip, Size: 18509 bytes --]

Re: New SELinux toolchain build system

[Daniel J Walsh]

Vikram Ambrose wrote:
> Attached to this email is a tarball snapshot of a new build system
> for the SELinux toolchain that I have been working on for the last
> couple of months.
> 
> This autoconf/automake build system has many advantages over the
> current Makefiles.
> 
> 1) It is capable of correctly cross compiling all libraries and
>  tools, including the python wrappers.
> 2) It allows the distro maintainer to set sysconfdir, prefix,
>  pamdir, startupdir etc..
> 3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
>  Autoconf code, formatted to ~80 char line length.
> 4) It lets RPM spec and .deb control take advantage of the autoconf
>  infrastructure already built into these package formats, allowing
>  distro maintainers to quickly and effortlessly build distro
>  packages.
> 5) It is completely self dependent, ie. one can build the complete
>  toolchain now resolving dependencies within the build, without
>  needing to link against a system wide library.
> 
> Due to an incompatibility with libtool, related to the current
> naming convention of the python wrappers and classes, a patch is
> supplied to rename:
>  *) _selinux.so to libpyselinux.so
>  *) _semanage.so to libpysemanage.so
>  *) _audit2why.so to libpyaudit2why.so
> 
> Similarily, the update also renames:
>  *) selinux.py to __init__.py
>  *) semanage.py to __init__.py
> and installs these plugins into their own sub site-packages folder.
> 
> For Example: libselinux/src/Makefile.am
> line 35 # When the upstream tree is patched this should be removed
> line 36 # and selinux.py must be renamed to __init__.py
> line 37 __init__.py : selinux.py
> line 38        cp selinux.py $@
> 
> A similar measure is taken in libsemanage/src/Makefile.am for semanage.py.
> 
> On a similar note, the code currently uses a macro called SHARED,
> which is currently being wrapped with a forced -include libtool_compat.h
> using -DPIC (a libtool defined macro). This too can be removed if the code
> can be reformatted.
> 
> This build system has been thoroughly tested to function correctly.
> Compiling natively for localhost, as well as cross compiling for
> the following platforms:
>  *) PPC32
>  *) PPC64
>  *) ARM (Versatile family)
>  *) Common PC X86_64
> 
> It has also been tested on mainstream Linux distributions such
> as Ubuntu 8.04 and Fedora 9.
> 
> This new build system is a replacement for the existing build system.
> They cannot be used concurrently. I leave no guarantee on backward
> compatibility after applying this update.
> 
> Recently a Ruby wrapper was added to the SELinux trunk, this build
> system does not build it. Though with the current infrastructure in
> place, it is a very simple procedure for anyone familiar with ruby
> to include it.
> 
> This contribution comes out of the integration efforts WindRiver has
> recently taken to add a SELinux feature to WindRiver Linux. It comes
> with no obligation or indemnity from WindRiver or myself. Distributed
> as not copyrighted, public domain software, in accordance with
> libselinux.
> 
> I ask if this update can be merged into the main SELinux trunk.
> I will help coordinate the merge if necessary.
> 
> Sincerely,
> 
> Vikram Ambrose
> 
This sounds good to me. We could use this to clean up some of the spec
files. x86_64 library stuff.  I have no problem fixing the spec files up
to use this if it becomes the new build environment.  Just need to make
sure python - include selinux and friends works.

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: New SELinux toolchain build system

[Joshua Brindle]

Vikram Ambrose wrote:
> Attached to this email is a tarball snapshot of a new build system
> for the SELinux toolchain that I have been working on for the last
> couple of months.
> 
> This autoconf/automake build system has many advantages over the
> current Makefiles.
> 
> 1) It is capable of correctly cross compiling all libraries and
>  tools, including the python wrappers.
> 2) It allows the distro maintainer to set sysconfdir, prefix,
>  pamdir, startupdir etc..
> 3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
>  Autoconf code, formatted to ~80 char line length.
> 4) It lets RPM spec and .deb control take advantage of the autoconf
>  infrastructure already built into these package formats, allowing
>  distro maintainers to quickly and effortlessly build distro
>  packages.
> 5) It is completely self dependent, ie. one can build the complete
>  toolchain now resolving dependencies within the build, without
>  needing to link against a system wide library.
> 
> Due to an incompatibility with libtool, related to the current
> naming convention of the python wrappers and classes, a patch is
> supplied to rename:
>  *) _selinux.so to libpyselinux.so
>  *) _semanage.so to libpysemanage.so
>  *) _audit2why.so to libpyaudit2why.so
> 
> Similarily, the update also renames:
>  *) selinux.py to __init__.py
>  *) semanage.py to __init__.py
> and installs these plugins into their own sub site-packages folder.
> 
> For Example: libselinux/src/Makefile.am
> line 35 # When the upstream tree is patched this should be removed
> line 36 # and selinux.py must be renamed to __init__.py
> line 37 __init__.py : selinux.py
> line 38        cp selinux.py $@
> 
> A similar measure is taken in libsemanage/src/Makefile.am for semanage.py.
> 
> On a similar note, the code currently uses a macro called SHARED,
> which is currently being wrapped with a forced -include libtool_compat.h
> using -DPIC (a libtool defined macro). This too can be removed if the code
> can be reformatted.
> 
> This build system has been thoroughly tested to function correctly.
> Compiling natively for localhost, as well as cross compiling for
> the following platforms:
>  *) PPC32
>  *) PPC64
>  *) ARM (Versatile family)
>  *) Common PC X86_64
> 
> It has also been tested on mainstream Linux distributions such
> as Ubuntu 8.04 and Fedora 9.
> 
> This new build system is a replacement for the existing build system.
> They cannot be used concurrently. I leave no guarantee on backward
> compatibility after applying this update.
> 
> Recently a Ruby wrapper was added to the SELinux trunk, this build
> system does not build it. Though with the current infrastructure in
> place, it is a very simple procedure for anyone familiar with ruby
> to include it.
> 
> This contribution comes out of the integration efforts WindRiver has
> recently taken to add a SELinux feature to WindRiver Linux. It comes
> with no obligation or indemnity from WindRiver or myself. Distributed
> as not copyrighted, public domain software, in accordance with
> libselinux.
> 
> I ask if this update can be merged into the main SELinux trunk.
> I will help coordinate the merge if necessary.
> 

I've just started looking at this. We need to be sure that it covers current make targets like make swigify which generate the c wrappers from swig interface files (we run this before checkins). Also make test needs to work.

Also there is no top level ./configure so this won't let you build the entire repo. I'd like something (even if it was just the makefile that ran ./configure in each dir.

I also get a failure:
[root@misterfreeze libsepol]# autoreconf -iv
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal 
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf
autoreconf: running: /usr/bin/autoheader
autoreconf: running: automake --add-missing --copy --no-force
configure.ac:17: installing `./compile'
configure.ac:6: installing `./install-sh'
configure.ac:6: installing `./missing'
include/Makefile.am:2: whitespace following trailing backslash
include/Makefile.am:5: whitespace following trailing backslash
include/Makefile.am:8: whitespace following trailing backslash
include/Makefile.am:9: whitespace following trailing backslash
include/Makefile.am:11: whitespace following trailing backslash
include/Makefile.am:14: whitespace following trailing backslash
include/Makefile.am:20: whitespace following trailing backslash
src/Makefile.am:10: Libtool library used but `LIBTOOL' is undefined
src/Makefile.am:10:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
src/Makefile.am:10:   to `configure.ac' and run `aclocal' and `autoconf' again.
src/Makefile.am:10:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
src/Makefile.am:10:   its definition is in aclocal's search path.
src/Makefile.am: installing `./depcomp'
Makefile.am: installing `./INSTALL'
Makefile.am: required file `./NEWS' not found
Makefile.am: required file `./README' not found
Makefile.am: required file `./AUTHORS' not found
autoreconf: automake failed with exit status: 1



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: New SELinux toolchain build system

[Vikram Ambrose]

[-- Attachment #1: Type: text/plain, Size: 5975 bytes --]

Joshua Brindle wrote:
> Vikram Ambrose wrote:
>   
>> Attached to this email is a tarball snapshot of a new build system
>> for the SELinux toolchain that I have been working on for the last
>> couple of months.
>>
>> This autoconf/automake build system has many advantages over the
>> current Makefiles.
>>
>> 1) It is capable of correctly cross compiling all libraries and
>>  tools, including the python wrappers.
>> 2) It allows the distro maintainer to set sysconfdir, prefix,
>>  pamdir, startupdir etc..
>> 3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
>>  Autoconf code, formatted to ~80 char line length.
>> 4) It lets RPM spec and .deb control take advantage of the autoconf
>>  infrastructure already built into these package formats, allowing
>>  distro maintainers to quickly and effortlessly build distro
>>  packages.
>> 5) It is completely self dependent, ie. one can build the complete
>>  toolchain now resolving dependencies within the build, without
>>  needing to link against a system wide library.
>>
>> Due to an incompatibility with libtool, related to the current
>> naming convention of the python wrappers and classes, a patch is
>> supplied to rename:
>>  *) _selinux.so to libpyselinux.so
>>  *) _semanage.so to libpysemanage.so
>>  *) _audit2why.so to libpyaudit2why.so
>>
>> Similarily, the update also renames:
>>  *) selinux.py to __init__.py
>>  *) semanage.py to __init__.py
>> and installs these plugins into their own sub site-packages folder.
>>
>> For Example: libselinux/src/Makefile.am
>> line 35 # When the upstream tree is patched this should be removed
>> line 36 # and selinux.py must be renamed to __init__.py
>> line 37 __init__.py : selinux.py
>> line 38        cp selinux.py $@
>>
>> A similar measure is taken in libsemanage/src/Makefile.am for semanage.py.
>>
>> On a similar note, the code currently uses a macro called SHARED,
>> which is currently being wrapped with a forced -include libtool_compat.h
>> using -DPIC (a libtool defined macro). This too can be removed if the code
>> can be reformatted.
>>
>> This build system has been thoroughly tested to function correctly.
>> Compiling natively for localhost, as well as cross compiling for
>> the following platforms:
>>  *) PPC32
>>  *) PPC64
>>  *) ARM (Versatile family)
>>  *) Common PC X86_64
>>
>> It has also been tested on mainstream Linux distributions such
>> as Ubuntu 8.04 and Fedora 9.
>>
>> This new build system is a replacement for the existing build system.
>> They cannot be used concurrently. I leave no guarantee on backward
>> compatibility after applying this update.
>>
>> Recently a Ruby wrapper was added to the SELinux trunk, this build
>> system does not build it. Though with the current infrastructure in
>> place, it is a very simple procedure for anyone familiar with ruby
>> to include it.
>>
>> This contribution comes out of the integration efforts WindRiver has
>> recently taken to add a SELinux feature to WindRiver Linux. It comes
>> with no obligation or indemnity from WindRiver or myself. Distributed
>> as not copyrighted, public domain software, in accordance with
>> libselinux.
>>
>> I ask if this update can be merged into the main SELinux trunk.
>> I will help coordinate the merge if necessary.
>>
>>     
>
> I've just started looking at this. We need to be sure that it covers current make targets like make swigify which generate the c wrappers from swig interface files (we run this before checkins). Also make test needs to work.
>
> Also there is no top level ./configure so this won't let you build the entire repo. I'd like something (even if it was just the makefile that ran ./configure in each dir.
>
> I also get a failure:
> [root@misterfreeze libsepol]# autoreconf -iv
> autoreconf: Entering directory `.'
> autoreconf: configure.ac: not using Gettext
> autoreconf: running: aclocal 
> autoreconf: configure.ac: tracing
> autoreconf: configure.ac: not using Libtool
> autoreconf: running: /usr/bin/autoconf
> autoreconf: running: /usr/bin/autoheader
> autoreconf: running: automake --add-missing --copy --no-force
> configure.ac:17: installing `./compile'
> configure.ac:6: installing `./install-sh'
> configure.ac:6: installing `./missing'
> include/Makefile.am:2: whitespace following trailing backslash
> include/Makefile.am:5: whitespace following trailing backslash
> include/Makefile.am:8: whitespace following trailing backslash
> include/Makefile.am:9: whitespace following trailing backslash
> include/Makefile.am:11: whitespace following trailing backslash
> include/Makefile.am:14: whitespace following trailing backslash
> include/Makefile.am:20: whitespace following trailing backslash
> src/Makefile.am:10: Libtool library used but `LIBTOOL' is undefined
> src/Makefile.am:10:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
> src/Makefile.am:10:   to `configure.ac' and run `aclocal' and `autoconf' again.
> src/Makefile.am:10:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
> src/Makefile.am:10:   its definition is in aclocal's search path.
> src/Makefile.am: installing `./depcomp'
> Makefile.am: installing `./INSTALL'
> Makefile.am: required file `./NEWS' not found
> Makefile.am: required file `./README' not found
> Makefile.am: required file `./AUTHORS' not found
> autoreconf: automake failed with exit status: 1
>
>
>   
Formatting error caused by the sanitation process. An updated tarball is 
attached to this email, with the following changes from the previous:

1) Fixed trailing white space in libsepol/src/Makefile.am
2) Added --enable-swig option to libselinux and libsemanage to regenerate
   c-wrapper
3) Added very very basic top level makefile, proof-of-concept to build 
entire
   trunk. make -f Makefile.toolchain PREFIX=/some/where (Requested by 
Joshua.B)
4) Minor python related cleanups in configure.ac for libselinux and
   libsemanage


Vikram

-- 
Vikram Ambrose | Linux Products Division | WindRiver Corporation


[-- Attachment #2: selinux-autotools-14082008.tar.bz2 --]
[-- Type: application/x-bzip, Size: 11545 bytes --]

Re: New SELinux toolchain build system

[Joshua Brindle]

Vikram Ambrose wrote:
> Joshua Brindle wrote:
>> Vikram Ambrose wrote:
>>  
>>> Attached to this email is a tarball snapshot of a new build system
>>> for the SELinux toolchain that I have been working on for the last
>>> couple of months.
>>>
>>> This autoconf/automake build system has many advantages over the
>>> current Makefiles.
>>>
>>> 1) It is capable of correctly cross compiling all libraries and
>>>  tools, including the python wrappers.
>>> 2) It allows the distro maintainer to set sysconfdir, prefix,
>>>  pamdir, startupdir etc..
>>> 3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
>>>  Autoconf code, formatted to ~80 char line length.
>>> 4) It lets RPM spec and .deb control take advantage of the autoconf
>>>  infrastructure already built into these package formats, allowing
>>>  distro maintainers to quickly and effortlessly build distro
>>>  packages.
>>> 5) It is completely self dependent, ie. one can build the complete
>>>  toolchain now resolving dependencies within the build, without
>>>  needing to link against a system wide library.
>>>
>>> Due to an incompatibility with libtool, related to the current
>>> naming convention of the python wrappers and classes, a patch is
>>> supplied to rename:
>>>  *) _selinux.so to libpyselinux.so
>>>  *) _semanage.so to libpysemanage.so
>>>  *) _audit2why.so to libpyaudit2why.so
>>>
>>> Similarily, the update also renames:
>>>  *) selinux.py to __init__.py
>>>  *) semanage.py to __init__.py
>>> and installs these plugins into their own sub site-packages folder.
>>>
>>> For Example: libselinux/src/Makefile.am
>>> line 35 # When the upstream tree is patched this should be removed
>>> line 36 # and selinux.py must be renamed to __init__.py
>>> line 37 __init__.py : selinux.py
>>> line 38        cp selinux.py $@
>>>
>>> A similar measure is taken in libsemanage/src/Makefile.am for
>>> semanage.py.
>>>
>>> On a similar note, the code currently uses a macro called SHARED,
>>> which is currently being wrapped with a forced -include libtool_compat.h
>>> using -DPIC (a libtool defined macro). This too can be removed if the
>>> code
>>> can be reformatted.
>>>
>>> This build system has been thoroughly tested to function correctly.
>>> Compiling natively for localhost, as well as cross compiling for
>>> the following platforms:
>>>  *) PPC32
>>>  *) PPC64
>>>  *) ARM (Versatile family)
>>>  *) Common PC X86_64
>>>
>>> It has also been tested on mainstream Linux distributions such
>>> as Ubuntu 8.04 and Fedora 9.
>>>
>>> This new build system is a replacement for the existing build system.
>>> They cannot be used concurrently. I leave no guarantee on backward
>>> compatibility after applying this update.
>>>
>>> Recently a Ruby wrapper was added to the SELinux trunk, this build
>>> system does not build it. Though with the current infrastructure in
>>> place, it is a very simple procedure for anyone familiar with ruby
>>> to include it.
>>>
>>> This contribution comes out of the integration efforts WindRiver has
>>> recently taken to add a SELinux feature to WindRiver Linux. It comes
>>> with no obligation or indemnity from WindRiver or myself. Distributed
>>> as not copyrighted, public domain software, in accordance with
>>> libselinux.
>>>
>>> I ask if this update can be merged into the main SELinux trunk.
>>> I will help coordinate the merge if necessary.
>>>
>>>     
>>
>> I've just started looking at this. We need to be sure that it covers
>> current make targets like make swigify which generate the c wrappers
>> from swig interface files (we run this before checkins). Also make
>> test needs to work.
>>
>> Also there is no top level ./configure so this won't let you build the
>> entire repo. I'd like something (even if it was just the makefile that
>> ran ./configure in each dir.
>>
>> I also get a failure:
>> [root@misterfreeze libsepol]# autoreconf -iv
>> autoreconf: Entering directory `.'
>> autoreconf: configure.ac: not using Gettext
>> autoreconf: running: aclocal autoreconf: configure.ac: tracing
>> autoreconf: configure.ac: not using Libtool
>> autoreconf: running: /usr/bin/autoconf
>> autoreconf: running: /usr/bin/autoheader
>> autoreconf: running: automake --add-missing --copy --no-force
>> configure.ac:17: installing `./compile'
>> configure.ac:6: installing `./install-sh'
>> configure.ac:6: installing `./missing'
>> include/Makefile.am:2: whitespace following trailing backslash
>> include/Makefile.am:5: whitespace following trailing backslash
>> include/Makefile.am:8: whitespace following trailing backslash
>> include/Makefile.am:9: whitespace following trailing backslash
>> include/Makefile.am:11: whitespace following trailing backslash
>> include/Makefile.am:14: whitespace following trailing backslash
>> include/Makefile.am:20: whitespace following trailing backslash
>> src/Makefile.am:10: Libtool library used but `LIBTOOL' is undefined
>> src/Makefile.am:10:   The usual way to define `LIBTOOL' is to add
>> `AC_PROG_LIBTOOL'
>> src/Makefile.am:10:   to `configure.ac' and run `aclocal' and
>> `autoconf' again.
>> src/Makefile.am:10:   If `AC_PROG_LIBTOOL' is in `configure.ac', make
>> sure
>> src/Makefile.am:10:   its definition is in aclocal's search path.
>> src/Makefile.am: installing `./depcomp'
>> Makefile.am: installing `./INSTALL'
>> Makefile.am: required file `./NEWS' not found
>> Makefile.am: required file `./README' not found
>> Makefile.am: required file `./AUTHORS' not found
>> autoreconf: automake failed with exit status: 1
>>
>>
>>   
> Formatting error caused by the sanitation process. An updated tarball is
> attached to this email, with the following changes from the previous:
> 
> 1) Fixed trailing white space in libsepol/src/Makefile.am
> 2) Added --enable-swig option to libselinux and libsemanage to regenerate
>   c-wrapper
> 3) Added very very basic top level makefile, proof-of-concept to build
> entire
>   trunk. make -f Makefile.toolchain PREFIX=/some/where (Requested by
> Joshua.B)
> 4) Minor python related cleanups in configure.ac for libselinux and
>   libsemanage
> 

Dan, Manoj, Caleb, you can skip to the bottom of this email

I have a branch with the latest patch, my current observations:
* checkpolicy/test isn't built (this is included on fedora and must be built)
* when I generated the configure scripts in a fresh repo apparently it felt the need to relicense libselinux as gplv3 (added a COPYING file)
* the CFLAGS we previously had have been completely removed, we need those back, particularly the warning ones and they aren't necessarily consistent:
libsepol/src/Makefile:CFLAGS ?= -Werror -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute
libsemanage/src/Makefile:CFLAGS ?= -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute -Wno-unused-parameter
checkpolicy/Makefile:CFLAGS ?= -g -Wall -Werror -Wshadow -O2 -pipe -fno-strict-aliasing
libselinux/src/Makefile:CFLAGS ?= -Werror -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute

Also, we filter out Werror from generated files (parser and lexer files, swig generated files) because they often cause warnings and we can't help it

* make clean from the top doesn't actually clean anything, it just removes the makefiles

* there is no way to distclean from the top level, eg., make the repository look like it will when we package up a release

* Why doesn't configure automatically find python? the vast majority of selinux installations are going to require the python bits so it should default to on and I'm not sure what the use of the autotools infrastructure is if it can't find the resources we need automatically

* configure is currently doing alot of tests for unnecessary things, such as a fortran compiler. This is one of the problems I have using autotools, it takes time and adds complexity for no reason

* building an embedded libselinux (--disable-rpm --disable-avc --disable-bool) fails:
	/usr/bin/ld: .libs/libselinux_la-load_policy.o: relocation R_X86_64_PC32 against undefined symbol `security_get_boolean_names_internal' can not be used when making a shared object; recompile with -fPIC

I think that is probably enough for now..

--

I created an autotools branch on the upstream git repo so all new patches should be based on that instead of a tarball. 

Also, I'd like distributions to start testing this branch as they are less likely to be affected by the issues raised above. Distributions will likely be mostly affected by the change of python bindings from _*.so to libpy*.so, and also will need to run configure with --with-python=/usr/include/python2.5 to have a useful build

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: New SELinux toolchain build system

[Vikram Ambrose]

Joshua Brindle wrote:
> Vikram Ambrose wrote:
>   
>> Joshua Brindle wrote:
>>     
>>> Vikram Ambrose wrote:
>>>  
>>>       
>>>> Attached to this email is a tarball snapshot of a new build system
>>>> for the SELinux toolchain that I have been working on for the last
>>>> couple of months.
>>>>
>>>> This autoconf/automake build system has many advantages over the
>>>> current Makefiles.
>>>>
>>>> 1) It is capable of correctly cross compiling all libraries and
>>>>  tools, including the python wrappers.
>>>> 2) It allows the distro maintainer to set sysconfdir, prefix,
>>>>  pamdir, startupdir etc..
>>>> 3) It employs no "hacks", it is to-the-book, clean, GNU Automake,
>>>>  Autoconf code, formatted to ~80 char line length.
>>>> 4) It lets RPM spec and .deb control take advantage of the autoconf
>>>>  infrastructure already built into these package formats, allowing
>>>>  distro maintainers to quickly and effortlessly build distro
>>>>  packages.
>>>> 5) It is completely self dependent, ie. one can build the complete
>>>>  toolchain now resolving dependencies within the build, without
>>>>  needing to link against a system wide library.
>>>>
>>>> Due to an incompatibility with libtool, related to the current
>>>> naming convention of the python wrappers and classes, a patch is
>>>> supplied to rename:
>>>>  *) _selinux.so to libpyselinux.so
>>>>  *) _semanage.so to libpysemanage.so
>>>>  *) _audit2why.so to libpyaudit2why.so
>>>>
>>>> Similarily, the update also renames:
>>>>  *) selinux.py to __init__.py
>>>>  *) semanage.py to __init__.py
>>>> and installs these plugins into their own sub site-packages folder.
>>>>
>>>> For Example: libselinux/src/Makefile.am
>>>> line 35 # When the upstream tree is patched this should be removed
>>>> line 36 # and selinux.py must be renamed to __init__.py
>>>> line 37 __init__.py : selinux.py
>>>> line 38        cp selinux.py $@
>>>>
>>>> A similar measure is taken in libsemanage/src/Makefile.am for
>>>> semanage.py.
>>>>
>>>> On a similar note, the code currently uses a macro called SHARED,
>>>> which is currently being wrapped with a forced -include libtool_compat.h
>>>> using -DPIC (a libtool defined macro). This too can be removed if the
>>>> code
>>>> can be reformatted.
>>>>
>>>> This build system has been thoroughly tested to function correctly.
>>>> Compiling natively for localhost, as well as cross compiling for
>>>> the following platforms:
>>>>  *) PPC32
>>>>  *) PPC64
>>>>  *) ARM (Versatile family)
>>>>  *) Common PC X86_64
>>>>
>>>> It has also been tested on mainstream Linux distributions such
>>>> as Ubuntu 8.04 and Fedora 9.
>>>>
>>>> This new build system is a replacement for the existing build system.
>>>> They cannot be used concurrently. I leave no guarantee on backward
>>>> compatibility after applying this update.
>>>>
>>>> Recently a Ruby wrapper was added to the SELinux trunk, this build
>>>> system does not build it. Though with the current infrastructure in
>>>> place, it is a very simple procedure for anyone familiar with ruby
>>>> to include it.
>>>>
>>>> This contribution comes out of the integration efforts WindRiver has
>>>> recently taken to add a SELinux feature to WindRiver Linux. It comes
>>>> with no obligation or indemnity from WindRiver or myself. Distributed
>>>> as not copyrighted, public domain software, in accordance with
>>>> libselinux.
>>>>
>>>> I ask if this update can be merged into the main SELinux trunk.
>>>> I will help coordinate the merge if necessary.
>>>>
>>>>     
>>>>         
>>> I've just started looking at this. We need to be sure that it covers
>>> current make targets like make swigify which generate the c wrappers
>>> from swig interface files (we run this before checkins). Also make
>>> test needs to work.
>>>
>>> Also there is no top level ./configure so this won't let you build the
>>> entire repo. I'd like something (even if it was just the makefile that
>>> ran ./configure in each dir.
>>>
>>> I also get a failure:
>>> [root@misterfreeze libsepol]# autoreconf -iv
>>> autoreconf: Entering directory `.'
>>> autoreconf: configure.ac: not using Gettext
>>> autoreconf: running: aclocal autoreconf: configure.ac: tracing
>>> autoreconf: configure.ac: not using Libtool
>>> autoreconf: running: /usr/bin/autoconf
>>> autoreconf: running: /usr/bin/autoheader
>>> autoreconf: running: automake --add-missing --copy --no-force
>>> configure.ac:17: installing `./compile'
>>> configure.ac:6: installing `./install-sh'
>>> configure.ac:6: installing `./missing'
>>> include/Makefile.am:2: whitespace following trailing backslash
>>> include/Makefile.am:5: whitespace following trailing backslash
>>> include/Makefile.am:8: whitespace following trailing backslash
>>> include/Makefile.am:9: whitespace following trailing backslash
>>> include/Makefile.am:11: whitespace following trailing backslash
>>> include/Makefile.am:14: whitespace following trailing backslash
>>> include/Makefile.am:20: whitespace following trailing backslash
>>> src/Makefile.am:10: Libtool library used but `LIBTOOL' is undefined
>>> src/Makefile.am:10:   The usual way to define `LIBTOOL' is to add
>>> `AC_PROG_LIBTOOL'
>>> src/Makefile.am:10:   to `configure.ac' and run `aclocal' and
>>> `autoconf' again.
>>> src/Makefile.am:10:   If `AC_PROG_LIBTOOL' is in `configure.ac', make
>>> sure
>>> src/Makefile.am:10:   its definition is in aclocal's search path.
>>> src/Makefile.am: installing `./depcomp'
>>> Makefile.am: installing `./INSTALL'
>>> Makefile.am: required file `./NEWS' not found
>>> Makefile.am: required file `./README' not found
>>> Makefile.am: required file `./AUTHORS' not found
>>> autoreconf: automake failed with exit status: 1
>>>
>>>
>>>   
>>>       
>> Formatting error caused by the sanitation process. An updated tarball is
>> attached to this email, with the following changes from the previous:
>>
>> 1) Fixed trailing white space in libsepol/src/Makefile.am
>> 2) Added --enable-swig option to libselinux and libsemanage to regenerate
>>   c-wrapper
>> 3) Added very very basic top level makefile, proof-of-concept to build
>> entire
>>   trunk. make -f Makefile.toolchain PREFIX=/some/where (Requested by
>> Joshua.B)
>> 4) Minor python related cleanups in configure.ac for libselinux and
>>   libsemanage
>>
>>     
>
> Dan, Manoj, Caleb, you can skip to the bottom of this email
>
> I have a branch with the latest patch, my current observations:
> * checkpolicy/test isn't built (this is included on fedora and must be built)
>   

That was done intentionally, I don't use it, so I didn't build it. That 
doesn't mean no one else can add 3 lines to a new Makefile.am.
> * when I generated the configure scripts in a fresh repo apparently it felt the need to relicense libselinux as gplv3 (added a COPYING file)
>   
If you do not provide a COPYING file, autoconf will generate one for 
you, so I suggest a COPYING, and any other dist file (eg, Changelog, 
NEWS, AUTHORS etc..) be made by hand and placed in the tree.
> * the CFLAGS we previously had have been completely removed, we need those back, particularly the warning ones and they aren't necessarily consistent:
> libsepol/src/Makefile:CFLAGS ?= -Werror -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute
> libsemanage/src/Makefile:CFLAGS ?= -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute -Wno-unused-parameter
> checkpolicy/Makefile:CFLAGS ?= -g -Wall -Werror -Wshadow -O2 -pipe -fno-strict-aliasing
> libselinux/src/Makefile:CFLAGS ?= -Werror -Wall -W -Wundef -Wshadow -Wmissing-noreturn -Wmissing-format-attribute
>
> Also, we filter out Werror from generated files (parser and lexer files, swig generated files) because they often cause warnings and we can't help it
>
>   
there is a libblah_la_CFLAGS = in each Makefile.am, you can tack those 
onto it. But i strongly suggest you only add the necessary, eg  -g, -O2 
etc.. are not mandatory, and should not be forced on the package 
builder. autoconf has support for such by adding them on the configure 
line,
eg ./configure CFLAGS="-Werror -pipe -Wall" --prefix=/blah etc..

> * make clean from the top doesn't actually clean anything, it just removes the makefiles
>
> * there is no way to distclean from the top level, eg., make the repository look like it will when we package up a release
>   
The top level makefile was just a proof-of-concept. If you need a top 
level makefile, then write one. The one i supplied was only intended to 
show that the tree builds.
> * Why doesn't configure automatically find python? the vast majority of selinux installations are going to require the python bits so it should default to on and I'm not sure what the use of the autotools infrastructure is if it can't find the resources we need automatically
>   
This is actually quite a complicated topic. Python is not very good at 
being used in a cross platform environment. And thus autoconf cannot be 
blamed for not being able to use python correctly in this manner. 
Autoconf depends on the "python" interpreter for information on which 
version and where python is installed. Now the problem with this is 
that, the build machine must have an identical python to your target 
machine, which is not possible on eg, x86_32 and ppc_64, as python 
generates arch dependent C-headers instead of #ifdef'ing the various 
architectures. There are more intricate problems that I could write a 
paper on, but i doubt you're interested. (eg Building on Linux for 
Windows, BSD, Solaris or vice versa, Or broken build systems like that 
of Redhat/Fedora)

To cut a long story short: No, you cannot build python wrappers outside 
your target _properly_ without explicitly telling the build system which 
python to use. To counter this, I have got autoconf to pick up the local 
python and suggest it to the user, this way autoconf doesn't guess and 
then get blamed for doing the "wrong thing". The user is the only one to 
blame if the package doesn't build.
> * configure is currently doing alot of tests for unnecessary things, such as a fortran compiler. This is one of the problems I have using autotools, it takes time and adds complexity for no reason
>   
I'll look into it, but a lot of it is done implicitly.
> * building an embedded libselinux (--disable-rpm --disable-avc --disable-bool) fails:
> 	/usr/bin/ld: .libs/libselinux_la-load_policy.o: relocation R_X86_64_PC32 against undefined symbol `security_get_boolean_names_internal' can not be used when making a shared object; recompile with -fPIC
>
>   
Yup, i'll look at that too.
> I think that is probably enough for now..
>
> --
>
> I created an autotools branch on the upstream git repo so all new patches should be based on that instead of a tarball. 
>
> Also, I'd like distributions to start testing this branch as they are less likely to be affected by the issues raised above. Distributions will likely be mostly affected by the change of python bindings from _*.so to libpy*.so, and also will need to run configure with --with-python=/usr/include/python2.5 to have a useful build
>   
yes, and --enable-swig too. As the pre-generated .c file will not be 
patched.


Working hard over the weekend hey method? give me svn write access to 
your autotools branch, this will speed up the initial ground work. I am 
moving onto other wild and wonderful things next week and would like to 
get this working for you guys as soon as possible.

Vikram.

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: New SELinux toolchain build system

[Joshua Brindle]

Vikram Ambrose wrote:
> Joshua Brindle wrote:
 
>> * make clean from the top doesn't actually clean anything, it just
>> removes the makefiles
>>
>> * there is no way to distclean from the top level, eg., make the
>> repository look like it will when we package up a release
>>   
> The top level makefile was just a proof-of-concept. If you need a top
> level makefile, then write one. The one i supplied was only intended to
> show that the tree builds.

So, the things you expect others to do include:
make the top level makefile useful (clean, distclean, debug build, etc)
fix the CFLAGS in all the app/lib makefiles
add the dispol/dismod makefile
add the ruby swig support
add make test targets to all lib Makefiles

Anything else?

I'm not adverse to the patch set, and when/if we ack and merge it, it will be our responsibility to maintain but I'm worried about it not being completed before that. I'm not an autotools expert (or even a beginner) and neither are the other maintainers. As such I don't see much motivation to finish up the patch set for you in order to get it in a mergeable state (and this isn't abnormal for opensource projects, try dropping an incomplete patch on lkml and see where it gets you).

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: New SELinux toolchain build system

[Vikram Ambrose]

Joshua Brindle wrote:
> Vikram Ambrose wrote:
>   
>> Joshua Brindle wrote:
>>     
>  
>   
>>> * make clean from the top doesn't actually clean anything, it just
>>> removes the makefiles
>>>
>>> * there is no way to distclean from the top level, eg., make the
>>> repository look like it will when we package up a release
>>>   
>>>       
>> The top level makefile was just a proof-of-concept. If you need a top
>> level makefile, then write one. The one i supplied was only intended to
>> show that the tree builds.
>>     
>
> So, the things you expect others to do include:
> make the top level makefile useful (clean, distclean, debug build, etc)
>   

I'll do that one if i get time this week.

> fix the CFLAGS in all the app/lib makefiles
>   

Like i said previously, do not force unnecessary CFLAGS onto the package 
builder. If you need special flags add it to your configure line, 
./configure CFLAGS="-Wall -I/blah -fPIC" --prefix=/usr --bindir=/bin 
etc...  A good place for these would be in the top level Makefile, which 
would be a hard coded makefile.

> add the dispol/dismod makefile
>   

I'm doing this one

> add the ruby swig support
> add make test targets to all lib Makefiles
>
>   

I'll try to do these this week, but I cannot test it, so be warned.

> Anything else?
>   

Nope.

> I'm not adverse to the patch set, and when/if we ack and merge it, it will be our responsibility to maintain but I'm worried about it not being completed before that. I'm not an autotools expert (or even a beginner) and neither are the other maintainers. As such I don't see much motivation to finish up the patch set for you in order to get it in a mergeable state (and this isn't abnormal for opensource projects, try dropping an incomplete patch on lkml and see where it gets you).
>   

Surprised, a little, so I'll try my best to help with anything I can, 
but time is not on my side.

Updates are in the mix, coming to a mailing list near you! :)


Vikram

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.