man-pages HEAD (eventually 6.8) build-system improvements

man-pages HEAD (eventually 6.8) build-system improvements

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 8464 bytes --]

Hi!

I'm excited to share list of improvements to our build system that I
think will be meaningful to downstram distributors:

-  I've reorganized the build-system files in a much more intuitive way,
   and the most direct benefit of this is that the build dependencies
   are now implicit in the file names; so much, that I've removed the
   documentation for it (which got obsolete very easily), and replaced
   the `make help` help with a script that will tell the dependencies:

	$ make help
	To see a list of targets, run:
		$ make nothing -p \
		| grep '^\.PHONY:' \
		| tr ' ' '\n' \
		| grep -v '^\.PHONY:' \
		| sort;

	To see a list of variables, run:
		$ find GNUmakefile share/mk/configure -type f \
		| sort \
		| xargs grep '^[^[:space:]].*=' \
		| sed 's/=.*/=/' \
		| grep -v -e ':DEFAULT_.*=' -e ':MAKEFILE_.*INCLUDED :=';

	To see a list of dependencies (package/program), run:
		$ find share/mk/configure/build-depends -type f \
		| sed 's,share/mk/configure/build-depends/,,' \
		| sed 's,\.mk,,' \
		| sort;

   Since I needed to make a choice, the build dependencies are expressed
   in terms of Debian packages, and the names of the binaries in Debian.
   You'll need to translate these to your distro.  On Debian, you could
   script the installation of dependencies:

	$ find share/mk/configure/build-depends -type f \
	| sed 's,share/mk/configure/build-depends/,,' \
	| sed 's,\.mk,,' \
	| sed 's,/.*,,' \
	| grep -v checkpatch \
	| xargs apt-get install;

   (This already proved useful when discussing about an issue with a
   downstream packager, so I could list some commands to reproduce some
   behavior in a clean installation of Debian (in a Docker container).)

   (checkpatch is the script checkpatch.pl from the Linux kernel
   sources, which I have a package for, but need to polish it for
   public distribution.)

   As you can see from that `make help`, variables exposed to users for
   configuration are also more intuitively placed, within a ./configure/
   subdirectory of the build system.  Anything not in that subdir is
   just implementation details, not intended for users to tweak them.

   I've also made target names more consistent (for example,
   'build-book' is now 'build-pdf-book', and is triggered by the parent
   'build-pdf').  In general, -* subtargets are always triggered by the
   target named by a prefix of it.  Hopefully, that's also intuitive
   enough that need not be documented (and of course, you can see it's
   now another script in `make help`).  This differs from autotools
   behavior, which is to only build as much as your current system
   --with its installed dependencies-- can run; I believe that behavior
   is bogus, as different people running the same make(1) target on
   different systems, will see different behavior.  IMO, the right thing
   to do is to behave the same everywhere, and fail hard when something
   can't be done due to a missing dependency.  With subtargets, we do
   allow building only partially, if some system doesn't want to run or
   build some stuff, but users need to explicitly specify so.

-  We stamp the date and version in the manual pages at `make install`
   time.  This means now the (unreleased) and (date) placeholders will
   be in the repository, but when one installs from source with
   `make install`, the pages will be marked with the date of the last
   commit that modified a page, and the version extracted from
   git-describe(1).

   This should help fix the manual pages at Michael's <man7.org>, which
   now have the (unreleased) and (date) placeholders.  I'm not 100% if
   he's running `make install` or if he is just copying the pages from
   the repository; I hope he is doing the former, in which case they'll
   get the stamp when 6.8 is released and he gets that version.

-  The version is not stamped on the pages distributed in the release
   tarballs anymore; we now only stamp the page date.  This will allow
   distributions to stamp their own versions, such as 6.8-1, by using
   the $EXTRAVERSION variable.  So, distributions will be able to run

	$ make install EXTRAVERSION=-1 prefix=/usr DESTDIR=tmpdir

   And get their version suffix stamped after the upstream version.

-  The PDF book is now stable enough, and I decided to add an install
   target for it:

	$ make install-pdf-book

   This means distros can start installing the PDF book in their
   systems if they want, to please those who want to read the manual
   typeset, without having to download it from the website.

	"The manual was intended to be typeset; some detail is sacrificed
	on terminals."  (man(1), _Unix Time-Sharing System Programmer's
	Manual_, Eighth Edition, Volume 1, February 1985)

   The script for producing the book was contributed by Deri James.

-  The build system works on other projects (this was already possible,
   but limited to just some features).  I've been using it to produce
   PDF books of the manual pages in the shadow project, and also forked
   it to write a build system for the liba2i library (of recent
   creation)[1] with minimal changes.  That helped find the assumptions
   made that depended on our project, and changed them to make them more
   generic.

   [1]  <https://git.kernel.org/pub/scm/libs/liba2i/liba2i.git/>

-  Already in 6.7, but noteworthy.  The build system now has a list of
   lints and checks known to fail, and doesn't run them by default.
   This allows downstream packagers run `make lint build check` without
   having to make exceptions.  Any errors are now regressions, and we
   should be careful to not introduce them.  With time, I'll try to
   remove the internal exceptions, although some aren't easily fixable.

   It is also easier on contributors, which now can just
   `make lint build check` after their patches, and expect it to not
   give any errors (else, they screwed it).

-  Already in 6.7, but noteworthy.  We have a 'distcheck' target, which
   does the usual stuff described by GNU autotools' 'distcheck', but it
   is better, as we have a GNUmakefile-based build system, and can
   express dependencies better.  We run in parallel as much as is
   possible, and don't need to do any read-only magic stuff, since we
   always run out-of-tree builds, contained in <.tmp/>.

   It doesn't do 'installcheck' (we don't even have it; what would we
   actually test?), though, but I don't think that's sensible, since an
   install check should run mandb(8) to actually have a proper install,
   but I think that would be too intrusive, because that's
   system-dependent, and I believe users should decide to run mandb(8)
   manually after `make install` (if they use man-db at all!), and it's
   not our business (consider users installing into /opt and not wanting
   to actually modify their systems).

   It goes a step beyond GNU's 'distcheck': we run `make dist` from
   within the extracted tarball, just like autotools, but then we make
   sure that the second tarball is identical byte-per-byte to the first
   one, by running diffoscope(1) to diff both tarballs.

Please comment any doubts you have about these features.

Have a lovely day!
Alex


P.S.:  Here's the list of build dependencies, as of $now:

$ find share/mk/configure/build-depends -type f \
        | sed 's,share/mk/configure/build-depends/,,' \
        | sed 's,\.mk,,' \
        | sort;
binutils/ld
bsdextrautils/col
bzip2/bzip2
checkpatch/checkpatch
clang-tidy/clang-tidy
clang/clang
coreutils/cat
coreutils/cp
coreutils/echo
coreutils/expr
coreutils/head
coreutils/install
coreutils/ln
coreutils/mkdir
coreutils/realpath
coreutils/rm
coreutils/sort
coreutils/stat
coreutils/tac
coreutils/tail
coreutils/test
coreutils/touch
coreutils/true
cpp/cpp
cppcheck/cppcheck
cpplint/cpplint
diffoscope/diffoscope
findutils/find
findutils/xargs
gcc/cc
git/git
grep/grep
groff-base/eqn
groff-base/grops
groff-base/grotty
groff-base/nroff
groff-base/pic
groff-base/preconv
groff-base/tbl
groff-base/troff
groff/gropdf
groff/post-grohtml
gzip/gzip
iwyu/iwyu
libc-bin/locale
lzip/lzip
man/man
mandoc/mandoc
moreutils/sponge
pkgconf/pkgconf
sed/sed
tar/tar
xz-utils/xz

-- 
<https://www.alejandro-colomar.es/>
Looking for a remote C programming job at the moment.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]