[PATCH 0 of 2 v3] improve checking for documentation tools and formatting

[PATCH 0 of 2 v3] improve checking for documentation tools and formatting

[Matt Wilson]

This version addresses feedback from v2. Please let me know if there
are any other concerns.

Matt

[PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Matt Wilson]

It is sometimes hard to discover all the optional tools that should be
on a system to build all available Xen documentation. By checking for
documentation generation tools at ./configure time and displaying a
warning, Xen packagers will more easily learn about new optional build
dependencies, like markdown, when they are introduced.

Signed-off-by: Matt Wilson <msw@amazon.com>

---
Changes since v1:
 * require that ./configure be run before building docs
 * remove Docs.mk and make Tools.mk the canonical locaiton where
   docs tools are defined (via ./configure)
 * fold in checking for markdown_py

Changes since v2:
 * drop the AX_DOCS_TOOL_PROG variant, just use AX_DOCS_TOOL_PROGS
 * use the lowercase version of the autoconf variable name for the
   help text

diff -r d7e4efa17fb0 -r 3b3582f4dc42 README
--- a/README	Tue Aug 28 15:35:08 2012 -0700
+++ b/README	Thu Sep 06 21:31:51 2012 -0700
@@ -28,8 +28,9 @@
 your system. For full documentation, see the Xen User Manual. If this
 is a pre-built release then you can find the manual at:
  dist/install/usr/share/doc/xen/pdf/user.pdf
-If you have a source release, then 'make -C docs' will build the
-manual at docs/pdf/user.pdf.
+If you have a source release and the required documentation generation
+tools, then './configure; make -C docs' will build the manual at
+docs/pdf/user.pdf.
 
 Quick-Start Guide
 =================
@@ -59,7 +60,6 @@
     * GNU gettext
     * 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc debs)
     * ACPI ASL compiler (iasl)
-    * markdown
 
 In addition to the above there are a number of optional build
 prerequisites. Omitting these will cause the related features to be
@@ -67,6 +67,7 @@
     * Development install of Ocaml (e.g. ocaml-nox and
       ocaml-findlib). Required to build ocaml components which
       includes the alternative ocaml xenstored.
+    * markdown
 
 Second, you need to acquire a suitable kernel for use in domain 0. If
 possible you should use a kernel provided by your OS distributor. If
diff -r d7e4efa17fb0 -r 3b3582f4dc42 config/Tools.mk.in
--- a/config/Tools.mk.in	Tue Aug 28 15:35:08 2012 -0700
+++ b/config/Tools.mk.in	Thu Sep 06 21:31:51 2012 -0700
@@ -22,6 +22,18 @@
 LD86                := @LD86@
 BCC                 := @BCC@
 IASL                := @IASL@
+PS2PDF              := @PS2PDF@
+DVIPS               := @DVIPS@
+LATEX               := @LATEX@
+FIG2DEV             := @FIG2DEV@
+LATEX2HTML          := @LATEX2HTML@
+DOXYGEN             := @DOXYGEN@
+POD2MAN             := @POD2MAN@
+POD2HTML            := @POD2HTML@
+POD2TEXT            := @POD2TEXT@
+DOT                 := @DOT@
+NEATO               := @NEATO@
+MARKDOWN            := @MARKDOWN@
 
 # Extra folder for libs/includes
 PREPEND_INCLUDES    := @PREPEND_INCLUDES@
diff -r d7e4efa17fb0 -r 3b3582f4dc42 docs/Makefile
--- a/docs/Makefile	Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/Makefile	Thu Sep 06 21:31:51 2012 -0700
@@ -2,7 +2,7 @@
 
 XEN_ROOT=$(CURDIR)/..
 include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk
 
 VERSION		= xen-unstable
 
@@ -26,10 +26,12 @@
 
 .PHONY: build
 build: html txt man-pages
-	@if which $(DOT) 1>/dev/null 2>/dev/null ; then              \
-	$(MAKE) -C xen-api build ; else                              \
-        echo "Graphviz (dot) not installed; skipping xen-api." ; fi
+ifdef DOT
+	$(MAKE) -C xen-api build
 	rm -f *.aux *.dvi *.bbl *.blg *.glo *.idx *.ilg *.log *.ind *.toc
+else
+	@echo "Graphviz (dot) not installed; skipping xen-api."
+endif
 
 .PHONY: dev-docs
 dev-docs: python-dev-docs
@@ -42,18 +44,21 @@
 
 .PHONY: python-dev-docs
 python-dev-docs:
-	@mkdir -v -p api/tools/python
-	@set -e ; if which $(DOXYGEN) 1>/dev/null 2>/dev/null; then \
-        echo "Running doxygen to generate Python tools APIs ... "; \
-	$(DOXYGEN) Doxyfile;                                       \
-	$(MAKE) -C api/tools/python/latex ; else                   \
-        echo "Doxygen not installed; skipping python-dev-docs."; fi
+ifdef DOXYGEN
+	@echo "Running doxygen to generate Python tools APIs ... "
+	mkdir -v -p api/tools/python
+	$(DOXYGEN) Doxyfile && $(MAKE) -C api/tools/python/latex
+else
+	@echo "Doxygen not installed; skipping python-dev-docs."
+endif
 
 .PHONY: man-pages
 man-pages:
-	@if which $(POD2MAN) 1>/dev/null 2>/dev/null; then \
-	$(MAKE) $(DOC_MAN1) $(DOC_MAN5); else              \
-	echo "pod2man not installed; skipping man-pages."; fi
+ifdef POD2MAN
+	$(MAKE) $(DOC_MAN1) $(DOC_MAN5)
+else
+	@echo "pod2man not installed; skipping man-pages."
+endif
 
 man1/%.1: man/%.pod.1 Makefile
 	$(INSTALL_DIR) $(@D)
@@ -94,30 +99,40 @@
 	perl -w -- ./gen-html-index -i INDEX html $(DOC_HTML)
 
 html/%.html: %.markdown
-	@$(INSTALL_DIR) $(@D)
-	@set -e ; if which $(MARKDOWN) 1>/dev/null 2>/dev/null; then \
-	echo "Running markdown to generate $*.html ... "; \
+	$(INSTALL_DIR) $(@D)
+ifdef MARKDOWN
+	@echo "Running markdown to generate $*.html ... "
 	$(MARKDOWN) $< > $@.tmp ; \
-	$(call move-if-changed,$@.tmp,$@) ; else \
-	echo "markdown not installed; skipping $*.html."; fi
+	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "markdown not installed; skipping $*.html."
+endif
 
 html/%.txt: %.txt
-	@$(INSTALL_DIR) $(@D)
+	$(INSTALL_DIR) $(@D)
 	cp $< $@
 
 html/man/%.1.html: man/%.pod.1 Makefile
 	$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
 	$(POD2HTML) --infile=$< --outfile=$@.tmp
 	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "pod2html not installed; skipping $<."
+endif
 
 html/man/%.5.html: man/%.pod.5 Makefile
 	$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
 	$(POD2HTML) --infile=$< --outfile=$@.tmp
 	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "pod2html not installed; skipping $<."
+endif
 
 html/hypercall/index.html: ./xen-headers
 	rm -rf $(@D)
-	@$(INSTALL_DIR) $(@D)
+	$(INSTALL_DIR) $(@D)
 	./xen-headers -O $(@D) \
 		-T 'arch-x86_64 - Xen public headers' \
 		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 -X arch-arm \
@@ -137,11 +152,24 @@
 
 txt/man/%.1.txt: man/%.pod.1 Makefile
 	$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
 	$(POD2TEXT) $< $@.tmp
 	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "pod2text not installed; skipping $<."
+endif
 
 txt/man/%.5.txt: man/%.pod.5 Makefile
 	$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
 	$(POD2TEXT) $< $@.tmp
 	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "pod2text not installed; skipping $<."
+endif
 
+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+	$(error You have to run ./configure before building docs)
+endif
+
diff -r d7e4efa17fb0 -r 3b3582f4dc42 docs/xen-api/Makefile
--- a/docs/xen-api/Makefile	Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/xen-api/Makefile	Thu Sep 06 21:31:51 2012 -0700
@@ -2,7 +2,7 @@
 
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk
 
 
 TEX := $(wildcard *.tex)
@@ -42,3 +42,8 @@
 .PHONY: clean
 clean:
 	rm -f *.pdf *.ps *.dvi *.aux *.log *.out $(EPSDOT)
+
+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+	$(error You have to run ./configure before building docs)
+endif
diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/configure.ac
--- a/tools/configure.ac	Tue Aug 28 15:35:08 2012 -0700
+++ b/tools/configure.ac	Thu Sep 06 21:31:51 2012 -0700
@@ -34,6 +34,7 @@
 m4_include([m4/curses.m4])
 m4_include([m4/pthread.m4])
 m4_include([m4/ptyfuncs.m4])
+m4_include([m4/docs_tool.m4])
 
 # Enable/disable options
 AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP])
@@ -80,6 +81,18 @@
 AC_PATH_PROG([BISON], [bison])
 AC_PATH_PROG([FLEX], [flex])
 AX_PATH_PROG_OR_FAIL([PERL], [perl])
+AX_DOCS_TOOL_PROGS([PS2PDF], [ps2pdf])
+AX_DOCS_TOOL_PROGS([DVIPS], [dvips])
+AX_DOCS_TOOL_PROGS([LATEX], [latex])
+AX_DOCS_TOOL_PROGS([FIG2DEV], [fig2dev])
+AX_DOCS_TOOL_PROGS([LATEX2HTML], [latex2html])
+AX_DOCS_TOOL_PROGS([DOXYGEN], [doxygen])
+AX_DOCS_TOOL_PROGS([POD2MAN], [pod2man])
+AX_DOCS_TOOL_PROGS([POD2HTML], [pod2html])
+AX_DOCS_TOOL_PROGS([POD2TEXT], [pod2text])
+AX_DOCS_TOOL_PROGS([DOT], [dot])
+AX_DOCS_TOOL_PROGS([NEATO], [neato])
+AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py])
 AS_IF([test "x$xapi" = "xy"], [
     AX_PATH_PROG_OR_FAIL([CURL], [curl-config])
     AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/m4/docs_tool.m4	Thu Sep 06 21:31:51 2012 -0700
@@ -0,0 +1,10 @@
+# $1: autoconf variable name
+# $2: list of programs to check for
+AC_DEFUN([AX_DOCS_TOOL_PROGS], [
+dnl
+    AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
+    AC_PATH_PROGS([$1], [$2])
+    AS_IF([! test -x "$ac_cv_path_$1"], [
+        AC_MSG_WARN([$2 is not available so some documentation won't be built])
+    ])
+])

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Ian Campbell]

> diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/tools/m4/docs_tool.m4	Thu Sep 06 21:31:51 2012 -0700
> @@ -0,0 +1,10 @@
> +# $1: autoconf variable name
> +# $2: list of programs to check for
> +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> +dnl
> +    AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> +    AC_PATH_PROGS([$1], [$2])
> +    AS_IF([! test -x "$ac_cv_path_$1"], [
> +        AC_MSG_WARN([$2 is not available so some documentation won't be built])

Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
list of potential command names this will look a bit odd I think?

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Matt Wilson]

On Mon, Sep 10, 2012 at 05:09:10PM +0100, Ian Campbell wrote:
> 
> > diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> > --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> > +++ b/tools/m4/docs_tool.m4	Thu Sep 06 21:31:51 2012 -0700
> > @@ -0,0 +1,10 @@
> > +# $1: autoconf variable name
> > +# $2: list of programs to check for
> > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> > +dnl
> > +    AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> > +    AC_PATH_PROGS([$1], [$2])
> > +    AS_IF([! test -x "$ac_cv_path_$1"], [
> > +        AC_MSG_WARN([$2 is not available so some documentation won't be built])
> 
> Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
> list of potential command names this will look a bit odd I think?

Yea, that could be better. Though I kind of like that it shows the
programs that were checked. It'd look like:
 
 markdown markdown_py is not available so some documentation won't be built

Maybe just drop 'is' to avoid the verb mismatch.

Matt

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Ian Campbell]

On Mon, 2012-09-10 at 21:46 +0100, Matt Wilson wrote:
> On Mon, Sep 10, 2012 at 05:09:10PM +0100, Ian Campbell wrote:
> > 
> > > diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> > > --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> > > +++ b/tools/m4/docs_tool.m4	Thu Sep 06 21:31:51 2012 -0700
> > > @@ -0,0 +1,10 @@
> > > +# $1: autoconf variable name
> > > +# $2: list of programs to check for
> > > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> > > +dnl
> > > +    AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> > > +    AC_PATH_PROGS([$1], [$2])
> > > +    AS_IF([! test -x "$ac_cv_path_$1"], [
> > > +        AC_MSG_WARN([$2 is not available so some documentation won't be built])
> > 
> > Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
> > list of potential command names this will look a bit odd I think?
> 
> Yea, that could be better. Though I kind of like that it shows the
> programs that were checked. It'd look like:
>  
>  markdown markdown_py is not available so some documentation won't be built
> 
> Maybe just drop 'is' to avoid the verb mismatch.

Or "m4_tolower($1) ($2) is not available ..."? (properly m4 quoted by
someone who knows what they are doing, of course ;-))

Ian.

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Matt Wilson]

On Tue, Sep 11, 2012 at 08:54:10AM +0100, Ian Campbell wrote:
> > 
> > Yea, that could be better. Though I kind of like that it shows the
> > programs that were checked. It'd look like:
> >  
> >  markdown markdown_py is not available so some documentation won't be built
> > 
> > Maybe just drop 'is' to avoid the verb mismatch.
> 
> Or "m4_tolower($1) ($2) is not available ..."? (properly m4 quoted by
> someone who knows what they are doing, of course ;-))

Right, so the desired output would be: 
  markdown (markdown markdown_py) is not available so some documentation won't be built

I can work on that, but it'll probably be a few days before I can get
back to it.

Matt

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Ian Campbell]

On Fri, 2012-09-07 at 05:35 +0100, Matt Wilson wrote:
> It is sometimes hard to discover all the optional tools that should be
> on a system to build all available Xen documentation. By checking for
> documentation generation tools at ./configure time and displaying a
> warning, Xen packagers will more easily learn about new optional build
> dependencies, like markdown, when they are introduced.
> 
> Signed-off-by: Matt Wilson <msw@amazon.com>

While investigating an (unrelated) issue with the cronjob that generates
http://xenbits.xen.org/docs/unstable/ I had a sudden though about a
downside of this approach.

Trying to run ./configure on xenbits results in:
        $ ./configure 
        checking build system type... x86_64-unknown-linux-gnu
        checking host system type... x86_64-unknown-linux-gnu
        checking for gcc... no
        checking for cc... no
        checking for cl.exe... no
        configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
        configure: error: no acceptable C compiler found in $PATH
        See `config.log' for more details

Which is a bit annoying if all you wanted was to build docs. I'm not
sure I really want to advocate putting gcc on xenbits. I suppose I could
put a dummy shell script in $PATH.

AIUI autoconf allows you to define that a configure script calls a
sub-configure script. Could we use this at the toplevel to call both
tools/configure and a new docs/configure? Then my docs scripts could
just call docs/configure?

FWIW the unrelated issue was:
        $ make -C docs/ clean
        /bin/sh: gcc: not found
        /bin/sh: arithmetic expression: expecting primary: ""
which I think is down to the gcc version check in Config.mk.

Ian.

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Matt Wilson]

On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
>         configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
>         configure: error: no acceptable C compiler found in $PATH
>         See `config.log' for more details
> 
> Which is a bit annoying if all you wanted was to build docs. I'm not
> sure I really want to advocate putting gcc on xenbits. I suppose I could
> put a dummy shell script in $PATH.

./configure is probably going to need a compiler that produces
executables.

> AIUI autoconf allows you to define that a configure script calls a
> sub-configure script. Could we use this at the toplevel to call both
> tools/configure and a new docs/configure? Then my docs scripts could
> just call docs/configure?

This sounds like a better idea.

> FWIW the unrelated issue was:
>         $ make -C docs/ clean
>         /bin/sh: gcc: not found
>         /bin/sh: arithmetic expression: expecting primary: ""
> which I think is down to the gcc version check in Config.mk.

While we're discussing expanding the coverage of ./configure, I
wonder: what (if any) is the resistance to requiring ./configure to
build the xen/ tree?

Matt

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Ian Campbell]

On Mon, 2012-09-17 at 18:08 +0100, Matt Wilson wrote:
> On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
> >         configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
> >         configure: error: no acceptable C compiler found in $PATH
> >         See `config.log' for more details
> > 
> > Which is a bit annoying if all you wanted was to build docs. I'm not
> > sure I really want to advocate putting gcc on xenbits. I suppose I could
> > put a dummy shell script in $PATH.
> 
> ./configure is probably going to need a compiler that produces
> executables.

True.

> 
> > AIUI autoconf allows you to define that a configure script calls a
> > sub-configure script. Could we use this at the toplevel to call both
> > tools/configure and a new docs/configure? Then my docs scripts could
> > just call docs/configure?
> 
> This sounds like a better idea.
> 
> > FWIW the unrelated issue was:
> >         $ make -C docs/ clean
> >         /bin/sh: gcc: not found
> >         /bin/sh: arithmetic expression: expecting primary: ""
> > which I think is down to the gcc version check in Config.mk.
> 
> While we're discussing expanding the coverage of ./configure, I
> wonder: what (if any) is the resistance to requiring ./configure to
> build the xen/ tree?

The hypervisor maintainers were pretty much unanimously against it.

There's some discussion in the archives around the time autoconf was
added and even a patch (applied and now reverted IIRC) to do as you
suggest.

Ian.

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Ian Jackson]

Matt Wilson writes ("Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time"):
> On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
> >         configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
> >         configure: error: no acceptable C compiler found in $PATH
> >         See `config.log' for more details
> > 
> > Which is a bit annoying if all you wanted was to build docs. I'm not
> > sure I really want to advocate putting gcc on xenbits. I suppose I could
> > put a dummy shell script in $PATH.
> 
> ./configure is probably going to need a compiler that produces
> executables.

Not necessarily.  If we don't ask for AC_CC or whatever it is, it
shouldn't.

Ian.

Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time

[Tim Deegan]

At 10:08 -0700 on 17 Sep (1347876501), Matt Wilson wrote:
> While we're discussing expanding the coverage of ./configure, I
> wonder: what (if any) is the resistance to requiring ./configure to
> build the xen/ tree?

There's nothing in the hypervisor build that needs to be adjusted to
match the run-time userspace.  So if you're a hypervisor maintainer it
just introduces all the charming idiosyncracies of autotools without
providing any benefit.  E.g., this:

 checking for yajl_alloc in -lyajl... no
 configure: error: Could not find yajl

should not stop me building Xen.  Also, as I said before, autoconf is
the thin end of a slippery wedge: the answer to any autotools problem is
always more autotools, and I choose to make my stand here rather than
have the argument later about why we shouldn't be using libtool. :)

Tim.

[PATCH 2 of 2 v3] docs: use elinks to format markdown-generated html to text

[Matt Wilson]

Markdown, while easy to read and write, isn't the most consumable
format for users reading documentation on a terminal. This patch uses
elinks (by default) to format markdown produced HTML into text files.

Signed-off-by: Matt Wilson <msw@amazon.com>

---
Changes since v3:
 * check for html to text dump tool in ./configure
 * switch to using elinks
 * allow command line flags to dump tool to be specified

Changes since v4:
 * none, just adjusted the commit message

diff -r 3b3582f4dc42 -r 6a1ee7eacd9c config/Tools.mk.in
--- a/config/Tools.mk.in	Thu Sep 06 21:31:51 2012 -0700
+++ b/config/Tools.mk.in	Thu Sep 06 21:33:05 2012 -0700
@@ -34,6 +34,8 @@
 DOT                 := @DOT@
 NEATO               := @NEATO@
 MARKDOWN            := @MARKDOWN@
+HTMLDUMP            := @HTMLDUMP@
+HTMLDUMPFLAGS       := @HTMLDUMPFLAGS@
 
 # Extra folder for libs/includes
 PREPEND_INCLUDES    := @PREPEND_INCLUDES@
diff -r 3b3582f4dc42 -r 6a1ee7eacd9c docs/Makefile
--- a/docs/Makefile	Thu Sep 06 21:31:51 2012 -0700
+++ b/docs/Makefile	Thu Sep 06 21:33:05 2012 -0700
@@ -146,9 +146,20 @@
 	$(call move-if-changed,$@.tmp,$@)
 
 txt/%.txt: %.markdown
-	$(INSTALL_DIR) $(@D)
-	cp $< $@.tmp
+	@$(INSTALL_DIR) $(@D)
+ifdef MARKDOWN
+ifdef HTMLDUMP
+	@echo "Running markdown to generate $*.txt ... "; \
+	$(MARKDOWN) $< | $(HTMLDUMP) $(HTMLDUMPFLAGS) > $@.tmp
 	$(call move-if-changed,$@.tmp,$@)
+else
+	@echo "html dump tool (like elinks) not installed; just copying $<." \;
+	cp $< $@;
+endif
+else
+	@echo "markdown not installed; just copying $<." \;
+	cp $< $@;
+endif
 
 txt/man/%.1.txt: man/%.pod.1 Makefile
 	$(INSTALL_DIR) $(@D)
diff -r 3b3582f4dc42 -r 6a1ee7eacd9c tools/configure.ac
--- a/tools/configure.ac	Thu Sep 06 21:31:51 2012 -0700
+++ b/tools/configure.ac	Thu Sep 06 21:33:05 2012 -0700
@@ -93,6 +93,15 @@
 AX_DOCS_TOOL_PROGS([DOT], [dot])
 AX_DOCS_TOOL_PROGS([NEATO], [neato])
 AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py])
+AC_ARG_VAR([HTMLDUMP],
+           [Path to html-to-text generation tool (default: elinks)])
+AC_PATH_PROG([HTMLDUMP], [elinks])
+AS_IF([! test -x "$ac_cv_path_HTMLDUMP"], [
+    AC_MSG_WARN([$ac_cv_path_HTMLDUMP is not available so text documentation will be unformatted markdown])
+])
+AC_SUBST([HTMLDUMPFLAGS], ["-dump"])
+AC_ARG_VAR([HTMLDUMPFLAGS], [Flags passed to html to text translation tool])
+
 AS_IF([test "x$xapi" = "xy"], [
     AX_PATH_PROG_OR_FAIL([CURL], [curl-config])
     AX_PATH_PROG_OR_FAIL([XML], [xml2-config])

Re: [PATCH 0 of 2 v3] improve checking for documentation tools and formatting

[Ian Campbell]

On Fri, 2012-09-07 at 05:35 +0100, Matt Wilson wrote:
> This version addresses feedback from v2. Please let me know if there
> are any other concerns.

In general I think I'm happy to take docs updates right up to the
release, but in this case since it also involves changes to autoconf and
the build system I think is now 4.3 material.

(I've not actually looked at the patches yet, just the diffstat)

Ian.

Re: [PATCH 0 of 2 v3] improve checking for documentation tools and formatting

[Matt Wilson]

On Fri, Sep 07, 2012 at 09:51:46AM +0100, Ian Campbell wrote:
> On Fri, 2012-09-07 at 05:35 +0100, Matt Wilson wrote:
> > This version addresses feedback from v2. Please let me know if there
> > are any other concerns.
> 
> In general I think I'm happy to take docs updates right up to the
> release, but in this case since it also involves changes to autoconf and
> the build system I think is now 4.3 material.

Well, I provided a patch that produced formatted text from markdown
that didn't change autoconf and the build system all that much, but
IanJ asked:
> Firstly, why lynx and not w3m ?  Is lynx even maintained upstream
> any more ?  Secondly, shouldn't we do the testing for the presence
> of markdown and the html formatter in configure ?
> Ian.

I think that there's very little risk in these changes, even at this
late stage for 4.2.0.

Matt

> (I've not actually looked at the patches yet, just the diffstat)
> 
> Ian.

Re: [PATCH 0 of 2 v3] improve checking for documentation tools and formatting

[Ian Campbell]

On Fri, 2012-09-07 at 21:28 +0100, Matt Wilson wrote:
> On Fri, Sep 07, 2012 at 09:51:46AM +0100, Ian Campbell wrote:
> > On Fri, 2012-09-07 at 05:35 +0100, Matt Wilson wrote:
> > > This version addresses feedback from v2. Please let me know if there
> > > are any other concerns.
> > 
> > In general I think I'm happy to take docs updates right up to the
> > release, but in this case since it also involves changes to autoconf and
> > the build system I think is now 4.3 material.
> 
> Well, I provided a patch that produced formatted text from markdown
> that didn't change autoconf and the build system all that much, but
> IanJ asked:
> > Firstly, why lynx and not w3m ?  Is lynx even maintained upstream
> > any more ?  Secondly, shouldn't we do the testing for the presence
> > of markdown and the html formatter in configure ?

I know this, and I think this patch is the right approach, it's just
unfortunately missed the 4.2.0 boat.

> I think that there's very little risk in these changes, even at this
> late stage for 4.2.0.

I think autoconf and build system changes are 4.3 material (and a
candidate for 4.2.1) at this point. Changing these has had unexpected
consequences numerous times.

Ian.