[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas De Schampheleire]

This patch series makes some improvements to the generation of the text
version of the manual:
- make sure that URLs are visible
- remove the 'logo' (logo.png) reference

To achieve this, three preparatory patches are needed.

Note that this series stands on its own, it can already be applied.
A subsequent series will make structural and content-wise changes to the
manual itself.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v4: fix 'make clean' (detected by Samuel)
v3: fix 'make manual' without .config (extra patch) (detected by Yann)
v2: refine patch 4 (changing URLs)

Thomas De Schampheleire (5)
  Makefile: unconditionally include pkg-utils.mk
  gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
  gendoc infra: add support for asciidoc configuration files
  manual-text: make sure URLs are displayed if a link text was provided
  manual-text: hide image representations

 Makefile                       |   2 ++
 docs/manual/asciidoc-text.conf |  23 +++++++++++++++++++++++
 docs/manual/manual.mk          |  28 +++++++++++++++++-----------
 package/Makefile.in            |   1 -
 4 files changed, 42 insertions(+), 12 deletions(-)

[Buildroot] [PATCH 1 of 5 v4 for 2014.08] Makefile: unconditionally include pkg-utils.mk

[Thomas De Schampheleire]

Currently, pkg-utils.mk (included via package/Makefile.in) is only included
when a configuration file already exists. This means that none of the
utilities it defines are available without .config.

In particular:
- the MESSAGE macro, causing pretty build output. Since some make targets
  can be run even without .config, like 'make manual', not having this
  pretty printing is odd.

- pkgname, pkgdir: in a subsequent patch, these functions will be used for
  the generation of the manual, and since this should work also without
  .config, we need these functions to be available.

This patch moves the include of pkg-utils.mk from package/Makefile.in to
Makefile, outside of the check for .config.

This is a quick fix. The full solution involves to minimize the amount of
Makefile code that is guarded by a check on .config. This approach will be
taken in the 2014.11 release cycle.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>

---
v4: no changes
v3: new patch, fixing the issue detected by Yann that 'make manual' no
longer worked without .config

 Makefile            |  2 ++
 package/Makefile.in |  1 -
 2 files changed, 2 insertions(+), 1 deletions(-)

diff -r 3bcf63a46bc1 -r 232206819ca1 Makefile
--- a/Makefile	Tue Aug 05 18:25:11 2014 -0300
+++ b/Makefile	Wed Aug 13 11:30:25 2014 +0200
@@ -271,6 +271,8 @@
 # Causes breakage with packages that needs host-ruby
 unexport RUBYOPT
 
+include package/pkg-utils.mk
+
 ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
 ################################################################################
diff -r 3bcf63a46bc1 -r 232206819ca1 package/Makefile.in
--- a/package/Makefile.in	Tue Aug 05 18:25:11 2014 -0300
+++ b/package/Makefile.in	Wed Aug 13 11:30:25 2014 +0200
@@ -385,7 +385,6 @@
 SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared
 endif
 
-include package/pkg-utils.mk
 include package/pkg-download.mk
 include package/pkg-autotools.mk
 include package/pkg-cmake.mk

[Buildroot] [PATCH 2 of 5 v4 for 2014.08] gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

[Thomas De Schampheleire]

In the gendoc infrastructure, using an assignment of the form
    FOO = docs/$(1)/bar
inside GENDOC_INNER does not work as expected: the $(1) value is empty here
and the value of FOO becomes 'docs//bar'.

Parameters $(2), $(3), etc. do not have this problem. The specific thing
about $(1) is that it is a parameter to GENDOC itself (indicating the
document to create) and passed transparently to GENDOC_INNER.

This is different from the package infrastructures, where $(1) is set from
$(pkgname). In fact, the same strategy could be used by the gendoc
infrastructure as well, as $(pkgname) resolves to 'manual' for file
docs/manual/manual.mk. This has the advantage that the described problem
does not occur.

Note that this means that if we want to use the same GENDOC infrastructure
for another document, it will have to reside in a separate directory than
the manual.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
v3: no changes
v2: no changes

 docs/manual/manual.mk |  20 ++++++++++----------
 1 files changed, 10 insertions(+), 10 deletions(-)

diff -r 232206819ca1 -r 9d9af165df7b docs/manual/manual.mk
--- a/docs/manual/manual.mk	Wed Aug 13 11:30:25 2014 +0200
+++ b/docs/manual/manual.mk	Sat Aug 09 18:18:58 2014 +0200
@@ -78,16 +78,16 @@
 # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies.
 ################################################################################
 define GENDOC
-$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
-$(call GENDOC_INNER,$(1),text,text,text,text)
-$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB)
-clean: $(1)-clean
-$(1)-clean:
-	$$(Q)$$(RM) -rf $$(O)/docs/$(1)
-.PHONY: $(1) $(1)-clean manual-update-lists
+$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
+$(call GENDOC_INNER,$(pkgname),text,text,text,text)
+$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB)
+clean: $(pkgname)-clean
+$(pkgname)-clean:
+	$$(Q)$$(RM) -rf $$(O)/docs/$(pkgname)
+.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists
 endef
 
 MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
-$(eval $(call GENDOC,manual))
+$(eval $(call GENDOC))

[Buildroot] [PATCH 3 of 5 v4 for 2014.08] gendoc infra: add support for asciidoc configuration files

[Thomas De Schampheleire]

This patch introduces support for asciidoc configuration files, specific for
each output format (html, text, pdf, ...). This is useful to make certain
tweaks in the document generation.

If a file docs/manual/asciidoc-<format>.conf is present, it is passed to
asciidoc as configuration file. If no file for the current format is
present, the options passed to asciidoc are empty.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/manual.mk |  8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff -r 9d9af165df7b -r 81d40b841073 docs/manual/manual.mk
--- a/docs/manual/manual.mk	Sat Aug 09 18:18:58 2014 +0200
+++ b/docs/manual/manual.mk	Sat Aug 09 12:20:16 2014 +0200
@@ -56,6 +56,11 @@
 
 manual-check-dependencies-$(3):
 
+MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf
+ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),)
+MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF)
+endif
+
 $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \
 			   $$($$(call UPPERCASE,$(1))_SOURCES) \
 			   manual-check-dependencies \
@@ -65,7 +70,8 @@
 	$$(Q)mkdir -p $$(@D)/.build
 	$$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build
 	$$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \
-	        -D $$(@D) $$(@D)/.build/$(1).txt
+	        -D $$(@D) $$(@D)/.build/$(1).txt \
+	        --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)"
 	-$$(Q)rm -rf $$(@D)/.build
 endef
 

[Buildroot] [PATCH 4 of 5 v4 for 2014.08] manual-text: make sure URLs are displayed if a link text was provided

[Thomas De Schampheleire]

When the asciidoc source contain URLs of the form:
    http://example.com[An example website]
the text representation of the manual would only contain:
    An example website
without displaying the actual URL.

This patch adds an asciidoc configuration file that sets the inline macros
for several URL types so that the display becomes:
    An example website [http://example.com]
For URLs where no link text was provided, the display becomes:
    http://example.com
which is the same as before.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>

---
v4: no changes
v3: no changes
v2:
- link to relevant documentation (which was hard to find)
- refine so that the display is identical to before when no link text is
  provided.

 docs/manual/asciidoc-text.conf |  17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)

diff -r 81d40b841073 -r 93b82c1cf8a4 docs/manual/asciidoc-text.conf
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/manual/asciidoc-text.conf	Sat Aug 09 19:42:30 2014 +0200
@@ -0,0 +1,17 @@
+# Refer to following asciidoc documentation:
+#     http://www.methods.co.nz/asciidoc/userguide.html
+# In particular sections "Macros" and "Attribute References"
+#
+# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL'
+[http-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[https-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[ftp-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[file-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[irc-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[mailto-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}

[Buildroot] [PATCH 5 of 5 v4 for 2014.08] manual-text: hide image representations

[Thomas De Schampheleire]

Images specified in the asciidoc sources (currently only the logo) are
displayed as the file name in the text version of the manual. This causes an
odd line to appear:
    logo.png

Avoid this by setting the image representation macros to {empty} in
manual-text.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/asciidoc-text.conf |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff -r 93b82c1cf8a4 -r 6b930ce9f5cb docs/manual/asciidoc-text.conf
--- a/docs/manual/asciidoc-text.conf	Sat Aug 09 19:42:30 2014 +0200
+++ b/docs/manual/asciidoc-text.conf	Sun Aug 10 17:55:54 2014 +0200
@@ -15,3 +15,9 @@
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
 [mailto-inlinemacro]
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
+
+# Hide image representation from text manual
+[image-inlinemacro]
+{empty}
+[image-blockmacro]
+{empty}

[Buildroot] [PATCH 2 of 5 v4 for 2014.08] gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

[Samuel Martin]

On Fri, Aug 15, 2014 at 3:40 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> In the gendoc infrastructure, using an assignment of the form
>     FOO = docs/$(1)/bar
> inside GENDOC_INNER does not work as expected: the $(1) value is empty here
> and the value of FOO becomes 'docs//bar'.
>
> Parameters $(2), $(3), etc. do not have this problem. The specific thing
> about $(1) is that it is a parameter to GENDOC itself (indicating the
> document to create) and passed transparently to GENDOC_INNER.
>
> This is different from the package infrastructures, where $(1) is set from
> $(pkgname). In fact, the same strategy could be used by the gendoc
> infrastructure as well, as $(pkgname) resolves to 'manual' for file
> docs/manual/manual.mk. This has the advantage that the described problem
> does not occur.
>
> Note that this means that if we want to use the same GENDOC infrastructure
> for another document, it will have to reside in a separate directory than
> the manual.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Reviewed-by: Samuel Martin <s.martin49@gmail.com>

Regards,

-- 
Samuel

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Samuel Martin]

Hi all,

On Fri, Aug 15, 2014 at 3:40 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> This patch series makes some improvements to the generation of the text
> version of the manual:
> - make sure that URLs are visible
> - remove the 'logo' (logo.png) reference
>
> To achieve this, three preparatory patches are needed.
>
> Note that this series stands on its own, it can already be applied.
> A subsequent series will make structural and content-wise changes to the
> manual itself.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

[the whole series; IMO this series is ready to go.]
Tested-by: Samuel Martin <s.martin49@gmail.com>

-- 
Samuel

[Buildroot] [PATCH 1 of 5 v4 for 2014.08] Makefile: unconditionally include pkg-utils.mk

[Yann E. MORIN]

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
> Currently, pkg-utils.mk (included via package/Makefile.in) is only included
> when a configuration file already exists. This means that none of the
> utilities it defines are available without .config.
> 
> In particular:
> - the MESSAGE macro, causing pretty build output. Since some make targets
>   can be run even without .config, like 'make manual', not having this
>   pretty printing is odd.
> 
> - pkgname, pkgdir: in a subsequent patch, these functions will be used for
>   the generation of the manual, and since this should work also without
>   .config, we need these functions to be available.
> 
> This patch moves the include of pkg-utils.mk from package/Makefile.in to
> Makefile, outside of the check for .config.
> 
> This is a quick fix. The full solution involves to minimize the amount of
> Makefile code that is guarded by a check on .config. This approach will be
> taken in the 2014.11 release cycle.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Reviewed-by: Samuel Martin <s.martin49@gmail.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Tested by generating the manual with and without a .config.
Tested that running make with no .config will popup the menuconfig.
Tested that a defconfig still builds OK.

Regards,
Yann E. MORIN.

> ---
> v4: no changes
> v3: new patch, fixing the issue detected by Yann that 'make manual' no
> longer worked without .config
> 
>  Makefile            |  2 ++
>  package/Makefile.in |  1 -
>  2 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff -r 3bcf63a46bc1 -r 232206819ca1 Makefile
> --- a/Makefile	Tue Aug 05 18:25:11 2014 -0300
> +++ b/Makefile	Wed Aug 13 11:30:25 2014 +0200
> @@ -271,6 +271,8 @@
>  # Causes breakage with packages that needs host-ruby
>  unexport RUBYOPT
>  
> +include package/pkg-utils.mk
> +
>  ifeq ($(BR2_HAVE_DOT_CONFIG),y)
>  
>  ################################################################################
> diff -r 3bcf63a46bc1 -r 232206819ca1 package/Makefile.in
> --- a/package/Makefile.in	Tue Aug 05 18:25:11 2014 -0300
> +++ b/package/Makefile.in	Wed Aug 13 11:30:25 2014 +0200
> @@ -385,7 +385,6 @@
>  SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared
>  endif
>  
> -include package/pkg-utils.mk
>  include package/pkg-download.mk
>  include package/pkg-autotools.mk
>  include package/pkg-cmake.mk

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2 of 5 v4 for 2014.08] gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

[Yann E. MORIN]

THomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
> In the gendoc infrastructure, using an assignment of the form
>     FOO = docs/$(1)/bar
> inside GENDOC_INNER does not work as expected: the $(1) value is empty here
> and the value of FOO becomes 'docs//bar'.
> 
> Parameters $(2), $(3), etc. do not have this problem. The specific thing
> about $(1) is that it is a parameter to GENDOC itself (indicating the
> document to create) and passed transparently to GENDOC_INNER.
> 
> This is different from the package infrastructures, where $(1) is set from
> $(pkgname). In fact, the same strategy could be used by the gendoc
> infrastructure as well, as $(pkgname) resolves to 'manual' for file
> docs/manual/manual.mk. This has the advantage that the described problem
> does not occur.
> 
> Note that this means that if we want to use the same GENDOC infrastructure
> for another document, it will have to reside in a separate directory than
> the manual.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Tested by building the manual with and without a .config.

Regards,
Yann E. MORIN.

> ---
> v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
> v3: no changes
> v2: no changes
> 
>  docs/manual/manual.mk |  20 ++++++++++----------
>  1 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff -r 232206819ca1 -r 9d9af165df7b docs/manual/manual.mk
> --- a/docs/manual/manual.mk	Wed Aug 13 11:30:25 2014 +0200
> +++ b/docs/manual/manual.mk	Sat Aug 09 18:18:58 2014 +0200
> @@ -78,16 +78,16 @@
>  # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies.
>  ################################################################################
>  define GENDOC
> -$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
> -$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
> -$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
> -$(call GENDOC_INNER,$(1),text,text,text,text)
> -$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB)
> -clean: $(1)-clean
> -$(1)-clean:
> -	$$(Q)$$(RM) -rf $$(O)/docs/$(1)
> -.PHONY: $(1) $(1)-clean manual-update-lists
> +$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
> +$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
> +$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
> +$(call GENDOC_INNER,$(pkgname),text,text,text,text)
> +$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB)
> +clean: $(pkgname)-clean
> +$(pkgname)-clean:
> +	$$(Q)$$(RM) -rf $$(O)/docs/$(pkgname)
> +.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists
>  endef
>  
>  MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
> -$(eval $(call GENDOC,manual))
> +$(eval $(call GENDOC))

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 3 of 5 v4 for 2014.08] gendoc infra: add support for asciidoc configuration files

[Yann E. MORIN]

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
> This patch introduces support for asciidoc configuration files, specific for
> each output format (html, text, pdf, ...). This is useful to make certain
> tweaks in the document generation.
> 
> If a file docs/manual/asciidoc-<format>.conf is present, it is passed to
> asciidoc as configuration file. If no file for the current format is
> present, the options passed to asciidoc are empty.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Samuel Martin <s.martin49@gmail.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
> v4: no changes
> v3: no changes
> v2: no changes
> 
>  docs/manual/manual.mk |  8 +++++++-
>  1 files changed, 7 insertions(+), 1 deletions(-)
> 
> diff -r 9d9af165df7b -r 81d40b841073 docs/manual/manual.mk
> --- a/docs/manual/manual.mk	Sat Aug 09 18:18:58 2014 +0200
> +++ b/docs/manual/manual.mk	Sat Aug 09 12:20:16 2014 +0200
> @@ -56,6 +56,11 @@
>  
>  manual-check-dependencies-$(3):
>  
> +MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf
> +ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),)
> +MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF)
> +endif
> +
>  $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \
>  			   $$($$(call UPPERCASE,$(1))_SOURCES) \
>  			   manual-check-dependencies \
> @@ -65,7 +70,8 @@
>  	$$(Q)mkdir -p $$(@D)/.build
>  	$$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build
>  	$$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \
> -	        -D $$(@D) $$(@D)/.build/$(1).txt
> +	        -D $$(@D) $$(@D)/.build/$(1).txt \
> +	        --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)"
>  	-$$(Q)rm -rf $$(@D)/.build
>  endef
>  

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 4 of 5 v4 for 2014.08] manual-text: make sure URLs are displayed if a link text was provided

[Yann E. MORIN]

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
> When the asciidoc source contain URLs of the form:
>     http://example.com[An example website]
> the text representation of the manual would only contain:
>     An example website
> without displaying the actual URL.
> 
> This patch adds an asciidoc configuration file that sets the inline macros
> for several URL types so that the display becomes:
>     An example website [http://example.com]
> For URLs where no link text was provided, the display becomes:
>     http://example.com
> which is the same as before.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Samuel Martin <s.martin49@gmail.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
> v4: no changes
> v3: no changes
> v2:
> - link to relevant documentation (which was hard to find)
> - refine so that the display is identical to before when no link text is
>   provided.
> 
>  docs/manual/asciidoc-text.conf |  17 +++++++++++++++++
>  1 files changed, 17 insertions(+), 0 deletions(-)
> 
> diff -r 81d40b841073 -r 93b82c1cf8a4 docs/manual/asciidoc-text.conf
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/docs/manual/asciidoc-text.conf	Sat Aug 09 19:42:30 2014 +0200
> @@ -0,0 +1,17 @@
> +# Refer to following asciidoc documentation:
> +#     http://www.methods.co.nz/asciidoc/userguide.html
> +# In particular sections "Macros" and "Attribute References"
> +#
> +# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL'
> +[http-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}
> +[https-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}
> +[ftp-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}
> +[file-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}
> +[irc-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}
> +[mailto-inlinemacro]
> +{0=}{0? &#91;}{name}:{target}{0?&#93;}

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 5 of 5 v4 for 2014.08] manual-text: hide image representations

[Yann E. MORIN]

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
> Images specified in the asciidoc sources (currently only the logo) are
> displayed as the file name in the text version of the manual. This causes an
> odd line to appear:
>     logo.png
> 
> Avoid this by setting the image representation macros to {empty} in
> manual-text.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Samuel Martin <s.martin49@gmail.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Tested that the filenames no longer appears, and that's the only
difference. :-)

Regards,
Yann E. MORIN.

> ---
> v4: no changes
> v3: no changes
> v2: no changes
> 
>  docs/manual/asciidoc-text.conf |  6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff -r 93b82c1cf8a4 -r 6b930ce9f5cb docs/manual/asciidoc-text.conf
> --- a/docs/manual/asciidoc-text.conf	Sat Aug 09 19:42:30 2014 +0200
> +++ b/docs/manual/asciidoc-text.conf	Sun Aug 10 17:55:54 2014 +0200
> @@ -15,3 +15,9 @@
>  {0=}{0? &#91;}{name}:{target}{0?&#93;}
>  [mailto-inlinemacro]
>  {0=}{0? &#91;}{name}:{target}{0?&#93;}
> +
> +# Hide image representation from text manual
> +[image-inlinemacro]
> +{empty}
> +[image-blockmacro]
> +{empty}

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:

> Thomas De Schampheleire (5)
>   Makefile: unconditionally include pkg-utils.mk
>   gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
>   gendoc infra: add support for asciidoc configuration files
>   manual-text: make sure URLs are displayed if a link text was provided
>   manual-text: hide image representations

Nice, you've added the list of patches to the cover letter :-)

I've applied the patch series, thanks to you, and to Yann and Samuel
for testing/reviewing.

Related to this patch series, here are a few things I noted:

 * When building the PDF and txt versions of the manual, I get one
   warning:

   a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

   It seems to be harmless as the result is really located in the right
   place (output/docs), but I found it worth mentioning.

 * Also, here the PDF version of the manual doesn't build:

   a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

   It's most likely due to something missing or a wrong version in my
   installation, but it also means we're maybe not checking for all the
   necessary tools before generating the manual.

Both of those 'issues' are not introduced by your patch series, they
existed before it.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas De Schampheleire]

On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Thomas De Schampheleire,
>
> On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:
>
>> Thomas De Schampheleire (5)
>>   Makefile: unconditionally include pkg-utils.mk
>>   gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
>>   gendoc infra: add support for asciidoc configuration files
>>   manual-text: make sure URLs are displayed if a link text was provided
>>   manual-text: hide image representations
>
> Nice, you've added the list of patches to the cover letter :-)

Yes :-)

>
> I've applied the patch series, thanks to you, and to Yann and Samuel
> for testing/reviewing.
>
> Related to this patch series, here are a few things I noted:
>
>  * When building the PDF and txt versions of the manual, I get one
>    warning:
>
>    a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
>
>    It seems to be harmless as the result is really located in the right
>    place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid
passing -D for non-html formats. Not sure whether it's worth the
effort, I will have a look.

>
>  * Also, here the PDF version of the manual doesn't build:
>
>    a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

I will have a look at this too, but I may need your help here.
Could you run the same command manually? It should show more output...

Thanks,
Thomas

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Samuel Martin]

On Sat, Aug 16, 2014 at 12:15 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Dear Thomas De Schampheleire,
>>
>> On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:
>>
>>> Thomas De Schampheleire (5)
>>>   Makefile: unconditionally include pkg-utils.mk
>>>   gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
>>>   gendoc infra: add support for asciidoc configuration files
>>>   manual-text: make sure URLs are displayed if a link text was provided
>>>   manual-text: hide image representations
>>
>> Nice, you've added the list of patches to the cover letter :-)
>
> Yes :-)
>
>>
>> I've applied the patch series, thanks to you, and to Yann and Samuel
>> for testing/reviewing.
>>
>> Related to this patch series, here are a few things I noted:
>>
>>  * When building the PDF and txt versions of the manual, I get one
>>    warning:
>>
>>    a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
>>
>>    It seems to be harmless as the result is really located in the right
>>    place (output/docs), but I found it worth mentioning.
>
> Yes, I see the same. We could pass format specific options to avoid
> passing -D for non-html formats. Not sure whether it's worth the
> effort, I will have a look.

I've noticed this too, but have not found a proper way to suppress it.

>
>>
>>  * Also, here the PDF version of the manual doesn't build:
>>
>>    a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

Few shortcomings:
- the generated appendix tables are only included in the html manual
(when running: make manual-clean manual). It seems these files are not
regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables,
because when they don't exits, the pdf manual generation succeeds.

This makes me think about refactoring the manual.mk to be closer to
the package ones, i.e. having it build directory under $(0)/build,
maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is
this pdf issue to fix.

>
> I will have a look at this too, but I may need your help here.
> Could you run the same command manually? It should show more output...
>
> Thanks,
> Thomas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

Regards,

-- 
Samuel

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Samuel Martin]

On Sat, Aug 16, 2014 at 1:00 PM, Samuel Martin <s.martin49@gmail.com> wrote:
> On Sat, Aug 16, 2014 at 12:15 PM, Thomas De Schampheleire
> <patrickdepinguin@gmail.com> wrote:
>> On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
[...]
>>>  * When building the PDF and txt versions of the manual, I get one
>>>    warning:
>>>
>>>    a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
>>>
>>>    It seems to be harmless as the result is really located in the right
>>>    place (output/docs), but I found it worth mentioning.
>>
>> Yes, I see the same. We could pass format specific options to avoid
>> passing -D for non-html formats. Not sure whether it's worth the
>> effort, I will have a look.
>
> I've noticed this too, but have not found a proper way to suppress it.

I have not looked into this.

>>
>>>
>>>  * Also, here the PDF version of the manual doesn't build:
>>>
>>>    a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
>
> Few shortcomings:
> - the generated appendix tables are only included in the html manual
> (when running: make manual-clean manual). It seems these files are not
> regenerated after the 'rm -rf $$(@D)/.build' :-/
> - the above error seems related to these generated appendix tables,
> because when they don't exits, the pdf manual generation succeeds.
>
> This makes me think about refactoring the manual.mk to be closer to
> the package ones, i.e. having it build directory under $(0)/build,
> maybe inheriting from generic-package...
> Anyway, this is just wild ideas; for the upcoming release there is
> this pdf issue to fix.
>

Well, after diving into asciidoc/dblatex/xslt, I eventually found the
problem and the the fix!

Short-long story:

When generating the pdf manual, a2x calls dblatex, which calls xsltproc.
Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and
reaches its limit set to 15000 when processing the target
package-list.txt file (empirically, this limit allows a 1147-entry
table).
To raise this limit, a --maxvars option exists in xsltproc program but
the latest release of libxslt is buggy and this option is not honored.
:-(
Hopefully the fix is already merged upstream [2].
Backporting this patch is enough to get a working xsltproc, but it
also means that we now need to build host-libxslt (and a .config) to
build the manual.
So, to build the manual we have to run:
$ make defconfig && make host-libxslt manual

The main drawback of this fix is that building the manual could now
require building some host-package.

This also makes echo to the include statement of the *.mk files
without having a .config file discussion.

Anyway, I will prepare a series fixing this, though it is not a
mandatory for the release

Note that we have already released a couple of Buildroot versions
without recieving any complaint about this.


[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c


Regards,

-- 
Samuel

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas Petazzoni]

Dear Samuel Martin,

On Sat, 16 Aug 2014 19:24:42 +0200, Samuel Martin wrote:

> When generating the pdf manual, a2x calls dblatex, which calls xsltproc.
> Here is a verbose log [1].
> During its execution, xsltproc checks for template recursion and
> reaches its limit set to 15000 when processing the target
> package-list.txt file (empirically, this limit allows a 1147-entry
> table).
> To raise this limit, a --maxvars option exists in xsltproc program but
> the latest release of libxslt is buggy and this option is not honored.
> :-(
> Hopefully the fix is already merged upstream [2].
> Backporting this patch is enough to get a working xsltproc, but it
> also means that we now need to build host-libxslt (and a .config) to
> build the manual.
> So, to build the manual we have to run:
> $ make defconfig && make host-libxslt manual
> 
> The main drawback of this fix is that building the manual could now
> require building some host-package.
> 
> This also makes echo to the include statement of the *.mk files
> without having a .config file discussion.
> 
> Anyway, I will prepare a series fixing this, though it is not a
> mandatory for the release
> 
> Note that we have already released a couple of Buildroot versions
> without recieving any complaint about this.

I'm not sure we should enforce building host-libxslt as a dependency of
the manual. Maybe just check the version of xsltproc and bail out with
an error message if it's not the appropriate version. I believe most
people are just using the pre-generated versions of the manual that are
online on Buildroot website. So it's up to us to have the appropriate
xsltproc version there.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas De Schampheleire]

Hi Samuel,

On Sat, Aug 16, 2014 at 7:24 PM, Samuel Martin <s.martin49@gmail.com> wrote:
> On Sat, Aug 16, 2014 at 1:00 PM, Samuel Martin <s.martin49@gmail.com> wrote:
>> On Sat, Aug 16, 2014 at 12:15 PM, Thomas De Schampheleire
>> <patrickdepinguin@gmail.com> wrote:
>>> On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
> [...]
>>>>  * When building the PDF and txt versions of the manual, I get one
>>>>    warning:
>>>>
>>>>    a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
>>>>
>>>>    It seems to be harmless as the result is really located in the right
>>>>    place (output/docs), but I found it worth mentioning.
>>>
>>> Yes, I see the same. We could pass format specific options to avoid
>>> passing -D for non-html formats. Not sure whether it's worth the
>>> effort, I will have a look.
>>
>> I've noticed this too, but have not found a proper way to suppress it.
>
> I have not looked into this.
>
>>>
>>>>
>>>>  * Also, here the PDF version of the manual doesn't build:
>>>>
>>>>    a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
>>
>> Few shortcomings:
>> - the generated appendix tables are only included in the html manual
>> (when running: make manual-clean manual). It seems these files are not
>> regenerated after the 'rm -rf $$(@D)/.build' :-/
>> - the above error seems related to these generated appendix tables,
>> because when they don't exits, the pdf manual generation succeeds.
>>
>> This makes me think about refactoring the manual.mk to be closer to
>> the package ones, i.e. having it build directory under $(0)/build,
>> maybe inheriting from generic-package...
>> Anyway, this is just wild ideas; for the upcoming release there is
>> this pdf issue to fix.
>>
>
> Well, after diving into asciidoc/dblatex/xslt, I eventually found the
> problem and the the fix!
>
> Short-long story:
>
> When generating the pdf manual, a2x calls dblatex, which calls xsltproc.
> Here is a verbose log [1].
> During its execution, xsltproc checks for template recursion and
> reaches its limit set to 15000 when processing the target
> package-list.txt file (empirically, this limit allows a 1147-entry
> table).
> To raise this limit, a --maxvars option exists in xsltproc program but
> the latest release of libxslt is buggy and this option is not honored.
> :-(
> Hopefully the fix is already merged upstream [2].
> Backporting this patch is enough to get a working xsltproc, but it
> also means that we now need to build host-libxslt (and a .config) to
> build the manual.
> So, to build the manual we have to run:
> $ make defconfig && make host-libxslt manual
>
> The main drawback of this fix is that building the manual could now
> require building some host-package.
>
> This also makes echo to the include statement of the *.mk files
> without having a .config file discussion.
>
> Anyway, I will prepare a series fixing this, though it is not a
> mandatory for the release
>
> Note that we have already released a couple of Buildroot versions
> without recieving any complaint about this.
>
>
> [1] http://code.bulix.org/1905fo-86779
> [2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

I must say I don't fully understand this.

There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf
manual using 'make manual-clean manual'. However, I don't see the same
thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line
(to fix the second point), then I do get this error.


2. None of the manuals except the html one (the first one) contain the
package lists. This is due to the removal of the .build directory
between each manual generation, which is the directory where the lists
are generated.
Removing the directory seems not really needed to me, so why not just
avoid that?

Thanks,
Thomas

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Samuel Martin]

Hi Thomas,

On Sat, Aug 16, 2014 at 8:35 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> Hi Samuel,
>
> On Sat, Aug 16, 2014 at 7:24 PM, Samuel Martin <s.martin49@gmail.com> wrote:
>> On Sat, Aug 16, 2014 at 1:00 PM, Samuel Martin <s.martin49@gmail.com> wrote:
>>> On Sat, Aug 16, 2014 at 12:15 PM, Thomas De Schampheleire
>>> <patrickdepinguin@gmail.com> wrote:
>>>> On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
>> [...]
>>>>>  * When building the PDF and txt versions of the manual, I get one
>>>>>    warning:
>>>>>
>>>>>    a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
>>>>>
>>>>>    It seems to be harmless as the result is really located in the right
>>>>>    place (output/docs), but I found it worth mentioning.
>>>>
>>>> Yes, I see the same. We could pass format specific options to avoid
>>>> passing -D for non-html formats. Not sure whether it's worth the
>>>> effort, I will have a look.
>>>
>>> I've noticed this too, but have not found a proper way to suppress it.
>>
>> I have not looked into this.
>>
>>>>
>>>>>
>>>>>  * Also, here the PDF version of the manual doesn't build:
>>>>>
>>>>>    a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0  "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
>>>
>>> Few shortcomings:
>>> - the generated appendix tables are only included in the html manual
>>> (when running: make manual-clean manual). It seems these files are not
>>> regenerated after the 'rm -rf $$(@D)/.build' :-/
>>> - the above error seems related to these generated appendix tables,
>>> because when they don't exits, the pdf manual generation succeeds.
>>>
>>> This makes me think about refactoring the manual.mk to be closer to
>>> the package ones, i.e. having it build directory under $(0)/build,
>>> maybe inheriting from generic-package...
>>> Anyway, this is just wild ideas; for the upcoming release there is
>>> this pdf issue to fix.
>>>
>>
>> Well, after diving into asciidoc/dblatex/xslt, I eventually found the
>> problem and the the fix!
>>
>> Short-long story:
>>
>> When generating the pdf manual, a2x calls dblatex, which calls xsltproc.
>> Here is a verbose log [1].
>> During its execution, xsltproc checks for template recursion and
>> reaches its limit set to 15000 when processing the target
>> package-list.txt file (empirically, this limit allows a 1147-entry
>> table).
>> To raise this limit, a --maxvars option exists in xsltproc program but
>> the latest release of libxslt is buggy and this option is not honored.
>> :-(
>> Hopefully the fix is already merged upstream [2].
>> Backporting this patch is enough to get a working xsltproc, but it
>> also means that we now need to build host-libxslt (and a .config) to
>> build the manual.
>> So, to build the manual we have to run:
>> $ make defconfig && make host-libxslt manual
>>
>> The main drawback of this fix is that building the manual could now
>> require building some host-package.
>>
>> This also makes echo to the include statement of the *.mk files
>> without having a .config file discussion.
>>
>> Anyway, I will prepare a series fixing this, though it is not a
>> mandatory for the release
>>
>> Note that we have already released a couple of Buildroot versions
>> without recieving any complaint about this.
>>
>>
>> [1] http://code.bulix.org/1905fo-86779
>> [2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c
>
> I must say I don't fully understand this.

Ok, I'll try to be clearer.

>
> There are two observations:
> 1. ThomasP and apparantly you too get an error on generating the pdf
> manual using 'make manual-clean manual'. However, I don't see the same
> thing, pdf builds fine for me.
> Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line
> (to fix the second point), then I do get this error.
>
>
> 2. None of the manuals except the html one (the first one) contain the
> package lists. This is due to the removal of the .build directory
> between each manual generation, which is the directory where the lists
> are generated.
> Removing the directory seems not really needed to me, so why not just
> avoid that?

First, there is this 2nd observation.
I agree with your suggestion; however since the manual generation
takes place in the output doc directory, we should clean the temporary
$(O)/docs/manual/.build directory at the end of the manual generation.
I propose to move this build directory under $(O)/build/manual/. I'm
cooking a patch for this

A consequence of this bug (to be confirmed) is that we have released a
couple of version with some missing pieces in the pdf manual included
in the tarball.

Another consequence of this bug is that it maskes the observation #1
when you build all the manual formats at the same time. That could
explain why you did not see the error before removing the "rm" call.
If you run 'make distclean && make manual-pdf' from a clean worktree,
you will trigger the dblatex error.

The error of the observation #1 is caused by xsltproc, which reaches
some template recursion limits when processing the big target package
table.
This limit can be raised thanks to the xsltproc option: --maxvars, but
the latest release of libxslt (1.1.28) is buggy and does not correctly
handle this argument when it is passed through the command line.
This xsltproc bug is fixed upstream on the libxslt master [1], but not
yet in any of its release.

So:
- we have already released a couple of version with some missing
pieces in manual.pdf;
- if someone wants to successfully build the pdf manual, he/she has to
make sure the xsltproc correctly support the --maxvars option (i.e. it
has been built with the patch [1]).
- if we want to make possible to anyone to build the pdf manual, then
the should provide our own xsltproc program. This introduces a
host-package dependency on the manual target, (so a .config dependency
and so on, and so forth).


As Yann (on IRC) and Thomas P. noticed, we have not received any
complaint about the missing part in the pdf manual, probably because
people mainly uses the online html doc and very few people needs to
build the manual.

So:
- we can certainly live with a broken pdf manual generation;
- the person doing the release (Peter) just has to make sure his
xsltproc is not buggy to properly generate the official manual (both
the online one and the one included in the release tarball).


Hope this is clearer.


[1] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

>
> Thanks,
> Thomas

Regards,

-- 
Samuel

[Buildroot] [PATCH 0 of 5 v4 for 2014.08] manual-text generation improvements

[Thomas De Schampheleire]

Hi Samuel,

On Sat, Aug 16, 2014 at 9:39 PM, Samuel Martin <s.martin49@gmail.com> wrote:
> Hi Thomas,
> Ok, I'll try to be clearer.
>
>>
>> There are two observations:
>> 1. ThomasP and apparantly you too get an error on generating the pdf
>> manual using 'make manual-clean manual'. However, I don't see the same
>> thing, pdf builds fine for me.
>> Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line
>> (to fix the second point), then I do get this error.
>>
>>
>> 2. None of the manuals except the html one (the first one) contain the
>> package lists. This is due to the removal of the .build directory
>> between each manual generation, which is the directory where the lists
>> are generated.
>> Removing the directory seems not really needed to me, so why not just
>> avoid that?
>
> First, there is this 2nd observation.
> I agree with your suggestion; however since the manual generation
> takes place in the output doc directory, we should clean the temporary
> $(O)/docs/manual/.build directory at the end of the manual generation.
> I propose to move this build directory under $(O)/build/manual/. I'm
> cooking a patch for this

If you do this, then another improvement is to extract the 'mkdir +
rsync' steps to a common target that is run only once. Currently, the
rsync is repeated for every format, while it is always the same
content.

>
> A consequence of this bug (to be confirmed) is that we have released a
> couple of version with some missing pieces in the pdf manual included
> in the tarball.
>
> Another consequence of this bug is that it maskes the observation #1
> when you build all the manual formats at the same time. That could
> explain why you did not see the error before removing the "rm" call.
> If you run 'make distclean && make manual-pdf' from a clean worktree,
> you will trigger the dblatex error.
>
> The error of the observation #1 is caused by xsltproc, which reaches
> some template recursion limits when processing the big target package
> table.
> This limit can be raised thanks to the xsltproc option: --maxvars, but
> the latest release of libxslt (1.1.28) is buggy and does not correctly
> handle this argument when it is passed through the command line.
> This xsltproc bug is fixed upstream on the libxslt master [1], but not
> yet in any of its release.
>
> So:
> - we have already released a couple of version with some missing
> pieces in manual.pdf;
> - if someone wants to successfully build the pdf manual, he/she has to
> make sure the xsltproc correctly support the --maxvars option (i.e. it
> has been built with the patch [1]).
> - if we want to make possible to anyone to build the pdf manual, then
> the should provide our own xsltproc program. This introduces a
> host-package dependency on the manual target, (so a .config dependency
> and so on, and so forth).
>
> As Yann (on IRC) and Thomas P. noticed, we have not received any
> complaint about the missing part in the pdf manual, probably because
> people mainly uses the online html doc and very few people needs to
> build the manual.
>
> So:
> - we can certainly live with a broken pdf manual generation;
> - the person doing the release (Peter) just has to make sure his
> xsltproc is not buggy to properly generate the official manual (both
> the online one and the one included in the release tarball).
>
>
> Hope this is clearer.

Yes, this is clearer, thanks.

Best regards,
Thomas