[Buildroot] Building Manual Errors

[Buildroot] Building Manual Errors

[rjbarnet at rockwellcollins.com]

I've been trying to build the manual for buildroot using 'make manual' 
using two different machines that have Ubuntu 10.04 LTS and 12.04 LTS.

On Ubuntu 10.04 there was an unlisted dependancy with python-argparse and 
even when I installed both python-argparse and asciidocs I still wasn't 
able to build the HTML version of the manual but the text version worked. 
Below is the error I got:

>>>   Generating HTML manual...
Traceback (most recent call last):
  File "/usr/bin/a2x", line 730, in <module>
    a2x.execute()
  File "/usr/bin/a2x", line 294, in execute
    self.__getattribute__('to_'+self.format)()  # Execute to_* functions.
  File "/usr/bin/a2x", line 493, in to_xhtml
    self.copy_resources(xhtml_file, src_dir, self.destination_dir)
  File "/usr/bin/a2x", line 443, in copy_resources
    lambda attrs: attrs.get('type') == 'text/css')
  File "/usr/bin/a2x", line 235, in find_resources
    parser.feed(open(f).read())
  File "/usr/lib/python2.6/HTMLParser.py", line 108, in feed
    self.goahead(0)
  File "/usr/lib/python2.6/HTMLParser.py", line 148, in goahead
    k = self.parse_starttag(i)
  File "/usr/lib/python2.6/HTMLParser.py", line 249, in parse_starttag
    attrvalue = self.unescape(attrvalue)
  File "/usr/lib/python2.6/HTMLParser.py", line 387, in unescape
    return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities, 
s)
  File "/usr/lib/python2.6/re.py", line 151, in sub
    return _compile(pattern, 0).sub(repl, string, count)
UnicodeDecodeError: 'ascii' codec can't decode byte 

On my Ubuntu 12.04 machine I can succesfully build the HTML version of the 
manual but not the text version

user at ubuntu:~/projects/buildroot/mainline$ make manual-txt
Writing the target-packages list in:
        output/docs/manual/.build/package-list.txt
Writing the host-packages list in:
        output/docs/manual/.build/host-package-list.txt
Writing the deprecated list in:
        output/docs/manual/.build/deprecated-list.txt
a2x: ERROR: w3m -cols 70 -dump -T text/html -no-graph 
/home/user/projects/buildroot/mainline/output/docs/manual/manual.text.html 
> /home/user/projects/buildroot/mainline/output/docs/manual/manual.text 
returned non-zero exit status 127
make: *** [output/docs/manual/manual.text] Error 1

I've seen in the manual that there is an issue with building the manual 
with debain but it really doesn't explain what that issue. I tried 
searching the mailing list for this issue along with bugzilla but came up 
empty handed in finding the exact issue. Its not the end of the world for 
me since I can kind of work around. Also if somebody has a known working 
configuration for building the manual that would be helpful to know.

Another random thought, is there any plans to build all the tools needed 
for doing compile of the manual within buildroot so then we are not 
dependent on the different distros of linux for building the manuals?

Thanks,
-Ryan



Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/821d6f2d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/821d6f2d/attachment.gif>

[Buildroot] Building Manual Errors

[Samuel Martin]

Hi Ryan,

2013/9/11 <rjbarnet@rockwellcollins.com>
>
> I've been trying to build the manual for buildroot using 'make manual'
using two different machines that have Ubuntu 10.04 LTS and 12.04 LTS.
>
> On Ubuntu 10.04 there was an unlisted dependancy with python-argparse
My bad, I forgot this deps when I updated the manual.

> and even when I installed both python-argparse and asciidocs I still
wasn't able to build the HTML version of the manual but the text version
worked. Below is the error I got:
>
> >>>   Generating HTML manual...
> Traceback (most recent call last):
>   File "/usr/bin/a2x", line 730, in <module>
>     a2x.execute()
>   File "/usr/bin/a2x", line 294, in execute
>     self.__getattribute__('to_'+self.format)()  # Execute to_* functions.
>   File "/usr/bin/a2x", line 493, in to_xhtml
>     self.copy_resources(xhtml_file, src_dir, self.destination_dir)
>   File "/usr/bin/a2x", line 443, in copy_resources
>     lambda attrs: attrs.get('type') == 'text/css')
>   File "/usr/bin/a2x", line 235, in find_resources
>     parser.feed(open(f).read())
>   File "/usr/lib/python2.6/HTMLParser.py", line 108, in feed
>     self.goahead(0)
>   File "/usr/lib/python2.6/HTMLParser.py", line 148, in goahead
>     k = self.parse_starttag(i)
>   File "/usr/lib/python2.6/HTMLParser.py", line 249, in parse_starttag
>     attrvalue = self.unescape(attrvalue)
>   File "/usr/lib/python2.6/HTMLParser.py", line 387, in unescape
>     return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));",
replaceEntities, s)
>   File "/usr/lib/python2.6/re.py", line 151, in sub
>     return _compile(pattern, 0).sub(repl, string, count)
> UnicodeDecodeError: 'ascii' codec can't decode byte
>
> On my Ubuntu 12.04 machine I can succesfully build the HTML version of
the manual but not the text version
>
> user at ubuntu:~/projects/buildroot/mainline$ make manual-txt
> Writing the target-packages list in:
>         output/docs/manual/.build/package-list.txt
> Writing the host-packages list in:
>         output/docs/manual/.build/host-package-list.txt
> Writing the deprecated list in:
>         output/docs/manual/.build/deprecated-list.txt
> a2x: ERROR: w3m -cols 70 -dump -T text/html -no-graph
/home/user/projects/buildroot/mainline/output/docs/manual/manual.text.html
> /home/user/projects/buildroot/mainline/output/docs/manual/manual.text
returned non-zero exit status 127
> make: *** [output/docs/manual/manual.text] Error 1
I looks like a w3m error...

I know there could be some issues with asciidoc:
- there might be a minimal version for asciidoc to be able to build BR
manual (not sure, not tested with old versions);
- I know there is a bug in dblatex 0.3.4 (0.3.2 works fine on my distro:
archlinux) preventing from building the pdf manual.

>
> I've seen in the manual that there is an issue with building the manual
with debain but it really doesn't explain what that issue. I tried
searching the mailing list for this issue along with bugzilla but came up
empty handed in finding the exact issue. Its not the end of the world for
me since I can kind of work around. Also if somebody has a known working
configuration for building the manual that would be helpful to know.
>
> Another random thought, is there any plans to build all the tools needed
for doing compile of the manual within buildroot so then we are not
dependent on the different distros of linux for building the manuals?
Could be the solution.

I whish we had a buildfarm regularly building the manual (and pushing it on
the web).
Peter, any news about this?


Regards,

--
Samuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130911/32a9a1dc/attachment.html>

[Buildroot] Building Manual Errors

[Peter Korsgaard]

>>>>> "Samuel" == Samuel Martin <s.martin49@gmail.com> writes:

Hi,

 Samuel> I whish we had a buildfarm regularly building the manual (and pushing it on the
 Samuel> web).
 Samuel> Peter, any news about this?

Sorry, I completely forgot about that. I'll look into it.

-- 
Bye, Peter Korsgaard

[Buildroot] Building Manual Errors

[Ryan Barnett]

Samuel Martin <s.martin49@gmail.com> wrote on 09/11/2013 04:14:18 AM:
 
> Hi Ryan,
> 
> 2013/9/11 <rjbarnet@rockwellcollins.com>
> >
> > I've been trying to build the manual for buildroot using 'make 
> > manual' using two different machines that have Ubuntu 10.04 LTS and 
12.04 LTS.
> >
> > On Ubuntu 10.04 there was an unlisted dependancy with python-argparse
>
> My bad, I forgot this deps when I updated the manual.

No problem.

> > Another random thought, is there any plans to build all the tools 
> > needed for doing compile of the manual within buildroot so then we are
> > not dependent on the different distros of linux for building the 
manuals?
>
> Could be the solution.

I guess I was looking do some manual updates for the SELinux contributes 
that we are doing but I can't seem to build the manual to verify that
my formatting is correct since I haven't done much with ASCII docs.

Just briefly glancing through how many packages would have to added to 
buildroot in order to build all the packages need to compile the manual 
it looks like quite a few:

 * asciidoc
 * a2x
 * xsltproc
 * DocBook XSL Stylesheets
 * dblatex
 * FOP
 * w3m
 * Lynx
 * epubcheck
 * python argparse
 * python

There may be more but this was the dependancies that I saw when just
looking at what it would take to build asciidoc and a2x.

Thanks,
-Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130911/fb4f3387/attachment.html>

[Buildroot] Building Manual Errors

[Thomas De Schampheleire]

Hi Ryan,


On Wed, Sep 11, 2013 at 1:49 AM, <rjbarnet@rockwellcollins.com> wrote:
[..]
>
> On my Ubuntu 12.04 machine I can succesfully build the HTML version of the manual but not the text version
>
> user at ubuntu:~/projects/buildroot/mainline$ make manual-txt
> Writing the target-packages list in:
>         output/docs/manual/.build/package-list.txt
> Writing the host-packages list in:
>         output/docs/manual/.build/host-package-list.txt
> Writing the deprecated list in:
>         output/docs/manual/.build/deprecated-list.txt
> a2x: ERROR: w3m -cols 70 -dump -T text/html -no-graph /home/user/projects/buildroot/mainline/output/docs/manual/manual.text.html > /home/user/projects/buildroot/mainline/output/docs/manual/manual.text returned non-zero exit status 127
> make: *** [output/docs/manual/manual.text] Error 1
>
> I've seen in the manual that there is an issue with building the manual with debain but it really doesn't explain what that issue. I tried searching the mailing list for this issue along with bugzilla but came up empty handed in finding the exact issue. Its not the end of the world for me since I can kind of work around. Also if somebody has a known working configuration for building the manual that would be helpful to know.

In fact, I have the same thing, but the actual problem is that w3m is
missing on the system, which can be fixed with 'sudo apt-get install
w3m'. After this, the text version builds fine.

Samuel: shouldn't we check the dependencies for building the manual
before someone attemps to do that? On my Linux Mint system
(=Ubuntu-based) I had to install asciidoc (which pulled in a lot of
tex deps) and w3m.

Best regards,
Thomas

[Buildroot] Building Manual Errors

[Ryan Barnett]

Thomas,

Thomas De Schampheleire <patrickdepinguin@gmail.com> wrote on 09/12/2013 
11:18:06 AM:
 
> In fact, I have the same thing, but the actual problem is that w3m is
> missing on the system, which can be fixed with 'sudo apt-get install
> w3m'. After this, the text version builds fine.

Thanks! That fixed my problems, I guess I didn't even check to see if
I had that package installed. I didn't think of checking to see if w3m
was installed - thanks for pointing out the obvious :)

> Samuel: shouldn't we check the dependencies for building the manual
> before someone attemps to do that? On my Linux Mint system
> (=Ubuntu-based) I had to install asciidoc (which pulled in a lot of
> tex deps) and w3m.

One I idea I had to solve these dependancies issues between specific
versions and packages for all the different distro's is to add all 
of the packages that are needed for building manual or any other 
buildroot maintenance script (stuff that is in support/scripts) to
buildroot as host packages and then put together a 
buildroot_maintenance_defconfig that will build all of the packages
need for the host system to run this scripts. Let me know what 
you think about that and we can maybe continue this discussion 
in a different thread. 

> Best regards,
> Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/467b18d7/attachment.html>

[Buildroot] Building Manual Errors

[Thomas Petazzoni]

Dear Ryan Barnett,

On Thu, 12 Sep 2013 11:27:51 -0500, Ryan Barnett wrote:

> > Samuel: shouldn't we check the dependencies for building the manual
> > before someone attemps to do that? On my Linux Mint system
> > (=Ubuntu-based) I had to install asciidoc (which pulled in a lot of
> > tex deps) and w3m.
> 
> One I idea I had to solve these dependancies issues between specific
> versions and packages for all the different distro's is to add all 
> of the packages that are needed for building manual or any other 
> buildroot maintenance script (stuff that is in support/scripts) to
> buildroot as host packages and then put together a 
> buildroot_maintenance_defconfig that will build all of the packages
> need for the host system to run this scripts. Let me know what 
> you think about that and we can maybe continue this discussion 
> in a different thread. 

My understanding is that asciidoc requires a very large number of TeX /
LaTeX dependencies which I really don't think we want to package in
Buildroot, so I believe doing this is not such a good idea.

Normal users shouldn't need to build the manual, since it's available
online for reading. Only developers will care about this, and I believe
installing asciidoc is a reasonable requirement for developers.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Building Manual Errors

[Thomas De Schampheleire]

Hi,

On Thu, Sep 12, 2013 at 6:37 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Ryan Barnett,
>
> On Thu, 12 Sep 2013 11:27:51 -0500, Ryan Barnett wrote:
>
>> > Samuel: shouldn't we check the dependencies for building the manual
>> > before someone attemps to do that? On my Linux Mint system
>> > (=Ubuntu-based) I had to install asciidoc (which pulled in a lot of
>> > tex deps) and w3m.
>>
>> One I idea I had to solve these dependancies issues between specific
>> versions and packages for all the different distro's is to add all
>> of the packages that are needed for building manual or any other
>> buildroot maintenance script (stuff that is in support/scripts) to
>> buildroot as host packages and then put together a
>> buildroot_maintenance_defconfig that will build all of the packages
>> need for the host system to run this scripts. Let me know what
>> you think about that and we can maybe continue this discussion
>> in a different thread.
>
> My understanding is that asciidoc requires a very large number of TeX /
> LaTeX dependencies which I really don't think we want to package in
> Buildroot, so I believe doing this is not such a good idea.
>
> Normal users shouldn't need to build the manual, since it's available
> online for reading. Only developers will care about this, and I believe
> installing asciidoc is a reasonable requirement for developers.

I agree with this.
But I do think that it would be nice to check for missing
dependencies, being asciidoc and w3m.

Best regards,
Thomas

[Buildroot] Building Manual Errors

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Thu, 12 Sep 2013 18:51:22 +0200, Thomas De Schampheleire wrote:

> I agree with this.
> But I do think that it would be nice to check for missing
> dependencies, being asciidoc and w3m.

Agreed, but they should only be checked when the user actually tries to
build the documentation. I don't want to make these additional
mandatory dependencies.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Building Manual Errors

[Thomas De Schampheleire]

On Thu, Sep 12, 2013 at 6:56 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Thomas De Schampheleire,
>
> On Thu, 12 Sep 2013 18:51:22 +0200, Thomas De Schampheleire wrote:
>
>> I agree with this.
>> But I do think that it would be nice to check for missing
>> dependencies, being asciidoc and w3m.
>
> Agreed, but they should only be checked when the user actually tries to
> build the documentation. I don't want to make these additional
> mandatory dependencies.

Yes, fully agreed.
Anyone volunteering to make a patch for this? If not, I'm willing to
do this, but as time permits.

Best regards,
Thomas

[Buildroot] Building Manual Errors

[Ryan Barnett]

Thomas(es) -

Thomas De Schampheleire <patrickdepinguin@gmail.com> wrote on 09/12/2013 
12:02:45 PM:

> On Thu, Sep 12, 2013 at 6:56 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Dear Thomas De Schampheleire,
> >
> > On Thu, 12 Sep 2013 18:51:22 +0200, Thomas De Schampheleire wrote:
> >
> >> I agree with this.
> >> But I do think that it would be nice to check for missing
> >> dependencies, being asciidoc and w3m.

Didn't really think about the dependancies or the black hole that this 
would imply. I satisfied with just adding these checks and documenting 
a know working configuration.

> > Agreed, but they should only be checked when the user actually tries 
to
> > build the documentation. I don't want to make these additional
> > mandatory dependencies.
> 
> Yes, fully agreed.
> Anyone volunteering to make a patch for this? If not, I'm willing to
> do this, but as time permits.

I don't think I would have time to put the checks into the scripts. With
not being familiar with these it would be a time consuming addition.
However, when I'm updating the documentation for SELinux stuff I can add 
a known working configuration to the manual for my system and the 
versions too.

Thanks,
-Ryan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/87c242f7/attachment.html>

[Buildroot] Building Manual Errors

[Thomas De Schampheleire]

Hi Ryan, Samuel,


On Wed, Sep 11, 2013 at 1:49 AM, <rjbarnet@rockwellcollins.com> wrote:

> I've been trying to build the manual for buildroot using 'make manual'
> using two different machines that have Ubuntu 10.04 LTS and 12.04 LTS.
>
> On Ubuntu 10.04 there was an unlisted dependancy with python-argparse and
> even when I installed both python-argparse and asciidocs I still wasn't
> able to build the HTML version of the manual but the text version worked.
> Below is the error I got:
>
> >>>   Generating HTML manual...
> Traceback (most recent call last):
>   File "/usr/bin/a2x", line 730, in <module>
>     a2x.execute()
>   File "/usr/bin/a2x", line 294, in execute
>     self.__getattribute__('to_'+self.format)()  # Execute to_* functions.
>   File "/usr/bin/a2x", line 493, in to_xhtml
>     self.copy_resources(xhtml_file, src_dir, self.destination_dir)
>   File "/usr/bin/a2x", line 443, in copy_resources
>     lambda attrs: attrs.get('type') == 'text/css')
>   File "/usr/bin/a2x", line 235, in find_resources
>     parser.feed(open(f).read())
>   File "/usr/lib/python2.6/HTMLParser.py", line 108, in feed
>     self.goahead(0)
>   File "/usr/lib/python2.6/HTMLParser.py", line 148, in goahead
>     k = self.parse_starttag(i)
>   File "/usr/lib/python2.6/HTMLParser.py", line 249, in parse_starttag
>     attrvalue = self.unescape(attrvalue)
>   File "/usr/lib/python2.6/HTMLParser.py", line 387, in unescape
>     return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities,
> s)
>   File "/usr/lib/python2.6/re.py", line 151, in sub
>     return _compile(pattern, 0).sub(repl, string, count)
> UnicodeDecodeError: 'ascii' codec can't decode byte
>


I found this:
https://groups.google.com/forum/#!topic/asciidoc/sPvOIQzhY2Q

There, they mention that asciidoc 8.6.3 fixes the issue. Could you verify
that the Ubuntu 10.04 system has asciidoc < 8.6.3, while the Ubuntu 12.04
system has >= 8.6.3 ?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130919/6b1ac432/attachment.html>

[Buildroot] Building Manual Errors

[Ryan Barnett]

Thomas/Samuel,

Thomas De Schampheleire <patrickdepinguin@gmail.com> wrote on 09/19/2013 
10:03:45 AM:

> I found this:
> https://groups.google.com/forum/#!topic/asciidoc/sPvOIQzhY2Q
> 
> There, they mention that asciidoc 8.6.3 fixes the issue. Could you 
> verify that the Ubuntu 10.04 system has asciidoc < 8.6.3, while the 
> Ubuntu 12.04 system has >= 8.6.3 ?

I've verified the version of asciidoc that I have on Ubuntu 10.04 is 8.5.2 
and 
Ubuntu 12.04 is 8.6.6. So that is why I get different results.

> Thanks, 
> Thomas 

Thanks,
-Ryan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130919/0123a510/attachment.html>