generated HTML contains broken links

generated HTML contains broken links

[Eric Hanchrow]

I'm just starting to play with git, and have checked it out (with "git
clone git://git.kernel.org/pub/scm/git/git.git"), and built the
documentation (cd Documentation; make), on Cygwin.  I notice that the
generated HTML docs are full of broken links -- for example, my file
C:/cygwin/usr/local/src/git/Documentation/git.html includes this:

        git<a href="git-instaweb">1</a>

but there is no document named "git-instaweb" on my disk; instead,
it's named "git-instaweb.html".

I'm using asciidoc version 8.2.4, if it matters.
-- 
I write [from 5 AM to 7 AM] to discover what I think.  After
all, the bars aren't open that early.
        -- Daniel Boorstin, former Librarian of Congress

Re: generated HTML contains broken links

[Dan McGee]

On 12/28/2007 05:49 PM, Eric Hanchrow wrote:
> I'm just starting to play with git, and have checked it out (with "git
> clone git://git.kernel.org/pub/scm/git/git.git"), and built the
> documentation (cd Documentation; make), on Cygwin.  I notice that the
> generated HTML docs are full of broken links -- for example, my file
> C:/cygwin/usr/local/src/git/Documentation/git.html includes this:
> 
>         git<a href="git-instaweb">1</a>
> 
> but there is no document named "git-instaweb" on my disk; instead,
> it's named "git-instaweb.html".
> 
> I'm using asciidoc version 8.2.4, if it matters.

We noticed this with the upgrade from Asciidoc 8.2.2 -> 8.2.3 on our project. It is broken in both the manpages and the HTML generated documentation. I've included an example below. So far, I haven't had luck tracking down the reason but I am looking into trying to fix this tonight. If anyone else is better with this stuff, it would be great if you could take a look.

Relevant email on the pacman-dev list [1]:

On Nov 9, 2007 4:05 PM, Andrew Fyfe <andrew@neptune-one.net> wrote:
> Did little digging, the breakage/change is in now asciidoc converts from asciidoc to xml (so it's 
> not docbook-xsl). In 8.2.2 manlink:pacman.conf[5] expands to
> 
> <citerefentry><refentrytitle>pacman.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry>,
> 
> in 8.2.3 it expands to
> 
> man<ulink url="pacman.conf">5</ulink>,

Note that manlink is basically just a renamed gitlink, and can be found here:
<http://projects.archlinux.org/git/?p=pacman.git;a=blob;f=doc/asciidoc.conf>

-Dan

[1] <http://archlinux.org/pipermail/pacman-dev/2007-November/009937.html>



Generated man pages from Junio:
HOOKS
       This command can run commit-msg, pre-commit, and post-commit hooks. See
       [1]hooks for more information.

SEE ALSO
       git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)

AUTHOR
       Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
       <junkio@cox.net>

GIT
       Part of the git(7) suite

Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
HOOKS
       This command can run commit-msg, pre-commit, and post-commit hooks. See
       hooks[5] for more information.

SEE ALSO
       git1[1], git1[2], git1[8], git1[6], git1[9]

AUTHOR
       Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
       <junkio@cox.net>

GIT
       Part of the git7[10] suite

Re: generated HTML contains broken links

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

On Fri, Dec 28, 2007 at 09:01:17PM -0600, Dan McGee <dpmcgee@gmail.com> wrote:
> Generated man pages from Junio:
> HOOKS
>        This command can run commit-msg, pre-commit, and post-commit hooks. See
>        [1]hooks for more information.
> 
> SEE ALSO
>        git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)
> 
> AUTHOR
>        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
>        <junkio@cox.net>
> 
> GIT
>        Part of the git(7) suite
> 
> Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
> HOOKS
>        This command can run commit-msg, pre-commit, and post-commit hooks. See
>        hooks[5] for more information.
> 
> SEE ALSO
>        git1[1], git1[2], git1[8], git1[6], git1[9]
> 
> AUTHOR
>        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
>        <junkio@cox.net>
> 
> GIT
>        Part of the git7[10] suite

http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d

here is the commit that fixed the problem for pacman. what do you think,
should we rename gitlink to something else, too - or should we contact
upstream to notify them about they caused a breakage?

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: generated HTML contains broken links

[Dan McGee]

On Dec 29, 2007 9:57 AM, Miklos Vajna <vmiklos@frugalware.org> wrote:
> On Fri, Dec 28, 2007 at 09:01:17PM -0600, Dan McGee <dpmcgee@gmail.com> wrote:
> > Generated man pages from Junio:
> > HOOKS
> >        This command can run commit-msg, pre-commit, and post-commit hooks. See
> >        [1]hooks for more information.
> >
> > SEE ALSO
> >        git-add(1), git-rm(1), git-mv(1), git-merge(1), git-commit-tree(1)
> >
> > AUTHOR
> >        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
> >        <junkio@cox.net>
> >
> > GIT
> >        Part of the git(7) suite
> >
> > Man pages generated locally (with Asciidoc 8.2.3 or 8.2.5):
> > HOOKS
> >        This command can run commit-msg, pre-commit, and post-commit hooks. See
> >        hooks[5] for more information.
> >
> > SEE ALSO
> >        git1[1], git1[2], git1[8], git1[6], git1[9]
> >
> > AUTHOR
> >        Written by Linus Torvalds <torvalds@osdl.org> and Junio C Hamano
> >        <junkio@cox.net>
> >
> > GIT
> >        Part of the git7[10] suite
>
> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d
>
> here is the commit that fixed the problem for pacman. what do you think,
> should we rename gitlink to something else, too - or should we contact
> upstream to notify them about they caused a breakage?

I've tried twice now to send a patch to the list...please let me know
if it isn't working, because I am getting CCed on the send, and the
mailing list address is set in the sendemail.to variable. I did notice
that send-email doesn't even prompt you for confirmation of the
default "to" address if you have this variable set, so I'm not sure if
something weird is happening there.

-Dan

[PATCH] Documentation: rename gitlink macro to linkgit

[Dan McGee]

[-- Attachment #1: Type: text/plain, Size: 9586 bytes --]

On 12/29/2007 10:24 AM, Dan McGee wrote:
> On Dec 29, 2007 9:57 AM, Miklos Vajna <vmiklos@frugalware.org> wrote:
>> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=b3c6bdda38f7e370e1f80f02a61f1d3f08c1b57d
>>
>> here is the commit that fixed the problem for pacman. what do you think,
>> should we rename gitlink to something else, too - or should we contact
>> upstream to notify them about they caused a breakage?
> 
> I've tried twice now to send a patch to the list...please let me know
> if it isn't working, because I am getting CCed on the send, and the
> mailing list address is set in the sendemail.to variable. I did notice
> that send-email doesn't even prompt you for confirmation of the
> default "to" address if you have this variable set, so I'm not sure if
> something weird is happening there.
> 
> -Dan

>From 68bf426e810e732ff3f9f75ffcd69f777b538685 Mon Sep 17 00:00:00 2001
From: Dan McGee <dpmcgee@gmail.com>
Date: Sat, 29 Dec 2007 00:20:38 -0600
Subject: [PATCH] Documentation: rename gitlink macro to linkgit

Between AsciiDoc 8.2.2 and 8.2.3, the following change was made to the stock
Asciidoc configuration:

@@ -149,7 +153,10 @@
 # Inline macros.
 # Backslash prefix required for escape processing.
 # (?s) re flag for line spanning.
-(?su)[\\]?(?P<name>\w(\w|-)*?):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
+
+# Explicit so they can be nested.
+(?su)[\\]?(?P<name>(http|https|ftp|file|mailto|callto|image|link)):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
+
 # Anchor: [[[id]]]. Bibliographic anchor.
 (?su)[\\]?\[\[\[(?P<attrlist>[\w][\w-]*?)\]\]\]=anchor3
 # Anchor: [[id,xreflabel]]

This default regex now matches explicit values, and unfortunately in this
case gitlink was being matched by just 'link', causing the wrong inline
macro template to be applied. By renaming the macro, we can avoid being
matched by the wrong regex.

Signed-off-by: Dan McGee <dpmcgee@gmail.com>
---
 Documentation/asciidoc.conf              |    8 +-
 Documentation/blame-options.txt          |    2 +-
 Documentation/cmd-list.perl              |    2 +-
 Documentation/config.txt                 |  126 +++++++++++++++---------------
 Documentation/cvs-migration.txt          |   10 +-
 Documentation/diff-options.txt           |    4 +-
 Documentation/everyday.txt               |   48 ++++++------
 Documentation/fetch-options.txt          |    2 +-
 Documentation/git-add.txt                |   16 ++--
 Documentation/git-am.txt                 |   16 ++--
 Documentation/git-annotate.txt           |    4 +-
 Documentation/git-apply.txt              |   12 ++--
 Documentation/git-archimport.txt         |    2 +-
 Documentation/git-archive.txt            |    2 +-
 Documentation/git-bisect.txt             |    2 +-
 Documentation/git-blame.txt              |    8 +-
 Documentation/git-branch.txt             |   12 ++--
 Documentation/git-bundle.txt             |   10 +-
 Documentation/git-cat-file.txt           |    4 +-
 Documentation/git-check-attr.txt         |    4 +-
 Documentation/git-check-ref-format.txt   |    6 +-
 Documentation/git-checkout-index.txt     |    2 +-
 Documentation/git-checkout.txt           |    4 +-
 Documentation/git-cherry-pick.txt        |    4 +-
 Documentation/git-cherry.txt             |    2 +-
 Documentation/git-citool.txt             |    6 +-
 Documentation/git-clean.txt              |    4 +-
 Documentation/git-clone.txt              |    2 +-
 Documentation/git-commit-tree.txt        |    6 +-
 Documentation/git-commit.txt             |   30 ++++----
 Documentation/git-config.txt             |    2 +-
 Documentation/git-count-objects.txt      |    2 +-
 Documentation/git-cvsexportcommit.txt    |    2 +-
 Documentation/git-cvsimport.txt          |    4 +-
 Documentation/git-cvsserver.txt          |    4 +-
 Documentation/git-daemon.txt             |    2 +-
 Documentation/git-describe.txt           |    2 +-
 Documentation/git-diff-files.txt         |    2 +-
 Documentation/git-diff-index.txt         |    2 +-
 Documentation/git-diff-tree.txt          |    2 +-
 Documentation/git-diff.txt               |    8 +-
 Documentation/git-fast-export.txt        |   12 ++--
 Documentation/git-fast-import.txt        |   18 ++--
 Documentation/git-fetch-pack.txt         |    4 +-
 Documentation/git-fetch.txt              |    4 +-
 Documentation/git-filter-branch.txt      |   16 ++--
 Documentation/git-fmt-merge-msg.txt      |    4 +-
 Documentation/git-format-patch.txt       |    8 +-
 Documentation/git-fsck-objects.txt       |    2 +-
 Documentation/git-fsck.txt               |    2 +-
 Documentation/git-gc.txt                 |   14 ++--
 Documentation/git-get-tar-commit-id.txt  |    4 +-
 Documentation/git-grep.txt               |    2 +-
 Documentation/git-gui.txt                |    4 +-
 Documentation/git-hash-object.txt        |    2 +-
 Documentation/git-help.txt               |    6 +-
 Documentation/git-http-fetch.txt         |    2 +-
 Documentation/git-http-push.txt          |    2 +-
 Documentation/git-imap-send.txt          |    2 +-
 Documentation/git-index-pack.txt         |   10 +-
 Documentation/git-init-db.txt            |    2 +-
 Documentation/git-init.txt               |    2 +-
 Documentation/git-instaweb.txt           |    2 +-
 Documentation/git-log.txt                |   10 +-
 Documentation/git-lost-found.txt         |    4 +-
 Documentation/git-ls-files.txt           |    8 +-
 Documentation/git-ls-remote.txt          |    4 +-
 Documentation/git-ls-tree.txt            |    2 +-
 Documentation/git-mailinfo.txt           |    4 +-
 Documentation/git-mailsplit.txt          |    2 +-
 Documentation/git-merge-base.txt         |    2 +-
 Documentation/git-merge-file.txt         |    4 +-
 Documentation/git-merge-index.txt        |    2 +-
 Documentation/git-merge-one-file.txt     |    2 +-
 Documentation/git-merge-tree.txt         |    2 +-
 Documentation/git-merge.txt              |    8 +-
 Documentation/git-mergetool.txt          |    4 +-
 Documentation/git-mktag.txt              |    2 +-
 Documentation/git-mktree.txt             |    2 +-
 Documentation/git-mv.txt                 |    2 +-
 Documentation/git-name-rev.txt           |    4 +-
 Documentation/git-pack-objects.txt       |   10 +-
 Documentation/git-pack-redundant.txt     |    8 +-
 Documentation/git-pack-refs.txt          |    2 +-
 Documentation/git-parse-remote.txt       |    2 +-
 Documentation/git-patch-id.txt           |    2 +-
 Documentation/git-peek-remote.txt        |    2 +-
 Documentation/git-prune-packed.txt       |    6 +-
 Documentation/git-prune.txt              |    2 +-
 Documentation/git-pull.txt               |   10 +-
 Documentation/git-push.txt               |    4 +-
 Documentation/git-quiltimport.txt        |    2 +-
 Documentation/git-read-tree.txt          |    8 +-
 Documentation/git-rebase.txt             |   10 +-
 Documentation/git-receive-pack.txt       |    4 +-
 Documentation/git-reflog.txt             |    8 +-
 Documentation/git-relink.txt             |    2 +-
 Documentation/git-remote.txt             |   12 ++--
 Documentation/git-repack.txt             |   14 ++--
 Documentation/git-repo-config.txt        |    2 +-
 Documentation/git-request-pull.txt       |    2 +-
 Documentation/git-rerere.txt             |    4 +-
 Documentation/git-reset.txt              |    8 +-
 Documentation/git-rev-list.txt           |   20 +++---
 Documentation/git-rev-parse.txt          |    2 +-
 Documentation/git-revert.txt             |    4 +-
 Documentation/git-rm.txt                 |    4 +-
 Documentation/git-send-email.txt         |    2 +-
 Documentation/git-send-pack.txt          |    6 +-
 Documentation/git-sh-setup.txt           |    2 +-
 Documentation/git-shell.txt              |    2 +-
 Documentation/git-shortlog.txt           |    2 +-
 Documentation/git-show-branch.txt        |    2 +-
 Documentation/git-show-index.txt         |    2 +-
 Documentation/git-show-ref.txt           |    6 +-
 Documentation/git-show.txt               |    8 +-
 Documentation/git-stash.txt              |   10 +-
 Documentation/git-status.txt             |    6 +-
 Documentation/git-stripspace.txt         |    2 +-
 Documentation/git-submodule.txt          |    6 +-
 Documentation/git-svn.txt                |   12 ++--
 Documentation/git-symbolic-ref.txt       |    2 +-
 Documentation/git-tag.txt                |    2 +-
 Documentation/git-tar-tree.txt           |    2 +-
 Documentation/git-unpack-file.txt        |    2 +-
 Documentation/git-unpack-objects.txt     |    2 +-
 Documentation/git-update-index.txt       |   12 ++--
 Documentation/git-update-ref.txt         |    2 +-
 Documentation/git-update-server-info.txt |    2 +-
 Documentation/git-upload-archive.txt     |    2 +-
 Documentation/git-upload-pack.txt        |    2 +-
 Documentation/git-var.txt                |    8 +-
 Documentation/git-verify-pack.txt        |    2 +-
 Documentation/git-verify-tag.txt         |    2 +-
 Documentation/git-whatchanged.txt        |    2 +-
 Documentation/git-write-tree.txt         |    2 +-
 Documentation/git.txt                    |   20 +++---
 Documentation/gitattributes.txt          |   10 +-
 Documentation/gitcli.txt                 |    2 +-
 Documentation/gitignore.txt              |    6 +-
 Documentation/gitk.txt                   |    6 +-
 Documentation/gitmodules.txt             |    6 +-
 Documentation/glossary.txt               |   18 ++--
 143 files changed, 460 insertions(+), 460 deletions(-)

Looks like the patch was bigger than I thought. Gzipped and attached if that is acceptable.

[-- Attachment #2: 0001-Documentation-rename-gitlink-macro-to-linkgit.patch.gz --]
[-- Type: application/x-gzip, Size: 36244 bytes --]

Re: [PATCH] Documentation: rename gitlink macro to linkgit

[Junio C Hamano]

Dan McGee <dpmcgee@gmail.com> writes:

>>From 68bf426e810e732ff3f9f75ffcd69f777b538685 Mon Sep 17 00:00:00 2001
> From: Dan McGee <dpmcgee@gmail.com>
> Date: Sat, 29 Dec 2007 00:20:38 -0600
> Subject: [PATCH] Documentation: rename gitlink macro to linkgit
>
> Between AsciiDoc 8.2.2 and 8.2.3, the following change was made to the stock
> Asciidoc configuration:
>
> @@ -149,7 +153,10 @@
>  # Inline macros.
>  # Backslash prefix required for escape processing.
>  # (?s) re flag for line spanning.
> -(?su)[\\]?(?P<name>\w(\w|-)*?):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
> +
> +# Explicit so they can be nested.
> +(?su)[\\]?(?P<name>(http|https|ftp|file|mailto|callto|image|link)):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])=
> +
>  # Anchor: [[[id]]]. Bibliographic anchor.
>  (?su)[\\]?\[\[\[(?P<attrlist>[\w][\w-]*?)\]\]\]=anchor3
>  # Anchor: [[id,xreflabel]]
>
> This default regex now matches explicit values, and unfortunately in this
> case gitlink was being matched by just 'link', causing the wrong inline
> macro template to be applied. By renaming the macro, we can avoid being
> matched by the wrong regex.

What's already tagged, released to the wild and picked up by
distros cannot be taken back, so I'd most likely have to apply
your patch anyway, but I have to say I am not very amused.  I'd
call this a regression on AsciiDoc's part.

I would have expected some courtesy to make sure that updates to
AsciiDoc would not to break existing users, especially the ones
that they use as the top advertising material in the "Projects
using AsciiDoc" list at http://www.methods.co.nz/asciidoc/ ;-)

Stuart, is there anything we can help you to set up some
automated tests to catch AsciiDoc regression, so we do not have
to suffer like this again?

Re: [PATCH] Documentation: rename gitlink macro to linkgit

[Yannick Gingras]

Junio C Hamano <gitster@pobox.com> writes:

> Stuart, is there anything we can help you to set up some automated
> tests to catch AsciiDoc regression, so we do not have to suffer like
> this again?

We considered adding a nose test suite.  The upcoming v9.0 release
involves quite a bit of code massaging and we will definitely need an
extensive test suite.  But the test suite can only catch obvious
rendering failures so any help in eyeballing the output will be
appreciated.

Should we setup a distributed proof reading system like Project
Gutenberg?  I hope we don't need it.  But a few scripts to batch
render all the doc of popular projects would be nice.  We could upload
such snapshot render jobs where everyone could take a quick glance and
spot obvious rendering errors.

If you follow AsciiDoc's mailing list [1], we'll go through a series
of release candidates before the final 9.0.  "With enough eyeballs,
all bugs are shallow."

[1]: asciidoc-discuss@lists.metaperl.com

I reply to two mailing lists; I think this particular problem belongs
to asciidoc-discuss.

Best regards, 

-- 
Yannick Gingras

Re: [PATCH] Documentation: rename gitlink macro to linkgit

[Junio C Hamano]

Yannick Gingras <ygingras@ygingras.net> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Stuart, is there anything we can help you to set up some automated
>> tests to catch AsciiDoc regression, so we do not have to suffer like
>> this again?
>
> We considered adding a nose test suite.  The upcoming v9.0 release
> involves quite a bit of code massaging and we will definitely need an
> extensive test suite.  But the test suite can only catch obvious
> rendering failures so any help in eyeballing the output will be
> appreciated.

You could go fancy like that, but I suspect that an automated
test to compare the text dump (e.g. "links -dump doc.html")
generated by before and after version, perhaps with minimum
massaging, would go a long enough way.  At least that would have
caught the "gitlink" breakage, wouldn't it?

Re: [PATCH] Documentation: rename gitlink macro to linkgit

[Yannick Gingras]

Junio C Hamano <gitster@pobox.com> writes:

>> But the test suite can only catch obvious rendering failures so any
>> help in eyeballing the output will be appreciated.
>
> You could go fancy like that, but I suspect that an automated
> test to compare the text dump (e.g. "links -dump doc.html")
> generated by before and after version, perhaps with minimum
> massaging, would go a long enough way.  At least that would have
> caught the "gitlink" breakage, wouldn't it?

Yeah it would have caught it.  I'll hack something like that before
the 9.0 release.

-- 
Yannick Gingras