[PATCH 0/2] qqz and runlatex improvements

[PATCH 0/2] qqz and runlatex improvements

[Akira Yokosawa]

From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001
From: Akira Yokosawa <akiysw@gmail.com>
Date: Thu, 21 Apr 2016 23:02:59 +0900
Subject: [PATCH 0/2] qqz and runlatex improvements

Hi,

There have been two issues that bother me a little since I began to
build perfbook.pdf myself.
One is the cross-link references of quick quizzes that land on random
nearby places in the PDF.
The other is the premature termination of make with warning messages
displayed.
I think I have managed to fix both issues.

However, .sty file modification may introduce regressions and
should be fully tested before being pushed out. Can anybody on the
list test it and give feedback?

Thanks.

Akira Yokosawa (2):
  qqz: Improve accuracy of cross-links
  runlatex.sh: Fix while loop condition

 qqz.sty               | 14 ++++++++------
 utilities/runlatex.sh |  4 ++--
 2 files changed, 10 insertions(+), 8 deletions(-)

-- 
1.9.1

[PATCH 1/2] qqz: Improve accuracy of cross-links

[Akira Yokosawa]

From b1b643edfe021163754c6aa83f81669bc544f386 Mon Sep 17 00:00:00 2001
From: Akira Yokosawa <akiysw@gmail.com>
Date: Thu, 21 Apr 2016 21:29:57 +0900
Subject: [PATCH 1/2] qqz: Improve accuracy of cross-links

In commit 33b93f8258f5 ("qqz: Cross-link questions and answers"),
cross-links of quick quizzes and their answers were implemented.
However it uses the \stepcounter{} command, and direct references
of 'quickquizctr' counter is not output to the .aux file.
The symptom is when you click on a hyperlink in the resulting PDF,
you jump to a random nearby label such as that of Figure, Table,
or Section, etc. which happens just before the \label{} using
'quickquizctr'.

By using \refstepcounter{} instead and using 2 counters
'quickquizctr' and 'quickquizctrC' ('C' stands for 'Chapter')
and defining \theHNum at the head of qqz.sty, then redefining
it within the \QuickQuizAnsers command, matching QQ.n.m's and
QQA.n.m's are output to the .aux file correctly.

By applying this change, resulting hyperlinks become to jump
almost directly where you'd expect.

This change may have the effect of increasing the required number
of 'pdflatex' invocations to get final result. The following commit
will take care of it.

Signed-off-by: Akira Yokosawa <akiysw@gmail.com>
---
 qqz.sty | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/qqz.sty b/qqz.sty
index 7270468..438f515 100644
--- a/qqz.sty
+++ b/qqz.sty
@@ -15,7 +15,9 @@
 %%
 %% Authors: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

-\newcounter{quickquizctr}
+\newcounter{quickquizctr}[chapter]
+\newcounter{quickquizctrC}[section]
+\newcommand*{\theHNum}{\arabic{chapter}.\arabic{quickquizctr}}
 \newcommand{\QuickQuizAnswerChapter}{\textbf{Unknown QuickQAC!!!}}

 \newcommand{\QuickQHeading}[3]{\label{#1.#3}\hyperref[#2.#3]{\textbf{Quick Quiz #3:}}}
@@ -23,9 +25,9 @@
 \newcommand{\QuickQuizChapter}[2]{
 	\chapter{#2}
 	\label{#1}
-	\setcounter{quickquizctr}{0}}
+	}
 \newcommand{\QuickQuiz}[1]{
-	\stepcounter{quickquizctr}
+	\refstepcounter{quickquizctr}
 	\QuickQHeading{QQ}{QQA}{\thechapter.\thequickquizctr}}
 \newcommand{\QuickQuizAnswer}[1]{\rule{7pt}{7pt}}
 \newcommand{\QuickQuizEnd}{}
@@ -37,6 +39,7 @@
 % There is probably a better way to do this, but this does work.

 \newcommand{\QuickQuizAnswers}{
+	\renewcommand*{\theHNum}{\arabic{section}.\arabic{quickquizctrC}}
 	\chapter{Answers to Quick Quizzes}
 	\label{chp:Answers to Quick Quizzes}
 	~ \\
@@ -46,11 +49,10 @@

 \newcommand{\QuickQAC}[2]{
 	\edef\QuickQuizAnswerChapter{\getrefnumber{#1}}
-	\setcounter{quickquizctr}{0}
 	\section{#2}}

 \newcommand{\QuickQ}[1]{
-	\stepcounter{quickquizctr} ~ \\ ~ \\
-	\QuickQHeading{QQA}{QQ}{\QuickQuizAnswerChapter.\thequickquizctr} #1 ~ \\ }
+	\refstepcounter{quickquizctrC} ~ \\ ~ \\
+	\QuickQHeading{QQA}{QQ}{\QuickQuizAnswerChapter.\thequickquizctrC} #1 ~ \\ }

 \newcommand{\QuickA}[1]{\rule{7pt}{7pt} \\ ~ \\ \textbf{Answer:} \\ }
-- 
1.9.1

[PATCH 2/2] runlatex.sh: Fix while loop condition

[Akira Yokosawa]

From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001
From: Akira Yokosawa <akiysw@gmail.com>
Date: Thu, 21 Apr 2016 22:11:06 +0900
Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition

After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"),
'make' becomes to terminate prematurely with warning messages such as

>   Latex Warning: There were undefined references
>   Latex Warning: There were multiply-defined labels.

This can be dealt with loosening loop condition in runlatex.sh
which is given to 'grep' command.
Also, This commit increases the iteration max count to accommodate
the required number of 'pdflatex' invocations caused by qqz cross-link.

Signed-off-by: Akira Yokosawa <akiysw@gmail.com>
---
 utilities/runlatex.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
index 8d2d77a..15d96c1 100644
--- a/utilities/runlatex.sh
+++ b/utilities/runlatex.sh
@@ -47,7 +47,7 @@ then
 		echo "No bibliography directory, skipping bibtex."
 	fi
 fi
-while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
+while grep -q 'LaTeX Warning:' $basename.log
 do
 	iter=`expr $iter + 1`
 	echo "pdflatex $iter"
@@ -56,7 +56,7 @@ do
 	then
 		echo "----- Fatal latex error, see $basename.log for details. -----"
 	fi
-	if test "$iter" -eq 4
+	if test "$iter" -eq 5
 	then
 		echo "Iteration limit: $iter passes through pdflatex"
 		exit 1
-- 
1.9.1

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Paul E. McKenney]

On Thu, Apr 21, 2016 at 11:32:24PM +0900, Akira Yokosawa wrote:
> >From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001
> From: Akira Yokosawa <akiysw@gmail.com>
> Date: Thu, 21 Apr 2016 22:11:06 +0900
> Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition
> 
> After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"),
> 'make' becomes to terminate prematurely with warning messages such as
> 
> >   Latex Warning: There were undefined references
> >   Latex Warning: There were multiply-defined labels.
> 
> This can be dealt with loosening loop condition in runlatex.sh
> which is given to 'grep' command.
> Also, This commit increases the iteration max count to accommodate
> the required number of 'pdflatex' invocations caused by qqz cross-link.
> 
> Signed-off-by: Akira Yokosawa <akiysw@gmail.com>

I pulled this into a test branch, and it seemed to work for me, thank you!
I am running on Ubuntu, any experience on other platforms?

Please see below for a couple of questions.

							Thanx, Paul

> ---
>  utilities/runlatex.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
> index 8d2d77a..15d96c1 100644
> --- a/utilities/runlatex.sh
> +++ b/utilities/runlatex.sh
> @@ -47,7 +47,7 @@ then
>  		echo "No bibliography directory, skipping bibtex."
>  	fi
>  fi
> -while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
> +while grep -q 'LaTeX Warning:' $basename.log
>  do
>  	iter=`expr $iter + 1`
>  	echo "pdflatex $iter"

Given the loosening above, should we put a "grep LaTeX Warning:" here,
so that users could interrupt the build if they felt that rerunning
latex wasn't going to help?

> @@ -56,7 +56,7 @@ do
>  	then
>  		echo "----- Fatal latex error, see $basename.log for details. -----"

Also given the loosening above, should there be an "exit 1" here?

>  	fi
> -	if test "$iter" -eq 4
> +	if test "$iter" -eq 5
>  	then
>  		echo "Iteration limit: $iter passes through pdflatex"
>  		exit 1
> -- 
> 1.9.1
> 

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Akira Yokosawa]

Hi Paul,

On 2016/04/22 1:27, Paul E. McKenney wrote:
> On Thu, Apr 21, 2016 at 11:32:24PM +0900, Akira Yokosawa wrote:
>> >From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001
>> From: Akira Yokosawa <akiysw@gmail.com>
>> Date: Thu, 21 Apr 2016 22:11:06 +0900
>> Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition
>>
>> After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"),
>> 'make' becomes to terminate prematurely with warning messages such as
>>
>>>   Latex Warning: There were undefined references
>>>   Latex Warning: There were multiply-defined labels.
>>
>> This can be dealt with loosening loop condition in runlatex.sh
>> which is given to 'grep' command.
>> Also, This commit increases the iteration max count to accommodate
>> the required number of 'pdflatex' invocations caused by qqz cross-link.
>>
>> Signed-off-by: Akira Yokosawa <akiysw@gmail.com>
> 
> I pulled this into a test branch, and it seemed to work for me, thank you!
> I am running on Ubuntu, any experience on other platforms?
> 
> Please see below for a couple of questions.
> 
> 							Thanx, Paul
> 
>> ---
>>  utilities/runlatex.sh | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
>> index 8d2d77a..15d96c1 100644
>> --- a/utilities/runlatex.sh
>> +++ b/utilities/runlatex.sh
>> @@ -47,7 +47,7 @@ then
>>  		echo "No bibliography directory, skipping bibtex."
>>  	fi
>>  fi
>> -while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
>> +while grep -q 'LaTeX Warning:' $basename.log
>>  do
>>  	iter=`expr $iter + 1`
>>  	echo "pdflatex $iter"
> 
> Given the loosening above, should we put a "grep LaTeX Warning:" here,
> so that users could interrupt the build if they felt that rerunning
> latex wasn't going to help?

So, I loosened the condition too much. I tried another idea and it also seems to work.
See below for the patch. (Note that it is the output of
git diff 514f21cd4156c68e17 utilities/runlatex.sh )
I have not enough time to do git commit & git format patch right now...

>> @@ -56,7 +56,7 @@ do
>>  	then
>>  		echo "----- Fatal latex error, see $basename.log for details. -----"
> 
> Also given the loosening above, should there be an "exit 1" here?

Yes, I think so. (This is added in the following patch)
> 
>>  	fi
>> -	if test "$iter" -eq 4
>> +	if test "$iter" -eq 5
>>  	then
>>  		echo "Iteration limit: $iter passes through pdflatex"
>>  		exit 1
>> -- 
>> 1.9.1
>>
> 
> 

Can you try this one?
Thanks, Akira

---
diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
index 8d2d77a..eba7e16 100644
--- a/utilities/runlatex.sh
+++ b/utilities/runlatex.sh
@@ -47,7 +47,8 @@ then
                echo "No bibliography directory, skipping bibtex."
        fi
 fi
-while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
+while grep -q 'LaTeX Warning: There were undefined references' $basename.log || \
+    grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
 do
        iter=`expr $iter + 1`
        echo "pdflatex $iter"
@@ -55,8 +56,9 @@ do
        if grep -q '! Emergency stop.' $basename.log
        then
                echo "----- Fatal latex error, see $basename.log for details. -----"
+               exit 1
        fi
-       if test "$iter" -eq 4
+       if test "$iter" -eq 5
        then
                echo "Iteration limit: $iter passes through pdflatex"
                exit 1

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Paul E. McKenney]

On Fri, Apr 22, 2016 at 07:05:12AM +0900, Akira Yokosawa wrote:
> Hi Paul,
> 
> On 2016/04/22 1:27, Paul E. McKenney wrote:
> > On Thu, Apr 21, 2016 at 11:32:24PM +0900, Akira Yokosawa wrote:
> >> >From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001
> >> From: Akira Yokosawa <akiysw@gmail.com>
> >> Date: Thu, 21 Apr 2016 22:11:06 +0900
> >> Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition
> >>
> >> After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"),
> >> 'make' becomes to terminate prematurely with warning messages such as
> >>
> >>>   Latex Warning: There were undefined references
> >>>   Latex Warning: There were multiply-defined labels.
> >>
> >> This can be dealt with loosening loop condition in runlatex.sh
> >> which is given to 'grep' command.
> >> Also, This commit increases the iteration max count to accommodate
> >> the required number of 'pdflatex' invocations caused by qqz cross-link.
> >>
> >> Signed-off-by: Akira Yokosawa <akiysw@gmail.com>
> > 
> > I pulled this into a test branch, and it seemed to work for me, thank you!
> > I am running on Ubuntu, any experience on other platforms?
> > 
> > Please see below for a couple of questions.
> > 
> > 							Thanx, Paul
> > 
> >> ---
> >>  utilities/runlatex.sh | 4 ++--
> >>  1 file changed, 2 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
> >> index 8d2d77a..15d96c1 100644
> >> --- a/utilities/runlatex.sh
> >> +++ b/utilities/runlatex.sh
> >> @@ -47,7 +47,7 @@ then
> >>  		echo "No bibliography directory, skipping bibtex."
> >>  	fi
> >>  fi
> >> -while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
> >> +while grep -q 'LaTeX Warning:' $basename.log
> >>  do
> >>  	iter=`expr $iter + 1`
> >>  	echo "pdflatex $iter"
> > 
> > Given the loosening above, should we put a "grep LaTeX Warning:" here,
> > so that users could interrupt the build if they felt that rerunning
> > latex wasn't going to help?
> 
> So, I loosened the condition too much. I tried another idea and it also seems to work.
> See below for the patch. (Note that it is the output of
> git diff 514f21cd4156c68e17 utilities/runlatex.sh )
> I have not enough time to do git commit & git format patch right now...
> 
> >> @@ -56,7 +56,7 @@ do
> >>  	then
> >>  		echo "----- Fatal latex error, see $basename.log for details. -----"
> > 
> > Also given the loosening above, should there be an "exit 1" here?
> 
> Yes, I think so. (This is added in the following patch)
> > 
> >>  	fi
> >> -	if test "$iter" -eq 4
> >> +	if test "$iter" -eq 5
> >>  	then
> >>  		echo "Iteration limit: $iter passes through pdflatex"
> >>  		exit 1
> >> -- 
> >> 1.9.1
> >>
> > 
> > 
> 
> Can you try this one?
> Thanks, Akira

I did, though for some reason I had to hand-apply it on my test branch.
Not sure what "git apply" didn't like.

It does repeat upon undefined, which is good.  Nicer behavior after
bibliography changes!

There is another "Fatal latex error" earlier that I missed, and it would
be good to have an "exit 1" on that one as well.

The one thing I am still concerned about is a real undefined reference,
which would consume a full five iterations.  Not really sure what would
be improved behavior, though.  Thoughts?

							Thanx, Paul

> ---
> diff --git a/utilities/runlatex.sh b/utilities/runlatex.sh
> index 8d2d77a..eba7e16 100644
> --- a/utilities/runlatex.sh
> +++ b/utilities/runlatex.sh
> @@ -47,7 +47,8 @@ then
>                 echo "No bibliography directory, skipping bibtex."
>         fi
>  fi
> -while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
> +while grep -q 'LaTeX Warning: There were undefined references' $basename.log || \
> +    grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
>  do
>         iter=`expr $iter + 1`
>         echo "pdflatex $iter"
> @@ -55,8 +56,9 @@ do
>         if grep -q '! Emergency stop.' $basename.log
>         then
>                 echo "----- Fatal latex error, see $basename.log for details. -----"
> +               exit 1
>         fi
> -       if test "$iter" -eq 4
> +       if test "$iter" -eq 5
>         then
>                 echo "Iteration limit: $iter passes through pdflatex"
>                 exit 1
> --
> To unsubscribe from this list: send the line "unsubscribe perfbook" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Akira Yokosawa]

On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:
[snip]

>> Can you try this one?
>> Thanks, Akira
> 
> I did, though for some reason I had to hand-apply it on my test branch.
> Not sure what "git apply" didn't like.

Well, I'm afraid you tried to apply it after [PATCH 2/2] applied.
I should have made it clear it was a diff from master at that moment.
Sorry for your trouble.

> It does repeat upon undefined, which is good.  Nicer behavior after
> bibliography changes!
> 
> There is another "Fatal latex error" earlier that I missed, and it would
> be good to have an "exit 1" on that one as well.
> 
> The one thing I am still concerned about is a real undefined reference,
> which would consume a full five iterations.  Not really sure what would
> be improved behavior, though.  Thoughts?
> 
> 							Thanx, Paul

So your question is why you need to change the maximum value from 4 to 5,
isn't it?  That's because the count is checked at the end of the loop,
and even when the 4th round of pdflatex is successful, the check will
fire and 'make' will regard it as an error. At such a later stage, 
remaining warning should only be 'LaTeX Warning: Label(s) may have changed',
if any.

So I made a fairly invasive modification to the script so that the count
is checked at the beginning of the loop. Also the loop is separated for each
of the warning message.

Appended is the whole source code of the script (without the comment part at
the head). I'm not sure which point should I make a diff from. Note that
I truncated the warning messages given to grep. They should be long enough.

What do you think?

If it is OK, I'll resubmit the V2 of the patch series relative to the
current master.

                                                       Thanks, Akira.

--- 
#!/bin/sh
# [snip]
#

if test -z "$1"
then
	echo No latex file, aborting.
	exit 1
fi

basename=`echo $1 | sed -e 's/\.tex$//'`

iter=1
echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
	echo "----- Fatal latex error, see $basename.log for details. -----"
fi
if grep -q 'LaTeX Warning: There were undefined references' $basename.log
then
	if test -d "$2"
	then
		bibtex $basename || :
	else
		echo "No bibliography directory, skipping bibtex."
	fi
fi
while grep -q 'LaTeX Warning: There were undefined references' $basename.log
do
	if test "$iter" -eq 4
	then
		echo "Iteration limit: $iter passes through pdflatex"
		exit 1
	fi
	iter=`expr $iter + 1`
	echo "pdflatex $iter"
	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
	if grep -q '! Emergency stop.' $basename.log
	then
		echo "----- Fatal latex error, see $basename.log for details. -----"
		exit 1
	fi
done
while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log
do
	if test "$iter" -ge 4
	then
		echo "Iteration limit: $iter passes through pdflatex"
		exit 1
	fi
	iter=`expr $iter + 1`
	echo "pdflatex $iter"
	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
	if grep -q '! Emergency stop.' $basename.log
	then
		echo "----- Fatal latex error, see $basename.log for details. -----"
		exit 1
	fi
done
grep "LaTeX Warning:" $basename.log
exit 0

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Paul E. McKenney]

On Fri, Apr 22, 2016 at 09:58:24PM +0900, Akira Yokosawa wrote:
> On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:
> [snip]
> 
> >> Can you try this one?
> >> Thanks, Akira
> > 
> > I did, though for some reason I had to hand-apply it on my test branch.
> > Not sure what "git apply" didn't like.
> 
> Well, I'm afraid you tried to apply it after [PATCH 2/2] applied.
> I should have made it clear it was a diff from master at that moment.
> Sorry for your trouble.

I am pretty sure that I was applying it to the current master,
but I cannot prove anything at this point.  ;-)

> > It does repeat upon undefined, which is good.  Nicer behavior after
> > bibliography changes!
> > 
> > There is another "Fatal latex error" earlier that I missed, and it would
> > be good to have an "exit 1" on that one as well.
> > 
> > The one thing I am still concerned about is a real undefined reference,
> > which would consume a full five iterations.  Not really sure what would
> > be improved behavior, though.  Thoughts?
> > 
> > 							Thanx, Paul
> 
> So your question is why you need to change the maximum value from 4 to 5,
> isn't it?  That's because the count is checked at the end of the loop,
> and even when the 4th round of pdflatex is successful, the check will
> fire and 'make' will regard it as an error. At such a later stage, 
> remaining warning should only be 'LaTeX Warning: Label(s) may have changed',
> if any.

Yes, I do understand that.  Your (very welcome!) changes to the quick-quiz
links can require additional passes through latex.  Therefore, the script
should change to provide this.

To see what I am getting at, please add a "\ref{this is a typo}" or a
reference to any other non-\label string, then type "make".  This is a
pure error that no number of reruns will fix, but the build script will
nevertheless crank away multiple times.

Before, it would run bibtex and stop.  (Which, I freely admit, wasn't
particularly good behavior either!)

My question is whether it is possible to make the script let the user
know why it is re-running latex, so that the user can choose to kill
it early, if appropriate.  Or, alternatively (and for extra credit!),
recognize which labels are undefined, and do more passes if they are
quick-quiz links.

> So I made a fairly invasive modification to the script so that the count
> is checked at the beginning of the loop. Also the loop is separated for each
> of the warning message.
> 
> Appended is the whole source code of the script (without the comment part at
> the head). I'm not sure which point should I make a diff from. Note that
> I truncated the warning messages given to grep. They should be long enough.

Good idea!

> What do you think?

Please see below.

							Thanx, Paul

> If it is OK, I'll resubmit the V2 of the patch series relative to the
> current master.
> 
>                                                        Thanks, Akira.
> 
> --- 
> #!/bin/sh
> # [snip]
> #
> 
> if test -z "$1"
> then
> 	echo No latex file, aborting.
> 	exit 1
> fi
> 
> basename=`echo $1 | sed -e 's/\.tex$//'`
> 
> iter=1
> echo "pdflatex $iter"
> pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> if grep -q '! Emergency stop.' $basename.log
> then
> 	echo "----- Fatal latex error, see $basename.log for details. -----"

Please add "exit 1" here.  (Yes, this was a bug in my orignal.  But as
long as we are here...)

> fi
> if grep -q 'LaTeX Warning: There were undefined references' $basename.log
> then
> 	if test -d "$2"
> 	then
> 		bibtex $basename || :
> 	else
> 		echo "No bibliography directory, skipping bibtex."
> 	fi
> fi
> while grep -q 'LaTeX Warning: There were undefined references' $basename.log
> do
> 	if test "$iter" -eq 4
> 	then
> 		echo "Iteration limit: $iter passes through pdflatex"
> 		exit 1
> 	fi
> 	iter=`expr $iter + 1`

This would be one point to add some indication of why we are doing
another pass through the loop.  Might be as simple as adding "Undefined
references" to the "echo" below.  Or maybe even show the last symbol
listed as undefined.

> 	echo "pdflatex $iter"
> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> 	if grep -q '! Emergency stop.' $basename.log
> 	then
> 		echo "----- Fatal latex error, see $basename.log for details. -----"
> 		exit 1
> 	fi
> done
> while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log
> do
> 	if test "$iter" -ge 4
> 	then
> 		echo "Iteration limit: $iter passes through pdflatex"
> 		exit 1
> 	fi
> 	iter=`expr $iter + 1`

This would be one point to add some indication of why we are doing
another pass through the loop.  Might be as simple as adding "Label(s)
may have changed" to the "echo" below.

> 	echo "pdflatex $iter"
> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> 	if grep -q '! Emergency stop.' $basename.log
> 	then
> 		echo "----- Fatal latex error, see $basename.log for details. -----"
> 		exit 1
> 	fi
> done
> grep "LaTeX Warning:" $basename.log
> exit 0
> 

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Akira Yokosawa]

Hi, 

On 2016/04/22 06:34 -0700, Paul E. McKenney wrote:
> On Fri, Apr 22, 2016 at 09:58:24PM +0900, Akira Yokosawa wrote:
>> On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:

Oh, I miss-converted the date. This should have been "On 2016/04/21 16:54 -0700."

>> [snip]
>>
>>>> Can you try this one?
>>>> Thanks, Akira
>>>
>>> I did, though for some reason I had to hand-apply it on my test branch.
>>> Not sure what "git apply" didn't like.
>>
>> Well, I'm afraid you tried to apply it after [PATCH 2/2] applied.
>> I should have made it clear it was a diff from master at that moment.
>> Sorry for your trouble.
> 
> I am pretty sure that I was applying it to the current master,
> but I cannot prove anything at this point.  ;-)
> 
>>> It does repeat upon undefined, which is good.  Nicer behavior after
>>> bibliography changes!
>>>
>>> There is another "Fatal latex error" earlier that I missed, and it would
>>> be good to have an "exit 1" on that one as well.
>>>
>>> The one thing I am still concerned about is a real undefined reference,
>>> which would consume a full five iterations.  Not really sure what would
>>> be improved behavior, though.  Thoughts?
>>>
>>> 							Thanx, Paul
>>
>> So your question is why you need to change the maximum value from 4 to 5,
>> isn't it?  That's because the count is checked at the end of the loop,
>> and even when the 4th round of pdflatex is successful, the check will
>> fire and 'make' will regard it as an error. At such a later stage, 
>> remaining warning should only be 'LaTeX Warning: Label(s) may have changed',
>> if any.
> 
> Yes, I do understand that.  Your (very welcome!) changes to the quick-quiz
> links can require additional passes through latex.  Therefore, the script
> should change to provide this.
> 
> To see what I am getting at, please add a "\ref{this is a typo}" or a
> reference to any other non-\label string, then type "make".  This is a
> pure error that no number of reruns will fix, but the build script will
> nevertheless crank away multiple times.
> 
> Before, it would run bibtex and stop.  (Which, I freely admit, wasn't
> particularly good behavior either!)
> 
> My question is whether it is possible to make the script let the user
> know why it is re-running latex, so that the user can choose to kill
> it early, if appropriate.  Or, alternatively (and for extra credit!),
> recognize which labels are undefined, and do more passes if they are
> quick-quiz links.
>

I see. I'll try to add those features and improve the behavior. I'm not sure
how far I can do, but please wait for a while. And thank you so much for
your valuable suggestions.

                                                      Regards, Akira

>> So I made a fairly invasive modification to the script so that the count
>> is checked at the beginning of the loop. Also the loop is separated for each
>> of the warning message.
>>
>> Appended is the whole source code of the script (without the comment part at
>> the head). I'm not sure which point should I make a diff from. Note that
>> I truncated the warning messages given to grep. They should be long enough.
> 
> Good idea!
> 
>> What do you think?
> 
> Please see below.
> 
> 							Thanx, Paul
> 
>> If it is OK, I'll resubmit the V2 of the patch series relative to the
>> current master.
>>
>>                                                        Thanks, Akira.
>>
>> --- 
>> #!/bin/sh
>> # [snip]
>> #
>>
>> if test -z "$1"
>> then
>> 	echo No latex file, aborting.
>> 	exit 1
>> fi
>>
>> basename=`echo $1 | sed -e 's/\.tex$//'`
>>
>> iter=1
>> echo "pdflatex $iter"
>> pdflatex $basename > /dev/null 2>&1 < /dev/null || :
>> if grep -q '! Emergency stop.' $basename.log
>> then
>> 	echo "----- Fatal latex error, see $basename.log for details. -----"
> 
> Please add "exit 1" here.  (Yes, this was a bug in my orignal.  But as
> long as we are here...)
> 
>> fi
>> if grep -q 'LaTeX Warning: There were undefined references' $basename.log
>> then
>> 	if test -d "$2"
>> 	then
>> 		bibtex $basename || :
>> 	else
>> 		echo "No bibliography directory, skipping bibtex."
>> 	fi
>> fi
>> while grep -q 'LaTeX Warning: There were undefined references' $basename.log
>> do
>> 	if test "$iter" -eq 4
>> 	then
>> 		echo "Iteration limit: $iter passes through pdflatex"
>> 		exit 1
>> 	fi
>> 	iter=`expr $iter + 1`
> 
> This would be one point to add some indication of why we are doing
> another pass through the loop.  Might be as simple as adding "Undefined
> references" to the "echo" below.  Or maybe even show the last symbol
> listed as undefined.
> 
>> 	echo "pdflatex $iter"
>> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
>> 	if grep -q '! Emergency stop.' $basename.log
>> 	then
>> 		echo "----- Fatal latex error, see $basename.log for details. -----"
>> 		exit 1
>> 	fi
>> done
>> while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log
>> do
>> 	if test "$iter" -ge 4
>> 	then
>> 		echo "Iteration limit: $iter passes through pdflatex"
>> 		exit 1
>> 	fi
>> 	iter=`expr $iter + 1`
> 
> This would be one point to add some indication of why we are doing
> another pass through the loop.  Might be as simple as adding "Label(s)
> may have changed" to the "echo" below.
> 
>> 	echo "pdflatex $iter"
>> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
>> 	if grep -q '! Emergency stop.' $basename.log
>> 	then
>> 		echo "----- Fatal latex error, see $basename.log for details. -----"
>> 		exit 1
>> 	fi
>> done
>> grep "LaTeX Warning:" $basename.log
>> exit 0
>>
> 
> 

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Paul E. McKenney]

On Fri, Apr 22, 2016 at 11:20:24PM +0900, Akira Yokosawa wrote:
> Hi, 
> 
> On 2016/04/22 06:34 -0700, Paul E. McKenney wrote:
> > On Fri, Apr 22, 2016 at 09:58:24PM +0900, Akira Yokosawa wrote:
> >> On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:
> 
> Oh, I miss-converted the date. This should have been "On 2016/04/21 16:54 -0700."
> 
> >> [snip]
> >>
> >>>> Can you try this one?
> >>>> Thanks, Akira
> >>>
> >>> I did, though for some reason I had to hand-apply it on my test branch.
> >>> Not sure what "git apply" didn't like.
> >>
> >> Well, I'm afraid you tried to apply it after [PATCH 2/2] applied.
> >> I should have made it clear it was a diff from master at that moment.
> >> Sorry for your trouble.
> > 
> > I am pretty sure that I was applying it to the current master,
> > but I cannot prove anything at this point.  ;-)
> > 
> >>> It does repeat upon undefined, which is good.  Nicer behavior after
> >>> bibliography changes!
> >>>
> >>> There is another "Fatal latex error" earlier that I missed, and it would
> >>> be good to have an "exit 1" on that one as well.
> >>>
> >>> The one thing I am still concerned about is a real undefined reference,
> >>> which would consume a full five iterations.  Not really sure what would
> >>> be improved behavior, though.  Thoughts?
> >>>
> >>> 							Thanx, Paul
> >>
> >> So your question is why you need to change the maximum value from 4 to 5,
> >> isn't it?  That's because the count is checked at the end of the loop,
> >> and even when the 4th round of pdflatex is successful, the check will
> >> fire and 'make' will regard it as an error. At such a later stage, 
> >> remaining warning should only be 'LaTeX Warning: Label(s) may have changed',
> >> if any.
> > 
> > Yes, I do understand that.  Your (very welcome!) changes to the quick-quiz
> > links can require additional passes through latex.  Therefore, the script
> > should change to provide this.
> > 
> > To see what I am getting at, please add a "\ref{this is a typo}" or a
> > reference to any other non-\label string, then type "make".  This is a
> > pure error that no number of reruns will fix, but the build script will
> > nevertheless crank away multiple times.
> > 
> > Before, it would run bibtex and stop.  (Which, I freely admit, wasn't
> > particularly good behavior either!)
> > 
> > My question is whether it is possible to make the script let the user
> > know why it is re-running latex, so that the user can choose to kill
> > it early, if appropriate.  Or, alternatively (and for extra credit!),
> > recognize which labels are undefined, and do more passes if they are
> > quick-quiz links.
> 
> I see. I'll try to add those features and improve the behavior. I'm not sure
> how far I can do, but please wait for a while. And thank you so much for
> your valuable suggestions.

Sounds very good!  Thank you for your work thus far on improving the
quick-quiz links, and I look forward to seeing your patches!

>                                                       Regards, Akira
> 
> >> So I made a fairly invasive modification to the script so that the count
> >> is checked at the beginning of the loop. Also the loop is separated for each
> >> of the warning message.
> >>
> >> Appended is the whole source code of the script (without the comment part at
> >> the head). I'm not sure which point should I make a diff from. Note that
> >> I truncated the warning messages given to grep. They should be long enough.
> > 
> > Good idea!
> > 
> >> What do you think?
> > 
> > Please see below.
> > 
> > 							Thanx, Paul
> > 
> >> If it is OK, I'll resubmit the V2 of the patch series relative to the
> >> current master.
> >>
> >>                                                        Thanks, Akira.
> >>
> >> --- 
> >> #!/bin/sh
> >> # [snip]
> >> #
> >>
> >> if test -z "$1"
> >> then
> >> 	echo No latex file, aborting.
> >> 	exit 1
> >> fi
> >>
> >> basename=`echo $1 | sed -e 's/\.tex$//'`
> >>
> >> iter=1
> >> echo "pdflatex $iter"
> >> pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> >> if grep -q '! Emergency stop.' $basename.log
> >> then
> >> 	echo "----- Fatal latex error, see $basename.log for details. -----"
> > 
> > Please add "exit 1" here.  (Yes, this was a bug in my orignal.  But as
> > long as we are here...)
> > 
> >> fi
> >> if grep -q 'LaTeX Warning: There were undefined references' $basename.log
> >> then
> >> 	if test -d "$2"
> >> 	then
> >> 		bibtex $basename || :
> >> 	else
> >> 		echo "No bibliography directory, skipping bibtex."
> >> 	fi
> >> fi
> >> while grep -q 'LaTeX Warning: There were undefined references' $basename.log
> >> do
> >> 	if test "$iter" -eq 4
> >> 	then
> >> 		echo "Iteration limit: $iter passes through pdflatex"
> >> 		exit 1
> >> 	fi
> >> 	iter=`expr $iter + 1`
> > 
> > This would be one point to add some indication of why we are doing
> > another pass through the loop.  Might be as simple as adding "Undefined
> > references" to the "echo" below.  Or maybe even show the last symbol
> > listed as undefined.
> > 
> >> 	echo "pdflatex $iter"
> >> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> >> 	if grep -q '! Emergency stop.' $basename.log
> >> 	then
> >> 		echo "----- Fatal latex error, see $basename.log for details. -----"
> >> 		exit 1
> >> 	fi
> >> done
> >> while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log
> >> do
> >> 	if test "$iter" -ge 4
> >> 	then
> >> 		echo "Iteration limit: $iter passes through pdflatex"
> >> 		exit 1
> >> 	fi
> >> 	iter=`expr $iter + 1`
> > 
> > This would be one point to add some indication of why we are doing
> > another pass through the loop.  Might be as simple as adding "Label(s)
> > may have changed" to the "echo" below.
> > 
> >> 	echo "pdflatex $iter"
> >> 	pdflatex $basename > /dev/null 2>&1 < /dev/null || :
> >> 	if grep -q '! Emergency stop.' $basename.log
> >> 	then
> >> 		echo "----- Fatal latex error, see $basename.log for details. -----"
> >> 		exit 1
> >> 	fi
> >> done
> >> grep "LaTeX Warning:" $basename.log
> >> exit 0
> >>
> > 
> > 
> 

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Akira Yokosawa]

Hi, Paul.

I'm tempted to move the bibtex part out from runlatex.sh into Makefile.
If I make targets for .bbl files in Makefile, then the .bbl files won't be
necessary in the repository any more.

I'm wondering why those .bbl files are in the repository in the first place.
Was there any reason I can't think of?

Yes, I know controlling latex with Makefile is sometimes tricky because of
its side effects, but I think I can manage.

Just want to confirm the circumstances.

                                                        Thanks, Akira

Re: [PATCH 2/2] runlatex.sh: Fix while loop condition

[Paul E. McKenney]

On Sat, Apr 23, 2016 at 11:14:04AM +0900, Akira Yokosawa wrote:
> Hi, Paul.
> 
> I'm tempted to move the bibtex part out from runlatex.sh into Makefile.
> If I make targets for .bbl files in Makefile, then the .bbl files won't be
> necessary in the repository any more.
> 
> I'm wondering why those .bbl files are in the repository in the first place.
> Was there any reason I can't think of?
> 
> Yes, I know controlling latex with Makefile is sometimes tricky because of
> its side effects, but I think I can manage.
> 
> Just want to confirm the circumstances.

History.  That and laziness.  ;-)

Originally, the bib/*.bib files were not present, so the .bbl file
was absolutely necessary for anyone but me to build the book.  I
eventually got the bib/*.bib files to where they could be placed
in the archive, but never bothered updating things to get the .bbl
files out of git control.

Please do feel free to try updating the build as you suggest.

							Thanx, Paul