[Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Michael Tokarev]

Currently our texi2pod ignores @documentencoding even if it is set
properly in *.texi files.  This results in a mojibake in documents
generated from qemu.pod (which is generated from qemu-doc.texi by
texi2pod), because the rest of the tools assumes ASCII encoding.

This patch recognizes first @documentencoding in input and places
it at the beginning of output as =encoding directive.

Signed-Off-By: Michael Tokarev <mjt@tls.msk.ru>
---
 scripts/texi2pod.pl |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
index 9ed056a..94097fb 100755
--- a/scripts/texi2pod.pl
+++ b/scripts/texi2pod.pl
@@ -36,6 +36,7 @@ $fnno = 1;
 $inf = "";
 $ibase = "";
 @ipath = ();
+$encoding = undef;
 
 while ($_ = shift) {
     if (/^-D(.*)$/) {
@@ -97,6 +98,12 @@ while(<$inf>) {
     /^\@setfilename\s+([^.]+)/ and $fn = $1, next;
     /^\@settitle\s+([^.]+)/ and $tl = postprocess($1), next;
 
+    # Look for document encoding
+    /^\@documentencoding\s+([^.]+)/ and do {
+        $encoding = $1 unless defined $encoding;
+        next;
+    };
+
     # Identify a man title but keep only the one we are interested in.
     /^\@c\s+man\s+title\s+([A-Za-z0-9-]+)\s+(.+)/ and do {
 	if (exists $defs{$1}) {
@@ -336,6 +343,8 @@ $inf = pop @instack;
 
 die "No filename or title\n" unless defined $fn && defined $tl;
 
+print "=encoding $encoding\n\n" if defined $encoding;
+
 $sects{NAME} = "$fn \- $tl\n";
 $sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
 
-- 
1.7.2.5

[Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Michael Tokarev]

This option makes no difference for manpages which contains only
ascii chars.  But for manpages with actual UTF8 characters (qemu
docs contains these), this change allows to see real characters
instead of mojibakes or substitutes.

Signed-off-By: Michael Tokarev <mjt@tls.msk.ru>
---
 Makefile |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 2560b59..737cda2 100644
--- a/Makefile
+++ b/Makefile
@@ -337,28 +337,29 @@ QMP/qmp-commands.txt: $(SRC_PATH)/qmp-commands.hx
 qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx
 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"  GEN   $@")
 
+POD2MAN = pod2man --utf8
 qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi
 	$(call quiet-command, \
 	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu.pod && \
-	  pod2man --section=1 --center=" " --release=" " qemu.pod > $@, \
+	  $(POD2MAN) --section=1 --center=" " --release=" " qemu.pod > $@, \
 	  "  GEN   $@")
 
 qemu-img.1: qemu-img.texi qemu-img-cmds.texi
 	$(call quiet-command, \
 	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-img.pod && \
-	  pod2man --section=1 --center=" " --release=" " qemu-img.pod > $@, \
+	  $(POD2MAN) --section=1 --center=" " --release=" " qemu-img.pod > $@, \
 	  "  GEN   $@")
 
 fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
 	$(call quiet-command, \
 	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< fsdev/virtfs-proxy-helper.pod && \
-	  pod2man --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
+	  $(POD2MAN) --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
 	  "  GEN   $@")
 
 qemu-nbd.8: qemu-nbd.texi
 	$(call quiet-command, \
 	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-nbd.pod && \
-	  pod2man --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
+	  $(POD2MAN) --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
 	  "  GEN   $@")
 
 dvi: qemu-doc.dvi qemu-tech.dvi
-- 
1.7.2.5

Re: [Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Peter Maydell]

On 2 February 2012 14:16, Michael Tokarev <mjt@tls.msk.ru> wrote:
> +POD2MAN = pod2man --utf8
>  qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi
>        $(call quiet-command, \
>          perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu.pod && \
> -         pod2man --section=1 --center=" " --release=" " qemu.pod > $@, \
> +         $(POD2MAN) --section=1 --center=" " --release=" " qemu.pod > $@, \
>          "  GEN   $@")
>
>  qemu-img.1: qemu-img.texi qemu-img-cmds.texi
>        $(call quiet-command, \
>          perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-img.pod && \
> -         pod2man --section=1 --center=" " --release=" " qemu-img.pod > $@, \
> +         $(POD2MAN) --section=1 --center=" " --release=" " qemu-img.pod > $@, \
>          "  GEN   $@")
>
>  fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
>        $(call quiet-command, \
>          perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< fsdev/virtfs-proxy-helper.pod && \
> -         pod2man --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
> +         $(POD2MAN) --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
>          "  GEN   $@")
>
>  qemu-nbd.8: qemu-nbd.texi
>        $(call quiet-command, \
>          perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-nbd.pod && \
> -         pod2man --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
> +         $(POD2MAN) --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
>          "  GEN   $@")

So why does '--utf8' qualify as a common option that we specify just once in
the POD2MAN variable, but --center=" " and --release=" " don't?

Maybe we could reduce the duplication here with a makefile function?

-- PMM

Re: [Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Michael Tokarev]

On 02.02.2012 18:32, Peter Maydell wrote:
> On 2 February 2012 14:16, Michael Tokarev <mjt@tls.msk.ru> wrote:
>> +POD2MAN = pod2man --utf8
>>  qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi
>>        $(call quiet-command, \
>>          perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu.pod && \
>> -         pod2man --section=1 --center=" " --release=" " qemu.pod > $@, \
>> +         $(POD2MAN) --section=1 --center=" " --release=" " qemu.pod > $@, \
[]
> So why does '--utf8' qualify as a common option that we specify just once in
> the POD2MAN variable, but --center=" " and --release=" " don't?

Maybe --release is ok here, but I don't think --center is.  Because, well,
--release is the same for every manpage (it corresponds to qemu release
number of something of this kind), but --center may actually be used one
day with different values for different manpages.

Besides, this is a task for another patch, since this one "only" thing
this patch does is addresses the --utf8 issue.  Maybe it is so trivial
that adding --release here actually does fit nicely too.  I don't care
either way, what matters is to fix the mojibake which currently exists
in qemu.1 manpage.

> Maybe we could reduce the duplication here with a makefile function?

I'm not sure it is worth the extra complexity.  The way it is now is
more visible than a function.  Just IMHO anyway.

Thanks,

/mjt

Re: [Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Peter Maydell]

On 2 February 2012 14:46, Michael Tokarev <mjt@tls.msk.ru> wrote:
> Besides, this is a task for another patch, since this one "only" thing
> this patch does is addresses the --utf8 issue.  Maybe it is so trivial
> that adding --release here actually does fit nicely too.  I don't care
> either way, what matters is to fix the mojibake which currently exists
> in qemu.1 manpage.

If you don't care about fixing the duplication (which I understand)
you could just add --utf8 to all the pod2man command lines.

-- PMM

Re: [Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Michael Tokarev]

On 02.02.2012 18:49, Peter Maydell wrote:
> On 2 February 2012 14:46, Michael Tokarev <mjt@tls.msk.ru> wrote:
>> Besides, this is a task for another patch, since this one "only" thing
>> this patch does is addresses the --utf8 issue.  Maybe it is so trivial
>> that adding --release here actually does fit nicely too.  I don't care
>> either way, what matters is to fix the mojibake which currently exists
>> in qemu.1 manpage.
> 
> If you don't care about fixing the duplication (which I understand)
> you could just add --utf8 to all the pod2man command lines.

The good side-effect of doing it like this is an ability to override
POD2MAN on command line, just in case --utf8 does not work somehow.

Thanks,

/mjt

Re: [Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Michael Tokarev]

Ping?  It's been more than a month since this patch has been posted.

Maybe it is a good candidate for -trivial queue?

Thanks,

/mjt

On 02.02.2012 18:16, Michael Tokarev wrote:
> Currently our texi2pod ignores @documentencoding even if it is set
> properly in *.texi files.  This results in a mojibake in documents
> generated from qemu.pod (which is generated from qemu-doc.texi by
> texi2pod), because the rest of the tools assumes ASCII encoding.
> 
> This patch recognizes first @documentencoding in input and places
> it at the beginning of output as =encoding directive.
> 
> Signed-Off-By: Michael Tokarev <mjt@tls.msk.ru>
> ---
>  scripts/texi2pod.pl |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
> 
> diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
> index 9ed056a..94097fb 100755
> --- a/scripts/texi2pod.pl
> +++ b/scripts/texi2pod.pl
> @@ -36,6 +36,7 @@ $fnno = 1;
>  $inf = "";
>  $ibase = "";
>  @ipath = ();
> +$encoding = undef;
>  
>  while ($_ = shift) {
>      if (/^-D(.*)$/) {
> @@ -97,6 +98,12 @@ while(<$inf>) {
>      /^\@setfilename\s+([^.]+)/ and $fn = $1, next;
>      /^\@settitle\s+([^.]+)/ and $tl = postprocess($1), next;
>  
> +    # Look for document encoding
> +    /^\@documentencoding\s+([^.]+)/ and do {
> +        $encoding = $1 unless defined $encoding;
> +        next;
> +    };
> +
>      # Identify a man title but keep only the one we are interested in.
>      /^\@c\s+man\s+title\s+([A-Za-z0-9-]+)\s+(.+)/ and do {
>  	if (exists $defs{$1}) {
> @@ -336,6 +343,8 @@ $inf = pop @instack;
>  
>  die "No filename or title\n" unless defined $fn && defined $tl;
>  
> +print "=encoding $encoding\n\n" if defined $encoding;
> +
>  $sects{NAME} = "$fn \- $tl\n";
>  $sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
>  

Re: [Qemu-devel] [PATCH 2/2] Run pod2man with --utf8 option to enable utf8 in manpages

[Michael Tokarev]

Ping?  It's been more than a month since this patch has been posted.

Maybe it is a good candidate for -trivial queue?

Thanks,

/mjt

On 02.02.2012 18:16, Michael Tokarev wrote:
> This option makes no difference for manpages which contains only
> ascii chars.  But for manpages with actual UTF8 characters (qemu
> docs contains these), this change allows to see real characters
> instead of mojibakes or substitutes.
> 
> Signed-off-By: Michael Tokarev <mjt@tls.msk.ru>
> ---
>  Makefile |    9 +++++----
>  1 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 2560b59..737cda2 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -337,28 +337,29 @@ QMP/qmp-commands.txt: $(SRC_PATH)/qmp-commands.hx
>  qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx
>  	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"  GEN   $@")
>  
> +POD2MAN = pod2man --utf8
>  qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi
>  	$(call quiet-command, \
>  	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu.pod && \
> -	  pod2man --section=1 --center=" " --release=" " qemu.pod > $@, \
> +	  $(POD2MAN) --section=1 --center=" " --release=" " qemu.pod > $@, \
>  	  "  GEN   $@")
>  
>  qemu-img.1: qemu-img.texi qemu-img-cmds.texi
>  	$(call quiet-command, \
>  	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-img.pod && \
> -	  pod2man --section=1 --center=" " --release=" " qemu-img.pod > $@, \
> +	  $(POD2MAN) --section=1 --center=" " --release=" " qemu-img.pod > $@, \
>  	  "  GEN   $@")
>  
>  fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
>  	$(call quiet-command, \
>  	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< fsdev/virtfs-proxy-helper.pod && \
> -	  pod2man --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
> +	  $(POD2MAN) --section=1 --center=" " --release=" " fsdev/virtfs-proxy-helper.pod > $@, \
>  	  "  GEN   $@")
>  
>  qemu-nbd.8: qemu-nbd.texi
>  	$(call quiet-command, \
>  	  perl -Ww -- $(SRC_PATH)/scripts/texi2pod.pl $< qemu-nbd.pod && \
> -	  pod2man --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
> +	  $(POD2MAN) --section=8 --center=" " --release=" " qemu-nbd.pod > $@, \
>  	  "  GEN   $@")
>  
>  dvi: qemu-doc.dvi qemu-tech.dvi

Re: [Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Stefan Hajnoczi]

On Sun, Mar 11, 2012 at 01:56:41PM +0400, Michael Tokarev wrote:
> Ping?  It's been more than a month since this patch has been posted.
> 
> Maybe it is a good candidate for -trivial queue?

One thing I don't understand is where you actually use
@documentencoding?  I don't see it in you 2 patches.

Stefan

Re: [Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Michael Tokarev]

On 12.03.2012 12:21, Stefan Hajnoczi wrote:
> On Sun, Mar 11, 2012 at 01:56:41PM +0400, Michael Tokarev wrote:
>> Ping?  It's been more than a month since this patch has been posted.
>>
>> Maybe it is a good candidate for -trivial queue?
> 
> One thing I don't understand is where you actually use
> @documentencoding?  I don't see it in you 2 patches.

It is already used in the existing docs, that's the
whole point ;)

Thanks,

/mjt

Re: [Qemu-devel] [PATCH 1/2] Support @documentencoding in scripts/texi2pod.pl

[Stefan Hajnoczi]

On Mon, Mar 12, 2012 at 10:14 AM, Michael Tokarev <mjt@tls.msk.ru> wrote:
> On 12.03.2012 12:21, Stefan Hajnoczi wrote:
>> On Sun, Mar 11, 2012 at 01:56:41PM +0400, Michael Tokarev wrote:
>>> Ping?  It's been more than a month since this patch has been posted.
>>>
>>> Maybe it is a good candidate for -trivial queue?
>>
>> One thing I don't understand is where you actually use
>> @documentencoding?  I don't see it in you 2 patches.
>
> It is already used in the existing docs, that's the
> whole point ;)

Ah, it would have been good to explain that in the commit description :).

Stefan