[PATCH v2 0/4] docs/html/: Hypercall docs from headers

[PATCH v2 0/4] docs/html/: Hypercall docs from headers

[Ian Jackson]

This is the revised version of my hypercall docs series:
 - Addresses all comments from v1
 - Renamed output directory to docs/html/hypercall

[PATCH 1/4] docs/html/: Initial cut of header documentation massager

[Ian Jackson]

"xen-headers" generates HTML from header files.  So far this generates
just some type cross-references, if you say
   make -C docs html/hypercall/stamp

An index page, proper wiring into the build system, and a few more
annotations in the headers, and will be forthcoming.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 docs/Makefile    |    8 ++
 docs/xen-headers |  292 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 300 insertions(+), 0 deletions(-)
 create mode 100755 docs/xen-headers

diff --git a/docs/Makefile b/docs/Makefile
index 2054541..ad315cf 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -129,6 +129,14 @@ html/%.html: %.markdown
 	$(MARKDOWN) $< > $@.tmp ; \
 	$(call move-if-changed,$@.tmp,$@) ; fi
 
+html/hypercall/stamp:
+	@$(INSTALL_DIR) $(@D)
+	./xen-headers -O $(@D) \
+		-T 'arch-x86_64 - Xen public headers' \
+		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 \
+		../xen include/public
+	touch $@
+
 txt/%.txt: %.txt
 	$(INSTALL_DIR) $(@D)
 	cp $< $@.tmp
diff --git a/docs/xen-headers b/docs/xen-headers
new file mode 100755
index 0000000..e893f91
--- /dev/null
+++ b/docs/xen-headers
@@ -0,0 +1,292 @@
+#!/usr/bin/perl -w
+# usage: xen-headers OPTIONS... BASE-DIR INPUT-SUB-DIR...
+#  INPUT-SUB-DIR must be a relative path, and is interpreted
+#  relative to BASE-DIR.  Only files whose names end .h are processed
+# options:
+#   -O HTML-DIR             write html to this directory (mandatory)
+#   -T EXTRA-TITLE-HTML     tail of title string (used in <title>)
+#   -X GLOB | -I GLOB       include/exclude files matching;
+#                            glob patterns matched against /INPUT-SUB-FILE
+#                            first match wins; if no match, files included
+#   -D                      increase debug
+
+# Functionality:
+#  enum values --> selected function or struct
+#  type & function names, macro definitions --> definition
+#  function or struct selected by enum ++> ref to enum value
+
+#  definitions must start in LH column
+#  extra syntax:
+#    /* ` <definition>                          } parse as if <definition>
+#     * ` <definition>                          }  was not commented
+#   enum <name> { // <pattern>* => <func>()     } cross-reference
+#   enum <name> { // <pattern>* => struct <s>   }  enum values
+#   
+
+# 1st pass: find where things are defined and what references are wanted
+# 2rd pass: write out output
+
+use strict;
+use warnings;
+
+use Getopt::Long;
+use File::Find;
+use IO::File;
+
+Getopt::Long::Configure('bundling');
+
+our $outdir;
+our $debug=0;
+our $xtitle='';
+our @fglobs;
+
+sub includeexclude {
+    my ($yn, $optobj, $value) = @_;
+    push @fglobs, [ $value, $yn ];
+}
+
+GetOptions("O|output-dir=s" => \$outdir,
+           "D+" => \$debug,
+           "T=s" => \$xtitle,
+	   "I=s" => sub { includeexclude(1, @_); },
+	   "X=s" => sub { includeexclude(0, @_); })
+    or die;
+
+die unless defined $outdir;
+@ARGV>=2 or die;
+
+my ($basedir,@indirs) = @ARGV;
+
+# general globals
+our $pass;
+our %sdef; 
+# $sdef{$type}{$name} => {
+#     DefLocs => { "$leaf_path:$lineno" => $leaf_opath ,... }
+#     Xrefs => { "$leaf_path,$lineno" => "$xref", ... }
+#     Used => 1
+# }
+# $type might be  Func Struct Union Enum EnumVal
+
+# provided by the find() function
+our $leaf;
+our $leaf_opath;
+
+# reset at the start of each file
+our $o;
+our $in_enum;
+our @pending_xrefs;
+
+sub compile_fglobs () {
+    local ($_);
+    my $f = "sub file_wanted (\$) {\n    local (\$_) = \"/\$leaf\";\n";
+    foreach my $fglob (@fglobs) {
+	$_ = $fglob->[0];
+	$_ = "**$_**" unless m/[?*]/;
+	s/\W/\\$&/g;
+	s,\\\*\\\*,.*,g;
+	s,\\\*,[^/]*,g;
+	s,\\\?,[^/],g;
+	$f .= "    return $fglob->[1] if m,$_,o;\n";
+    }
+    $f .= "    return 1;\n}\n1;\n";
+    debug(3, $f);
+    eval $f or die "$@ ";
+}
+
+compile_fglobs();
+
+
+sub warning {
+    print STDERR "$leaf:$.: @_\n";
+}
+
+sub debug {
+    my $msglevel = scalar shift @_;
+    return unless $debug >= $msglevel;
+    print STDERR "DEBUG $pass $msglevel @_\n" or die $!;
+}
+
+sub in_enum ($$$) { $in_enum = [ @_ ]; } # [ $enumvalpfx, RefType, $refnamepfx ]
+
+sub aelem ($$$) {
+    my ($ntext,$ytext,$hparams) = @_;
+    return $ntext unless $hparams =~ m/\S/;
+    return "<a $hparams>$ytext</a>";
+}
+
+sub defn ($$$;$) {
+    my ($text,$type,$name,$hparams) = @_;
+    $hparams='' if !defined $hparams;
+    debug(2,"DEFN $. $type $name $hparams");
+    $sdef{$type}{$name}{DefLocs}{"$leaf:$."} = $leaf_opath;
+    my $xrefs = $sdef{$type}{$name}{Xrefs};
+    push @pending_xrefs, values %$xrefs if $xrefs;
+    $hparams .= " name=\"${type}_$name\"" if $sdef{$type}{$name}{Used};
+    return aelem($text, "<strong>$text</strong>", $hparams);
+}
+
+sub norm ($) {
+    local ($_) = @_;
+    my $no = '';
+    while (length) {
+	if (s/^(?:\s|^\W)+//) {
+	    $no .= $&;
+	} elsif (s/^(struct|union|enum)\s+(\w+)\b//) {
+	    $no .= ahref($&, (ucfirst $1), $2);
+	} elsif (s/^\w+\b//) {
+	    $no .= ahref($&, 'Func', $&);
+	} else {
+	    die "$_ ?";
+	}
+    }
+    return $no;
+}
+
+sub refhref ($$) {
+    my ($type,$name) = @_;
+    $sdef{$type}{$name}{Used} = 1;
+    my $locs = $sdef{$type}{$name}{DefLocs};
+    return '' unless $locs;
+    if ((scalar keys %$locs) != 1 && !$sdef{$type}{$name}{MultiWarned}) {
+	warning("multiple definitions of $type $name: $_")
+	    foreach keys %$locs;
+	$sdef{$type}{$name}{MultiWarned}=1;
+    }
+    my ($loc) = values %$locs;
+    return "href=\"$loc#${type}_$name\"";
+}
+
+sub ahref ($$$) {
+    my ($text,$type,$name) = @_;
+    return aelem($text,$text, refhref($type,$name));
+}
+
+sub defmacro ($) {
+    my ($valname) = @_;
+    if (!$in_enum) {
+	return $valname;
+    } elsif (substr($valname, 0, (length $in_enum->[0])) ne $in_enum->[0]) {
+	warning("in enum expecting $in_enum->[0]* got $valname");
+	return $valname;
+    } else {
+	my $reftype = $in_enum->[1];
+	my $refname = $in_enum->[2].substr($valname, (length $in_enum->[0]));
+	$sdef{$reftype}{$refname}{Xrefs}{$leaf,$.} =
+	    "[see <a href=\"$leaf_opath#EnumVal_$valname\">$valname</a>]";
+	$sdef{EnumVal}{$valname}{Used} = 1;
+	return defn($valname,'EnumVal',$valname, refhref($reftype,$refname));
+    }
+}
+
+sub out_xrefs ($) {
+    my ($linemapfunc) = @_;
+    foreach my $xref (@pending_xrefs) {
+	$o .= $linemapfunc->($xref);
+	$o .= "\n";
+    }
+    @pending_xrefs = ();
+}
+
+sub write_file ($$) {
+    my ($opath, $odata) = @_;
+    my $out = new IO::File "$opath.new", '>' or die "$opath $!";
+    print $out $odata or die $!;
+    rename "$opath.new", "$opath" or die "$opath $!";
+}
+
+sub process_file ($$) {
+    my ($infile, $outfile) = @_;
+    debug(1,"$pass $infile => $outfile");
+    my $in = new IO::File "$infile", '<' or die "$infile $!";
+
+    $o = '';
+    $in_enum = undef;
+    @pending_xrefs = ();
+
+    $o .= "<html><head><title>$leaf - $xtitle</title></head><body><pre>\n";
+    
+    while (<$in>) {
+	s/\&/\&amp;/g;
+	s/\</\&lt;/g;
+	s/\>/\&gt;/g;
+
+	if (m/^(.*\`)[ \t]*$/) {
+	    my $lhs = $1;
+	    out_xrefs(sub { "$1 $_[0]"; });
+	} elsif (m/^\s*$/) {
+	    out_xrefs(sub { sprintf "/* %70s */", $_[0]; });
+	}
+
+	# In case of comments, strip " /* ` " and " * ` ";
+	my $lstripped = s,^ \s* /? \* \s* \` \  ,,x ? $&: '';
+
+	# Strip trailing whitespace and perhaps trailing "*/" or "*"
+	s,(?: \s* \* /? )? \s* $,,x or die;
+	my $rstripped = $&;
+
+	# Now the actual functionality:
+
+	debug(3,"$. $_");
+
+	if (!m/^(?: __attribute__ | __pragma__ )\b/x &&
+	    s/^( (?: \w+\  )? ) (\w+[a-z]\w+) ( \( .*)$
+             / $1.defn($2,'Func',$2).norm($3) /xe) {
+	} elsif (s/^((struct|union|enum) \  (\w+)) ( \s+ \{ .* )$
+                  / defn($1,(ucfirst $2),$3).norm($4) /xe) {
+	    if ($2 eq 'enum') {
+		if (m,/[/*] (\w+)\* \=\&gt\; (\w+)\*\(\),) { 
+		    in_enum($1,'Func',$2)
+		} elsif (m,/[/*] (\w+)\* \=\&gt\; (struct) (\w+)\*,) { 
+		    in_enum($1,(ucfirst $2),$3);
+	        }
+	    }
+	} elsif (s/^( \s* \#define \s+ ) (\w+) ( \s+\S )
+                  / $1.defmacro($2).norm($3) /xe) {
+	} else {
+	    if (m/^\s*\}/) {
+		$in_enum = undef;
+	    }
+	    $_ = norm($_);
+	}
+
+	# Write the line out
+
+	if ($pass == 2) {
+	    $o .= $lstripped;
+	    $o .= $_;
+	    $o .= $rstripped;
+	}
+    }
+
+    warning("pending xrefs at end of file") if @pending_xrefs;
+
+    if ($pass == 2) {
+	$o .= "</pre></body></html>";
+	write_file($outfile, $o);
+    }
+}
+
+
+foreach $pass (qw(1 2)) {
+    find({ wanted => 
+	       sub {
+		   return unless m/\.h$/;
+		   lstat $File::Find::name or die "$File::Find::name $!";
+		   -f _ or die "$File::Find::name";
+		   substr($File::Find::name, 0, 1+length $basedir) 
+		       eq "$basedir/"
+		       or die "$File::Find::name $basedir";
+		   $leaf = substr($File::Find::name, 1+length $basedir);
+		   if (!file_wanted()) {
+		       debug(1,"$pass $File::Find::name excluded");
+		       return;
+		   }
+		   $leaf_opath = $leaf;
+		   $leaf_opath =~ s#/#,#g;
+		   $leaf_opath .= ".html";
+		   process_file($File::Find::name, $outdir.'/'.$leaf_opath);
+	   },
+	   no_chdir => 1,
+	 },
+	 map { "$basedir/$_" } @indirs);
+}
-- 
1.7.2.5

[PATCH 2/4] docs/html/: Annotations for two hypercalls

[Ian Jackson]

Add annotations for a couple of the hypercalls:
 HYPERVISOR_set_trap_table
 HYPERVISOR_mmu_update

We do this by
 * annotating the list of #defines for hypercall numbers
 * annotating the list of error values
 * providing a function prototype for the systematically-named functions
The header generator does the rest.

This exercise revealed a couple of infelicities:
 * In the actual source code, do_mmu_update is defined to return an int
   and do_set_trap_table a long.  However both functions return either
   -Efoo (on error) or 0 for success.
 * The error numbers are defined only in the private header file
   xen/include/xen/errno.h and then only with names which will
   typically clash with other projects.  It would be nice to include a
   public version of this header which defines XEN_E*.  But for now
   we run xen-headers on errno.h too.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 docs/Makefile                     |    2 +-
 xen/include/public/arch-x86/xen.h |    6 ++++++
 xen/include/public/xen.h          |   11 +++++++++--
 xen/include/xen/errno.h           |    5 +++++
 4 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/docs/Makefile b/docs/Makefile
index ad315cf..a2bbf2d 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -134,7 +134,7 @@ html/hypercall/stamp:
 	./xen-headers -O $(@D) \
 		-T 'arch-x86_64 - Xen public headers' \
 		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 \
-		../xen include/public
+		../xen include/public include/xen/errno.h
 	touch $@
 
 txt/%.txt: %.txt
diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
index 79ec633..d663cef 100644
--- a/xen/include/public/arch-x86/xen.h
+++ b/xen/include/public/arch-x86/xen.h
@@ -82,7 +82,13 @@ typedef unsigned long xen_pfn_t;
 typedef unsigned long xen_ulong_t;
 
 /*
+ * ` enum neg_errnoval
+ * ` HYPERVISOR_set_trap_table(const struct trap_info traps[]);
+ * `
+ */
+/*
  * Send an array of these to HYPERVISOR_set_trap_table().
+ * Terminate the array with a sentinel entry, with traps[].address==0.
  * The privilege level specifies which modes may enter a trap via a software
  * interrupt. On x86/64, since rings 1 and 2 are unavailable, we allocate
  * privilege levels as follows:
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index fde9fa5..f2c9e6f 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -55,6 +55,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
  * HYPERCALLS
  */
 
+/* ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*() */
+
 #define __HYPERVISOR_set_trap_table        0
 #define __HYPERVISOR_mmu_update            1
 #define __HYPERVISOR_set_gdt               2
@@ -105,6 +107,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
 #define __HYPERVISOR_arch_6               54
 #define __HYPERVISOR_arch_7               55
 
+/* ` } */
+
 /*
  * HYPERCALL COMPATIBILITY.
  */
@@ -163,8 +167,11 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
 #define NR_VIRQS       24
 
 /*
- * HYPERVISOR_mmu_update(reqs, count, pdone, foreigndom)
- * 
+ * ` enum neg_errnoval
+ * ` HYPERVISOR_mmu_update(const struct mmu_update reqs[],
+ * `                       unsigned count, unsigned *done_out,
+ * `                       unsigned foreigndom)
+ * `
  * @reqs is an array of mmu_update_t structures ((ptr, val) pairs).
  * @count is the length of the above array.
  * @pdone is an output parameter indicating number of completed operations
diff --git a/xen/include/xen/errno.h b/xen/include/xen/errno.h
index 7cf599f..39147be 100644
--- a/xen/include/xen/errno.h
+++ b/xen/include/xen/errno.h
@@ -1,6 +1,9 @@
 #ifndef _I386_ERRNO_H
 #define _I386_ERRNO_H
 
+/* ` enum neg_errnoval {  [ -Efoo for each Efoo in the list below ]  } */
+/* ` enum errnoval { */
+
 #define	EPERM		 1	/* Operation not permitted */
 #define	ENOENT		 2	/* No such file or directory */
 #define	ESRCH		 3	/* No such process */
@@ -129,4 +132,6 @@
 #define	ENOMEDIUM	123	/* No medium found */
 #define	EMEDIUMTYPE	124	/* Wrong medium type */
 
+/* ` } */
+
 #endif
-- 
1.7.2.5

[PATCH 3/4] docs/html/: Generate an "index.html" for hypercall docs

[Ian Jackson]

Generate an "index.html" containing various useful links.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 docs/xen-headers         |   59 ++++++++++++++++++++++++++++++++++++++++++++++
 xen/include/public/xen.h |    4 ++-
 2 files changed, 62 insertions(+), 1 deletions(-)

diff --git a/docs/xen-headers b/docs/xen-headers
index e893f91..dcf673e 100755
--- a/docs/xen-headers
+++ b/docs/xen-headers
@@ -17,6 +17,10 @@
 
 #  definitions must start in LH column
 #  extra syntax:
+#   `incontents <seq> <shortname> <anchor text html>...
+#                              make a table of contents entry; they
+#                              will be sorted by increasing seq, and
+#                              shortname will be used as the anchor target
 #    /* ` <definition>                          } parse as if <definition>
 #     * ` <definition>                          }  was not commented
 #   enum <name> { // <pattern>* => <func>()     } cross-reference
@@ -60,6 +64,8 @@ my ($basedir,@indirs) = @ARGV;
 # general globals
 our $pass;
 our %sdef; 
+our @incontents;
+our @outfiles;
 # $sdef{$type}{$name} => {
 #     DefLocs => { "$leaf_path:$lineno" => $leaf_opath ,... }
 #     Xrefs => { "$leaf_path,$lineno" => "$xref", ... }
@@ -187,6 +193,19 @@ sub out_xrefs ($) {
     @pending_xrefs = ();
 }
 
+sub incontents ($$$) {
+    my ($text, $seq, $anchor) = @_;
+    $anchor = "incontents_$anchor";
+    if ($pass==2) {
+	push @incontents, { 
+	    Seq => $seq,
+	    Href => "$leaf_opath#$anchor",
+	    Title => $text,
+	};
+    }
+    return "<a name=\"$anchor\"><strong>$text</strong></a>";
+}
+
 sub write_file ($$) {
     my ($opath, $odata) = @_;
     my $out = new IO::File "$opath.new", '>' or die "$opath $!";
@@ -242,6 +261,8 @@ sub process_file ($$) {
 	    }
 	} elsif (s/^( \s* \#define \s+ ) (\w+) ( \s+\S )
                   / $1.defmacro($2).norm($3) /xe) {
+	} elsif (s/( \`incontents \s+ (\d+) \s+ (\w+) \s+ )(\S .* \S)
+                 / norm($1).incontents($4, $2, $3) /xe) {
 	} else {
 	    if (m/^\s*\}/) {
 		$in_enum = undef;
@@ -261,11 +282,47 @@ sub process_file ($$) {
     warning("pending xrefs at end of file") if @pending_xrefs;
 
     if ($pass == 2) {
+	push @outfiles, [ $leaf, $leaf_opath ];
 	$o .= "</pre></body></html>";
 	write_file($outfile, $o);
     }
 }
 
+sub output_index () {
+    my $title = "contents - $xtitle";
+    $o = '';
+    $o .= <<END;
+<html><head><title>$title</title></head>
+<body>
+<h1>$title</h1>
+<h2>Starting points</h2>
+<ul>
+END
+    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} } @incontents) {
+	$o .= "<li><a href=\"$ic->{Href}\">$ic->{Title}</a></li>\n";
+    }
+    $o .= "</ul>\n";
+    my $forkind = sub {
+	my ($type,$desc,$pfx,$sfx) = @_;
+	$o .= "<h2>$desc</h2><ul>\n";
+	foreach my $name (sort keys %{ $sdef{$type} }) {
+	    my $href = refhref($type,$name);
+	    next unless $href =~ m/\S/;
+	    $o .= "<li><a $href>$pfx$name$sfx</a></li>\n";
+	}
+	$o .= "</ul>\n";
+    };
+    $forkind->('Func','Functions','','()');
+    $forkind->('Struct','Structs','struct ','');
+    $forkind->('Enum','Enums and sets of #defines','','');
+    $forkind->('EnumVal','Enum values and individual #defines','','');
+    $o .= "</ul>\n<h2>Files</h2><ul>\n";
+    foreach my $of (sort { $a->[0] cmp $b->[0] } @outfiles) {
+	$o .= "<li><a href=\"$of->[1]\">$of->[0]</a></li>\n";
+    }
+    $o .= "</ul></body></html>\n";
+    write_file("$outdir/index.html", $o);
+}
 
 foreach $pass (qw(1 2)) {
     find({ wanted => 
@@ -290,3 +347,5 @@ foreach $pass (qw(1 2)) {
 	 },
 	 map { "$basedir/$_" } @indirs);
 }
+
+output_index();
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index f2c9e6f..a5b6ad8 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -55,7 +55,9 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
  * HYPERCALLS
  */
 
-/* ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*() */
+/* `incontents 100 hcalls List of hypercalls
+ * ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*()
+ */
 
 #define __HYPERVISOR_set_trap_table        0
 #define __HYPERVISOR_mmu_update            1
-- 
1.7.2.5

[PATCH 4/4] docs/html/: Arrange for automatic build of hypercall docs

[Ian Jackson]

- Use index.html rather than a stamp file.
- Automatically generate dependencies.
- Wire into the docs build system

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 docs/Makefile    |    9 ++++++---
 docs/xen-headers |   14 ++++++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/docs/Makefile b/docs/Makefile
index a2bbf2d..413c1a2 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -15,7 +15,8 @@ DOC_MARKDOWN	:= $(wildcard misc/*.markdown)
 DOC_PS		:= $(patsubst src/%.tex,ps/%.ps,$(DOC_TEX))
 DOC_PDF		:= $(patsubst src/%.tex,pdf/%.pdf,$(DOC_TEX))
 DOC_HTML	:= $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX)) \
-		   $(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN))
+		   $(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN)) \
+		   html/hypercall/index.html
 DOC_TXT         := $(patsubst %.txt,txt/%.txt,$(wildcard misc/*.txt)) \
 		   $(patsubst %.markdown,txt/%.txt,$(DOC_MARKDOWN))
 
@@ -129,13 +130,15 @@ html/%.html: %.markdown
 	$(MARKDOWN) $< > $@.tmp ; \
 	$(call move-if-changed,$@.tmp,$@) ; fi
 
-html/hypercall/stamp:
+html/hypercall/index.html: ./xen-headers
+	rm -rf $(@D)
 	@$(INSTALL_DIR) $(@D)
 	./xen-headers -O $(@D) \
 		-T 'arch-x86_64 - Xen public headers' \
 		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 \
 		../xen include/public include/xen/errno.h
-	touch $@
+
+-include html/hypercall/.deps
 
 txt/%.txt: %.txt
 	$(INSTALL_DIR) $(@D)
diff --git a/docs/xen-headers b/docs/xen-headers
index dcf673e..8226d73 100755
--- a/docs/xen-headers
+++ b/docs/xen-headers
@@ -325,6 +325,12 @@ END
 }
 
 foreach $pass (qw(1 2)) {
+    my $depspath = "$outdir/.deps";
+    my $depsout;
+    if ($pass==2) {
+	$depsout = new IO::File "$depspath.new", 'w' or die $!;
+    }
+
     find({ wanted => 
 	       sub {
 		   return unless m/\.h$/;
@@ -341,11 +347,19 @@ foreach $pass (qw(1 2)) {
 		   $leaf_opath = $leaf;
 		   $leaf_opath =~ s#/#,#g;
 		   $leaf_opath .= ".html";
+		   print $depsout "$outdir/index.html: $File::Find::name\n"
+		       or die $!
+		       if $pass==2;
 		   process_file($File::Find::name, $outdir.'/'.$leaf_opath);
 	   },
 	   no_chdir => 1,
 	 },
 	 map { "$basedir/$_" } @indirs);
+
+    if ($pass==2) {
+	close $depsout or die $!;
+	rename "$depspath.new", "$depspath" or die $!;
+    }
 }
 
 output_index();
-- 
1.7.2.5

Re: [PATCH 1/4] docs/html/: Initial cut of header documentation massager

[Ian Campbell]

On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> "xen-headers" generates HTML from header files.  So far this generates
> just some type cross-references, if you say
>    make -C docs html/hypercall/stamp
> 
> An index page, proper wiring into the build system, and a few more
> annotations in the headers, and will be forthcoming.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

> [...]

> +GetOptions("O|output-dir=s" => \$outdir,
> +           "D+" => \$debug,
> +           "T=s" => \$xtitle,
> +	   "I=s" => sub { includeexclude(1, @_); },
> +	   "X=s" => sub { includeexclude(0, @_); })

Indentation is messed up (you've got a hard tab from somewhere).

With that fixed: 
Acked-by: Ian Campbell <ian.campbell@citrix.com>

(insofar as I can review Perl code...)

> +    or die;
> +
> +die unless defined $outdir;
> +@ARGV>=2 or die;
> +
> +my ($basedir,@indirs) = @ARGV;
> +
> +# general globals
> +our $pass;
> +our %sdef; 
> +# $sdef{$type}{$name} => {
> +#     DefLocs => { "$leaf_path:$lineno" => $leaf_opath ,... }
> +#     Xrefs => { "$leaf_path,$lineno" => "$xref", ... }
> +#     Used => 1
> +# }
> +# $type might be  Func Struct Union Enum EnumVal
> +
> +# provided by the find() function
> +our $leaf;
> +our $leaf_opath;
> +
> +# reset at the start of each file
> +our $o;
> +our $in_enum;
> +our @pending_xrefs;
> +
> +sub compile_fglobs () {
> +    local ($_);
> +    my $f = "sub file_wanted (\$) {\n    local (\$_) = \"/\$leaf\";\n";
> +    foreach my $fglob (@fglobs) {
> +	$_ = $fglob->[0];
> +	$_ = "**$_**" unless m/[?*]/;
> +	s/\W/\\$&/g;
> +	s,\\\*\\\*,.*,g;
> +	s,\\\*,[^/]*,g;
> +	s,\\\?,[^/],g;
> +	$f .= "    return $fglob->[1] if m,$_,o;\n";
> +    }
> +    $f .= "    return 1;\n}\n1;\n";
> +    debug(3, $f);
> +    eval $f or die "$@ ";
> +}
> +
> +compile_fglobs();
> +
> +
> +sub warning {
> +    print STDERR "$leaf:$.: @_\n";
> +}
> +
> +sub debug {
> +    my $msglevel = scalar shift @_;
> +    return unless $debug >= $msglevel;
> +    print STDERR "DEBUG $pass $msglevel @_\n" or die $!;
> +}
> +
> +sub in_enum ($$$) { $in_enum = [ @_ ]; } # [ $enumvalpfx, RefType, $refnamepfx ]
> +
> +sub aelem ($$$) {
> +    my ($ntext,$ytext,$hparams) = @_;
> +    return $ntext unless $hparams =~ m/\S/;
> +    return "<a $hparams>$ytext</a>";
> +}
> +
> +sub defn ($$$;$) {
> +    my ($text,$type,$name,$hparams) = @_;
> +    $hparams='' if !defined $hparams;
> +    debug(2,"DEFN $. $type $name $hparams");
> +    $sdef{$type}{$name}{DefLocs}{"$leaf:$."} = $leaf_opath;
> +    my $xrefs = $sdef{$type}{$name}{Xrefs};
> +    push @pending_xrefs, values %$xrefs if $xrefs;
> +    $hparams .= " name=\"${type}_$name\"" if $sdef{$type}{$name}{Used};
> +    return aelem($text, "<strong>$text</strong>", $hparams);
> +}
> +
> +sub norm ($) {
> +    local ($_) = @_;
> +    my $no = '';
> +    while (length) {
> +	if (s/^(?:\s|^\W)+//) {
> +	    $no .= $&;
> +	} elsif (s/^(struct|union|enum)\s+(\w+)\b//) {
> +	    $no .= ahref($&, (ucfirst $1), $2);
> +	} elsif (s/^\w+\b//) {
> +	    $no .= ahref($&, 'Func', $&);
> +	} else {
> +	    die "$_ ?";
> +	}
> +    }
> +    return $no;
> +}
> +
> +sub refhref ($$) {
> +    my ($type,$name) = @_;
> +    $sdef{$type}{$name}{Used} = 1;
> +    my $locs = $sdef{$type}{$name}{DefLocs};
> +    return '' unless $locs;
> +    if ((scalar keys %$locs) != 1 && !$sdef{$type}{$name}{MultiWarned}) {
> +	warning("multiple definitions of $type $name: $_")
> +	    foreach keys %$locs;
> +	$sdef{$type}{$name}{MultiWarned}=1;
> +    }
> +    my ($loc) = values %$locs;
> +    return "href=\"$loc#${type}_$name\"";
> +}
> +
> +sub ahref ($$$) {
> +    my ($text,$type,$name) = @_;
> +    return aelem($text,$text, refhref($type,$name));
> +}
> +
> +sub defmacro ($) {
> +    my ($valname) = @_;
> +    if (!$in_enum) {
> +	return $valname;
> +    } elsif (substr($valname, 0, (length $in_enum->[0])) ne $in_enum->[0]) {
> +	warning("in enum expecting $in_enum->[0]* got $valname");
> +	return $valname;
> +    } else {
> +	my $reftype = $in_enum->[1];
> +	my $refname = $in_enum->[2].substr($valname, (length $in_enum->[0]));
> +	$sdef{$reftype}{$refname}{Xrefs}{$leaf,$.} =
> +	    "[see <a href=\"$leaf_opath#EnumVal_$valname\">$valname</a>]";
> +	$sdef{EnumVal}{$valname}{Used} = 1;
> +	return defn($valname,'EnumVal',$valname, refhref($reftype,$refname));
> +    }
> +}
> +
> +sub out_xrefs ($) {
> +    my ($linemapfunc) = @_;
> +    foreach my $xref (@pending_xrefs) {
> +	$o .= $linemapfunc->($xref);
> +	$o .= "\n";
> +    }
> +    @pending_xrefs = ();
> +}
> +
> +sub write_file ($$) {
> +    my ($opath, $odata) = @_;
> +    my $out = new IO::File "$opath.new", '>' or die "$opath $!";
> +    print $out $odata or die $!;
> +    rename "$opath.new", "$opath" or die "$opath $!";
> +}
> +
> +sub process_file ($$) {
> +    my ($infile, $outfile) = @_;
> +    debug(1,"$pass $infile => $outfile");
> +    my $in = new IO::File "$infile", '<' or die "$infile $!";
> +
> +    $o = '';
> +    $in_enum = undef;
> +    @pending_xrefs = ();
> +
> +    $o .= "<html><head><title>$leaf - $xtitle</title></head><body><pre>\n";
> +    
> +    while (<$in>) {
> +	s/\&/\&amp;/g;
> +	s/\</\&lt;/g;
> +	s/\>/\&gt;/g;
> +
> +	if (m/^(.*\`)[ \t]*$/) {
> +	    my $lhs = $1;
> +	    out_xrefs(sub { "$1 $_[0]"; });
> +	} elsif (m/^\s*$/) {
> +	    out_xrefs(sub { sprintf "/* %70s */", $_[0]; });
> +	}
> +
> +	# In case of comments, strip " /* ` " and " * ` ";
> +	my $lstripped = s,^ \s* /? \* \s* \` \  ,,x ? $&: '';
> +
> +	# Strip trailing whitespace and perhaps trailing "*/" or "*"
> +	s,(?: \s* \* /? )? \s* $,,x or die;
> +	my $rstripped = $&;
> +
> +	# Now the actual functionality:
> +
> +	debug(3,"$. $_");
> +
> +	if (!m/^(?: __attribute__ | __pragma__ )\b/x &&
> +	    s/^( (?: \w+\  )? ) (\w+[a-z]\w+) ( \( .*)$
> +             / $1.defn($2,'Func',$2).norm($3) /xe) {
> +	} elsif (s/^((struct|union|enum) \  (\w+)) ( \s+ \{ .* )$
> +                  / defn($1,(ucfirst $2),$3).norm($4) /xe) {
> +	    if ($2 eq 'enum') {
> +		if (m,/[/*] (\w+)\* \=\&gt\; (\w+)\*\(\),) { 
> +		    in_enum($1,'Func',$2)
> +		} elsif (m,/[/*] (\w+)\* \=\&gt\; (struct) (\w+)\*,) { 
> +		    in_enum($1,(ucfirst $2),$3);
> +	        }
> +	    }
> +	} elsif (s/^( \s* \#define \s+ ) (\w+) ( \s+\S )
> +                  / $1.defmacro($2).norm($3) /xe) {
> +	} else {
> +	    if (m/^\s*\}/) {
> +		$in_enum = undef;
> +	    }
> +	    $_ = norm($_);
> +	}
> +
> +	# Write the line out
> +
> +	if ($pass == 2) {
> +	    $o .= $lstripped;
> +	    $o .= $_;
> +	    $o .= $rstripped;
> +	}
> +    }
> +
> +    warning("pending xrefs at end of file") if @pending_xrefs;
> +
> +    if ($pass == 2) {
> +	$o .= "</pre></body></html>";
> +	write_file($outfile, $o);
> +    }
> +}
> +
> +
> +foreach $pass (qw(1 2)) {
> +    find({ wanted => 
> +	       sub {
> +		   return unless m/\.h$/;
> +		   lstat $File::Find::name or die "$File::Find::name $!";
> +		   -f _ or die "$File::Find::name";
> +		   substr($File::Find::name, 0, 1+length $basedir) 
> +		       eq "$basedir/"
> +		       or die "$File::Find::name $basedir";
> +		   $leaf = substr($File::Find::name, 1+length $basedir);
> +		   if (!file_wanted()) {
> +		       debug(1,"$pass $File::Find::name excluded");
> +		       return;
> +		   }
> +		   $leaf_opath = $leaf;
> +		   $leaf_opath =~ s#/#,#g;
> +		   $leaf_opath .= ".html";
> +		   process_file($File::Find::name, $outdir.'/'.$leaf_opath);
> +	   },
> +	   no_chdir => 1,
> +	 },
> +	 map { "$basedir/$_" } @indirs);
> +}

Re: [PATCH 2/4] docs/html/: Annotations for two hypercalls

[Ian Campbell]

On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> Add annotations for a couple of the hypercalls:
>  HYPERVISOR_set_trap_table
>  HYPERVISOR_mmu_update
> 
> We do this by
>  * annotating the list of #defines for hypercall numbers
>  * annotating the list of error values
>  * providing a function prototype for the systematically-named functions
> The header generator does the rest.
> 
> This exercise revealed a couple of infelicities:
>  * In the actual source code, do_mmu_update is defined to return an int
>    and do_set_trap_table a long.  However both functions return either
>    -Efoo (on error) or 0 for success.

Should that be fixed or would that do more harm than good?

>  * The error numbers are defined only in the private header file
>    xen/include/xen/errno.h and then only with names which will
>    typically clash with other projects.  It would be nice to include a
>    public version of this header which defines XEN_E*.  But for now
>    we run xen-headers on errno.h too.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

> ---
>  docs/Makefile                     |    2 +-
>  xen/include/public/arch-x86/xen.h |    6 ++++++
>  xen/include/public/xen.h          |   11 +++++++++--
>  xen/include/xen/errno.h           |    5 +++++
>  4 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/Makefile b/docs/Makefile
> index ad315cf..a2bbf2d 100644
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -134,7 +134,7 @@ html/hypercall/stamp:
>  	./xen-headers -O $(@D) \
>  		-T 'arch-x86_64 - Xen public headers' \
>  		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 \
> -		../xen include/public
> +		../xen include/public include/xen/errno.h
>  	touch $@
>  
>  txt/%.txt: %.txt
> diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
> index 79ec633..d663cef 100644
> --- a/xen/include/public/arch-x86/xen.h
> +++ b/xen/include/public/arch-x86/xen.h
> @@ -82,7 +82,13 @@ typedef unsigned long xen_pfn_t;
>  typedef unsigned long xen_ulong_t;
>  
>  /*
> + * ` enum neg_errnoval
> + * ` HYPERVISOR_set_trap_table(const struct trap_info traps[]);
> + * `
> + */
> +/*
>   * Send an array of these to HYPERVISOR_set_trap_table().
> + * Terminate the array with a sentinel entry, with traps[].address==0.
>   * The privilege level specifies which modes may enter a trap via a software
>   * interrupt. On x86/64, since rings 1 and 2 are unavailable, we allocate
>   * privilege levels as follows:
> diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> index fde9fa5..f2c9e6f 100644
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -55,6 +55,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
>   * HYPERCALLS
>   */
>  
> +/* ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*() */
> +
>  #define __HYPERVISOR_set_trap_table        0
>  #define __HYPERVISOR_mmu_update            1
>  #define __HYPERVISOR_set_gdt               2
> @@ -105,6 +107,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
>  #define __HYPERVISOR_arch_6               54
>  #define __HYPERVISOR_arch_7               55
>  
> +/* ` } */
> +
>  /*
>   * HYPERCALL COMPATIBILITY.
>   */
> @@ -163,8 +167,11 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
>  #define NR_VIRQS       24
>  
>  /*
> - * HYPERVISOR_mmu_update(reqs, count, pdone, foreigndom)
> - * 
> + * ` enum neg_errnoval
> + * ` HYPERVISOR_mmu_update(const struct mmu_update reqs[],
> + * `                       unsigned count, unsigned *done_out,
> + * `                       unsigned foreigndom)
> + * `
>   * @reqs is an array of mmu_update_t structures ((ptr, val) pairs).
>   * @count is the length of the above array.
>   * @pdone is an output parameter indicating number of completed operations
> diff --git a/xen/include/xen/errno.h b/xen/include/xen/errno.h
> index 7cf599f..39147be 100644
> --- a/xen/include/xen/errno.h
> +++ b/xen/include/xen/errno.h
> @@ -1,6 +1,9 @@
>  #ifndef _I386_ERRNO_H
>  #define _I386_ERRNO_H
>  
> +/* ` enum neg_errnoval {  [ -Efoo for each Efoo in the list below ]  } */
> +/* ` enum errnoval { */
> +
>  #define	EPERM		 1	/* Operation not permitted */
>  #define	ENOENT		 2	/* No such file or directory */
>  #define	ESRCH		 3	/* No such process */
> @@ -129,4 +132,6 @@
>  #define	ENOMEDIUM	123	/* No medium found */
>  #define	EMEDIUMTYPE	124	/* Wrong medium type */
>  
> +/* ` } */
> +
>  #endif

Re: [PATCH 3/4] docs/html/: Generate an "index.html" for hypercall docs

[Ian Campbell]

On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> Generate an "index.html" containing various useful links.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

> ---
>  docs/xen-headers         |   59 ++++++++++++++++++++++++++++++++++++++++++++++
>  xen/include/public/xen.h |    4 ++-
>  2 files changed, 62 insertions(+), 1 deletions(-)
> 
> diff --git a/docs/xen-headers b/docs/xen-headers
> index e893f91..dcf673e 100755
> --- a/docs/xen-headers
> +++ b/docs/xen-headers
> @@ -17,6 +17,10 @@
>  
>  #  definitions must start in LH column
>  #  extra syntax:
> +#   `incontents <seq> <shortname> <anchor text html>...
> +#                              make a table of contents entry; they
> +#                              will be sorted by increasing seq, and
> +#                              shortname will be used as the anchor target
>  #    /* ` <definition>                          } parse as if <definition>
>  #     * ` <definition>                          }  was not commented
>  #   enum <name> { // <pattern>* => <func>()     } cross-reference
> @@ -60,6 +64,8 @@ my ($basedir,@indirs) = @ARGV;
>  # general globals
>  our $pass;
>  our %sdef; 
> +our @incontents;
> +our @outfiles;
>  # $sdef{$type}{$name} => {
>  #     DefLocs => { "$leaf_path:$lineno" => $leaf_opath ,... }
>  #     Xrefs => { "$leaf_path,$lineno" => "$xref", ... }
> @@ -187,6 +193,19 @@ sub out_xrefs ($) {
>      @pending_xrefs = ();
>  }
>  
> +sub incontents ($$$) {
> +    my ($text, $seq, $anchor) = @_;
> +    $anchor = "incontents_$anchor";
> +    if ($pass==2) {
> +	push @incontents, { 
> +	    Seq => $seq,
> +	    Href => "$leaf_opath#$anchor",
> +	    Title => $text,
> +	};
> +    }
> +    return "<a name=\"$anchor\"><strong>$text</strong></a>";
> +}
> +
>  sub write_file ($$) {
>      my ($opath, $odata) = @_;
>      my $out = new IO::File "$opath.new", '>' or die "$opath $!";
> @@ -242,6 +261,8 @@ sub process_file ($$) {
>  	    }
>  	} elsif (s/^( \s* \#define \s+ ) (\w+) ( \s+\S )
>                    / $1.defmacro($2).norm($3) /xe) {
> +	} elsif (s/( \`incontents \s+ (\d+) \s+ (\w+) \s+ )(\S .* \S)
> +                 / norm($1).incontents($4, $2, $3) /xe) {
>  	} else {
>  	    if (m/^\s*\}/) {
>  		$in_enum = undef;
> @@ -261,11 +282,47 @@ sub process_file ($$) {
>      warning("pending xrefs at end of file") if @pending_xrefs;
>  
>      if ($pass == 2) {
> +	push @outfiles, [ $leaf, $leaf_opath ];
>  	$o .= "</pre></body></html>";
>  	write_file($outfile, $o);
>      }
>  }
>  
> +sub output_index () {
> +    my $title = "contents - $xtitle";
> +    $o = '';
> +    $o .= <<END;
> +<html><head><title>$title</title></head>
> +<body>
> +<h1>$title</h1>
> +<h2>Starting points</h2>
> +<ul>
> +END
> +    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} } @incontents) {
> +	$o .= "<li><a href=\"$ic->{Href}\">$ic->{Title}</a></li>\n";
> +    }
> +    $o .= "</ul>\n";
> +    my $forkind = sub {
> +	my ($type,$desc,$pfx,$sfx) = @_;
> +	$o .= "<h2>$desc</h2><ul>\n";
> +	foreach my $name (sort keys %{ $sdef{$type} }) {
> +	    my $href = refhref($type,$name);
> +	    next unless $href =~ m/\S/;
> +	    $o .= "<li><a $href>$pfx$name$sfx</a></li>\n";
> +	}
> +	$o .= "</ul>\n";
> +    };
> +    $forkind->('Func','Functions','','()');
> +    $forkind->('Struct','Structs','struct ','');
> +    $forkind->('Enum','Enums and sets of #defines','','');
> +    $forkind->('EnumVal','Enum values and individual #defines','','');
> +    $o .= "</ul>\n<h2>Files</h2><ul>\n";
> +    foreach my $of (sort { $a->[0] cmp $b->[0] } @outfiles) {
> +	$o .= "<li><a href=\"$of->[1]\">$of->[0]</a></li>\n";
> +    }
> +    $o .= "</ul></body></html>\n";
> +    write_file("$outdir/index.html", $o);
> +}
>  
>  foreach $pass (qw(1 2)) {
>      find({ wanted => 
> @@ -290,3 +347,5 @@ foreach $pass (qw(1 2)) {
>  	 },
>  	 map { "$basedir/$_" } @indirs);
>  }
> +
> +output_index();
> diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> index f2c9e6f..a5b6ad8 100644
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -55,7 +55,9 @@ DEFINE_XEN_GUEST_HANDLE(xen_pfn_t);
>   * HYPERCALLS
>   */
>  
> -/* ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*() */
> +/* `incontents 100 hcalls List of hypercalls
> + * ` enum hypercall_num { // __HYPERVISOR_* => HYPERVISOR_*()
> + */
>  
>  #define __HYPERVISOR_set_trap_table        0
>  #define __HYPERVISOR_mmu_update            1

Re: [PATCH 4/4] docs/html/: Arrange for automatic build of hypercall docs

[Ian Campbell]

On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> - Use index.html rather than a stamp file.
> - Automatically generate dependencies.
> - Wire into the docs build system
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Looks good.

Let me know if you want me to rebase my docs series on this. I
suspect/hope the conflicts will be trivial...

Acked-by: Ian Campbell <ian.campbell@citrix.com>

> ---
>  docs/Makefile    |    9 ++++++---
>  docs/xen-headers |   14 ++++++++++++++
>  2 files changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/Makefile b/docs/Makefile
> index a2bbf2d..413c1a2 100644
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -15,7 +15,8 @@ DOC_MARKDOWN	:= $(wildcard misc/*.markdown)
>  DOC_PS		:= $(patsubst src/%.tex,ps/%.ps,$(DOC_TEX))
>  DOC_PDF		:= $(patsubst src/%.tex,pdf/%.pdf,$(DOC_TEX))
>  DOC_HTML	:= $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX)) \
> -		   $(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN))
> +		   $(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN)) \
> +		   html/hypercall/index.html
>  DOC_TXT         := $(patsubst %.txt,txt/%.txt,$(wildcard misc/*.txt)) \
>  		   $(patsubst %.markdown,txt/%.txt,$(DOC_MARKDOWN))
>  
> @@ -129,13 +130,15 @@ html/%.html: %.markdown
>  	$(MARKDOWN) $< > $@.tmp ; \
>  	$(call move-if-changed,$@.tmp,$@) ; fi
>  
> -html/hypercall/stamp:
> +html/hypercall/index.html: ./xen-headers
> +	rm -rf $(@D)
>  	@$(INSTALL_DIR) $(@D)
>  	./xen-headers -O $(@D) \
>  		-T 'arch-x86_64 - Xen public headers' \
>  		-X arch-ia64 -X arch-x86_32 -X xen-x86_32 \
>  		../xen include/public include/xen/errno.h
> -	touch $@
> +
> +-include html/hypercall/.deps
>  
>  txt/%.txt: %.txt
>  	$(INSTALL_DIR) $(@D)
> diff --git a/docs/xen-headers b/docs/xen-headers
> index dcf673e..8226d73 100755
> --- a/docs/xen-headers
> +++ b/docs/xen-headers
> @@ -325,6 +325,12 @@ END
>  }
>  
>  foreach $pass (qw(1 2)) {
> +    my $depspath = "$outdir/.deps";
> +    my $depsout;
> +    if ($pass==2) {
> +	$depsout = new IO::File "$depspath.new", 'w' or die $!;
> +    }
> +
>      find({ wanted => 
>  	       sub {
>  		   return unless m/\.h$/;
> @@ -341,11 +347,19 @@ foreach $pass (qw(1 2)) {
>  		   $leaf_opath = $leaf;
>  		   $leaf_opath =~ s#/#,#g;
>  		   $leaf_opath .= ".html";
> +		   print $depsout "$outdir/index.html: $File::Find::name\n"
> +		       or die $!
> +		       if $pass==2;
>  		   process_file($File::Find::name, $outdir.'/'.$leaf_opath);
>  	   },
>  	   no_chdir => 1,
>  	 },
>  	 map { "$basedir/$_" } @indirs);
> +
> +    if ($pass==2) {
> +	close $depsout or die $!;
> +	rename "$depspath.new", "$depspath" or die $!;
> +    }
>  }
>  
>  output_index();

Re: [PATCH 2/4] docs/html/: Annotations for two hypercalls

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2/4] docs/html/: Annotations for two hypercalls"):
> On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> > This exercise revealed a couple of infelicities:
> >  * In the actual source code, do_mmu_update is defined to return an int
> >    and do_set_trap_table a long.  However both functions return either
> >    -Efoo (on error) or 0 for success.
> 
> Should that be fixed or would that do more harm than good?

I wasn't sure whether I should go through all the hypercalls checking
and perhaps fixing up their return values.

> Acked-by: Ian Campbell <ian.campbell@citrix.com>

Thanks,
Ian.

Re: [PATCH 2/4] docs/html/: Annotations for two hypercalls

[Ian Campbell]

On Tue, 2011-11-29 at 15:29 +0000, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH 2/4] docs/html/: Annotations for two hypercalls"):
> > On Tue, 2011-11-29 at 15:01 +0000, Ian Jackson wrote:
> > > This exercise revealed a couple of infelicities:
> > >  * In the actual source code, do_mmu_update is defined to return an int
> > >    and do_set_trap_table a long.  However both functions return either
> > >    -Efoo (on error) or 0 for success.
> > 
> > Should that be fixed or would that do more harm than good?
> 
> I wasn't sure whether I should go through all the hypercalls checking
> and perhaps fixing up their return values.

I'd definitely wait for Keir's OK before spending any time on it. Even
then it's likely to be a lot of fiddly work to convince yourself the
change is correct/safe and not an ABI wart we are stuck with...

> 
> > Acked-by: Ian Campbell <ian.campbell@citrix.com>
> 
> Thanks,
> Ian.

Re: [PATCH 4/4] docs/html/: Arrange for automatic build of hypercall docs

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH 4/4] docs/html/: Arrange for automatic build of hypercall docs"):
> Let me know if you want me to rebase my docs series on this. I
> suspect/hope the conflicts will be trivial...

There was one trivial conflict.  Thanks for the acks, both series are
now in staging.

Ian.

Re: [PATCH 4/4] docs/html/: Arrange for automatic build of hypercall docs

[Olaf Hering]

On Tue, Nov 29, Ian Jackson wrote:

> - Use index.html rather than a stamp file.
> - Automatically generate dependencies.
> - Wire into the docs build system
> 

> +-include html/hypercall/.deps

This creates a new .deps file which is also installed during make
install. Unfortunately the rpm post-build checks reject such a file and
the package build fails. My quick hack to remove it during make install
looks like this:

--- a/docs/Makefile
+++ b/docs/Makefile
@@ -102,7 +102,7 @@ install: all
 	$(INSTALL_DIR) $(DESTDIR)$(MANDIR)
 	cp -dR man1 $(DESTDIR)$(MANDIR)
 	cp -dR man5 $(DESTDIR)$(MANDIR)
-	[ ! -d html ] || cp -dR html $(DESTDIR)$(DOCDIR)
+	if [ -d html ] ; then cp -dR html $(DESTDIR)$(DOCDIR) ; find $(DESTDIR)$(DOCDIR) -name .deps -print0 | xargs -0 --no-run-if-empty rm -f ; fi
 
 pdf/%.pdf: ps/%.ps
 	$(INSTALL_DIR) $(@D)


There is probably a better way to get rid of the file.

Olaf