[Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Anthony Liguori]

Right now our specs are written in psuedo-wiki syntax.  This series converts
them to markdown.  markdown is a simple markup format that's gaining in
popularity.

The big advantage of using markdown is that there are tools that can convert it
to relatively simple HTML.  That means we can build a make infrastructure that
generates a nice set of static web pages.

The syntax is also more human friendly than mediawiki syntax.

To see what the stylized version of this looks like, check out:

  https://github.com/aliguori/qemu/tree/markdown/docs/specs

[Qemu-devel] [PATCH 1/5] docs: convert qed_spec.txt to markdown

[Anthony Liguori]

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 docs/specs/qed_spec.md  |  221 +++++++++++++++++++++++++++++++++++++++++++++++
 docs/specs/qed_spec.txt |  138 -----------------------------
 2 files changed, 221 insertions(+), 138 deletions(-)
 create mode 100644 docs/specs/qed_spec.md
 delete mode 100644 docs/specs/qed_spec.txt

diff --git a/docs/specs/qed_spec.md b/docs/specs/qed_spec.md
new file mode 100644
index 0000000..d74984c
--- /dev/null
+++ b/docs/specs/qed_spec.md
@@ -0,0 +1,221 @@
+Specification
+=============
+
+The file format looks like this:
+
+    +----------+----------+----------+-----+
+    | cluster0 | cluster1 | cluster2 | ... |
+    +----------+----------+----------+-----+
+
+The first cluster begins with the *header*.  The header contains information
+about where regular clusters start; this allows the header to be extensible and
+store extra information about the image file.  A regular cluster may be a *data
+cluster*, an *L2*, or an *L1 table*.  L1 and L2 tables are composed of one or
+more contiguous clusters.
+
+Normally the file size will be a multiple of the cluster size.  If the file
+size is not a multiple, extra information after the last cluster may not be
+preserved if data is written.  Legitimate extra information should use space
+between the header and the first regular cluster.
+
+All fields are little-endian.
+
+Header
+------
+    Header {
+        uint32_t magic;               /* QED\0 */
+    
+        uint32_t cluster_size;        /* in bytes */
+        uint32_t table_size;          /* for L1 and L2 tables, in clusters */
+        uint32_t header_size;         /* in clusters */
+    
+        uint64_t features;            /* format feature bits */
+        uint64_t compat_features;     /* compat feature bits */
+        uint64_t autoclear_features;  /* self-resetting feature bits */
+   
+        uint64_t l1_table_offset;     /* in bytes */
+        uint64_t image_size;          /* total logical image size, in bytes */
+    
+        /* if (features & QED_F_BACKING_FILE) */
+        uint32_t backing_filename_offset; /* in bytes from start of header */
+        uint32_t backing_filename_size;   /* in bytes */
+    }
+
+Field descriptions:
+
+ * __cluster_size__ must be a power of 2 in range [2^12, 2^26].
+
+ * __table_size__ must be a power of 2 in range [1, 16].
+
+ * __header_size__ is the number of clusters used by the header and any
+   additional information stored before regular clusters.
+
+ * __features__, __compat_features__, and __autoclear_features__ are file format
+   extension bitmaps.  They work as follows:
+
+  - An image with unknown __features__ bits enabled must not be opened.  File
+    format changes that are not backwards-compatible must use __features__ bits.
+
+  - An image with unknown __compat_features__ bits enabled can be opened safely.
+    The unknown features are simply ignored and represent backwards-compatible
+    changes to the file format.
+
+  - An image with unknown __autoclear_features__ bits enable can be opened
+    safely after clearing the unknown bits.  This allows for
+    backwards-compatible changes to the file format which degrade gracefully and
+    can be re-enabled again by a new program later.
+
+ * __l1_table_offset__ is the offset of the first byte of the L1 table in the
+   image file and must be a multiple of __cluster_size__.
+
+ * __image_size__ is the block device size seen by the guest and must be a
+   multiple of 512 bytes.
+
+ * __backing_filename_offset__ and __backing_filename_size__ describe a string
+   in (byte offset, byte size) form.  It is not NUL-terminated and has no
+   alignment constraints.  The string must be stored within the first
+   __header_size__ clusters.  The backing filename may be an absolute path or
+   relative to the image file.
+
+Feature bits:
+
+ * QED_F_BACKING_FILE = 0x01.  The image uses a backing file.
+
+ * QED_F_NEED_CHECK = 0x02.  The image needs a consistency check before use.
+
+ * QED_F_BACKING_FORMAT_NO_PROBE = 0x04.  The backing file is a raw disk image
+   and no file format autodetection should be attempted.  This should be used
+   to ensure that raw backing files are never detected as an image format if
+   they happen to contain magic constants.
+
+There are currently no defined __compat_features__ or __autoclear_features__
+bits.
+
+Fields predicated on a feature bit are only used when that feature is set.  The
+fields always take up header space, regardless of whether or not the feature
+bit is set.
+
+Tables
+------
+
+Tables provide the translation from logical offsets in the block device to
+cluster offsets in the file.
+
+    #define TABLE_NOFFSETS (table_size * cluster_size / sizeof(uint64_t))
+     
+    Table {
+        uint64_t offsets[TABLE_NOFFSETS];
+    }
+
+The tables are organized as follows:
+
+                       +----------+
+                       | L1 table |
+                       +----------+
+                  ,------'  |  '------.
+             +----------+   |    +----------+
+             | L2 table |  ...   | L2 table |
+             +----------+        +----------+
+         ,------'  |  '------.
+    +----------+   |    +----------+
+    |   Data   |  ...   |   Data   |
+    +----------+        +----------+
+
+A table is made up of one or more contiguous clusters.  The table_size header
+field determines table size for an image file.  For example, cluster_size=64 KB
+and table_size=4 results in 256 KB tables.
+
+The logical image size must be less than or equal to the maximum possible size
+of clusters rooted by the L1 table:
+
+    header.image_size <= TABLE_NOFFSETS * TABLE_NOFFSETS * header.cluster_size
+
+L1, L2, and data cluster offsets must be aligned to header.cluster_size.  The
+following offsets have special meanings:
+
+### L2 table offsets
+
+ * 0 - unallocated.  The L2 table is not yet allocated.
+
+### Data cluster offsets
+
+ * 0 - unallocated.  The data cluster is not yet allocated.
+
+ * 1 - zero.  The data cluster contents are all zeroes and no cluster is
+   allocated.
+
+Future format extensions may wish to store per-offset information.  The least
+significant 12 bits of an offset are reserved for this purpose and must be set
+to zero.  Image files with cluster_size > 2^12 will have more unused bits which
+should also be zeroed.
+
+### Unallocated L2 tables and data clusters
+
+Reads to an unallocated area of the image file access the backing file.  If
+there is no backing file, then zeroes are produced.  The backing file may be
+smaller than the image file and reads of unallocated areas beyond the end of
+the backing file produce zeroes.
+
+Writes to an unallocated area cause a new data clusters to be allocated, and a
+new L2 table if that is also unallocated.  The new data cluster is populated
+with data from the backing file (or zeroes if no backing file) and the data
+being written.
+
+### Zero data clusters
+
+Zero data clusters are a space-efficient way of storing zeroed regions of the
+image.
+
+Reads to a zero data cluster produce zeroes.  Note that the difference between
+an unallocated and a zero data cluster is that zero data clusters stop the
+reading of contents from the backing file.
+
+Writes to a zero data cluster cause a new data cluster to be allocated.  The
+new data cluster is populated with zeroes and the data being written.
+
+### Logical offset translation
+
+Logical offsets are translated into cluster offsets as follows:
+
+     table_bits table_bits    cluster_bits
+     <--------> <--------> <--------------->
+    +----------+----------+-----------------+
+    | L1 index | L2 index |     byte offset |
+    +----------+----------+-----------------+
+    
+Structure of a logical offset
+   
+    offset_mask = ~(cluster_size - 1) # mask for the image file byte offset
+    
+    def logical_to_cluster_offset(l1_index, l2_index, byte_offset):
+      l2_offset = l1_table[l1_index]
+      l2_table = load_table(l2_offset)
+      cluster_offset = l2_table[l2_index] & offset_mask
+      return cluster_offset + byte_offset
+
+Consistency checking
+--------------------
+
+This section is informational and included to provide background on the use of
+the QED_F_NEED_CHECK __features__ bit.
+
+The QED_F_NEED_CHECK bit is used to mark an image as dirty before starting an
+operation that could leave the image in an inconsistent state if interrupted by
+a crash or power failure.  A dirty image must be checked on open because its
+metadata may not be consistent.
+
+Consistency check includes the following invariants:
+
+ 1. Each cluster is referenced once and only once.  It is an inconsistency to
+    have a cluster referenced more than once by L1 or L2 tables.  A cluster has
+    been leaked if it has no references.
+
+ 2. Offsets must be within the image file size and must be __cluster_size__
+    aligned.
+
+ 3. Table offsets must at least __table_size__ * __cluster_size__ bytes from the
+    end of the image file so that there is space for the entire table.
+
+The consistency check process starts by from __l1_table_offset__ and scans all
+L2 tables.  After the check completes with no other errors besides leaks, the
+QED_F_NEED_CHECK bit can be cleared and the image can be accessed.
diff --git a/docs/specs/qed_spec.txt b/docs/specs/qed_spec.txt
deleted file mode 100644
index 7982e05..0000000
--- a/docs/specs/qed_spec.txt
+++ /dev/null
@@ -1,138 +0,0 @@
-=Specification=
-
-The file format looks like this:
-
- +----------+----------+----------+-----+
- | cluster0 | cluster1 | cluster2 | ... |
- +----------+----------+----------+-----+
-
-The first cluster begins with the '''header'''.  The header contains information about where regular clusters start; this allows the header to be extensible and store extra information about the image file.  A regular cluster may be a '''data cluster''', an '''L2''', or an '''L1 table'''.  L1 and L2 tables are composed of one or more contiguous clusters.
-
-Normally the file size will be a multiple of the cluster size.  If the file size is not a multiple, extra information after the last cluster may not be preserved if data is written.  Legitimate extra information should use space between the header and the first regular cluster.
-
-All fields are little-endian.
-
-==Header==
- Header {
-     uint32_t magic;               /* QED\0 */
- 
-     uint32_t cluster_size;        /* in bytes */
-     uint32_t table_size;          /* for L1 and L2 tables, in clusters */
-     uint32_t header_size;         /* in clusters */
- 
-     uint64_t features;            /* format feature bits */
-     uint64_t compat_features;     /* compat feature bits */
-     uint64_t autoclear_features;  /* self-resetting feature bits */
-
-     uint64_t l1_table_offset;     /* in bytes */
-     uint64_t image_size;          /* total logical image size, in bytes */
- 
-     /* if (features & QED_F_BACKING_FILE) */
-     uint32_t backing_filename_offset; /* in bytes from start of header */
-     uint32_t backing_filename_size;   /* in bytes */
- }
-
-Field descriptions:
-* ''cluster_size'' must be a power of 2 in range [2^12, 2^26].
-* ''table_size'' must be a power of 2 in range [1, 16].
-* ''header_size'' is the number of clusters used by the header and any additional information stored before regular clusters.
-* ''features'', ''compat_features'', and ''autoclear_features'' are file format extension bitmaps.  They work as follows:
-** An image with unknown ''features'' bits enabled must not be opened.  File format changes that are not backwards-compatible must use ''features'' bits.
-** An image with unknown ''compat_features'' bits enabled can be opened safely.  The unknown features are simply ignored and represent backwards-compatible changes to the file format.
-** An image with unknown ''autoclear_features'' bits enable can be opened safely after clearing the unknown bits.  This allows for backwards-compatible changes to the file format which degrade gracefully and can be re-enabled again by a new program later.
-* ''l1_table_offset'' is the offset of the first byte of the L1 table in the image file and must be a multiple of ''cluster_size''.
-* ''image_size'' is the block device size seen by the guest and must be a multiple of 512 bytes.
-* ''backing_filename_offset'' and ''backing_filename_size'' describe a string in (byte offset, byte size) form.  It is not NUL-terminated and has no alignment constraints.  The string must be stored within the first ''header_size'' clusters.  The backing filename may be an absolute path or relative to the image file.
-
-Feature bits:
-* QED_F_BACKING_FILE = 0x01.  The image uses a backing file.
-* QED_F_NEED_CHECK = 0x02.  The image needs a consistency check before use.
-* QED_F_BACKING_FORMAT_NO_PROBE = 0x04.  The backing file is a raw disk image and no file format autodetection should be attempted.  This should be used to ensure that raw backing files are never detected as an image format if they happen to contain magic constants.
-
-There are currently no defined ''compat_features'' or ''autoclear_features'' bits.
-
-Fields predicated on a feature bit are only used when that feature is set.  The fields always take up header space, regardless of whether or not the feature bit is set.
-
-==Tables==
-
-Tables provide the translation from logical offsets in the block device to cluster offsets in the file.
-
- #define TABLE_NOFFSETS (table_size * cluster_size / sizeof(uint64_t))
-  
- Table {
-     uint64_t offsets[TABLE_NOFFSETS];
- }
-
-The tables are organized as follows:
-
-                    +----------+
-                    | L1 table |
-                    +----------+
-               ,------'  |  '------.
-          +----------+   |    +----------+
-          | L2 table |  ...   | L2 table |
-          +----------+        +----------+
-      ,------'  |  '------.
- +----------+   |    +----------+
- |   Data   |  ...   |   Data   |
- +----------+        +----------+
-
-A table is made up of one or more contiguous clusters.  The table_size header field determines table size for an image file.  For example, cluster_size=64 KB and table_size=4 results in 256 KB tables.
-
-The logical image size must be less than or equal to the maximum possible size of clusters rooted by the L1 table:
- header.image_size <= TABLE_NOFFSETS * TABLE_NOFFSETS * header.cluster_size
-
-L1, L2, and data cluster offsets must be aligned to header.cluster_size.  The following offsets have special meanings:
-
-===L2 table offsets===
-* 0 - unallocated.  The L2 table is not yet allocated.
-
-===Data cluster offsets===
-* 0 - unallocated.  The data cluster is not yet allocated.
-* 1 - zero.  The data cluster contents are all zeroes and no cluster is allocated.
-
-Future format extensions may wish to store per-offset information.  The least significant 12 bits of an offset are reserved for this purpose and must be set to zero.  Image files with cluster_size > 2^12 will have more unused bits which should also be zeroed.
-
-===Unallocated L2 tables and data clusters===
-Reads to an unallocated area of the image file access the backing file.  If there is no backing file, then zeroes are produced.  The backing file may be smaller than the image file and reads of unallocated areas beyond the end of the backing file produce zeroes.
-
-Writes to an unallocated area cause a new data clusters to be allocated, and a new L2 table if that is also unallocated.  The new data cluster is populated with data from the backing file (or zeroes if no backing file) and the data being written.
-
-===Zero data clusters===
-Zero data clusters are a space-efficient way of storing zeroed regions of the image.
-
-Reads to a zero data cluster produce zeroes.  Note that the difference between an unallocated and a zero data cluster is that zero data clusters stop the reading of contents from the backing file.
-
-Writes to a zero data cluster cause a new data cluster to be allocated.  The new data cluster is populated with zeroes and the data being written.
-
-===Logical offset translation===
-Logical offsets are translated into cluster offsets as follows:
-
-  table_bits table_bits    cluster_bits
-  <--------> <--------> <--------------->
- +----------+----------+-----------------+
- | L1 index | L2 index |     byte offset |
- +----------+----------+-----------------+
- 
-       Structure of a logical offset
-
- offset_mask = ~(cluster_size - 1) # mask for the image file byte offset
- 
- def logical_to_cluster_offset(l1_index, l2_index, byte_offset):
-   l2_offset = l1_table[l1_index]
-   l2_table = load_table(l2_offset)
-   cluster_offset = l2_table[l2_index] & offset_mask
-   return cluster_offset + byte_offset
-
-==Consistency checking==
-
-This section is informational and included to provide background on the use of the QED_F_NEED_CHECK ''features'' bit.
-
-The QED_F_NEED_CHECK bit is used to mark an image as dirty before starting an operation that could leave the image in an inconsistent state if interrupted by a crash or power failure.  A dirty image must be checked on open because its metadata may not be consistent.
-
-Consistency check includes the following invariants:
-# Each cluster is referenced once and only once.  It is an inconsistency to have a cluster referenced more than once by L1 or L2 tables.  A cluster has been leaked if it has no references.
-# Offsets must be within the image file size and must be ''cluster_size'' aligned.
-# Table offsets must at least ''table_size'' * ''cluster_size'' bytes from the end of the image file so that there is space for the entire table.
-
-The consistency check process starts by from ''l1_table_offset'' and scans all L2 tables.  After the check completes with no other errors besides leaks, the QED_F_NEED_CHECK bit can be cleared and the image can be accessed.
-- 
1.7.4.1

[Qemu-devel] [PATCH 2/5] docs: convert qcow2 specification to markdown

[Anthony Liguori]

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 docs/specs/qcow2.md  |  263 ++++++++++++++++++++++++++++++++++++++++++++++++++
 docs/specs/qcow2.txt |  260 -------------------------------------------------
 2 files changed, 263 insertions(+), 260 deletions(-)
 create mode 100644 docs/specs/qcow2.md
 delete mode 100644 docs/specs/qcow2.txt

diff --git a/docs/specs/qcow2.md b/docs/specs/qcow2.md
new file mode 100644
index 0000000..b9d6cdd
--- /dev/null
+++ b/docs/specs/qcow2.md
@@ -0,0 +1,263 @@
+General
+=======
+
+A qcow2 image file is organized in units of constant size, which are called
+(host) clusters. A cluster is the unit in which all allocations are done,
+both for actual guest data and for image metadata.
+
+Likewise, the virtual disk as seen by the guest is divided into (guest)
+clusters of the same size.
+
+All numbers in qcow2 are stored in Big Endian byte order.
+
+Header
+======
+
+The first cluster of a qcow2 image contains the file header:
+
+    Byte  0 -  3:   magic
+                    QCOW magic string ("QFI\xfb")
+
+          4 -  7:   version
+                    Version number (only valid value is 2)
+
+          8 - 15:   backing_file_offset
+                    Offset into the image file at which the backing file name
+                    is stored (NB: The string is not null terminated). 0 if the
+                    image doesn't have a backing file.
+
+         16 - 19:   backing_file_size
+                    Length of the backing file name in bytes. Must not be
+                    longer than 1023 bytes. Undefined if the image doesn't have
+                    a backing file.
+
+         20 - 23:   cluster_bits
+                    Number of bits that are used for addressing an offset
+                    within a cluster (1 << cluster_bits is the cluster size).
+                    Must not be less than 9 (i.e. 512 byte clusters).
+
+                    Note: qemu as of today has an implementation limit of 2 MB
+                    as the maximum cluster size and won't be able to open images
+                    with larger cluster sizes.
+
+         24 - 31:   size
+                    Virtual disk size in bytes
+
+         32 - 35:   crypt_method
+                    0 for no encryption
+                    1 for AES encryption
+
+         36 - 39:   l1_size
+                    Number of entries in the active L1 table
+
+         40 - 47:   l1_table_offset
+                    Offset into the image file at which the active L1 table
+                    starts. Must be aligned to a cluster boundary.
+
+         48 - 55:   refcount_table_offset
+                    Offset into the image file at which the refcount table
+                    starts. Must be aligned to a cluster boundary.
+
+         56 - 59:   refcount_table_clusters
+                    Number of clusters that the refcount table occupies
+
+         60 - 63:   nb_snapshots
+                    Number of snapshots contained in the image
+
+         64 - 71:   snapshots_offset
+                    Offset into the image file at which the snapshot table
+                    starts. Must be aligned to a cluster boundary.
+
+Directly after the image header, optional sections called header extensions can
+be stored. Each extension has a structure like the following:
+
+    Byte  0 -  3:   Header extension type:
+                        0x00000000 - End of the header extension area
+                        0xE2792ACA - Backing file format name
+                        other      - Unknown header extension, can be safely
+                                     ignored
+
+          4 -  7:   Length of the header extension data
+
+          8 -  n:   Header extension data
+
+          n -  m:   Padding to round up the header extension size to the next
+                    multiple of 8.
+
+The remaining space between the end of the header extension area and the end of
+the first cluster can be used for other data. Usually, the backing file name is
+stored there.
+
+
+Host cluster management
+=======================
+
+qcow2 manages the allocation of host clusters by maintaining a reference count
+for each host cluster. A refcount of 0 means that the cluster is free, 1 means
+that it is used, and >= 2 means that it is used and any write access must
+perform a COW (copy on write) operation.
+
+The refcounts are managed in a two-level table. The first level is called
+refcount table and has a variable size (which is stored in the header). The
+refcount table can cover multiple clusters, however it needs to be contiguous
+in the image file.
+
+It contains pointers to the second level structures which are called refcount
+blocks and are exactly one cluster in size.
+
+Given a offset into the image file, the refcount of its cluster can be obtained
+as follows:
+
+    refcount_block_entries = (cluster_size / sizeof(uint16_t))
+
+    refcount_block_index = (offset / cluster_size) % refcount_block_entries
+    refcount_table_index = (offset / cluster_size) / refcount_block_entries
+
+    refcount_block = load_cluster(refcount_table[refcount_table_index]);
+    return refcount_block[refcount_block_index];
+
+Refcount table entry:
+
+    Bit  0 -  8:    Reserved (set to 0)
+
+         9 - 63:    Bits 9-63 of the offset into the image file at which the
+                    refcount block starts. Must be aligned to a cluster
+                    boundary.
+
+                    If this is 0, the corresponding refcount block has not yet
+                    been allocated. All refcounts managed by this refcount block
+                    are 0.
+
+Refcount block entry:
+
+    Bit  0 - 15:    Reference count of the cluster
+
+
+Cluster mapping
+===============
+
+Just as for refcounts, qcow2 uses a two-level structure for the mapping of
+guest clusters to host clusters. They are called L1 and L2 table.
+
+The L1 table has a variable size (stored in the header) and may use multiple
+clusters, however it must be contiguous in the image file. L2 tables are
+exactly one cluster in size.
+
+Given a offset into the virtual disk, the offset into the image file can be
+obtained as follows:
+
+    l2_entries = (cluster_size / sizeof(uint64_t))
+
+    l2_index = (offset / cluster_size) % l2_entries
+    l1_index = (offset / cluster_size) / l2_entries
+
+    l2_table = load_cluster(l1_table[l1_index]);
+    cluster_offset = l2_table[l2_index];
+
+    return cluster_offset + (offset % cluster_size)
+
+L1 table entry:
+
+    Bit  0 -  8:    Reserved (set to 0)
+
+         9 - 55:    Bits 9-55 of the offset into the image file at which the L2
+                    table starts. Must be aligned to a cluster boundary. If the
+                    offset is 0, the L2 table and all clusters described by this
+                    L2 table are unallocated.
+
+        56 - 62:    Reserved (set to 0)
+
+             63:    0 for an L2 table that is unused or requires COW, 1 if its
+                    refcount is exactly one. This information is only accurate
+                    in the active L1 table.
+
+L2 table entry (for normal clusters):
+
+    Bit  0 -  8:    Reserved (set to 0)
+
+         9 - 55:    Bits 9-55 of host cluster offset. Must be aligned to a
+                    cluster boundary. If the offset is 0, the cluster is
+                    unallocated.
+
+        56 - 61:    Reserved (set to 0)
+
+             62:    0 (this cluster is not compressed)
+
+             63:    0 for a cluster that is unused or requires COW, 1 if its
+                    refcount is exactly one. This information is only accurate
+                    in L2 tables that are reachable from the the active L1
+                    table.
+
+L2 table entry (for compressed clusters; x = 62 - (cluster_size - 8)):
+
+    Bit  0 -  x:    Host cluster offset. This is usually _not_ aligned to a
+                    cluster boundary!
+
+       x+1 - 61:    Compressed size of the images in sectors of 512 bytes
+
+             62:    1 (this cluster is compressed using zlib)
+
+             63:    0 for a cluster that is unused or requires COW, 1 if its
+                    refcount is exactly one. This information is only accurate
+                    in L2 tables that are reachable from the the active L1
+                    table.
+
+If a cluster is unallocated, read requests shall read the data from the backing
+file. If there is no backing file or the backing file is smaller than the image,
+they shall read zeros for all parts that are not covered by the backing file.
+
+Snapshots
+=========
+
+qcow2 supports internal snapshots. Their basic principle of operation is to
+switch the active L1 table, so that a different set of host clusters are
+exposed to the guest.
+
+When creating a snapshot, the L1 table should be copied and the refcount of all
+L2 tables and clusters reachable from this L1 table must be increased, so that
+a write causes a COW and isn't visible in other snapshots.
+
+When loading a snapshot, bit 63 of all entries in the new active L1 table and
+all L2 tables referenced by it must be reconstructed from the refcount table
+as it doesn't need to be accurate in inactive L1 tables.
+
+A directory of all snapshots is stored in the snapshot table, a contiguous area
+in the image file, whose starting offset and length are given by the header
+fields snapshots_offset and nb_snapshots. The entries of the snapshot table
+have variable length, depending on the length of ID, name and extra data.
+
+Snapshot table entry:
+
+    Byte 0 -  7:    Offset into the image file at which the L1 table for the
+                    snapshot starts. Must be aligned to a cluster boundary.
+
+         8 - 11:    Number of entries in the L1 table of the snapshots
+
+        12 - 13:    Length of the unique ID string describing the snapshot
+
+        14 - 15:    Length of the name of the snapshot
+
+        16 - 19:    Time at which the snapshot was taken in seconds since the
+                    Epoch
+
+        20 - 23:    Subsecond part of the time at which the snapshot was taken
+                    in nanoseconds
+
+        24 - 31:    Time that the guest was running until the snapshot was
+                    taken in nanoseconds
+
+        32 - 35:    Size of the VM state in bytes. 0 if no VM state is saved.
+                    If there is VM state, it starts at the first cluster
+                    described by first L1 table entry that doesn't describe a
+                    regular guest cluster (i.e. VM state is stored like guest
+                    disk content, except that it is stored at offsets that are
+                    larger than the virtual disk presented to the guest)
+
+        36 - 39:    Size of extra data in the table entry (used for future
+                    extensions of the format)
+
+        variable:   Extra data for future extensions. Must be ignored.
+
+        variable:   Unique ID string for the snapshot (not null terminated)
+
+        variable:   Name of the snapshot (not null terminated)
diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
deleted file mode 100644
index e792953..0000000
--- a/docs/specs/qcow2.txt
+++ /dev/null
@@ -1,260 +0,0 @@
-== General ==
-
-A qcow2 image file is organized in units of constant size, which are called
-(host) clusters. A cluster is the unit in which all allocations are done,
-both for actual guest data and for image metadata.
-
-Likewise, the virtual disk as seen by the guest is divided into (guest)
-clusters of the same size.
-
-All numbers in qcow2 are stored in Big Endian byte order.
-
-
-== Header ==
-
-The first cluster of a qcow2 image contains the file header:
-
-    Byte  0 -  3:   magic
-                    QCOW magic string ("QFI\xfb")
-
-          4 -  7:   version
-                    Version number (only valid value is 2)
-
-          8 - 15:   backing_file_offset
-                    Offset into the image file at which the backing file name
-                    is stored (NB: The string is not null terminated). 0 if the
-                    image doesn't have a backing file.
-
-         16 - 19:   backing_file_size
-                    Length of the backing file name in bytes. Must not be
-                    longer than 1023 bytes. Undefined if the image doesn't have
-                    a backing file.
-
-         20 - 23:   cluster_bits
-                    Number of bits that are used for addressing an offset
-                    within a cluster (1 << cluster_bits is the cluster size).
-                    Must not be less than 9 (i.e. 512 byte clusters).
-
-                    Note: qemu as of today has an implementation limit of 2 MB
-                    as the maximum cluster size and won't be able to open images
-                    with larger cluster sizes.
-
-         24 - 31:   size
-                    Virtual disk size in bytes
-
-         32 - 35:   crypt_method
-                    0 for no encryption
-                    1 for AES encryption
-
-         36 - 39:   l1_size
-                    Number of entries in the active L1 table
-
-         40 - 47:   l1_table_offset
-                    Offset into the image file at which the active L1 table
-                    starts. Must be aligned to a cluster boundary.
-
-         48 - 55:   refcount_table_offset
-                    Offset into the image file at which the refcount table
-                    starts. Must be aligned to a cluster boundary.
-
-         56 - 59:   refcount_table_clusters
-                    Number of clusters that the refcount table occupies
-
-         60 - 63:   nb_snapshots
-                    Number of snapshots contained in the image
-
-         64 - 71:   snapshots_offset
-                    Offset into the image file at which the snapshot table
-                    starts. Must be aligned to a cluster boundary.
-
-Directly after the image header, optional sections called header extensions can
-be stored. Each extension has a structure like the following:
-
-    Byte  0 -  3:   Header extension type:
-                        0x00000000 - End of the header extension area
-                        0xE2792ACA - Backing file format name
-                        other      - Unknown header extension, can be safely
-                                     ignored
-
-          4 -  7:   Length of the header extension data
-
-          8 -  n:   Header extension data
-
-          n -  m:   Padding to round up the header extension size to the next
-                    multiple of 8.
-
-The remaining space between the end of the header extension area and the end of
-the first cluster can be used for other data. Usually, the backing file name is
-stored there.
-
-
-== Host cluster management ==
-
-qcow2 manages the allocation of host clusters by maintaining a reference count
-for each host cluster. A refcount of 0 means that the cluster is free, 1 means
-that it is used, and >= 2 means that it is used and any write access must
-perform a COW (copy on write) operation.
-
-The refcounts are managed in a two-level table. The first level is called
-refcount table and has a variable size (which is stored in the header). The
-refcount table can cover multiple clusters, however it needs to be contiguous
-in the image file.
-
-It contains pointers to the second level structures which are called refcount
-blocks and are exactly one cluster in size.
-
-Given a offset into the image file, the refcount of its cluster can be obtained
-as follows:
-
-    refcount_block_entries = (cluster_size / sizeof(uint16_t))
-
-    refcount_block_index = (offset / cluster_size) % refcount_block_entries
-    refcount_table_index = (offset / cluster_size) / refcount_block_entries
-
-    refcount_block = load_cluster(refcount_table[refcount_table_index]);
-    return refcount_block[refcount_block_index];
-
-Refcount table entry:
-
-    Bit  0 -  8:    Reserved (set to 0)
-
-         9 - 63:    Bits 9-63 of the offset into the image file at which the
-                    refcount block starts. Must be aligned to a cluster
-                    boundary.
-
-                    If this is 0, the corresponding refcount block has not yet
-                    been allocated. All refcounts managed by this refcount block
-                    are 0.
-
-Refcount block entry:
-
-    Bit  0 - 15:    Reference count of the cluster
-
-
-== Cluster mapping ==
-
-Just as for refcounts, qcow2 uses a two-level structure for the mapping of
-guest clusters to host clusters. They are called L1 and L2 table.
-
-The L1 table has a variable size (stored in the header) and may use multiple
-clusters, however it must be contiguous in the image file. L2 tables are
-exactly one cluster in size.
-
-Given a offset into the virtual disk, the offset into the image file can be
-obtained as follows:
-
-    l2_entries = (cluster_size / sizeof(uint64_t))
-
-    l2_index = (offset / cluster_size) % l2_entries
-    l1_index = (offset / cluster_size) / l2_entries
-
-    l2_table = load_cluster(l1_table[l1_index]);
-    cluster_offset = l2_table[l2_index];
-
-    return cluster_offset + (offset % cluster_size)
-
-L1 table entry:
-
-    Bit  0 -  8:    Reserved (set to 0)
-
-         9 - 55:    Bits 9-55 of the offset into the image file at which the L2
-                    table starts. Must be aligned to a cluster boundary. If the
-                    offset is 0, the L2 table and all clusters described by this
-                    L2 table are unallocated.
-
-        56 - 62:    Reserved (set to 0)
-
-             63:    0 for an L2 table that is unused or requires COW, 1 if its
-                    refcount is exactly one. This information is only accurate
-                    in the active L1 table.
-
-L2 table entry (for normal clusters):
-
-    Bit  0 -  8:    Reserved (set to 0)
-
-         9 - 55:    Bits 9-55 of host cluster offset. Must be aligned to a
-                    cluster boundary. If the offset is 0, the cluster is
-                    unallocated.
-
-        56 - 61:    Reserved (set to 0)
-
-             62:    0 (this cluster is not compressed)
-
-             63:    0 for a cluster that is unused or requires COW, 1 if its
-                    refcount is exactly one. This information is only accurate
-                    in L2 tables that are reachable from the the active L1
-                    table.
-
-L2 table entry (for compressed clusters; x = 62 - (cluster_size - 8)):
-
-    Bit  0 -  x:    Host cluster offset. This is usually _not_ aligned to a
-                    cluster boundary!
-
-       x+1 - 61:    Compressed size of the images in sectors of 512 bytes
-
-             62:    1 (this cluster is compressed using zlib)
-
-             63:    0 for a cluster that is unused or requires COW, 1 if its
-                    refcount is exactly one. This information is only accurate
-                    in L2 tables that are reachable from the the active L1
-                    table.
-
-If a cluster is unallocated, read requests shall read the data from the backing
-file. If there is no backing file or the backing file is smaller than the image,
-they shall read zeros for all parts that are not covered by the backing file.
-
-
-== Snapshots ==
-
-qcow2 supports internal snapshots. Their basic principle of operation is to
-switch the active L1 table, so that a different set of host clusters are
-exposed to the guest.
-
-When creating a snapshot, the L1 table should be copied and the refcount of all
-L2 tables and clusters reachable from this L1 table must be increased, so that
-a write causes a COW and isn't visible in other snapshots.
-
-When loading a snapshot, bit 63 of all entries in the new active L1 table and
-all L2 tables referenced by it must be reconstructed from the refcount table
-as it doesn't need to be accurate in inactive L1 tables.
-
-A directory of all snapshots is stored in the snapshot table, a contiguous area
-in the image file, whose starting offset and length are given by the header
-fields snapshots_offset and nb_snapshots. The entries of the snapshot table
-have variable length, depending on the length of ID, name and extra data.
-
-Snapshot table entry:
-
-    Byte 0 -  7:    Offset into the image file at which the L1 table for the
-                    snapshot starts. Must be aligned to a cluster boundary.
-
-         8 - 11:    Number of entries in the L1 table of the snapshots
-
-        12 - 13:    Length of the unique ID string describing the snapshot
-
-        14 - 15:    Length of the name of the snapshot
-
-        16 - 19:    Time at which the snapshot was taken in seconds since the
-                    Epoch
-
-        20 - 23:    Subsecond part of the time at which the snapshot was taken
-                    in nanoseconds
-
-        24 - 31:    Time that the guest was running until the snapshot was
-                    taken in nanoseconds
-
-        32 - 35:    Size of the VM state in bytes. 0 if no VM state is saved.
-                    If there is VM state, it starts at the first cluster
-                    described by first L1 table entry that doesn't describe a
-                    regular guest cluster (i.e. VM state is stored like guest
-                    disk content, except that it is stored at offsets that are
-                    larger than the virtual disk presented to the guest)
-
-        36 - 39:    Size of extra data in the table entry (used for future
-                    extensions of the format)
-
-        variable:   Extra data for future extensions. Must be ignored.
-
-        variable:   Unique ID string for the snapshot (not null terminated)
-
-        variable:   Name of the snapshot (not null terminated)
-- 
1.7.4.1

[Qemu-devel] [PATCH 3/5] docs: convert ivshmem device spec to markdown

[Anthony Liguori]

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 docs/specs/ivshmem_device_spec.md  |   97 ++++++++++++++++++++++++++++++++++++
 docs/specs/ivshmem_device_spec.txt |   96 -----------------------------------
 2 files changed, 97 insertions(+), 96 deletions(-)
 create mode 100644 docs/specs/ivshmem_device_spec.md
 delete mode 100644 docs/specs/ivshmem_device_spec.txt

diff --git a/docs/specs/ivshmem_device_spec.md b/docs/specs/ivshmem_device_spec.md
new file mode 100644
index 0000000..70226e6
--- /dev/null
+++ b/docs/specs/ivshmem_device_spec.md
@@ -0,0 +1,97 @@
+Device Specification for Inter-VM shared memory device
+======================================================
+
+The Inter-VM shared memory device is designed to share a region of memory to
+userspace in multiple virtual guests.  The memory region does not belong to any
+guest, but is a POSIX memory object on the host.  Optionally, the device may
+support sending interrupts to other guests sharing the same memory region.
+
+
+The Inter-VM PCI device
+=======================
+
+BARs
+----
+
+The device supports three BARs.  BAR0 is a 1 Kbyte MMIO region to support
+registers.  BAR1 is used for MSI-X when it is enabled in the device.  BAR2 is
+used to map the shared memory object from the host.  The size of BAR2 is
+specified when the guest is started and must be a power of 2 in size.
+
+Registers
+---------
+
+The device currently supports 4 registers of 32-bits each.  Registers
+are used for synchronization between guests sharing the same memory object when
+interrupts are supported (this requires using the shared memory server).
+
+The server assigns each VM an ID number and sends this ID number to the Qemu
+process when the guest starts.
+
+    enum ivshmem_registers {
+        IntrMask = 0,
+        IntrStatus = 4,
+        IVPosition = 8,
+        Doorbell = 12
+    };
+
+The first two registers are the interrupt mask and status registers.  Mask and
+status are only used with pin-based interrupts.  They are unused with MSI
+interrupts.
+
+**Status Register:** The status register is set to 1 when an interrupt occurs.
+
+**Mask Register:** The mask register is bitwise ANDed with the interrupt status
+and the result will raise an interrupt if it is non-zero.  However, since 1 is
+the only value the status will be set to, it is only the first bit of the mask
+that has any effect.  Therefore interrupts can be masked by setting the first
+bit to 0 and unmasked by setting the first bit to 1.
+
+**IVPosition Register:** The IVPosition register is read-only and reports the
+guest's ID number.  The guest IDs are non-negative integers.  When using the
+server, since the server is a separate process, the VM ID will only be set when
+the device is ready (shared memory is received from the server and accessible
+via the device).  If the device is not ready, the IVPosition will return -1.
+Applications should ensure that they have a valid VM ID before accessing the
+shared memory.
+
+**Doorbell Register:**  To interrupt another guest, a guest must write to the
+Doorbell register.  The doorbell register is 32-bits, logically divided into
+two 16-bit fields.  The high 16-bits are the guest ID to interrupt and the low
+16-bits are the interrupt vector to trigger.  The semantics of the value
+written to the doorbell depends on whether the device is using MSI or a regular
+pin-based interrupt.  In short, MSI uses vectors while regular interrupts set
+the status register.
+
+### Regular Interrupts
+
+If regular interrupts are used (due to either a guest not supporting MSI or the
+user specifying not to use them on startup) then the value written to the lower
+16-bits of the Doorbell register results is arbitrary and will trigger an
+interrupt in the destination guest.
+
+### Message Signalled Interrupts
+
+A ivshmem device may support multiple MSI vectors.  If so, the lower 16-bits
+written to the Doorbell register must be between 0 and the maximum number of
+vectors the guest supports.  The lower 16 bits written to the doorbell is the
+MSI vector that will be raised in the destination guest.  The number of MSI
+vectors is configurable but it is set when the VM is started.
+
+The important thing to remember with MSI is that it is only a signal, no status
+is set (since MSI interrupts are not shared).  All information other than the
+interrupt itself should be communicated via the shared memory region.  Devices
+supporting multiple MSI vectors can use different vectors to indicate different
+events have occurred.  The semantics of interrupt vectors are left to the
+user's discretion.
+
+
+Usage in the Guest
+==================
+
+The shared memory device is intended to be used with the provided UIO driver.
+Very little configuration is needed.  The guest should map BAR0 to access the
+registers (an array of 32-bit ints allows simple writing) and map BAR2 to
+access the shared memory region itself.  The size of the shared memory region
+is specified when the guest (or shared memory server) is started.  A guest may
+map the whole shared memory region or only part of it.
diff --git a/docs/specs/ivshmem_device_spec.txt b/docs/specs/ivshmem_device_spec.txt
deleted file mode 100644
index 23dd2ba..0000000
--- a/docs/specs/ivshmem_device_spec.txt
+++ /dev/null
@@ -1,96 +0,0 @@
-
-Device Specification for Inter-VM shared memory device
-------------------------------------------------------
-
-The Inter-VM shared memory device is designed to share a region of memory to
-userspace in multiple virtual guests.  The memory region does not belong to any
-guest, but is a POSIX memory object on the host.  Optionally, the device may
-support sending interrupts to other guests sharing the same memory region.
-
-
-The Inter-VM PCI device
------------------------
-
-*BARs*
-
-The device supports three BARs.  BAR0 is a 1 Kbyte MMIO region to support
-registers.  BAR1 is used for MSI-X when it is enabled in the device.  BAR2 is
-used to map the shared memory object from the host.  The size of BAR2 is
-specified when the guest is started and must be a power of 2 in size.
-
-*Registers*
-
-The device currently supports 4 registers of 32-bits each.  Registers
-are used for synchronization between guests sharing the same memory object when
-interrupts are supported (this requires using the shared memory server).
-
-The server assigns each VM an ID number and sends this ID number to the Qemu
-process when the guest starts.
-
-enum ivshmem_registers {
-    IntrMask = 0,
-    IntrStatus = 4,
-    IVPosition = 8,
-    Doorbell = 12
-};
-
-The first two registers are the interrupt mask and status registers.  Mask and
-status are only used with pin-based interrupts.  They are unused with MSI
-interrupts.
-
-Status Register: The status register is set to 1 when an interrupt occurs.
-
-Mask Register: The mask register is bitwise ANDed with the interrupt status
-and the result will raise an interrupt if it is non-zero.  However, since 1 is
-the only value the status will be set to, it is only the first bit of the mask
-that has any effect.  Therefore interrupts can be masked by setting the first
-bit to 0 and unmasked by setting the first bit to 1.
-
-IVPosition Register: The IVPosition register is read-only and reports the
-guest's ID number.  The guest IDs are non-negative integers.  When using the
-server, since the server is a separate process, the VM ID will only be set when
-the device is ready (shared memory is received from the server and accessible via
-the device).  If the device is not ready, the IVPosition will return -1.
-Applications should ensure that they have a valid VM ID before accessing the
-shared memory.
-
-Doorbell Register:  To interrupt another guest, a guest must write to the
-Doorbell register.  The doorbell register is 32-bits, logically divided into
-two 16-bit fields.  The high 16-bits are the guest ID to interrupt and the low
-16-bits are the interrupt vector to trigger.  The semantics of the value
-written to the doorbell depends on whether the device is using MSI or a regular
-pin-based interrupt.  In short, MSI uses vectors while regular interrupts set the
-status register.
-
-Regular Interrupts
-
-If regular interrupts are used (due to either a guest not supporting MSI or the
-user specifying not to use them on startup) then the value written to the lower
-16-bits of the Doorbell register results is arbitrary and will trigger an
-interrupt in the destination guest.
-
-Message Signalled Interrupts
-
-A ivshmem device may support multiple MSI vectors.  If so, the lower 16-bits
-written to the Doorbell register must be between 0 and the maximum number of
-vectors the guest supports.  The lower 16 bits written to the doorbell is the
-MSI vector that will be raised in the destination guest.  The number of MSI
-vectors is configurable but it is set when the VM is started.
-
-The important thing to remember with MSI is that it is only a signal, no status
-is set (since MSI interrupts are not shared).  All information other than the
-interrupt itself should be communicated via the shared memory region.  Devices
-supporting multiple MSI vectors can use different vectors to indicate different
-events have occurred.  The semantics of interrupt vectors are left to the
-user's discretion.
-
-
-Usage in the Guest
-------------------
-
-The shared memory device is intended to be used with the provided UIO driver.
-Very little configuration is needed.  The guest should map BAR0 to access the
-registers (an array of 32-bit ints allows simple writing) and map BAR2 to
-access the shared memory region itself.  The size of the shared memory region
-is specified when the guest (or shared memory server) is started.  A guest may
-map the whole shared memory region or only part of it.
-- 
1.7.4.1

[Qemu-devel] [PATCH 4/5] docs: convert acpi_pci_hotplug to markdown

[Anthony Liguori]

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 docs/specs/acpi_pci_hotplug.md  |   38 ++++++++++++++++++++++++++++++++++++++
 docs/specs/acpi_pci_hotplug.txt |   37 -------------------------------------
 2 files changed, 38 insertions(+), 37 deletions(-)
 create mode 100644 docs/specs/acpi_pci_hotplug.md
 delete mode 100644 docs/specs/acpi_pci_hotplug.txt

diff --git a/docs/specs/acpi_pci_hotplug.md b/docs/specs/acpi_pci_hotplug.md
new file mode 100644
index 0000000..9621187
--- /dev/null
+++ b/docs/specs/acpi_pci_hotplug.md
@@ -0,0 +1,38 @@
+QEMU<->ACPI BIOS PCI hotplug interface
+======================================
+
+QEMU supports PCI hotplug via ACPI, for PCI bus 0. This document describes the
+interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access)
+----------------------------------------------------
+
+Generic ACPI GPE block. Bit 1 (GPE.1) used to notify PCI hotplug/eject event to
+ACPI BIOS, via SCI interrupt.
+
+PCI slot injection notification pending (IO port 0xae00-0xae03, 4-byte access)
+------------------------------------------------------------------------------
+
+Slot injection notification pending. One bit per slot.
+
+Read by ACPI BIOS GPE.1 handler to notify OS of injection events.
+
+PCI slot removal notification (IO port 0xae04-0xae07, 4-byte access)
+--------------------------------------------------------------------
+
+Slot removal notification pending. One bit per slot.
+
+Read by ACPI BIOS GPE.1 handler to notify OS of removal events.
+
+PCI device eject (IO port 0xae08-0xae0b, 4-byte access)
+-------------------------------------------------------
+
+Used by ACPI BIOS _EJ0 method to request device removal. One bit per slot.
+
+Reads return 0.
+
+PCI removability status (IO port 0xae0c-0xae0f, 4-byte access)
+--------------------------------------------------------------
+
+Used by ACPI BIOS _RMV method to indicate removability status to OS. One bit
+per slot.
diff --git a/docs/specs/acpi_pci_hotplug.txt b/docs/specs/acpi_pci_hotplug.txt
deleted file mode 100644
index f0f74a7..0000000
--- a/docs/specs/acpi_pci_hotplug.txt
+++ /dev/null
@@ -1,37 +0,0 @@
-QEMU<->ACPI BIOS PCI hotplug interface
---------------------------------------
-
-QEMU supports PCI hotplug via ACPI, for PCI bus 0. This document
-describes the interface between QEMU and the ACPI BIOS.
-
-ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
------------------------------------------
-
-Generic ACPI GPE block. Bit 1 (GPE.1) used to notify PCI hotplug/eject
-event to ACPI BIOS, via SCI interrupt.
-
-PCI slot injection notification pending (IO port 0xae00-0xae03, 4-byte access):
----------------------------------------------------------------
-Slot injection notification pending. One bit per slot.
-
-Read by ACPI BIOS GPE.1 handler to notify OS of injection
-events.
-
-PCI slot removal notification (IO port 0xae04-0xae07, 4-byte access):
------------------------------------------------------
-Slot removal notification pending. One bit per slot.
-
-Read by ACPI BIOS GPE.1 handler to notify OS of removal
-events.
-
-PCI device eject (IO port 0xae08-0xae0b, 4-byte access):
-----------------------------------------
-
-Used by ACPI BIOS _EJ0 method to request device removal. One bit per slot.
-Reads return 0.
-
-PCI removability status (IO port 0xae0c-0xae0f, 4-byte access):
------------------------------------------------
-
-Used by ACPI BIOS _RMV method to indicate removability status to OS. One
-bit per slot.
-- 
1.7.4.1

[Qemu-devel] [PATCH 5/5] docs: add readme to specifications directory

[Anthony Liguori]

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 docs/specs/README.md |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
 create mode 100644 docs/specs/README.md

diff --git a/docs/specs/README.md b/docs/specs/README.md
new file mode 100644
index 0000000..29fec3f
--- /dev/null
+++ b/docs/specs/README.md
@@ -0,0 +1,8 @@
+QEMU Virtual Device Specifications
+==================================
+
+This directory contains specifications for devices and/or interfaces that were
+created in QEMU and do not have a physical analog.
+
+The latest version of these specifications are always located in the qemu.org
+git tree.
-- 
1.7.4.1

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Stefan Weil]

Am 14.11.2011 23:41, schrieb Anthony Liguori:
> Right now our specs are written in psuedo-wiki syntax.  This series converts
> them to markdown.  markdown is a simple markup format that's gaining in
> popularity.
>
> The big advantage of using markdown is that there are tools that can convert it
> to relatively simple HTML.  That means we can build a make infrastructure that
> generates a nice set of static web pages.
>
> The syntax is also more human friendly than mediawiki syntax.
>
> To see what the stylized version of this looks like, check out:
>
>    https://github.com/aliguori/qemu/tree/markdown/docs/specs

Maybe the file names could also be cleaned: either remove _spec
as it is not needed (or do you add _src to source file names),
or add it to all file names.

Regards,
Stefan Weil

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Avi Kivity]

On 11/15/2011 12:41 AM, Anthony Liguori wrote:
> Right now our specs are written in psuedo-wiki syntax.  This series converts
> them to markdown.  markdown is a simple markup format that's gaining in
> popularity.
>
> The big advantage of using markdown is that there are tools that can convert it
> to relatively simple HTML.  That means we can build a make infrastructure that
> generates a nice set of static web pages.
>
> The syntax is also more human friendly than mediawiki syntax.
>
> To see what the stylized version of this looks like, check out:
>
>   https://github.com/aliguori/qemu/tree/markdown/docs/specs
>
>

Nice.  Suggest you enable rename detection, to make patches like these
easier to read (not that it truly matters in the particular case).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Anthony Liguori]

On 11/15/2011 02:28 AM, Avi Kivity wrote:
> On 11/15/2011 12:41 AM, Anthony Liguori wrote:
>> Right now our specs are written in psuedo-wiki syntax.  This series converts
>> them to markdown.  markdown is a simple markup format that's gaining in
>> popularity.
>>
>> The big advantage of using markdown is that there are tools that can convert it
>> to relatively simple HTML.  That means we can build a make infrastructure that
>> generates a nice set of static web pages.
>>
>> The syntax is also more human friendly than mediawiki syntax.
>>
>> To see what the stylized version of this looks like, check out:
>>
>>    https://github.com/aliguori/qemu/tree/markdown/docs/specs
>>
>>
>
> Nice.  Suggest you enable rename detection, to make patches like these
> easier to read (not that it truly matters in the particular case).

I haven't figured out yet how to make this sane to merge, but I've also 
converted qemu-doc.texi to a bunch of separate markdown files[1].

The info is fairly out of date.  I'll try to get patches out RSN so that we can 
all take a pass at trying to modernize some of the sections before the release.

[1] https://github.com/aliguori/qemu/tree/markdown/docs/manual

Regards,

Anthony Liguori

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Avi Kivity]

On 11/15/2011 03:44 PM, Anthony Liguori wrote:
> On 11/15/2011 02:28 AM, Avi Kivity wrote:
>> On 11/15/2011 12:41 AM, Anthony Liguori wrote:
>>> Right now our specs are written in psuedo-wiki syntax.  This series
>>> converts
>>> them to markdown.  markdown is a simple markup format that's gaining in
>>> popularity.
>>>
>>> The big advantage of using markdown is that there are tools that can
>>> convert it
>>> to relatively simple HTML.  That means we can build a make
>>> infrastructure that
>>> generates a nice set of static web pages.
>>>
>>> The syntax is also more human friendly than mediawiki syntax.
>>>
>>> To see what the stylized version of this looks like, check out:
>>>
>>>    https://github.com/aliguori/qemu/tree/markdown/docs/specs
>>>
>>>
>>
>> Nice.  Suggest you enable rename detection, to make patches like these
>> easier to read (not that it truly matters in the particular case).
>
> I haven't figured out yet how to make this sane to merge, but I've
> also converted qemu-doc.texi to a bunch of separate markdown files[1].
>
> The info is fairly out of date.  I'll try to get patches out RSN so
> that we can all take a pass at trying to modernize some of the
> sections before the release.
>
> [1] https://github.com/aliguori/qemu/tree/markdown/docs/manual
>

Does markdown support rendering into man pages?

A similar alternative is asciidoc, which is used by git.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Alex Bradbury]

On 15 November 2011 13:51, Avi Kivity <avi@redhat.com> wrote:
> Does markdown support rendering into man pages?

You can do this via pandoc:
http://johnmacfarlane.net/pandoc/

Alex

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Anthony Liguori]

On 11/15/2011 07:51 AM, Avi Kivity wrote:
> On 11/15/2011 03:44 PM, Anthony Liguori wrote:
>>> Nice.  Suggest you enable rename detection, to make patches like these
>>> easier to read (not that it truly matters in the particular case).
>>
>> I haven't figured out yet how to make this sane to merge, but I've
>> also converted qemu-doc.texi to a bunch of separate markdown files[1].
>>
>> The info is fairly out of date.  I'll try to get patches out RSN so
>> that we can all take a pass at trying to modernize some of the
>> sections before the release.
>>
>> [1] https://github.com/aliguori/qemu/tree/markdown/docs/manual
>>
>
> Does markdown support rendering into man pages?
>
> A similar alternative is asciidoc, which is used by git.

I was thinking of doing a2x for the man pages (which is more or less what git does).

The man pages are generated by qemu-doc.texi so I think I'm going to have to 
strip out the extracted info, but leave enough in qemu-doc.texi so that we can 
keep generating the man pages.  Once we clean up the user docs a bit, we can 
convert the man pages too.

Regards,

Anthony Liguori

>

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Stefano Stabellini]

On Tue, 15 Nov 2011, Alex Bradbury wrote:
> On 15 November 2011 13:51, Avi Kivity <avi@redhat.com> wrote:
> > Does markdown support rendering into man pages?
> 
> You can do this via pandoc:
> http://johnmacfarlane.net/pandoc/
 
Actually we are having the very same issue on xen right now: we have a
manual written in markdown and we would like to render it into a man
page. I found that pandoc generates man pages of terrible quality, but
ronn works pretty well. Unfortunately it is not easy to find it in
distros.
Overall I think it is better to use something else than markdown for man
pages whenever possible.

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Stefan Weil]

Am 15.11.2011 14:44, schrieb Anthony Liguori:
> On 11/15/2011 02:28 AM, Avi Kivity wrote:
>> On 11/15/2011 12:41 AM, Anthony Liguori wrote:
>>> Right now our specs are written in psuedo-wiki syntax.  This series 
>>> converts
>>> them to markdown.  markdown is a simple markup format that's gaining in
>>> popularity.
>>>
>>> The big advantage of using markdown is that there are tools that can 
>>> convert it
>>> to relatively simple HTML.  That means we can build a make 
>>> infrastructure that
>>> generates a nice set of static web pages.
>>>
>>> The syntax is also more human friendly than mediawiki syntax.
>>>
>>> To see what the stylized version of this looks like, check out:
>>>
>>>    https://github.com/aliguori/qemu/tree/markdown/docs/specs
>>>
>>>
>>
>> Nice.  Suggest you enable rename detection, to make patches like these
>> easier to read (not that it truly matters in the particular case).
>
> I haven't figured out yet how to make this sane to merge, but I've 
> also converted qemu-doc.texi to a bunch of separate markdown files[1].
>
> The info is fairly out of date.  I'll try to get patches out RSN so 
> that we can all take a pass at trying to modernize some of the 
> sections before the release.
>
> [1] https://github.com/aliguori/qemu/tree/markdown/docs/manual
>
> Regards,
>
> Anthony Liguori

Markdown is nice for separated specs and README files,
but is it also possible to create a complete printed manual
from all markdown files?

I thought of a different solution: keep qemu-doc.texi and qemu-tech.texi
and integrate texi files made from markdown files (with pandoc) into
these two documents to get complete manuals.

Regards,

Stefan Weil

Re: [Qemu-devel] [PATCH 0/5] docs: convert specifications to markdown

[Anthony Liguori]

On 11/15/2011 12:06 PM, Stefan Weil wrote:
> Am 15.11.2011 14:44, schrieb Anthony Liguori:
> Markdown is nice for separated specs and README files,
> but is it also possible to create a complete printed manual
> from all markdown files?

I think high quality information beats pretty formatting any day.  The problem I 
see with our current documentation strategy is that it's very monolithic (one 
big printed manual) and it's in a format that many people consider obscure.

I think small stand alone documents that are easy to author are going to result 
in a better end user experience than one big document with pretty formatting.

Regards,

Anthony Liguori

> I thought of a different solution: keep qemu-doc.texi and qemu-tech.texi
> and integrate texi files made from markdown files (with pandoc) into
> these two documents to get complete manuals.
>
> Regards,
>
> Stefan Weil
>