From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43277) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XV6cW-0004PK-FW for qemu-devel@nongnu.org; Fri, 19 Sep 2014 18:25:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XV6cO-0004Y9-Ay for qemu-devel@nongnu.org; Fri, 19 Sep 2014 18:25:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26695) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XV6cO-0004W7-37 for qemu-devel@nongnu.org; Fri, 19 Sep 2014 18:25:32 -0400 From: Eric Blake Date: Fri, 19 Sep 2014 16:24:49 -0600 Message-Id: <1411165504-18198-5-git-send-email-eblake@redhat.com> In-Reply-To: <1411165504-18198-1-git-send-email-eblake@redhat.com> References: <1411165504-18198-1-git-send-email-eblake@redhat.com> Subject: [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Luiz Capitulino , Fam Zheng , Markus Armbruster , wenchaoqemu@gmail.com Go into more details about the various types of valid expressions in a qapi schema, including tweaks to document fixes being done later in the current patch series. Signed-off-by: Eric Blake --- docs/qapi-code-gen.txt | 249 ++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 217 insertions(+), 32 deletions(-) diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt index 8313ba6..3a79629 100644 --- a/docs/qapi-code-gen.txt +++ b/docs/qapi-code-gen.txt @@ -22,35 +22,136 @@ code are used. This file defines the types, commands, and events used by QMP. It should fully describe the interface used by QMP. -This file is designed to be loosely based on JSON although it's technically -executable Python. While dictionaries are used, they are parsed as -OrderedDicts so that ordering is preserved. +A QAPI file is designed to be loosely based on JSON although it's +technically executable Python. A valid QAPI schema consists of a list +of top-level expressions, with no commas between them. While +dictionaries are used, they are parsed as OrderedDicts so that +ordering is preserved; ordering doesn't matter for top-level +expressions, but does matter within 'data' members. QAPI input is +written using 'single quotes' instead of JSON's "double quotes" (in +contrast, QMP is strict JSON and only uses "double quotes"). As in +JSON, trailing commas are not permitted in arrays or dictionaries. -There are two basic syntaxes used, type definitions and command definitions. +Comments are allowed; anything between an unquoted # and the following +newline is ignored. Although there is not yet a documentation +generator, a form of stylized comments has developed for consistently +documenting details about an expression and when it was added to the +schema. The documentation is delimited between two lines of ##, then +the first line names the expression, an optional overview is provided, +then individual documentation about each member of 'data' is provided, +and finally, a 'Since: x.y.z' tag lists the release that introduced +the expression. Optional fields are tagged with the phrase +'#optional', often with their default value; and extensions added +after the expression was first released are also given a '(since +x.y.z)' comment. For example: -The first syntax defines a type and is represented by a dictionary. There are -three kinds of user-defined types that are supported: complex types, -enumeration types and union types. + ## + # @BlockStats: + # + # Statistics of a virtual block device or a block backing device. + # + # @device: #optional If the stats are for a virtual block device, the name + # corresponding to the virtual block device. + # + # @stats: A @BlockDeviceStats for the device. + # + # @parent: #optional This describes the file block device if it has one. + # + # @backing: #optional This describes the backing block device if it has one. + # (Since 2.0) + # + # Since: 0.14.0 + ## + { 'type': 'BlockStats', + 'data': {'*device': 'str', 'stats': 'BlockDeviceStats', + '*parent': 'BlockStats', + '*backing': 'BlockStats'} } -Generally speaking, types definitions should always use CamelCase for the type -names. Command names should be all lower case with words separated by a hyphen. +The schema sets up a series of types, as well as commands and events +that will use those types. Forward references are allowed: the parser +scans in two passes, where the first pass learns all type names, and +the second validates the schema and generates the code. This allows +the definition of complex structs that can have mutually recursive +types, and allows for indefinite nesting of QMP that satisfies the +schema. A type name should not be defined more than once. +There are six top-level expressions recognized by the parser: +'include', 'command', 'type', 'enum', 'union', and 'event'. There are +several built-in types, such as 'int' and 'str'; additionally, the +top-level expressions can define complex types, enumeration types, and +union types. The 'command' expression can refer to existing types by +name, or list an anonymous type as a dictionary. Listing a type name +inside an array refers to a single-dimension array of that type; +multi-dimension arrays are not directly supported (although an array +of a complex struct that contains an array member is possible). + +Generally speaking, types definitions should always use CamelCase for +user-defined type names, while built-in types are lowercase. Command +names, and field names within a type, should be all lower case with +words separated by a hyphen. However, some existing older commands +and complex types use underscore; when extending such expressions, +consistency is preferred over blindly avoiding underscore. Event +names should be ALL_CAPS with words separated by underscore. The +special string '**' appears for some commands that manually perform +their own type checking rather than relying on the type-safe code +produced by the qapi code generators. + +Any command, type, or field name beginning with "x-" is marked +experimental, and may be withdrawn in a future release. Downstream +vendors may add extensions; such extensions should begin with a prefix +matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of +the vendor), even if the rest of the command or field name uses dash +(example: __com.redhat_drive-mirror). Other than the dots used in +RFQDN of a downstream extension, all command, type, and field names +should begin with a letter, and contain only ASCII letters, numbers, +dash, and underscore. It is okay to reuse names that might collide +with programming languages; the generator will rename a field named +"default" in the QAPI to "q_default" in the generated C code. + + +=== Built-in Types === + +The following types are built-in to the parser: + 'str' - arbitrary UTF-8 string + 'int' - 64-bit signed integer (although the C code may place further + restrictions on acceptable range) + 'number' - floating point number + 'bool' - JSON value of true or false + 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum + bit size + 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts + 'size' - like 'uint64', but allows scaled suffix from command line + visitor === Includes === +Usage: { 'include': 'str' } + The QAPI schema definitions can be modularized using the 'include' directive: { 'include': 'path/to/file.json'} The directive is evaluated recursively, and include paths are relative to the -file using the directive. Multiple includes of the same file are safe. +file using the directive. Multiple includes of the same file are +safe. No other keys should appear in the expression, and the include +value should be a string. + +As a matter of style, it is a good idea to have all files be +self-contained, but at the moment, nothing prevents an included file +from making a forward reference to a type that is only introduced by +an outer file. The parser may be made stricter in the future to +prevent incomplete include files. === Complex types === -A complex type is a dictionary containing a single key whose value is a -dictionary. This corresponds to a struct in C or an Object in JSON. An -example of a complex type is: +Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' } + +A complex type is a dictionary containing a single 'data' key whose +value is a dictionary. This corresponds to a struct in C or an Object +in JSON. Each value of the 'data' dictionary must be the name of a +complex, enum, union, or built-in type, or a one-element array +containing a type name. An example of a complex type is: { 'type': 'MyType', 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } } @@ -100,15 +201,42 @@ both fields like this: { "file": "/some/place/my-image", "backing": "/some/place/my-backing-file" } + === Enumeration types === -An enumeration type is a dictionary containing a single key whose value is a -list of strings. An example enumeration is: +Usage: { 'enum': 'str', 'data': [ 'str' ] } + +An enumeration type is a dictionary containing a single 'data' key +whose value is a list of strings. An example enumeration is: { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] } +Nothing prevents an empty enumeration, although it is probably not +useful. The list of strings should be lower case; if an enum name +represents multiple words, use '-' between words. The string 'max' is +not allowed as an enum value, and values should not be repeated. + +The enumeration values are passed as strings over the QMP protocol, +but are encoded as C enum integral values in generated code. While +the C code starts numbering at 0, it is better to use explicit +comparisons to enum values than implicit comparisons to 0; the C code +will also include a generated enum member ending in _MAX for tracking +the size of the enum, useful when using common functions for +converting between strings and enum values. Since the wire format +always passes by name, it is acceptable to reorder or add new +enumeration members in any location without breaking QMP clients; +however, removing enum values would break compatibility. For any +complex type that has a field that will only contain a finite set of +string values, using an enum type for that field is better than +open-coding the field to be type 'str'. + + === Union types === +Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name', + '*discriminator': 'enum-type-name' } +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} } + Union types are used to let the user choose between several different data types. A union type is defined using a dictionary as explained in the following paragraphs. @@ -153,8 +281,10 @@ And it looks like this on the wire: Flat union types avoid the nesting on the wire. They are used whenever a specific field of the base type is declared as the discriminator ('type' is -then no longer generated). The discriminator must be of enumeration type. -The above example can then be modified as follows: +then no longer generated). The discriminator must be of enumeration +type, and the keys of the 'data' dictionary must match the enumeration +keys (although not necessarily in the same order). The above example +can then be modified as follows: { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] } { 'type': 'BlockdevCommonOptions', @@ -200,23 +330,78 @@ This example allows using both of the following example objects: === Commands === -Commands are defined by using a list containing three members. The first -member is the command name, the second member is a dictionary containing -arguments, and the third member is the return type. +Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name', + '*returns': 'type', + '*gen': 'no', '*success-response': 'no' } -An example command is: +Commands are defined by using a dictionary containing several members, +where three members are most common. The 'data' member is optional; if +absent, the command accepts an optional empty dictionary. If present, +it must be the string name of a complex type, a one-element array +containing the name of a complex type, or a dictionary that declares +an anonymous type with the same semantics as a 'type' expression, with +one exception noted below. The 'returns' member is optional; if +absent, the command returns an empty dictionary. If present, it must +be the string name of a complex or built-in type, a one-element array +containing the name of a complex or built-in type, or a dictionary +that declares an anonymous type with the same semantics as a 'type' +expression, with one exception noted below. + +Although it is permitted to have the 'returns' member name a built-in +type, any command that does this cannot be extended to return +additional information in the future; thus, new commands should +strongly consider returning a dictionary-based type, even if it only +contains one field at the present. Besides, all commands return a +dictionary to report failure. + +Some example commands: + + { 'command': 'my-first-command', + 'data': { 'arg1': 'str', '*arg2': 'str' } } + { 'type': 'MyType', 'data': { '*value': 'str' } } + { 'command': 'my-second-command', + 'returns': [ 'MyType' ] } + +which would validate this QMP transaction: + + => { "execute": "my-first-command", + "arguments": { "arg1": "hello" } } + <= { "return": { } } + => { "execute": "my-second-command" } + <= { "return": [ { "value": "one" }, { } ] } + +In rare cases, QAPI cannot express a type-safe representation of a +corresponding QMP command. In these cases, if the command expression +includes the key 'gen' with value 'no', then the 'data' or 'returns' +member that intends to bypass generated type-safety and do its own +manual validation should use '**' rather than a valid type name. +Please try to avoid adding new commands that rely on this, and instead +use type-safe unions. For an example of bypass usage: + + { 'command': 'netdev_add', + 'data': {'type': 'str', 'id': 'str', '*props': '**'}, + 'gen': 'no' } + +Normally, the QAPI schema is used to describe synchronous exchanges, +where a response is expected. But in some cases, the action of a +command is expected to change state in a way that a successful +response is not possible (a failure message still returns a +dictionary). In this case, the command expression should include the +optional key 'success-response' with value 'no'. - { 'command': 'my-command', - 'data': { 'arg1': 'str', '*arg2': 'str' }, - 'returns': 'str' } === Events === -Events are defined with the keyword 'event'. When 'data' is also specified, -additional info will be included in the event. Finally there will be C API -generated in qapi-event.h; when called by QEMU code, a message with timestamp -will be emitted on the wire. If timestamp is -1, it means failure to retrieve -host time. +Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' } + +Events are defined with the keyword 'event'. It is not allowed to +name an event 'MAX', since the generator also produces a C enumeration +of all event names with a generated _MAX value at the end. When +'data' is also specified, additional info will be included in the +event, with similar semantics to a 'type' expression. Finally there +will be C API generated in qapi-event.h; when called by QEMU code, a +message with timestamp will be emitted on the wire. If timestamp is +-1, it means failure to retrieve host time. An example event is: @@ -311,7 +496,7 @@ Example: #ifndef EXAMPLE_QAPI_TYPES_H #define EXAMPLE_QAPI_TYPES_H -[Builtin types omitted...] +[Built-in types omitted...] typedef struct UserDefOne UserDefOne; @@ -324,7 +509,7 @@ Example: struct UserDefOneList *next; } UserDefOneList; -[Functions on builtin types omitted...] +[Functions on built-in types omitted...] struct UserDefOne { @@ -423,7 +608,7 @@ Example: #ifndef EXAMPLE_QAPI_VISIT_H #define EXAMPLE_QAPI_VISIT_H -[Visitors for builtin types omitted...] +[Visitors for built-in types omitted...] void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp); void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp); -- 1.9.3