[PATCH v11 07/13] admin: Add device resource objects admin commands

[Parav Pandit <parav@nvidia.com>]

Add generic administration commands create, modify, query and destroy
device resource object.

Each resource object defines resource specific attributes for the commands.
Once the resource object is created by the driver, it can be query/modify or
destroyed by the driver.

Each resource object is identified using a unique resource id per
resource type.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
changelog:
v10->v11:
- addressed comments from Satananda
- improved driver normative for two resource objects with same ID
v9->v10:
- added hyperreference to quickly jump to command definition
- renamed rid to just id as it is part of the resource related structs
- removed length field from the commands as it can be derived
- rephrase for group_member_id
- squashed the patch that added the error code
- renamed and rephrase 'device resource' to 'device resource object'
- removed performance part of commit log for future and implementation
  creativity

v8->v9:
- changed command name from read_data/read_result to query_data/query_result
  to match the opcode name and description
- added fields_bmap bitmap to support future support of optional fields
  in the resources without inventing new set of commands, for example
  multiple pages in the VQ, PASID in VQ, flow counter id in flow filter rule
  etc.
- fixed spelling from 'destroyd' to 'destroyed'
- fixed 'upto' to 'up to'
---
 admin-cmds-capabilities.tex     |   3 +
 admin-cmds-resource-objects.tex | 199 ++++++++++++++++++++++++++++++++
 admin.tex                       |  14 ++-
 conformance.tex                 |   2 +
 4 files changed, 216 insertions(+), 2 deletions(-)

diff --git a/admin-cmds-capabilities.tex b/admin-cmds-capabilities.tex
index 07b8ddb..8119a56 100644
--- a/admin-cmds-capabilities.tex
+++ b/admin-cmds-capabilities.tex
@@ -210,6 +210,9 @@ \subsubsection{Device and driver capabilities}\label{sec:Basic Facilities of a V
 
 Upon a device reset, the device MUST reset all driver capabilities.
 
+The device SHOULD treat the driver resource limits as zero if the
+driver has not set such capability, unless otherwise explicitly stated.
+
 \drivernormative{\paragraph}{Device and driver capabilities}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities}
 
 The driver MUST send the command VIRTIO_ADMIN_CMD_DRIVER_CAP_SET before
diff --git a/admin-cmds-resource-objects.tex b/admin-cmds-resource-objects.tex
index a74cb3b..291f7e6 100644
--- a/admin-cmds-resource-objects.tex
+++ b/admin-cmds-resource-objects.tex
@@ -67,3 +67,202 @@ \subsubsection{Device resource objects}\label{sec:Basic Facilities of a Virtio D
 can create resources up to the published \field{limit}. The driver can
 destroy and recreate the resource one or multiple times. Upon device reset,
 all resource objects created by the driver are destroyed within the device.
+
+Following administration commands control device resource objects,
+they are only supported for the self group type:
+
+\begin{enumerate}
+\item VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE
+\item VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY
+\item VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY
+\item VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY
+\end{enumerate}
+
+Each resource object administration command uses a common header
+\field{struct virtio_admin_cmd_resource_obj_cmd_hdr}.
+
+\begin{lstlisting}
+struct virtio_admin_cmd_resource_obj_cmd_hdr {
+        le16 type;
+        u8 reserved[2];
+        le32 id; /* Indicates unique resource object id per resource object type */
+};
+
+\end{lstlisting}
+
+\field{type} refers to the device resource object type.
+\field{id} uniquely identifies the resource object of a specified \field{type}.
+
+\paragraph{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE}
+\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE}
+
+This command creates the specified resource object of \field{type} identified by the
+resource id \field{id}. The valid range of \field{id} is defined by the
+device in the related device capability. The driver assigns the unique \field{id}
+for the resource for the specified \field{type}.
+
+For the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE, \field{opcode} is set to 0xa.
+\field{group_member_id} is set to zero.
+The \field{command_specific_data} is in the format
+\field{struct virtio_admin_cmd_resource_obj_create_data}.
+\field{resource_obj_specific_data} refers to the resource object specific data.
+Each resource uses a different \field{resource_obj_specific_data} and is described
+separately.
+
+\field{flags} is reserved for future extension for optional resource object attributes and
+is set to 0. Each resource object uses a different value for
+\field{flags} and it is described separately.
+
+\begin{lstlisting}
+struct virtio_admin_cmd_resource_obj_create_data {
+        struct virtio_admin_cmd_resource_obj_cmd_hdr hdr;
+        le64 flags;
+        u8 resource_obj_specific_data[];
+};
+\end{lstlisting}
+
+When the command completes successfully, the resource object is created by the
+device and the device can immediately begin using it.
+This command has no command specific result.
+
+\paragraph{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY}
+\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY}
+
+This command modifies the attributes of an existing device resource object.
+For the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY, \field{opcode} is set to 0xb.
+The \field{command_specific_data} is in the format
+\field{struct virtio_admin_cmd_resource_modify_data}.
+\field{group_member_id} is set to zero.
+\field{id} identifies the resource object of type \field{type} whose attributes
+to modify.
+This command modifies the attributes supplied in \field{resource_obj_specific_data}.
+
+\begin{lstlisting}
+struct virtio_admin_cmd_resource_modify_data {
+        struct virtio_admin_cmd_resource_obj_cmd_hdr hdr;
+        le64 flags;
+        u8 resource_obj_specific_data[];
+};
+\end{lstlisting}
+
+This command has no command specific result.
+When the command completes successfully, attributes of the resource object is
+set to the values supplied in \field{resource_obj_specific_data}.
+
+\paragraph{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY}
+\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY}
+
+This command queries attributes of the existing resource object.
+For the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY, \field{opcode} is set to 0xc.
+\field{group_member_id} is set to zero.
+The \field{command_specific_data} is in the format
+\field{struct virtio_admin_cmd_resource_obj_query_data}.
+\field{id} identifies the existing resource object of type \field{type} whose
+attributes to query.
+
+\begin{lstlisting}
+struct virtio_admin_cmd_resource_obj_query_data {
+        struct virtio_admin_cmd_resource_obj_cmd_hdr hdr;
+        le64 flags;
+};
+\end{lstlisting}
+
+\begin{lstlisting}
+struct virtio_admin_cmd_resource_obj_query_result {
+        u8 resource_obj_specific_result[];
+};
+\end{lstlisting}
+
+\field{command_specific_result} is in the format
+\field{virtio_admin_cmd_resource_obj_query_result}.
+
+When the command completes successfully, the attributes of the specified
+resource object are are set in \field{resource_obj_specific_data}.
+
+\paragraph{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY}
+\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY}
+
+This command destroys the previously created device resource object.
+For the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY, \field{opcode} is set to 0xd.
+The \field{command_specific_data} is in the format
+\field{struct virtio_admin_cmd_resource_obj_cmd_hdr}.
+\field{group_member_id} is set to zero.
+\field{id} identifies the existing resource object of type \field{type}.
+
+This command destroys the specified resource object of \field{type} identified
+by \field{id}, which is previously created using
+VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE command.
+
+This command has no command specific result.
+When the command completes successfully, the resource object is destroyed from the device.
+
+\devicenormative{\paragraph}{Device resource objects}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
+
+The device SHOULD complete the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE
+with \field{status} set to VIRTIO_ADMIN_STATUS_EEXIST if a resource object already exists
+with supplied resource \field{id} for the specified \field{type}.
+
+The device SHOULD complete the commands VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY,
+VIRTIO_ADMIN_CMD_RESOURCE_QUERY and
+VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY with \field{status} set to
+VIRTIO_ADMIN_STATUS_ENXIO if the specified resource object does not exist.
+
+The device SHOULD set \field{status} to VIRTIO_ADMIN_STATUS_ENOSPC for the
+command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE if the device fail to create the
+resource object.
+
+The device SHOULD complete the commands VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY or
+VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY commands with \field{status} set to
+VIRTIO_ADMIN_STATUS_EBUSY if other resource objects depend on the resource object
+being modified or destroyed.
+
+The device MUST allow recreating the resource object using the command
+VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE which was previously
+destroyed using the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY respectively
+without undergoing a device reset.
+
+The device SHOULD allow creating the resource object using
+the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE with any resource
+id as long as the resource object is not created.
+
+The device MAY fail the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE even if the
+resources within the device have not reached up to the \field{max_limit}
+but the device MAY have reached an internal limit.
+
+When a capability represents a number of resource objects, the device SHOULD
+allow creating as many resource objects as represented by the driver capability.
+
+The device MUST NOT have any side effects on the resource object when the command
+VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY fails.
+
+The device MUST complete the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY
+with \field{resource_obj_specific_data} which is matching the
+\field{resource_obj_specific_data} of last VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE
+or VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY command.
+
+On device reset, the device MUST destroy all the resource objects which
+have been created.
+
+\drivernormative{\paragraph}{Device resource objects}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
+
+The driver MUST not create a second resource object of the same type with same
+ID using command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE before destroying the
+previously created resource object.
+
+The driver MUST NOT create more resource objects of a specified \field{type} using
+command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE than the maximum limit set by the
+driver capability.
+
+The driver SHOULD NOT modify, query and destroy the resource object which is
+already destroyed previously by the driver.
+
+The driver SHOULD NOT destroy the resource object on which other resource objects
+are depending; the driver SHOULD destroy all the resource objects which do not depend
+on other resource objects.
+
+The driver MUST NOT set the capability related to the resource objects if the
+resource objects have been created using the command VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE
+and not yet destroyed.
+
+The driver MUST send the command VIRTIO_ADMIN_CMD_DRIVER_CAP_SET before using
+any resources related to such capability.
diff --git a/admin.tex b/admin.tex
index d8a19c4..5b908ae 100644
--- a/admin.tex
+++ b/admin.tex
@@ -142,7 +142,15 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
 \hline
 0x0009 & \hyperref[par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities / VIRTIO_ADMIN_CMD_DRIVER_CAP_SET]{VIRTIO_ADMIN_CMD_DRIVER_CAP_SET} & Set the driver capabilities \\
 \hline
-0x000a - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd}    \\
+0x000a & \hyperref[par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE]{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE} & Create a device resource object \\
+\hline
+0x000c & \hyperref[par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY]{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_MODIFY} & Modify a device resource object \\
+\hline
+0x000b & \hyperref[par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY]{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_QUERY} & Query a device resource object \\
+\hline
+0x000d & \hyperref[par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects / VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY]{VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY} & Destroy a device resource object \\
+\hline
+0x000e - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd}    \\
 \hline
 0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure)    \\
 \hline
@@ -168,7 +176,7 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
 \hline \hline
 00   & VIRTIO_ADMIN_STATUS_OK    & successful completion  \\
 \hline
-06   & VIRTIO_ADMIN_STATUS_ENXIO & no such capability \\
+06   & VIRTIO_ADMIN_STATUS_ENXIO & no such capability or resource\\
 \hline
 11   & VIRTIO_ADMIN_STATUS_EAGAIN    & try again \\
 \hline
@@ -178,6 +186,8 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
 \hline
 22   & VIRTIO_ADMIN_STATUS_EINVAL    & invalid command \\
 \hline
+28   & VIRTIO_ADMIN_STATUS_ENOSPC    & resources exhausted on device \\
+\hline
 other   & -    & group administration command error  \\
 \hline
 \end{tabular}
diff --git a/conformance.tex b/conformance.tex
index 8b77eba..297b78a 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -100,6 +100,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{drivernormative:General Initialization And Device Operation / Device Cleanup}
 \item \ref{drivernormative:Reserved Feature Bits}
 \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities}
+\item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
 \end{itemize}
 
 \conformance{\subsection}{PCI Driver Conformance}\label{sec:Conformance / Driver Conformance / PCI Driver Conformance}
@@ -183,6 +184,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
 \item \ref{devicenormative:Reserved Feature Bits}
 \item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities}
+\item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
 \end{itemize}
 
 \conformance{\subsection}{PCI Device Conformance}\label{sec:Conformance / Device Conformance / PCI Device Conformance}
-- 
2.34.1