[virtio-dev] [PATCH 06/11] introduction: Introduce transitional MMR interface

Parav Pandit
2023-03-30T23:00:41Z

Introduce terminology for the transitional MMR device and transitional
MMR driver.

Add description of the transitional MMR device. It is a PCI
device that implements legacy virtio common configuration registers
followed by legacy device specific registers in a memory region at
an offset.

This enables hypervisor such as vfio driver to emulate
I/O region towards the guest at BAR0. By doing so VFIO driver can
translate read/write accesses on I/O region from the guest
to the device memory region.

High level comparison of 1.x, transitional & transitional MMR
sriov vf device:

+------------------+ +--------------------+ +--------------------+
|virtio 1.x | |Transitional | |Transitional |
|SRIOV VF | |SRIOV VF | |MMR SRIOV VF |
| | | | | |
++---------------+ | ++---------------+ | ++---------------+ |
||dev_id = | | ||dev_id = | | ||dev_id = | |
||{0x1040-0x106C}| | ||{0x1000-0x103f}| | ||{0x10f9-0x10ff}| |
|+---------------+ | |+---------------+ | |+---------------+ |
| | | | | |
|+------------+ | |+------------+ | |+-----------------+ |
||Memory BAR | | ||Memory BAR | | ||Memory BAR | |
|+------------+ | |+------------+ | || | |
| | | | || +--------------+| |
| | |+-----------------+ | || |legacy virtio || |
| | ||IOBAR impossible | | || |+ dev cfg || |
| | |+-----------------+ | || |registers || |
| | | | || +--------------+| |
| | | | |+-----------------+ |
+------------------+ +--------------------+ +--------------------+

Motivation and background:
PCIe and system limitations:
1. PCIe VFs do not support IOBAR cited at [1].

Perhaps the PCIe spec could be extended, however it would be only
useful for virtio transitional devices. Even if such an extension
is present, there are other system limitations described below in (2)
and (3).

2. cpu io port space limit and fragmentation
x86_64 is limited to only 64K worth of IO port space at [2],
which is shared with many other onboard system peripherals which
are behind PCIe bridge; such I/O region also needs to be aligned
to 4KB at PCIe bridge level cited at [3]. This can lead to a I/O space
fragmentation. Due to this fragmentation and alignment need,
actual usable range is small.

3. IO space access of PCI device is done through non-posted message
which requires higher completion time in the PCIe fabric for
round trip travel.

[1] PCIe spec citation:
VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.

[2] cpu arch citiation:
Intel 64 and IA-32 Architectures Software Developer’s Manual
The processor’s I/O address space is separate and distinct from
the physical-memory address space. The I/O address space consists
of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.

[3] PCIe spec citation:
If a bridge implements an I/O address range,...I/O address range
will be aligned to a 4 KB boundary.

Co-developed-by: Satananda Burla <sburla@marvell.com>
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
introduction.tex | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

diff --git a/introduction.tex b/introduction.tex
index e8b34e3..9a0f96a 100644
--- a/introduction.tex
+++ b/introduction.tex
@@ -161,6 +161,20 @@ \subsection{Legacy Interface: Terminology}\label{intro:Legacy
have a need for backwards compatibility!
\end{note}

+\begin{description}
+\item[Transitional MMR Device]
+ is a PCI device which exposes legacy virtio configuration
+ registers followed by legacy device configuration registers as
+ memory mapped registers (MMR) at an offset in a memory region
+ BAR, has no I/O region BAR, having its own PCI Device ID range,
+ and follows the rest of the functionalities of the transitional device.
+\end{description}
+
+\begin{description}
+\item[Transitional MMR Driver]
+ is a PCI device driver that supports the Transitional MMR device.
+\end{description}
+
Devices or drivers with no legacy compatibility are referred to as
non-transitional devices and drivers, respectively.

@@ -174,6 +188,22 @@ \subsection{Transition from earlier specification drafts}\label{sec:Transition f
sections tagged "Legacy Interface" in the section title.
These highlight the changes made since the earlier drafts.

+\subsection{Transitional MMR interface: specification drafts}\label{sec:Transitional MMR interface: specification drafts}
+
+The transitional MMR device and driver differs from the
+transitional device and driver respectively in few areas. Such
+differences are contained in sections named
+'Transitional MMR interface', like this one. When no differences
+are mentioned explicitly, the transitional MMR device and driver
+follow exactly the same functionalities as that of the
+transitional device and driver respectively.
+
+\begin{note}
+Transitional MMR interface is only required to support backward
+compatibility. It should not be implemented unless there is a need
+for the backward compatibility.
+\end{note}
+
\section{Structure Specifications}\label{sec:Structure Specifications}

Many device and driver in-memory structure layouts are documented using
--
2.26.2