Re: [PATCH] transport-pci: Allow storing the configuration in the BAR

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 6715 bytes --]

On 10/1/26 21:04, alistair23@gmail.com wrote:
> From: Alistair Francis <alistair.francis@wdc.com>
> 
> Most physical PCI endpoint capable PCIe controllers do not allow modifying the
> capability list, and thus are unable to advertise the Common configuration
> capability.
> 
> This results in physical PCIe endpoints implementing virtio-pci falling
> back to the legacy interface and then needing hacks to deal with DMA
> issues [1].
> 
> One solution to this is to allow the device to store the configuration
> data in the BAR [2]. This can be used as a fallback mechanism so we
> don't break any existing setups. New drivers and devices can attempt
> to read the configuration from the capabilities first and then fallback
> to the BAR.
> 
> We also need to add a go_busy field so that the driver and device have a
> basic message box to communicate. This is required for
> device_feature_select for example.
> 
> 1: https://lkml.org/lkml/2026/9/1/185
> 2: https://lkml.org/lkml/2026/9/1/2370
> 
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  transport-pci.tex | 85 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 85 insertions(+)
> 
> diff --git a/transport-pci.tex b/transport-pci.tex
> index 95b08b8..7d09169 100644
> --- a/transport-pci.tex
> +++ b/transport-pci.tex
> @@ -262,6 +262,83 @@ \subsection{Virtio Structure PCI Capabilities}\label{sec:Virtio Transport Option
>  fields provide the most significant 32 bits of a total 64 bit offset and
>  length within the BAR specified by \field{cap.bar}.
>  
> +\subsubsection{Virtio Structure Capabilities in a memory BAR}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Capabilities in a memory BAR}
> +
> +Some devices, for example implementions using physical PCI endpoint
> +controllers, are unable to add capabilities to the existing PCI capability
> +list. This means they are unable to advertise the virtio structure
> +capabilities as described above.
> +
> +In this case a device MAY instead publish the same capability structures in
> +one of its memory BARs, at offset VIRTIO_PCI_CAP_BAR_OFFSET (0), prefixed by
> +the following header:
> +
> +\begin{lstlisting}
> +#define VIRTIO_PCI_CAP_BAR_OFFSET 0
> +#define VIRTIO_PCI_CAP_BAR_MAGIC  0x74726976 /* "virt" */
> +#define VIRTIO_PCI_CAP_BAR_VERSION_ONE  1
> +
> +struct virtio_pci_bar_cap_hdr {
> +        le32 magic;             /* VIRTIO_PCI_CAP_BAR_MAGIC */
> +        u8 hdr_ver;             /* Version of this header */
> +        u8 num_caps;            /* Number of capability entries that follow */
> +        u8 go_busy;             /* Indicates to start an operation and when it's ongoing */
> +        u8 padding[1];
> +};
> +\end{lstlisting}
> +
> +The fields are interpreted as follows:
> +
> +\begin{description}
> +\item[\field{magic}]
> +        Magic value that identifies the presence of capabilities in this BAR.
> +        Must be set to VIRTIO_PCI_CAP_BAR_MAGIC.
> +
> +\item[\field{hdr_ver}]
> +        The version of this header (virtio_pci_bar_cap_hdr), currently
> +        only version VIRTIO_PCI_CAP_BAR_VERSION_ONE is supported.
> +
> +\item[\field{num_caps}]
> +        The number of struct virtio_pci_cap entries that follow this header.
> +
> +\item[\field{go_busy}]
> +        The driver writes VIRTIO_PCI_CAP_BAR_VERSION_ONE to this field to indicate
> +        that the device should take action. The device is considered busy until
> +        the driver reads this as 0.
> +\end{description}
> +
> +The header is followed by \field{num_caps} struct virtio_pci_cap entries
> +(see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / Virtio Structure PCI Capabilities}),
> +the first of which begins immediately after struct
> +virtio_pci_bar_cap_hdr.
> +
> +The entries are packed one immediately after another. Each entry is interpreted
> +the same as if they are in configuration space and locate the
> +corresponding structure within a memory BAR.
> +
> +Note that the \field{cap_vndr} and \field{cap_next} fields are not used in
> +this case.
> +
> +For read-write communication with the device, for example \field{device_feature_select},
> +the driver should set the value of the field then write the header version to
> +\field{go_busy}. This indicates to the device that an action is required.
> +
> +The device will then read the fields, take action according to the sections above
> +and when complete clear the \field{go_busy}. The driver MUST NOT read or write
> +other fields until \field{go_busy} has been cleared by the device.
> +
> +Note the driver can write to multiple fields before triggering \field{go_busy}. The
> +device MUST then take all actions before clearing \field{go_busy}.
> +
> +A device that publishes capabilities in a memory BAR MUST place struct
> +virtio_pci_bar_cap_hdr at offset VIRTIO_PCI_CAP_BAR_OFFSET within a memory
> +BAR.
> +
> +A device MUST follow struct virtio_pci_bar_cap_hdr with \field{num_caps} struct virtio_pci_cap entries.
> +
> +The device MUST provide the same capabilities it usually would, just in the memory
> +BAR.
> +
>  \drivernormative{\subsubsection}{Virtio Structure PCI Capabilities}{Virtio Transport Options / Virtio Over PCI Bus / Virtio Structure PCI Capabilities}
>  
>  The driver MUST ignore any vendor-specific capability structure which has
> @@ -270,6 +347,14 @@ \subsection{Virtio Structure PCI Capabilities}\label{sec:Virtio Transport Option
>  The driver SHOULD use the first instance of each virtio structure type they can
>  support.
>  
> +The driver SHOULD first attempt to use the virtio structure capabilities in
> +the PCI configuration space. If no common configuration capability is
> +present, the driver MAY scan each memory BAR at offset
> +VIRTIO_PCI_CAP_BAR_OFFSET for a struct virtio_pci_bar_cap_hdr whose
> +\field{magic} equals VIRTIO_PCI_CAP_BAR_MAGIC. If found, use the
> +capability entries published there as described in
> +\ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Capabilities in a memory BAR}.
> +
>  The driver MUST accept a \field{cap_len} value which is larger than specified here.
>  
>  The driver MUST ignore any vendor-specific capability structure which has

How is the driver supposed to know that the memory BAR contains that
structure, and not unrelated data that happens to be correctly formed?

From what I can tell, feature bit 31 is currently not used for any
purpose.  Setting it could be used to indicate that the structure
is present.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]