Re: [PATCH] transport-pci: Allow storing the configuration in the BAR

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 8154 bytes --]

On 10/22/26 00:27, Alistair wrote:
> On Fri, 2026-10-02 at 01:48 -0400, Demi Marie Obenour wrote:
>> On 10/1/26 21:04, alistair23@gmail.com wrote:
>>> From: Alistair Francis <alistair.francis@wdc.com>
>>>
>>> Most physical PCI endpoint capable PCIe controllers do not allow
>>> modifying the
>>> capability list, and thus are unable to advertise the Common
>>> configuration
>>> capability.
>>>
>>> This results in physical PCIe endpoints implementing virtio-pci
>>> falling
>>> back to the legacy interface and then needing hacks to deal with
>>> DMA
>>> issues [1].
>>>
>>> One solution to this is to allow the device to store the
>>> configuration
>>> data in the BAR [2]. This can be used as a fallback mechanism so we
>>> don't break any existing setups. New drivers and devices can
>>> attempt
>>> to read the configuration from the capabilities first and then
>>> fallback
>>> to the BAR.
>>>
>>> We also need to add a go_busy field so that the driver and device
>>> have a
>>> basic message box to communicate. This is required for
>>> device_feature_select for example.
>>>
>>> 1: https://lkml.org/lkml/2026/9/1/185
>>> 2: https://lkml.org/lkml/2026/9/1/2370
>>>
>>> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
>>> ---
>>>  transport-pci.tex | 85
>>> +++++++++++++++++++++++++++++++++++++++++++++++
>>>  1 file changed, 85 insertions(+)
>>>
>>> diff --git a/transport-pci.tex b/transport-pci.tex
>>> index 95b08b8..7d09169 100644
>>> --- a/transport-pci.tex
>>> +++ b/transport-pci.tex
>>> @@ -262,6 +262,83 @@ \subsection{Virtio Structure PCI
>>> Capabilities}\label{sec:Virtio Transport Option
>>>  fields provide the most significant 32 bits of a total 64 bit
>>> offset and
>>>  length within the BAR specified by \field{cap.bar}.
>>>  
>>> +\subsubsection{Virtio Structure Capabilities in a memory
>>> BAR}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI
>>> Device Layout / Capabilities in a memory BAR}
>>> +
>>> +Some devices, for example implementions using physical PCI
>>> endpoint
>>> +controllers, are unable to add capabilities to the existing PCI
>>> capability
>>> +list. This means they are unable to advertise the virtio structure
>>> +capabilities as described above.
>>> +
>>> +In this case a device MAY instead publish the same capability
>>> structures in
>>> +one of its memory BARs, at offset VIRTIO_PCI_CAP_BAR_OFFSET (0),
>>> prefixed by
>>> +the following header:
>>> +
>>> +\begin{lstlisting}
>>> +#define VIRTIO_PCI_CAP_BAR_OFFSET 0
>>> +#define VIRTIO_PCI_CAP_BAR_MAGIC  0x74726976 /* "virt" */
>>> +#define VIRTIO_PCI_CAP_BAR_VERSION_ONE  1
>>> +
>>> +struct virtio_pci_bar_cap_hdr {
>>> +        le32 magic;             /* VIRTIO_PCI_CAP_BAR_MAGIC */
>>> +        u8 hdr_ver;             /* Version of this header */
>>> +        u8 num_caps;            /* Number of capability entries
>>> that follow */
>>> +        u8 go_busy;             /* Indicates to start an operation
>>> and when it's ongoing */
>>> +        u8 padding[1];
>>> +};
>>> +\end{lstlisting}
>>> +
>>> +The fields are interpreted as follows:
>>> +
>>> +\begin{description}
>>> +\item[\field{magic}]
>>> +        Magic value that identifies the presence of capabilities
>>> in this BAR.
>>> +        Must be set to VIRTIO_PCI_CAP_BAR_MAGIC.
>>> +
>>> +\item[\field{hdr_ver}]
>>> +        The version of this header (virtio_pci_bar_cap_hdr),
>>> currently
>>> +        only version VIRTIO_PCI_CAP_BAR_VERSION_ONE is supported.
>>> +
>>> +\item[\field{num_caps}]
>>> +        The number of struct virtio_pci_cap entries that follow
>>> this header.
>>> +
>>> +\item[\field{go_busy}]
>>> +        The driver writes VIRTIO_PCI_CAP_BAR_VERSION_ONE to this
>>> field to indicate
>>> +        that the device should take action. The device is
>>> considered busy until
>>> +        the driver reads this as 0.
>>> +\end{description}
>>> +
>>> +The header is followed by \field{num_caps} struct virtio_pci_cap
>>> entries
>>> +(see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus /
>>> Virtio Structure PCI Capabilities}),
>>> +the first of which begins immediately after struct
>>> +virtio_pci_bar_cap_hdr.
>>> +
>>> +The entries are packed one immediately after another. Each entry
>>> is interpreted
>>> +the same as if they are in configuration space and locate the
>>> +corresponding structure within a memory BAR.
>>> +
>>> +Note that the \field{cap_vndr} and \field{cap_next} fields are not
>>> used in
>>> +this case.
>>> +
>>> +For read-write communication with the device, for example
>>> \field{device_feature_select},
>>> +the driver should set the value of the field then write the header
>>> version to
>>> +\field{go_busy}. This indicates to the device that an action is
>>> required.
>>> +
>>> +The device will then read the fields, take action according to the
>>> sections above
>>> +and when complete clear the \field{go_busy}. The driver MUST NOT
>>> read or write
>>> +other fields until \field{go_busy} has been cleared by the device.
>>> +
>>> +Note the driver can write to multiple fields before triggering
>>> \field{go_busy}. The
>>> +device MUST then take all actions before clearing \field{go_busy}.
>>> +
>>> +A device that publishes capabilities in a memory BAR MUST place
>>> struct
>>> +virtio_pci_bar_cap_hdr at offset VIRTIO_PCI_CAP_BAR_OFFSET within
>>> a memory
>>> +BAR.
>>> +
>>> +A device MUST follow struct virtio_pci_bar_cap_hdr with
>>> \field{num_caps} struct virtio_pci_cap entries.
>>> +
>>> +The device MUST provide the same capabilities it usually would,
>>> just in the memory
>>> +BAR.
>>> +
>>>  \drivernormative{\subsubsection}{Virtio Structure PCI
>>> Capabilities}{Virtio Transport Options / Virtio Over PCI Bus /
>>> Virtio Structure PCI Capabilities}
>>>  
>>>  The driver MUST ignore any vendor-specific capability structure
>>> which has
>>> @@ -270,6 +347,14 @@ \subsection{Virtio Structure PCI
>>> Capabilities}\label{sec:Virtio Transport Option
>>>  The driver SHOULD use the first instance of each virtio structure
>>> type they can
>>>  support.
>>>  
>>> +The driver SHOULD first attempt to use the virtio structure
>>> capabilities in
>>> +the PCI configuration space. If no common configuration capability
>>> is
>>> +present, the driver MAY scan each memory BAR at offset
>>> +VIRTIO_PCI_CAP_BAR_OFFSET for a struct virtio_pci_bar_cap_hdr
>>> whose
>>> +\field{magic} equals VIRTIO_PCI_CAP_BAR_MAGIC. If found, use the
>>> +capability entries published there as described in
>>> +\ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI
>>> Device Layout / Capabilities in a memory BAR}.
>>> +
>>>  The driver MUST accept a \field{cap_len} value which is larger
>>> than specified here.
>>>  
>>>  The driver MUST ignore any vendor-specific capability structure
>>> which has
>>
>> How is the driver supposed to know that the memory BAR contains that
>> structure, and not unrelated data that happens to be correctly
>> formed?
> 
> Sorry for the delay, I lost this email while I was travelling.
> 
> I don't have an answer here. The driver is supposed to search for
> VIRTIO_PCI_CAP_BAR_MAGIC at offset VIRTIO_PCI_CAP_BAR_OFFSET, which
> could exist independently.
> 
>>
>> From what I can tell, feature bit 31 is currently not used for any
>> purpose.  Setting it could be used to indicate that the structure
>> is present.
> 
> Do you mean have the driver probe for feature bit 31 (after searching
> for VIRTIO_PCI_CAP_BAR_MAGIC at offset VIRTIO_PCI_CAP_BAR_OFFSET) to
> confirm that it is supported?
> 
> Alistair

I meant having a feature bit that is compatible with the legacy
interface, but which tells the driver that the device is really
a modern driver with its common configuration capability in a
specified BAR.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]