Linux PCI subsystem development
 help / color / mirror / Atom feed
From: Zhi Wang <zhiw@nvidia.com>
To: Gary Guo <gary@garyguo.net>
Cc: <rust-for-linux@vger.kernel.org>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <dakr@kernel.org>,
	<aliceryhl@google.com>, <bhelgaas@google.com>,
	<kwilczynski@kernel.org>, <ojeda@kernel.org>, <boqun@kernel.org>,
	<bjorn3_gh@protonmail.com>, <lossin@kernel.org>,
	<a.hindborg@kernel.org>, <tmgross@umich.edu>,
	<markus.probst@posteo.de>, <cjia@nvidia.com>, <smitra@nvidia.com>,
	<ankita@nvidia.com>, <aniketa@nvidia.com>, <kwankhede@nvidia.com>,
	<targupta@nvidia.com>, <kjaju@nvidia.com>, <alkumar@nvidia.com>,
	<acourbot@nvidia.com>, <joelagnelf@nvidia.com>,
	<jhubbard@nvidia.com>, <zhiwang@kernel.org>,
	<daniel.almeida@collabora.com>, <tamird@kernel.org>,
	<work@onurozkan.dev>
Subject: Re: [PATCH v7 1/1] rust: pci: add extended capability and SR-IOV support
Date: Wed, 19 Aug 2026 11:35:15 +0300	[thread overview]
Message-ID: <20260819113515.55bc1132@inno-dell> (raw)
In-Reply-To: <DKNU588R9B98.2UEHK3YLII989@garyguo.net>

On Thu, 13 Aug 2026 14:04:00 +0100
"Gary Guo" <gary@garyguo.net> wrote:

Hi Gary:

Thanks for the review. I have already addressed all of them in v8.

Z.

> On Tue Aug 4, 2026 at 5:16 PM BST, Zhi Wang wrote:
> > Rust PCI drivers have no typed interface for locating and accessing
> > PCIe extended capabilities.
> >
> > The SR-IOV extended capability describes VF topology and VF BARs.
> > Expose this information through the Rust PCI abstraction so drivers
> > can use the existing typed configuration-space accessors instead of
> > raw bindings.
> >
> > Define ExtCapability to associate a capability ID with a register
> > layout, and add ConfigSpace::find_ext_capability() to locate and
> > project that layout. Bound the view at the next capability or the
> > end of extended configuration space. Add ExtSriovRegs and a decoded
> > VF BAR iterator that reads and validates all six VF BAR register
> > slots up front, yields decoded BAR addresses and widths in logical
> > order, and keeps the raw configuration-space slot advancement
> > internal. Since PCI_EXT_CAP_NEXT() is a function-like macro, expose
> > it through a Rust helper.
> >
> > Link:
> > https://lore.kernel.org/rust-for-linux/20260730182954.783568-1-zhiw@nvidia.com/
> > Cc: Alexandre Courbot <acourbot@nvidia.com> Cc: Gary Guo
> > <gary@garyguo.net> Signed-off-by: Zhi Wang <zhiw@nvidia.com>
> > ---
> >  rust/helpers/pci.c     |   5 +
> >  rust/kernel/pci.rs     |   8 ++
> >  rust/kernel/pci/cap.rs | 317
> > +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 330
> > insertions(+) create mode 100644 rust/kernel/pci/cap.rs
> >
> > diff --git a/rust/helpers/pci.c b/rust/helpers/pci.c
> > index 4ebf256dff23..b946b14d79e4 100644
> > --- a/rust/helpers/pci.c
> > +++ b/rust/helpers/pci.c
> > @@ -24,6 +24,11 @@ __rust_helper bool rust_helper_dev_is_pci(const
> > struct device *dev) return dev_is_pci(dev);
> >  }
> >  
> > +__rust_helper u32 rust_helper_pci_ext_cap_next(u32 header)
> > +{
> > +	return PCI_EXT_CAP_NEXT(header);
> > +}
> > +
> >  #ifndef CONFIG_PCI_IOV
> >  __rust_helper unsigned int
> >  rust_helper_pci_sriov_get_totalvfs(struct pci_dev *pdev)
> > diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
> > index 9f19ccd5905c..008c2770a3f3 100644
> > --- a/rust/kernel/pci.rs
> > +++ b/rust/kernel/pci.rs
> > @@ -32,10 +32,18 @@
> >      },
> >  };
> >  
> > +mod cap;
> >  mod id;
> >  mod io;
> >  mod irq;
> >  
> > +pub use self::cap::{
> > +    ExtCapId,
> > +    ExtCapability,
> > +    ExtSriovCapability,
> > +    ExtSriovRegs,
> > +    ExtSriovVfBar, //
> > +};
> >  pub use self::id::{
> >      Class,
> >      ClassMask,
> > diff --git a/rust/kernel/pci/cap.rs b/rust/kernel/pci/cap.rs
> > new file mode 100644
> > index 000000000000..c49de8682f6d
> > --- /dev/null
> > +++ b/rust/kernel/pci/cap.rs
> > @@ -0,0 +1,317 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! PCI extended capability support.
> > +
> > +use super::{
> > +    io::ConfigSpaceBackend,
> > +    ConfigSpace,
> > +    Extended, //
> > +};
> > +use crate::{
> > +    bindings,
> > +    io::{
> > +        Io,
> > +        IoBackend,
> > +        Region, //
> > +    },
> > +    num::Bounded,
> > +    prelude::*,
> > +};
> > +
> > +/// Number of VF BAR register slots in an SR-IOV capability.
> > +// CAST: `PCI_SRIOV_NUM_BARS` is 6, which fits in `usize`.
> > +const NUM_VF_BARS: usize = bindings::PCI_SRIOV_NUM_BARS as usize;
> > +
> > +/// PCI extended capability IDs.
> > +#[repr(u16)]
> > +#[derive(Debug, Clone, Copy, PartialEq, Eq)]
> > +pub enum ExtCapId {
> > +    /// Single Root I/O Virtualization.
> > +    // CAST: `PCI_EXT_CAP_ID_SRIOV` is `0x10`, which fits in `u16`.
> > +    Sriov = bindings::PCI_EXT_CAP_ID_SRIOV as u16,
> > +}
> 
> This might also make sense as a `struct` with assoc constants, so it
> can accept unknown values.
> 
>     #[repr(transparent)]
>     #[derive(...)]
>     pub struct ExtCapId(u16);
> 
>     impl ExtCapId {
>         pub const SRIOV: Self = Self(bindings::PCI_EXT_CAP_ID_SRIOV
> as u16); }
> 
> > +
> > +impl ExtCapId {
> > +    #[inline]
> > +    fn as_raw(self) -> u16 {
> > +        self as u16
> > +    }
> > +}
> > +
> > +/// A typed PCI extended capability register layout.
> > +///
> > +/// Implementors describe the register layout of one extended
> > capability. The layout must start at +/// the extended capability
> > header, and [`Self::ID`] must identify that layout. +pub trait
> > ExtCapability: FromBytes + IntoBytes {
> > +    /// PCI extended capability ID for this register layout.
> > +    const ID: ExtCapId;
> > +}
> > +
> > +impl<'a> ConfigSpace<'a, Extended> {
> > +    /// Finds and projects an extended capability into its typed
> > register layout.
> > +    ///
> > +    /// Returns [`None`] if the device does not implement the
> > capability.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```no_run
> > +    /// use kernel::pci;
> > +    ///
> > +    /// fn probe_sriov(
> > +    ///     pdev: &pci::Device<kernel::device::Bound>,
> > +    /// ) -> Result<(), kernel::error::Error> {
> > +    ///     let Some(sriov) = pdev
> > +    ///         .config_space_extended()?
> > +    ///         .find_ext_capability::<pci::ExtSriovRegs>()?
> > +    ///     else {
> > +    ///         return Ok(());
> > +    ///     };
> > +    ///
> > +    ///     let total_vfs = kernel::io_read!(sriov, .total_vfs);
> > +    ///     let vf_offset = kernel::io_read!(sriov, .vf_offset);
> > +    ///     let mut vf_bars = sriov.vf_bars()?;
> > +    ///     let bar0 =
> > vf_bars.next().ok_or(kernel::error::code::EINVAL)?;
> 
> Error codes are in the prelude.
> 
> > +    ///     let bar1 =
> > vf_bars.next().ok_or(kernel::error::code::EINVAL)?;
> > +    ///     let bar2 =
> > vf_bars.next().ok_or(kernel::error::code::EINVAL)?;
> > +    ///
> > +    ///     Ok(())
> > +    /// }
> > +    /// ```
> > +    pub fn find_ext_capability<C: ExtCapability>(&self) ->
> > Result<Option<ConfigSpace<'a, C>>> {
> > +        let offset = usize::from(
> > +            // SAFETY: `self.pdev` is valid by the type invariant
> > of `ConfigSpace`.
> > +            unsafe {
> > +
> > bindings::pci_find_ext_capability(self.pdev.as_raw(),
> > i32::from(C::ID.as_raw()))
> > +            },
> > +        );
> > +
> > +        if offset == 0 {
> > +            return Ok(None);
> > +        }
> > +
> > +        let size = self.calculate_ext_cap_size(offset)?;
> > +
> > +        let base = ConfigSpaceBackend::as_ptr(*self)
> > +            .cast::<u8>()
> > +            .wrapping_add(offset);
> > +        let ptr = Region::<0>::ptr_try_from_raw_parts_mut(base,
> > size)?; +
> > +        // SAFETY: `offset` was returned by
> > `pci_find_ext_capability`, and
> > +        // `calculate_ext_cap_size` bounds `ptr` at the next
> > capability or the end of the extended
> > +        // configuration space. `ptr_try_from_raw_parts_mut`
> > verified the region layout.
> > +        let capability = unsafe {
> > ConfigSpaceBackend::project_view(*self, ptr) }; +
> > +        capability.try_cast::<C>().map(Some)
> 
> Is this cast failure is supposed to be error condition?
> 
> > +    }
> > +
> > +    /// Calculates the size of the extended capability at `offset`.
> > +    ///
> > +    /// The capability extends to the next extended capability, or
> > to the end of the extended
> > +    /// configuration space if it is the last one. `offset` must
> > be a DWORD-aligned offset within
> > +    /// the extended configuration space returned by
> > `pci_find_ext_capability`. Returns an error if
> > +    /// the capability header is outside the extended
> > configuration space.
> > +    fn calculate_ext_cap_size(&self, offset: usize) ->
> > Result<usize> {
> > +        let header = self.try_read32(offset)?;
> > +        // SAFETY: Pure bit manipulation, no preconditions.
> > +        // CAST: The next-cap pointer is a 12-bit field (max
> > 0xFFC), always fits in `usize`.
> > +        let next = unsafe { bindings::pci_ext_cap_next(header) }
> > as usize; +
> > +        Ok(if next > offset {
> > +            next - offset
> > +        } else {
> > +            self.size() - offset
> > +        })
> > +    }
> > +}
> > +
> > +/// SR-IOV register layout per PCIe spec (64 bytes starting at cap
> > offset). +#[repr(C)]
> > +#[derive(FromBytes, IntoBytes)]
> > +pub struct ExtSriovRegs {
> > +    /// Extended capability header.
> > +    pub header: u32,
> 
> I think this shouldn't be public.
> 
> > +    /// SR-IOV capabilities.
> > +    pub cap: u32,
> > +    /// SR-IOV control.
> > +    pub ctrl: u16,
> > +    /// SR-IOV status.
> > +    pub status: u16,
> > +    /// Initial VFs.
> > +    pub initial_vfs: u16,
> > +    /// Total VFs.
> > +    pub total_vfs: u16,
> > +    /// Number of VFs.
> > +    pub num_vfs: u16,
> > +    /// Function dependency link.
> > +    pub func_dep_link: u8,
> > +    _reserved_0: u8,
> > +    /// First VF offset.
> > +    pub vf_offset: u16,
> > +    /// VF stride.
> > +    pub vf_stride: u16,
> > +    _reserved_1: u16,
> > +    /// VF device ID.
> > +    pub vf_device_id: u16,
> > +    /// Supported page sizes.
> > +    pub supported_page_sizes: u32,
> > +    /// System page size.
> > +    pub system_page_size: u32,
> > +    /// VF BARs (BAR0–BAR5).
> > +    pub vf_bar: [u32; NUM_VF_BARS],
> > +    /// VF migration state array offset.
> > +    pub migration_state: u32,
> > +}
> > +
> > +impl ExtCapability for ExtSriovRegs {
> > +    const ID: ExtCapId = ExtCapId::Sriov;
> > +}
> > +
> > +/// A typed view of an SR-IOV extended capability.
> > +pub type ExtSriovCapability<'a> = ConfigSpace<'a, ExtSriovRegs>;
> > +
> > +#[derive(Debug, Clone, Copy, PartialEq, Eq)]
> > +enum VfBarMemoryType {
> > +    Bits32,
> > +    Bits64,
> > +}
> > +
> > +impl TryFrom<Bounded<u32, 2>> for VfBarMemoryType {
> > +    type Error = Error;
> > +
> > +    fn try_from(value: Bounded<u32, 2>) -> Result<Self> {
> > +        match value.get() {
> > +            0b00 => Ok(Self::Bits32),
> > +            0b10 => Ok(Self::Bits64),
> > +            _ => Err(EINVAL),
> > +        }
> > +    }
> > +}
> > +
> > +impl From<VfBarMemoryType> for Bounded<u32, 2> {
> > +    fn from(value: VfBarMemoryType) -> Self {
> > +        match value {
> > +            VfBarMemoryType::Bits32 => Self::new::<0b00>(),
> > +            VfBarMemoryType::Bits64 => Self::new::<0b10>(),
> > +        }
> > +    }
> > +}
> > +
> > +crate::bitfield! {
> > +    /// Low DWORD of an SR-IOV VF BAR.
> > +    struct VfBarLow(u32) {
> > +        /// Base address bits 31:4.
> > +        31:4 address;
> > +        /// Whether the address range is prefetchable.
> > +        3:3 prefetchable => bool;
> > +        /// Memory BAR type.
> > +        2:1 memory_type ?=> VfBarMemoryType;
> > +        /// Whether this is an I/O-space BAR.
> > +        0:0 io_space => bool;
> > +    }
> > +}
> > +
> > +/// A decoded VF BAR register encoding.
> > +#[derive(Debug, Clone, Copy, PartialEq, Eq)]
> > +pub struct ExtSriovVfBar {
> > +    /// The BAR address without PCI attribute bits.
> > +    pub address: u64,
> > +
> > +    /// Whether the BAR is 64-bit.
> > +    pub is_64bit: bool,
> > +}
> > +
> > +/// Iterator over decoded VF BAR register encodings.
> > +///
> > +/// `slots` contains the six consecutive 32-bit registers VF BAR0
> > through VF BAR5. A 32-bit +/// memory BAR encoding uses one
> > register. A 64-bit memory BAR encoding uses that register for +///
> > bits 31:0 and the immediately following register for bits 63:32.
> > +/// +/// # Invariants
> > +///
> > +/// - `config_slot <= NUM_VF_BARS`.
> > +/// - If `config_slot < NUM_VF_BARS`, it identifies the next
> > register to interpret as a BAR low +///   DWORD. Its address-space
> > encoding is memory and its type encoding is either 32-bit or
> > 64-bit. +/// - If that low DWORD encodes a 64-bit BAR, `config_slot
> > + 1 < NUM_VF_BARS`, and the register at +///   `config_slot + 1` is
> > its upper DWORD. +struct ExtSriovVfBars {
> > +    slots: [u32; NUM_VF_BARS],
> > +    config_slot: usize,
> > +}
> > +
> > +impl ExtSriovVfBars {
> > +    fn new(slots: [u32; NUM_VF_BARS]) -> Result<Self> {
> > +        let mut config_slot = 0;
> > +
> > +        while config_slot < NUM_VF_BARS {
> > +            let low = VfBarLow::from(slots[config_slot]);
> > +
> > +            if low.io_space() {
> > +                return Err(EINVAL);
> > +            }
> > +
> > +            let is_64bit = low.memory_type()? ==
> > VfBarMemoryType::Bits64; +
> > +            if is_64bit {
> > +                if config_slot + 1 >= NUM_VF_BARS {
> > +                    return Err(EINVAL);
> > +                }
> > +
> > +                config_slot += 2;
> > +            } else {
> > +                config_slot += 1;
> > +            }
> 
> So you're already decoding the vf bars here, but the `next` does an
> additional decoding. I think you should just decode them in one place.
> 
> Best,
> Gary
> 
> > +        }
> > +
> > +        Ok(Self {
> > +            slots,
> > +            config_slot: 0,
> > +        })
> > +    }
> > +}
> > +
> > +impl Iterator for ExtSriovVfBars {
> > +    type Item = ExtSriovVfBar;
> > +
> > +    fn next(&mut self) -> Option<Self::Item> {
> > +        if self.config_slot >= NUM_VF_BARS {
> > +            return None;
> > +        }
> > +
> > +        let config_slot = self.config_slot;
> > +        let low = VfBarLow::from(self.slots[config_slot]);
> > +        let is_64bit = matches!(low.memory_type(),
> > Ok(VfBarMemoryType::Bits64));
> > +        let low_address = u64::from(low.address()) <<
> > VfBarLow::ADDRESS_SHIFT; +
> > +        let address = if is_64bit {
> > +            let high = self.slots[config_slot + 1];
> > +            self.config_slot += 2;
> > +            (u64::from(high) << 32) | low_address
> > +        } else {
> > +            self.config_slot += 1;
> > +            low_address
> > +        };
> > +
> > +        Some(ExtSriovVfBar { address, is_64bit })
> > +    }
> > +}
> > +
> > +impl ExtSriovCapability<'_> {
> > +    /// Returns an iterator over decoded VF BAR register encodings.
> > +    ///
> > +    /// The iterator tracks the six raw VF BAR register slots
> > internally. A 32-bit encoding yields
> > +    /// one entry and advances by one slot; a 64-bit encoding
> > combines two slots into one entry.
> > +    ///
> > +    /// A zero-valued low DWORD is yielded as a 32-bit BAR at
> > address zero; this method does not
> > +    /// probe whether a BAR is implemented.
> > +    ///
> > +    /// Returns [`EINVAL`] and logs an error if a BAR low DWORD
> > does not encode a 32-bit or 64-bit
> > +    /// memory BAR, or if a 64-bit encoding has no upper DWORD.
> > +    pub fn vf_bars(&self) -> Result<impl Iterator<Item =
> > ExtSriovVfBar>> {
> > +        let slots: [u32; NUM_VF_BARS] =
> > +            core::array::from_fn(|slot| crate::io_read!(*self,
> > .vf_bar[panic: slot])); +
> > +        ExtSriovVfBars::new(slots).inspect_err(|_| {
> > +            dev_err!(self.pdev, "invalid VF BAR encoding in SR-IOV
> > capability\n");
> > +        })
> > +    }
> > +}
> 
> 


  reply	other threads:[~2026-08-19  8:35 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04 16:16 [PATCH v7 0/1] Rust PCI capability infrastructure and SR-IOV support Zhi Wang
2026-08-04 16:16 ` [PATCH v7 1/1] rust: pci: add extended capability " Zhi Wang
2026-08-04 16:33   ` sashiko-bot
2026-08-13 13:04   ` Gary Guo
2026-08-19  8:35     ` Zhi Wang [this message]
2026-08-13 13:56   ` Danilo Krummrich
2026-08-19  8:36     ` Zhi Wang
2026-08-13  6:54 ` [PATCH v7 0/1] Rust PCI capability infrastructure " Zhi Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260819113515.55bc1132@inno-dell \
    --to=zhiw@nvidia.com \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=aliceryhl@google.com \
    --cc=alkumar@nvidia.com \
    --cc=aniketa@nvidia.com \
    --cc=ankita@nvidia.com \
    --cc=bhelgaas@google.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=cjia@nvidia.com \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=gary@garyguo.net \
    --cc=jhubbard@nvidia.com \
    --cc=joelagnelf@nvidia.com \
    --cc=kjaju@nvidia.com \
    --cc=kwankhede@nvidia.com \
    --cc=kwilczynski@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lossin@kernel.org \
    --cc=markus.probst@posteo.de \
    --cc=ojeda@kernel.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=smitra@nvidia.com \
    --cc=tamird@kernel.org \
    --cc=targupta@nvidia.com \
    --cc=tmgross@umich.edu \
    --cc=work@onurozkan.dev \
    --cc=zhiwang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox