Re: [PATCH 07/11] PCI: Add PCIe Device 3 Extended Capability enumeration

["Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 4923 bytes --]

On Thu, 5 Dec 2024, Dan Williams wrote:

> PCIe 6.2 Section 7.7.9 Device 3 Extended Capability Structure,
> enumerates new link capabilities and status added for Gen 6 devices. One
> of the link details enumerated in that register block is the "Segment
> Captured" status in the Device Status 3 register. That status is
> relevant for enabling IDE (Integrity & Data Encryption) whereby
> Selective IDE streams can be limited to a given requester id range
> within a given segment.
> 
> If a device has captured its Segment value then it knows that PCIe Flit
> Mode is enabled via all links in the path that a configuration write
> traversed. IDE establishment requires that "Segment Base" in
> IDE RID Association Register 2 (PCIe 6.2 Section 7.9.26.5.4.2) be
> programmed if the RID association mechanism is in effect.
> 
> When / if IDE + Flit Mode capable devices arrive, the PCI core needs to
> setup the segment base when using the RID association facility, but no
> known deployments today depend on this.
> 
> Cc: Lukas Wunner <lukas@wunner.de>
> Cc: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Samuel Ortiz <sameo@rivosinc.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
>  drivers/pci/pci.h             |   11 +++++++++++
>  drivers/pci/probe.c           |    1 +
>  include/linux/pci.h           |    1 +
>  include/uapi/linux/pci_regs.h |    7 +++++++
>  4 files changed, 20 insertions(+)
> 
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 0537fc72d5be..6565eb72ded2 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -444,6 +444,17 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { }
>  static inline void pci_doe_disconnected(struct pci_dev *pdev) { }
>  #endif
>  
> +static inline void pci_dev3_init(struct pci_dev *pdev)
> +{
> +	u16 cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DEV3);
> +	u32 val = 0;
> +
> +	if (!cap)
> +		return;
> +	pci_read_config_dword(pdev, cap + PCI_DEV3_STA, &val);
> +	pdev->fm_enabled = !!(val & PCI_DEV3_STA_SEGMENT);
> +}
> +
>  #ifdef CONFIG_PCI_NPEM
>  void pci_npem_create(struct pci_dev *dev);
>  void pci_npem_remove(struct pci_dev *dev);
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 7cddde3cb0ed..6c1fe6354d26 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2517,6 +2517,7 @@ static void pci_init_capabilities(struct pci_dev *dev)
>  	pci_rcec_init(dev);		/* Root Complex Event Collector */
>  	pci_doe_init(dev);		/* Data Object Exchange */
>  	pci_tph_init(dev);		/* TLP Processing Hints */
> +	pci_dev3_init(dev);		/* Device 3 capabilities */
>  	pci_ide_init(dev);		/* Link Integrity and Data Encryption */
>  	pci_tsm_init(dev);		/* TEE Security Manager connection */
>  
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index a0900e7d2012..10d035395a43 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -443,6 +443,7 @@ struct pci_dev {
>  	unsigned int	pasid_enabled:1;	/* Process Address Space ID */
>  	unsigned int	pri_enabled:1;		/* Page Request Interface */
>  	unsigned int	tph_enabled:1;		/* TLP Processing Hints */
> +	unsigned int	fm_enabled:1;		/* Flit Mode (segment captured) */
>  	unsigned int	is_managed:1;		/* Managed via devres */
>  	unsigned int	is_msi_managed:1;	/* MSI release via devres installed */
>  	unsigned int	needs_freset:1;		/* Requires fundamental reset */
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 19bba65a262c..c61231861b51 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -749,6 +749,7 @@
>  #define PCI_EXT_CAP_ID_NPEM	0x29	/* Native PCIe Enclosure Management */
>  #define PCI_EXT_CAP_ID_PL_32GT  0x2A    /* Physical Layer 32.0 GT/s */
>  #define PCI_EXT_CAP_ID_DOE	0x2E	/* Data Object Exchange */
> +#define PCI_EXT_CAP_ID_DEV3	0x2F	/* Device 3 Capability/Control/Status */
>  #define PCI_EXT_CAP_ID_IDE	0x30    /* Integrity and Data Encryption */
>  #define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_IDE
>  
> @@ -1210,6 +1211,12 @@
>  #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL		0x00ff0000
>  #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX	0xff000000
>  
> +/* Device 3 Extended Capability */
> +#define PCI_DEV3_CAP		0x4	/* Device 3 Capabilities Register */
> +#define PCI_DEV3_CTL		0x8	/* Device 3 Control Register */

Should save/restore too be added for DEV3_CTL?

-- 
 i.

> +#define PCI_DEV3_STA		0xc	/* Device 3 Status Register */
> +#define  PCI_DEV3_STA_SEGMENT	0x8	/* Segment Captured (end-to-end flit-mode detected) */
> +
>  /* Compute Express Link (CXL r3.1, sec 8.1.5) */
>  #define PCI_DVSEC_CXL_PORT				3
>  #define PCI_DVSEC_CXL_PORT_CTL				0x0c
>