* [PATCH 0/2] Save and restore pci properties to support FLR
@ 2017-05-30 16:25 Ashok Raj
2017-05-30 16:25 ` [PATCH 1/2] PCI: Cache PRI and PASID bits in pci_dev Ashok Raj
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Ashok Raj @ 2017-05-30 16:25 UTC (permalink / raw)
To: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci,
linux-kernel
Cc: Ashok Raj, David Woodhouse, iommu
Resending Jean's patch so it can be included earlier than his large
SVM commits. Original patch https://patchwork.kernel.org/patch/9593891
was ack'ed by Bjorn. Let's commit these separately since we need
functionality earlier.
Resending this series as requested by Jean.
CQ Tang (1):
PCI: Save properties required to handle FLR for replay purposes.
Jean-Philippe Brucker (1):
PCI: Cache PRI and PASID bits in pci_dev
drivers/pci/ats.c | 88 ++++++++++++++++++++++++++++++++++++++++---------
drivers/pci/pci.c | 3 ++
include/linux/pci-ats.h | 10 ++++++
include/linux/pci.h | 8 +++++
4 files changed, 94 insertions(+), 15 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/2] PCI: Cache PRI and PASID bits in pci_dev 2017-05-30 16:25 [PATCH 0/2] Save and restore pci properties to support FLR Ashok Raj @ 2017-05-30 16:25 ` Ashok Raj 2017-05-30 16:25 ` [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes Ashok Raj 2017-05-30 20:58 ` [PATCH 0/2] Save and restore pci properties to support FLR Bjorn Helgaas 2 siblings, 0 replies; 7+ messages in thread From: Ashok Raj @ 2017-05-30 16:25 UTC (permalink / raw) To: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/pci/ats.c | 23 +++++++++++++++++++++++ include/linux/pci.h | 2 ++ 2 files changed, 25 insertions(+) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index eeb9fb2..2126497 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -153,6 +153,9 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) u32 max_requests; int pos; + if (WARN_ON(pdev->pri_enabled)) + return -EBUSY; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return -EINVAL; @@ -170,6 +173,8 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) control |= PCI_PRI_CTRL_ENABLE; pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); + pdev->pri_enabled = 1; + return 0; } EXPORT_SYMBOL_GPL(pci_enable_pri); @@ -185,6 +190,9 @@ void pci_disable_pri(struct pci_dev *pdev) u16 control; int pos; + if (WARN_ON(!pdev->pri_enabled)) + return; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return; @@ -192,6 +200,8 @@ void pci_disable_pri(struct pci_dev *pdev) pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); control &= ~PCI_PRI_CTRL_ENABLE; pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); + + pdev->pri_enabled = 0; } EXPORT_SYMBOL_GPL(pci_disable_pri); @@ -207,6 +217,9 @@ int pci_reset_pri(struct pci_dev *pdev) u16 control; int pos; + if (WARN_ON(pdev->pri_enabled)) + return -EBUSY; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); if (!pos) return -EINVAL; @@ -239,6 +252,9 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) u16 control, supported; int pos; + if (WARN_ON(pdev->pasid_enabled)) + return -EBUSY; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return -EINVAL; @@ -259,6 +275,8 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); + pdev->pasid_enabled = 1; + return 0; } EXPORT_SYMBOL_GPL(pci_enable_pasid); @@ -273,11 +291,16 @@ void pci_disable_pasid(struct pci_dev *pdev) u16 control = 0; int pos; + if (WARN_ON(!pdev->pasid_enabled)) + return; + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); if (!pos) return; pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); + + pdev->pasid_enabled = 0; } EXPORT_SYMBOL_GPL(pci_disable_pasid); diff --git a/include/linux/pci.h b/include/linux/pci.h index eb3da1a..bee980e 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -351,6 +351,8 @@ struct pci_dev { unsigned int msix_enabled:1; unsigned int ari_enabled:1; /* ARI forwarding */ unsigned int ats_enabled:1; /* Address Translation Service */ + unsigned int pasid_enabled:1; /* Process Address Space ID */ + unsigned int pri_enabled:1; /* Page Request Interface */ unsigned int is_managed:1; unsigned int needs_freset:1; /* Dev requires fundamental reset */ unsigned int state_saved:1; -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes. 2017-05-30 16:25 [PATCH 0/2] Save and restore pci properties to support FLR Ashok Raj 2017-05-30 16:25 ` [PATCH 1/2] PCI: Cache PRI and PASID bits in pci_dev Ashok Raj @ 2017-05-30 16:25 ` Ashok Raj 2017-05-30 19:50 ` Bjorn Helgaas 2017-05-30 20:58 ` Bjorn Helgaas 2017-05-30 20:58 ` [PATCH 0/2] Save and restore pci properties to support FLR Bjorn Helgaas 2 siblings, 2 replies; 7+ messages in thread From: Ashok Raj @ 2017-05-30 16:25 UTC (permalink / raw) To: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel Cc: CQ Tang, David Woodhouse, iommu, Ashok Raj From: CQ Tang <cq.tang@intel.com> Requires: https://patchwork.kernel.org/patch/9593891 After a FLR, pci-states need to be restored. This patch saves PASID features and PRI reqs cached. To: Bjorn Helgaas <bhelgaas@google.com> To: Joerg Roedel <joro@8bytes.org> To: linux-pci@vger.kernel.org To: linux-kernel@vger.kernel.org Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org> Cc: iommu@lists.linux-foundation.org Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> --- drivers/pci/ats.c | 65 +++++++++++++++++++++++++++++++++++++------------ drivers/pci/pci.c | 3 +++ include/linux/pci-ats.h | 10 ++++++++ include/linux/pci.h | 6 +++++ 4 files changed, 69 insertions(+), 15 deletions(-) diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c index 2126497..a769955 100644 --- a/drivers/pci/ats.c +++ b/drivers/pci/ats.c @@ -160,17 +160,16 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) if (!pos) return -EINVAL; - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status); - if ((control & PCI_PRI_CTRL_ENABLE) || - !(status & PCI_PRI_STATUS_STOPPED)) + if (!(status & PCI_PRI_STATUS_STOPPED)) return -EBUSY; pci_read_config_dword(pdev, pos + PCI_PRI_MAX_REQ, &max_requests); reqs = min(max_requests, reqs); + pdev->pri_reqs_alloc = reqs; pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); - control |= PCI_PRI_CTRL_ENABLE; + control = PCI_PRI_CTRL_ENABLE; pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); pdev->pri_enabled = 1; @@ -206,6 +205,29 @@ void pci_disable_pri(struct pci_dev *pdev) EXPORT_SYMBOL_GPL(pci_disable_pri); /** + * pci_restore_pri_state - Restore PRI + * @pdev: PCI device structure + * + */ +void pci_restore_pri_state(struct pci_dev *pdev) +{ + u16 control = PCI_PRI_CTRL_ENABLE; + u32 reqs = pdev->pri_reqs_alloc; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); + if (!pos) + return; + + if (!pdev->pri_enabled) + return; + + pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); + pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); +} +EXPORT_SYMBOL_GPL(pci_restore_pri_state); + +/** * pci_reset_pri - Resets device's PRI state * @pdev: PCI device structure * @@ -224,12 +246,7 @@ int pci_reset_pri(struct pci_dev *pdev) if (!pos) return -EINVAL; - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); - if (control & PCI_PRI_CTRL_ENABLE) - return -EBUSY; - - control |= PCI_PRI_CTRL_RESET; - + control = PCI_PRI_CTRL_RESET; pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); return 0; @@ -259,12 +276,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) if (!pos) return -EINVAL; - pci_read_config_word(pdev, pos + PCI_PASID_CTRL, &control); pci_read_config_word(pdev, pos + PCI_PASID_CAP, &supported); - - if (control & PCI_PASID_CTRL_ENABLE) - return -EINVAL; - supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV; /* User wants to enable anything unsupported? */ @@ -272,6 +284,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) return -EINVAL; control = PCI_PASID_CTRL_ENABLE | features; + pdev->pasid_features = features; pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); @@ -305,6 +318,28 @@ void pci_disable_pasid(struct pci_dev *pdev) EXPORT_SYMBOL_GPL(pci_disable_pasid); /** + * pci_restore_pasid_state - Restore PASID capabilities. + * @pdev: PCI device structure + * + */ +void pci_restore_pasid_state(struct pci_dev *pdev) +{ + u16 control; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); + if (!pos) + return; + + if (!pdev->pasid_enabled) + return; + + control = PCI_PASID_CTRL_ENABLE | pdev->pasid_features; + pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); +} +EXPORT_SYMBOL_GPL(pci_restore_pasid_state); + +/** * pci_pasid_features - Check which PASID features are supported * @pdev: PCI device structure * diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 7904d02..c9a6510 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -28,6 +28,7 @@ #include <linux/pm_runtime.h> #include <linux/pci_hotplug.h> #include <linux/vmalloc.h> +#include <linux/pci-ats.h> #include <asm/setup.h> #include <asm/dma.h> #include <linux/aer.h> @@ -1171,6 +1172,8 @@ void pci_restore_state(struct pci_dev *dev) /* PCI Express register must be restored first */ pci_restore_pcie_state(dev); + pci_restore_pasid_state(dev); + pci_restore_pri_state(dev); pci_restore_ats_state(dev); pci_restore_vc_state(dev); diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h index 57e0b82..782fb8e 100644 --- a/include/linux/pci-ats.h +++ b/include/linux/pci-ats.h @@ -7,6 +7,7 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs); void pci_disable_pri(struct pci_dev *pdev); +void pci_restore_pri_state(struct pci_dev *pdev); int pci_reset_pri(struct pci_dev *pdev); #else /* CONFIG_PCI_PRI */ @@ -20,6 +21,10 @@ static inline void pci_disable_pri(struct pci_dev *pdev) { } +static inline void pci_restore_pri_state(struct pci_dev *pdev) +{ +} + static inline int pci_reset_pri(struct pci_dev *pdev) { return -ENODEV; @@ -31,6 +36,7 @@ static inline int pci_reset_pri(struct pci_dev *pdev) int pci_enable_pasid(struct pci_dev *pdev, int features); void pci_disable_pasid(struct pci_dev *pdev); +void pci_restore_pasid_state(struct pci_dev *pdev); int pci_pasid_features(struct pci_dev *pdev); int pci_max_pasids(struct pci_dev *pdev); @@ -45,6 +51,10 @@ static inline void pci_disable_pasid(struct pci_dev *pdev) { } +static inline void pci_restore_pasid_state(struct pci_dev *pdev) +{ +} + static inline int pci_pasid_features(struct pci_dev *pdev) { return -EINVAL; diff --git a/include/linux/pci.h b/include/linux/pci.h index bee980e..1ddb8e0 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -395,6 +395,12 @@ struct pci_dev { u8 ats_stu; /* ATS Smallest Translation Unit */ atomic_t ats_ref_cnt; /* number of VFs with ATS enabled */ #endif +#ifdef CONFIG_PCI_PRI + u32 pri_reqs_alloc; /* Number of PRI requests allocated */ +#endif +#ifdef CONFIG_PCI_PASID + u16 pasid_features; +#endif phys_addr_t rom; /* Physical address of ROM if it's not from the BAR */ size_t romlen; /* Length of ROM if it's not from the BAR */ char *driver_override; /* Driver name to force a match */ -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes. 2017-05-30 16:25 ` [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes Ashok Raj @ 2017-05-30 19:50 ` Bjorn Helgaas 2017-05-30 18:53 ` Raj, Ashok 2017-05-30 20:58 ` Bjorn Helgaas 1 sibling, 1 reply; 7+ messages in thread From: Bjorn Helgaas @ 2017-05-30 19:50 UTC (permalink / raw) To: Ashok Raj Cc: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel, CQ Tang, David Woodhouse, iommu On Tue, May 30, 2017 at 09:25:49AM -0700, Ashok Raj wrote: > From: CQ Tang <cq.tang@intel.com> > > Requires: https://patchwork.kernel.org/patch/9593891 The above patch (9593891) is not in my tree or Linus' tree, so I can't do anything with this yet. > After a FLR, pci-states need to be restored. This patch saves PASID features > and PRI reqs cached. > > To: Bjorn Helgaas <bhelgaas@google.com> > To: Joerg Roedel <joro@8bytes.org> > To: linux-pci@vger.kernel.org > To: linux-kernel@vger.kernel.org > Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> > Cc: David Woodhouse <dwmw2@infradead.org> > Cc: iommu@lists.linux-foundation.org > > Signed-off-by: CQ Tang <cq.tang@intel.com> > Signed-off-by: Ashok Raj <ashok.raj@intel.com> > --- > drivers/pci/ats.c | 65 +++++++++++++++++++++++++++++++++++++------------ > drivers/pci/pci.c | 3 +++ > include/linux/pci-ats.h | 10 ++++++++ > include/linux/pci.h | 6 +++++ > 4 files changed, 69 insertions(+), 15 deletions(-) > > diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c > index 2126497..a769955 100644 > --- a/drivers/pci/ats.c > +++ b/drivers/pci/ats.c > @@ -160,17 +160,16 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status); > - if ((control & PCI_PRI_CTRL_ENABLE) || > - !(status & PCI_PRI_STATUS_STOPPED)) > + if (!(status & PCI_PRI_STATUS_STOPPED)) > return -EBUSY; > > pci_read_config_dword(pdev, pos + PCI_PRI_MAX_REQ, &max_requests); > reqs = min(max_requests, reqs); > + pdev->pri_reqs_alloc = reqs; > pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > > - control |= PCI_PRI_CTRL_ENABLE; > + control = PCI_PRI_CTRL_ENABLE; > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > pdev->pri_enabled = 1; > @@ -206,6 +205,29 @@ void pci_disable_pri(struct pci_dev *pdev) > EXPORT_SYMBOL_GPL(pci_disable_pri); > > /** > + * pci_restore_pri_state - Restore PRI > + * @pdev: PCI device structure > + * > + */ > +void pci_restore_pri_state(struct pci_dev *pdev) > +{ > + u16 control = PCI_PRI_CTRL_ENABLE; > + u32 reqs = pdev->pri_reqs_alloc; > + int pos; > + > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); > + if (!pos) > + return; > + > + if (!pdev->pri_enabled) > + return; > + > + pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > + pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > +} > +EXPORT_SYMBOL_GPL(pci_restore_pri_state); > + > +/** > * pci_reset_pri - Resets device's PRI state > * @pdev: PCI device structure > * > @@ -224,12 +246,7 @@ int pci_reset_pri(struct pci_dev *pdev) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > - if (control & PCI_PRI_CTRL_ENABLE) > - return -EBUSY; > - > - control |= PCI_PRI_CTRL_RESET; > - > + control = PCI_PRI_CTRL_RESET; > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > return 0; > @@ -259,12 +276,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PASID_CTRL, &control); > pci_read_config_word(pdev, pos + PCI_PASID_CAP, &supported); > - > - if (control & PCI_PASID_CTRL_ENABLE) > - return -EINVAL; > - > supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV; > > /* User wants to enable anything unsupported? */ > @@ -272,6 +284,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > return -EINVAL; > > control = PCI_PASID_CTRL_ENABLE | features; > + pdev->pasid_features = features; > > pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > > @@ -305,6 +318,28 @@ void pci_disable_pasid(struct pci_dev *pdev) > EXPORT_SYMBOL_GPL(pci_disable_pasid); > > /** > + * pci_restore_pasid_state - Restore PASID capabilities. > + * @pdev: PCI device structure > + * > + */ > +void pci_restore_pasid_state(struct pci_dev *pdev) > +{ > + u16 control; > + int pos; > + > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); > + if (!pos) > + return; > + > + if (!pdev->pasid_enabled) > + return; > + > + control = PCI_PASID_CTRL_ENABLE | pdev->pasid_features; > + pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > +} > +EXPORT_SYMBOL_GPL(pci_restore_pasid_state); > + > +/** > * pci_pasid_features - Check which PASID features are supported > * @pdev: PCI device structure > * > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 7904d02..c9a6510 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -28,6 +28,7 @@ > #include <linux/pm_runtime.h> > #include <linux/pci_hotplug.h> > #include <linux/vmalloc.h> > +#include <linux/pci-ats.h> > #include <asm/setup.h> > #include <asm/dma.h> > #include <linux/aer.h> > @@ -1171,6 +1172,8 @@ void pci_restore_state(struct pci_dev *dev) > > /* PCI Express register must be restored first */ > pci_restore_pcie_state(dev); > + pci_restore_pasid_state(dev); > + pci_restore_pri_state(dev); > pci_restore_ats_state(dev); > pci_restore_vc_state(dev); > > diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h > index 57e0b82..782fb8e 100644 > --- a/include/linux/pci-ats.h > +++ b/include/linux/pci-ats.h > @@ -7,6 +7,7 @@ > > int pci_enable_pri(struct pci_dev *pdev, u32 reqs); > void pci_disable_pri(struct pci_dev *pdev); > +void pci_restore_pri_state(struct pci_dev *pdev); > int pci_reset_pri(struct pci_dev *pdev); > > #else /* CONFIG_PCI_PRI */ > @@ -20,6 +21,10 @@ static inline void pci_disable_pri(struct pci_dev *pdev) > { > } > > +static inline void pci_restore_pri_state(struct pci_dev *pdev) > +{ > +} > + > static inline int pci_reset_pri(struct pci_dev *pdev) > { > return -ENODEV; > @@ -31,6 +36,7 @@ static inline int pci_reset_pri(struct pci_dev *pdev) > > int pci_enable_pasid(struct pci_dev *pdev, int features); > void pci_disable_pasid(struct pci_dev *pdev); > +void pci_restore_pasid_state(struct pci_dev *pdev); > int pci_pasid_features(struct pci_dev *pdev); > int pci_max_pasids(struct pci_dev *pdev); > > @@ -45,6 +51,10 @@ static inline void pci_disable_pasid(struct pci_dev *pdev) > { > } > > +static inline void pci_restore_pasid_state(struct pci_dev *pdev) > +{ > +} > + > static inline int pci_pasid_features(struct pci_dev *pdev) > { > return -EINVAL; > diff --git a/include/linux/pci.h b/include/linux/pci.h > index bee980e..1ddb8e0 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -395,6 +395,12 @@ struct pci_dev { > u8 ats_stu; /* ATS Smallest Translation Unit */ > atomic_t ats_ref_cnt; /* number of VFs with ATS enabled */ > #endif > +#ifdef CONFIG_PCI_PRI > + u32 pri_reqs_alloc; /* Number of PRI requests allocated */ > +#endif > +#ifdef CONFIG_PCI_PASID > + u16 pasid_features; > +#endif > phys_addr_t rom; /* Physical address of ROM if it's not from the BAR */ > size_t romlen; /* Length of ROM if it's not from the BAR */ > char *driver_override; /* Driver name to force a match */ > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes. 2017-05-30 19:50 ` Bjorn Helgaas @ 2017-05-30 18:53 ` Raj, Ashok 0 siblings, 0 replies; 7+ messages in thread From: Raj, Ashok @ 2017-05-30 18:53 UTC (permalink / raw) To: Bjorn Helgaas Cc: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel, CQ Tang, David Woodhouse, iommu, ashok.raj On Tue, May 30, 2017 at 02:50:33PM -0500, Bjorn Helgaas wrote: > On Tue, May 30, 2017 at 09:25:49AM -0700, Ashok Raj wrote: > > From: CQ Tang <cq.tang@intel.com> > > > > Requires: https://patchwork.kernel.org/patch/9593891 > > The above patch (9593891) is not in my tree or Linus' tree, so I can't > do anything with this yet. I resent the patch as part of this series.. maybe should have massaged the commit message.. my bad :-(. Its the first patch in this series. Jean mentioned it might take him a while before the SVM patches stabilize. Since this patch is pretty much stand alone, he asked if i could resend it along with the one i need. Sorry for any confusion. Cheers, Ashok > > > After a FLR, pci-states need to be restored. This patch saves PASID features > > and PRI reqs cached. > > > > To: Bjorn Helgaas <bhelgaas@google.com> > > To: Joerg Roedel <joro@8bytes.org> > > To: linux-pci@vger.kernel.org > > To: linux-kernel@vger.kernel.org > > Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> > > Cc: David Woodhouse <dwmw2@infradead.org> > > Cc: iommu@lists.linux-foundation.org > > > > Signed-off-by: CQ Tang <cq.tang@intel.com> > > Signed-off-by: Ashok Raj <ashok.raj@intel.com> > > --- > > drivers/pci/ats.c | 65 +++++++++++++++++++++++++++++++++++++------------ > > drivers/pci/pci.c | 3 +++ > > include/linux/pci-ats.h | 10 ++++++++ > > include/linux/pci.h | 6 +++++ > > 4 files changed, 69 insertions(+), 15 deletions(-) > > > > diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c > > index 2126497..a769955 100644 > > --- a/drivers/pci/ats.c > > +++ b/drivers/pci/ats.c > > @@ -160,17 +160,16 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) > > if (!pos) > > return -EINVAL; > > > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > > pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status); > > - if ((control & PCI_PRI_CTRL_ENABLE) || > > - !(status & PCI_PRI_STATUS_STOPPED)) > > + if (!(status & PCI_PRI_STATUS_STOPPED)) > > return -EBUSY; > > > > pci_read_config_dword(pdev, pos + PCI_PRI_MAX_REQ, &max_requests); > > reqs = min(max_requests, reqs); > > + pdev->pri_reqs_alloc = reqs; > > pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > > > > - control |= PCI_PRI_CTRL_ENABLE; > > + control = PCI_PRI_CTRL_ENABLE; > > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > > > pdev->pri_enabled = 1; > > @@ -206,6 +205,29 @@ void pci_disable_pri(struct pci_dev *pdev) > > EXPORT_SYMBOL_GPL(pci_disable_pri); > > > > /** > > + * pci_restore_pri_state - Restore PRI > > + * @pdev: PCI device structure > > + * > > + */ > > +void pci_restore_pri_state(struct pci_dev *pdev) > > +{ > > + u16 control = PCI_PRI_CTRL_ENABLE; > > + u32 reqs = pdev->pri_reqs_alloc; > > + int pos; > > + > > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); > > + if (!pos) > > + return; > > + > > + if (!pdev->pri_enabled) > > + return; > > + > > + pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > > + pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > +} > > +EXPORT_SYMBOL_GPL(pci_restore_pri_state); > > + > > +/** > > * pci_reset_pri - Resets device's PRI state > > * @pdev: PCI device structure > > * > > @@ -224,12 +246,7 @@ int pci_reset_pri(struct pci_dev *pdev) > > if (!pos) > > return -EINVAL; > > > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > > - if (control & PCI_PRI_CTRL_ENABLE) > > - return -EBUSY; > > - > > - control |= PCI_PRI_CTRL_RESET; > > - > > + control = PCI_PRI_CTRL_RESET; > > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > > > return 0; > > @@ -259,12 +276,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > > if (!pos) > > return -EINVAL; > > > > - pci_read_config_word(pdev, pos + PCI_PASID_CTRL, &control); > > pci_read_config_word(pdev, pos + PCI_PASID_CAP, &supported); > > - > > - if (control & PCI_PASID_CTRL_ENABLE) > > - return -EINVAL; > > - > > supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV; > > > > /* User wants to enable anything unsupported? */ > > @@ -272,6 +284,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > > return -EINVAL; > > > > control = PCI_PASID_CTRL_ENABLE | features; > > + pdev->pasid_features = features; > > > > pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > > > > @@ -305,6 +318,28 @@ void pci_disable_pasid(struct pci_dev *pdev) > > EXPORT_SYMBOL_GPL(pci_disable_pasid); > > > > /** > > + * pci_restore_pasid_state - Restore PASID capabilities. > > + * @pdev: PCI device structure > > + * > > + */ > > +void pci_restore_pasid_state(struct pci_dev *pdev) > > +{ > > + u16 control; > > + int pos; > > + > > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); > > + if (!pos) > > + return; > > + > > + if (!pdev->pasid_enabled) > > + return; > > + > > + control = PCI_PASID_CTRL_ENABLE | pdev->pasid_features; > > + pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > > +} > > +EXPORT_SYMBOL_GPL(pci_restore_pasid_state); > > + > > +/** > > * pci_pasid_features - Check which PASID features are supported > > * @pdev: PCI device structure > > * > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > > index 7904d02..c9a6510 100644 > > --- a/drivers/pci/pci.c > > +++ b/drivers/pci/pci.c > > @@ -28,6 +28,7 @@ > > #include <linux/pm_runtime.h> > > #include <linux/pci_hotplug.h> > > #include <linux/vmalloc.h> > > +#include <linux/pci-ats.h> > > #include <asm/setup.h> > > #include <asm/dma.h> > > #include <linux/aer.h> > > @@ -1171,6 +1172,8 @@ void pci_restore_state(struct pci_dev *dev) > > > > /* PCI Express register must be restored first */ > > pci_restore_pcie_state(dev); > > + pci_restore_pasid_state(dev); > > + pci_restore_pri_state(dev); > > pci_restore_ats_state(dev); > > pci_restore_vc_state(dev); > > > > diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h > > index 57e0b82..782fb8e 100644 > > --- a/include/linux/pci-ats.h > > +++ b/include/linux/pci-ats.h > > @@ -7,6 +7,7 @@ > > > > int pci_enable_pri(struct pci_dev *pdev, u32 reqs); > > void pci_disable_pri(struct pci_dev *pdev); > > +void pci_restore_pri_state(struct pci_dev *pdev); > > int pci_reset_pri(struct pci_dev *pdev); > > > > #else /* CONFIG_PCI_PRI */ > > @@ -20,6 +21,10 @@ static inline void pci_disable_pri(struct pci_dev *pdev) > > { > > } > > > > +static inline void pci_restore_pri_state(struct pci_dev *pdev) > > +{ > > +} > > + > > static inline int pci_reset_pri(struct pci_dev *pdev) > > { > > return -ENODEV; > > @@ -31,6 +36,7 @@ static inline int pci_reset_pri(struct pci_dev *pdev) > > > > int pci_enable_pasid(struct pci_dev *pdev, int features); > > void pci_disable_pasid(struct pci_dev *pdev); > > +void pci_restore_pasid_state(struct pci_dev *pdev); > > int pci_pasid_features(struct pci_dev *pdev); > > int pci_max_pasids(struct pci_dev *pdev); > > > > @@ -45,6 +51,10 @@ static inline void pci_disable_pasid(struct pci_dev *pdev) > > { > > } > > > > +static inline void pci_restore_pasid_state(struct pci_dev *pdev) > > +{ > > +} > > + > > static inline int pci_pasid_features(struct pci_dev *pdev) > > { > > return -EINVAL; > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > index bee980e..1ddb8e0 100644 > > --- a/include/linux/pci.h > > +++ b/include/linux/pci.h > > @@ -395,6 +395,12 @@ struct pci_dev { > > u8 ats_stu; /* ATS Smallest Translation Unit */ > > atomic_t ats_ref_cnt; /* number of VFs with ATS enabled */ > > #endif > > +#ifdef CONFIG_PCI_PRI > > + u32 pri_reqs_alloc; /* Number of PRI requests allocated */ > > +#endif > > +#ifdef CONFIG_PCI_PASID > > + u16 pasid_features; > > +#endif > > phys_addr_t rom; /* Physical address of ROM if it's not from the BAR */ > > size_t romlen; /* Length of ROM if it's not from the BAR */ > > char *driver_override; /* Driver name to force a match */ > > -- > > 2.7.4 > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes. 2017-05-30 16:25 ` [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes Ashok Raj 2017-05-30 19:50 ` Bjorn Helgaas @ 2017-05-30 20:58 ` Bjorn Helgaas 1 sibling, 0 replies; 7+ messages in thread From: Bjorn Helgaas @ 2017-05-30 20:58 UTC (permalink / raw) To: Ashok Raj Cc: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel, CQ Tang, David Woodhouse, iommu On Tue, May 30, 2017 at 09:25:49AM -0700, Ashok Raj wrote: > From: CQ Tang <cq.tang@intel.com> > > Requires: https://patchwork.kernel.org/patch/9593891 > > > After a FLR, pci-states need to be restored. This patch saves PASID features > and PRI reqs cached. > > To: Bjorn Helgaas <bhelgaas@google.com> > To: Joerg Roedel <joro@8bytes.org> > To: linux-pci@vger.kernel.org > To: linux-kernel@vger.kernel.org > Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> > Cc: David Woodhouse <dwmw2@infradead.org> > Cc: iommu@lists.linux-foundation.org > > Signed-off-by: CQ Tang <cq.tang@intel.com> > Signed-off-by: Ashok Raj <ashok.raj@intel.com> > --- > drivers/pci/ats.c | 65 +++++++++++++++++++++++++++++++++++++------------ > drivers/pci/pci.c | 3 +++ > include/linux/pci-ats.h | 10 ++++++++ > include/linux/pci.h | 6 +++++ > 4 files changed, 69 insertions(+), 15 deletions(-) > > diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c > index 2126497..a769955 100644 > --- a/drivers/pci/ats.c > +++ b/drivers/pci/ats.c > @@ -160,17 +160,16 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status); > - if ((control & PCI_PRI_CTRL_ENABLE) || > - !(status & PCI_PRI_STATUS_STOPPED)) > + if (!(status & PCI_PRI_STATUS_STOPPED)) > return -EBUSY; > > pci_read_config_dword(pdev, pos + PCI_PRI_MAX_REQ, &max_requests); > reqs = min(max_requests, reqs); > + pdev->pri_reqs_alloc = reqs; > pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > > - control |= PCI_PRI_CTRL_ENABLE; > + control = PCI_PRI_CTRL_ENABLE; > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > pdev->pri_enabled = 1; > @@ -206,6 +205,29 @@ void pci_disable_pri(struct pci_dev *pdev) > EXPORT_SYMBOL_GPL(pci_disable_pri); > > /** > + * pci_restore_pri_state - Restore PRI > + * @pdev: PCI device structure > + * > + */ > +void pci_restore_pri_state(struct pci_dev *pdev) > +{ > + u16 control = PCI_PRI_CTRL_ENABLE; > + u32 reqs = pdev->pri_reqs_alloc; > + int pos; > + > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI); > + if (!pos) > + return; > + > + if (!pdev->pri_enabled) > + return; I propose swapping the order of these tests, so that if PRI is not enabled, we don't have to search for the capability. Similarly for PASID below. I made these changes and re-indented these functions on my branch. No action required unless you object to these changes. > + > + pci_write_config_dword(pdev, pos + PCI_PRI_ALLOC_REQ, reqs); > + pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > +} > +EXPORT_SYMBOL_GPL(pci_restore_pri_state); > + > +/** > * pci_reset_pri - Resets device's PRI state > * @pdev: PCI device structure > * > @@ -224,12 +246,7 @@ int pci_reset_pri(struct pci_dev *pdev) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PRI_CTRL, &control); > - if (control & PCI_PRI_CTRL_ENABLE) > - return -EBUSY; > - > - control |= PCI_PRI_CTRL_RESET; > - > + control = PCI_PRI_CTRL_RESET; > pci_write_config_word(pdev, pos + PCI_PRI_CTRL, control); > > return 0; > @@ -259,12 +276,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > if (!pos) > return -EINVAL; > > - pci_read_config_word(pdev, pos + PCI_PASID_CTRL, &control); > pci_read_config_word(pdev, pos + PCI_PASID_CAP, &supported); > - > - if (control & PCI_PASID_CTRL_ENABLE) > - return -EINVAL; > - > supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV; > > /* User wants to enable anything unsupported? */ > @@ -272,6 +284,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) > return -EINVAL; > > control = PCI_PASID_CTRL_ENABLE | features; > + pdev->pasid_features = features; > > pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > > @@ -305,6 +318,28 @@ void pci_disable_pasid(struct pci_dev *pdev) > EXPORT_SYMBOL_GPL(pci_disable_pasid); > > /** > + * pci_restore_pasid_state - Restore PASID capabilities. > + * @pdev: PCI device structure > + * > + */ > +void pci_restore_pasid_state(struct pci_dev *pdev) > +{ > + u16 control; > + int pos; > + > + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID); > + if (!pos) > + return; > + > + if (!pdev->pasid_enabled) > + return; > + > + control = PCI_PASID_CTRL_ENABLE | pdev->pasid_features; > + pci_write_config_word(pdev, pos + PCI_PASID_CTRL, control); > +} > +EXPORT_SYMBOL_GPL(pci_restore_pasid_state); > + > +/** > * pci_pasid_features - Check which PASID features are supported > * @pdev: PCI device structure > * > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 7904d02..c9a6510 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -28,6 +28,7 @@ > #include <linux/pm_runtime.h> > #include <linux/pci_hotplug.h> > #include <linux/vmalloc.h> > +#include <linux/pci-ats.h> > #include <asm/setup.h> > #include <asm/dma.h> > #include <linux/aer.h> > @@ -1171,6 +1172,8 @@ void pci_restore_state(struct pci_dev *dev) > > /* PCI Express register must be restored first */ > pci_restore_pcie_state(dev); > + pci_restore_pasid_state(dev); > + pci_restore_pri_state(dev); > pci_restore_ats_state(dev); > pci_restore_vc_state(dev); > > diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h > index 57e0b82..782fb8e 100644 > --- a/include/linux/pci-ats.h > +++ b/include/linux/pci-ats.h > @@ -7,6 +7,7 @@ > > int pci_enable_pri(struct pci_dev *pdev, u32 reqs); > void pci_disable_pri(struct pci_dev *pdev); > +void pci_restore_pri_state(struct pci_dev *pdev); > int pci_reset_pri(struct pci_dev *pdev); > > #else /* CONFIG_PCI_PRI */ > @@ -20,6 +21,10 @@ static inline void pci_disable_pri(struct pci_dev *pdev) > { > } > > +static inline void pci_restore_pri_state(struct pci_dev *pdev) > +{ > +} > + > static inline int pci_reset_pri(struct pci_dev *pdev) > { > return -ENODEV; > @@ -31,6 +36,7 @@ static inline int pci_reset_pri(struct pci_dev *pdev) > > int pci_enable_pasid(struct pci_dev *pdev, int features); > void pci_disable_pasid(struct pci_dev *pdev); > +void pci_restore_pasid_state(struct pci_dev *pdev); > int pci_pasid_features(struct pci_dev *pdev); > int pci_max_pasids(struct pci_dev *pdev); > > @@ -45,6 +51,10 @@ static inline void pci_disable_pasid(struct pci_dev *pdev) > { > } > > +static inline void pci_restore_pasid_state(struct pci_dev *pdev) > +{ > +} > + > static inline int pci_pasid_features(struct pci_dev *pdev) > { > return -EINVAL; > diff --git a/include/linux/pci.h b/include/linux/pci.h > index bee980e..1ddb8e0 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -395,6 +395,12 @@ struct pci_dev { > u8 ats_stu; /* ATS Smallest Translation Unit */ > atomic_t ats_ref_cnt; /* number of VFs with ATS enabled */ > #endif > +#ifdef CONFIG_PCI_PRI > + u32 pri_reqs_alloc; /* Number of PRI requests allocated */ > +#endif > +#ifdef CONFIG_PCI_PASID > + u16 pasid_features; > +#endif > phys_addr_t rom; /* Physical address of ROM if it's not from the BAR */ > size_t romlen; /* Length of ROM if it's not from the BAR */ > char *driver_override; /* Driver name to force a match */ > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/2] Save and restore pci properties to support FLR 2017-05-30 16:25 [PATCH 0/2] Save and restore pci properties to support FLR Ashok Raj 2017-05-30 16:25 ` [PATCH 1/2] PCI: Cache PRI and PASID bits in pci_dev Ashok Raj 2017-05-30 16:25 ` [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes Ashok Raj @ 2017-05-30 20:58 ` Bjorn Helgaas 2 siblings, 0 replies; 7+ messages in thread From: Bjorn Helgaas @ 2017-05-30 20:58 UTC (permalink / raw) To: Ashok Raj Cc: Jean-Phillipe Brucker, Bjorn Helgaas, Joerg Roedel, linux-pci, linux-kernel, David Woodhouse, iommu On Tue, May 30, 2017 at 09:25:47AM -0700, Ashok Raj wrote: > Resending Jean's patch so it can be included earlier than his large > SVM commits. Original patch https://patchwork.kernel.org/patch/9593891 > was ack'ed by Bjorn. Let's commit these separately since we need > functionality earlier. > > Resending this series as requested by Jean. > > CQ Tang (1): > PCI: Save properties required to handle FLR for replay purposes. > > Jean-Philippe Brucker (1): > PCI: Cache PRI and PASID bits in pci_dev > > drivers/pci/ats.c | 88 ++++++++++++++++++++++++++++++++++++++++--------- > drivers/pci/pci.c | 3 ++ > include/linux/pci-ats.h | 10 ++++++ > include/linux/pci.h | 8 +++++ > 4 files changed, 94 insertions(+), 15 deletions(-) Applied to pci/virtualization for v4.13. See response to 2/2 for minor changes I made there. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-05-30 20:58 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-05-30 16:25 [PATCH 0/2] Save and restore pci properties to support FLR Ashok Raj 2017-05-30 16:25 ` [PATCH 1/2] PCI: Cache PRI and PASID bits in pci_dev Ashok Raj 2017-05-30 16:25 ` [PATCH 2/2] PCI: Save properties required to handle FLR for replay purposes Ashok Raj 2017-05-30 19:50 ` Bjorn Helgaas 2017-05-30 18:53 ` Raj, Ashok 2017-05-30 20:58 ` Bjorn Helgaas 2017-05-30 20:58 ` [PATCH 0/2] Save and restore pci properties to support FLR Bjorn Helgaas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).