* [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices @ 2017-07-20 0:46 Feng Kan 2017-07-20 22:22 ` Alex Williamson 0 siblings, 1 reply; 6+ messages in thread From: Feng Kan @ 2017-07-20 0:46 UTC (permalink / raw) To: linux-kernel, linux-pci, bhelgaas, alex.williamson; +Cc: Feng Kan The APM X-Gene PCIe root port does not support ACS at this point. However, the hw provides isolation and source validation through the SMMU. Turn on ACS but disable all the peer to peer features. Signed-off-by: Feng Kan <fkan@apm.com> --- drivers/pci/quirks.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 085fb78..0f8f1cd 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) return acs_flags ? 0 : 1; } +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) +{ + /* + * XGene root matching this quirk do not allow peer-to-peer + * transactions with others, allowing masking out these bits as if they + * were unimplemented in the ACS capability. + */ + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); + + return acs_flags ? 0 : 1; +} + /* * Many Intel PCH root ports do provide ACS-like features to disable peer * transactions and validate bus numbers in requests, but do not provide an @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ /* Cavium ThunderX */ { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, + /* APM XGene */ + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, { 0 } }; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices 2017-07-20 0:46 [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices Feng Kan @ 2017-07-20 22:22 ` Alex Williamson 2017-07-21 20:20 ` Feng Kan 0 siblings, 1 reply; 6+ messages in thread From: Alex Williamson @ 2017-07-20 22:22 UTC (permalink / raw) To: Feng Kan; +Cc: linux-kernel, linux-pci, bhelgaas On Wed, 19 Jul 2017 17:46:51 -0700 Feng Kan <fkan@apm.com> wrote: > The APM X-Gene PCIe root port does not support ACS at this point. > However, the hw provides isolation and source validation through > the SMMU. Turn on ACS but disable all the peer to peer features. > > Signed-off-by: Feng Kan <fkan@apm.com> > --- > drivers/pci/quirks.c | 15 +++++++++++++++ > 1 file changed, 15 insertions(+) > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index 085fb78..0f8f1cd 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > return acs_flags ? 0 : 1; > } > > +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) > +{ > + /* > + * XGene root matching this quirk do not allow peer-to-peer > + * transactions with others, allowing masking out these bits as if they > + * were unimplemented in the ACS capability. > + */ > + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | > + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); > + > + return acs_flags ? 0 : 1; > +} > + > /* > * Many Intel PCH root ports do provide ACS-like features to disable peer > * transactions and validate bus numbers in requests, but do not provide an > @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) > { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ > /* Cavium ThunderX */ > { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, > + /* APM XGene */ > + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, > { 0 } > }; > Sorry, I'm not yet convinced there's an equivalent of SV, if a device spoofs a different bdf and it reaches the smmu, what prevents that from simply referencing the context for that alternate bdf? The point of root port SV is to ensure that any forwarded transaction is within the downstream bdf range. Thanks, Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices 2017-07-20 22:22 ` Alex Williamson @ 2017-07-21 20:20 ` Feng Kan 2017-07-24 2:06 ` Alex Williamson 0 siblings, 1 reply; 6+ messages in thread From: Feng Kan @ 2017-07-21 20:20 UTC (permalink / raw) To: Alex Williamson; +Cc: linux-kernel@vger.kernel.org, linux-pci, Bjorn Helgaas On Thu, Jul 20, 2017 at 3:22 PM, Alex Williamson <alex.williamson@redhat.com> wrote: > On Wed, 19 Jul 2017 17:46:51 -0700 > Feng Kan <fkan@apm.com> wrote: > >> The APM X-Gene PCIe root port does not support ACS at this point. >> However, the hw provides isolation and source validation through >> the SMMU. Turn on ACS but disable all the peer to peer features. >> >> Signed-off-by: Feng Kan <fkan@apm.com> >> --- >> drivers/pci/quirks.c | 15 +++++++++++++++ >> 1 file changed, 15 insertions(+) >> >> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c >> index 085fb78..0f8f1cd 100644 >> --- a/drivers/pci/quirks.c >> +++ b/drivers/pci/quirks.c >> @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) >> return acs_flags ? 0 : 1; >> } >> >> +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) >> +{ >> + /* >> + * XGene root matching this quirk do not allow peer-to-peer >> + * transactions with others, allowing masking out these bits as if they >> + * were unimplemented in the ACS capability. >> + */ >> + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | >> + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); >> + >> + return acs_flags ? 0 : 1; >> +} >> + >> /* >> * Many Intel PCH root ports do provide ACS-like features to disable peer >> * transactions and validate bus numbers in requests, but do not provide an >> @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) >> { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ >> /* Cavium ThunderX */ >> { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, >> + /* APM XGene */ >> + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, >> { 0 } >> }; >> > > Sorry, I'm not yet convinced there's an equivalent of SV, if a device > spoofs a different bdf and it reaches the smmu, what prevents that from > simply referencing the context for that alternate bdf? Perhaps I am not understanding the question correctly. The bdf forms a stream id which is used to provide an context. Since there is no actual context created by an alternate bdf, the transaction would be rejected by the SMMU. The point of > root port SV is to ensure that any forwarded transaction is within the > downstream bdf range. Thanks, > > Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices 2017-07-21 20:20 ` Feng Kan @ 2017-07-24 2:06 ` Alex Williamson 2017-07-24 17:33 ` Feng Kan 0 siblings, 1 reply; 6+ messages in thread From: Alex Williamson @ 2017-07-24 2:06 UTC (permalink / raw) To: Feng Kan; +Cc: linux-kernel@vger.kernel.org, linux-pci, Bjorn Helgaas On Fri, 21 Jul 2017 13:20:18 -0700 Feng Kan <fkan@apm.com> wrote: > On Thu, Jul 20, 2017 at 3:22 PM, Alex Williamson > <alex.williamson@redhat.com> wrote: > > On Wed, 19 Jul 2017 17:46:51 -0700 > > Feng Kan <fkan@apm.com> wrote: > > > >> The APM X-Gene PCIe root port does not support ACS at this point. > >> However, the hw provides isolation and source validation through > >> the SMMU. Turn on ACS but disable all the peer to peer features. > >> > >> Signed-off-by: Feng Kan <fkan@apm.com> > >> --- > >> drivers/pci/quirks.c | 15 +++++++++++++++ > >> 1 file changed, 15 insertions(+) > >> > >> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > >> index 085fb78..0f8f1cd 100644 > >> --- a/drivers/pci/quirks.c > >> +++ b/drivers/pci/quirks.c > >> @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > >> return acs_flags ? 0 : 1; > >> } > >> > >> +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) > >> +{ > >> + /* > >> + * XGene root matching this quirk do not allow peer-to-peer > >> + * transactions with others, allowing masking out these bits as if they > >> + * were unimplemented in the ACS capability. > >> + */ > >> + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | > >> + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); > >> + > >> + return acs_flags ? 0 : 1; > >> +} > >> + > >> /* > >> * Many Intel PCH root ports do provide ACS-like features to disable peer > >> * transactions and validate bus numbers in requests, but do not provide an > >> @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) > >> { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ > >> /* Cavium ThunderX */ > >> { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, > >> + /* APM XGene */ > >> + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, > >> { 0 } > >> }; > >> > > > > Sorry, I'm not yet convinced there's an equivalent of SV, if a device > > spoofs a different bdf and it reaches the smmu, what prevents that from > > simply referencing the context for that alternate bdf? > Perhaps I am not understanding the question correctly. The bdf forms a > stream id which is used to provide an context. Since there is no actual > context created by an alternate bdf, the transaction would be rejected > by the SMMU. Unless that context does exist. Take for example a basic topology: -[0000:00]-+-00.0 +-01.0-[01]----00.0 +-02.0-[02]----00.0 Assume 00:01.0 and 00:02.0 are root ports and 01:00.0 and 02:00.0 are their respective downstream endpoint. A single iommu would typically handle both of these endpoints using the requester ID, aka stream ID, to reference the appropriate context. Source validation at the root port makes sure that any forwarded transaction has a requester ID that falls between the secondary and subordinate bus number range of the root port. For instance, if device 01:00.0 was able to generate a transaction with a requester ID of 02:00.0, source validation at the root port would generate an ACS violation. If the root port does not support source validation, the transaction might successfully reference the iommu context for the other endpoint. Therefore I don't understand what property of the SMMU is able to prevent this spoofing technique if the root port provides no protection on its own. Thanks, Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices 2017-07-24 2:06 ` Alex Williamson @ 2017-07-24 17:33 ` Feng Kan 2017-07-24 18:48 ` Alex Williamson 0 siblings, 1 reply; 6+ messages in thread From: Feng Kan @ 2017-07-24 17:33 UTC (permalink / raw) To: Alex Williamson; +Cc: linux-kernel@vger.kernel.org, linux-pci, Bjorn Helgaas On Sun, Jul 23, 2017 at 7:06 PM, Alex Williamson <alex.williamson@redhat.com> wrote: > On Fri, 21 Jul 2017 13:20:18 -0700 > Feng Kan <fkan@apm.com> wrote: > >> On Thu, Jul 20, 2017 at 3:22 PM, Alex Williamson >> <alex.williamson@redhat.com> wrote: >> > On Wed, 19 Jul 2017 17:46:51 -0700 >> > Feng Kan <fkan@apm.com> wrote: >> > >> >> The APM X-Gene PCIe root port does not support ACS at this point. >> >> However, the hw provides isolation and source validation through >> >> the SMMU. Turn on ACS but disable all the peer to peer features. >> >> >> >> Signed-off-by: Feng Kan <fkan@apm.com> >> >> --- >> >> drivers/pci/quirks.c | 15 +++++++++++++++ >> >> 1 file changed, 15 insertions(+) >> >> >> >> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c >> >> index 085fb78..0f8f1cd 100644 >> >> --- a/drivers/pci/quirks.c >> >> +++ b/drivers/pci/quirks.c >> >> @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) >> >> return acs_flags ? 0 : 1; >> >> } >> >> >> >> +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) >> >> +{ >> >> + /* >> >> + * XGene root matching this quirk do not allow peer-to-peer >> >> + * transactions with others, allowing masking out these bits as if they >> >> + * were unimplemented in the ACS capability. >> >> + */ >> >> + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | >> >> + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); >> >> + >> >> + return acs_flags ? 0 : 1; >> >> +} >> >> + >> >> /* >> >> * Many Intel PCH root ports do provide ACS-like features to disable peer >> >> * transactions and validate bus numbers in requests, but do not provide an >> >> @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) >> >> { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ >> >> /* Cavium ThunderX */ >> >> { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, >> >> + /* APM XGene */ >> >> + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, >> >> { 0 } >> >> }; >> >> >> > >> > Sorry, I'm not yet convinced there's an equivalent of SV, if a device >> > spoofs a different bdf and it reaches the smmu, what prevents that from >> > simply referencing the context for that alternate bdf? >> Perhaps I am not understanding the question correctly. The bdf forms a >> stream id which is used to provide an context. Since there is no actual >> context created by an alternate bdf, the transaction would be rejected >> by the SMMU. > > Unless that context does exist. Take for example a basic topology: > > -[0000:00]-+-00.0 > +-01.0-[01]----00.0 > +-02.0-[02]----00.0 > > Assume 00:01.0 and 00:02.0 are root ports and 01:00.0 and 02:00.0 are > their respective downstream endpoint. A single iommu would typically > handle both of these endpoints using the requester ID, aka stream ID, to > reference the appropriate context. Source validation at the root port > makes sure that any forwarded transaction has a requester ID that falls > between the secondary and subordinate bus number range of the root > port. For instance, if device 01:00.0 was able to generate a > transaction with a requester ID of 02:00.0, source validation at the > root port would generate an ACS violation. If the root port does not > support source validation, the transaction might successfully reference > the iommu context for the other endpoint. Therefore I don't understand > what property of the SMMU is able to prevent this spoofing technique if > the root port provides no protection on its own. Thanks, Yes, you are right. I left out some details to our implementation. Our implementation of the stream ID is not just the BDF. The stream ID also contains a port ID at the upper bits to indicate the difference between the PCIe ports. The scenario you explained up above should not happen. > > Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices 2017-07-24 17:33 ` Feng Kan @ 2017-07-24 18:48 ` Alex Williamson 0 siblings, 0 replies; 6+ messages in thread From: Alex Williamson @ 2017-07-24 18:48 UTC (permalink / raw) To: Feng Kan; +Cc: linux-kernel@vger.kernel.org, linux-pci, Bjorn Helgaas On Mon, 24 Jul 2017 10:33:25 -0700 Feng Kan <fkan@apm.com> wrote: > On Sun, Jul 23, 2017 at 7:06 PM, Alex Williamson > <alex.williamson@redhat.com> wrote: > > On Fri, 21 Jul 2017 13:20:18 -0700 > > Feng Kan <fkan@apm.com> wrote: > > > >> On Thu, Jul 20, 2017 at 3:22 PM, Alex Williamson > >> <alex.williamson@redhat.com> wrote: > >> > On Wed, 19 Jul 2017 17:46:51 -0700 > >> > Feng Kan <fkan@apm.com> wrote: > >> > > >> >> The APM X-Gene PCIe root port does not support ACS at this point. > >> >> However, the hw provides isolation and source validation through > >> >> the SMMU. Turn on ACS but disable all the peer to peer features. > >> >> > >> >> Signed-off-by: Feng Kan <fkan@apm.com> > >> >> --- > >> >> drivers/pci/quirks.c | 15 +++++++++++++++ > >> >> 1 file changed, 15 insertions(+) > >> >> > >> >> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > >> >> index 085fb78..0f8f1cd 100644 > >> >> --- a/drivers/pci/quirks.c > >> >> +++ b/drivers/pci/quirks.c > >> >> @@ -4120,6 +4120,19 @@ static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > >> >> return acs_flags ? 0 : 1; > >> >> } > >> >> > >> >> +static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) > >> >> +{ > >> >> + /* > >> >> + * XGene root matching this quirk do not allow peer-to-peer > >> >> + * transactions with others, allowing masking out these bits as if they > >> >> + * were unimplemented in the ACS capability. > >> >> + */ > >> >> + acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | > >> >> + PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); > >> >> + > >> >> + return acs_flags ? 0 : 1; > >> >> +} > >> >> + > >> >> /* > >> >> * Many Intel PCH root ports do provide ACS-like features to disable peer > >> >> * transactions and validate bus numbers in requests, but do not provide an > >> >> @@ -4368,6 +4381,8 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) > >> >> { 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */ > >> >> /* Cavium ThunderX */ > >> >> { PCI_VENDOR_ID_CAVIUM, PCI_ANY_ID, pci_quirk_cavium_acs }, > >> >> + /* APM XGene */ > >> >> + { PCI_VENDOR_ID_AMCC, 0xE004, pci_quirk_xgene_acs }, > >> >> { 0 } > >> >> }; > >> >> > >> > > >> > Sorry, I'm not yet convinced there's an equivalent of SV, if a device > >> > spoofs a different bdf and it reaches the smmu, what prevents that from > >> > simply referencing the context for that alternate bdf? > >> Perhaps I am not understanding the question correctly. The bdf forms a > >> stream id which is used to provide an context. Since there is no actual > >> context created by an alternate bdf, the transaction would be rejected > >> by the SMMU. > > > > Unless that context does exist. Take for example a basic topology: > > > > -[0000:00]-+-00.0 > > +-01.0-[01]----00.0 > > +-02.0-[02]----00.0 > > > > Assume 00:01.0 and 00:02.0 are root ports and 01:00.0 and 02:00.0 are > > their respective downstream endpoint. A single iommu would typically > > handle both of these endpoints using the requester ID, aka stream ID, to > > reference the appropriate context. Source validation at the root port > > makes sure that any forwarded transaction has a requester ID that falls > > between the secondary and subordinate bus number range of the root > > port. For instance, if device 01:00.0 was able to generate a > > transaction with a requester ID of 02:00.0, source validation at the > > root port would generate an ACS violation. If the root port does not > > support source validation, the transaction might successfully reference > > the iommu context for the other endpoint. Therefore I don't understand > > what property of the SMMU is able to prevent this spoofing technique if > > the root port provides no protection on its own. Thanks, > Yes, you are right. I left out some details to our implementation. Our > implementation > of the stream ID is not just the BDF. The stream ID also contains a > port ID at the > upper bits to indicate the difference between the PCIe ports. The scenario you > explained up above should not happen. Ok, great. Can we update the comment in the patch to address that SV is implicit in the generation of a port specific stream ID since it cannot be hand-waved away as part of p2p? Thanks, Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-07-24 18:48 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-07-20 0:46 [PATCH V2] pci: quirk: Apply APM ACS quirk to XGene devices Feng Kan 2017-07-20 22:22 ` Alex Williamson 2017-07-21 20:20 ` Feng Kan 2017-07-24 2:06 ` Alex Williamson 2017-07-24 17:33 ` Feng Kan 2017-07-24 18:48 ` Alex Williamson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).