pci overmapping

pci overmapping

[Yoder Stuart-B08248]

Kumar,

In fsl_pci.c there is a change you made a while back:
  "powerpc/fsl: Setup PCI inbound window based on actual amount of memory"

...and there is this comment in the code:

    /* PCIe can overmap inbound & outbound since RX & TX are separated */
    if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {

You are implying that PCIe can overmap and PCI can't.  Why is
that?   (I'm assuming that 'overmap' means that inbound window
can extend beyond the end of ram.)

We are facing an issue for a non-PCI express controller (i.e. without
PCI_CAP_ID_EXP) emulated by QEMU.  In our case we need the ATMU
window to extend beyond the end of ram, since the window size has
to be a power of 2.

Thanks,
Stuart

Re: pci overmapping

[Scott Wood]

On 05/02/2013 12:05:42 PM, Yoder Stuart-B08248 wrote:
> Kumar,
>=20
> In fsl_pci.c there is a change you made a while back:
>   "powerpc/fsl: Setup PCI inbound window based on actual amount of =20
> memory"
>=20
> ...and there is this comment in the code:
>=20
>     /* PCIe can overmap inbound & outbound since RX & TX are =20
> separated */
>     if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {
>=20
> You are implying that PCIe can overmap and PCI can't.  Why is
> that?   (I'm assuming that 'overmap' means that inbound window
> can extend beyond the end of ram.)

Shouldn't the concern be whether we're overlapping outbound, not merely =20
whether we go beyond the end of RAM?

And couldn't inbound/outbound overlap be an issue even on PCIe, if =20
there's a PCI bridge underneath it?

-Scott=

RE: pci overmapping

[Sethi Varun-B16395]

> -----Original Message-----
> From: Wood Scott-B07421
> Sent: Thursday, May 02, 2013 10:44 PM
> To: Yoder Stuart-B08248
> Cc: galak@kernel.crashing.org; Sethi Varun-B16395; linuxppc-
> dev@lists.ozlabs.org
> Subject: Re: pci overmapping
>=20
> On 05/02/2013 12:05:42 PM, Yoder Stuart-B08248 wrote:
> > Kumar,
> >
> > In fsl_pci.c there is a change you made a while back:
> >   "powerpc/fsl: Setup PCI inbound window based on actual amount of
> > memory"
> >
> > ...and there is this comment in the code:
> >
> >     /* PCIe can overmap inbound & outbound since RX & TX are separated
> > */
> >     if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {
> >
> > You are implying that PCIe can overmap and PCI can't.  Why is
> > that?   (I'm assuming that 'overmap' means that inbound window
> > can extend beyond the end of ram.)
>=20
> Shouldn't the concern be whether we're overlapping outbound, not merely
> whether we go beyond the end of RAM?
>=20
> And couldn't inbound/outbound overlap be an issue even on PCIe, if
> there's a PCI bridge underneath it?
I believe that the overlap problem would be avoided in case of 36 bit physi=
cal support (outbound window address would be high order 36 bit value), rig=
ht?

-Varun

Re: pci overmapping

[Scott Wood]

On 05/02/2013 01:09:53 PM, Sethi Varun-B16395 wrote:
>=20
>=20
> > -----Original Message-----
> > From: Wood Scott-B07421
> > Sent: Thursday, May 02, 2013 10:44 PM
> > To: Yoder Stuart-B08248
> > Cc: galak@kernel.crashing.org; Sethi Varun-B16395; linuxppc-
> > dev@lists.ozlabs.org
> > Subject: Re: pci overmapping
> >
> > On 05/02/2013 12:05:42 PM, Yoder Stuart-B08248 wrote:
> > > Kumar,
> > >
> > > In fsl_pci.c there is a change you made a while back:
> > >   "powerpc/fsl: Setup PCI inbound window based on actual amount of
> > > memory"
> > >
> > > ...and there is this comment in the code:
> > >
> > >     /* PCIe can overmap inbound & outbound since RX & TX are =20
> separated
> > > */
> > >     if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {
> > >
> > > You are implying that PCIe can overmap and PCI can't.  Why is
> > > that?   (I'm assuming that 'overmap' means that inbound window
> > > can extend beyond the end of ram.)
> >
> > Shouldn't the concern be whether we're overlapping outbound, not =20
> merely
> > whether we go beyond the end of RAM?
> >
> > And couldn't inbound/outbound overlap be an issue even on PCIe, if
> > there's a PCI bridge underneath it?
> I believe that the overlap problem would be avoided in case of 36 bit =20
> physical support (outbound window address would be high order 36 bit =20
> value), right?

Not necessarily -- it depends on what the bus addresses are, not the =20
host addresses.

-Scott=

RE: pci overmapping

[Sethi Varun-B16395]

> -----Original Message-----
> From: Wood Scott-B07421
> Sent: Thursday, May 02, 2013 11:50 PM
> To: Sethi Varun-B16395
> Cc: Wood Scott-B07421; Yoder Stuart-B08248; galak@kernel.crashing.org;
> linuxppc-dev@lists.ozlabs.org
> Subject: Re: pci overmapping
>=20
> On 05/02/2013 01:09:53 PM, Sethi Varun-B16395 wrote:
> >
> >
> > > -----Original Message-----
> > > From: Wood Scott-B07421
> > > Sent: Thursday, May 02, 2013 10:44 PM
> > > To: Yoder Stuart-B08248
> > > Cc: galak@kernel.crashing.org; Sethi Varun-B16395; linuxppc-
> > > dev@lists.ozlabs.org
> > > Subject: Re: pci overmapping
> > >
> > > On 05/02/2013 12:05:42 PM, Yoder Stuart-B08248 wrote:
> > > > Kumar,
> > > >
> > > > In fsl_pci.c there is a change you made a while back:
> > > >   "powerpc/fsl: Setup PCI inbound window based on actual amount of
> > > > memory"
> > > >
> > > > ...and there is this comment in the code:
> > > >
> > > >     /* PCIe can overmap inbound & outbound since RX & TX are
> > separated
> > > > */
> > > >     if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {
> > > >
> > > > You are implying that PCIe can overmap and PCI can't.  Why is
> > > > that?   (I'm assuming that 'overmap' means that inbound window
> > > > can extend beyond the end of ram.)
> > >
> > > Shouldn't the concern be whether we're overlapping outbound, not
> > merely
> > > whether we go beyond the end of RAM?
> > >
> > > And couldn't inbound/outbound overlap be an issue even on PCIe, if
> > > there's a PCI bridge underneath it?
> > I believe that the overlap problem would be avoided in case of 36 bit
> > physical support (outbound window address would be high order 36 bit
> > value), right?
>=20
> Not necessarily -- it depends on what the bus addresses are, not the host
> addresses.
The fsl_pci code already checks for the overlap condition. The inbound wind=
ow range is set to the minimum of memory end address or the start of pci ou=
tbound address. In our case, the guest memory end address (host physical ad=
dress derived from the huge tlb page) is less than the pci outbound start a=
ddress. But, it could be the other way round as well. In both cases if the =
inbound memory can't map the end of RAM address (for the guest), the fsl_pc=
i driver would enable swiotlb.

For the first case where end of memory is < start of pci outbound address, =
we can avoid enabling swiotlb (when end address is not a power of two) by o=
ver committing the inbound address window.

-Varun