Re: [PATCH 8/8] Cell IOMMU fixed mapping support

[Michael Ellerman <michael@ellerman.id.au>]

[-- Attachment #1: Type: text/plain, Size: 3030 bytes --]

On Tue, 2008-01-29 at 15:36 -0600, Olof Johansson wrote:
> On Wed, Jan 30, 2008 at 08:18:15AM +1100, Benjamin Herrenschmidt wrote:
> > 
> > On Wed, 2008-01-30 at 02:13 +1100, Michael Ellerman wrote:
> > > On Tue, 2008-01-29 at 09:15 -0600, Olof Johansson wrote:
> > > > On Wed, Jan 30, 2008 at 01:14:03AM +1100, Michael Ellerman wrote:
> > > > 
> > > > > For example a machine with 4GB of memory would end up with the normal
> > > > > IOMMU window from 0-2GB and the fixed mapping window from 2GB to 6GB. In
> > > > > this case a 64-bit device wishing to DMA to 1GB would be told to DMA to
> > > > > 3GB, plus any offset required by firmware. The firmware offset is encoded
> > > > > in the "dma-ranges" property.
> > > > 
> > > > Shouldn't the fixed mapping be between 4G and 8G (and the offset for 1G
> > > > is at 5G), to account for the MMIO range at 2-4G?
> > > 
> > > I don't think so, ie. it works setup like that, but I'm not entirely
> > > sure why. Presumably the 2-4GB for MMIO is only for cycles heading out
> > > of the CPU.
> > 
> > No no no... it's because on the PCI segment, it's all offset up
> > remember ?
> > 
> > Basically, the PCI host bridge on these has 2 interesting windows for
> > us:
> > 
> > 0....2G			-> This goes up to memory @0 (via a couple of 
> >                            layers)
> > 
> > 0x80*....0xF*		-> This goes untranslated to the PLB5 which
> >   			   drops the top bits and does some other 
> >                            manipulations, which allows to access, among
> >                            others the full 32GB of the cell inbound 
> >                            range.
> > 
> > The MMIO region of 2...4G is on the PCI (outbound from the Cell is yet
> > another range of addresses with different constraints but that ends up
> > generating cycles between 2 and 4G on the PCI segment).
> > 
> > If we had set the direct mapped region so that it uses 2G...N on PCI, we
> > would indeed be toast. But instead, the addresses for direct DMA that we
> > hand out to devices are in the 0x80* region and go hit the cell
> > directly, they never match MMIO.
> 
> Yeah, ok. That makes more sense. Thanks for the clarification.

Right, that's the firmware offset I mentioned in the changelog - 2am is
not a good time to think about these things.

> Michael, btw, I wonder if it would make sense to duplicate the patch
> description at the top of the file as well, since it'll be lost in the
> change log for people who don't go back and read history, and having
> the intentions documented in the file could be a good idea.

Yeah that probably makes sense. I've got most of the fixed mapping code
in a block, so I'll put a comment above that section. New patch coming.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]