PCI MMIO flushing, write-combining etc

PCI MMIO flushing, write-combining etc

[Krzysztof Halasa]

Hi,

Just a few simple questions for you PCI experts (please correct any wrong
assumptions):


Imagine we have a PCI card with 2 MMIO regions: prefetchable PR1 and
non-prefetchable NPR2 (and we are not limited to Pentium-class machines).

I understand writes to PR1 can be reordered, merged, and delayed.
What should I do to flush the write buffers? I understand reading from
PR1 would do. Would reading from NPR2 flush PR1 write buffers?
Would writing to NPR2 flush them?

Now NPR2, the non-prefetchable MMIO region.
Is it possible that the writes there are reordered, merged and/or
delayed (delayed = not making it to the PCI device when the writel()
completes)?


We have ioremap() and ioremap_nocache(). What is the exact difference
between them? Would the ioremap_nocache() disable all A) read- and
B) write-caching on a) prefetchable MMIO b) non-prefetchable MMIO ?

Would the ioremap() enable A) read- and A) write-caching on
a) prefetchable MMIO b) non-prefetchable MMIO ?


Thank you.
-- 
Krzysztof Halasa
Network Administrator

Re: PCI MMIO flushing, write-combining etc

[Alan Cox]

On Fri, 2002-08-16 at 00:45, Krzysztof Halasa wrote:
> I understand writes to PR1 can be reordered, merged, and delayed.
> What should I do to flush the write buffers? I understand reading from
> PR1 would do. Would reading from NPR2 flush PR1 write buffers?
> Would writing to NPR2 flush them?

That one I can't actually remember. 

> Now NPR2, the non-prefetchable MMIO region.
> Is it possible that the writes there are reordered, merged and/or
> delayed (delayed = not making it to the PCI device when the writel()
> completes)?

All PCI writes are posted. Think of PCI as messages otherwise you'll go
slowly insane debugging code. If you want to know your write completed
you need to read, when the read returns both have completed

> We have ioremap() and ioremap_nocache(). What is the exact difference
> between them? Would the ioremap_nocache() disable all A) read- and
> B) write-caching on a) prefetchable MMIO b) non-prefetchable MMIO ?

They make no difference

Re: PCI MMIO flushing, write-combining etc

[Krzysztof Halasa]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > I understand writes to PR1 can be reordered, merged, and delayed.
> > What should I do to flush the write buffers? I understand reading from
> > PR1 would do. Would reading from NPR2 flush PR1 write buffers?
> > Would writing to NPR2 flush them?
> 
> That one I can't actually remember.

Ok. What PCI spec 2.1 says is, basically, that we don't need to worry about
such things. Writes can't be reordered - all the reads and writes are
in CPU (or any other PCI master) order, exactly as on ISA.
Writes can be merged on prefetchable region (so we don't necessarily want
to mark I/O MMIO as prefetchable, if the hardware doesn't like merging).

All writes can be posted, and they are flushed before a read initiated
by the same master (i.e. CPU) reaches the same PCI target (so it's enough
to readl() any region, either I/O or MMIO). Looks like we only need to
flush posted writes when there are specific timing requirements (something
like writel(reset); sleep 100 ns, writel(no_reset) - or when we want to be
sure that, say, card interrupts are off when we do something critical
elsewhere).
-- 
Krzysztof Halasa
Network Administrator