Re: PCI Memory mapping

[Marc Leeman <marc.leeman@barco.com>]

> actually, if you have 32-byte cache lines on your cpu, and you have,
> umm, something maybe 4-way set associative (I've forgotten how it
> works) that might explain it. I don't know what cpu's have what cache
> sizes.

This was also one of the paths we were considering, but the following
tests have been performed in order to validate or invalidate this
assumption:
1. I was under the impression that 'consistent_alloc' (which I
activited by defining CONFIG_NOT_COHERENT_CACHE in the kernel
configuration for our board) assured that the pages were not using
cache, at least the following lines lead me to this:

[malm@bpscltpd linux-2.4.17_bl21]$ vim ./arch/ppc/mm/cachemap.c

<...>
        /*
         * we need to ensure that there are no cachelines in use,
         * or worse dirty in this area.
         */
        invalidate_dcache_range(page, page + size);
<...>

and

<...>
        flags = _PAGE_KERNEL | _PAGE_NO_CACHE;

        err = 0;
        for (i = 0; i < size && err == 0; i += PAGE_SIZE)
                err = map_page(va+i, pa+i, flags);
<...>

As told before, no change with or without this define enabled that uses
consistent_alloc.

2. Next I tried to invalidate the cache the naive way:
I refilled the used buffer with 0xCA in userspace and copied it back to
kernelspace (in the PCI mappded buffer). Next I transferred the next
'useful' data from user to kernel space. Still completely the same: the
data showing up (corrupted) on the DSP side was the 'old' lines of the
buffer last transferred (i.e. the buffer before bringing 0xCA into
kernel space).

3. I took the previous point one step further: I filled the kernel
buffer with 0xCA before copying in the userbuffer. Still the same old
data is showing up on the DSP side. Still, I understand that none of
these two techniques assure purging the cache.

4. I added sync's and the like which should sync cache and mem, still no
change.

Apparently, the closest I can come to pinpointing the location of the
problem at the moment is that 'something' must be happening at the point
where the DSP starts reading from the PPC memory. When no such read is
performed, the problem does not occur. The other way around, the DSP can
write into the PPC memory without corrupting data.

Maybe reading of the DSP over PCI and the resulting DMA setup causes a
[hardware|software] timing problem with purging the cache (which should
not be used for these pages to start with (see [1]), or the DSP ack's
too early on and reading these 97 bytes after the DSP ACK causes some
hardware lock to be released, giving the DSP just enough time to
correctly fetch the data before the PPC starts pumping in new, ...

  marc.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/