New API for non cache coherent ppc cpu's

New API for non cache coherent ppc cpu's

[Armin Kuster]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]



To all NOT_COHERENT_CACHE users

About a month ago a new API made its way into the ppc,
consistent_sync_page.  For CONFIG_NOT_COHERENT_CACHE  processors,
requires
proper flushing of the page being used.  Please review and provide feed
back

-- armin




sample from patch

void consistent_sync_page(struct page *page, unsigned long offset,
size_t size, int direction)
{
        unsigned long start = (unsigned long)page->virtual;
        unsigned long end   = start + size;

        switch (direction) {
        case PCI_DMA_NONE:
                BUG();
        case PCI_DMA_FROMDEVICE:        /* invalidate only */
                invalidate_dcache_range(start, end);
                break;
        case PCI_DMA_TODEVICE:          /* writeback only */
                clean_dcache_range(start, end);
                break;
        case PCI_DMA_BIDIRECTIONAL:     /* writeback and invalidate */
                flush_dcache_range(start, end);
                break;
        }

 }

[-- Attachment #2: cachemap_1114.patch.gz --]
[-- Type: application/octet-stream, Size: 730 bytes --]

Re: New API for non cache coherent ppc cpu's

[Roman Zippel]

Hi,

On Tue, 20 Nov 2001, Armin Kuster wrote:

> About a month ago a new API made its way into the ppc,
> consistent_sync_page.

Another one?
We have now 3 APIs for this:
- cache_(push|clear): that's the old m68k API (and also used by APUS)
- dma_cache_(inv|wback|wback_inv): used by mips(64), parisc, sh
- consistent_sync_page: ppc

>  For CONFIG_NOT_COHERENT_CACHE  processors,
> requires
> proper flushing of the page being used.  Please review and provide feed
> back

Two comments:
- I'm not sure about the page argument, although I'd like to see it, the
problem is the drivers usually don't have a page pointer.
- I think it was never defined, what should be done if offset/size isn't
cache line aligned. That's especially a problem in the invalidate only
case. I'd prefer to make this illegal, as it's mostly not a problem for
drivers.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Armin Kuster]

Roman Zippel wrote:
>
> Hi,
>
> On Tue, 20 Nov 2001, Armin Kuster wrote:
>
> > About a month ago a new API made its way into the ppc,
> > consistent_sync_page.
>
> Another one?
> We have now 3 APIs for this:
> - cache_(push|clear): that's the old m68k API (and also used by APUS)
> - dma_cache_(inv|wback|wback_inv): used by mips(64), parisc, sh
> - consistent_sync_page: ppc
>
> >  For CONFIG_NOT_COHERENT_CACHE  processors,
> > requires
> > proper flushing of the page being used.  Please review and provide feed
> > back
>
> Two comments:
> - I'm not sure about the page argument, although I'd like to see it, the
> problem is the drivers usually don't have a page pointer.

example as it is used today.

mapping = pci_map_page(pdev,
			       virt_to_page(cmd->request_buffer),
			       ((unsigned long)cmd->request_buffer &
				~PAGE_MASK),
			       cmd->request_bufflen, dma_dir);

/*
 * pci_{map,unmap}_single_page maps a kernel page to a dma_addr_t.
identical
 * to pci_map_single, but takes a struct page instead of a virtual
address
 */
static inline dma_addr_t pci_map_page(struct pci_dev *hwdev, struct page
*page,
				      unsigned long offset, size_t size,
				      int direction)
{
	if (direction == PCI_DMA_NONE)
		BUG();
	consistent_sync_page(page, offset, size, direction);
	return (page - mem_map) * PAGE_SIZE + PCI_DRAM_OFFSET + offset;
}

> - I think it was never defined, what should be done if offset/size isn't
> cache line aligned. That's especially a problem in the invalidate only
> case. I'd prefer to make this illegal, as it's mostly not a problem for
> drivers.
>

wouldn't it also be a problem for consistent_sync?  we don't check for
offset/size are cache line aligned.

> bye, Roman



-- armin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Roman Zippel]

Hi,

Armin Kuster wrote:

> mapping = pci_map_page(pdev,
>                                virt_to_page(cmd->request_buffer),

This is the problem, virt_to_page doesn't work kmap'ed highmem pages.

> > - I think it was never defined, what should be done if offset/size isn't
> > cache line aligned. That's especially a problem in the invalidate only
> > case. I'd prefer to make this illegal, as it's mostly not a problem for
> > drivers.
>
> wouldn't it also be a problem for consistent_sync?  we don't check for
> offset/size are cache line aligned.

Yes, same problem.

bye, Roman

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Paul Mackerras]

Armin Kuster writes:

> To all NOT_COHERENT_CACHE users
>
> About a month ago a new API made its way into the ppc,
> consistent_sync_page.  For CONFIG_NOT_COHERENT_CACHE  processors,
> requires
> proper flushing of the page being used.  Please review and provide feed
> back

If we have to have a consistent_sync_page, it should be purely a local
function in our implementation of the official DMA mapping API - see
Documentation/DMA-mapping.txt.  Drivers should be using functions such
as pci_alloc_consistent, pci_map_single, pci_dma_sync_single,
pci_unmap_single, etc.  The implementation of those routines should do
the correct cache flushing - if it doesn't then we need to fix it.

If you're talking about non-PCI devices, use the pci DMA API but just
pass NULL for the dev (we need to make sure that will work ok on the
non-cache-coherent cpus).

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Dan Malek]

Paul Mackerras wrote:


> If we have to have a consistent_sync_page, it should be purely a local
> function in our implementation of the official DMA mapping API - see
> Documentation/DMA-mapping.txt.

That's exactly what it is.  We are not creating a new API, and it shouldn't
have been described as such.

> ....  Drivers should be using functions such
> as pci_alloc_consistent, pci_map_single, pci_dma_sync_single,
> pci_unmap_single, etc.  The implementation of those routines should do
> the correct cache flushing - if it doesn't then we need to fix it.

They do.  Recently a pci_sync_page was added for some reason (because
Linux doesn't have a real VM implementation I guess :-).  We just needed
to add the underlying function to support it.  I think it popped up in
one of the SCSI drivers.

> If you're talking about non-PCI devices, use the pci DMA API but just
> pass NULL for the dev (we need to make sure that will work ok on the
> non-cache-coherent cpus).

Ummm....I don't normally do this because using these functions requires
the CONFIG_PCI option, which isn't supported or has bad effects when
requested on processors that don't have PCI.  For non-PCI devices,
I call the consistent_* functions, which other architectures support
and do as well.  So, we kind of have a common interface here for non-PCI
devices as well.

I stole all of this cache coherency stuff from ARM and Sparc (or MIPS,
I don't remember) when I did it the first time.

Thanks.


	-- Dan


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Roman Zippel]

Hi,

Dan Malek wrote:

> For non-PCI devices,
> I call the consistent_* functions, which other architectures support
> and do as well.  So, we kind of have a common interface here for non-PCI
> devices as well.
>
> I stole all of this cache coherency stuff from ARM and Sparc (or MIPS,
> I don't remember) when I did it the first time.

The only other architecture which defines it is ARM?!
The only API which exists at least as dummys everywhere is
dma_cache_(inv|wback).

bye, Roman

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Paul Mackerras]

Dan Malek writes:

> That's exactly what it is.  We are not creating a new API, and it shouldn't
> have been described as such.

OK... Armin's message wasn't exactly clear... :)

> They do.  Recently a pci_sync_page was added for some reason (because

To go with pci_map_page and pci_unmap_page.  These give you the
ability to do DMA to highmem pages without using bounce buffers.

> > If you're talking about non-PCI devices, use the pci DMA API but just
> > pass NULL for the dev (we need to make sure that will work ok on the
> > non-cache-coherent cpus).
>
> Ummm....I don't normally do this because using these functions requires
> the CONFIG_PCI option, which isn't supported or has bad effects when
> requested on processors that don't have PCI.  For non-PCI devices,
> I call the consistent_* functions, which other architectures support
> and do as well.  So, we kind of have a common interface here for non-PCI
> devices as well.

My preference would be to make the one set of functions work
everywhere.  OTOH these non-PCI devices you're talking about are
presumably very specific to PPC embedded chips so maybe it doesn't
matter so much.

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Dan Malek]

Paul Mackerras wrote:


> To go with pci_map_page and pci_unmap_page.  These give you the
> ability to do DMA to highmem pages without using bounce buffers.

IIRC, there was a virt_to_bus in the code at one time, making it
unusable for highmem pages, which is why I couldn't understand
the appearance of the function.


> My preference would be to make the one set of functions work
> everywhere.

Mine, too.  I just wish this was a common concept across a variety
of our I/O interfaces.

> ....  OTOH these non-PCI devices you're talking about are
> presumably very specific to PPC embedded chips so maybe it doesn't
> matter so much.

That is always a challenge with the embedded parts.  It is often
very convenient to make the (promper, IMHO) assumption this software
isn't going to be used outside of these parts.

Thanks.


	-- Dan


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Roman Zippel]

Hi,

Paul Mackerras wrote:

> If we have to have a consistent_sync_page, it should be purely a local
> function in our implementation of the official DMA mapping API - see
> Documentation/DMA-mapping.txt.  Drivers should be using functions such
> as pci_alloc_consistent, pci_map_single, pci_dma_sync_single,
> pci_unmap_single, etc.  The implementation of those routines should do
> the correct cache flushing - if it doesn't then we need to fix it.

This document only describes DMA _mappings_, it doesn't say anything
about cache coherency.

> If you're talking about non-PCI devices, use the pci DMA API but just
> pass NULL for the dev (we need to make sure that will work ok on the
> non-cache-coherent cpus).

That's the other problem, "non-PCI" sounds like ISA there, what about
other buses?

bye, Roman

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New API for non cache coherent ppc cpu's

[Paul Mackerras]

Roman Zippel writes:

> This document only describes DMA _mappings_, it doesn't say anything
> about cache coherency.

It talks about the conditions under which the cpu and the DMA device
will see a consistent view of memory.  The section on the streaming
DMA mappings could be more explicit, but the idea is that between the
pci_map_{single,sg} and the corresponding pci_unmap_*, the device owns
the region and the cpu shouldn't touch it without first calling
pci_dma_sync_*.  (Hmmm, there is possibly a hole here - the API should
maybe require that you call a sync function both before and after
touching the memory.)  At other times the cpu owns the region and
should see a consistent view of that memory.  That implies some cache
flushing in pci_map_* and/or pci_unmap_* on non-cache-coherent
systems.

> That's the other problem, "non-PCI" sounds like ISA there, what about
> other buses?

That is implementation-dependent.  My view is that we should make this
API work for every kind of bus we have, if possible.

Note also that in 2.5 the struct pci_dev is going to transmogrify into
a "struct device" that will be used for I/O devices on any kind of
bus.

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/