From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from el-out-1112.google.com (el-out-1112.google.com [209.85.162.176]) by ozlabs.org (Postfix) with ESMTP id 809FCDDF6F for ; Fri, 6 Mar 2009 16:38:28 +1100 (EST) Received: by el-out-1112.google.com with SMTP id s27so237994ele.3 for ; Thu, 05 Mar 2009 21:38:26 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: <64ac01180903052032i75792fa6x864f91c511b3f228@mail.gmail.com> Date: Thu, 5 Mar 2009 23:38:25 -0600 Message-ID: <64ac01180903052138k4226022cq840d497dccecd939@mail.gmail.com> Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited From: Ben Menchaca To: Liu Dave-R63238 Content-Type: multipart/alternative; boundary=002215046c8b9697d704646cb1c8 Cc: linuxppc-dev@ozlabs.org List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --002215046c8b9697d704646cb1c8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 wrote: > and what settings is DMA description bit 3? > > > -----Original Message----- > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces+daveliu = > freescale.com@ozlabs.org] > > On Behalf Of Liu Dave-R63238 > > Sent: Friday, March 06, 2009 1:22 PM > > To: Ben Menchaca; linuxppc-dev@ozlabs.org > > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > > PEX_RDMA_CTRL[SNOOP]? > > > > What is the freq settings? CORE/CSB bus. > > > > Thanks, Dave > > > > ________________________________ > > > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces+daveliu = > freescale.com@ozlabs.org] > > On Behalf Of Ben Menchaca > > Sent: Friday, March 06, 2009 12:33 PM > > To: linuxppc-dev@ozlabs.org > > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > > > > I am working on a Freescale 8314e design, and the > > embedded device is configured as a PCI-e endpoint running a > > 2.6.27-5 kernel. For context, we have written a kernel > > module which, among other things, uses the RDMA/WDMA engine > > in the PCI-e IP block. On the host side, these DMAs are > > coherent. However, on the embedded side, things are quite a > > bit less rosy; we must manually flush/invalidate cache lines > > for WDMA/RDMAs to occur successfully. After speaking with > > (several) FAEs at Freescale, we believe there is a > > configuration issue that is the cause, but we have yet to > > have anyone successfully point to it. > > > > Disabling the data cache altogether resolves the issue > > entirely, but of course, also completely tanks performance. > > As a temporary workaround, I would like to simply mark the > > pages (obtained currently via dma_alloc_coherent) involved as > > cache-inhibited. I have attempted to do this via some > > snippets remaining in fec.c (va_to_pte, uncache_pte to set > > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > > almost certainly braindead; va_to_pte is not a part of the > > 83xx source, as far as I can tell; 8xx only. > > > > A quick pointer in the correct direction for marking > > pages as cache-inhibited on a 2.6.27-5 kernel would be > > appreciated, or if my approach to a workaround is flawed, a > > pointer to the correct way would be great. > > > > Ben Menchaca > > > > > > _______________________________________________ > > Linuxppc-dev mailing list > > Linuxppc-dev@ozlabs.org > > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > > > > --002215046c8b9697d704646cb1c8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable 1.=A0 BAT2 in linux is set to = WIMG=3D0010, and covers=20 all 64M
2.=A0 PEX_DEVICE_CONTROL in PCI-E Config Space (0x54):=20 0x1020
3.=A0 PEX_xDMA_CTRL is set to 0x00000401 at the init= iation of the=20 DMA.
4.=A0 OWAR0 is set to 0xFFFFF005, so NSNP is 0.
5.=A0 The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared.= =A0

Core is 400MHz, and CSB is 133MHz.

- Ben

On Th= u, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrot= e:
and what settings= =A0is DMA description bit 3?

> -----Original Message-----
> From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-dev= -bounces+daveliu=3Df= reescale.com@ozlabs.org= ]
> =A0On Behalf Of Liu Dave-R6323= 8
> Sent: Friday, March 06, 2009 1:22 PM
> To: Ben Menchaca; linuxppc-= dev@ozlabs.org
> Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited
>
> Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and
> PEX_RDMA_CTRL[SNOOP]?
>
> What is the freq settings? CORE/CSB bus.
>
> Thanks, Dave
>
> ________________________________
>
> =A0 =A0 =A0 From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-dev= -bounces+daveliu=3Df= reescale.com@ozlabs.org= ]
> =A0On Behalf Of Ben Menchaca
> =A0 =A0 =A0 Sent: Friday, March 06, 2009 12:33 PM
> =A0 =A0 =A0 To: linuxppc-de= v@ozlabs.org
> =A0 =A0 =A0 Subject: 83xx: Marking or Allocating Pages as Cache-Inhibi= ted
>
>
> =A0 =A0 =A0 I am working on a Freescale 8314e design, and the
> embedded device is configured as a PCI-e endpoint running a
> 2.6.27-5 kernel. =A0For context, we have written a kernel
> module which, among other things, uses the RDMA/WDMA engine
> in the PCI-e IP block. =A0On the host side, these DMAs are
> coherent. =A0However, on the embedded side, things are quite a
> bit less rosy; we must manually flush/invalidate cache lines
> for WDMA/RDMAs to occur successfully. =A0After speaking with
> (several) FAEs at Freescale, we believe there is a
> configuration issue that is the cause, but we have yet to
> have anyone successfully point to it.
>
> =A0 =A0 =A0 Disabling the data cache altogether resolves the issue
> entirely, but of course, also completely tanks performance.
> As a temporary workaround, I would like to simply mark the
> pages (obtained currently via dma_alloc_coherent) involved as
> cache-inhibited. =A0I have attempted to do this via some
> snippets remaining in fec.c (va_to_pte, uncache_pte to set
> _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is
> almost certainly braindead; va_to_pte is not a part of the
> 83xx source, as far as I can tell; 8xx only.
>
> =A0 =A0 =A0 A quick pointer in the correct direction for marking
> pages as cache-inhibited on a 2.6.27-5 kernel would be
> appreciated, or if my approach to a workaround is flawed, a
> pointer to the correct way would be great.
>
> =A0 =A0 =A0 Ben Menchaca
>
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org=
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>
>

--002215046c8b9697d704646cb1c8--