From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-gx0-f174.google.com (mail-gx0-f174.google.com [209.85.217.174]) by ozlabs.org (Postfix) with ESMTP id 50991DDF9B for ; Sat, 7 Mar 2009 03:12:14 +1100 (EST) Received: by gxk22 with SMTP id 22so1300629gxk.9 for ; Fri, 06 Mar 2009 08:12:13 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: <64ac01180903052032i75792fa6x864f91c511b3f228@mail.gmail.com> <64ac01180903052138k4226022cq840d497dccecd939@mail.gmail.com> <64ac01180903052210r2aee7246g50fd0656c491a40b@mail.gmail.com> Date: Fri, 6 Mar 2009 10:12:12 -0600 Message-ID: <64ac01180903060812n1f355207lb3fa6de3ed17ae41@mail.gmail.com> Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited From: Ben Menchaca To: Liu Dave-R63238 Content-Type: multipart/alternative; boundary=00221532cba03342fc0464758ca0 Cc: linuxppc-dev@ozlabs.org List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --00221532cba03342fc0464758ca0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Testing now...it looks like it (almost) works, though! Why does setting no-snoop cause snooping to work? More on the effect on setting that bit in a few minutes...need more testing. ACR is 0x00030300. - Ben On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 wrote: > Did you enable the descriptor bit 3 to have a try? > > ------------------------------ > *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] > *Sent:* Friday, March 06, 2009 2:10 PM > > *To:* Liu Dave-R63238 > *Cc:* linuxppc-dev@ozlabs.org > *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited > > I can look at ACR morning...although I can say with a fair amount of > certainty that I have not changed it from the POR value. > > I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this > seems a bit counterintuitive to me. > > What is the hope regarding these two? Some combination I am not seeing? > > > On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 wrote: > >> what is the value of ACR register? >> >> ------------------------------ >> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >> *Sent:* Friday, March 06, 2009 1:38 PM >> *To:* Liu Dave-R63238 >> *Cc:* linuxppc-dev@ozlabs.org >> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >> >> 1. BAT2 in linux is set to WIMG=0010, and covers all 64M >> 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 >> 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. >> 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. >> 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore >> the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. >> >> Core is 400MHz, and CSB is 133MHz. >> >> - Ben >> >> On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 wrote: >> >>> and what settings is DMA description bit 3? >>> >>> > -----Original Message----- >>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>> > [mailto:linuxppc-dev-bounces+daveliu = >>> freescale.com@ozlabs.org] >>> > On Behalf Of Liu Dave-R63238 >>> > Sent: Friday, March 06, 2009 1:22 PM >>> > To: Ben Menchaca; linuxppc-dev@ozlabs.org >>> > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> > >>> > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and >>> > PEX_RDMA_CTRL[SNOOP]? >>> > >>> > What is the freq settings? CORE/CSB bus. >>> > >>> > Thanks, Dave >>> > >>> > ________________________________ >>> > >>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>> > [mailto:linuxppc-dev-bounces+daveliu = >>> freescale.com@ozlabs.org] >>> > On Behalf Of Ben Menchaca >>> > Sent: Friday, March 06, 2009 12:33 PM >>> > To: linuxppc-dev@ozlabs.org >>> > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> > >>> > >>> > I am working on a Freescale 8314e design, and the >>> > embedded device is configured as a PCI-e endpoint running a >>> > 2.6.27-5 kernel. For context, we have written a kernel >>> > module which, among other things, uses the RDMA/WDMA engine >>> > in the PCI-e IP block. On the host side, these DMAs are >>> > coherent. However, on the embedded side, things are quite a >>> > bit less rosy; we must manually flush/invalidate cache lines >>> > for WDMA/RDMAs to occur successfully. After speaking with >>> > (several) FAEs at Freescale, we believe there is a >>> > configuration issue that is the cause, but we have yet to >>> > have anyone successfully point to it. >>> > >>> > Disabling the data cache altogether resolves the issue >>> > entirely, but of course, also completely tanks performance. >>> > As a temporary workaround, I would like to simply mark the >>> > pages (obtained currently via dma_alloc_coherent) involved as >>> > cache-inhibited. I have attempted to do this via some >>> > snippets remaining in fec.c (va_to_pte, uncache_pte to set >>> > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is >>> > almost certainly braindead; va_to_pte is not a part of the >>> > 83xx source, as far as I can tell; 8xx only. >>> > >>> > A quick pointer in the correct direction for marking >>> > pages as cache-inhibited on a 2.6.27-5 kernel would be >>> > appreciated, or if my approach to a workaround is flawed, a >>> > pointer to the correct way would be great. >>> > >>> > Ben Menchaca >>> > >>> > >>> > _______________________________________________ >>> > Linuxppc-dev mailing list >>> > Linuxppc-dev@ozlabs.org >>> > https://ozlabs.org/mailman/listinfo/linuxppc-dev >>> > >>> > >>> >> >> > --00221532cba03342fc0464758ca0 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Testing now...it looks like it (almost) works, though!=A0 Why does setting = no-snoop cause snooping to work?=A0 More on the effect on setting that bit = in a few minutes...need more testing.
=A0
ACR is 0x00030300.

-= Ben

On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R6= 3238 <DaveLiu= @freescale.com> wrote:
Did you enable the descriptor bit 3 to have a=20 try?

From: Ben Mench= aca=20 [mailto:ben.m= enchaca@gmail.com]
Sent: Friday, March 06, 2009 2:10= =20 PM

To: Liu Dave-R63238
= Cc:=20 linuxppc-dev= @ozlabs.org
Subject: Re: 83xx: Marking or Allocating=20 Pages as Cache-Inhibited

=
I can look at ACR morning...although I can say with a fair amo= unt=20 of certainty that I have not changed it from the POR value.

I will= try=20 enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this seems a = bit=20 counterintuitive to me.

What is the hope regarding these two?=A0= =20 Some combination I am not seeing?


On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R632= 38 <DaveLiu@freescale.com> wrote:
what is=20 the value of ACR register?

From: Ben Menchaca [mailto:ben.menchaca@gmai= l.com]
Sent: Friday, March=20 06, 2009 1:38 PM
To: Liu Dave-R63238
Cc: linuxppc-dev@ozlabs.o= rg
Subject: Re: 83xx:=20 Marking or Allocating Pages as Cache-Inhibited

1.=A0 BAT2 in= linux is set=20 to WIMG=3D0010, and covers all 64M
2.=A0 PEX_DEVICE_CONTROL in PCI= -E=20 Config Space (0x54): 0x1020
3.=A0 PEX_xDMA_CTRL is set to=20 0x00000401 at the initiation of the DMA.
4.=A0 OWAR0 = is=20 set to 0xFFFFF005, so NSNP is 0.
5.=A0 The DMA descriptor (randoml= y=20 chosen when I hit a trigger...just ignore the size...) contains 0002A= FF3=20 at offset 0, so nosnoops are=20 cleared.=A0

Core is=20 400MHz, and CSB is 133MHz.

- Ben

On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-= R63238=20 <DaveLiu@freescale.com> wrote:
and=20 what settings =A0is DMA description bit 3?

> -----Original Message-----
> From:=20 linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org]
> =A0On Behalf Of Liu Dave-R63238
> Sent: Friday, Mar= ch=20 06, 2009 1:22 PM
> To: Ben Menchaca; linuxppc-dev@ozlabs.org
> Su= bject: RE: 83xx:=20 Marking or Allocating Pages as Cache-Inhibited
>
> Did = you=20 enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and
>=20 PEX_RDMA_CTRL[SNOOP]?
>
> What is the freq settings?=20 CORE/CSB bus.
>
> Thanks, Dave
>
>=20 ________________________________
>
> =A0 =A0 =A0=20 From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org]
> =A0On=20 Behalf Of Ben Menchaca
> =A0 =A0 =A0 Sent: Friday, March=20 06, 2009 12:33 PM
> =A0 =A0 =A0 To: linuxppc-dev@ozlabs.org
> =A0= =A0 =A0=20 Subject: 83xx: Marking or Allocating Pages as=20 Cache-Inhibited
>
>
> =A0 =A0 =A0 I am=20 working on a Freescale 8314e design, and the
> embedded devic= e is=20 configured as a PCI-e endpoint running a
> 2.6.27-5 kernel.= =20 =A0For context, we have written a kernel
> module which, amon= g=20 other things, uses the RDMA/WDMA engine
> in the PCI-e IP blo= ck.=20 =A0On the host side, these DMAs are
> coherent. =A0However,= =20 on the embedded side, things are quite a
> bit less rosy; we = must=20 manually flush/invalidate cache lines
> for WDMA/RDMAs to occ= ur=20 successfully. =A0After speaking with
> (several) FAEs at=20 Freescale, we believe there is a
> configuration issue that i= s the=20 cause, but we have yet to
> have anyone successfully point to= =20 it.
>
> =A0 =A0 =A0 Disabling the data cache=20 altogether resolves the issue
> entirely, but of course, also= =20 completely tanks performance.
> As a temporary workaround, I = would=20 like to simply mark the
> pages (obtained currently via=20 dma_alloc_coherent) involved as
> cache-inhibited. =A0I have= =20 attempted to do this via some
> snippets remaining in fec.c= =20 (va_to_pte, uncache_pte to set
> _PAGE_NO_CACHE, flush_tlb_pa= ge,=20 then unmap_pte), but this is
> almost certainly braindead;=20 va_to_pte is not a part of the
> 83xx source, as far as I can= =20 tell; 8xx only.
>
> =A0 =A0 =A0 A quick pointer in=20 the correct direction for marking
> pages as cache-inhibited = on a=20 2.6.27-5 kernel would be
> appreciated, or if my approach to = a=20 workaround is flawed, a
> pointer to the correct way would be= =20 great.
>
> =A0 =A0 =A0 Ben=20 Menchaca
>
>
>=20 _______________________________________________
> Linuxppc-de= v=20 mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/m= ailman/listinfo/linuxppc-dev
>
>



--00221532cba03342fc0464758ca0--