* 83xx: Marking or Allocating Pages as Cache-Inhibited @ 2009-03-06 4:32 Ben Menchaca 2009-03-06 5:22 ` Liu Dave-R63238 0 siblings, 1 reply; 17+ messages in thread From: Ben Menchaca @ 2009-03-06 4:32 UTC (permalink / raw) To: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 1352 bytes --] I am working on a Freescale 8314e design, and the embedded device is configured as a PCI-e endpoint running a 2.6.27-5 kernel. For context, we have written a kernel module which, among other things, uses the RDMA/WDMA engine in the PCI-e IP block. On the host side, these DMAs are coherent. However, on the embedded side, things are quite a bit less rosy; we must manually flush/invalidate cache lines for WDMA/RDMAs to occur successfully. After speaking with (several) FAEs at Freescale, we believe there is a configuration issue that is the cause, but we have yet to have anyone successfully point to it. Disabling the data cache altogether resolves the issue entirely, but of course, also completely tanks performance. As a temporary workaround, I would like to simply mark the pages (obtained currently via dma_alloc_coherent) involved as cache-inhibited. I have attempted to do this via some snippets remaining in fec.c (va_to_pte, uncache_pte to set _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is almost certainly braindead; va_to_pte is not a part of the 83xx source, as far as I can tell; 8xx only. A quick pointer in the correct direction for marking pages as cache-inhibited on a 2.6.27-5 kernel would be appreciated, or if my approach to a workaround is flawed, a pointer to the correct way would be great. Ben Menchaca [-- Attachment #2: Type: text/html, Size: 1394 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 4:32 83xx: Marking or Allocating Pages as Cache-Inhibited Ben Menchaca @ 2009-03-06 5:22 ` Liu Dave-R63238 2009-03-06 5:27 ` Liu Dave-R63238 0 siblings, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-06 5:22 UTC (permalink / raw) To: Ben Menchaca, linuxppc-dev Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and PEX_RDMA_CTRL[SNOOP]? What is the freq settings? CORE/CSB bus. Thanks, Dave ________________________________ From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org [mailto:linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org] On = Behalf Of Ben Menchaca Sent: Friday, March 06, 2009 12:33 PM To: linuxppc-dev@ozlabs.org Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited =09 =09 I am working on a Freescale 8314e design, and the embedded device is configured as a PCI-e endpoint running a 2.6.27-5 kernel. For context, we have written a kernel module which, among other things, uses the RDMA/WDMA engine in the PCI-e IP block. On the host side, these DMAs are coherent. However, on the embedded side, things are quite a bit less rosy; we must manually flush/invalidate cache lines for WDMA/RDMAs to occur successfully. After speaking with (several) FAEs at Freescale, we believe there is a configuration issue that is the cause, but we have yet to have anyone successfully point to it. =20 =09 Disabling the data cache altogether resolves the issue entirely, but of course, also completely tanks performance. As a temporary workaround, I would like to simply mark the pages (obtained currently via dma_alloc_coherent) involved as cache-inhibited. I have attempted to do this via some snippets remaining in fec.c (va_to_pte, uncache_pte to set _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is almost certainly braindead; va_to_pte is not a part of the 83xx source, as far as I can tell; 8xx only. =09 A quick pointer in the correct direction for marking pages as cache-inhibited on a 2.6.27-5 kernel would be appreciated, or if my approach to a workaround is flawed, a pointer to the correct way would be great. =09 Ben Menchaca =09 ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:22 ` Liu Dave-R63238 @ 2009-03-06 5:27 ` Liu Dave-R63238 2009-03-06 5:38 ` Ben Menchaca 0 siblings, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-06 5:27 UTC (permalink / raw) To: Liu Dave-R63238, Ben Menchaca, linuxppc-dev and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org=20 > [mailto:linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >=20 > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and=20 > PEX_RDMA_CTRL[SNOOP]? >=20 > What is the freq settings? CORE/CSB bus. >=20 > Thanks, Dave >=20 > ________________________________ >=20 > From: linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu=3Dfreescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > =09 > =09 > I am working on a Freescale 8314e design, and the=20 > embedded device is configured as a PCI-e endpoint running a=20 > 2.6.27-5 kernel. For context, we have written a kernel=20 > module which, among other things, uses the RDMA/WDMA engine=20 > in the PCI-e IP block. On the host side, these DMAs are=20 > coherent. However, on the embedded side, things are quite a=20 > bit less rosy; we must manually flush/invalidate cache lines=20 > for WDMA/RDMAs to occur successfully. After speaking with=20 > (several) FAEs at Freescale, we believe there is a=20 > configuration issue that is the cause, but we have yet to=20 > have anyone successfully point to it. =20 > =09 > Disabling the data cache altogether resolves the issue=20 > entirely, but of course, also completely tanks performance. =20 > As a temporary workaround, I would like to simply mark the=20 > pages (obtained currently via dma_alloc_coherent) involved as=20 > cache-inhibited. I have attempted to do this via some=20 > snippets remaining in fec.c (va_to_pte, uncache_pte to set=20 > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is=20 > almost certainly braindead; va_to_pte is not a part of the=20 > 83xx source, as far as I can tell; 8xx only. > =09 > A quick pointer in the correct direction for marking=20 > pages as cache-inhibited on a 2.6.27-5 kernel would be=20 > appreciated, or if my approach to a workaround is flawed, a=20 > pointer to the correct way would be great. > =09 > Ben Menchaca > =09 >=20 > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev >=20 >=20 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:27 ` Liu Dave-R63238 @ 2009-03-06 5:38 ` Ben Menchaca 2009-03-06 5:40 ` Liu Dave-R63238 2009-03-06 5:49 ` Liu Dave-R63238 0 siblings, 2 replies; 17+ messages in thread From: Ben Menchaca @ 2009-03-06 5:38 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 3182 bytes --] 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > and what settings is DMA description bit 3? > > > -----Original Message----- > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= > freescale.com@ozlabs.org] > > On Behalf Of Liu Dave-R63238 > > Sent: Friday, March 06, 2009 1:22 PM > > To: Ben Menchaca; linuxppc-dev@ozlabs.org > > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > > PEX_RDMA_CTRL[SNOOP]? > > > > What is the freq settings? CORE/CSB bus. > > > > Thanks, Dave > > > > ________________________________ > > > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= > freescale.com@ozlabs.org] > > On Behalf Of Ben Menchaca > > Sent: Friday, March 06, 2009 12:33 PM > > To: linuxppc-dev@ozlabs.org > > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > > > > I am working on a Freescale 8314e design, and the > > embedded device is configured as a PCI-e endpoint running a > > 2.6.27-5 kernel. For context, we have written a kernel > > module which, among other things, uses the RDMA/WDMA engine > > in the PCI-e IP block. On the host side, these DMAs are > > coherent. However, on the embedded side, things are quite a > > bit less rosy; we must manually flush/invalidate cache lines > > for WDMA/RDMAs to occur successfully. After speaking with > > (several) FAEs at Freescale, we believe there is a > > configuration issue that is the cause, but we have yet to > > have anyone successfully point to it. > > > > Disabling the data cache altogether resolves the issue > > entirely, but of course, also completely tanks performance. > > As a temporary workaround, I would like to simply mark the > > pages (obtained currently via dma_alloc_coherent) involved as > > cache-inhibited. I have attempted to do this via some > > snippets remaining in fec.c (va_to_pte, uncache_pte to set > > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > > almost certainly braindead; va_to_pte is not a part of the > > 83xx source, as far as I can tell; 8xx only. > > > > A quick pointer in the correct direction for marking > > pages as cache-inhibited on a 2.6.27-5 kernel would be > > appreciated, or if my approach to a workaround is flawed, a > > pointer to the correct way would be great. > > > > Ben Menchaca > > > > > > _______________________________________________ > > Linuxppc-dev mailing list > > Linuxppc-dev@ozlabs.org > > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > > > > [-- Attachment #2: Type: text/html, Size: 5138 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:38 ` Ben Menchaca @ 2009-03-06 5:40 ` Liu Dave-R63238 2009-03-06 6:10 ` Ben Menchaca 2009-03-06 5:49 ` Liu Dave-R63238 1 sibling, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-06 5:40 UTC (permalink / raw) To: Ben Menchaca; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 3629 bytes --] what is the value of ACR register? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 1:38 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > PEX_RDMA_CTRL[SNOOP]? > > What is the freq settings? CORE/CSB bus. > > Thanks, Dave > > ________________________________ > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > I am working on a Freescale 8314e design, and the > embedded device is configured as a PCI-e endpoint running a > 2.6.27-5 kernel. For context, we have written a kernel > module which, among other things, uses the RDMA/WDMA engine > in the PCI-e IP block. On the host side, these DMAs are > coherent. However, on the embedded side, things are quite a > bit less rosy; we must manually flush/invalidate cache lines > for WDMA/RDMAs to occur successfully. After speaking with > (several) FAEs at Freescale, we believe there is a > configuration issue that is the cause, but we have yet to > have anyone successfully point to it. > > Disabling the data cache altogether resolves the issue > entirely, but of course, also completely tanks performance. > As a temporary workaround, I would like to simply mark the > pages (obtained currently via dma_alloc_coherent) involved as > cache-inhibited. I have attempted to do this via some > snippets remaining in fec.c (va_to_pte, uncache_pte to set > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > almost certainly braindead; va_to_pte is not a part of the > 83xx source, as far as I can tell; 8xx only. > > A quick pointer in the correct direction for marking > pages as cache-inhibited on a 2.6.27-5 kernel would be > appreciated, or if my approach to a workaround is flawed, a > pointer to the correct way would be great. > > Ben Menchaca > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > [-- Attachment #2: Type: text/html, Size: 6497 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:40 ` Liu Dave-R63238 @ 2009-03-06 6:10 ` Ben Menchaca 2009-03-06 6:30 ` Liu Dave-R63238 0 siblings, 1 reply; 17+ messages in thread From: Ben Menchaca @ 2009-03-06 6:10 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 3969 bytes --] I can look at ACR morning...although I can say with a fair amount of certainty that I have not changed it from the POR value. I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this seems a bit counterintuitive to me. What is the hope regarding these two? Some combination I am not seeing? On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > what is the value of ACR register? > > ------------------------------ > *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] > *Sent:* Friday, March 06, 2009 1:38 PM > *To:* Liu Dave-R63238 > *Cc:* linuxppc-dev@ozlabs.org > *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited > > 1. BAT2 in linux is set to WIMG=0010, and covers all 64M > 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 > 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. > 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. > 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore > the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. > > Core is 400MHz, and CSB is 133MHz. > > - Ben > > On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > >> and what settings is DMA description bit 3? >> >> > -----Original Message----- >> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= >> freescale.com@ozlabs.org] >> > On Behalf Of Liu Dave-R63238 >> > Sent: Friday, March 06, 2009 1:22 PM >> > To: Ben Menchaca; linuxppc-dev@ozlabs.org >> > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >> > >> > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and >> > PEX_RDMA_CTRL[SNOOP]? >> > >> > What is the freq settings? CORE/CSB bus. >> > >> > Thanks, Dave >> > >> > ________________________________ >> > >> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= >> freescale.com@ozlabs.org] >> > On Behalf Of Ben Menchaca >> > Sent: Friday, March 06, 2009 12:33 PM >> > To: linuxppc-dev@ozlabs.org >> > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited >> > >> > >> > I am working on a Freescale 8314e design, and the >> > embedded device is configured as a PCI-e endpoint running a >> > 2.6.27-5 kernel. For context, we have written a kernel >> > module which, among other things, uses the RDMA/WDMA engine >> > in the PCI-e IP block. On the host side, these DMAs are >> > coherent. However, on the embedded side, things are quite a >> > bit less rosy; we must manually flush/invalidate cache lines >> > for WDMA/RDMAs to occur successfully. After speaking with >> > (several) FAEs at Freescale, we believe there is a >> > configuration issue that is the cause, but we have yet to >> > have anyone successfully point to it. >> > >> > Disabling the data cache altogether resolves the issue >> > entirely, but of course, also completely tanks performance. >> > As a temporary workaround, I would like to simply mark the >> > pages (obtained currently via dma_alloc_coherent) involved as >> > cache-inhibited. I have attempted to do this via some >> > snippets remaining in fec.c (va_to_pte, uncache_pte to set >> > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is >> > almost certainly braindead; va_to_pte is not a part of the >> > 83xx source, as far as I can tell; 8xx only. >> > >> > A quick pointer in the correct direction for marking >> > pages as cache-inhibited on a 2.6.27-5 kernel would be >> > appreciated, or if my approach to a workaround is flawed, a >> > pointer to the correct way would be great. >> > >> > Ben Menchaca >> > >> > >> > _______________________________________________ >> > Linuxppc-dev mailing list >> > Linuxppc-dev@ozlabs.org >> > https://ozlabs.org/mailman/listinfo/linuxppc-dev >> > >> > >> > > [-- Attachment #2: Type: text/html, Size: 6902 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 6:10 ` Ben Menchaca @ 2009-03-06 6:30 ` Liu Dave-R63238 2009-03-06 16:12 ` Ben Menchaca 0 siblings, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-06 6:30 UTC (permalink / raw) To: Ben Menchaca; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 4553 bytes --] Did you enable the descriptor bit 3 to have a try? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 2:10 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited I can look at ACR morning...although I can say with a fair amount of certainty that I have not changed it from the POR value. I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this seems a bit counterintuitive to me. What is the hope regarding these two? Some combination I am not seeing? On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: what is the value of ACR register? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 1:38 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > PEX_RDMA_CTRL[SNOOP]? > > What is the freq settings? CORE/CSB bus. > > Thanks, Dave > > ________________________________ > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > I am working on a Freescale 8314e design, and the > embedded device is configured as a PCI-e endpoint running a > 2.6.27-5 kernel. For context, we have written a kernel > module which, among other things, uses the RDMA/WDMA engine > in the PCI-e IP block. On the host side, these DMAs are > coherent. However, on the embedded side, things are quite a > bit less rosy; we must manually flush/invalidate cache lines > for WDMA/RDMAs to occur successfully. After speaking with > (several) FAEs at Freescale, we believe there is a > configuration issue that is the cause, but we have yet to > have anyone successfully point to it. > > Disabling the data cache altogether resolves the issue > entirely, but of course, also completely tanks performance. > As a temporary workaround, I would like to simply mark the > pages (obtained currently via dma_alloc_coherent) involved as > cache-inhibited. I have attempted to do this via some > snippets remaining in fec.c (va_to_pte, uncache_pte to set > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > almost certainly braindead; va_to_pte is not a part of the > 83xx source, as far as I can tell; 8xx only. > > A quick pointer in the correct direction for marking > pages as cache-inhibited on a 2.6.27-5 kernel would be > appreciated, or if my approach to a workaround is flawed, a > pointer to the correct way would be great. > > Ben Menchaca > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > [-- Attachment #2: Type: text/html, Size: 8562 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 6:30 ` Liu Dave-R63238 @ 2009-03-06 16:12 ` Ben Menchaca 2009-03-06 16:30 ` Ben Menchaca 2009-03-07 21:51 ` Benjamin Herrenschmidt 0 siblings, 2 replies; 17+ messages in thread From: Ben Menchaca @ 2009-03-06 16:12 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 4687 bytes --] Testing now...it looks like it (almost) works, though! Why does setting no-snoop cause snooping to work? More on the effect on setting that bit in a few minutes...need more testing. ACR is 0x00030300. - Ben On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > Did you enable the descriptor bit 3 to have a try? > > ------------------------------ > *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] > *Sent:* Friday, March 06, 2009 2:10 PM > > *To:* Liu Dave-R63238 > *Cc:* linuxppc-dev@ozlabs.org > *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited > > I can look at ACR morning...although I can say with a fair amount of > certainty that I have not changed it from the POR value. > > I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this > seems a bit counterintuitive to me. > > What is the hope regarding these two? Some combination I am not seeing? > > > On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > >> what is the value of ACR register? >> >> ------------------------------ >> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >> *Sent:* Friday, March 06, 2009 1:38 PM >> *To:* Liu Dave-R63238 >> *Cc:* linuxppc-dev@ozlabs.org >> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >> >> 1. BAT2 in linux is set to WIMG=0010, and covers all 64M >> 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 >> 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. >> 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. >> 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore >> the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. >> >> Core is 400MHz, and CSB is 133MHz. >> >> - Ben >> >> On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: >> >>> and what settings is DMA description bit 3? >>> >>> > -----Original Message----- >>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= >>> freescale.com@ozlabs.org] >>> > On Behalf Of Liu Dave-R63238 >>> > Sent: Friday, March 06, 2009 1:22 PM >>> > To: Ben Menchaca; linuxppc-dev@ozlabs.org >>> > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> > >>> > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and >>> > PEX_RDMA_CTRL[SNOOP]? >>> > >>> > What is the freq settings? CORE/CSB bus. >>> > >>> > Thanks, Dave >>> > >>> > ________________________________ >>> > >>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu>= >>> freescale.com@ozlabs.org] >>> > On Behalf Of Ben Menchaca >>> > Sent: Friday, March 06, 2009 12:33 PM >>> > To: linuxppc-dev@ozlabs.org >>> > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> > >>> > >>> > I am working on a Freescale 8314e design, and the >>> > embedded device is configured as a PCI-e endpoint running a >>> > 2.6.27-5 kernel. For context, we have written a kernel >>> > module which, among other things, uses the RDMA/WDMA engine >>> > in the PCI-e IP block. On the host side, these DMAs are >>> > coherent. However, on the embedded side, things are quite a >>> > bit less rosy; we must manually flush/invalidate cache lines >>> > for WDMA/RDMAs to occur successfully. After speaking with >>> > (several) FAEs at Freescale, we believe there is a >>> > configuration issue that is the cause, but we have yet to >>> > have anyone successfully point to it. >>> > >>> > Disabling the data cache altogether resolves the issue >>> > entirely, but of course, also completely tanks performance. >>> > As a temporary workaround, I would like to simply mark the >>> > pages (obtained currently via dma_alloc_coherent) involved as >>> > cache-inhibited. I have attempted to do this via some >>> > snippets remaining in fec.c (va_to_pte, uncache_pte to set >>> > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is >>> > almost certainly braindead; va_to_pte is not a part of the >>> > 83xx source, as far as I can tell; 8xx only. >>> > >>> > A quick pointer in the correct direction for marking >>> > pages as cache-inhibited on a 2.6.27-5 kernel would be >>> > appreciated, or if my approach to a workaround is flawed, a >>> > pointer to the correct way would be great. >>> > >>> > Ben Menchaca >>> > >>> > >>> > _______________________________________________ >>> > Linuxppc-dev mailing list >>> > Linuxppc-dev@ozlabs.org >>> > https://ozlabs.org/mailman/listinfo/linuxppc-dev >>> > >>> > >>> >> >> > [-- Attachment #2: Type: text/html, Size: 8684 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 16:12 ` Ben Menchaca @ 2009-03-06 16:30 ` Ben Menchaca 2009-03-09 2:30 ` Liu Dave-R63238 2009-03-09 2:56 ` Liu Dave-R63238 2009-03-07 21:51 ` Benjamin Herrenschmidt 1 sibling, 2 replies; 17+ messages in thread From: Ben Menchaca @ 2009-03-06 16:30 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 5496 bytes --] Thank you for your help! That bit resolved all of the RDMA/WDMA coherency issues on the CSB side...except: We expose a 1MB region of memory from CSB via a BAR (BAR1, if it matters) to the Host. This region is also not behaving correctly with respect to coherency on SOME hosts; again, disabling our cache makes it work correctly on all hosts. We have set PEX_DEVICE_CONTROL in PCI-E Config Space (0x54) to 0x2010 (sorry about the endianness below). We thought that CLEARING the no-snoop bit here would indicate that snooping was required for this region...is this a similar issue? - Ben On Fri, Mar 6, 2009 at 10:12 AM, Ben Menchaca <ben.menchaca@gmail.com>wrote: > Testing now...it looks like it (almost) works, though! Why does setting > no-snoop cause snooping to work? More on the effect on setting that bit in > a few minutes...need more testing. > > ACR is 0x00030300. > > - Ben > > > On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > >> Did you enable the descriptor bit 3 to have a try? >> >> ------------------------------ >> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >> *Sent:* Friday, March 06, 2009 2:10 PM >> >> *To:* Liu Dave-R63238 >> *Cc:* linuxppc-dev@ozlabs.org >> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >> >> I can look at ACR morning...although I can say with a fair amount of >> certainty that I have not changed it from the POR value. >> >> I will try enabling No Snoop for CSB in the descriptor (bit 3, >> yes?)...this seems a bit counterintuitive to me. >> >> What is the hope regarding these two? Some combination I am not seeing? >> >> >> On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: >> >>> what is the value of ACR register? >>> >>> ------------------------------ >>> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >>> *Sent:* Friday, March 06, 2009 1:38 PM >>> *To:* Liu Dave-R63238 >>> *Cc:* linuxppc-dev@ozlabs.org >>> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> >>> 1. BAT2 in linux is set to WIMG=0010, and covers all 64M >>> 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 >>> 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. >>> 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. >>> 5. The DMA descriptor (randomly chosen when I hit a trigger...just >>> ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. >>> >>> >>> Core is 400MHz, and CSB is 133MHz. >>> >>> - Ben >>> >>> On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: >>> >>>> and what settings is DMA description bit 3? >>>> >>>> > -----Original Message----- >>>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>>> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu> >>>> =freescale.com@ozlabs.org] >>>> > On Behalf Of Liu Dave-R63238 >>>> > Sent: Friday, March 06, 2009 1:22 PM >>>> > To: Ben Menchaca; linuxppc-dev@ozlabs.org >>>> > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >>>> > >>>> > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and >>>> > PEX_RDMA_CTRL[SNOOP]? >>>> > >>>> > What is the freq settings? CORE/CSB bus. >>>> > >>>> > Thanks, Dave >>>> > >>>> > ________________________________ >>>> > >>>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>>> > [mailto:linuxppc-dev-bounces+daveliu <linuxppc-dev-bounces%2Bdaveliu> >>>> =freescale.com@ozlabs.org] >>>> > On Behalf Of Ben Menchaca >>>> > Sent: Friday, March 06, 2009 12:33 PM >>>> > To: linuxppc-dev@ozlabs.org >>>> > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited >>>> > >>>> > >>>> > I am working on a Freescale 8314e design, and the >>>> > embedded device is configured as a PCI-e endpoint running a >>>> > 2.6.27-5 kernel. For context, we have written a kernel >>>> > module which, among other things, uses the RDMA/WDMA engine >>>> > in the PCI-e IP block. On the host side, these DMAs are >>>> > coherent. However, on the embedded side, things are quite a >>>> > bit less rosy; we must manually flush/invalidate cache lines >>>> > for WDMA/RDMAs to occur successfully. After speaking with >>>> > (several) FAEs at Freescale, we believe there is a >>>> > configuration issue that is the cause, but we have yet to >>>> > have anyone successfully point to it. >>>> > >>>> > Disabling the data cache altogether resolves the issue >>>> > entirely, but of course, also completely tanks performance. >>>> > As a temporary workaround, I would like to simply mark the >>>> > pages (obtained currently via dma_alloc_coherent) involved as >>>> > cache-inhibited. I have attempted to do this via some >>>> > snippets remaining in fec.c (va_to_pte, uncache_pte to set >>>> > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is >>>> > almost certainly braindead; va_to_pte is not a part of the >>>> > 83xx source, as far as I can tell; 8xx only. >>>> > >>>> > A quick pointer in the correct direction for marking >>>> > pages as cache-inhibited on a 2.6.27-5 kernel would be >>>> > appreciated, or if my approach to a workaround is flawed, a >>>> > pointer to the correct way would be great. >>>> > >>>> > Ben Menchaca >>>> > >>>> > >>>> > _______________________________________________ >>>> > Linuxppc-dev mailing list >>>> > Linuxppc-dev@ozlabs.org >>>> > https://ozlabs.org/mailman/listinfo/linuxppc-dev >>>> > >>>> > >>>> >>> >>> >> > [-- Attachment #2: Type: text/html, Size: 10299 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 16:30 ` Ben Menchaca @ 2009-03-09 2:30 ` Liu Dave-R63238 2009-03-09 2:56 ` Liu Dave-R63238 1 sibling, 0 replies; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-09 2:30 UTC (permalink / raw) To: Ben Menchaca; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 6343 bytes --] Hi Ben, The second issue. you told me "some hosts" has problem, and some hosts worked well. what is the problem-hosts? The issue seems like the hosts did set the NO SNOOP attribute bit at TLP. The PEX_DEVICE_CONTROL is standard PCI configuration space register, it controls the behavior of the initiator's transaction. For 8315, it is outbound, not inbound transaction. Thanks, Dave ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Saturday, March 07, 2009 12:30 AM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited Thank you for your help! That bit resolved all of the RDMA/WDMA coherency issues on the CSB side...except: We expose a 1MB region of memory from CSB via a BAR (BAR1, if it matters) to the Host. This region is also not behaving correctly with respect to coherency on SOME hosts; again, disabling our cache makes it work correctly on all hosts. We have set PEX_DEVICE_CONTROL in PCI-E Config Space (0x54) to 0x2010 (sorry about the endianness below). We thought that CLEARING the no-snoop bit here would indicate that snooping was required for this region...is this a similar issue? - Ben On Fri, Mar 6, 2009 at 10:12 AM, Ben Menchaca <ben.menchaca@gmail.com> wrote: Testing now...it looks like it (almost) works, though! Why does setting no-snoop cause snooping to work? More on the effect on setting that bit in a few minutes...need more testing. ACR is 0x00030300. - Ben On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: Did you enable the descriptor bit 3 to have a try? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 2:10 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited I can look at ACR morning...although I can say with a fair amount of certainty that I have not changed it from the POR value. I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this seems a bit counterintuitive to me. What is the hope regarding these two? Some combination I am not seeing? On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: what is the value of ACR register? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 1:38 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > PEX_RDMA_CTRL[SNOOP]? > > What is the freq settings? CORE/CSB bus. > > Thanks, Dave > > ________________________________ > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > I am working on a Freescale 8314e design, and the > embedded device is configured as a PCI-e endpoint running a > 2.6.27-5 kernel. For context, we have written a kernel > module which, among other things, uses the RDMA/WDMA engine > in the PCI-e IP block. On the host side, these DMAs are > coherent. However, on the embedded side, things are quite a > bit less rosy; we must manually flush/invalidate cache lines > for WDMA/RDMAs to occur successfully. After speaking with > (several) FAEs at Freescale, we believe there is a > configuration issue that is the cause, but we have yet to > have anyone successfully point to it. > > Disabling the data cache altogether resolves the issue > entirely, but of course, also completely tanks performance. > As a temporary workaround, I would like to simply mark the > pages (obtained currently via dma_alloc_coherent) involved as > cache-inhibited. I have attempted to do this via some > snippets remaining in fec.c (va_to_pte, uncache_pte to set > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > almost certainly braindead; va_to_pte is not a part of the > 83xx source, as far as I can tell; 8xx only. > > A quick pointer in the correct direction for marking > pages as cache-inhibited on a 2.6.27-5 kernel would be > appreciated, or if my approach to a workaround is flawed, a > pointer to the correct way would be great. > > Ben Menchaca > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > [-- Attachment #2: Type: text/html, Size: 14731 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 16:30 ` Ben Menchaca 2009-03-09 2:30 ` Liu Dave-R63238 @ 2009-03-09 2:56 ` Liu Dave-R63238 2009-03-09 3:27 ` Ben Menchaca 1 sibling, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-09 2:56 UTC (permalink / raw) To: Ben Menchaca; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 6343 bytes --] Hi Ben, The second issue. you told me "some hosts" has problem, and some hosts worked well. what is the problem-hosts? The issue seems like the hosts did set the NO SNOOP attribute bit at TLP. The PEX_DEVICE_CONTROL is standard PCI configuration space register, it controls the behavior of the initiator's transaction. For 8315, it is outbound, not inbound transaction. Thanks, Dave ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Saturday, March 07, 2009 12:30 AM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited Thank you for your help! That bit resolved all of the RDMA/WDMA coherency issues on the CSB side...except: We expose a 1MB region of memory from CSB via a BAR (BAR1, if it matters) to the Host. This region is also not behaving correctly with respect to coherency on SOME hosts; again, disabling our cache makes it work correctly on all hosts. We have set PEX_DEVICE_CONTROL in PCI-E Config Space (0x54) to 0x2010 (sorry about the endianness below). We thought that CLEARING the no-snoop bit here would indicate that snooping was required for this region...is this a similar issue? - Ben On Fri, Mar 6, 2009 at 10:12 AM, Ben Menchaca <ben.menchaca@gmail.com> wrote: Testing now...it looks like it (almost) works, though! Why does setting no-snoop cause snooping to work? More on the effect on setting that bit in a few minutes...need more testing. ACR is 0x00030300. - Ben On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: Did you enable the descriptor bit 3 to have a try? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 2:10 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited I can look at ACR morning...although I can say with a fair amount of certainty that I have not changed it from the POR value. I will try enabling No Snoop for CSB in the descriptor (bit 3, yes?)...this seems a bit counterintuitive to me. What is the hope regarding these two? Some combination I am not seeing? On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: what is the value of ACR register? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 1:38 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > PEX_RDMA_CTRL[SNOOP]? > > What is the freq settings? CORE/CSB bus. > > Thanks, Dave > > ________________________________ > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > I am working on a Freescale 8314e design, and the > embedded device is configured as a PCI-e endpoint running a > 2.6.27-5 kernel. For context, we have written a kernel > module which, among other things, uses the RDMA/WDMA engine > in the PCI-e IP block. On the host side, these DMAs are > coherent. However, on the embedded side, things are quite a > bit less rosy; we must manually flush/invalidate cache lines > for WDMA/RDMAs to occur successfully. After speaking with > (several) FAEs at Freescale, we believe there is a > configuration issue that is the cause, but we have yet to > have anyone successfully point to it. > > Disabling the data cache altogether resolves the issue > entirely, but of course, also completely tanks performance. > As a temporary workaround, I would like to simply mark the > pages (obtained currently via dma_alloc_coherent) involved as > cache-inhibited. I have attempted to do this via some > snippets remaining in fec.c (va_to_pte, uncache_pte to set > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > almost certainly braindead; va_to_pte is not a part of the > 83xx source, as far as I can tell; 8xx only. > > A quick pointer in the correct direction for marking > pages as cache-inhibited on a 2.6.27-5 kernel would be > appreciated, or if my approach to a workaround is flawed, a > pointer to the correct way would be great. > > Ben Menchaca > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > [-- Attachment #2: Type: text/html, Size: 14763 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-09 2:56 ` Liu Dave-R63238 @ 2009-03-09 3:27 ` Ben Menchaca 0 siblings, 0 replies; 17+ messages in thread From: Ben Menchaca @ 2009-03-09 3:27 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 7201 bytes --] The only problem host found so far is a newer Asus 970FX motherboard, regardless of OS. We are seeing that some time after LTSSM finishes, but long before OS load, our PEX_DEVICE_CONTROL register is changed. On the "working" motherboards, NO_SNOOP is enabled; if I read the spec correctly, this means that TLPs with no-snoop are permitted. On the non-working motherboards, NO_SNOOP is disabled; this is supposed to mean that TLPs with no-snoop are not permitted (again, if I understand correctly). Is it possible that there has been another misinterpretation of this bit? Something regarding the generation of snoops on CSB when it is cleared? This bit could be a complete red herring; it was one of the few differences in the config space, however, so it was my best guess. - Ben On Sun, Mar 8, 2009 at 9:56 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: > Hi Ben, > > The second issue. you told me "some hosts" has problem, > and some hosts worked well. > > what is the problem-hosts? > > The issue seems like the hosts did set the NO SNOOP attribute > bit at TLP. > > The PEX_DEVICE_CONTROL is standard PCI configuration space > register, it controls the behavior of the initiator's transaction. > For 8315, it is outbound, not inbound transaction. > > Thanks, Dave > > ------------------------------ > *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] > *Sent:* Saturday, March 07, 2009 12:30 AM > *To:* Liu Dave-R63238 > *Cc:* linuxppc-dev@ozlabs.org > *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Thank you for your help! That bit resolved all of the RDMA/WDMA coherency > issues on the CSB side...except: > > We expose a 1MB region of memory from CSB via a BAR (BAR1, if it matters) > to the Host. This region is also not behaving correctly with respect to > coherency on SOME hosts; again, disabling our cache makes it work correctly > on all hosts. We have set PEX_DEVICE_CONTROL in PCI-E Config Space (0x54) > to 0x2010 (sorry about the endianness below). We thought that CLEARING > the no-snoop bit here would indicate that snooping was required for this > region...is this a similar issue? > > - Ben > > On Fri, Mar 6, 2009 at 10:12 AM, Ben Menchaca <ben.menchaca@gmail.com>wrote: > >> Testing now...it looks like it (almost) works, though! Why does setting >> no-snoop cause snooping to work? More on the effect on setting that bit in >> a few minutes...need more testing. >> >> ACR is 0x00030300. >> >> - Ben >> >> >> On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: >> >>> Did you enable the descriptor bit 3 to have a try? >>> >>> ------------------------------ >>> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >>> *Sent:* Friday, March 06, 2009 2:10 PM >>> >>> *To:* Liu Dave-R63238 >>> *Cc:* linuxppc-dev@ozlabs.org >>> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >>> >>> I can look at ACR morning...although I can say with a fair amount of >>> certainty that I have not changed it from the POR value. >>> >>> I will try enabling No Snoop for CSB in the descriptor (bit 3, >>> yes?)...this seems a bit counterintuitive to me. >>> >>> What is the hope regarding these two? Some combination I am not seeing? >>> >>> >>> On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 <DaveLiu@freescale.com>wrote: >>> >>>> what is the value of ACR register? >>>> >>>> ------------------------------ >>>> *From:* Ben Menchaca [mailto:ben.menchaca@gmail.com] >>>> *Sent:* Friday, March 06, 2009 1:38 PM >>>> *To:* Liu Dave-R63238 >>>> *Cc:* linuxppc-dev@ozlabs.org >>>> *Subject:* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >>>> >>>> 1. BAT2 in linux is set to WIMG=0010, and covers all 64M >>>> 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 >>>> 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. >>>> 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. >>>> 5. The DMA descriptor (randomly chosen when I hit a trigger...just >>>> ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. >>>> >>>> >>>> Core is 400MHz, and CSB is 133MHz. >>>> >>>> - Ben >>>> >>>> On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com >>>> > wrote: >>>> >>>>> and what settings is DMA description bit 3? >>>>> >>>>> > -----Original Message----- >>>>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>>>> > [mailto:linuxppc-dev-bounces+daveliu<linuxppc-dev-bounces%2Bdaveliu> >>>>> =freescale.com@ozlabs.org] >>>>> > On Behalf Of Liu Dave-R63238 >>>>> > Sent: Friday, March 06, 2009 1:22 PM >>>>> > To: Ben Menchaca; linuxppc-dev@ozlabs.org >>>>> > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited >>>>> > >>>>> > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and >>>>> > PEX_RDMA_CTRL[SNOOP]? >>>>> > >>>>> > What is the freq settings? CORE/CSB bus. >>>>> > >>>>> > Thanks, Dave >>>>> > >>>>> > ________________________________ >>>>> > >>>>> > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org >>>>> > [mailto:linuxppc-dev-bounces+daveliu<linuxppc-dev-bounces%2Bdaveliu> >>>>> =freescale.com@ozlabs.org] >>>>> > On Behalf Of Ben Menchaca >>>>> > Sent: Friday, March 06, 2009 12:33 PM >>>>> > To: linuxppc-dev@ozlabs.org >>>>> > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited >>>>> > >>>>> > >>>>> > I am working on a Freescale 8314e design, and the >>>>> > embedded device is configured as a PCI-e endpoint running a >>>>> > 2.6.27-5 kernel. For context, we have written a kernel >>>>> > module which, among other things, uses the RDMA/WDMA engine >>>>> > in the PCI-e IP block. On the host side, these DMAs are >>>>> > coherent. However, on the embedded side, things are quite a >>>>> > bit less rosy; we must manually flush/invalidate cache lines >>>>> > for WDMA/RDMAs to occur successfully. After speaking with >>>>> > (several) FAEs at Freescale, we believe there is a >>>>> > configuration issue that is the cause, but we have yet to >>>>> > have anyone successfully point to it. >>>>> > >>>>> > Disabling the data cache altogether resolves the issue >>>>> > entirely, but of course, also completely tanks performance. >>>>> > As a temporary workaround, I would like to simply mark the >>>>> > pages (obtained currently via dma_alloc_coherent) involved as >>>>> > cache-inhibited. I have attempted to do this via some >>>>> > snippets remaining in fec.c (va_to_pte, uncache_pte to set >>>>> > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is >>>>> > almost certainly braindead; va_to_pte is not a part of the >>>>> > 83xx source, as far as I can tell; 8xx only. >>>>> > >>>>> > A quick pointer in the correct direction for marking >>>>> > pages as cache-inhibited on a 2.6.27-5 kernel would be >>>>> > appreciated, or if my approach to a workaround is flawed, a >>>>> > pointer to the correct way would be great. >>>>> > >>>>> > Ben Menchaca >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > Linuxppc-dev mailing list >>>>> > Linuxppc-dev@ozlabs.org >>>>> > https://ozlabs.org/mailman/listinfo/linuxppc-dev >>>>> > >>>>> > >>>>> >>>> >>>> >>> >> > [-- Attachment #2: Type: text/html, Size: 14998 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 16:12 ` Ben Menchaca 2009-03-06 16:30 ` Ben Menchaca @ 2009-03-07 21:51 ` Benjamin Herrenschmidt 2009-03-08 3:15 ` Ben Menchaca 1 sibling, 1 reply; 17+ messages in thread From: Benjamin Herrenschmidt @ 2009-03-07 21:51 UTC (permalink / raw) To: Ben Menchaca; +Cc: linuxppc-dev, Liu Dave-R63238 On Fri, 2009-03-06 at 10:12 -0600, Ben Menchaca wrote: > Testing now...it looks like it (almost) works, though! Why does > setting no-snoop cause snooping to work? More on the effect on > setting that bit in a few minutes...need more testing. Maybe they got the documentation for that bit backward ? :-) Cheers, Ben. > ACR is 0x00030300. > > - Ben > > On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238 > <DaveLiu@freescale.com> wrote: > Did you enable the descriptor bit 3 to have a try? > > > ______________________________________________________ > From: Ben Menchaca [mailto:ben.menchaca@gmail.com] > > Sent: Friday, March 06, 2009 2:10 PM > > > To: Liu Dave-R63238 > Cc: linuxppc-dev@ozlabs.org > Subject: Re: 83xx: Marking or Allocating Pages as > Cache-Inhibited > > > > > > I can look at ACR morning...although I can say with a > fair amount of certainty that I have not changed it > from the POR value. > > I will try enabling No Snoop for CSB in the descriptor > (bit 3, yes?)...this seems a bit counterintuitive to > me. > > What is the hope regarding these two? Some > combination I am not seeing? > > > On Thu, Mar 5, 2009 at 11:40 PM, Liu Dave-R63238 > <DaveLiu@freescale.com> wrote: > what is the value of ACR register? > > > ______________________________________ > From: Ben Menchaca > [mailto:ben.menchaca@gmail.com] > Sent: Friday, March 06, 2009 1:38 PM > To: Liu Dave-R63238 > Cc: linuxppc-dev@ozlabs.org > Subject: Re: 83xx: Marking or > Allocating Pages as Cache-Inhibited > > > > > 1. BAT2 in linux is set to WIMG=0010, > and covers all 64M > 2. PEX_DEVICE_CONTROL in PCI-E Config > Space (0x54): 0x1020 > 3. PEX_xDMA_CTRL is set to 0x00000401 > at the initiation of the DMA. > 4. OWAR0 is set to 0xFFFFF005, so > NSNP is 0. > 5. The DMA descriptor (randomly > chosen when I hit a trigger...just > ignore the size...) contains 0002AFF3 > at offset 0, so nosnoops are > cleared. > > Core is 400MHz, and CSB is 133MHz. > > - Ben > > On Thu, Mar 5, 2009 at 11:27 PM, Liu > Dave-R63238 <DaveLiu@freescale.com> > wrote: > and what settings is DMA > description bit 3? > > > -----Original Message----- > > From: linuxppc-dev-bounces > +daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces > +daveliu=freescale.com@ozlabs.org] > > > > On Behalf Of Liu > Dave-R63238 > > Sent: Friday, March 06, 2009 > 1:22 PM > > To: Ben Menchaca; > linuxppc-dev@ozlabs.org > > Subject: RE: 83xx: Marking > or Allocating Pages as > Cache-Inhibited > > > > Did you enable the snoop bit > at PEX_WDMA_CTRL[SNOOP] and > > PEX_RDMA_CTRL[SNOOP]? > > > > What is the freq settings? > CORE/CSB bus. > > > > Thanks, Dave > > > > > ________________________________ > > > > From: > linuxppc-dev-bounces > +daveliu=freescale.com@ozlabs.org > > [mailto:linuxppc-dev-bounces > +daveliu=freescale.com@ozlabs.org] > > On Behalf Of Ben Menchaca > > Sent: Friday, March > 06, 2009 12:33 PM > > To: > linuxppc-dev@ozlabs.org > > Subject: 83xx: Marking > or Allocating Pages as > Cache-Inhibited > > > > > > I am working on a > Freescale 8314e design, and > the > > embedded device is > configured as a PCI-e endpoint > running a > > 2.6.27-5 kernel. For > context, we have written a > kernel > > module which, among other > things, uses the RDMA/WDMA > engine > > in the PCI-e IP block. On > the host side, these DMAs are > > coherent. However, on the > embedded side, things are > quite a > > bit less rosy; we must > manually flush/invalidate > cache lines > > for WDMA/RDMAs to occur > successfully. After speaking > with > > (several) FAEs at Freescale, > we believe there is a > > configuration issue that is > the cause, but we have yet to > > have anyone successfully > point to it. > > > > Disabling the data > cache altogether resolves the > issue > > entirely, but of course, > also completely tanks > performance. > > As a temporary workaround, I > would like to simply mark the > > pages (obtained currently > via dma_alloc_coherent) > involved as > > cache-inhibited. I have > attempted to do this via some > > snippets remaining in fec.c > (va_to_pte, uncache_pte to set > > _PAGE_NO_CACHE, > flush_tlb_page, then > unmap_pte), but this is > > almost certainly braindead; > va_to_pte is not a part of the > > 83xx source, as far as I can > tell; 8xx only. > > > > A quick pointer in the > correct direction for marking > > pages as cache-inhibited on > a 2.6.27-5 kernel would be > > appreciated, or if my > approach to a workaround is > flawed, a > > pointer to the correct way > would be great. > > > > Ben Menchaca > > > > > > > > _______________________________________________ > > Linuxppc-dev mailing list > > Linuxppc-dev@ozlabs.org > > > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > > > > > > > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-07 21:51 ` Benjamin Herrenschmidt @ 2009-03-08 3:15 ` Ben Menchaca 0 siblings, 0 replies; 17+ messages in thread From: Ben Menchaca @ 2009-03-08 3:15 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Liu Dave-R63238 [-- Attachment #1: Type: text/plain, Size: 762 bytes --] On Sat, Mar 7, 2009 at 3:51 PM, Benjamin Herrenschmidt < benh@kernel.crashing.org> wrote: > On Fri, 2009-03-06 at 10:12 -0600, Ben Menchaca wrote: > > Testing now...it looks like it (almost) works, though! Why does > > setting no-snoop cause snooping to work? More on the effect on > > setting that bit in a few minutes...need more testing. > > Maybe they got the documentation for that bit backward ? :-) For posterity...it does appear that this is the case. I don't have a bus analyzer to watch the transaction, but a JTAG trigger caught the update happening if and only if bit 3 was set in the (R|W)DMA descriptor. My FAE at FS said he is watching this thread, so hopefully some doc errata can be generated so others can avoid my confusion :-). - Ben [-- Attachment #2: Type: text/html, Size: 1091 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:38 ` Ben Menchaca 2009-03-06 5:40 ` Liu Dave-R63238 @ 2009-03-06 5:49 ` Liu Dave-R63238 2009-03-08 15:19 ` Timur Tabi 1 sibling, 1 reply; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-06 5:49 UTC (permalink / raw) To: Liu Dave-R63238, Ben Menchaca; +Cc: linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 4004 bytes --] could you try to set '1' to DMA description bit3? ________________________________ From: Liu Dave-R63238 Sent: Friday, March 06, 2009 1:41 PM To: 'Ben Menchaca' Cc: linuxppc-dev@ozlabs.org Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited what is the value of ACR register? ________________________________ From: Ben Menchaca [mailto:ben.menchaca@gmail.com] Sent: Friday, March 06, 2009 1:38 PM To: Liu Dave-R63238 Cc: linuxppc-dev@ozlabs.org Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 1. BAT2 in linux is set to WIMG=0010, and covers all 64M 2. PEX_DEVICE_CONTROL in PCI-E Config Space (0x54): 0x1020 3. PEX_xDMA_CTRL is set to 0x00000401 at the initiation of the DMA. 4. OWAR0 is set to 0xFFFFF005, so NSNP is 0. 5. The DMA descriptor (randomly chosen when I hit a trigger...just ignore the size...) contains 0002AFF3 at offset 0, so nosnoops are cleared. Core is 400MHz, and CSB is 133MHz. - Ben On Thu, Mar 5, 2009 at 11:27 PM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: and what settings is DMA description bit 3? > -----Original Message----- > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Liu Dave-R63238 > Sent: Friday, March 06, 2009 1:22 PM > To: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: RE: 83xx: Marking or Allocating Pages as Cache-Inhibited > > Did you enable the snoop bit at PEX_WDMA_CTRL[SNOOP] and > PEX_RDMA_CTRL[SNOOP]? > > What is the freq settings? CORE/CSB bus. > > Thanks, Dave > > ________________________________ > > From: linuxppc-dev-bounces+daveliu=freescale.com@ozlabs.org > [mailto:linuxppc-dev-bounces+daveliu <mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale.com@ozlabs.org] > On Behalf Of Ben Menchaca > Sent: Friday, March 06, 2009 12:33 PM > To: linuxppc-dev@ozlabs.org > Subject: 83xx: Marking or Allocating Pages as Cache-Inhibited > > > I am working on a Freescale 8314e design, and the > embedded device is configured as a PCI-e endpoint running a > 2.6.27-5 kernel. For context, we have written a kernel > module which, among other things, uses the RDMA/WDMA engine > in the PCI-e IP block. On the host side, these DMAs are > coherent. However, on the embedded side, things are quite a > bit less rosy; we must manually flush/invalidate cache lines > for WDMA/RDMAs to occur successfully. After speaking with > (several) FAEs at Freescale, we believe there is a > configuration issue that is the cause, but we have yet to > have anyone successfully point to it. > > Disabling the data cache altogether resolves the issue > entirely, but of course, also completely tanks performance. > As a temporary workaround, I would like to simply mark the > pages (obtained currently via dma_alloc_coherent) involved as > cache-inhibited. I have attempted to do this via some > snippets remaining in fec.c (va_to_pte, uncache_pte to set > _PAGE_NO_CACHE, flush_tlb_page, then unmap_pte), but this is > almost certainly braindead; va_to_pte is not a part of the > 83xx source, as far as I can tell; 8xx only. > > A quick pointer in the correct direction for marking > pages as cache-inhibited on a 2.6.27-5 kernel would be > appreciated, or if my approach to a workaround is flawed, a > pointer to the correct way would be great. > > Ben Menchaca > > > _______________________________________________ > Linuxppc-dev mailing list > Linuxppc-dev@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-dev > > [-- Attachment #2: Type: text/html, Size: 7380 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-06 5:49 ` Liu Dave-R63238 @ 2009-03-08 15:19 ` Timur Tabi 2009-03-09 2:33 ` Liu Dave-R63238 0 siblings, 1 reply; 17+ messages in thread From: Timur Tabi @ 2009-03-08 15:19 UTC (permalink / raw) To: Liu Dave-R63238; +Cc: Ben Menchaca, linuxppc-dev On Fri, Mar 6, 2009 at 12:49 AM, Liu Dave-R63238 <DaveLiu@freescale.com> wrote: > could you try to set '1' to DMA description bit3? Dave, I'm looking at the 8315e reference manual. What is "DMA description bit 3"? -- Timur Tabi Linux kernel developer at Freescale ^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: 83xx: Marking or Allocating Pages as Cache-Inhibited 2009-03-08 15:19 ` Timur Tabi @ 2009-03-09 2:33 ` Liu Dave-R63238 0 siblings, 0 replies; 17+ messages in thread From: Liu Dave-R63238 @ 2009-03-09 2:33 UTC (permalink / raw) To: Tabi Timur-B04825; +Cc: Ben Menchaca, linuxppc-dev Timur, See the section 14.7.1 DMA Description Format at page 14-112 of MPC8315ERM rev.1. > -----Original Message----- > From: timur.tabi@gmail.com [mailto:timur.tabi@gmail.com] On=20 > Behalf Of Tabi Timur-B04825 > Sent: Sunday, March 08, 2009 11:20 PM > To: Liu Dave-R63238 > Cc: Ben Menchaca; linuxppc-dev@ozlabs.org > Subject: Re: 83xx: Marking or Allocating Pages as Cache-Inhibited >=20 > On Fri, Mar 6, 2009 at 12:49 AM, Liu Dave-R63238=20 > <DaveLiu@freescale.com> wrote: > > could you try to set '1' to DMA description bit3? >=20 > Dave, >=20 > I'm looking at the 8315e reference manual. What is "DMA=20 > description bit 3"? >=20 > -- > Timur Tabi > Linux kernel developer at Freescale >=20 >=20 ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2009-03-09 3:27 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-03-06 4:32 83xx: Marking or Allocating Pages as Cache-Inhibited Ben Menchaca 2009-03-06 5:22 ` Liu Dave-R63238 2009-03-06 5:27 ` Liu Dave-R63238 2009-03-06 5:38 ` Ben Menchaca 2009-03-06 5:40 ` Liu Dave-R63238 2009-03-06 6:10 ` Ben Menchaca 2009-03-06 6:30 ` Liu Dave-R63238 2009-03-06 16:12 ` Ben Menchaca 2009-03-06 16:30 ` Ben Menchaca 2009-03-09 2:30 ` Liu Dave-R63238 2009-03-09 2:56 ` Liu Dave-R63238 2009-03-09 3:27 ` Ben Menchaca 2009-03-07 21:51 ` Benjamin Herrenschmidt 2009-03-08 3:15 ` Ben Menchaca 2009-03-06 5:49 ` Liu Dave-R63238 2009-03-08 15:19 ` Timur Tabi 2009-03-09 2:33 ` Liu Dave-R63238
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.