[RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Hello folks,

I'm currently working on a Linux 2.6.x port for a MCP7448 based board.
And I'm observing that some memory areas become corrupted on early
kernel startup. Namely some parts of kernel page table, that leads to
sudden kernel crashes. After some investigations I've managed to narrow
the problem down to a _set_L2CR() function that seems to not work for
this processor and the memory become corrupted after the cache is
flushed and re-enabled.

Note that I'm working with arch/ppc code. I've looked through
arch/powerpc sources and found some fixes for MPC7450 processors family
in l2cr_6xx.S. Then I've backported them to arch/ppc but it didn't help.

I read the MPC7450 reference manual and it describes a method to flush
the cache using L2 hardware flushing feature. The cache flushing
procedure consist of several steps and some of them are dictated by
MPC7448 errata (www.freescale.com/files/32bit/doc/errata/MPC7448CE.pdf,
Erratum no.3).

When I implemented that method in _set_L2CR() (patch included below)
following the MPC7450 reference manual, things worked fine. With these
changes the kernel works stable. The same result I get when I force a
boot loader to not enable the L2 cache before starting the kernel.

So, I have couple questions:

First, I'm looking for a help and advice why the current _set_L2CR()
implementation may not work for MPC7450 (namely 7448 with 1Mb L2 cache
installed). Is it a bug in _set_L2CR()  or a hardware problem. I've
mentioned above about MPC7450 hardware bug in L2 hardware flushing
mechanism. May it be applicable to common cache flushing procedure based
on sequence of lwz/dcbf instructions?

I'd like to note that I've had a hard time to find a root cause of the
problem in the current _set_L2CR() implementation. I changed a size of
memory used to fill and then flush the cache, tried to apply the
workaround for the MPC7448 hardware bug above and nothing helped except
the patch below.

Second, Is this patch acceptable?

Any feedback will be appreciated.

Thanks in advance,
Vlad.

---

 arch/powerpc/kernel/l2cr_6xx.S |   38 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 38 insertions(+), 0 deletions(-)

diff --git a/arch/powerpc/kernel/l2cr_6xx.S b/arch/powerpc/kernel/l2cr_6xx.S
index 858f28a..e6d96a6 100644
--- a/arch/powerpc/kernel/l2cr_6xx.S
+++ b/arch/powerpc/kernel/l2cr_6xx.S
@@ -156,6 +156,19 @@ END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
 	  ****/
 
 BEGIN_FTR_SECTION
+	/* MPC7450 provides a L2 hardware flushing mechanism
+	 * implemented through L2CR[L2HWF] bit. Refer to MPC7450
+	 * reference manual p. 3-53 for details.
+	 */
+
+	/* Set L2CR[L2IO], L2CR[L2DO] to lock the cache */
+	mfspr	r4, SPRN_L2CR
+	oris	r4, r4, 0x0011
+	sync
+	mtspr	SPRN_L2CR, r4
+	sync
+	isync
+	
 	/* Disable L2 prefetch on some 745x and try to ensure
 	 * L2 prefetch engines are idle. As explained by errata
 	 * text, we can't be sure they are, we just hope very hard
@@ -173,6 +186,31 @@ BEGIN_FTR_SECTION
 	dcbf	0,r4
 	dcbf	0,r4
 	dcbf	0,r4
+
+	/* Flush and invalidate L2:
+	 * Set L2HWF and wait till it is cleared.
+	 */
+	mfspr	r4, SPRN_L2CR
+	ori	r4, r4, 0x0800
+	sync
+	mtspr	SPRN_L2CR, r4
+	sync
+	isync
+	
+1:	mfspr	r4, SPRN_L2CR
+	rlwinm.	r0,r4,0,20,20
+	bne	1b
+
+	/* Clear L2IO, L2DO to unlock the cache */
+	mfspr	r4, SPRN_L2CR
+	rlwinm  r4, r4, 0, 12, 10
+	rlwinm	r4, r4, 0, 16, 14
+	sync
+	mtspr	SPRN_L2CR, r4
+	sync
+	isync
+
+	b	2f
 END_FTR_SECTION_IFSET(CPU_FTR_SPEC7450)
 
 	/* TODO: use HW flush assist when available */

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

> I read the MPC7450 reference manual and it describes a method to flush
> the cache using L2 hardware flushing feature. The cache flushing
> procedure consist of several steps and some of them are dictated by
> MPC7448 errata (www.freescale.com/files/32bit/doc/errata/MPC7448CE.pdf,
> Erratum no.3).

Are these errata 7448-only?  If not, I wonder what is
done on PowerMacs?

> First, I'm looking for a help and advice why the current _set_L2CR()
> implementation may not work for MPC7450 (namely 7448 with 1Mb L2 cache
> installed). Is it a bug in _set_L2CR()  or a hardware problem.

I think that if anyone here could answer this straight
away, the source code would have been fixed already ;-)

> I've
> mentioned above about MPC7450 hardware bug in L2 hardware flushing
> mechanism. May it be applicable to common cache flushing procedure 
> based
> on sequence of lwz/dcbf instructions?

Dunno, too lazy to download that PDF, perhaps you can
quote the relevant part?

> Second, Is this patch acceptable?

Looks reasonable enough to me...  if it works (on all
things considered "7450" by the kernel).

>  	/* TODO: use HW flush assist when available */

You want to get rid of this old comment though -- and
perhaps branch over the non-hardware-assisted cache
flushing code.


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Segher Boessenkool wrote:

>> I read the MPC7450 reference manual and it describes a method to flush
>> the cache using L2 hardware flushing feature. The cache flushing
>> procedure consist of several steps and some of them are dictated by
>> MPC7448 errata (www.freescale.com/files/32bit/doc/errata/MPC7448CE.pdf,
>> Erratum no.3).
>
>
> Are these errata 7448-only?  If not, I wonder what is
> done on PowerMacs?


This is a errata for 7448 only. I've looked through errata for other 
7450 processors (7450, 7457) and they contain the same erratum for L2 
cache: "L2 hardware flush may not flush every line from the L2 cache" 
The workaround for this problem is: "Set the IONLY and DONLY bits in the 
L2CR prior to the L2 hardware flush", and the projected solution is: 
"The workaround has been documented in the MPC7450 RISC Microprocessor 
Family User’s Manual as the correct way to flush the L2 cache"

I have tried to apply the same workaround for a common 
non-hardware-assistant _set_L2CR() implementation. I tried to completely 
lock the cache before a whole flushing procedure, tried to set the L2DO 
before filling the cache, fill it and then set the L2IO and issue a 
number of dcbf's. But nothing helped.

>
>> First, I'm looking for a help and advice why the current _set_L2CR()
>> implementation may not work for MPC7450 (namely 7448 with 1Mb L2 cache
>> installed). Is it a bug in _set_L2CR()  or a hardware problem.
>
>
> I think that if anyone here could answer this straight
> away, the source code would have been fixed already ;-)

Yeah, I guess I'm a first person who encountered this problem.

>
>> I've
>> mentioned above about MPC7450 hardware bug in L2 hardware flushing
>> mechanism. May it be applicable to common cache flushing procedure based
>> on sequence of lwz/dcbf instructions?
>
>
> Dunno, too lazy to download that PDF, perhaps you can
> quote the relevant part?
>
>> Second, Is this patch acceptable?
>
>
> Looks reasonable enough to me...  if it works (on all
> things considered "7450" by the kernel).

I've double checked this. All processors considered 7450 in the kernel 
are covered by MPC7450 RISC Microprocessor Family Reference Manual where 
hardware cache flushing procedure is described.

>
>>      /* TODO: use HW flush assist when available */
>
>
> You want to get rid of this old comment though -- and
> perhaps branch over the non-hardware-assisted cache
> flushing code.

Ok, I agree that the comment is obsolete now. Would you please explain 
why the branch over non-hardware-assisted code should be removed as 
well. Technically the cache is flushed and there is no need to use extra 
commands to fill and then re-flush the cache.

Thank you,
Vlad.

>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Mark A. Greer]

On Thu, Jun 14, 2007 at 09:12:11PM +0400, Vladislav Buzov wrote:
> Segher Boessenkool wrote:

> >>      /* TODO: use HW flush assist when available */
> >
> >
> > You want to get rid of this old comment though -- and
> > perhaps branch over the non-hardware-assisted cache
> > flushing code.
> 
> Ok, I agree that the comment is obsolete now. Would you please explain 
> why the branch over non-hardware-assisted code should be removed as 
> well. Technically the cache is flushed and there is no need to use extra 
> commands to fill and then re-flush the cache.

I think Segher is saying that you can skip the manual invalidation too
(although he said "flushing", I think he really meant "invalidation"--
the manual flushing is already skipped).  If I'm reading the manual
correctly, L2HWF does the invalidation as well so both the manual
flushing and invalidation can be skipped.

Mark

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>> Are these errata 7448-only?  If not, I wonder what is
>> done on PowerMacs?
>
> This is a errata for 7448 only. I've looked through errata for other=20=

> 7450 processors (7450, 7457) and they contain the same erratum for L2=20=

> cache: "L2 hardware flush may not flush every line from the L2 cache"=20=

> The workaround for this problem is: "Set the IONLY and DONLY bits in=20=

> the L2CR prior to the L2 hardware flush", and the projected solution=20=

> is: "The workaround has been documented in the MPC7450 RISC=20
> Microprocessor Family User=92s Manual as the correct way to flush the =
L2=20
> cache"

Okay, so it is for all members of the 7450 family.  Good :-)

>> Looks reasonable enough to me...  if it works (on all
>> things considered "7450" by the kernel).
>
> I've double checked this. All processors considered 7450 in the kernel=20=

> are covered by MPC7450 RISC Microprocessor Family Reference Manual=20
> where hardware cache flushing procedure is described.

Nice.  Now all that remains to be done is for the patch to
be tested on actual hardware for at least a few of those
other CPUs.  Any volunteers?

>>>      /* TODO: use HW flush assist when available */
>>
>> You want to get rid of this old comment though -- and
>> perhaps branch over the non-hardware-assisted cache
>> flushing code.
>
> Ok, I agree that the comment is obsolete now. Would you please explain=20=

> why the branch over non-hardware-assisted code should be removed as=20
> well. Technically the cache is flushed and there is no need to use=20
> extra commands to fill and then re-flush the cache.

I said such a branch should be added, not removed -- it
appears from your reply that you are skipping it already
and I just missed it?


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>>>>      /* TODO: use HW flush assist when available */
>>>
>>> You want to get rid of this old comment though -- and
>>> perhaps branch over the non-hardware-assisted cache
>>> flushing code.
>>
>> Ok, I agree that the comment is obsolete now. Would you please explain
>> why the branch over non-hardware-assisted code should be removed as
>> well. Technically the cache is flushed and there is no need to use 
>> extra
>> commands to fill and then re-flush the cache.
>
> I think Segher is saying that you can skip the manual invalidation too
> (although he said "flushing", I think he really meant "invalidation"--
> the manual flushing is already skipped).

Erm yes.

> If I'm reading the manual
> correctly, L2HWF does the invalidation as well so both the manual
> flushing and invalidation can be skipped.

Yeah.  Not that it really matters of course.


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav D. Buzov]

Segher Boessenkool wrote:
>>> Are these errata 7448-only?  If not, I wonder what is
>>> done on PowerMacs?
>>
>> This is a errata for 7448 only. I've looked through errata for other 
>> 7450 processors (7450, 7457) and they contain the same erratum for L2 
>> cache: "L2 hardware flush may not flush every line from the L2 cache" 
>> The workaround for this problem is: "Set the IONLY and DONLY bits in 
>> the L2CR prior to the L2 hardware flush", and the projected solution 
>> is: "The workaround has been documented in the MPC7450 RISC 
>> Microprocessor Family User’s Manual as the correct way to flush the 
>> L2 cache"
>
> Okay, so it is for all members of the 7450 family.  Good :-)
>
>>> Looks reasonable enough to me...  if it works (on all
>>> things considered "7450" by the kernel).
>>
>> I've double checked this. All processors considered 7450 in the 
>> kernel are covered by MPC7450 RISC Microprocessor Family Reference 
>> Manual where hardware cache flushing procedure is described.
>
> Nice.  Now all that remains to be done is for the patch to
> be tested on actual hardware for at least a few of those
> other CPUs.  Any volunteers?
Unfortunately I have only 7448 on hand :(
>
>>>>      /* TODO: use HW flush assist when available */
>>>
>>> You want to get rid of this old comment though -- and
>>> perhaps branch over the non-hardware-assisted cache
>>> flushing code.
>>
>> Ok, I agree that the comment is obsolete now. Would you please 
>> explain why the branch over non-hardware-assisted code should be 
>> removed as well. Technically the cache is flushed and there is no 
>> need to use extra commands to fill and then re-flush the cache.
>
> I said such a branch should be added, not removed -- it
> appears from your reply that you are skipping it already
> and I just missed it?
I have a branch over a manual cache flushing code in the patch:

+	b	2f
 END_FTR_SECTION_IFSET(CPU_FTR_SPEC7450)

That is a branch to the next step after cache flushing - cache disabling and setting up all the L2CR configuration bits passed to _set_L2CR()

Vlad.


>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav D. Buzov]

Segher Boessenkool wrote:
>>>>>      /* TODO: use HW flush assist when available */
>>>>
>>>> You want to get rid of this old comment though -- and
>>>> perhaps branch over the non-hardware-assisted cache
>>>> flushing code.
>>>
>>> Ok, I agree that the comment is obsolete now. Would you please explain
>>> why the branch over non-hardware-assisted code should be removed as
>>> well. Technically the cache is flushed and there is no need to use 
>>> extra
>>> commands to fill and then re-flush the cache.
>>
>> I think Segher is saying that you can skip the manual invalidation too
>> (although he said "flushing", I think he really meant "invalidation"--
>> the manual flushing is already skipped).
>
> Erm yes.
>
>> If I'm reading the manual
>> correctly, L2HWF does the invalidation as well so both the manual
>> flushing and invalidation can be skipped.
>
> Yeah.  Not that it really matters of course.
Hrm. I don't think that cache invalidation should be skipped. It is done 
after _set_L2CR() explicitly disabled the cache, in part of cache 
enabling procedure. Note that cache is flushed only if _set_L2CR() is 
called for already enabled cache. So, to skip cache invalidation there 
is a need to somehow track whether the cache has been 
flushed/invalidated before disabling or not. Since the manual 
invalidation does not break anything I think it is better to leave it as 
is rather than overload a _set_L2CR() logic.

Vlad.
>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>>> I've double checked this. All processors considered 7450 in the 
>>> kernel are covered by MPC7450 RISC Microprocessor Family Reference 
>>> Manual where hardware cache flushing procedure is described.
>>
>> Nice.  Now all that remains to be done is for the patch to
>> be tested on actual hardware for at least a few of those
>> other CPUs.  Any volunteers?

> Unfortunately I have only 7448 on hand :(

I was calling out for other list members to test your
patch :-)


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

> Hrm. I don't think that cache invalidation should be skipped. It is 
> done after _set_L2CR() explicitly disabled the cache, in part of cache 
> enabling procedure. Note that cache is flushed only if _set_L2CR() is 
> called for already enabled cache. So, to skip cache invalidation there 
> is a need to somehow track whether the cache has been 
> flushed/invalidated before disabling or not. Since the manual 
> invalidation does not break anything I think it is better to leave it 
> as is rather than overload a _set_L2CR() logic.

Yeah just leave it, it definitely is the safer thing to do.
Do fix the comment though ;-)


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav D. Buzov]

Segher Boessenkool wrote:
>> Hrm. I don't think that cache invalidation should be skipped. It is 
>> done after _set_L2CR() explicitly disabled the cache, in part of 
>> cache enabling procedure. Note that cache is flushed only if 
>> _set_L2CR() is called for already enabled cache. So, to skip cache 
>> invalidation there is a need to somehow track whether the cache has 
>> been flushed/invalidated before disabling or not. Since the manual 
>> invalidation does not break anything I think it is better to leave it 
>> as is rather than overload a _set_L2CR() logic.
>
> Yeah just leave it, it definitely is the safer thing to do.
> Do fix the comment though ;-)
Once somebody tests the patch on other 7450 processors I'll remove the 
comment and send a new patch. Is it ok? Or I should create a new patch 
first?

Vlad.
>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>> Do fix the comment though ;-)
> Once somebody tests the patch on other 7450 processors I'll remove the 
> comment and send a new patch. Is it ok? Or I should create a new patch 
> first?

No, this is perfectly fine.  Thanks!


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Mark A. Greer]

On Fri, Jun 15, 2007 at 10:14:24AM +0200, Segher Boessenkool wrote:

> > I've double checked this. All processors considered 7450 in the kernel 
> > are covered by MPC7450 RISC Microprocessor Family Reference Manual 
> > where hardware cache flushing procedure is described.
> 
> Nice.  Now all that remains to be done is for the patch to
> be tested on actual hardware for at least a few of those
> other CPUs.  Any volunteers?

I just booted & played around on a prpmc2800e (7447A, 512MB) with
this patch and it seemed to work fine.

I *may* have some old sandpoint 745x modules that still work but
I'll have to get the sandpoint working in arch/powerpc again.
I'll try to get to that in a week or so.

Mark

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>>> I've double checked this. All processors considered 7450 in the 
>>> kernel
>>> are covered by MPC7450 RISC Microprocessor Family Reference Manual
>>> where hardware cache flushing procedure is described.
>>
>> Nice.  Now all that remains to be done is for the patch to
>> be tested on actual hardware for at least a few of those
>> other CPUs.  Any volunteers?
>
> I just booted & played around on a prpmc2800e (7447A, 512MB) with
> this patch and it seemed to work fine.

Good to hear, thank you!

> I *may* have some old sandpoint 745x modules that still work but
> I'll have to get the sandpoint working in arch/powerpc again.
> I'll try to get to that in a week or so.

Don't sweat it, I'm sure someone with a powermac 7450
or 7455 will show up soon enough.  Any takers?


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Benjamin Herrenschmidt]

On Fri, 2007-06-15 at 14:20 -0700, Mark A. Greer wrote:
> On Fri, Jun 15, 2007 at 10:14:24AM +0200, Segher Boessenkool wrote:
> 
> > > I've double checked this. All processors considered 7450 in the kernel 
> > > are covered by MPC7450 RISC Microprocessor Family Reference Manual 
> > > where hardware cache flushing procedure is described.
> > 
> > Nice.  Now all that remains to be done is for the patch to
> > be tested on actual hardware for at least a few of those
> > other CPUs.  Any volunteers?
> 
> I just booted & played around on a prpmc2800e (7447A, 512MB) with
> this patch and it seemed to work fine.
> 
> I *may* have some old sandpoint 745x modules that still work but
> I'll have to get the sandpoint working in arch/powerpc again.
> I'll try to get to that in a week or so.

The thing that needs torture is the powerbook sleep/wakeup code. That's
where all the cache flushing issues have been biting in the past.

Ben.

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Benjamin Herrenschmidt wrote:

>On Fri, 2007-06-15 at 14:20 -0700, Mark A. Greer wrote:
>  
>
>>On Fri, Jun 15, 2007 at 10:14:24AM +0200, Segher Boessenkool wrote:
>>
>>    
>>
>>>>I've double checked this. All processors considered 7450 in the kernel 
>>>>are covered by MPC7450 RISC Microprocessor Family Reference Manual 
>>>>where hardware cache flushing procedure is described.
>>>>        
>>>>
>>>Nice.  Now all that remains to be done is for the patch to
>>>be tested on actual hardware for at least a few of those
>>>other CPUs.  Any volunteers?
>>>      
>>>
>>I just booted & played around on a prpmc2800e (7447A, 512MB) with
>>this patch and it seemed to work fine.
>>
>>I *may* have some old sandpoint 745x modules that still work but
>>I'll have to get the sandpoint working in arch/powerpc again.
>>I'll try to get to that in a week or so.
>>    
>>
>
>The thing that needs torture is the powerbook sleep/wakeup code. That's
>where all the cache flushing issues have been biting in the past.
>
>Ben.
>  
>
Note that 745x processors have L3 cache installed and may have the same 
problem requiring similar code modifications to use L3 hardware flushing 
mechanism.
So, my patch may not cure the cache flushing problem at all on such 
processors.

Vlad.

>
>_______________________________________________
>Linuxppc-dev mailing list
>Linuxppc-dev@ozlabs.org
>https://ozlabs.org/mailman/listinfo/linuxppc-dev
>  
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

> Note that 745x processors have L3 cache installed and may have the same
> problem requiring similar code modifications to use L3 hardware 
> flushing
> mechanism.

What does the erratum say?

The L3 is a very different beast from the L2, IIRC it is
a pure victim cache so it cannot have this problem at all?


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Segher Boessenkool wrote:

>> Note that 745x processors have L3 cache installed and may have the same
>> problem requiring similar code modifications to use L3 hardware flushing
>> mechanism.
>
>
> What does the erratum say?

The erratum says nothing about any HW bugs with L3 cache flush. I just 
mentioned that the L3 cache flush operation described in MPC7450 
Reference manual is similar to the L2 using the L3 cache hardware 
flushing mechanism. For instance, it requires a complete L3 locking 
before flushing.

>
> The L3 is a very different beast from the L2, IIRC it is
> a pure victim cache so it cannot have this problem at all?

I'm not sure if it is a pure victim cache. I read the MPC7450 reference 
manual and see that the L3 cache operates similarly to the L2. The main 
difference between those caches is that L3 uses an external SRAM memory 
while L2 is a pure on-chip cache.

Vlad.

>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Segher Boessenkool wrote:

>> I read the MPC7450 reference manual and it describes a method to flush
>> the cache using L2 hardware flushing feature. The cache flushing
>> procedure consist of several steps and some of them are dictated by
>> MPC7448 errata (www.freescale.com/files/32bit/doc/errata/MPC7448CE.pdf,
>> Erratum no.3).
>
>
> Are these errata 7448-only?  If not, I wonder what is
> done on PowerMacs?
>
>> First, I'm looking for a help and advice why the current _set_L2CR()
>> implementation may not work for MPC7450 (namely 7448 with 1Mb L2 cache
>> installed). Is it a bug in _set_L2CR()  or a hardware problem.
>
>
> I think that if anyone here could answer this straight
> away, the source code would have been fixed already ;-)

I think I can try to answer this question. Please, look through my 
thoughts below and correct me if I'm somewhere wrong.

The current scheme of flushing the caches is based on a number of 
consecutive lwz/dcbf instructions. A contiguous memory region (started 
from zero) is read by series of lwz commands and then cache is flushed 
using a sequence of dcbf instructions with addresses from this memory 
range. If I understand correctly, to get this approach working it is 
required to guarantee that after reading the memory region, each line in 
a cache should be used and keep data from this region. Otherwise, if 
some cache lines keep data from another address range they will not be 
flushed by the dcbf instructions sequence.

Further, how cache lines are utilized is dictated by a cache lines 
replacement policy. I didn't go in to details deeply, but on MPC7450 L1 
cache lines replacement policy seems to  satisfy the requirement above. 
At least the MPC7450 reference manual describes L1 cache flushing 
algorithm based on a sequence of lwz/dcbf instructions.

But regarding to L2/L3 caches, the manual describes two different cache 
line replacement policies. And the both are pseudo-random and differ by 
implementation of random number generator. It means that a cache line in 
a set is chosen randomly, and that, in turn, means that there is a 
probability that some cache lines are not used during reading of the 
contiguous memory region and not flushed by the dcbf instruction sequence.

For example, on MPC7448 there is a eight-way set-associative 1Mb L2 
cache that consist of 2048 sets x 8 ways per set. And even if a set N 
has been accessed M times (M > 8) there is a chance that some cache line 
is set N has never been used, but another line has been used twice or 
more. Of course, the probability of such situation decreases with 
increasing of N.

Current _set_L2CR() implementation reads first 4Mb of memory to flush 
the L2 cache. I have increased this size up to 16 Mb and now things work 
fine. But I don't think that is a right way to fix the problem because 
there is no any way to define the upper limit of memory size to 
guarantee flushing of each cache line. 16Mb is too large though. It 
seems more reasonable to use a stable and guaranteed way to flush the 
cache implemented in hardware.

Vlad.

>
>> I've
>> mentioned above about MPC7450 hardware bug in L2 hardware flushing
>> mechanism. May it be applicable to common cache flushing procedure based
>> on sequence of lwz/dcbf instructions?
>
>
> Dunno, too lazy to download that PDF, perhaps you can
> quote the relevant part?
>
>> Second, Is this patch acceptable?
>
>
> Looks reasonable enough to me...  if it works (on all
> things considered "7450" by the kernel).
>
>>      /* TODO: use HW flush assist when available */
>
>
> You want to get rid of this old comment though -- and
> perhaps branch over the non-hardware-assisted cache
> flushing code.
>
>
> Segher
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>>> Note that 745x processors have L3 cache installed and may have the 
>>> same
>>> problem requiring similar code modifications to use L3 hardware 
>>> flushing
>>> mechanism.
>>
>> What does the erratum say?
>
> The erratum says nothing about any HW bugs with L3 cache flush. I just 
> mentioned that the L3 cache flush operation described in MPC7450 
> Reference manual is similar to the L2 using the L3 cache hardware 
> flushing mechanism. For instance, it requires a complete L3 locking 
> before flushing.

Then I think we should use that mechanism in the Linux kernel.
Anything else is waiting for bugs to bite.

>> The L3 is a very different beast from the L2, IIRC it is
>> a pure victim cache so it cannot have this problem at all?
>
> I'm not sure if it is a pure victim cache. I read the MPC7450 
> reference manual and see that the L3 cache operates similarly to the 
> L2. The main difference between those caches is that L3 uses an 
> external SRAM memory while L2 is a pure on-chip cache.

Yeah but it isn't involved in any prefetching AFAIK.  Anyway,
moot point, since we really should do the recommended flush
algorithm (esp. since we clearly do not sufficiently understand
the details of the L3 operation!)


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Segher Boessenkool]

>>> First, I'm looking for a help and advice why the current _set_L2CR()
>>> implementation may not work for MPC7450 (namely 7448 with 1Mb L2 
>>> cache
>>> installed). Is it a bug in _set_L2CR()  or a hardware problem.
>>
>> I think that if anyone here could answer this straight
>> away, the source code would have been fixed already ;-)
>
> I think I can try to answer this question. Please, look through my 
> thoughts below and correct me if I'm somewhere wrong.

You forgot step 0: the goal of flushing the caches here
is to make sure there is no data at all in there after it
has finished.

> The current scheme of flushing the caches is based on a number of 
> consecutive lwz/dcbf instructions. A contiguous memory region (started 
> from zero) is read by series of lwz commands and then cache is flushed 
> using a sequence of dcbf instructions with addresses from this memory 
> range. If I understand correctly, to get this approach working it is 
> required to guarantee that after reading the memory region, each line 
> in a cache should be used and keep data from this region. Otherwise, 
> if some cache lines keep data from another address range they will not 
> be flushed by the dcbf instructions sequence.

Yes, you need to ensure there is nothing interfering (SMP
agents, DMA agents, prefetch engines...), and you need to
know the line replacement policy, to make this work;
furthermore, you need to be quite careful in your code to
make sure the intended L2 stores are the _only_ L2 traffic
you generate.

> Further, how cache lines are utilized is dictated by a cache lines 
> replacement policy. I didn't go in to details deeply, but on MPC7450 
> L1 cache lines replacement policy seems to  satisfy the requirement 
> above. At least the MPC7450 reference manual describes L1 cache 
> flushing algorithm based on a sequence of lwz/dcbf instructions.
>
> But regarding to L2/L3 caches, the manual describes two different 
> cache line replacement policies. And the both are pseudo-random

At least on is the "standard" PowerPC pseudo-LRU tree, no?
That one flushes fine using this strategy.

> and differ by implementation of random number generator. It means that 
> a cache line in a set is chosen randomly, and that, in turn, means 
> that there is a probability that some cache lines are not used during 
> reading of the contiguous memory region and not flushed by the dcbf 
> instruction sequence.

Knowing that there is no "outside" interference, and knowing
the "random" algorithm, can give plenty guarantees.

> For example, on MPC7448 there is a eight-way set-associative 1Mb L2 
> cache that consist of 2048 sets x 8 ways per set. And even if a set N 
> has been accessed M times (M > 8) there is a chance that some cache 
> line is set N has never been used, but another line has been used 
> twice or more. Of course, the probability of such situation decreases 
> with increasing of N.

You can make sure, too.  Just trying to statistically get
to the point where you are sure the whole cache is flushed
is not going to work *at all*, you need to use deeper knowledge
of how the cache works.

> Current _set_L2CR() implementation reads first 4Mb of memory to flush 
> the L2 cache. I have increased this size up to 16 Mb and now things 
> work fine. But I don't think that is a right way to fix the problem 
> because there is no any way to define the upper limit of memory size 
> to guarantee flushing of each cache line. 16Mb is too large though. It 
> seems more reasonable to use a stable and guaranteed way to flush the 
> cache implemented in hardware.

Yes, use the hardware flush mechanism.  Please :-)

[I think the erratum is about insn fetches to L2 that you
have no way too stop.  <handwaving>Something like that,
anyway.</handwaving>]


Segher

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Benjamin Herrenschmidt]

> > The erratum says nothing about any HW bugs with L3 cache flush. I just 
> > mentioned that the L3 cache flush operation described in MPC7450 
> > Reference manual is similar to the L2 using the L3 cache hardware 
> > flushing mechanism. For instance, it requires a complete L3 locking 
> > before flushing.
> 
> Then I think we should use that mechanism in the Linux kernel.
> Anything else is waiting for bugs to bite.

I just figured out ... we actually already have all of that cache flush
code :-) I wrote most of it in fact. It's just that for some (bad)
reasons, it's somewhat hidden in arch/powerpc/platforms/powermac/cache.S

So I think best would be to take it from there and make it more
generic ...

Cheers,
Ben.

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Benjamin Herrenschmidt wrote:

>>>The erratum says nothing about any HW bugs with L3 cache flush. I just 
>>>mentioned that the L3 cache flush operation described in MPC7450 
>>>Reference manual is similar to the L2 using the L3 cache hardware 
>>>flushing mechanism. For instance, it requires a complete L3 locking 
>>>before flushing.
>>>      
>>>
>>Then I think we should use that mechanism in the Linux kernel.
>>Anything else is waiting for bugs to bite.
>>    
>>
>
>I just figured out ... we actually already have all of that cache flush
>code :-) I wrote most of it in fact. It's just that for some (bad)
>reasons, it's somewhat hidden in arch/powerpc/platforms/powermac/cache.S
>
>So I think best would be to take it from there and make it more
>generic ...
>  
>
I'm just wondering who is supposed to do that. I can copy the code from 
cache.S to l2cr_6xx.S and test it on 7448 since I have only this 
processor on hand. So, I can't test the L3.      

I've looked through cache.S and see it contains a dcbf loop over 4Mb 
along with L2, L3 HW cache flushing. The comments says 'Due to a bug 
with the HW flush on some CPU revs, we occasionally experience data 
corruption...'. Could you please clarify which CPU revisions have this 
bug and whether it is the same bug described in errata and requiring a 
complete cache locking before flushing? Do we still need to use the dcbf 
loop in _set_L2CR() for MPC7450 processors?

Thanks,
Vlad.

>Cheers,
>Ben.
>
>
>  
>

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Benjamin Herrenschmidt]

On Thu, 2007-06-28 at 14:47 +0400, Vladislav Buzov wrote:
> 
> I've looked through cache.S and see it contains a dcbf loop over 4Mb 
> along with L2, L3 HW cache flushing. The comments says 'Due to a bug 
> with the HW flush on some CPU revs, we occasionally experience data 
> corruption...'. Could you please clarify which CPU revisions have this 
> bug and whether it is the same bug described in errata and requiring a 
> complete cache locking before flushing? Do we still need to use the dcbf 
> loop in _set_L2CR() for MPC7450 processors?

Can't remember ... that code is pretty old now.

Cheers,
Ben.

Re: [RFC/PATCH] powerpc: MPC7450 L2 HW cache flush feature utilization

[Vladislav Buzov]

Segher Boessenkool wrote:

>>>> First, I'm looking for a help and advice why the current _set_L2CR()
>>>> implementation may not work for MPC7450 (namely 7448 with 1Mb L2 cache
>>>> installed). Is it a bug in _set_L2CR()  or a hardware problem.
>>>
>>>
>>> I think that if anyone here could answer this straight
>>> away, the source code would have been fixed already ;-)
>>
>>
>> I think I can try to answer this question. Please, look through my 
>> thoughts below and correct me if I'm somewhere wrong.
>
>
> You forgot step 0: the goal of flushing the caches here
> is to make sure there is no data at all in there after it
> has finished.
>
>> The current scheme of flushing the caches is based on a number of 
>> consecutive lwz/dcbf instructions. A contiguous memory region 
>> (started from zero) is read by series of lwz commands and then cache 
>> is flushed using a sequence of dcbf instructions with addresses from 
>> this memory range. If I understand correctly, to get this approach 
>> working it is required to guarantee that after reading the memory 
>> region, each line in a cache should be used and keep data from this 
>> region. Otherwise, if some cache lines keep data from another address 
>> range they will not be flushed by the dcbf instructions sequence.
>
>
> Yes, you need to ensure there is nothing interfering (SMP
> agents, DMA agents, prefetch engines...), and you need to
> know the line replacement policy, to make this work;
> furthermore, you need to be quite careful in your code to
> make sure the intended L2 stores are the _only_ L2 traffic
> you generate.
>
>> Further, how cache lines are utilized is dictated by a cache lines 
>> replacement policy. I didn't go in to details deeply, but on MPC7450 
>> L1 cache lines replacement policy seems to  satisfy the requirement 
>> above. At least the MPC7450 reference manual describes L1 cache 
>> flushing algorithm based on a sequence of lwz/dcbf instructions.
>>
>> But regarding to L2/L3 caches, the manual describes two different 
>> cache line replacement policies. And the both are pseudo-random
>
>
> At least on is the "standard" PowerPC pseudo-LRU tree, no?
> That one flushes fine using this strategy.

The PLRU is implemented in the the L1 cache only, And yes, it does allow 
to flush the cache using the lwz/dcbf sequence.

The L2/L3 caches use two different pseudo-random number generators based 
on CPU clocks. The first one is just 3-bit modulo 8 counter incremented 
on every clock cycle. On each cache miss it is used to choice a cache 
line in a set. The second is more complicated and consists of 16 latches 
clocked on every clock cycle with 3 XOR functions. And L2/L3 take 
"random" numbers from 3 latches of 16.

>
>> and differ by implementation of random number generator. It means 
>> that a cache line in a set is chosen randomly, and that, in turn, 
>> means that there is a probability that some cache lines are not used 
>> during reading of the contiguous memory region and not flushed by the 
>> dcbf instruction sequence.
>
>
> Knowing that there is no "outside" interference, and knowing
> the "random" algorithm, can give plenty guarantees.

Agree. In case of cache flushing there is no interference and all the 
instructions use known numbers of clock cycles, so it is possible (at 
least in theory) to predict all the "random" numbers and to say whether 
all the cache lines will be used or not in each particular case.  But 
the nature of the algorithms does not guarantee the preferable result in 
general.

>
>> For example, on MPC7448 there is a eight-way set-associative 1Mb L2 
>> cache that consist of 2048 sets x 8 ways per set. And even if a set N 
>> has been accessed M times (M > 8) there is a chance that some cache 
>> line is set N has never been used, but another line has been used 
>> twice or more. Of course, the probability of such situation decreases 
>> with increasing of N.
>
Opps, misprint.. I meant M in last sentence :-)

Thank you,
Vlad.

>
> You can make sure, too.  Just trying to statistically get
> to the point where you are sure the whole cache is flushed
> is not going to work *at all*, you need to use deeper knowledge
> of how the cache works.
>
>> Current _set_L2CR() implementation reads first 4Mb of memory to flush 
>> the L2 cache. I have increased this size up to 16 Mb and now things 
>> work fine. But I don't think that is a right way to fix the problem 
>> because there is no any way to define the upper limit of memory size 
>> to guarantee flushing of each cache line. 16Mb is too large though. 
>> It seems more reasonable to use a stable and guaranteed way to flush 
>> the cache implemented in hardware.
>
>
> Yes, use the hardware flush mechanism.  Please :-)
>
> [I think the erratum is about insn fetches to L2 that you
> have no way too stop.  <handwaving>Something like that,
> anyway.</handwaving>]
>
>
> Segher
>