Problems in 2.6 memory management on 8xx

Problems in 2.6 memory management on 8xx

[Detlev Zundel]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

Hi,

working on a 2.6.16 kernel on a 870 CPU, I ran into this strange
behaviour exemplified by the simple attached demo program.  An icbi
from userspace on an address that is mapped only lazily gets into an -
though interruptible - loop. Locking the icbi target in question with
mlock circumvents this problem.

I tested this code on 2.6.21 on 4xx and 82xx only to prove that those
combinations, as expected, don't have this problem.  Vitaly Bordug was
kind enough to test the code on an 8xx hw supported by a recent kernel
and acknowledged it still being present.

As I don't have time to investigate this any further, I at least want
to document this problem with this post.  Maybe someone else can help
out here.

Thanks
  Detlev

-- 
-- Question authority!
-- Yeah, says who?
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu@denx.de

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: icbi.c --]
[-- Type: text/x-csrc, Size: 437 bytes --]

/* Test code to show problem with handling of icbi in userspace on
   PowerPC in Linux 2.6 */
#include <stdio.h>
#include <unistd.h>

extern void test();

int main(int argc, char *argv[])
{
#if 0
	/* Lock the text segment of the test routine so that the pages will
	   be present in RAM - otherwise the ICBI will not work */
	mlock((void *)&test, 3*4096);
#endif
	test();
	printf("move on - there's nothing to see here\n");
	return 0;
}

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: icbi_trigger.S --]
[-- Type: text/x-csrc, Size: 463 bytes --]

	.globl	test

/* Align to MMU page size, i.e. 2^12 = 4k */
	.align 12
page0:	
test:
	mflr	%r20	

	/* invalidate first cache line at page1 */
	lis	%r24, page0@h
	ori	%r24, %r24, page0@l
	li	%r23, 0x1000
	icbi	%r24, %r23

	b	test_return

	.org	page0+4096
/*
 page1 is on a new 4k page - only fill it with "mtmq r0" (sigill)
*/
page1:

/* Ensure that page2 again is on a new 4k page */
        .org	page1+4096
page2:

test_return:
	mtlr	%r20			/* restore lr */
	blr

[-- Attachment #4: Makefile --]
[-- Type: text/plain, Size: 151 bytes --]

CC=$(CROSS_COMPILE)gcc
AS=$(CROSS_COMPILE)as
CFLAGS=-g

%.o:	%.S
	$(CC) -c -o $@ $<	

all:	icbi

icbi:	icbi.o icbi_trigger.o

clean:
	rm -f *.o *~ icbi

Re: Problems in 2.6 memory management on 8xx

[Joakim Tjernlund]

On Thu, 2007-05-24 at 15:07 +0200, Detlev Zundel wrote:
> Hi,
> 
> working on a 2.6.16 kernel on a 870 CPU, I ran into this strange
> behaviour exemplified by the simple attached demo program.  An icbi
> from userspace on an address that is mapped only lazily gets into an -
> though interruptible - loop. Locking the icbi target in question with
> mlock circumvents this problem.

8xx is buggy w.r.t cache instructions. They do not update the
DAR register in the TLB miss/TLB error handlers.
The TLB miss handler does not use the DAR reg but the TLB error
handler do. Thats why it works when you mlock the memory.

This bug isn't documented but Freescale has confirmed it.
You can search the archives some years back for more info.

 Jocke

> 
> I tested this code on 2.6.21 on 4xx and 82xx only to prove that those
> combinations, as expected, don't have this problem.  Vitaly Bordug was
> kind enough to test the code on an 8xx hw supported by a recent kernel
> and acknowledged it still being present.
> 
> As I don't have time to investigate this any further, I at least want
> to document this problem with this post.  Maybe someone else can help
> out here.
> 
> Thanks
>   Detlev
> 
> plain text document attachment (Makefile)
> CC=$(CROSS_COMPILE)gcc
> AS=$(CROSS_COMPILE)as
> CFLAGS=-g
> 
> %.o:	%.S
> 	$(CC) -c -o $@ $<	
> 
> all:	icbi
> 
> icbi:	icbi.o icbi_trigger.o
> 
> clean:
> 	rm -f *.o *~ icbi
> _______________________________________________ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev

Re: Problems in 2.6 memory management on 8xx

[Joakim Tjernlund]

On Thu, 2007-05-24 at 17:23 +0200, Joakim Tjernlund wrote:
> On Thu, 2007-05-24 at 15:07 +0200, Detlev Zundel wrote:
> > Hi,
> > 
> > working on a 2.6.16 kernel on a 870 CPU, I ran into this strange
> > behaviour exemplified by the simple attached demo program.  An icbi
> > from userspace on an address that is mapped only lazily gets into an -
> > though interruptible - loop. Locking the icbi target in question with
> > mlock circumvents this problem.
> 
> 8xx is buggy w.r.t cache instructions. They do not update the
> DAR register in the TLB miss/TLB error handlers.
> The TLB miss handler does not use the DAR reg but the TLB error
> handler do. Thats why it works when you mlock the memory.
> 
> This bug isn't documented but Freescale has confirmed it.
> You can search the archives some years back for more info.
> 
>  Jocke

BTW, it is possible to workaround this problem in the kernel by
tagging DAR with an impossible value and compare DAR against it
in the DTLB Error handler. If a match, then do a instruction decode
to get the regs involved and calculate the faulting address.

I did this several years ago for 2.4 in assembler and posted
it, but it was rejected.
One should bail out to handle_page_fault and do the
calculations there instead(less likely to break that way)

Found one version of the patch here:
http://patchwork.ozlabs.org/linuxppc/patch?id=1307

 Jocke

Re: Problems in 2.6 memory management on 8xx

[Detlev Zundel]

Hi Joakim,

> On Thu, 2007-05-24 at 17:23 +0200, Joakim Tjernlund wrote:
>> On Thu, 2007-05-24 at 15:07 +0200, Detlev Zundel wrote:
>> > Hi,
>> > 
>> > working on a 2.6.16 kernel on a 870 CPU, I ran into this strange
>> > behaviour exemplified by the simple attached demo program.  An icbi
>> > from userspace on an address that is mapped only lazily gets into an -
>> > though interruptible - loop. Locking the icbi target in question with
>> > mlock circumvents this problem.
>> 
>> 8xx is buggy w.r.t cache instructions. They do not update the
>> DAR register in the TLB miss/TLB error handlers.
>> The TLB miss handler does not use the DAR reg but the TLB error
>> handler do. Thats why it works when you mlock the memory.
>> 
>> This bug isn't documented but Freescale has confirmed it.
>> You can search the archives some years back for more info.
>> 
>>  Jocke
>
> BTW, it is possible to workaround this problem in the kernel by
> tagging DAR with an impossible value and compare DAR against it
> in the DTLB Error handler. If a match, then do a instruction decode
> to get the regs involved and calculate the faulting address.
>
> I did this several years ago for 2.4 in assembler and posted
> it, but it was rejected.
> One should bail out to handle_page_fault and do the
> calculations there instead(less likely to break that way)
>
> Found one version of the patch here:
> http://patchwork.ozlabs.org/linuxppc/patch?id=1307

Thanks for shedding some light on this problem.  I already found the
patch you refer to and also wondered why it was never accepted.  Was
there a technical reason or did it simply slip everybodys attention?

Cheers
  Detlev

-- 
Men are born ignorant, not stupid; they are made stupid by education.
					  --Bertrand Russell
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu@denx.de

RE: Problems in 2.6 memory management on 8xx

[Joakim Tjernlund]

> -----Original Message-----
> From: Detlev Zundel [mailto:dzu@denx.de] 
> Sent: den 24 maj 2007 18:46
> To: joakim.tjernlund@transmode.se
> Cc: linuxppc-dev@ozlabs.org
> Subject: Re: Problems in 2.6 memory management on 8xx
> 
> Hi Joakim,
> 
> > On Thu, 2007-05-24 at 17:23 +0200, Joakim Tjernlund wrote:
> >> On Thu, 2007-05-24 at 15:07 +0200, Detlev Zundel wrote:
> >> > Hi,
> >> > 
> >> > working on a 2.6.16 kernel on a 870 CPU, I ran into this strange
> >> > behaviour exemplified by the simple attached demo 
> program.  An icbi
> >> > from userspace on an address that is mapped only lazily 
> gets into an -
> >> > though interruptible - loop. Locking the icbi target in 
> question with
> >> > mlock circumvents this problem.
> >> 
> >> 8xx is buggy w.r.t cache instructions. They do not update the
> >> DAR register in the TLB miss/TLB error handlers.
> >> The TLB miss handler does not use the DAR reg but the TLB error
> >> handler do. Thats why it works when you mlock the memory.
> >> 
> >> This bug isn't documented but Freescale has confirmed it.
> >> You can search the archives some years back for more info.
> >> 
> >>  Jocke
> >
> > BTW, it is possible to workaround this problem in the kernel by
> > tagging DAR with an impossible value and compare DAR against it
> > in the DTLB Error handler. If a match, then do a instruction decode
> > to get the regs involved and calculate the faulting address.
> >
> > I did this several years ago for 2.4 in assembler and posted
> > it, but it was rejected.
> > One should bail out to handle_page_fault and do the
> > calculations there instead(less likely to break that way)
> >
> > Found one version of the patch here:
> > http://patchwork.ozlabs.org/linuxppc/patch?id=1307
> 
> Thanks for shedding some light on this problem.  I already found the
> patch you refer to and also wondered why it was never accepted.  Was
> there a technical reason or did it simply slip everybodys attention?

Can't really rember the details, but I think Dan Malek didn't
like it because it was hard to maintain and usally one can avoid
the problem.
I have been using that patch on our 860/862 boards for years now
and it works fine.

 Jocke