Follow up on 4 Gig of DDR on MPC8548E

Follow up on 4 Gig of DDR on MPC8548E

[Morrison, Tom]

All,

We are running into a very strange problem that I=20
am hoping someone here might have some insight upon.

Setup:
   1) 4 Gig of DDR RAM (at physical addresses 0-0xF_FFFF_FFFF)

   2) LTIB/Linux Kernel (2.6.11++) - I know we are behind the
	times, but it was too risky to go with latest/greatest.

   3) U-boot passes a Max MEM of 3 GIG to the kernel at boot...
	But, we are using on the bootcmd =3D=3D> mem=3D2 GIG + a few pages

Symptoms:
  a) 	Kernel boots - but fails / and hangs when it attempts to=20
	open the initial console:

>> EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended
>> VFS: Mounted root (ext2 filesystem).
>>  316k init
>> EXT2-fs error (device sda1): ext2_check_page: bad entry in directory
#2: >> unaligned directory entry - offset=3D0, inode=3D128,
rec_len=3D8961,name_len=3D69
>> Warning: unable to open an initial console.

--- a little debugging indicates that it is trying to open the      ----
--- /dev/console device - and one of first physical pages allocated ----
--- is above the 2 GIG boundary (0x80000000)                        ----

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3Dfurther =
tests=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
  =20
   b) if we pass a 'mem=3D2GIG' or less (even though 4 GIG of DDR) -=20
	it works correctly, and we boot up normally...

   d) Further, If we do #b, and ioremap a physical address=20
	above the 2GIG and read/write those addresses - it works.....
	=3D=3D> This tells me the memory is working...

My gut tells me this might be something to do with the 2=20
Gig boundary and specifically a "signed" versus "unsigned"=20
address/offsets mismatch maybe somewhere in the file system??

We are debugging this - but it is painful and slow process=20
to narrow the root problem - I am hoping somebody around=20
here has some knowledge of this type of failure and can help us!

All the Best~!

Tom Morrison
Principal S/W Engineer
Empirix, Inc (www.empirix.com)
tmorrison@empirix.com
(781) 266 - 3567

RE: Follow up on 4 Gig of DDR on MPC8548E

[Rune Torgersen]

> From: Morrison, Tom
> Sent: Friday, June 22, 2007 12:50 PM
> Setup:
>    1) 4 Gig of DDR RAM (at physical addresses 0-0xF_FFFF_FFFF)

0x0000_0000 to 0xffff_ffff or do you have 64GB of mem...

> >> EXT2-fs warning: mounting unchecked fs, running e2fsck is=20
> recommended
> >> VFS: Mounted root (ext2 filesystem).
> >>  316k init
> >> EXT2-fs error (device sda1): ext2_check_page: bad entry in=20
> directory
> #2: >> unaligned directory entry - offset=3D0, inode=3D128,
> rec_len=3D8961,name_len=3D69
> >> Warning: unable to open an initial console.

> My gut tells me this might be something to do with the 2=20
> Gig boundary and specifically a "signed" versus "unsigned"=20
> address/offsets mismatch maybe somewhere in the file system??

If you have all physical mem in the first 32bit, where are your PCI
windows set?
And in modst cases the PCI devices (if they are bus-mastring ) need 1-1
inbound mapping to be able to write to memory.

RE: Follow up on 4 Gig of DDR on MPC8548E

[Morrison, Tom]

>> 0x0000_0000 to 0xffff_ffff or do you have 64GB of mem...

4 GIG DDR Memory...that we say...there is 3 GIG for the moment...
=20
the LAW's are  setup in priority order - thus, if you have overlapping
regions (aka: PCI & DDR memory) the one with the lower #'d LAW
is the one it is mapped to....

>> If you have all physical mem in the first 32bit, where are your PCI
>> windows set?
>> And in modst cases the PCI devices (if they are bus-mastring ) need =
1-1
>> inbound mapping to be able to write to memory.

you can use the LAW's to remap anything to anywhere...
see above for priority - but in our case, we are mapping
the PCI/PEX/Local Bus to the last 4 GIG at a higher=20
priority LAW than the DDR...that is what the MPC8548=20
spec says how it works...and it does - if you don't tell
the kernel there is more than 2GIG (and ioremap that last
GIG to access the full 'other' 2 GIG...
=20
that is not the issue here....
=20
apparently after more investigations - it looks like there is something =
in the ext2 driver code
that is mal-adjusted....I haven't talked to the guy today who was =
looking at that - but the ext2
driver code that was openning a 'virtual file' / console - had some =
problems mapping that
space - again, my gut is telling me more stronger there is a problem =
with signed/unsigned...
now deeper in the ext2 code...

Am at the Freescale Technical Forum the next few days - if any of you =
guys
are down here in Florida...I am intending to track down a few freescale =
folk
on this issue...:-)
=20
Tom

RE: Follow up on 4 Gig of DDR on MPC8548E

[Benjamin Herrenschmidt]

> apparently after more investigations - it looks like there is something in the ext2 driver code
> that is mal-adjusted....I haven't talked to the guy today who was looking at that - but the ext2
> driver code that was openning a 'virtual file' / console - had some problems mapping that
> space - again, my gut is telling me more stronger there is a problem with signed/unsigned...
> now deeper in the ext2 code...

I very much doubt there is a signed/unsigned issue in ext2 or
elsewhere...

You have CONFIG_HIGHMEM ? What are your setting for KERNELBASE and
PAGE_OFFSET ?

Ben.

RE: Follow up on 4 Gig of DDR on MPC8548E

[Morrison, Tom]

>> apparently after more investigations - it looks like there is =
something in the ext2 driver code
>> that is mal-adjusted....I haven't talked to the guy today who was =
looking at that - but the ext2
>> driver code that was openning a 'virtual file' / console - had some =
problems mapping that
>> space - again, my gut is telling me more stronger there is a problem =
with signed/unsigned...
>> now deeper in the ext2 code...

> I very much doubt there is a signed/unsigned issue in ext2 or =
elsewhere...
>
> You have CONFIG_HIGHMEM ? What are your setting for KERNELBASE and =
PAGE_OFFSET ?

Ben,
=20
I am sorry it I alluded to ext2 code itself having a signed/unsigned =
problem -=20
just that the problem was manifesting itself inside the ext2 code in a =
very=20
strange way ...with something that I guessed below it that was violating =

a signed/unsigned API....
=20
and I was wrong with this hunch (gotta check my hunch meter)...
It was a nuance in the configuration of inbound windows for PCI/PEX=20
that was dribbling onto the memory that was consistently being i/o =
mapped
to the ext2 driver - which then got confused and bailed completely.
=20
We now have it working as we want it to with all 4 GIG of=20
DDR working as we want it.
=20
Thank you for pushing us to push through the hunch and get to the=20
root cause. It was a tough one to find.
=20
Tom
=20