Porting the 2.6.38 linux kernel to ARM11 MPcore

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Molnár Gábor]

Dear everyone,

im working on a project wich aims at porting the 2.6.38 kernel to an
arm11 mpcore based board.

The kernel fails to boot, in the paging_init func, it throws an
exception (pc: FFFF_000C) when executing the third mcr instruction in
the list below:

0xc000bf34  <>:    blx	r3
0xc000bf38  <>:    mov	r5, #0	; 0x0
0xc000bf3c  <>:    mcr	15, 0, r5, cr7, cr10, {4}
0xc000bf40  <>:    mcr	15, 0, r5, cr8, cr6, {0}
0xc000bf44  <>:    mcr	15, 0, r5, cr8, cr5, {0} //throws an exception
here, Invalidate Instruction TLB Register
0xc000bf48  <>:    mcr	15, 0, r5, cr7, cr5, {6}
0xc000bf4c  <>:    mcr	15, 0, r5, cr7, cr10, {4}
0xc000bf50  <>:    mcr	15, 0, r5, cr7, cr5, {4}
0xc000bf54  <>:    ldr	lr, [pc, #176]	; 0xc000c00c

The kernel has been patched with the 2.6.38-arm1 patch, and configured
to be as minimal as possible (the vmlinux.elf is 1.8MB) and
cross-compiled with the?Codesourcery gcc toolchain.
The minimal size is improtant because the aim is to make the kernel
run with only 5-6MB of memory (from c0000000 to c0500000).
The 'boot_loader' i wrote passes the atag pointers (basically core and
memory) and the kernel seemingly reads and parses them fine. The
__log_buf before the exception:

<5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
<4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
<4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
<4>Machine: ARM-RealView PB11MPCore
<5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).

The last one might be solved with setting the high mem option before
compiling (it might be a problem as i load the whole kernel to that
region).

Im only a beginner,?and im sorry for the low quality of the question,
i appreciate any help. Even comments on how 'accomplishable' this
project is. Also, the board is not a realview board, but it might not
be an issue at this stage.

Thank you in advance.

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Catalin Marinas]

On 18 November 2011 11:54, Moln?r G?bor <conqrx@gmail.com> wrote:
> im working on a project wich aims at porting the 2.6.38 kernel to an
> arm11 mpcore based board.
>
> The kernel fails to boot, in the paging_init func, it throws an
> exception (pc: FFFF_000C) when executing the third mcr instruction in
> the list below:
>
> 0xc000bf34 ?<>: ? ?blx ?r3
> 0xc000bf38 ?<>: ? ?mov ?r5, #0 ?; 0x0
> 0xc000bf3c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
> 0xc000bf40 ?<>: ? ?mcr ?15, 0, r5, cr8, cr6, {0}
> 0xc000bf44 ?<>: ? ?mcr ?15, 0, r5, cr8, cr5, {0} //throws an exception
> here, Invalidate Instruction TLB Register
> 0xc000bf48 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {6}
> 0xc000bf4c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
> 0xc000bf50 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {4}
> 0xc000bf54 ?<>: ? ?ldr ?lr, [pc, #176] ?; 0xc000c00c

So you get a prefetch abort as soon as the TLBs are invalidated, which
means that the page table is not set up correctly.

What's your RAM configuration (start address, size)?

> The kernel has been patched with the 2.6.38-arm1 patch, and configured
> to be as minimal as possible (the vmlinux.elf is 1.8MB) and
> cross-compiled with the?Codesourcery gcc toolchain.
> The minimal size is improtant because the aim is to make the kernel
> run with only 5-6MB of memory (from c0000000 to c0500000).
> The 'boot_loader' i wrote passes the atag pointers (basically core and
> memory) and the kernel seemingly reads and parses them fine. The
> __log_buf before the exception:
>
> <5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
> G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
> <4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
> <4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
> <4>Machine: ARM-RealView PB11MPCore
> <5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).

Does your physical RAM start at 0xc0000000 and it is all ignored?
Setting highmem would fix as all your lowmem (including the kernel
code) would be unmapped, hence the fault. What's the VMALLOC_END
defined to?

-- 
Catalin

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Molnár Gábor]

Thank you for your reply!

2011/11/21 Catalin Marinas <catalin.marinas@arm.com>:
> On 18 November 2011 11:54, Moln?r G?bor <conqrx@gmail.com> wrote:
>> im working on a project wich aims at porting the 2.6.38 kernel to an
>> arm11 mpcore based board.
>>
>> The kernel fails to boot, in the paging_init func, it throws an
>> exception (pc: FFFF_000C) when executing the third mcr instruction in
>> the list below:
>>
>> 0xc000bf34 ?<>: ? ?blx ?r3
>> 0xc000bf38 ?<>: ? ?mov ?r5, #0 ?; 0x0
>> 0xc000bf3c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
>> 0xc000bf40 ?<>: ? ?mcr ?15, 0, r5, cr8, cr6, {0}
>> 0xc000bf44 ?<>: ? ?mcr ?15, 0, r5, cr8, cr5, {0} //throws an exception
>> here, Invalidate Instruction TLB Register
>> 0xc000bf48 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {6}
>> 0xc000bf4c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
>> 0xc000bf50 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {4}
>> 0xc000bf54 ?<>: ? ?ldr ?lr, [pc, #176] ?; 0xc000c00c
>
> So you get a prefetch abort as soon as the TLBs are invalidated, which
> means that the page table is not set up correctly.
>
> What's your RAM configuration (start address, size)?

In the current configuration the start address is c0000000 and the
size is 500000.
The .elf kernel image is loaded at the c0008000 'start' address.

>> The kernel has been patched with the 2.6.38-arm1 patch, and configured
>> to be as minimal as possible (the vmlinux.elf is 1.8MB) and
>> cross-compiled with the?Codesourcery gcc toolchain.
>> The minimal size is improtant because the aim is to make the kernel
>> run with only 5-6MB of memory (from c0000000 to c0500000).
>> The 'boot_loader' i wrote passes the atag pointers (basically core and
>> memory) and the kernel seemingly reads and parses them fine. The
>> __log_buf before the exception:
>>
>> <5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
>> G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
>> <4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
>> <4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
>> <4>Machine: ARM-RealView PB11MPCore
>> <5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).
>
> Does your physical RAM start at 0xc0000000 and it is all ignored?
> Setting highmem would fix as all your lowmem (including the kernel
> code) would be unmapped, hence the fault. What's the VMALLOC_END
> defined to?

Yes it seems to be all ignored, i will check VMALLOC_END as soon as i can.

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Molnár Gábor]

> Does your physical RAM start at 0xc0000000 and it is all ignored?
> Setting highmem would fix as all your lowmem (including the kernel
> code) would be unmapped, hence the fault. What's the VMALLOC_END
> defined to?

#define VMALLOC_END             0xf8000000

For the platform i use. The size of vmalloc is the default (128MB).

If HIGHMEM is not set, this area is executed:

>/*
>792                  * Check whether this memory bank would entirely overlap
>793                  * the vmalloc area.
>794                  */
>795                 if (__va(bank->start) >= vmalloc_min ||
>796                     __va(bank->start) < (void *)PAGE_OFFSET) {
>797                         printk(KERN_NOTICE "Ignoring RAM at %.8lx-%.8lx "
>798                                "(vmalloc region overlap).\n",
>799                                bank->start, bank->start + bank->size - 1);
>800                         continue;
>801                 }

static void * __initdata vmalloc_min = (void *)(VMALLOC_END - SZ_128M);

The kernel is loaded on the phisical address: 0xc0000000-0xc05000000.
(This region is given as atag_mem as well).

At the current configuration i believe vmalloc should be at
0xF0000000-0xF8000000.

Why does this overlap with 0xC0000000-0xC0500000?



2011/11/22 Moln?r G?bor <conqrx@gmail.com>:
> Thank you for your reply!
>
> 2011/11/21 Catalin Marinas <catalin.marinas@arm.com>:
>> On 18 November 2011 11:54, Moln?r G?bor <conqrx@gmail.com> wrote:
>>> im working on a project wich aims at porting the 2.6.38 kernel to an
>>> arm11 mpcore based board.
>>>
>>> The kernel fails to boot, in the paging_init func, it throws an
>>> exception (pc: FFFF_000C) when executing the third mcr instruction in
>>> the list below:
>>>
>>> 0xc000bf34 ?<>: ? ?blx ?r3
>>> 0xc000bf38 ?<>: ? ?mov ?r5, #0 ?; 0x0
>>> 0xc000bf3c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
>>> 0xc000bf40 ?<>: ? ?mcr ?15, 0, r5, cr8, cr6, {0}
>>> 0xc000bf44 ?<>: ? ?mcr ?15, 0, r5, cr8, cr5, {0} //throws an exception
>>> here, Invalidate Instruction TLB Register
>>> 0xc000bf48 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {6}
>>> 0xc000bf4c ?<>: ? ?mcr ?15, 0, r5, cr7, cr10, {4}
>>> 0xc000bf50 ?<>: ? ?mcr ?15, 0, r5, cr7, cr5, {4}
>>> 0xc000bf54 ?<>: ? ?ldr ?lr, [pc, #176] ?; 0xc000c00c
>>
>> So you get a prefetch abort as soon as the TLBs are invalidated, which
>> means that the page table is not set up correctly.
>>
>> What's your RAM configuration (start address, size)?
>
> In the current configuration the start address is c0000000 and the
> size is 500000.
> The .elf kernel image is loaded at the c0008000 'start' address.
>
>>> The kernel has been patched with the 2.6.38-arm1 patch, and configured
>>> to be as minimal as possible (the vmlinux.elf is 1.8MB) and
>>> cross-compiled with the?Codesourcery gcc toolchain.
>>> The minimal size is improtant because the aim is to make the kernel
>>> run with only 5-6MB of memory (from c0000000 to c0500000).
>>> The 'boot_loader' i wrote passes the atag pointers (basically core and
>>> memory) and the kernel seemingly reads and parses them fine. The
>>> __log_buf before the exception:
>>>
>>> <5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
>>> G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
>>> <4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
>>> <4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
>>> <4>Machine: ARM-RealView PB11MPCore
>>> <5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).
>>
>> Does your physical RAM start at 0xc0000000 and it is all ignored?
>> Setting highmem would fix as all your lowmem (including the kernel
>> code) would be unmapped, hence the fault. What's the VMALLOC_END
>> defined to?
>
> Yes it seems to be all ignored, i will check VMALLOC_END as soon as i can.
>

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Catalin Marinas]

On Wed, Nov 23, 2011 at 02:14:06PM +0000, Moln?r G?bor wrote:
> > Does your physical RAM start at 0xc0000000 and it is all ignored?
> > Setting highmem would fix as all your lowmem (including the kernel
> > code) would be unmapped, hence the fault. What's the VMALLOC_END
> > defined to?
> 
> #define VMALLOC_END             0xf8000000
> 
> For the platform i use. The size of vmalloc is the default (128MB).
> 
> If HIGHMEM is not set, this area is executed:
> 
> >/*
> >792                  * Check whether this memory bank would entirely overlap
> >793                  * the vmalloc area.
> >794                  */
> >795                 if (__va(bank->start) >= vmalloc_min ||
> >796                     __va(bank->start) < (void *)PAGE_OFFSET) {
> >797                         printk(KERN_NOTICE "Ignoring RAM at %.8lx-%.8lx "
> >798                                "(vmalloc region overlap).\n",
> >799                                bank->start, bank->start + bank->size - 1);
> >800                         continue;
> >801                 }
> 
> static void * __initdata vmalloc_min = (void *)(VMALLOC_END - SZ_128M);
> 
> The kernel is loaded on the phisical address: 0xc0000000-0xc05000000.
> (This region is given as atag_mem as well).
> 
> At the current configuration i believe vmalloc should be at
> 0xF0000000-0xF8000000.
> 
> Why does this overlap with 0xC0000000-0xC0500000?

So it doesn't, in which case you may want to place some printk's in the
kernel around the message reporting that your block of RAM is ignored.

-- 
Catalin

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Arnd Bergmann]

On Friday 18 November 2011, Moln?r G?bor wrote:
> im working on a project wich aims at porting the 2.6.38 kernel to an
> arm11 mpcore based board.
> 
> ...
>
> Im only a beginner, and im sorry for the low quality of the question,
> i appreciate any help. Even comments on how 'accomplishable' this
> project is. Also, the board is not a realview board, but it might not
> be an issue at this stage.

Hi Moln?r,

I hope I'm not discouraging you too much, but from a high-level perspective
this sounds like a rather pointless goal, mostly because 2.6.38 is
rather old now and doing a port for that version would end up looking
very different from how we are doing things for new kernel releases.

Now if you were thinking about doing a port for mainline Linux (currently
that means targetting developing on 3.2-rc and targetting 3.3 or 3.4),
things would look much brighter.

The most important question is what hardware "platform" this port is for,
not so much the CPU core. The only two platform we support in Linux with
an ARM11 MPCore are cns3xxx and realview. You already mentioned that you
don't target realview. If your board is instead based on a Cavium Networks
SOC, things should be fairly easy and you only need to add a new machine
description in arch/arm/mach-cns3xxx. If you are working with an SOC
from a different family that is already supported by Linux, you have
to add support for the specific SOC, and the work involved in that 
depends a lot on the particular SOC.

For any other SOC that has no support whatsoever in Linux, I would suggest
getting the latest Linux kernel from git (3.2-rc2) and looking at the
highbank, prima2, picoxcell and zynq platforms in there. The code you
need to write should look similar to any of these, rather than the realview
platform, which is a fairly unusual one. In case of a new SOC platform,
you should be prepared to also write device drivers for any vendor
specific component that you want to support, e.g. network, mmc, audio,
so I would recommend first researching how many of the components on
the board already have drivers in Linux.

	Arnd

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Molnár Gábor]

> I hope I'm not discouraging you too much

Absolutely not, in fact, thank you for the really useful insight and
suggestions, much appreciated.


2011/11/23 Arnd Bergmann <arnd@arndb.de>:
> On Friday 18 November 2011, Moln?r G?bor wrote:
>> im working on a project wich aims at porting the 2.6.38 kernel to an
>> arm11 mpcore based board.
>>
>> ...
>>
>> Im only a beginner, and im sorry for the low quality of the question,
>> i appreciate any help. Even comments on how 'accomplishable' this
>> project is. Also, the board is not a realview board, but it might not
>> be an issue at this stage.
>
> Hi Moln?r,
>
> I hope I'm not discouraging you too much, but from a high-level perspective
> this sounds like a rather pointless goal, mostly because 2.6.38 is
> rather old now and doing a port for that version would end up looking
> very different from how we are doing things for new kernel releases.
>
> Now if you were thinking about doing a port for mainline Linux (currently
> that means targetting developing on 3.2-rc and targetting 3.3 or 3.4),
> things would look much brighter.
>
> The most important question is what hardware "platform" this port is for,
> not so much the CPU core. The only two platform we support in Linux with
> an ARM11 MPCore are cns3xxx and realview. You already mentioned that you
> don't target realview. If your board is instead based on a Cavium Networks
> SOC, things should be fairly easy and you only need to add a new machine
> description in arch/arm/mach-cns3xxx. If you are working with an SOC
> from a different family that is already supported by Linux, you have
> to add support for the specific SOC, and the work involved in that
> depends a lot on the particular SOC.
>
> For any other SOC that has no support whatsoever in Linux, I would suggest
> getting the latest Linux kernel from git (3.2-rc2) and looking at the
> highbank, prima2, picoxcell and zynq platforms in there. The code you
> need to write should look similar to any of these, rather than the realview
> platform, which is a fairly unusual one. In case of a new SOC platform,
> you should be prepared to also write device drivers for any vendor
> specific component that you want to support, e.g. network, mmc, audio,
> so I would recommend first researching how many of the components on
> the board already have drivers in Linux.
>
> ? ? ? ?Arnd
>

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Russell King - ARM Linux]

On Fri, Nov 18, 2011 at 12:54:42PM +0100, Moln?r G?bor wrote:
> <5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
> G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
> <4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
> <4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
> <4>Machine: ARM-RealView PB11MPCore
> <5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).

This tells me your PHYS_OFFSET is set incorrectly.

Porting the 2.6.38 linux kernel to ARM11 MPcore

[Molnár Gábor]

2011/11/23 Russell King - ARM Linux <linux@arm.linux.org.uk>:
> On Fri, Nov 18, 2011 at 12:54:42PM +0100, Moln?r G?bor wrote:
>> <5>Linux version 2.6.38-rc8 (VirtualBox) (gcc version 4.5.2 (Sourcery
>> G++ Lite 2011.03-41) ) #6 Wed Nov 9 21:59:29 CET 2011
>> <4>CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=08c5387f"
>> <4>CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
>> <4>Machine: ARM-RealView PB11MPCore
>> <5>Ignoring RAM at c0000000-c04fffff (vmalloc region overlap).
>
> This tells me your PHYS_OFFSET is set incorrectly.
>

Indeed it is, thank you!