Separate/external initramfs, ATAG's, kernel panics ... Oh My!

Separate/external initramfs, ATAG's, kernel panics ... Oh My!

[Brian Hutchinson]

Greetings,

I have a board with a ARM926EJS core that we have booted with a rootfs
on JFFS2 up to this point.? For performance (need some things to run
out of RAM), I've been trying to figure out how to get separate
initramfs working and have gone down several rabbit trails after
reading all I could on the u-boot and ARM list archives.

First, while reading the fine ARM boot document about ATAG's, I
realized that the u-boot (version 1.1.6) that shipped with our board
had #define CONFIG_SETUP_MEMORY_TAGS and CONFIG_INITRD_TAG commented
out.

Thinking that this was my problem ... I recompiled u-boot only to
start getting the Kernel telling me it couldn't recognize my ATAG_MEM
definitions and that INITRD: was outside physical memory.

So, I restored the original u-boot and then started passing initrd=
only to have the system hang.

Firing up OpenOCD/gdb and checking out init/initramfs.c left me even
more puzzled.? I set a breakpoint to make sure I was getting to
populate_rootfs ... and I am ... but it isn't doing anything!

Reading all the fine documentation on initramfs ...
CONFIG_BLK_DEV_INITRD=y is needed but CONFIG_BLK_DEV_RAM is not so I
don't have that set although it looks like maybe I should have it
after looking at the code.

My init/initramfs.c populate_rootfs in my 2.6.28 kernel looks like this:

static int __init populate_rootfs(void)
{
??? char *err = unpack_to_rootfs(__initramfs_start,
??? ??? ??? ?__initramfs_end - __initramfs_start, 0);
??? if (err)
??? ??? panic(err);
??? if (initrd_start) {
#ifdef CONFIG_BLK_DEV_RAM
??? ??? int fd;
??? ??? printk(KERN_INFO "checking if image is initramfs...");
??? ??? err = unpack_to_rootfs((char *)initrd_start,
??? ??? ??? initrd_end - initrd_start, 1);
??? ??? if (!err) {
??? ??? ??? printk(" it is\n");
??? ??? ??? unpack_to_rootfs((char *)initrd_start,
??? ??? ??? ??? initrd_end - initrd_start, 0);
??? ??? ??? free_initrd();
??? ??? ??? return 0;
??? ??? }
??? ??? printk("it isn't (%s); looks like an initrd\n", err);
??? ??? fd = sys_open("/initrd.image", O_WRONLY|O_CREAT, 0700);
??? ??? if (fd >= 0) {
??? ??? ??? sys_write(fd, (char *)initrd_start,
??? ??? ??? ??? ??? initrd_end - initrd_start);
??? ??? ??? sys_close(fd);
??? ??? ??? free_initrd();
??? ??? }
#else
??? ??? printk(KERN_INFO "Unpacking initramfs...");
??? ??? err = unpack_to_rootfs((char *)initrd_start,
??? ??? ??? initrd_end - initrd_start, 0);
??? ??? if (err)
??? ??? ??? panic(err);
??? ??? printk(" done\n");
??? ??? free_initrd();
#endif
??? }
??? return 0;
}

With OpenOCD, I discovered that __initramfs_end and __initramfs_start
don't appear to have anything to do with my initrd= line passed in
with bootargs.
The test of initrd_start fails because my initrd_start is zero.  I'm
not sure yet how initrd_start gets set but I did see something in a
#ifdef CONFIG_KEXEC that did something with it.  I don't have
CONFIG_KEXEC.

So at this point I guess my questions are:
1.  How do I get __initramfs_start and __initramfs_start to reflect my
initrd= bootargs argument?  Do I have to fix my ATAG's problem for
this?
2.  How does initrd_start get set?
3.  Do I need CONFIG_KEXEC?

What I've done so far is take the rootfs that was used with JFFS and
created a init->busybox soft link.  If I build the kernel with the
initramfs linked in (with CONFIG_INITRAMFS_SOURCE), the initramfs
works just fine so I know my cpio.gz is OK.  If I place the cpio.gz in
flash or tftp it to try separate initramfs ... it hangs the system.

I've tried to wrap the cpio.gz in a u-boot image with:
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x200000 -e 0x200000 -n
uInitramfs_busybox -d /home/hutch/squashfs/test.cpio.gz
uInitramfs_busybox

This was used with a bootargs line in uboot of:
bootargs initrd=0x200000
ip=$ipaddr:$serverip:$gatewayip:$netmask:$hostname:$netdev:any
console=$consoledev,$baudrate $othbootargs;

Kernel is located at 0x20080000 and the cpio.gz mkimage file was
stored at 0x20280000

bootcmd in uboot was bootm 0x20080000 0x20280000

I also tried to tftp just the cpio.gz file (without the u-boot header)
to 0x200000 and then bootm 0x20080000 with a initrd=0x200000 with the
same result ... system hang because it doesn't try to do anything with
my initramfs and I didn't supply a root= so it falls back and then
panics because it can't find a filesystem.

Hopefully I've provided enough info to show I've been digging through
this but something just isn't clicking yet.  Hopefully someone with
more experience can show me the way!

Regards,

Brian

Separate/external initramfs, ATAG's, kernel panics ... Oh My!

[Russell King - ARM Linux]

On Mon, Feb 22, 2010 at 05:07:21PM -0500, Brian Hutchinson wrote:
> First, while reading the fine ARM boot document about ATAG's, I
> realized that the u-boot (version 1.1.6) that shipped with our board
> had #define CONFIG_SETUP_MEMORY_TAGS and CONFIG_INITRD_TAG commented
> out.
> 
> Thinking that this was my problem ... I recompiled u-boot only to
> start getting the Kernel telling me it couldn't recognize my ATAG_MEM
> definitions and that INITRD: was outside physical memory.

What do you mean "it couldn't recognize my ATAG_MEM definitions" ?

The "INITRD: was outside physical memory" seems entirely reasonable
given what you're saying below:

> I've tried to wrap the cpio.gz in a u-boot image with:
> mkimage -A arm -O linux -T ramdisk -C gzip -a 0x200000 -e 0x200000 -n
> uInitramfs_busybox -d /home/hutch/squashfs/test.cpio.gz
> uInitramfs_busybox
> 
> This was used with a bootargs line in uboot of:
> bootargs initrd=0x200000
> ip=$ipaddr:$serverip:$gatewayip:$netmask:$hostname:$netdev:any
> console=$consoledev,$baudrate $othbootargs;
> 
> Kernel is located at 0x20080000 and the cpio.gz mkimage file was
> stored at 0x20280000

Okay - you say the kernel is at 0x20080000, but this can't be, because
it needs to have a 32K offset - I assume you mean it's at 0x20008000.
So this presumably means that physical memory starts at 0x20000000.

So, the first problem with the above is that you're telling uboot to
load your ramdisk into physical address 0x200000 - below physical memory.
(In that case, is it unsurprising that the ATAG_INITRD caused a complaint?)

The second problem is that 0x20208000 - 0x20008000 = 0x00200000, or 2MB.
With the size of kernels today, 2MB is not sufficient space.  You want it
to be at least 16MB above the kernel.

The problem is that if the image gets corrupted, but it still looks like
a gzip, the gunzip code can get horribly confused and loop infinitely -
it sounds like this is what's happening.

Separate/external initramfs, ATAG's, kernel panics ... Oh My!

[Brian Hutchinson]

> What do you mean "it couldn't recognize my ATAG_MEM definitions" ?

Sorry, I was trying to provide enough info but not be too long. ?I'll
shed more light ...
OK don't laugh, but my u-boot code didn't appear setup right because
when I uncommented the #define CONFIG_SETUP_MEMORY_TAGS I started to
get unrecognized lines in the kernel (see below) so I tried to jam
something to see if it made a difference so here is the
setup_memory_tags I tried:

#ifdef CONFIG_SETUP_MEMORY_TAGS
static void setup_memory_tags (bd_t *bd)
{
// ? ? ?int i;

// ? ? ?for (i = 0; i < CONFIG_NR_DRAM_BANKS; i++) {
// ? ? ? ? ? ? ?params->hdr.tag = ATAG_MEM;
// ? ? ? ? ? ? ?params->hdr.size = tag_size (tag_mem32);
//
// ? ? ? ? ? ? ?params->u.mem.start = bd->bi_dram[i].start;
// ? ? ? ? ? ? ?params->u.mem.size = bd->bi_dram[i].size;
//
// ? ? ? ? ? ? ?params = tag_next (params);
// ? ? ?}
? ? ? ? ? ? ? ?// bth
? ? ? ? ? ? ? ?params->hdr.tag = ATAG_MEM;
? ? ? ? ? ? ? ?params->hdr.size = tag_size (tag_mem32);
? ? ? ? ? ? ? ?params->u.mem.start = PHYS_SDRAM_1;
? ? ? ? ? ? ? ?params->u.mem.size = PHYS_SDRAM_1_SIZE;
? ? ? ? ? ? ? ?params = tag_next (params);

? ? ? ? ? ? ? ?params->hdr.tag = ATAG_MEM;
? ? ? ? ? ? ? ?params->hdr.size = tag_size (tag_mem32);
? ? ? ? ? ? ? ?params->u.mem.start = PHYS_SDRAM_2;
? ? ? ? ? ? ? ?params->u.mem.size = PHYS_SDRAM_2_SIZE;
? ? ? ? ? ? ? ?params = tag_next (params);

? ? ? ? ? ? ? ?params->hdr.tag = ATAG_MEM;
? ? ? ? ? ? ? ?params->hdr.size = tag_size (tag_mem32);
? ? ? ? ? ? ? ?params->u.mem.start = PHYS_SDRAM_3;
? ? ? ? ? ? ? ?params->u.mem.size = PHYS_SDRAM_3_SIZE;
? ? ? ? ? ? ? ?params = tag_next (params);

? ? ? ? ? ? ? ?params->hdr.tag = ATAG_MEM;
? ? ? ? ? ? ? ?params->hdr.size = tag_size (tag_mem32);
? ? ? ? ? ? ? ?params->u.mem.start = PHYS_SDRAM_4;
? ? ? ? ? ? ? ?params->u.mem.size = PHYS_SDRAM_4_SIZE;
? ? ? ? ? ? ? ?params = tag_next (params);


}

I did this hack because my u-boot board_config.h file looks like:

/*-----------------------------------------------------------------------------
?* DDR2 RAM Memory Map
?*/
#ifndef CONFIG_PC20X_2_DDR_RAM_BANKS
/* We want a 4 DDR Bank setup then */

#define CONFIG_NR_DRAM_BANKS ? ?4 ? ? ? ? ? ? ? ? ? /* We have 4 RAM
banks... */
#define PHYS_SDRAM_1 ? ? ? ? ? ?0x00000000
#define PHYS_SDRAM_1_SIZE ? ? ? (32*1024*1024) ? ? ?/* 32 Mbytes */
#define PHYS_SDRAM_2 ? ? ? ? ? ?0x04000000
#define PHYS_SDRAM_2_SIZE ? ? ? (32*1024*1024) ? ? ?/* 32 Mbytes */
#define PHYS_SDRAM_3 ? ? ? ? ? ?0x08000000
#define PHYS_SDRAM_3_SIZE ? ? ? (32*1024*1024) ? ? ?/* 32 Mbytes */
#define PHYS_SDRAM_4 ? ? ? ? ? ?0x0C000000
#define PHYS_SDRAM_4_SIZE ? ? ? (32*1024*1024) ? ? ?/* 32 Mbytes */

... Which didn't appear to help. ?No matter what I did the kernel
would display these messages:

Ignoring unrecognised tag 0x00000000
Ignoring unrecognised tag 0x00000000
Ignoring unrecognised tag 0x00000000
Ignoring unrecognised tag 0x00000000

So something isn't quite right with the ATAG_MEM tags ... that is when
I commented out the ATAG's again and tried to just use the initrd=
bootarg.

>
> The "INITRD: was outside physical memory" seems entirely reasonable
> given what you're saying below:
>

snip

>
> Okay - you say the kernel is at 0x20080000, but this can't be, because
> it needs to have a 32K offset - I assume you mean it's at 0x20008000.
> So this presumably means that physical memory starts at 0x20000000.

Sorry, I should of just posted the u-boot display ... a picture is
worth a 1000 words:

## Booting image at 20080000 ...
? Image Name: ? Linux-2.6.28-picochip-3.1.0
? Created: ? ? ?2010-02-19 ?21:37:59 UTC
? Image Type: ? ARM Linux Kernel Image (uncompressed)
? Data Size: ? ?1608236 Bytes = ?1.5 MB
? Load Address: 00008000
? Entry Point: ?00008000
? Verifying Checksum ... OK
OK
## Loading Ramdisk Image at 20280000 ...
? Image Name: ? uInitramfs_busybox
? Created: ? ? ?2010-02-22 ?19:03:56 UTC
? Image Type: ? ARM Linux RAMDisk Image (gzip compressed)
? Data Size: ? ?13221592 Bytes = 12.6 MB
? Load Address: 00200000
? Entry Point: ?00200000
? Verifying Checksum ... OK

Starting kernel ...

0x20080000 & 0x20280000 are the NOR flash locations of a uImage and
cpio.gz respectively.
Kernel entry is 0x8000 and to be honest ... I'm not 100% up on how the
ARM kernel starts (haven't been working with this target long) so I
was guessing at where the cpio.gz image should be located. ?At first
when I did mkimage on the cpio image, ?I used -a 0 and -e 0 which is
what I read in some documentation (I assume this would work if my
ATAGs were not hosed up) and then I tried other options like tftp'ing
the cpio.gz itself and other combinations and that is how I arrived at
the example I gave. ?I've been trying all kinds of stuff trying to get
this to work and understand what is going on.

The board has 128M of RAM. ?It looks to me like u-boot recognizes this
correctly but since ATAG_MEM's are commented out ... I'm not quite
sure how Linux figures out the memory since these lines look suspect
as the kernel is coming up:

Memory: 16MB 16MB 16MB 16MB = 64MB total
Memory: 61584KB available (2800K code, 336K data, 116K init)

Regards,

Brian