[PATCH 0/15] bootwrapper: support for kexec to zImage

[PATCH 0/15] bootwrapper: support for kexec to zImage

[Milton Miller]

This series creates a 32 bit zImage wrapper for a 32 or 64 bit PowerPC
Linux kernel.   This allows you to kexec a zImage with its compressed
vmlinux instead of the uncompressed vmlinux elf.  The elf is also
packaged as a 64 bit elf for use by kexec-tools for 64 bit kernels.

This series also adds code to extract the vmlinux elf file from an
initramfs (or other platform supplied location).  This allows one
to load the elf once (compressed) and use it both for userspace and
for the actual boot.

milton

[PATCH 1/15] boot: find initrd location from device-tree

[Milton Miller]

Some platforms have a boot agent that can create or modify properties in
the device-tree and load images into memory.  Provide a helper to set
loader_info used by prep_initrd().

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Moved to devtree.c as suggested.

define UNIT_MAX hardcoded as requested.

Print exactly why we are not propagting loader supplied initrd knowledge to
wrapper code.

Having start_prop and end_prop as variables allows several source lines
to fit in 80 columns.

Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 03:44:41.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 03:45:51.000000000 -0500
@@ -159,6 +159,7 @@ void dt_fixup_clock(const char *path, u3
 void __dt_fixup_mac_addresses(u32 startindex, ...);
 #define dt_fixup_mac_addresses(...) \
 	__dt_fixup_mac_addresses(0, __VA_ARGS__, NULL)
+void dt_find_initrd(void);
 
 
 static inline void *find_node_by_linuxphandle(const u32 linuxphandle)
Index: work.git/arch/powerpc/boot/types.h
===================================================================
--- work.git.orig/arch/powerpc/boot/types.h	2007-07-10 03:44:41.000000000 -0500
+++ work.git/arch/powerpc/boot/types.h	2007-07-10 03:45:51.000000000 -0500
@@ -12,6 +12,8 @@ typedef short			s16;
 typedef int			s32;
 typedef long long		s64;
 
+#define UINT_MAX	0xFFFFFFFF
+
 #define min(x,y) ({ \
 	typeof(x) _x = (x);	\
 	typeof(y) _y = (y);	\
Index: work.git/arch/powerpc/boot/devtree.c
===================================================================
--- work.git.orig/arch/powerpc/boot/devtree.c	2007-07-10 03:44:41.000000000 -0500
+++ work.git/arch/powerpc/boot/devtree.c	2007-07-10 03:45:51.000000000 -0500
@@ -1,6 +1,7 @@
 /*
  * devtree.c - convenience functions for device tree manipulation
  * Copyright 2007 David Gibson, IBM Corporation.
+ * Copyright 2007 Milton Miller, IBM Corporation.
  * Copyright (c) 2007 Freescale Semiconductor, Inc.
  *
  * Authors: David Gibson <david@gibson.dropbear.id.au>
@@ -305,3 +306,68 @@ int dt_xlate_addr(void *node, u32 *buf, 
 	memcpy(dt_xlate_buf, buf, buflen);
 	return dt_xlate(node, 0, buflen / 4, xlated_addr, NULL);
 }
+
+/**
+ * dt_find_initrd - set loader initrd location based on existing properties
+ *
+ * finds the linux,initrd-start and linux,initrd-end properties in
+ * the /chosen node and sets the loader initrd fields accordingly.
+ *
+ * Use this if your loader sets the properties to allow other code to
+ * relocate the tree and/or cause r3 and r4 to be set on true OF
+ * platforms.
+ */
+void dt_find_initrd(void)
+{
+	int rc;
+	unsigned long long initrd_start, initrd_end;
+	void *devp;
+	static const char start_prop[] = "linux,initrd-start";
+	static const char end_prop[] = "linux,initrd-end";
+
+	devp = finddevice("/chosen");
+	if (! devp) {
+		return;
+	}
+
+	rc = getprop(devp, start_prop, &initrd_start, sizeof(initrd_start));
+	if (rc < 0)
+		return;				/* not found */
+	/* The properties had to be 8 bytes until 2.6.22 */
+	if (rc == sizeof(unsigned long)) {
+		unsigned long tmp;
+		memcpy(&tmp, &initrd_start, rc);
+		initrd_start = tmp;
+	} else if (rc != sizeof(initrd_start)) {	/* now they can be 4 */
+		printf("unexpected length of %s in /chosen!\n\r", start_prop);
+		return;
+	}
+
+	rc = getprop(devp, end_prop, &initrd_end, sizeof(initrd_end));
+	if (rc < 0) {
+		printf("chosen has %s but no %s!\n\r", start_prop, end_prop);
+		return;
+	}
+	if (rc == sizeof(unsigned long)) {
+		unsigned long tmp;
+		memcpy(&tmp, &initrd_end, rc);
+		initrd_end = tmp;
+	} else if (rc != sizeof(initrd_end)) {
+		printf("unexpected length of %s in /chosen!\n\r", end_prop);
+		return;
+	}
+
+	/* Check for presence, ignore if (partially) loaded above 32 bits */
+	if (initrd_start == initrd_end) {
+		printf("ignoring empty device-tree supplied initrd\n");
+	} else if (initrd_start > initrd_end) {
+		printf("ignoring device-tree supplied initrd: start 0x%llx"
+				" > end 0x%llx \n", initrd_start, initrd_end);
+	} else if (initrd_end > UINT_MAX) {
+		printf("ignoring device-tree supplied initrd:"
+				" end 0x%llx > 32 bits\n", initrd_end);
+	} else {
+		loader_info.initrd_addr = initrd_start;
+		loader_info.initrd_size  = initrd_end - initrd_start;
+	}
+}

[PATCH 2/15] boot: record header bytes in gunzip_start

[Milton Miller]

Record the number of header bytes skipped in the total bytes read field.

This is needed for the initramfs parsing code to find the end of the zip file.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 

Index: work.git/arch/powerpc/boot/gunzip_util.c
===================================================================
--- work.git.orig/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:44:41.000000000 -0500
+++ work.git/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:47:35.000000000 -0500
@@ -78,6 +78,7 @@ void gunzip_start(struct gunzip_state *s
 			fatal("inflateInit2 returned %d\n\r", r);
 	}
 
+	state->s.total_in = hdrlen;
 	state->s.next_in = src + hdrlen;
 	state->s.avail_in = srclen - hdrlen;
 }

[PATCH 3/15] boot: simplfy gunzip_finish

[Milton Miller]

Call gunzip_partial to calculate the remaining length and copy the
data to the user buffer.   This makes it shorter and reduces
duplication.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 

Index: work.git/arch/powerpc/boot/gunzip_util.c
===================================================================
--- work.git.orig/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:47:35.000000000 -0500
+++ work.git/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:47:39.000000000 -0500
@@ -194,13 +194,10 @@ int gunzip_finish(struct gunzip_state *s
 {
 	int len;
 
+	len = gunzip_partial(state, dst, dstlen);
+
 	if (state->s.workspace) {
-		len = gunzip_partial(state, dst, dstlen);
 		zlib_inflateEnd(&state->s);
-	} else {
-		/* uncompressed image */
-		len = min(state->s.avail_in, (unsigned)dstlen);
-		memcpy(dst, state->s.next_in, len);
 	}
 
 	return len;

[PATCH 4/15] bootwrapper: smp support code

[Milton Miller]

Support code to move cpus around, both a spin loop and c code to
move the cpus before uncompressing and copying the kernel to 0.

The low level code is designed to be included in a crt0 or other
assembly file because it may need to be at a fixed location or there
may be other entry point requirements.

Note: this code works with kernel head_64.S.   head_6xx.S needs the
0x60 entry point (it currently uses something closer to 0xC0; but
the similar code is at 0xC4); the other heads don't appear to support
SMP.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Cleaned up documentation.   Hopefully the format is correct.

Removed the #if 1 #else #endif

Move hidden asm code outside function instead of branching over it.

This code has previously survied days of kexec stress and also works
when the next stage is itself (ie zBoot to zImage).

Index: work.git/arch/powerpc/boot/marshal_low.S
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/marshal_low.S	2007-07-10 03:48:20.000000000 -0500
@@ -0,0 +1,103 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright 2007 IBM Corporation.
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ *
+ */
+
+#include "ppc_asm.h"
+
+	.text
+	/*
+	 * This code is designed to be a kexec entry point block.
+	 * That is, it has both code for the master cpu that begins
+	 * at offset 0 as linked into the image, and a sequence of
+	 * 0x100 bytes that, when copied to address 0, forms the
+	 * wait loop for slave cpus.  Each slave should have its
+	 * unique hardware cpu identifier in r3 before entering
+	 * this code.
+	 */
+	.globl	master
+master: b	_zimage_start_plat
+
+	.global slave_wait
+slave_wait:
+	/* r3 cpu id, r4 slaves_wait, r5 cpu bit, r6 cpu mask word offset */
+
+	/* set our bit in the slaves mask  */
+98:	lwarx	r7,r4,r6
+	or	r8,r7,r5
+	stwcx.	r8,r4,r6
+	bne	98b
+
+	and.	r8,r7,r5
+	bnel-	err_slave
+
+99:	lwz	r7,gohere-slave_wait(r4)
+	cmpwi	0,r7,0
+	beq	99b
+	mtctr	r7
+	mr	r4,r7
+	bctr
+
+
+	.global gohere
+gohere:	.long	0			/* when set the slave moves */
+
+
+err_slave:
+	stw	r5,slave_error-slave_wait(4)	/* no locking */
+	blr
+
+	.globl	slave_error		/* set when slave detects error */
+slave_error:
+	.long	0
+
+	/*
+	 * The slaves may be in 32 or 64 bit mode, we don't care
+	 * r3 is the slave cpu number, matching the device tree.
+	 */
+	.org	master+0x60
+	.globl	slave
+slave:	bl	1f
+1:	mflr	r4
+	addi	r4,r4,slave_wait-1b	/* code assumes r4=slave_wait */
+	li	r5,1
+	rlwnm	r5,r5,r3,0,31		/* bit within word */
+	rlwinm	r6,r3,32-5+2,4,29	/* word in array */
+	addi	r6,r6,slaves-slave_wait	/* relative to r4, slave_wait */
+	b	slave_wait
+
+	.org	master+0x80	/* put locked bitmask data in another line */
+	.global	slaves
+slaves:
+
+	.globl slaves_end;
+slaves_end = 0f
+
+#if 0
+	/* today, the 32 bit kernel starts slaves at 0xc0
+	 * but this limits us to cpu to 512 vs 1024
+	 */
+	.org	master+0xc0
+0:	b	slave
+#endif
+
+
+	.org	master+0x100		/* we must fit in 0x100 bytes */
+0:
+
Index: work.git/arch/powerpc/boot/marshal.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/marshal.c	2007-07-10 03:48:20.000000000 -0500
@@ -0,0 +1,275 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright (C) 2007 IBM Corporation.
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+
+#include "ops.h"
+#include "stdio.h"
+#include "reg.h"
+
+extern unsigned int gohere[], master[], slave_wait[], slaves[], slaves_end[];
+extern unsigned int slave_error[1];
+
+static unsigned int slaves_run_here[SMP_SLAVE_SIZE / sizeof(unsigned int)];
+static unsigned int *slaves_were_here = master;
+static unsigned int *slaves_goto_here = master;
+
+/**
+ * check_slave_errors - check if the slaves have set the error flag.
+ * @slaves_here ... the location that the slaves should be spinning.
+ */
+static void check_slave_errors(unsigned int *slaves_here)
+{
+	unsigned int *error = slave_error - master + slaves_here;
+
+	if (*error) {
+		printf("WARNING: error detected by one or more slave cpus!!\n\r"
+		"WARNING: This probably means you have duplicate cpu ids\n\r");
+		/* exit() */
+	}
+}
+
+/**
+ * wait_slaves_moved - wait for the slaves to catch up
+ *
+ * Wait until all slaves that checked in the previous location have
+ * checked into the current location.  Seperate so we can do other
+ * work while we wait for them to catch up.
+ */
+void wait_slaves_moved(void)
+{
+	int offset = slaves - master;
+	int len = sizeof(slaves_end[0]) * (slaves_end - slaves);
+	int printed = 0;
+	unsigned int *to = slaves_goto_here;
+	unsigned int *from = slaves_were_here;
+
+	from += offset;
+	to += offset;
+
+	if (from == to)
+		return;
+
+	while (memcmp(from, to, len)) {
+		if (!printed) {
+			printf("waiting for slave cpus to move...");
+			printed = 1;
+			HMT_LOW;
+			barrier();
+		}
+		/* check from is superset of to */
+	}
+	if (printed) {
+		HMT_MEDIUM;
+		printf("done.\n\r");
+	}
+
+	slaves_were_here = slaves_goto_here;
+}
+
+/**
+ * move_slaves_here - move slaves to a specified address.
+ * @addr: location of %SMP_SLAVE_SIZE buffer to place code and spin
+ *
+ * Tell slaves to go from their current location to a buffer @addr
+ * of %SMP_SLAVE_SIZE bytes somewhere in memory.
+ */
+void move_slaves_here(void *addr)
+{
+	unsigned int *move_slaves_here = addr;
+	unsigned int *tell_them = gohere - master + slaves_goto_here;
+	unsigned int *goto_here = slave_wait - master + move_slaves_here;
+	unsigned int *wait_here = gohere - master + move_slaves_here;
+
+	if (move_slaves_here == slaves_goto_here)
+		return;				/* already there */
+
+	wait_slaves_moved();			/* one move at a time */
+
+	printf("moving slave cpus from %p to %p\n\r", slaves_goto_here,
+		move_slaves_here);
+
+	memcpy(move_slaves_here, master, SMP_SLAVE_SIZE);
+	memset(move_slaves_here + (slaves - master), 0,
+		(slaves_end - slaves) * sizeof(slaves_end[0]));
+	*wait_here = 0;
+
+	flush_cache(move_slaves_here, SMP_SLAVE_SIZE);
+
+	check_slave_errors(slaves_were_here);
+
+	*tell_them = (unsigned int)goto_here;
+	slaves_goto_here = move_slaves_here;
+}
+
+/**
+ * move_slaves_up - move slaves from somewhere low to our bss.
+ * Call before decompressing the kernel to address 0.
+ */
+void move_slaves_up(void)
+{
+	move_slaves_here(slaves_run_here);
+}
+
+/**
+ * slaves_are_low - Assert that the slaves are spinning at 0, and move them
+ * Assert that the slaves are running in a copy of the marshall code
+ * that was copied to address 0.  Ask them to go up to our bss, as we
+ * know we have to move them away from 0.
+ */
+void slaves_are_low(void)
+{
+	slaves_goto_here = slaves_were_here = (void *)0;
+	move_slaves_up();
+}
+
+/**
+ * wait_slave_checkout - wait for slaves to execute checkout store.
+ * @checkout - slave checkout flag array
+ *
+ * Wait for every slave who checked in at slaves_were_here to
+ * perform the stb to @checkout before the branch to self spin loop.
+ */
+static void wait_slave_checkout(char *checkout)
+{
+	unsigned int *end = slaves_end - master + slaves_were_here;
+	unsigned int *from = slaves - master + slaves_were_here;;
+	unsigned int bit;
+	int i, ncpus = 0;
+	char *waiting = "waiting on slaves to go to kernel...";
+
+	for (i=0; from < end; from++)
+		for (bit = 1; bit; i++, bit <<= 1)
+			if (*from & bit) {
+				ncpus++;
+				while (!checkout[i]) {
+					if (waiting) {
+						printf(waiting);
+						waiting = NULL;
+					}
+					HMT_LOW;
+					barrier();
+				}
+			}
+
+	if (waiting == NULL)
+		printf("done.\n\r");
+
+	printf("moved %d slaves to the kernel.\n\r", ncpus);
+}
+
+/* The slave checkin code ... used by checkout_slaves_to_kernel below */
+extern unsigned int slave_checkout_begin[], slave_checkout_spin[];
+asm ("\
+	.globl	slave_checkout_begin 	;\
+	.globl	slave_checkout_spin	;\
+slave_checkout_begin:			;\
+	lwz	7,0(0)			;\
+	li	8,1			;\
+	stbx	8,7,3			;\
+slave_checkout_spin:			;\
+	b	$			;\
+");
+
+
+/**
+ * checkout_slaves_to_kernel - send SMP slaves to the kernel
+ * @tell_them - the expected marshalling buffer for the slaves
+ *
+ * Actively move slaves spinning on @tell_them to 0x60.  Since we
+ * don't know what code is there, replace it with our one code that
+ * ends with a byte store and branch to self, with the branch at 0x60.
+ * After the stores complete, we can restore the rest of the line,
+ * flush, then restore the remaining line.
+ */
+static void checkout_slaves_to_kernel(unsigned int *tell_them)
+{
+	int to, spin;
+	unsigned int *from, *low, save[SMP_SLAVE_SIZE/sizeof(unsigned int)];
+	char *checkout;
+
+	checkout = malloc(1024);
+	if (checkout == NULL)
+		fatal("can't malloc slave checkout buffer");
+	memset(checkout, 0, 1024);
+
+	low = (unsigned int *)0;
+	memcpy(save, low, SMP_SLAVE_SIZE);
+
+	to = spin = 0x60 / sizeof(int);
+
+	to++;
+	from = slave_checkout_spin;
+	while (from >= slave_checkout_begin)
+		low[--to] = *from--;
+
+	low[0] = (unsigned int)checkout;
+	flush_cache(low, SMP_SLAVE_SIZE);
+
+	*tell_them = (unsigned int)(low + to);
+
+	wait_slave_checkout(checkout);
+
+	/* at this point, all have completed the store at %0x5c and are at
+	 * the branch to self at %0x60.   Restore the rest of the vector,
+	 * flush cache, then do the final store replacing the spin and
+	 * flush again.
+	 */
+	low[0] = save[0];
+	for (;to < spin; to++)
+		low[to] = save[to];
+	flush_cache(low, SMP_SLAVE_SIZE);
+	low[to] = save[to];
+	flush_cache(low, SMP_SLAVE_SIZE);
+
+}
+
+/**
+ * send_slaves_to_kernel - send SMP slaves to the kernel
+ * @vmlinux_addr: address vmlinux was decompressed to (where to get slave loop)
+ *
+ * Send slaves currently running in the marshalling system to the slave code
+ * in the next kernel which has been uncompressed at address @vmlinux_addr.
+ * Copies the first %SMP_SLAVE_SIZE bytes of the image to address %0 and
+ * then tells the slaves to go to %0x60.
+ */
+void send_slaves_to_kernel(void *vmlinux_addr)
+{
+	unsigned int *tell_them = gohere - master + slaves_goto_here;
+
+	if ((unsigned long)slaves_goto_here < SMP_SLAVE_SIZE) {
+		if ((unsigned long)vmlinux_addr < SMP_SLAVE_SIZE)
+			fatal("ERROR: slaves were not marshaled before "
+					"decompressing the kernel to 0!\n");
+		move_slaves_up();
+		send_slaves_to_kernel(vmlinux_addr);
+		return;
+	}
+
+	wait_slaves_moved();
+
+	if (vmlinux_addr) {
+		memcpy((void *)0, vmlinux_addr, SMP_SLAVE_SIZE);
+		flush_cache((void *)0, SMP_SLAVE_SIZE);
+	} else {
+		printf("kernel was decompressed to 0\n\r");
+	}
+	check_slave_errors(slaves_goto_here);
+
+	checkout_slaves_to_kernel(tell_them);
+}
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 03:48:11.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 03:48:20.000000000 -0500
@@ -42,6 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
+		marshal.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c
Index: work.git/arch/powerpc/boot/reg.h
===================================================================
--- work.git.orig/arch/powerpc/boot/reg.h	2007-07-10 03:48:11.000000000 -0500
+++ work.git/arch/powerpc/boot/reg.h	2007-07-10 03:48:20.000000000 -0500
@@ -19,4 +19,9 @@ static inline u32 mfpvr(void)
 register void *__stack_pointer asm("r1");
 #define get_sp()	(__stack_pointer)
 
+#define HMT_MEDIUM	asm volatile("or 2,2,2")
+#define HMT_LOW		asm volatile("or 1,1,1")
+#define barrier()	asm volatile("":::"memory")
+
+
 #endif	/* _PPC_BOOT_REG_H */
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 03:48:15.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 03:48:20.000000000 -0500
@@ -18,6 +18,7 @@
 #define	COMMAND_LINE_SIZE	512
 #define	MAX_PATH_LEN		256
 #define	MAX_PROP_LEN		256 /* What should this be? */
+#define	SMP_SLAVE_SIZE		256 /* Size of SMP slave block, kexec/kernel */
 
 typedef void (*kernel_entry_t)(unsigned long r3, unsigned long r4, void *r5);
 
@@ -84,10 +85,17 @@ int ns16550_console_init(void *devp, str
 int mpsc_console_init(void *devp, struct serial_console_data *scdp);
 void *simple_alloc_init(char *base, unsigned long heap_size,
 			unsigned long granularity, unsigned long max_allocs);
-extern void flush_cache(void *, unsigned long);
+void flush_cache(void *, unsigned long);
 int dt_xlate_reg(void *node, int res, unsigned long *addr, unsigned long *size);
 int dt_xlate_addr(void *node, u32 *buf, int buflen, unsigned long *xlated_addr);
 
+/* marshal slave cpus around to kernel */
+void move_slaves_up(void);
+void move_slaves_here(void *where);
+void send_slaves_to_kernel(void *vmlinux_addr);
+void slaves_are_low(void);
+void wait_slaves_moved(void);
+
 static inline void *finddevice(const char *name)
 {
 	return (dt_ops.finddevice) ? dt_ops.finddevice(name) : NULL;

[PATCH 5/15] bootwrapper: occupied memory ranges

[Milton Miller]

Add a set of library routines to manage gross memory allocations.

This code uses an array in bss to store upto 32 entrys with merging
representing a range of memory below rmo_end (aka end of real mode
memory at 0).

To use this code, a platform would set rmo_end, call occupy_memory,
then then call init_malloc for fine grain allocation.

Also, an optional vmlinux_alloc.   

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Fixed a bug when extending across multiple ranges.

Cleaned up documentation

Index: work.git/arch/powerpc/boot/memranges.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/memranges.c	2007-07-10 03:48:29.000000000 -0500
@@ -0,0 +1,242 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corporation 2007
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+
+#include "ops.h"
+#include "stdio.h"
+#include "flatdevtree.h"
+#include "page.h"
+#include "types.h"
+
+extern char _start[], _end[];
+
+void *rmo_end;
+
+static struct {
+	void *start, *end;
+} ranges[32];
+static int num_ranges;
+
+/**
+ * add_occupied_range - mark a range as occupied
+ * @start: start of range pointer
+ * @end: end of range pointer
+ *
+ * Mark the range from @start to @end as occupied.
+ * Ignore anything above rmo_end.
+ */
+void add_occupied_range(void *start, void *end)
+{
+	int i, j;
+
+	if (start == end)
+		return;
+	if (start > rmo_end)
+		return;
+	if (end > rmo_end)
+		end = rmo_end;
+	if (start > end)
+		fatal("%s: BUG: start %p > end %p\n\r", __FUNCTION__,
+				start, end);
+
+	printf("add %p %p: ", start, end);
+
+	for (i=0; i < num_ranges; i++)
+		if (start <= ranges[i].end)
+			break;
+
+	/* extend and merge any overlapping ranges */
+	if (i < num_ranges && end >= ranges[i].start) {
+		ranges[i].start = min(start, ranges[i].start);
+		for (j=i; j < num_ranges; j++)
+			if (end >= ranges[j].start)
+				end = max(end, ranges[j].end);
+			else
+				break;
+		ranges[i].end = end;
+
+		if (j == i + 1) {
+			printf("extending range %d to %p %p\n\r", i,
+				ranges[i].start, ranges[i].end);
+		} else {
+			printf("merged ranges %d to %d now %p %p\n\r", i, j,
+				ranges[i].start, ranges[i].end);
+
+			++i;
+			memmove(&ranges[i], &ranges[j],
+				(num_ranges - j) * sizeof(ranges[0]));
+			num_ranges -= j-i;
+		}
+	} else {
+		/* insert a new range */
+		if (num_ranges >= ARRAY_SIZE(ranges) - 1)
+			fatal("Too many memory ranges to track\n");
+
+		printf("inserting range %d between %p and %p\n\r",
+				i, i ? ranges[i-1].end : 0,
+				i == num_ranges ? rmo_end : ranges[i].start);
+
+		memmove(&ranges[i+1], &ranges[i],
+				(num_ranges - i) * sizeof(ranges[0]));
+		num_ranges++;
+
+		ranges[i].start = start;
+		ranges[i].end = end;
+	}
+}
+
+/**
+ * add_occupied_range_ulong - mark a range as occupied
+ * @start: start of block to occupy
+ * @end: start of block to occupy
+ *
+ * Call add_occupied_range() after casting to ulong @start and @end to
+ * void * pointers.
+ */
+void add_occupied_range_ulong(unsigned long start, unsigned long end)
+{
+	add_occupied_range((void *)start, (void *)end);
+}
+
+/**
+ * add_known_ranges - occupy some known regions
+ * @dt_blob: a flattend device tree to occupy, or NULL to skip
+ *
+ * call add_occupied_range() for the wrapper, loader supplied initrd,
+ * and, if not %NULL, the device tree blob @dt_blob and any reserved
+ * memory ranges therein.
+ */
+void add_known_ranges(struct boot_param_header *dt_blob)
+{
+	unsigned long long rstart, rlen, rend, *rsrv;
+
+	add_occupied_range(_start, _end);
+
+	add_occupied_range_ulong(loader_info.initrd_addr,
+		loader_info.initrd_addr + loader_info.initrd_size);
+
+	if (dt_blob == NULL)
+		return;
+
+	add_occupied_range(dt_blob, (void *)dt_blob + dt_blob->totalsize);
+
+	/* only support 8-byte reserve map.  Only care about < 4G */
+	rsrv = (void *)dt_blob + dt_blob->off_mem_rsvmap;
+	do {
+		rstart = *rsrv++;
+		rlen = *rsrv++;
+		rend = rstart + rlen;
+
+		if (rlen && rstart < UINT_MAX) {
+			if (rend < UINT_MAX)
+				add_occupied_range_ulong(rstart, rend);
+			else
+				add_occupied_range_ulong(rstart, UINT_MAX);
+		}
+	} while (rlen);
+}
+
+/**
+ * ranges_init_malloc - initialize malloc heap in a free memory range.
+ *
+ * Call simple_alloc_init using the largest gap between occupied ranges.
+ * Does not consider before the first or after the last range.
+ */
+void ranges_init_malloc(void)
+{
+	int i;
+	unsigned long size = 0;
+	void *heap_start, *heap_end;
+
+	/*
+	 * Allow the beginning for the kernel and the end for
+	 * other things the platform might want to have reserved.
+	 */
+
+	heap_start = NULL;		/* avoid gcc warning */
+	for (i=1; i < num_ranges; i++) {
+		unsigned long newsize;
+
+		newsize = ranges[i].start - ranges[i-1].end;
+		if (newsize > size) {
+			size = newsize;
+			heap_start = ranges[i-1].end;
+		}
+	}
+
+	if (size < 4 * 1024 * 1024)
+		fatal("Can't find a sutiable gap (largest 0x%lx)", size);
+
+	printf("putting heap between %p and %p size 0x%lx\n\r", heap_start,
+			heap_start + size, size);
+	heap_end = simple_alloc_init(heap_start, size * 7 / 8,
+			PAGE_SIZE, /* max num alloc */ 4096);
+	if (heap_end > (heap_start + size))
+		fatal("heap alloc overflowed gap (%p)\n\r", heap_end);
+
+	add_occupied_range(heap_start, heap_end);
+}
+
+/**
+ * ranges_vmlinux_alloc - an optonal kernel allocator.
+ * @size: the image size of the kernel
+ *
+ * Searches for a location to put the kernel, then reserve that range
+ * and the area to which the kernel will relocate itself.   First try
+ * address %0.  If that is blocked by a previos call to add_occupied_range(),
+ * try malloc().  If that also fails search for free space between the
+ * occupied ranges or between the last range and rmo_end.
+ */
+void *ranges_vmlinux_alloc(unsigned long size)
+{
+	void *addr;
+	int i;
+
+	/* Assume _start to _end is occupied */
+	addr = (void *)0;
+	if (addr + size < ranges[0].start)
+		goto occupy;
+
+	addr = malloc(size);
+	if (addr)
+		goto out;
+
+	for (i=1; i < num_ranges; i++) {
+		if (size < ranges[i].start - ranges[i-1].end)
+			goto occupy_range;
+	}
+	if (size < rmo_end - ranges[i-1].end)
+		goto occupy_range;
+
+	fatal("Unable to find a 0x%lx byte gap for the kernel\n", size);
+
+occupy_range:
+	addr = ranges[i-1].end;
+occupy:
+	add_occupied_range(addr, addr + size);
+out:
+	/*
+	 * Assume the kernel will decompress to 0, but don't implicity
+	 * create a new gap below the current first range.
+	 */
+	if ((unsigned long)ranges[0].end < size)
+		add_occupied_range_ulong(0, size);
+
+	return addr;
+}
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 03:48:20.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 03:48:29.000000000 -0500
@@ -22,6 +22,8 @@
 
 typedef void (*kernel_entry_t)(unsigned long r3, unsigned long r4, void *r5);
 
+struct boot_param_header;
+
 /* Platform specific operations */
 struct platform_ops {
 	void	(*fixups)(void);
@@ -96,6 +98,14 @@ void send_slaves_to_kernel(void *vmlinux
 void slaves_are_low(void);
 void wait_slaves_moved(void);
 
+/* memory ranges */
+extern void *rmo_end;
+void add_occupied_range(void *start, void *end);
+void add_occupied_range_ulong(unsigned long start, unsigned long end);
+void add_known_ranges(struct boot_param_header *dt_blob);
+void ranges_init_malloc(void);
+void *ranges_vmlinux_alloc(unsigned long size);
+
 static inline void *finddevice(const char *name)
 {
 	return (dt_ops.finddevice) ? dt_ops.finddevice(name) : NULL;
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 03:48:20.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 03:48:29.000000000 -0500
@@ -42,7 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
-		marshal.c \
+		marshal.c memranges.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c

[PATCH 6/15] bootwrapper: switch 64 bit cpus to 32 bit mode

[Milton Miller]

Add code to check if the processor is in 64 or 32 bit mode using
only instructions from the 32 bit subset.  If the processor is in
64 bit mode, switch to 32 bit mode by clearing MSR[SF].

Also add a 64 bit procedure descriptor to use as a elf64 entry
point.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Merged into crt0.S instead of a separate file with a separate entry
point.

Index: kernel/arch/powerpc/boot/crt0.S
===================================================================
--- kernel.orig/arch/powerpc/boot/crt0.S	2007-07-10 03:33:36.000000000 -0500
+++ kernel/arch/powerpc/boot/crt0.S	2007-07-10 03:39:08.000000000 -0500
@@ -17,11 +17,47 @@
 _zimage_start_opd:
 	.long	_zimage_start, 0, 0, 0
 
+	/* a procedure descriptor used when pretending to be elf64_powerpc */
+	.balign	8
+	.globl	_zimage_start_64
+_zimage_start_64:
+	.long	0, _zimage_start	/* big endian, supported reloc ppc32 */
+	.long	0, 0, 0, 0, 0, 0
+
+
 	.weak	_zimage_start
 	.globl	_zimage_start
 _zimage_start:
 	.globl	_zimage_start_lib
 _zimage_start_lib:
+	/* Check if the processor is running in 32 bit mode, using
+	 * only 32 bit instructions which should be safe on 32 and
+	 * 64 bit processors.
+	 *
+	 * Subtract the bottom 32 bits of MSR from the full value
+	 * recording the result.  Since MSR[SF] is in the high word,
+	 * the result will be not-equal iff in 32 bit mode (either
+	 * the processor is a 32 bit processor or MSR[SF] = 0).
+	 */
+	mfmsr	r0		/* grab whole msr		*/
+	rlwinm	r8,r0,0,0,31	/* extract bottom word		*/
+	subf.	r8,r8,r0	/* subtract, same?		*/
+	beq	0f		/* yes: we are 32 bit mode	*/
+
+	/* We are in 64-bit mode.  This program must run in 32 bit
+	 * mode.  Assume we are actually running somewhere in the
+	 * low 32 bits of the address space, so we can just turn
+	 * off MSR[SF] which is bit 0.
+	 */
+	.machine push
+	.machine "ppc64"
+	rldicl	r0,r0,0,1
+	sync
+	mtmsrd	r0
+	isync
+	.machine pop
+0:	/* We are now in 32-bit mode */
+
 	/* Work out the offset between the address we were linked at
 	   and the address where we're running. */
 	bl	1f

[PATCH 7/15] bootwrapper: Add kexec callable zImage wrapper

[Milton Miller]

This code creates a 32 bit zImage wrapper for a 32 or 64 bit PowerPC
Linux kernel.   This allows you to kexec a zImage with its compressed
vmlinux instead of the uncompressed vmlinux elf.  The elf is also
packaged as a 64 bit elf for use by kexec-tools for 64 bit kernels.

Limitations:

The memory node off the root with a name starting with "memory" must
contain enough free memory (not in the reserved ranges) in the first
reg range to uncompress the the kernel with padding.

Signed-off-by: Milton Miller <miltonm@bga.com>
---
I split a few more routines out from this file.

I removed the call serial_console_init because some of the serial
console require a loader supplied virtural address mapping which
will not exist when called from kexec.

Successfully boots from and to a 64 bit kernel when loaded at 0 and
when loaded after the kernel _end on machines with with initrds,
rtas, and various other data reserved.

The next patch needed to load using kexec-tools without an externally
generated flat device tree.

Index: work.git/arch/powerpc/boot/kexec.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/kexec.c	2007-07-10 16:37:30.000000000 -0500
@@ -0,0 +1,181 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corporation 2007
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+
+#include "ops.h"
+#include "stdio.h"
+#include "flatdevtree.h"
+#include "page.h"
+#include "types.h"
+
+extern char _start[];
+extern char _end[];
+
+BSS_STACK(16*1024);
+
+static void find_console_from_tree(void)
+{
+	int rc = -1;
+
+	if (rc) {
+		/* no console, oh well */
+	}
+}
+
+/* fixme: use find_device_by_type "memory" and xlate_reg */
+static void find_rmo_end(void)
+{
+	unsigned int na, ns, reg[4], *rp;
+	void *devp;
+	int rc;
+
+	devp = finddevice("/");
+	if (!devp)
+		fatal("Ack, device-tree root");
+	rc = getprop(devp, "#address-cells", &na, sizeof(na));
+	if (rc != sizeof(na))
+		fatal("Ack, no #address-cells in root");
+	rc = getprop(devp, "#size-cells", &ns, sizeof(ns));
+	if (rc != sizeof(ns))
+		fatal("Ack, no #size-cells in root");
+	if (!na || !ns || na + ns > ARRAY_SIZE(reg))
+		fatal("#addr-cells or #size-cells unusable");
+	do {
+		devp = finddevice("/memory@0");
+		if (!devp)
+			devp = finddevice("/memory");
+		if (!devp)
+			devp = finddevice("/memory@00000000");
+		if (!devp)
+			devp = finddevice("/memory@0000000000000000");
+		if (!devp)
+			fatal("Ack, can't find  memory");
+		rc = getprop(devp, "reg", reg, sizeof(reg));
+		if (rc < (na + ns) * sizeof(int))
+			fatal("Ack, no valid reg property in memory");
+
+		rp = &reg[0];
+		while (na--) {
+			if (*rp)
+				fatal("didn't find memory at 0");
+			rp++;
+		}
+		while (--ns) {
+			if (*rp) {
+				/* first range >= 4G , use 2G */
+				rmo_end = (void *)0x80000000;
+				return;
+			}
+			rp++;
+		}
+	} while (0);
+
+	rmo_end = (void *)*rp;
+}
+
+/**
+ * setup_initial_heap - setup a small heap in the bss
+ * Using a preallocated heap, setup for scanning the device tree.
+ * Intended for the initial read while the tree will remain read-only so
+ * a minimal malloc and search limit can be used.  This way we don't have
+ * lots of data or bss to clear.
+ */
+static void setup_initial_heap(void)
+{
+	static char initial_heap[8*1024];
+	void *heap_end;
+
+	heap_end = simple_alloc_init(initial_heap,
+			sizeof(initial_heap) * 7 / 8,
+			sizeof(long), 64);
+
+	if (heap_end - sizeof(initial_heap) > (void *)&initial_heap[0])
+		fatal("Initial heap too big\n\r");
+}
+
+static void early_scan_flat_tree(struct boot_param_header *dt_blob)
+{
+	int rc;
+
+	rc = ft_init(dt_blob, dt_blob->totalsize, 50);
+	if (rc)
+		fatal("couldn't initialize device-tree\n\r");
+
+	find_rmo_end();
+	dt_find_initrd();
+}
+
+static void init_flat_tree(struct boot_param_header *dt_blob)
+{
+	int rc;
+
+	rc = ft_init(dt_blob, dt_blob->totalsize, /* max_finddevice */ 1024);
+	if (rc)
+		fatal("Unable to initialize device_tree library!\n\r");
+}
+
+static void *saved_vmlinux_addr;
+
+static void *kexec_vmlinux_alloc(unsigned long size)
+{
+	void *addr;
+
+	addr = ranges_vmlinux_alloc(size);
+
+	saved_vmlinux_addr = addr;
+	return addr;
+}
+
+static void kexec_fixups(void)
+{
+	wait_slaves_moved();
+}
+
+static unsigned long (*finalize_chain)(void);
+
+static unsigned long kexec_finalize(void)
+{
+	send_slaves_to_kernel(saved_vmlinux_addr);
+
+	return finalize_chain();
+}
+
+void kexec_platform_init(struct boot_param_header *dt_blob)
+{
+	slaves_are_low();
+	move_slaves_up();
+
+	setup_initial_heap();
+	early_scan_flat_tree(dt_blob);
+
+	/* drivers can malloc and read the tree, but not realloc later
+	 * or modify the tree now.
+	 */
+	if (!console_ops.write)
+		find_console_from_tree();
+
+	add_known_ranges(dt_blob);
+	ranges_init_malloc();
+	init_flat_tree(dt_blob);
+
+	platform_ops.vmlinux_alloc = kexec_vmlinux_alloc;
+	platform_ops.fixups = kexec_fixups;
+	finalize_chain = dt_ops.finalize;
+	dt_ops.finalize = kexec_finalize;
+}
Index: work.git/arch/powerpc/boot/crt0_kexec.S
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/crt0_kexec.S	2007-07-10 16:37:30.000000000 -0500
@@ -0,0 +1,45 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright 2007 IBM Corporation.
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ *
+ */
+	/*
+	 * The kernel calls out to the first image with
+	 * r3 = boot cpu, r4 = entrypoint, r5 = 0
+	 *
+	 * kexec-tools purgatory calls this as it would a linux kernel:
+	 * r3 = boot block, r4 = entrypoint, r5 = 0
+	 *
+	 * The boot block boot_cpu field has been filled in.
+	 *
+	 * kexec-tools and its purgatory are suppposed to copy SMP_SLAVE_SIZE
+	 * bytes from the from entry point, but aparently instead it copies
+	 * from the image start.
+	 */
+	.globl _zimage_start
+_zimage_start:
+
+#include "marshal_low.S"
+
+	.globl	_zimage_start_plat
+_zimage_start_plat:
+	b	_zimage_start_lib
+
+	.globl	platform_init
+platform_init:
+	b	kexec_platform_init
Index: work.git/arch/powerpc/boot/wrapper
===================================================================
--- work.git.orig/arch/powerpc/boot/wrapper	2007-07-10 16:30:15.000000000 -0500
+++ work.git/arch/powerpc/boot/wrapper	2007-07-10 16:37:30.000000000 -0500
@@ -133,6 +133,12 @@ coff)
     platformo=$object/of.o
     lds=$object/zImage.coff.lds
     ;;
+kexec)
+    platformo=$object/crt0_kexec.o
+    ;;
+kexec64)
+    platformo="-e _zimage_start_64 $object/crt0_kexec.o"
+    ;;
 miboot|uboot)
     # miboot and U-boot want just the bare bits, not an ELF binary
     ext=bin
@@ -190,6 +196,9 @@ uboot)
     fi
     exit 0
     ;;
+kexec64)
+    ${CROSS}objcopy -O elf64-powerpc $ofile
+    ;;
 esac
 
 addsec() {
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 16:34:28.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 16:37:30.000000000 -0500
@@ -42,11 +42,11 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
-		marshal.c memranges.c \
+		marshal.c memranges.c kexec.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c
-src-plat := of.c cuboot-83xx.c cuboot-85xx.c holly.c \
+src-plat := of.c cuboot-83xx.c cuboot-85xx.c holly.c crt0_kexec.S \
 		cuboot-ebony.c treeboot-ebony.c prpmc2800.c \
 		ps3-head.S ps3-hvcall.S ps3.c
 src-boot := $(src-wlib) $(src-plat) empty.c
@@ -126,6 +126,9 @@ quiet_cmd_wrap	= WRAP    $@
       cmd_wrap	=$(CONFIG_SHELL) $(wrapper) -c -o $@ -p $2 $(CROSSWRAP) \
 		$(if $3, -s $3)$(if $4, -d $4)$(if $5, -i $5) vmlinux
 
+kexec-$(CONFIG_PPC32)			+= zImage.kexec
+kexec-$(CONFIG_PPC64)			+= zImage.kexec64
+
 image-$(CONFIG_PPC_PSERIES)		+= zImage.pseries
 image-$(CONFIG_PPC_MAPLE)		+= zImage.pseries
 image-$(CONFIG_PPC_IBM_CELL_BLADE)	+= zImage.pseries
@@ -138,6 +141,7 @@ image-$(CONFIG_PPC_HOLLY)		+= zImage.hol
 image-$(CONFIG_PPC_PRPMC2800)		+= zImage.prpmc2800
 image-$(CONFIG_PPC_ISERIES)		+= zImage.iseries
 image-$(CONFIG_DEFAULT_UIMAGE)		+= uImage
+image-$(CONFIG_KEXEC)			+= $(kexec-y)
 
 ifneq ($(CONFIG_DEVICE_TREE),"")
 image-$(CONFIG_PPC_83xx)		+= cuImage.83xx
@@ -151,7 +155,7 @@ ifeq ($(CONFIG_PPC32),y)
 image-$(CONFIG_PPC_PMAC)	+= zImage.coff zImage.miboot
 endif
 
-initrd-  := $(patsubst zImage%, zImage.initrd%, $(image-n) $(image-))
+initrd-  := $(patsubst zImage%, zImage.initrd%, $(image-n) $(image-) $(kexec-))
 initrd-y := $(patsubst zImage%, zImage.initrd%, \
 		$(patsubst treeImage%, treeImage.initrd%, $(image-y)))
 initrd-y := $(filter-out $(image-y), $(initrd-y))
@@ -219,8 +223,8 @@ install: $(CONFIGURE) $(addprefix $(obj)
 	sh -x $(srctree)/$(src)/install.sh "$(KERNELRELEASE)" vmlinux System.map "$(INSTALL_PATH)" $<
 
 # anything not in $(targets)
-clean-files += $(image-) $(initrd-) zImage zImage.initrd cuImage.* treeImage.* \
-	otheros.bld
+clean-files += $(image-) $(initrd-) zImage zImage.initrd cuImage.* \
+	treeImage.* $(kexec-)
 
 # clean up files cached by wrapper
 clean-kernel := vmlinux.strip vmlinux.bin
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 16:34:28.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 16:37:30.000000000 -0500
@@ -90,6 +90,7 @@ void *simple_alloc_init(char *base, unsi
 void flush_cache(void *, unsigned long);
 int dt_xlate_reg(void *node, int res, unsigned long *addr, unsigned long *size);
 int dt_xlate_addr(void *node, u32 *buf, int buflen, unsigned long *xlated_addr);
+void kexec_platform_init(struct boot_param_header *dt_blob);
 
 /* marshal slave cpus around to kernel */
 void move_slaves_up(void);

[PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

kexec-tools still produces a version 2 device tree, while the
libraries in the wrapper only support version 16 and later.

Add a routine to convert a v2 flat device tree to a v16 one inplace
by inserting OF_DT_NOP and chomping full path.  Make space for new
headers by moving and then chomping the OF_DT_NOPs.

Signed-off-by: Milton Miller <miltonm@bga.com>
---
Rediffed Makefile

Index: work.git/arch/powerpc/boot/flatdevtree_conv.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/flatdevtree_conv.c	2007-07-10 16:08:20.000000000 -0500
@@ -0,0 +1,280 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corporation 2007
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+#include "flatdevtree.h"
+#include "stdio.h"
+#include "ops.h"
+
+#define MIN_VERSION 2
+#define OUT_VERSION 16
+#define OUT_COMPAT 16
+
+#ifdef NO_CHECK
+static int check_v123_tree(u32 *start, u32 *limit)
+{
+	return 0;
+}
+#else
+/**
+ * check_v123_tree - check integrety of a version 1, 2, or 3 tree
+ * @start: the start of the device tree struct
+ * @limit: the end of the region for the struct
+ * structural checks on device_tree
+ */
+static int check_v123_tree(u32 *start, u32 *limit)
+{
+	u32 len;
+	int depth = 0;
+	u32 *dtp = start;
+
+	while (dtp < limit)
+		switch (*dtp) {
+		case OF_DT_END:
+			if (depth)
+				return -1;
+			return ++dtp - start;
+		case OF_DT_NOP:
+			dtp++;
+			break;
+		case OF_DT_END_NODE:
+			dtp++;
+			depth--;
+			break;
+		case OF_DT_BEGIN_NODE:
+			len = strlen((char *)(++dtp));
+			/* check path is suffix to previous? */
+			dtp += 1 + (len / 4);
+			depth++;
+			break;
+		case OF_DT_PROP:
+			len = dtp[1];
+			dtp += 3;
+			if ((len >= 8) && ((long)dtp & 4))
+				dtp++;
+			dtp += (len + 3) / 4;
+			break;
+		default:
+			return -1;
+		}
+	return -1;	/* no OF_DT_END */
+}
+#endif
+
+/**
+ * nop_to_v16 - add %OF_DT_NOP to hide alignment differences
+ * @dtp: pointer to the beginning of the struct area to modify
+ * insert %OF_DT_NOP into the dt_struct @dtp to make it v16 from v1, 2, or 3.
+ */
+static int nop_to_v16(u32 *dtp)
+{
+	int nops = 0;
+	char *p, *s;
+	int len;
+	u32 *next;
+
+	while (*dtp != OF_DT_END)
+		switch (*dtp) {
+		case OF_DT_BEGIN_NODE:
+			/* v2 & v3 names are full path, v16+ is relative */
+			p = (char *)(++dtp);
+			len = strlen(p);
+			next = dtp + 1 + len / 4;
+
+			for (s = p + len; *s != '/'; s--)
+				if (s == p)
+					fatal("name %s has no '/'", p);
+
+			len -= s++ - p;		/* not the slash but the nul */
+			memmove(p, s, len);
+			while (len % 4)
+				p[len++] = '\0';
+			dtp += len / 4;
+			while (dtp != next) {
+				*dtp++ = OF_DT_NOP;
+				nops++;
+			}
+			break;
+		case OF_DT_PROP:
+			/* convert from align_8 to align_4 via prefixing nop */
+			len = dtp[1];
+			if ((len >= 8) && !((long)dtp & 4)) {
+				memmove(dtp+1, dtp, 12);
+				*dtp++ = OF_DT_NOP;
+				nops++;
+			}
+			dtp += 3 + (len + 3)/4;
+			break;
+		default:
+			fatal("%s: unrecognised tag %d at %p\n", __FUNCTION__,
+				*dtp, dtp);
+		case OF_DT_NOP:
+			nops ++;
+			/* fall through */
+		case OF_DT_END_NODE:
+			dtp ++;
+			break;
+		}
+	return nops;
+}
+
+#if MIN_VERSION < 3 || OUT_VERSION > 16
+/**
+ * move_nops_fwd - move nops in a v16 dt_struct to the beginning
+ * @start - device tree starting address
+ * @count - number of %OF_DT_NOP cells to move
+ */
+static void move_nops_fwd(u32 *start, int count)
+{
+	u32 *dtp = start;
+	int len;
+	while (count)
+		switch (*dtp) {
+		case OF_DT_NOP:
+			memmove(start+1,start,(dtp-start) * 4);
+			*start++ = OF_DT_NOP;
+			dtp++;
+			count--;
+			break;
+		case OF_DT_END_NODE:
+			dtp++;
+			break;
+		case OF_DT_BEGIN_NODE:
+			len = strlen((char *)(++dtp));
+			dtp += 1 + len / 4;
+			break;
+		case OF_DT_PROP:
+			len = dtp[1];
+			dtp += 3 + (len + 3) / 4;
+			break;
+		case OF_DT_END:
+			fatal("Not enough nops -- need %d more\n", count);
+			return;
+		default:
+			fatal("%s: unknown tag %d at %p", __FUNCTION__, *dtp, dtp)
+		}
+}
+#endif
+
+/**
+ * conv_flattree_inplace upgrade the version of a boot_param_header
+ * @tree: pointer to the device tree header to convert
+ *
+ * Converts a v1, 2, 3 device tree (of at least MIN_VERSION)
+ * in place to OUT_VERSION (16) format, usable by flatdevtree.c
+ */
+void conv_flattree_inplace(struct boot_param_header *tree)
+{
+	u32 *dtp;
+	u32 need = 0, nops;
+	int slen;
+
+	if (tree->magic != OF_DT_HEADER)
+		fatal("%s: no magic", __FUNCTION__);
+
+	if (tree->last_comp_version > 3)
+		return;			/* don't know what to do */
+
+	if (tree->version < MIN_VERSION) {
+		printf("%s: Warning: can't handle version %d tree\n",
+				__FUNCTION__, tree->version);
+		return;
+	}
+
+	if (tree->version < 2)
+		need++;		/* boot_cpu_id */
+
+	if (tree->version < 3)
+		need++;		/* dt_string_size */
+
+	if (OUT_VERSION > 16)
+		need++;		/* dt_struct_size */
+
+	dtp = (void *)tree + tree->off_dt_struct;
+
+	slen = check_v123_tree(dtp, (void *)tree + tree->totalsize);
+	if (slen < 0)
+		fatal("device tree check failed\n");
+
+	nops = nop_to_v16(dtp);
+
+	if (need & 1)		/* keep 8 byte alignment of mem reserve */
+		need++;
+
+	if (need > nops)
+		fatal("Didn't find enough space to add new header fields\n\r"
+			"(needed %d found %d tree %p)", need, nops, tree);
+
+	/* ok now compress the dtb struct */
+	move_nops_fwd(dtp, need);
+	dtp += need;
+
+	/*
+	 * move mem_rsvmap and dt_strings if they are before dt_struct
+	 * onto our nops .  Adjust start addresses for the 3 sections.
+	 */
+	if ((tree->off_mem_rsvmap < tree->off_dt_struct) ||
+		(tree->off_dt_strings < tree->off_dt_struct)) {
+		int start, end;
+		void *ptr;
+
+		if (tree->off_mem_rsvmap < tree->off_dt_strings)
+			start = tree->off_mem_rsvmap;
+		else
+			start = tree->off_dt_strings;
+
+		end = tree->off_dt_struct;
+		ptr = (void *)tree + start;
+
+		memmove(ptr + 4 * need, ptr, end - start);
+
+		if (tree->off_mem_rsvmap < tree->off_dt_struct)
+			tree->off_mem_rsvmap += 4 * need;
+		if (tree->off_dt_strings < tree->off_dt_struct)
+			tree->off_dt_strings += 4 * need;
+	}
+	tree->off_dt_struct += 4 * need;
+
+	/* ok now we have space to extend the header. */
+	if (tree->version < 2 && MIN_VERSION < 2) {
+		tree->boot_cpuid_phys = 0;	/* default, caller can fix */
+	}
+
+	/* calculate size of dt_strings_size */
+	if (tree->version < 3 && MIN_VERSION < 3) {
+		int end = tree->totalsize;
+
+		if (tree->off_dt_strings < tree->off_mem_rsvmap)
+			end = tree->off_mem_rsvmap;
+
+		if ((tree->off_dt_strings < tree->off_dt_struct) &&
+				(end > tree->off_dt_struct))
+			end = tree->off_dt_struct;
+
+		tree->dt_strings_size = end - tree->off_dt_strings;
+	}
+
+#if OUT_VERSION > 16
+	tree->dt_struct_size = 4 * slen;
+#endif
+
+	tree->version = OUT_VERSION;
+	tree->last_comp_version = OUT_COMPAT;
+
+	return;
+}
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 16:05:13.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 16:05:13.000000000 -0500
@@ -42,7 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
-		marshal.c memranges.c kexec.c \
+		flatdevtree_conv.c marshal.c memranges.c kexec.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 16:05:13.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 16:05:13.000000000 -0500
@@ -80,7 +80,10 @@ struct loader_info {
 };
 extern struct loader_info loader_info;
 
+struct boot_param_header;
+
 void start(void);
+void conv_flattree_inplace(struct boot_param_header *tree);
 int ft_init(void *dt_blob, unsigned int max_size, unsigned int max_find_device);
 int serial_console_init(void);
 int ns16550_console_init(void *devp, struct serial_console_data *scdp);
Index: work.git/arch/powerpc/boot/kexec.c
===================================================================
--- work.git.orig/arch/powerpc/boot/kexec.c	2007-07-10 16:05:13.000000000 -0500
+++ work.git/arch/powerpc/boot/kexec.c	2007-07-10 16:05:13.000000000 -0500
@@ -162,6 +162,7 @@ void kexec_platform_init(struct boot_par
 	move_slaves_up();
 
 	setup_initial_heap();
+	conv_flattree_inplace(dt_blob);
 	early_scan_flat_tree(dt_blob);
 
 	/* drivers can malloc and read the tree, but not realloc later

[PATCH 9/15] bootwrapper: rtas support

[Milton Miller]

This code provides a console write that calls put-term-char.

To avoid PIC relocation of the absolute rtas addresses, hide the
actual call to rtas in assembly and declare all variables as int.

An instantiated rtas will be protected by a reserved range in the
device tree, so no explicit call to add_occupied_range is needed
here.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Removed use_call_rtas(); instead put asm() at the file level.

Index: work.git/arch/powerpc/boot/rtas.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/rtas.c	2007-07-10 04:39:43.000000000 -0500
@@ -0,0 +1,146 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright 2007 IBM Corporation.
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ *
+ */
+
+#include "ops.h"
+#include "stdio.h"
+#include "flatdevtree.h"
+
+static int rtas_entry;
+static int rtas_base;
+static int rtas_size;
+
+static void find_rtas(void)
+{
+	int rc;
+	void *devp;
+	char *str;
+
+	devp = finddevice("/rtas");
+	if (! devp)
+		return;
+
+	str = "linux,rtas-entry";
+	rc = getprop(devp, str, &rtas_entry, sizeof(rtas_entry));
+	if (rc < 0)
+		return;
+	if (rc != sizeof(rtas_entry))
+		goto fail;
+
+	str = "rtas-size";
+	rc = getprop(devp, str, &rtas_size, sizeof(rtas_size));
+	if (rc < 0)
+		return;
+	if (rc != sizeof(rtas_size))
+		goto fail;
+
+	str = "linux,rtas-base";
+	rc = getprop(devp, str, &rtas_base, sizeof(rtas_base));
+	if (rc < 0) {
+		printf("rtas-size but no linux,rtas-base in /rtas.  "
+			"disabling wrapper rtas interface\n\r");
+		rtas_entry = 0;
+		return;
+	}
+
+	if (rc != sizeof(rtas_base))
+		goto fail;
+
+	return;
+
+
+fail:
+	printf("Unexpected length %d of %s property in /rtas.\n\r"
+			"disabling wrapper rtas interface\n\r", rc, str);
+	rtas_entry = 0;
+	return;
+}
+
+/*
+ * PIC relocation of function pointers happens at call time.
+ * We have an absolute out-of-image address.   So tell C they
+ * are just integers, and hide the call as an out-of-file
+ * function.
+ */
+__asm__ (
+		"	.globl call_rtas\n"
+		"	call_rtas: mtctr 5\n"
+		"	bctr\n"
+		);
+
+void call_rtas(int args[], int base, int entry);
+
+
+static int put_term_char;
+
+static void rtas_put_term_write(const char *buf, int len)
+{
+	int i, args[5];
+
+	args[0] = put_term_char;
+	args[1] = 1;	/* num inputs */
+	args[2] = 1;	/* num outputs */
+
+	for (i=0; i < len; ) {
+		args[3] = buf[i];
+		args[4] = 0;
+
+		call_rtas(args, rtas_base, rtas_entry);
+		if (args[4] == 0)		/* SUCCESS */
+			i++;
+		else if (args[4] == -1)		/* HARDWARE_ERROR */
+			break;
+		/* else retry */
+	}
+}
+
+int rtas_console_init(void)
+{
+	void *devp;
+	int rc;
+
+
+	devp = finddevice("/rtas");
+	if (!devp)
+		return -1;
+
+	if (!rtas_entry)
+		find_rtas();
+	if (!rtas_entry)
+		return -1;
+
+	rc = getprop(devp, "put-term-char", &put_term_char,
+			sizeof(put_term_char));
+	if (rc == sizeof(put_term_char))
+		console_ops.write = rtas_put_term_write;
+	else
+		put_term_char = -1;
+
+	return put_term_char == -1 ? -1 : 0;
+}
+
+/* for debug, hard code */
+void use_rtas_console(int entry, int base, int tc)
+{
+	rtas_entry = entry;
+	rtas_base = base;
+	put_term_char = tc;
+	console_ops.write = rtas_put_term_write;
+}
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 04:40:11.000000000 -0500
@@ -94,6 +94,8 @@ void flush_cache(void *, unsigned long);
 int dt_xlate_reg(void *node, int res, unsigned long *addr, unsigned long *size);
 int dt_xlate_addr(void *node, u32 *buf, int buflen, unsigned long *xlated_addr);
 void kexec_platform_init(struct boot_param_header *dt_blob);
+int rtas_console_init(void);
+void use_rtas_console(int entry, int base, int tc);
 
 /* marshal slave cpus around to kernel */
 void move_slaves_up(void);
Index: work.git/arch/powerpc/boot/kexec.c
===================================================================
--- work.git.orig/arch/powerpc/boot/kexec.c	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/kexec.c	2007-07-10 04:40:11.000000000 -0500
@@ -33,6 +33,9 @@ static void find_console_from_tree(void)
 {
 	int rc = -1;
 
+	rc = rtas_console_init();
+	if (!rc)
+		return;
 	if (rc) {
 		/* no console, oh well */
 	}
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 04:40:11.000000000 -0500
@@ -43,7 +43,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
 		flatdevtree_conv.c marshal.c memranges.c kexec.c \
-		ns16550.c serial.c simple_alloc.c div64.S util.S \
+		ns16550.c serial.c simple_alloc.c div64.S util.S rtas.c \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c
 src-plat := of.c cuboot-83xx.c cuboot-85xx.c holly.c crt0_kexec.S \

[PATCH 10/15] bootwrapper: add cpio file extraction library.

[Milton Miller]

Add a library to search through a cpio or initramfs to a specified
path contained in a cpio.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Status: tested and working.  

This file is designed to also be usable in a stand alone user space
application.

Index: work.git/arch/powerpc/boot/cpio.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/cpio.c	2007-07-10 04:03:40.000000000 -0500
@@ -0,0 +1,306 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright 2007 IBM Corporation.
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ *
+ */
+
+#include "cpio.h"
+#include "string.h"
+
+struct cpio_header {
+	char ino[8];
+	char mode[8];
+	char uid[8];
+	char gid[8];
+	char nlink[8];
+	char mtime[8];
+	char filesize[8];
+	char maj[8];
+	char min[8];
+	char rmaj[8];
+	char rmin[8];
+	char namesize[8];
+	char chksum[8];
+} cpio_header_buf;
+
+static int check_magic(char magic[6])
+{
+	return !memcmp(magic,"070701",6) || !memcmp(magic,"070702",6);
+}
+
+static int read_magic(struct gunzip_state *stream)
+{
+	int len;
+	char magic[6];
+
+	len = gunzip_partial(stream, magic, sizeof(magic));
+	if (len == 0)
+		return 0;
+
+	if (len == sizeof(magic) && check_magic(magic))
+		return len;
+
+
+	/* Not the right magic or short read.  We might have stumbled
+	 * onto a compressed archive immediately following an
+	 * uncompressed one, or just some NUL bytes at the end of the
+	 * archive.  Inform the higher layers by the negative length.
+	 */
+	return -len;
+}
+
+static int get_cpio_header(struct gunzip_state *stream)
+{
+	int len;
+
+	len = read_magic(stream);
+	if (len <= 0)
+		return len;
+
+	gunzip_exactly(stream, &cpio_header_buf, sizeof(cpio_header_buf));
+	len += sizeof(cpio_header_buf);
+
+	return len;
+}
+
+static unsigned int cpio_str_to_num(char hexascii[8])
+{
+	unsigned int num = 0;
+	char c;
+	int d;
+
+	for (d=0; d < 8; d++) {
+		c = hexascii[d];
+		num <<= 4;
+		if (c >= '0' && c <= '9') {
+			num += c - '0';
+		} else if (c >= 'A' && c <= 'F') {
+			num += c - 'A' + 10;
+		} else if (c >= 'a' && c <= 'f') {
+			num += c - 'a' + 10;
+		} else {
+			cpio_error("bad cpio archive header: "
+					"invalid a hex digit");
+		}
+	}
+
+	return num;
+}
+
+static char name_buf[MAX_PATH+1];
+static const char cpio_end[] = "TRAILER!!!";
+#define CPIO_END_LEN sizeof(cpio_end)
+
+/* check_next_file
+ * Look for @path in @stream.  Set @consumed to the number of bytes
+ * succesfully read and processed.  return 1 on match, 0 for discarding
+ * an unmatched file, -1 on end of archive (either detected trailer or
+ * EOF on stream), or -(1 + bytes read) for a short read or bad magic
+ * number.  (For the short or bad read, the consumed is not changed).
+ */
+static int check_next_file(char *path, int pathlen,
+		struct gunzip_state *stream, int *consumed)
+{
+	int len, total, match;
+
+	if (pathlen > MAX_PATH) {
+		cpio_error("path too long to search\n");
+	}
+	total = get_cpio_header(stream);
+	if (total <= 0)
+		return total - 1;
+
+	len = cpio_str_to_num(cpio_header_buf.namesize);
+
+	if (len == pathlen || len == CPIO_END_LEN) {
+		gunzip_exactly(stream, name_buf, len);
+		total += len;
+		match = !strcmp(name_buf, path);
+		if (!match && !cpio_str_to_num(cpio_header_buf.filesize))
+			match = -!strcmp(name_buf, cpio_end);
+	} else {
+		gunzip_discard(stream, len);
+		total += len;
+		name_buf[0] = '\0';
+		match = 0;
+	}
+
+	len = total % 4;
+	if (len) {
+		gunzip_discard(stream, 4 - len);
+		total += 4 - len;
+	}
+
+	if (!match) {
+		len = cpio_str_to_num(cpio_header_buf.filesize);
+		gunzip_discard(stream, len);
+		total += len;
+
+		len = total % 4;
+		if (len) {
+			gunzip_discard(stream, 4 - len);
+			total += 4 - len;
+		}
+	}
+
+	*consumed += total;
+	return match;
+}
+
+static char *this_buf;
+static int this_archive;
+
+/* find_in_cpio.
+ * find a pathname @path in a single cpio archive described by @stream.
+ * Return is the same as check_next_file.
+ */
+int find_in_cpio(char *path, struct gunzip_state *stream)
+{
+	int found;
+	int pathlen = strlen(path) + 1;
+
+	this_archive = 0;
+	do {
+		found = check_next_file(path, pathlen, stream, &this_archive);
+	} while (found == 0);
+
+	return found;
+}
+
+/* find_in_initramfs
+ * Search a initramfs buffer for a given path name.  Returns 0 on
+ * not found, or 1 with @state ready to read the file.
+ *
+ * Note: this returns the first match, not the last.   The kernel
+ * decompressor  effectivly uses the last match.  This code also
+ * doesn't worry about the directories in the path existing.
+ */
+int find_in_initramfs(char *path, char *buf, int len,
+		struct gunzip_state *stream)
+{
+	int found, total = 0;
+	int pathlen = strlen(path) + 1;
+	int *ibuf;
+
+	do {
+		/* get to word boundary, but stop if not NUL */
+		for (; total % 4 && total < len - 4; total++)
+			if (buf[total])
+				break;
+
+		if ((total % 4) == 0) {
+			/* fast forward over NUL words.  */
+			for (ibuf = (int *)&buf[total];
+					total < len - sizeof(*ibuf) && !*ibuf;
+					ibuf++)
+			total += sizeof(*ibuf);
+		}
+
+		/* check remainder of a short archive -- it must all be
+		 * zero as both gzip header and cpio header are bigger
+		 * than this.
+		 */
+		if (total >= len - 4) {
+			for (;total < len; total++) {
+				if (buf[len]) {
+					cpio_error("Junk at end of buffer");
+				}
+			}
+			break;
+		}  else if (total % 4) {
+			/*
+			 * If we are unalinged and not at the end of the buffer
+			 * we must be following a compressed archive.  Only
+			 * NUL and gzip headers are allowed.  We skipped NUL.
+			 */
+			if (!(buf[total] == 0x1b && buf[total + 1] == 0x8b))
+				cpio_error("unalinged junk in buffer");
+		}
+
+		this_buf = buf + total;
+		this_archive = 0;
+
+		gunzip_start(stream, this_buf, len - total);
+
+		do {
+			found = check_next_file(path, pathlen, stream,
+					&this_archive);
+		} while (!found);
+
+		if (found > 0)
+			return found;
+
+		if (stream->s.workspace) {
+			int discard;
+
+			if (found < -1) {
+				cpio_error("Junk in compressed archive");
+			}
+
+			/* we either found EOF or TRAILER!!!.  In the later
+			 * case we need to discard to the end of the gzip
+			 * contents.
+			 */
+			do {
+				discard =  gunzip_partial(stream, name_buf,
+					MAX_PATH);
+				this_archive += discard;
+			} while (discard);
+
+			/*
+			 * Peek at how many input bytes were consumed.
+			 * reset our consumed input.
+			 */
+			total += stream->s.total_in + 8;
+
+			/* clean up zlib */
+			discard = gunzip_finish(stream, name_buf, 0);
+		} else {
+			if (this_archive % 4) {
+				cpio_error("Archive not multiple of 4?");
+			}
+			total += this_archive;
+
+			if (!this_archive) {
+				cpio_error("junk between archives");
+			}
+			/* don't check the < -1 of found, it might be an
+			 * archive.  This will be caught by the !this_archive
+			 * check on the next loop.
+			 */
+		}
+	} while (total < len);
+
+	return 0;
+}
+
+void get_cpio_info(void **archive_start, int *consumed)
+{
+	*archive_start = this_buf;
+	*consumed = this_archive;
+}
+
+int get_cpio_file_mode(void)
+{
+	return cpio_str_to_num(cpio_header_buf.mode);
+}
+
+int get_cpio_file_size(void)
+{
+	return cpio_str_to_num(cpio_header_buf.filesize);
+}
Index: work.git/arch/powerpc/boot/cpio.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/cpio.h	2007-07-10 04:03:40.000000000 -0500
@@ -0,0 +1,12 @@
+#include "gunzip_util.h"
+
+extern int find_in_cpio(char *path, struct gunzip_state *stream);
+extern int find_in_initramfs(char *path, char *buf, int len,
+		struct gunzip_state *state);
+extern void get_cpio_info(void **archive_start, int *consumed);
+extern int get_cpio_file_size(void);
+extern int get_cpio_file_mode(void);
+
+#define MAX_PATH 256
+
+extern void cpio_error(char *msg);
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 04:02:24.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 04:03:57.000000000 -0500
@@ -36,13 +36,13 @@ $(obj)/ebony.o: BOOTCFLAGS += -mcpu=440
 
 zlib       := inffast.c inflate.c inftrees.c
 zlibheader := inffast.h inffixed.h inflate.h inftrees.h infutil.h
-zliblinuxheader := zlib.h zconf.h zutil.h
+zliblinuxheader := zlib.h zconf.h zutil.h stat.h
 
 $(addprefix $(obj)/,$(zlib) gunzip_util.o main.o): \
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
-		flatdevtree_conv.c marshal.c memranges.c kexec.c \
+		flatdevtree_conv.c marshal.c memranges.c kexec.c cpio.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S rtas.c \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c

[PATCH 11/15] bootwrapper: allow vmlinuz to be an external payload

[Milton Miller]

Allow the boot wrapper to obtain the vmlinux elf from an external
platform supplied location instead of embedding it at link time.

When used with the cpio library, the kernel can be stored as a file
an file in the initramfs, allowing the same (compressed) copy of
the vmlinux elf to be used for both the kernel to execute and any
user space tools such as kexec (for reboot), crash dump, or oprofile.

Another use would be to uncompress directly from a memory mapped
region such as a flash partition.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Check for no find_vmlinuz method before calling it.

The source and source len varables are passed to the routine only
to allow them to be printed later.

This patch follows the current policy of using linker attached data
in preference to platform supplied data (matching the initrd and
command line).

While the platform could supply archive-start, length, and offset,
in loader data, it would require decompressing the skiped portion
twice when the offset is not known in advance.

The prpmc2800 code shows that this hook to find the kernel and the
the code to uncompress the header may want to be separated from the
vmlinux alloc, decompress, and flush code.  I haven't explored an
actual split.

Index: work.git/arch/powerpc/boot/main.c
===================================================================
--- work.git.orig/arch/powerpc/boot/main.c	2007-07-10 03:44:40.000000000 -0500
+++ work.git/arch/powerpc/boot/main.c	2007-07-10 04:04:02.000000000 -0500
@@ -47,10 +47,19 @@ static struct addr_range prep_kernel(voi
 	struct elf_info ei;
 	int len;
 
-	/* gunzip the ELF header of the kernel */
-	gunzip_start(&gzstate, vmlinuz_addr, vmlinuz_size);
-	gunzip_exactly(&gzstate, elfheader, sizeof(elfheader));
+	/* Initialze zlib.  Any attached kernel overrides find_vmlinuz */
+	if (vmlinuz_size)
+		gunzip_start(&gzstate, vmlinuz_addr, vmlinuz_size);
+	else
+		if (platform_ops.find_vmlinuz)
+			platform_ops.find_vmlinuz(&gzstate, &vmlinuz_addr,
+					&vmlinuz_size);
+		else
+			fatal("Can't find a kernel to boot: no attached "
+				"vmlinuz and no find_vmlinuz method\n\r");
 
+	/* gunzip and parse the ELF header of the kernel */
+	gunzip_exactly(&gzstate, elfheader, sizeof(elfheader));
 	if (!parse_elf64(elfheader, &ei) && !parse_elf32(elfheader, &ei))
 		fatal("Error: not a valid PPC32 or PPC64 ELF file!\n\r");
 
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 04:01:57.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 04:04:02.000000000 -0500
@@ -24,6 +24,8 @@ typedef void (*kernel_entry_t)(unsigned 
 
 struct boot_param_header;
 
+struct gunzip_state;
+
 /* Platform specific operations */
 struct platform_ops {
 	void	(*fixups)(void);
@@ -33,6 +35,8 @@ struct platform_ops {
 	void *	(*realloc)(void *ptr, unsigned long size);
 	void	(*exit)(void);
 	void *	(*vmlinux_alloc)(unsigned long size);
+ 	void	(*find_vmlinuz)(struct gunzip_state *, void **srcp,
+ 			unsigned long *lenp);
 };
 extern struct platform_ops platform_ops;
 

[PATCH 12/15] bootwrapper: extract the vmlinux from initramfs

[Milton Miller]

Teach the kexec platform to find the vmlinux from the initramfs.

The implementation searches the initramfs for a file specified
by the boot-file property in chosen.


Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Status: working.

Index: work.git/arch/powerpc/boot/kexec.c
===================================================================
--- work.git.orig/arch/powerpc/boot/kexec.c	2007-07-10 04:01:57.000000000 -0500
+++ work.git/arch/powerpc/boot/kexec.c	2007-07-10 04:04:09.000000000 -0500
@@ -23,6 +23,8 @@
 #include "flatdevtree.h"
 #include "page.h"
 #include "types.h"
+#include "cpio.h"
+#include "stat.h"
 
 extern char _start[];
 extern char _end[];
@@ -92,6 +94,51 @@ static void find_rmo_end(void)
 	rmo_end = (void *)*rp;
 }
 
+void find_vmlinux_in_initramfs(struct gunzip_state *state,
+		void **srcp, unsigned long *lenp)
+{
+	void *devp;
+	int rc;
+	char path[MAX_PATH];
+
+	if (!loader_info.initrd_size)
+		dt_find_initrd();
+	if (!loader_info.initrd_size)
+		fatal("find_vmlinux: no initramfs");
+	devp = finddevice("/chosen");
+	if (!devp)
+		fatal("find_vmlinux: no /chosen to find vmlinux");
+	rc = getprop(devp, "boot-file", path, sizeof(path));
+	if (rc < 0)
+		fatal("find_vmlinux: no boot-file property in /chosen")
+	else if (rc == 0 || rc > MAX_PATH)
+		fatal("boot-file too long in /chosen")
+
+	rc = find_in_initramfs(path, (void *)loader_info.initrd_addr,
+			loader_info.initrd_size, state);
+	if (!rc)
+		fatal("find_vmlinux: couldn't find boot-file %s in initramfs",
+				path);
+
+	rc = get_cpio_file_mode();
+	if (!S_ISREG(rc))
+		fatal("find_vmlinux: boot-file %s is not a regular file",
+				path)
+
+	get_cpio_info(srcp, &rc);
+	*lenp = get_cpio_file_size();
+}
+
+void cpio_error(char *msg)
+{
+	void *srcp;
+	int read;
+
+	get_cpio_info(&srcp, &read);
+
+	fatal("cpio_error at %p + 0x%x(%d): %s", srcp, read, read, msg);
+}
+
 /**
  * setup_initial_heap - setup a small heap in the bss
  * Using a preallocated heap, setup for scanning the device tree.
@@ -178,6 +225,7 @@ void kexec_platform_init(struct boot_par
 	ranges_init_malloc();
 	init_flat_tree(dt_blob);
 
+	platform_ops.find_vmlinuz = find_vmlinux_in_initramfs;
 	platform_ops.vmlinux_alloc = kexec_vmlinux_alloc;
 	platform_ops.fixups = kexec_fixups;
 	finalize_chain = dt_ops.finalize;

[PATCH 13/15] bootwrapper: attach an empty vmlinux

[Milton Miller]

Allow the boot wrapper code to be linked without an attached vmlinux.

Rather than invent a new syntax to invoke the wrapper, attach the
stripped version of empty.o, which produces the same result.

This new intermediary is called zBoot.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
Initial attempts to remove the dependency on vmlinux in the arch Makefile
have lead to multiple make commands competing in the boot directory and
destroying each other.

Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 16:38:01.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 16:41:01.000000000 -0500
@@ -124,7 +124,8 @@ endif
 # args (to if_changed): 1 = (this rule), 2 = platform, 3 = dts 4=dtb 5=initrd
 quiet_cmd_wrap	= WRAP    $@
       cmd_wrap	=$(CONFIG_SHELL) $(wrapper) -c -o $@ -p $2 $(CROSSWRAP) \
-		$(if $3, -s $3)$(if $4, -d $4)$(if $5, -i $5) vmlinux
+		$(if $3, -s $3)$(if $4, -d $4)$(if $5, -i $5) \
+		$(if $6,$6,vmlinux)
 
 kexec-$(CONFIG_PPC32)			+= zImage.kexec
 kexec-$(CONFIG_PPC64)			+= zImage.kexec64
@@ -142,6 +143,7 @@ image-$(CONFIG_PPC_PRPMC2800)		+= zImage
 image-$(CONFIG_PPC_ISERIES)		+= zImage.iseries
 image-$(CONFIG_DEFAULT_UIMAGE)		+= uImage
 image-$(CONFIG_KEXEC)			+= $(kexec-y)
+image-$(CONFIG_KEXEC)			+= $(patsubst zImage%,zBoot%,$(kexec-y))
 
 ifneq ($(CONFIG_DEVICE_TREE),"")
 image-$(CONFIG_PPC_83xx)		+= cuImage.83xx
@@ -190,6 +192,9 @@ $(obj)/vmlinux.strip: vmlinux
 $(obj)/zImage.iseries: vmlinux
 	$(STRIP) -s -R .comment $< -o $@
 
+$(obj)/zBoot.%: $(wrapperbits)
+	$(call if_changed,wrap,$*,,,,$(obj)/empty.o)
+
 $(obj)/zImage.ps3: vmlinux  $(wrapper) $(wrapperbits) $(srctree)/$(src)/dts/ps3.dts
 	$(STRIP) -s -R .comment $< -o vmlinux.strip
 	$(call cmd,wrap,ps3,$(srctree)/$(src)/dts/ps3.dts,,)
@@ -227,7 +232,7 @@ clean-files += $(image-) $(initrd-) zIma
 	treeImage.* $(kexec-)
 
 # clean up files cached by wrapper
-clean-kernel := vmlinux.strip vmlinux.bin
+clean-kernel := vmlinux.strip vmlinux.bin empty.o.bin
 clean-kernel += $(addsuffix .gz,$(clean-kernel))
 # If not absolute clean-files are relative to $(obj).
 clean-files += $(addprefix $(objtree)/, $(clean-kernel))
Index: work.git/arch/powerpc/boot/wrapper
===================================================================
--- work.git.orig/arch/powerpc/boot/wrapper	2007-07-10 16:37:30.000000000 -0500
+++ work.git/arch/powerpc/boot/wrapper	2007-07-10 16:41:01.000000000 -0500
@@ -161,6 +161,13 @@ ps3)
     ;;
 esac
 
+if [ "$kernel" = $object/empty.o ] ; then
+    ext=bin
+    objflags="-O binary"
+    gzip=
+fi
+
+
 vmz="$tmpdir/`basename \"$kernel\"`.$ext"
 if [ -z "$cacheit" -o ! -f "$vmz$gzip" -o "$vmz$gzip" -ot "$kernel" ]; then
     ${CROSS}objcopy $objflags "$kernel" "$vmz.$$"

[PATCH 14/15] boot: add a hook to start cpus

[Milton Miller]

The kexec code is doing strange contortions with dtops.finalize and
platform_ops.vmlinux_alloc to manage the slave cpus.   Add a hook
with the needed information.

Signed-off-by: Milton Miller <miltonm@bga.com>
---
Index: work.git/arch/powerpc/boot/kexec.c
===================================================================
--- work.git.orig/arch/powerpc/boot/kexec.c	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/kexec.c	2007-07-10 04:40:01.000000000 -0500
@@ -180,32 +180,11 @@ static void init_flat_tree(struct boot_p
 		fatal("Unable to initialize device_tree library!\n\r");
 }
 
-static void *saved_vmlinux_addr;
-
-static void *kexec_vmlinux_alloc(unsigned long size)
-{
-	void *addr;
-
-	addr = ranges_vmlinux_alloc(size);
-
-	saved_vmlinux_addr = addr;
-	return addr;
-}
-
 static void kexec_fixups(void)
 {
 	wait_slaves_moved();
 }
 
-static unsigned long (*finalize_chain)(void);
-
-static unsigned long kexec_finalize(void)
-{
-	send_slaves_to_kernel(saved_vmlinux_addr);
-
-	return finalize_chain();
-}
-
 void kexec_platform_init(struct boot_param_header *dt_blob)
 {
 	slaves_are_low();
@@ -226,8 +205,7 @@ void kexec_platform_init(struct boot_par
 	init_flat_tree(dt_blob);
 
 	platform_ops.find_vmlinuz = find_vmlinux_in_initramfs;
-	platform_ops.vmlinux_alloc = kexec_vmlinux_alloc;
+	platform_ops.vmlinux_alloc = ranges_vmlinux_alloc;
 	platform_ops.fixups = kexec_fixups;
-	finalize_chain = dt_ops.finalize;
-	dt_ops.finalize = kexec_finalize;
+	platform_ops.start_smp = send_slaves_to_kernel;
 }
Index: work.git/arch/powerpc/boot/main.c
===================================================================
--- work.git.orig/arch/powerpc/boot/main.c	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/main.c	2007-07-10 04:39:43.000000000 -0500
@@ -212,6 +212,9 @@ void start(void)
 	else
 		printf(" using OF tree (promptr=%p)\n\r", loader_info.promptr);
 
+	if (platform_ops.start_smp)
+		platform_ops.start_smp(vmlinux.addr);
+
 	if (console_ops.close)
 		console_ops.close();
 
Index: work.git/arch/powerpc/boot/ops.h
===================================================================
--- work.git.orig/arch/powerpc/boot/ops.h	2007-07-10 04:39:43.000000000 -0500
+++ work.git/arch/powerpc/boot/ops.h	2007-07-10 04:39:43.000000000 -0500
@@ -35,6 +35,7 @@ struct platform_ops {
 	void *	(*realloc)(void *ptr, unsigned long size);
 	void	(*exit)(void);
 	void *	(*vmlinux_alloc)(unsigned long size);
+	void	(*start_smp)(void *vmlinux);
  	void	(*find_vmlinuz)(struct gunzip_state *, void **srcp,
  			unsigned long *lenp);
 };

[PATCH/EXAMPLE 15/15] bootwrapper: example sreset marshalling

[Milton Miller]

An example using the marshalling code that can be entered by sreset.
By linking the marshalling code is at 0 and differentiating a cpu
id from a device tree, it also works for kexec.

Signed-off-by: Milton Miller <miltonm@bga.com>
--- 
For reference only, not intended to be merged.

Index: work.git/arch/powerpc/boot/crt0_sample.S
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/crt0_sample.S	2007-07-10 16:47:25.000000000 -0500
@@ -0,0 +1,73 @@
+/*
+ * Copyright 2006-2007 Milton Miller, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+	/*
+	 * This can be either a kexec started image or a sreset initiated
+	 * one.  kexec-tools purgatory is suppposed to copy from entry
+	 * point, but instead copies from image start, so put marshal_low
+	 * at address 0.
+	 */
+
+	.globl _zimage_start
+_zimage_start:
+#include "marshal_low.S"
+
+	.org 0x100
+	bl	get_cpu_id
+1:	mflr	0
+	lis	r4,1b@ha
+	addi	r4,r4,1b@l
+	subf	r0,r4,r0
+
+	lis	r4,elected_master@ha
+	addi	r4,r4,elected_master@l
+	add	r4,r4,r0
+2:	lwarx	r6,r0,r4
+	cmpwi	r6,0
+	bge	3f
+	stwcx.	r3,r0,r4
+	bne-	2b
+	lwz	r6,0(r4)
+
+3:	lis	r4,cpus_found@ha
+	addi	r4,r4,cpus_found@l
+	add	r4,r4,r0
+4:	lwarx	r12,r0,r4
+	addi	r12,r12,1
+	stwcx.	r12,r0,r4
+	bne-	4b
+
+	cmpw	r6,r3
+	bne	slave
+
+	mr	r4,r0
+	li	r5,0
+	b	master
+
+
+	.globl	_zimage_start_plat
+_zimage_start_plat:
+	b	_zimage_start_lib
+
+	.weak	get_cpu_id
+get_cpu_id:
+
+get_pir:
+	mfspr	r3,1023	/* SPRN_PIR */
+	blr
+
+	.balign 8
+	.globl elected_master
+elected_master:
+	.long -1
+	.globl cpus_expected
+cpus_expected:
+	.long 8
+	.globl cpus_found
+cpus_found:
+	.long 0
Index: work.git/arch/powerpc/boot/sample.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ work.git/arch/powerpc/boot/sample.c	2007-07-10 16:48:45.000000000 -0500
@@ -0,0 +1,62 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corporation 2007
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+
+#include "ops.h"
+#include "reg.h"
+#include "flatdevtree.h"
+
+extern struct boot_param_header _dtb_start[], _dtb_end[];
+struct boot_param_header *sample_dt_blob;
+
+extern unsigned int get_cpu_id(void);
+extern unsigned int cpus_found, cpus_expected;
+
+void platform_init(unsigned long boot_cpu_id)
+{
+	if (boot_cpu_id > 1024) {
+		sample_dt_blob = (void *)boot_cpu_id;
+		boot_cpu_id = get_cpu_id();
+	}
+
+	if (!sample_dt_blob)
+		if (_dtb_start != _dtb_end)
+			sample_dt_blob = _dtb_start;
+
+	if (!sample_dt_blob)
+		sample_dt_blob = 31 * 1024 * 1024 + (void *)0;
+
+	if (sample_dt_blob->magic != OF_DT_HEADER)
+		fatal("No device tree at %p\n", sample_dt_blob);
+
+	if (sample_dt_blob->version < 2)
+		conv_flattree_inplace(sample_dt_blob);
+
+	sample_dt_blob->boot_cpuid_phys = boot_cpu_id;
+
+	if (cpus_found && cpus_found < cpus_expected) {
+		HMT_LOW;
+		while (cpus_found < cpus_expected) {
+			barrier();
+		}
+		HMT_MEDIUM;
+	}
+
+	kexec_platform_init(sample_dt_blob);
+}
Index: work.git/arch/powerpc/boot/Makefile
===================================================================
--- work.git.orig/arch/powerpc/boot/Makefile	2007-07-10 16:41:01.000000000 -0500
+++ work.git/arch/powerpc/boot/Makefile	2007-07-10 16:47:45.000000000 -0500
@@ -48,7 +48,7 @@ src-wlib := string.S crt0.S stdio.c main
 		44x.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c
 src-plat := of.c cuboot-83xx.c cuboot-85xx.c holly.c crt0_kexec.S \
 		cuboot-ebony.c treeboot-ebony.c prpmc2800.c \
-		ps3-head.S ps3-hvcall.S ps3.c
+		ps3-head.S ps3-hvcall.S ps3.c crt0_sample.S sample.c
 src-boot := $(src-wlib) $(src-plat) empty.c
 
 src-boot := $(addprefix $(obj)/, $(src-boot))
@@ -128,7 +128,7 @@ quiet_cmd_wrap	= WRAP    $@
 		$(if $6,$6,vmlinux)
 
 kexec-$(CONFIG_PPC32)			+= zImage.kexec
-kexec-$(CONFIG_PPC64)			+= zImage.kexec64
+kexec-$(CONFIG_PPC64)			+= zImage.kexec64 zImage.sample
 
 image-$(CONFIG_PPC_PSERIES)		+= zImage.pseries
 image-$(CONFIG_PPC_MAPLE)		+= zImage.pseries
Index: work.git/arch/powerpc/boot/wrapper
===================================================================
--- work.git.orig/arch/powerpc/boot/wrapper	2007-07-10 16:41:01.000000000 -0500
+++ work.git/arch/powerpc/boot/wrapper	2007-07-10 16:47:25.000000000 -0500
@@ -139,6 +139,9 @@ kexec)
 kexec64)
     platformo="-e _zimage_start_64 $object/crt0_kexec.o"
     ;;
+sample)
+    platformo="$object/crt0_sample.o $object/sample.o"
+    ;;
 miboot|uboot)
     # miboot and U-boot want just the bare bits, not an ELF binary
     ext=bin

Re: [PATCH 6/15] bootwrapper: switch 64 bit cpus to 32 bit mode

[Segher Boessenkool]

> +	/* Check if the processor is running in 32 bit mode, using
> +	 * only 32 bit instructions which should be safe on 32 and
> +	 * 64 bit processors.
> +	 *
> +	 * Subtract the bottom 32 bits of MSR from the full value
> +	 * recording the result.  Since MSR[SF] is in the high word,
> +	 * the result will be not-equal iff in 32 bit mode (either
> +	 * the processor is a 32 bit processor or MSR[SF] = 0).
> +	 */

The other way around -- the processor is 64-bit and MSR[SF]=1.

Great trick btw! :-)

> +	mfmsr	r0		/* grab whole msr		*/
> +	rlwinm	r8,r0,0,0,31	/* extract bottom word		*/
> +	subf.	r8,r8,r0	/* subtract, same?		*/
> +	beq	0f		/* yes: we are 32 bit mode	*/

The code is fine though.


Segher

Re: [PATCH 11/15] bootwrapper: allow vmlinuz to be an external payload

[Segher Boessenkool]

> Allow the boot wrapper to obtain the vmlinux elf from an external
> platform supplied location instead of embedding it at link time.
>
> When used with the cpio library, the kernel can be stored as a file
> an file in the initramfs, allowing the same (compressed) copy of
> the vmlinux elf to be used for both the kernel to execute and any
> user space tools such as kexec (for reboot), crash dump, or oprofile.

Awesome!


Segher

Re: [PATCH 7/15] bootwrapper: Add kexec callable zImage wrapper

[Segher Boessenkool]

> The memory node off the root with a name starting with "memory" must
> contain enough free memory (not in the reserved ranges) in the first
> reg range to uncompress the the kernel with padding.

There can be many nodes called "/memory".  You probably
want to use the node pointed to by /chosen/memory instead?


Segher

Re: [PATCH 1/15] boot: find initrd location from device-tree

[David Gibson]

On Tue, Jul 10, 2007 at 05:07:38PM -0500, Milton Miller wrote:
> Some platforms have a boot agent that can create or modify properties in
> the device-tree and load images into memory.  Provide a helper to set
> loader_info used by prep_initrd().
> 
> Signed-off-by: Milton Miller <miltonm@bga.com>

Looks reasonable.

Acked-by: David Gibson <david@gibson.dropbear.id.au>

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 2/15] boot: record header bytes in gunzip_start

[David Gibson]

On Tue, Jul 10, 2007 at 05:08:05PM -0500, Milton Miller wrote:
> 
> Record the number of header bytes skipped in the total bytes read field.
> 
> This is needed for the initramfs parsing code to find the end of the zip file.
> 
> Signed-off-by: Milton Miller <miltonm@bga.com>

Ok... I assume you've checked that this is actually the correct
semantics for that field of the zlib structure?

> Index: work.git/arch/powerpc/boot/gunzip_util.c
> ===================================================================
> --- work.git.orig/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:44:41.000000000 -0500
> +++ work.git/arch/powerpc/boot/gunzip_util.c	2007-07-10 03:47:35.000000000 -0500
> @@ -78,6 +78,7 @@ void gunzip_start(struct gunzip_state *s
>  			fatal("inflateInit2 returned %d\n\r", r);
>  	}
>  
> +	state->s.total_in = hdrlen;
>  	state->s.next_in = src + hdrlen;
>  	state->s.avail_in = srclen - hdrlen;
>  }
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 3/15] boot: simplfy gunzip_finish

[David Gibson]

On Tue, Jul 10, 2007 at 05:08:32PM -0500, Milton Miller wrote:
> 
> Call gunzip_partial to calculate the remaining length and copy the
> data to the user buffer.   This makes it shorter and reduces
> duplication.
> 
> Signed-off-by: Milton Miller <miltonm@bga.com>

Hrm... I guess this is sufficient.  Since we'll shortly be blowing
away the state anyway, the lack of call to inflateEnd doesn't really
matter.  Originally I was going to make this function call fatal() if
the destination didn't have enough space to contain the decompressed
tail of the image, but obviously I decided aginst that.

But, if you're going to do this, you might as well get rid of
gunzip_finish() entirely, and have the callers using gunzip_partial
instead.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 3/15] boot: simplfy gunzip_finish

[Milton Miller]

On Jul 18, 2007, at 9:39 PM, David Gibson wrote:

> On Tue, Jul 10, 2007 at 05:08:32PM -0500, Milton Miller wrote:
>>
>> Call gunzip_partial to calculate the remaining length and copy the
>> data to the user buffer.   This makes it shorter and reduces
>> duplication.
>>
>> Signed-off-by: Milton Miller <miltonm@bga.com>
>
> Hrm... I guess this is sufficient.  Since we'll shortly be blowing
> away the state anyway, the lack of call to inflateEnd doesn't really
> matter.  Originally I was going to make this function call fatal() if
> the destination didn't have enough space to contain the decompressed
> tail of the image, but obviously I decided aginst that.
>
> But, if you're going to do this, you might as well get rid of
> gunzip_finish() entirely, and have the callers using gunzip_partial
> instead.

Huh?   I stilll call inflateEnd if s.workspace is not NULL, I just 
don't duplicate gunzip_partial in gunzip_finish.  Calling inflateEnd 
will mean a future call would return an error in the compressed case 
instead of the behavior in the non-compressed case that changes with 
this patch from get same data again to get further data that may exist 
after the patch.

Yes, one can argue that we don't need to call inflateEnd, but I didn't 
make that change in this patch.

Or were you suggesting that we really don't need the finish function?

milton

Re: [PATCH 3/15] boot: simplfy gunzip_finish

[David Gibson]

On Wed, Jul 18, 2007 at 11:01:37PM -0500, Milton Miller wrote:
> 
> On Jul 18, 2007, at 9:39 PM, David Gibson wrote:
> 
> > On Tue, Jul 10, 2007 at 05:08:32PM -0500, Milton Miller wrote:
> >>
> >> Call gunzip_partial to calculate the remaining length and copy the
> >> data to the user buffer.   This makes it shorter and reduces
> >> duplication.
> >>
> >> Signed-off-by: Milton Miller <miltonm@bga.com>
> >
> > Hrm... I guess this is sufficient.  Since we'll shortly be blowing
> > away the state anyway, the lack of call to inflateEnd doesn't really
> > matter.  Originally I was going to make this function call fatal() if
> > the destination didn't have enough space to contain the decompressed
> > tail of the image, but obviously I decided aginst that.
> >
> > But, if you're going to do this, you might as well get rid of
> > gunzip_finish() entirely, and have the callers using gunzip_partial
> > instead.
> 
> Huh?   I stilll call inflateEnd if s.workspace is not NULL, I just 
> don't duplicate gunzip_partial in gunzip_finish.  Calling inflateEnd 

Oh, yes, oops.  Missed the couple of lines there that didn't have a
'-' in front.

> will mean a future call would return an error in the compressed case 
> instead of the behavior in the non-compressed case that changes with 
> this patch from get same data again to get further data that may exist 
> after the patch.
> 
> Yes, one can argue that we don't need to call inflateEnd, but I didn't 
> make that change in this patch.
> 
> Or were you suggesting that we really don't need the finish function?

Well, we probably don't.  But for the time being.

Acked-by: David Gibson <david@gibson.dropbear.id.au>

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 2/15] boot: record header bytes in gunzip_start

[Milton Miller]

On Jul 18, 2007, at 9:11 PM, David Gibson wrote:

> On Tue, Jul 10, 2007 at 05:08:05PM -0500, Milton Miller wrote:
>>
>> Record the number of header bytes skipped in the total bytes read 
>> field.
>>
>> This is needed for the initramfs parsing code to find the end of the 
>> zip file.
>>
>> Signed-off-by: Milton Miller <miltonm@bga.com>
>
> Ok... I assume you've checked that this is actually the correct
> semantics for that field of the zlib structure?

 From looking at the comments in include/linux/zlib.h, I think its 
consistent.  It talks about looking at that field for statistics and 
for the application to record how many bytes were read during 
compression.  It mentions it brifely in the inflateSync to show how 
many bytes have been processed.   So I say its consistent to say we 
have processed the header bytes in the input file.

It appears to be for recording the number of bytes processed over all 
iterations.

I dig into zlib to see if this matches the beahvior of decompressing a 
zlib header, but expect it would match.


milton

[PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

kexec-tools still produces a version 2 device tree, while the
libraries in the wrapper only support version 16 and later.

Add a routine to convert a v2 flat device tree to a v16 one inplace
by inserting OF_DT_NOP and chomping full path.  Make space for new
headers by moving and then chomping the OF_DT_NOPs.

Signed-off-by: Milton Miller <miltonm@bga.com>
---
vs 12175
Rediffed Makefile, ops, kexec.c

Index: kernel/arch/powerpc/boot/flatdevtree_conv.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ kernel/arch/powerpc/boot/flatdevtree_conv.c	2007-09-20 17:49:04.000000000 -0500
@@ -0,0 +1,280 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corporation 2007
+ *
+ * Authors: Milton Miller <miltonm@bga.com>
+ */
+#include "flatdevtree.h"
+#include "stdio.h"
+#include "ops.h"
+
+#define MIN_VERSION 2
+#define OUT_VERSION 16
+#define OUT_COMPAT 16
+
+#ifdef NO_CHECK
+static int check_v123_tree(u32 *start, u32 *limit)
+{
+	return 0;
+}
+#else
+/**
+ * check_v123_tree - check integrety of a version 1, 2, or 3 tree
+ * @start: the start of the device tree struct
+ * @limit: the end of the region for the struct
+ * structural checks on device_tree
+ */
+static int check_v123_tree(u32 *start, u32 *limit)
+{
+	u32 len;
+	int depth = 0;
+	u32 *dtp = start;
+
+	while (dtp < limit)
+		switch (*dtp) {
+		case OF_DT_END:
+			if (depth)
+				return -1;
+			return ++dtp - start;
+		case OF_DT_NOP:
+			dtp++;
+			break;
+		case OF_DT_END_NODE:
+			dtp++;
+			depth--;
+			break;
+		case OF_DT_BEGIN_NODE:
+			len = strlen((char *)(++dtp));
+			/* check path is suffix to previous? */
+			dtp += 1 + (len / 4);
+			depth++;
+			break;
+		case OF_DT_PROP:
+			len = dtp[1];
+			dtp += 3;
+			if ((len >= 8) && ((long)dtp & 4))
+				dtp++;
+			dtp += (len + 3) / 4;
+			break;
+		default:
+			return -1;
+		}
+	return -1;	/* no OF_DT_END */
+}
+#endif
+
+/**
+ * nop_to_v16 - add %OF_DT_NOP to hide alignment differences
+ * @dtp: pointer to the beginning of the struct area to modify
+ * insert %OF_DT_NOP into the dt_struct @dtp to make it v16 from v1, 2, or 3.
+ */
+static int nop_to_v16(u32 *dtp)
+{
+	int nops = 0;
+	char *p, *s;
+	int len;
+	u32 *next;
+
+	while (*dtp != OF_DT_END)
+		switch (*dtp) {
+		case OF_DT_BEGIN_NODE:
+			/* v2 & v3 names are full path, v16+ is relative */
+			p = (char *)(++dtp);
+			len = strlen(p);
+			next = dtp + 1 + len / 4;
+
+			for (s = p + len; *s != '/'; s--)
+				if (s == p)
+					fatal("name %s has no '/'", p);
+
+			len -= s++ - p;		/* not the slash but the nul */
+			memmove(p, s, len);
+			while (len % 4)
+				p[len++] = '\0';
+			dtp += len / 4;
+			while (dtp != next) {
+				*dtp++ = OF_DT_NOP;
+				nops++;
+			}
+			break;
+		case OF_DT_PROP:
+			/* convert from align_8 to align_4 via prefixing nop */
+			len = dtp[1];
+			if ((len >= 8) && !((long)dtp & 4)) {
+				memmove(dtp+1, dtp, 12);
+				*dtp++ = OF_DT_NOP;
+				nops++;
+			}
+			dtp += 3 + (len + 3)/4;
+			break;
+		default:
+			fatal("%s: unrecognised tag %d at %p\n", __FUNCTION__,
+				*dtp, dtp);
+		case OF_DT_NOP:
+			nops ++;
+			/* fall through */
+		case OF_DT_END_NODE:
+			dtp ++;
+			break;
+		}
+	return nops;
+}
+
+#if MIN_VERSION < 3 || OUT_VERSION > 16
+/**
+ * move_nops_fwd - move nops in a v16 dt_struct to the beginning
+ * @start - device tree starting address
+ * @count - number of %OF_DT_NOP cells to move
+ */
+static void move_nops_fwd(u32 *start, int count)
+{
+	u32 *dtp = start;
+	int len;
+	while (count)
+		switch (*dtp) {
+		case OF_DT_NOP:
+			memmove(start+1,start,(dtp-start) * 4);
+			*start++ = OF_DT_NOP;
+			dtp++;
+			count--;
+			break;
+		case OF_DT_END_NODE:
+			dtp++;
+			break;
+		case OF_DT_BEGIN_NODE:
+			len = strlen((char *)(++dtp));
+			dtp += 1 + len / 4;
+			break;
+		case OF_DT_PROP:
+			len = dtp[1];
+			dtp += 3 + (len + 3) / 4;
+			break;
+		case OF_DT_END:
+			fatal("Not enough nops -- need %d more\n", count);
+			return;
+		default:
+			fatal("%s: unknown tag %d at %p", __FUNCTION__, *dtp, dtp)
+		}
+}
+#endif
+
+/**
+ * conv_flattree_inplace upgrade the version of a boot_param_header
+ * @tree: pointer to the device tree header to convert
+ *
+ * Converts a v1, 2, 3 device tree (of at least MIN_VERSION)
+ * in place to OUT_VERSION (16) format, usable by flatdevtree.c
+ */
+void conv_flattree_inplace(struct boot_param_header *tree)
+{
+	u32 *dtp;
+	u32 need = 0, nops;
+	int slen;
+
+	if (tree->magic != OF_DT_HEADER)
+		fatal("%s: no magic", __FUNCTION__);
+
+	if (tree->last_comp_version > 3)
+		return;			/* don't know what to do */
+
+	if (tree->version < MIN_VERSION) {
+		printf("%s: Warning: can't handle version %d tree\n",
+				__FUNCTION__, tree->version);
+		return;
+	}
+
+	if (tree->version < 2)
+		need++;		/* boot_cpu_id */
+
+	if (tree->version < 3)
+		need++;		/* dt_string_size */
+
+	if (OUT_VERSION > 16)
+		need++;		/* dt_struct_size */
+
+	dtp = (void *)tree + tree->off_dt_struct;
+
+	slen = check_v123_tree(dtp, (void *)tree + tree->totalsize);
+	if (slen < 0)
+		fatal("device tree check failed\n");
+
+	nops = nop_to_v16(dtp);
+
+	if (need & 1)		/* keep 8 byte alignment of mem reserve */
+		need++;
+
+	if (need > nops)
+		fatal("Didn't find enough space to add new header fields\n\r"
+			"(needed %d found %d tree %p)", need, nops, tree);
+
+	/* ok now compress the dtb struct */
+	move_nops_fwd(dtp, need);
+	dtp += need;
+
+	/*
+	 * move mem_rsvmap and dt_strings if they are before dt_struct
+	 * onto our nops .  Adjust start addresses for the 3 sections.
+	 */
+	if ((tree->off_mem_rsvmap < tree->off_dt_struct) ||
+		(tree->off_dt_strings < tree->off_dt_struct)) {
+		int start, end;
+		void *ptr;
+
+		if (tree->off_mem_rsvmap < tree->off_dt_strings)
+			start = tree->off_mem_rsvmap;
+		else
+			start = tree->off_dt_strings;
+
+		end = tree->off_dt_struct;
+		ptr = (void *)tree + start;
+
+		memmove(ptr + 4 * need, ptr, end - start);
+
+		if (tree->off_mem_rsvmap < tree->off_dt_struct)
+			tree->off_mem_rsvmap += 4 * need;
+		if (tree->off_dt_strings < tree->off_dt_struct)
+			tree->off_dt_strings += 4 * need;
+	}
+	tree->off_dt_struct += 4 * need;
+
+	/* ok now we have space to extend the header. */
+	if (tree->version < 2 && MIN_VERSION < 2) {
+		tree->boot_cpuid_phys = 0;	/* default, caller can fix */
+	}
+
+	/* calculate size of dt_strings_size */
+	if (tree->version < 3 && MIN_VERSION < 3) {
+		int end = tree->totalsize;
+
+		if (tree->off_dt_strings < tree->off_mem_rsvmap)
+			end = tree->off_mem_rsvmap;
+
+		if ((tree->off_dt_strings < tree->off_dt_struct) &&
+				(end > tree->off_dt_struct))
+			end = tree->off_dt_struct;
+
+		tree->dt_strings_size = end - tree->off_dt_strings;
+	}
+
+#if OUT_VERSION > 16
+	tree->dt_struct_size = 4 * slen;
+#endif
+
+	tree->version = OUT_VERSION;
+	tree->last_comp_version = OUT_COMPAT;
+
+	return;
+}
Index: kernel/arch/powerpc/boot/Makefile
===================================================================
--- kernel.orig/arch/powerpc/boot/Makefile	2007-09-20 17:42:24.000000000 -0500
+++ kernel/arch/powerpc/boot/Makefile	2007-09-20 17:49:04.000000000 -0500
@@ -42,7 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
 	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
 
 src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
-		marshal.c memranges.c kexec.c \
+		flatdevtree_conv.c marshal.c memranges.c kexec.c \
 		ns16550.c serial.c simple_alloc.c div64.S util.S \
 		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
 		4xx.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c bamboo.c \
Index: kernel/arch/powerpc/boot/ops.h
===================================================================
--- kernel.orig/arch/powerpc/boot/ops.h	2007-09-20 17:42:24.000000000 -0500
+++ kernel/arch/powerpc/boot/ops.h	2007-09-20 17:49:04.000000000 -0500
@@ -81,7 +81,10 @@ struct loader_info {
 };
 extern struct loader_info loader_info;
 
+struct boot_param_header;
+
 void start(void);
+void conv_flattree_inplace(struct boot_param_header *tree);
 int ft_init(void *dt_blob, unsigned int max_size, unsigned int max_find_device);
 int serial_console_init(void);
 int ns16550_console_init(void *devp, struct serial_console_data *scdp);
Index: kernel/arch/powerpc/boot/kexec.c
===================================================================
--- kernel.orig/arch/powerpc/boot/kexec.c	2007-09-20 17:42:24.000000000 -0500
+++ kernel/arch/powerpc/boot/kexec.c	2007-09-20 17:49:04.000000000 -0500
@@ -99,6 +99,7 @@ void kexec_platform_init(struct boot_par
 	move_slaves_up();
 
 	setup_initial_heap();
+	conv_flattree_inplace(dt_blob);
 	init_flat_tree(dt_blob);
 	/*
 	 * drivers can malloc and read the tree, but not realloc later

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[David Gibson]

On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
> kexec-tools still produces a version 2 device tree, while the
> libraries in the wrapper only support version 16 and later.
> 
> Add a routine to convert a v2 flat device tree to a v16 one inplace
> by inserting OF_DT_NOP and chomping full path.  Make space for new
> headers by moving and then chomping the OF_DT_NOPs.
> 
> Signed-off-by: Milton Miller <miltonm@bga.com>
> ---
> vs 12175
> Rediffed Makefile, ops, kexec.c
> 
> Index: kernel/arch/powerpc/boot/flatdevtree_conv.c
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ kernel/arch/powerpc/boot/flatdevtree_conv.c	2007-09-20 17:49:04.000000000 -0500
> @@ -0,0 +1,280 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
> + *
> + * Copyright IBM Corporation 2007
> + *
> + * Authors: Milton Miller <miltonm@bga.com>
> + */
> +#include "flatdevtree.h"
> +#include "stdio.h"
> +#include "ops.h"
> +
> +#define MIN_VERSION 2
> +#define OUT_VERSION 16

Should output version 17.  In any case, don't try to be so general -
just convert v123 (all basically the same) to latest (i.e. v17)
without all the #if nonsense.

> +#define OUT_COMPAT 16
> +
> +#ifdef NO_CHECK
> +static int check_v123_tree(u32 *start, u32 *limit)
> +{
> +	return 0;
> +}
> +#else
> +/**
> + * check_v123_tree - check integrety of a version 1, 2, or 3 tree
> + * @start: the start of the device tree struct
> + * @limit: the end of the region for the struct
> + * structural checks on device_tree
> + */
> +static int check_v123_tree(u32 *start, u32 *limit)

What is the point of this check?  If the device tree is corrupt, we're
stuffed anyway, so why bother?

> +{
> +	u32 len;
> +	int depth = 0;
> +	u32 *dtp = start;
> +
> +	while (dtp < limit)
> +		switch (*dtp) {
> +		case OF_DT_END:
> +			if (depth)
> +				return -1;
> +			return ++dtp - start;
> +		case OF_DT_NOP:
> +			dtp++;
> +			break;
> +		case OF_DT_END_NODE:
> +			dtp++;
> +			depth--;
> +			break;
> +		case OF_DT_BEGIN_NODE:
> +			len = strlen((char *)(++dtp));
> +			/* check path is suffix to previous? */
> +			dtp += 1 + (len / 4);
> +			depth++;
> +			break;
> +		case OF_DT_PROP:
> +			len = dtp[1];
> +			dtp += 3;
> +			if ((len >= 8) && ((long)dtp & 4))
> +				dtp++;
> +			dtp += (len + 3) / 4;
> +			break;
> +		default:
> +			return -1;
> +		}
> +	return -1;	/* no OF_DT_END */
> +}
> +#endif
> +
> +/**
> + * nop_to_v16 - add %OF_DT_NOP to hide alignment differences
> + * @dtp: pointer to the beginning of the struct area to modify
> + * insert %OF_DT_NOP into the dt_struct @dtp to make it v16 from v1, 2, or 3.
> + */
> +static int nop_to_v16(u32 *dtp)
> +{
> +	int nops = 0;
> +	char *p, *s;
> +	int len;
> +	u32 *next;
> +
> +	while (*dtp != OF_DT_END)
> +		switch (*dtp) {
> +		case OF_DT_BEGIN_NODE:
> +			/* v2 & v3 names are full path, v16+ is relative */
> +			p = (char *)(++dtp);
> +			len = strlen(p);
> +			next = dtp + 1 + len / 4;
> +
> +			for (s = p + len; *s != '/'; s--)
> +				if (s == p)
> +					fatal("name %s has no '/'", p);
> +
> +			len -= s++ - p;		/* not the slash but the nul */
> +			memmove(p, s, len);
> +			while (len % 4)
> +				p[len++] = '\0';
> +			dtp += len / 4;
> +			while (dtp != next) {
> +				*dtp++ = OF_DT_NOP;
> +				nops++;
> +			}
> +			break;
> +		case OF_DT_PROP:
> +			/* convert from align_8 to align_4 via prefixing nop */
> +			len = dtp[1];
> +			if ((len >= 8) && !((long)dtp & 4)) {
> +				memmove(dtp+1, dtp, 12);
> +				*dtp++ = OF_DT_NOP;
> +				nops++;
> +			}
> +			dtp += 3 + (len + 3)/4;
> +			break;
> +		default:
> +			fatal("%s: unrecognised tag %d at %p\n", __FUNCTION__,
> +				*dtp, dtp);
> +		case OF_DT_NOP:
> +			nops ++;
> +			/* fall through */
> +		case OF_DT_END_NODE:
> +			dtp ++;
> +			break;
> +		}
> +	return nops;
> +}
> +
> +#if MIN_VERSION < 3 || OUT_VERSION > 16
> +/**
> + * move_nops_fwd - move nops in a v16 dt_struct to the beginning
> + * @start - device tree starting address
> + * @count - number of %OF_DT_NOP cells to move
> + */
> +static void move_nops_fwd(u32 *start, int count)

What on earth is the point of this.  The NOPs are perfectly valid
scattered within the tree, why go to all this trouble to shuffle them
about.

> +{
> +	u32 *dtp = start;
> +	int len;
> +	while (count)
> +		switch (*dtp) {
> +		case OF_DT_NOP:
> +			memmove(start+1,start,(dtp-start) * 4);
> +			*start++ = OF_DT_NOP;
> +			dtp++;
> +			count--;
> +			break;
> +		case OF_DT_END_NODE:
> +			dtp++;
> +			break;
> +		case OF_DT_BEGIN_NODE:
> +			len = strlen((char *)(++dtp));
> +			dtp += 1 + len / 4;
> +			break;
> +		case OF_DT_PROP:
> +			len = dtp[1];
> +			dtp += 3 + (len + 3) / 4;
> +			break;
> +		case OF_DT_END:
> +			fatal("Not enough nops -- need %d more\n", count);
> +			return;
> +		default:
> +			fatal("%s: unknown tag %d at %p", __FUNCTION__, *dtp, dtp)
> +		}
> +}
> +#endif
> +
> +/**
> + * conv_flattree_inplace upgrade the version of a boot_param_header
> + * @tree: pointer to the device tree header to convert
> + *
> + * Converts a v1, 2, 3 device tree (of at least MIN_VERSION)
> + * in place to OUT_VERSION (16) format, usable by flatdevtree.c
> + */
> +void conv_flattree_inplace(struct boot_param_header *tree)
> +{
> +	u32 *dtp;
> +	u32 need = 0, nops;
> +	int slen;
> +
> +	if (tree->magic != OF_DT_HEADER)
> +		fatal("%s: no magic", __FUNCTION__);

More pointless tests...

> +	if (tree->last_comp_version > 3)
> +		return;			/* don't know what to do */

Rather, don't need to do anything.

> +
> +	if (tree->version < MIN_VERSION) {
> +		printf("%s: Warning: can't handle version %d tree\n",
> +				__FUNCTION__, tree->version);
> +		return;
> +	}
> +
> +	if (tree->version < 2)
> +		need++;		/* boot_cpu_id */
> +
> +	if (tree->version < 3)
> +		need++;		/* dt_string_size */
> +
> +	if (OUT_VERSION > 16)
> +		need++;		/* dt_struct_size */
> +
> +	dtp = (void *)tree + tree->off_dt_struct;
> +
> +	slen = check_v123_tree(dtp, (void *)tree + tree->totalsize);
> +	if (slen < 0)
> +		fatal("device tree check failed\n");
> +
> +	nops = nop_to_v16(dtp);
> +
> +	if (need & 1)		/* keep 8 byte alignment of mem reserve */
> +		need++;
> +
> +	if (need > nops)
> +		fatal("Didn't find enough space to add new header fields\n\r"
> +			"(needed %d found %d tree %p)", need, nops, tree);
> +
> +	/* ok now compress the dtb struct */
> +	move_nops_fwd(dtp, need);
> +	dtp += need;
> +
> +	/*
> +	 * move mem_rsvmap and dt_strings if they are before dt_struct
> +	 * onto our nops .  Adjust start addresses for the 3 sections.
> +	 */

Hrm.  Do we really need to worry about this case.  You may be
producing v2 trees in kexec-tools, but do they actually have the
blocks out of order?  dtc certainly never produced them that way.

> +	if ((tree->off_mem_rsvmap < tree->off_dt_struct) ||
> +		(tree->off_dt_strings < tree->off_dt_struct)) {
> +		int start, end;
> +		void *ptr;
> +
> +		if (tree->off_mem_rsvmap < tree->off_dt_strings)
> +			start = tree->off_mem_rsvmap;
> +		else
> +			start = tree->off_dt_strings;
> +
> +		end = tree->off_dt_struct;
> +		ptr = (void *)tree + start;
> +
> +		memmove(ptr + 4 * need, ptr, end - start);
> +
> +		if (tree->off_mem_rsvmap < tree->off_dt_struct)
> +			tree->off_mem_rsvmap += 4 * need;
> +		if (tree->off_dt_strings < tree->off_dt_struct)
> +			tree->off_dt_strings += 4 * need;
> +	}
> +	tree->off_dt_struct += 4 * need;
> +
> +	/* ok now we have space to extend the header. */
> +	if (tree->version < 2 && MIN_VERSION < 2) {
> +		tree->boot_cpuid_phys = 0;	/* default, caller can fix */
> +	}
> +
> +	/* calculate size of dt_strings_size */
> +	if (tree->version < 3 && MIN_VERSION < 3) {
> +		int end = tree->totalsize;
> +
> +		if (tree->off_dt_strings < tree->off_mem_rsvmap)
> +			end = tree->off_mem_rsvmap;
> +
> +		if ((tree->off_dt_strings < tree->off_dt_struct) &&
> +				(end > tree->off_dt_struct))
> +			end = tree->off_dt_struct;
> +
> +		tree->dt_strings_size = end - tree->off_dt_strings;
> +	}
> +
> +#if OUT_VERSION > 16
> +	tree->dt_struct_size = 4 * slen;
> +#endif
> +
> +	tree->version = OUT_VERSION;
> +	tree->last_comp_version = OUT_COMPAT;
> +
> +	return;
> +}
> Index: kernel/arch/powerpc/boot/Makefile
> ===================================================================
> --- kernel.orig/arch/powerpc/boot/Makefile	2007-09-20 17:42:24.000000000 -0500
> +++ kernel/arch/powerpc/boot/Makefile	2007-09-20 17:49:04.000000000 -0500
> @@ -42,7 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
>  	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix $(obj)/,$(zlibheader))
>  
>  src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c flatdevtree_misc.c \
> -		marshal.c memranges.c kexec.c \
> +		flatdevtree_conv.c marshal.c memranges.c kexec.c \
>  		ns16550.c serial.c simple_alloc.c div64.S util.S \
>  		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
>  		4xx.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c bamboo.c \
> Index: kernel/arch/powerpc/boot/ops.h
> ===================================================================
> --- kernel.orig/arch/powerpc/boot/ops.h	2007-09-20 17:42:24.000000000 -0500
> +++ kernel/arch/powerpc/boot/ops.h	2007-09-20 17:49:04.000000000 -0500
> @@ -81,7 +81,10 @@ struct loader_info {
>  };
>  extern struct loader_info loader_info;
>  
> +struct boot_param_header;
> +
>  void start(void);
> +void conv_flattree_inplace(struct boot_param_header *tree);
>  int ft_init(void *dt_blob, unsigned int max_size, unsigned int max_find_device);
>  int serial_console_init(void);
>  int ns16550_console_init(void *devp, struct serial_console_data *scdp);
> Index: kernel/arch/powerpc/boot/kexec.c
> ===================================================================
> --- kernel.orig/arch/powerpc/boot/kexec.c	2007-09-20 17:42:24.000000000 -0500
> +++ kernel/arch/powerpc/boot/kexec.c	2007-09-20 17:49:04.000000000 -0500
> @@ -99,6 +99,7 @@ void kexec_platform_init(struct boot_par
>  	move_slaves_up();
>  
>  	setup_initial_heap();
> +	conv_flattree_inplace(dt_blob);
>  	init_flat_tree(dt_blob);
>  	/*
>  	 * drivers can malloc and read the tree, but not realloc later
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

On Sep 23, 2007, at 10:36 PM, David Gibson wrote:

> On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
>> kexec-tools still produces a version 2 device tree, while the
>> libraries in the wrapper only support version 16 and later.
>>
>> Add a routine to convert a v2 flat device tree to a v16 one inplace
>> by inserting OF_DT_NOP and chomping full path.  Make space for new
>> headers by moving and then chomping the OF_DT_NOPs.
>>
>> Signed-off-by: Milton Miller <miltonm@bga.com>
>> ---
>> vs 12175
>> Rediffed Makefile, ops, kexec.c
>>
>> Index: kernel/arch/powerpc/boot/flatdevtree_conv.c
>> ===================================================================
>> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
>> +++ kernel/arch/powerpc/boot/flatdevtree_conv.c	2007-09-20 
>> 17:49:04.000000000 -0500
>> @@ -0,0 +1,280 @@
>> +/*
>> + * This program is free software; you can redistribute it and/or 
>> modify
>> + * it under the terms of the GNU General Public License as published 
>> by
>> + * the Free Software Foundation; either version 2 of the License, or
>> + * (at your option) any later version.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program; if not, write to the Free Software
>> + * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  
>> 02110-1301, USA.
>> + *
>> + * Copyright IBM Corporation 2007
>> + *
>> + * Authors: Milton Miller <miltonm@bga.com>
>> + */
>> +#include "flatdevtree.h"
>> +#include "stdio.h"
>> +#include "ops.h"
>> +
>> +#define MIN_VERSION 2
>> +#define OUT_VERSION 16
>
> Should output version 17.  In any case, don't try to be so general -
> just convert v123 (all basically the same) to latest (i.e. v17)
> without all the #if nonsense.

Outputing v17 instead of 16 requires more words to be added to the 
header, and the library does fine with v16.  Actually the v1 trees has 
some other differences such as initrd addresses were kernel linear not 
real, cpus were assigned logical numbers  ... so while the structure 
didn't change except for the header field, the contents did.  Actually, 
when converting v3 to v16 some of the code issn't needed, the ifs allow 
the code size to be reduced.

>
>> +#define OUT_COMPAT 16
>> +
>> +#ifdef NO_CHECK
>> +static int check_v123_tree(u32 *start, u32 *limit)
>> +{
>> +	return 0;
>> +}
>> +#else
>> +/**
>> + * check_v123_tree - check integrety of a version 1, 2, or 3 tree
>> + * @start: the start of the device tree struct
>> + * @limit: the end of the region for the struct
>> + * structural checks on device_tree
>> + */
>> +static int check_v123_tree(u32 *start, u32 *limit)
>
> What is the point of this check?  If the device tree is corrupt, we're
> stuffed anyway, so why bother?

Hence the ifdef NO_CHECK.   When developing, sometimes its nice to know 
if its your input or your program.  These functions are destructive to 
an improperlly formed tree, and in non-obvious ways.  When debugging, 
it's not hard to hardcode console write or read the printf string 
buffer with a hardware debugger to see error messages.  That said, it 
could be removed.

>
>> +{
>> +	u32 len;
>> +	int depth = 0;
>> +	u32 *dtp = start;
>> +
>> +	while (dtp < limit)
>> +		switch (*dtp) {
>> +		case OF_DT_END:
>> +			if (depth)
>> +				return -1;
>> +			return ++dtp - start;
>> +		case OF_DT_NOP:
>> +			dtp++;
>> +			break;
>> +		case OF_DT_END_NODE:
>> +			dtp++;
>> +			depth--;
>> +			break;
>> +		case OF_DT_BEGIN_NODE:
>> +			len = strlen((char *)(++dtp));
>> +			/* check path is suffix to previous? */
>> +			dtp += 1 + (len / 4);
>> +			depth++;
>> +			break;
>> +		case OF_DT_PROP:
>> +			len = dtp[1];
>> +			dtp += 3;
>> +			if ((len >= 8) && ((long)dtp & 4))
>> +				dtp++;
>> +			dtp += (len + 3) / 4;
>> +			break;
>> +		default:
>> +			return -1;
>> +		}
>> +	return -1;	/* no OF_DT_END */
>> +}
>> +#endif
>> +
>> +/**
>> + * nop_to_v16 - add %OF_DT_NOP to hide alignment differences
>> + * @dtp: pointer to the beginning of the struct area to modify
>> + * insert %OF_DT_NOP into the dt_struct @dtp to make it v16 from v1, 
>> 2, or 3.
>> + */
>> +static int nop_to_v16(u32 *dtp)
>> +{
>> +	int nops = 0;
>> +	char *p, *s;
>> +	int len;
>> +	u32 *next;
>> +
>> +	while (*dtp != OF_DT_END)
>> +		switch (*dtp) {
>> +		case OF_DT_BEGIN_NODE:
>> +			/* v2 & v3 names are full path, v16+ is relative */
>> +			p = (char *)(++dtp);
>> +			len = strlen(p);
>> +			next = dtp + 1 + len / 4;
>> +
>> +			for (s = p + len; *s != '/'; s--)
>> +				if (s == p)
>> +					fatal("name %s has no '/'", p);
>> +
>> +			len -= s++ - p;		/* not the slash but the nul */
>> +			memmove(p, s, len);
>> +			while (len % 4)
>> +				p[len++] = '\0';
>> +			dtp += len / 4;
>> +			while (dtp != next) {
>> +				*dtp++ = OF_DT_NOP;
>> +				nops++;
>> +			}
>> +			break;
>> +		case OF_DT_PROP:
>> +			/* convert from align_8 to align_4 via prefixing nop */
>> +			len = dtp[1];
>> +			if ((len >= 8) && !((long)dtp & 4)) {
>> +				memmove(dtp+1, dtp, 12);
>> +				*dtp++ = OF_DT_NOP;
>> +				nops++;
>> +			}
>> +			dtp += 3 + (len + 3)/4;
>> +			break;
>> +		default:
>> +			fatal("%s: unrecognised tag %d at %p\n", __FUNCTION__,
>> +				*dtp, dtp);
>> +		case OF_DT_NOP:
>> +			nops ++;
>> +			/* fall through */
>> +		case OF_DT_END_NODE:
>> +			dtp ++;
>> +			break;
>> +		}
>> +	return nops;
>> +}
>> +
>> +#if MIN_VERSION < 3 || OUT_VERSION > 16
>> +/**
>> + * move_nops_fwd - move nops in a v16 dt_struct to the beginning
>> + * @start - device tree starting address
>> + * @count - number of %OF_DT_NOP cells to move
>> + */
>> +static void move_nops_fwd(u32 *start, int count)
>
> What on earth is the point of this.  The NOPs are perfectly valid
> scattered within the tree, why go to all this trouble to shuffle them
> about.

And if you notice, there is a "how many to move" argument.  The point 
of moving them to the front of the tree is the v17 device tree header 
takes more space than the v3 one, and the v2 header is smaller than 
both v17 and v16 header.  Since I am converting the tree in place, the 
space has to come from somewhere.  Since we are pretty much guaranteed 
to get several nops, this function moves them forward so they can be 
overwritten.  In practice we move 1-3 NOPS from the early properties > 
8 bytes and early grandchild nodes (eg /cpus/PowerPC,xxx).

>
>> +{
>> +	u32 *dtp = start;
>> +	int len;
>> +	while (count)
>> +		switch (*dtp) {
>> +		case OF_DT_NOP:
>> +			memmove(start+1,start,(dtp-start) * 4);
>> +			*start++ = OF_DT_NOP;
>> +			dtp++;
>> +			count--;
>> +			break;
>> +		case OF_DT_END_NODE:
>> +			dtp++;
>> +			break;
>> +		case OF_DT_BEGIN_NODE:
>> +			len = strlen((char *)(++dtp));
>> +			dtp += 1 + len / 4;
>> +			break;
>> +		case OF_DT_PROP:
>> +			len = dtp[1];
>> +			dtp += 3 + (len + 3) / 4;
>> +			break;
>> +		case OF_DT_END:
>> +			fatal("Not enough nops -- need %d more\n", count);
>> +			return;
>> +		default:
>> +			fatal("%s: unknown tag %d at %p", __FUNCTION__, *dtp, dtp)
>> +		}
>> +}
>> +#endif
>> +
>> +/**
>> + * conv_flattree_inplace upgrade the version of a boot_param_header
>> + * @tree: pointer to the device tree header to convert
>> + *
>> + * Converts a v1, 2, 3 device tree (of at least MIN_VERSION)
>> + * in place to OUT_VERSION (16) format, usable by flatdevtree.c
>> + */
>> +void conv_flattree_inplace(struct boot_param_header *tree)
>> +{
>> +	u32 *dtp;
>> +	u32 need = 0, nops;
>> +	int slen;
>> +
>> +	if (tree->magic != OF_DT_HEADER)
>> +		fatal("%s: no magic", __FUNCTION__);
>
> More pointless tests...

It says you didn't pass a tree ... yea, its dumb, but I prefer an 
explicit error to figuring out where a machine check came from.

>
>> +	if (tree->last_comp_version > 3)
>> +		return;			/* don't know what to do */
>
> Rather, don't need to do anything.

If the tree is >= 16 then we don't presently need to do anything.   If 
there is a theoritical v4 tree we don't know what to do.  And if output 
version is 17 but input is 16 we don't know what to do either, because 
there likely aren't nops in the tree to consume.  I suppose we could 
preceed it with a check for version == OUTPUT_VERSION, but then I'm 
sure I'd get pointless differentation :-).

>
>> +
>> +	if (tree->version < MIN_VERSION) {
>> +		printf("%s: Warning: can't handle version %d tree\n",
>> +				__FUNCTION__, tree->version);
>> +		return;
>> +	}
>> +
>> +	if (tree->version < 2)
>> +		need++;		/* boot_cpu_id */
>> +
>> +	if (tree->version < 3)
>> +		need++;		/* dt_string_size */
>> +
>> +	if (OUT_VERSION > 16)
>> +		need++;		/* dt_struct_size */
>> +
>> +	dtp = (void *)tree + tree->off_dt_struct;
>> +
>> +	slen = check_v123_tree(dtp, (void *)tree + tree->totalsize);
>> +	if (slen < 0)
>> +		fatal("device tree check failed\n");
>> +
>> +	nops = nop_to_v16(dtp);
>> +
>> +	if (need & 1)		/* keep 8 byte alignment of mem reserve */
>> +		need++;
>> +
>> +	if (need > nops)
>> +		fatal("Didn't find enough space to add new header fields\n\r"
>> +			"(needed %d found %d tree %p)", need, nops, tree);
>> +
>> +	/* ok now compress the dtb struct */
>> +	move_nops_fwd(dtp, need);
>> +	dtp += need;
>> +
>> +	/*
>> +	 * move mem_rsvmap and dt_strings if they are before dt_struct
>> +	 * onto our nops .  Adjust start addresses for the 3 sections.
>> +	 */
>
> Hrm.  Do we really need to worry about this case.  You may be
> producing v2 trees in kexec-tools, but do they actually have the
> blocks out of order?  dtc certainly never produced them that way.

Out of order?  There has never been a spec as to the order of the 
blocks, only the implicit assumption that they follow the device tree 
header in a reasonably packed sequence.  booting-without-of says it 
must be in ram,; the offsets are unsigned 32 bit quantities.

As to the order, used, the first implemntation was the kernel which 
writes memreserve, strings, then struct (both the openfirmware client 
in prom_init and the iSeries procedural library in dt.c).  The second 
implentation written is a procedural based library (similar to iseries, 
never published, but still used internally) that starts with a 
pre-built header and string table, builds the dt_struct as the 
functions are called, and when finished copies the memreserve table and 
fills in the dt_size, total size, and memreserve offset.

fs2dt writes memreserve, struct, then strings.  Aparently the same as 
dtc.  But yes, the strings can be before the struct, and the mem 
reserve may or may not be when the strings are before the struct.

>
>> +	if ((tree->off_mem_rsvmap < tree->off_dt_struct) ||
>> +		(tree->off_dt_strings < tree->off_dt_struct)) {
>> +		int start, end;
>> +		void *ptr;
>> +
>> +		if (tree->off_mem_rsvmap < tree->off_dt_strings)
>> +			start = tree->off_mem_rsvmap;
>> +		else
>> +			start = tree->off_dt_strings;
>> +
>> +		end = tree->off_dt_struct;
>> +		ptr = (void *)tree + start;
>> +
>> +		memmove(ptr + 4 * need, ptr, end - start);
>> +
>> +		if (tree->off_mem_rsvmap < tree->off_dt_struct)
>> +			tree->off_mem_rsvmap += 4 * need;
>> +		if (tree->off_dt_strings < tree->off_dt_struct)
>> +			tree->off_dt_strings += 4 * need;
>> +	}
>> +	tree->off_dt_struct += 4 * need;
>> +
>> +	/* ok now we have space to extend the header. */
>> +	if (tree->version < 2 && MIN_VERSION < 2) {
>> +		tree->boot_cpuid_phys = 0;	/* default, caller can fix */
>> +	}
>> +
>> +	/* calculate size of dt_strings_size */
>> +	if (tree->version < 3 && MIN_VERSION < 3) {
>> +		int end = tree->totalsize;
>> +
>> +		if (tree->off_dt_strings < tree->off_mem_rsvmap)
>> +			end = tree->off_mem_rsvmap;
>> +
>> +		if ((tree->off_dt_strings < tree->off_dt_struct) &&
>> +				(end > tree->off_dt_struct))
>> +			end = tree->off_dt_struct;
>> +
>> +		tree->dt_strings_size = end - tree->off_dt_strings;
>> +	}
>> +
>> +#if OUT_VERSION > 16
>> +	tree->dt_struct_size = 4 * slen;
>> +#endif
>> +
>> +	tree->version = OUT_VERSION;
>> +	tree->last_comp_version = OUT_COMPAT;
>> +
>> +	return;
>> +}
>> Index: kernel/arch/powerpc/boot/Makefile
>> ===================================================================
>> --- kernel.orig/arch/powerpc/boot/Makefile	2007-09-20 
>> 17:42:24.000000000 -0500
>> +++ kernel/arch/powerpc/boot/Makefile	2007-09-20 17:49:04.000000000 
>> -0500
>> @@ -42,7 +42,7 @@ $(addprefix $(obj)/,$(zlib) gunzip_util.
>>  	$(addprefix $(obj)/,$(zliblinuxheader)) $(addprefix 
>> $(obj)/,$(zlibheader))
>>
>>  src-wlib := string.S crt0.S stdio.c main.c flatdevtree.c 
>> flatdevtree_misc.c \
>> -		marshal.c memranges.c kexec.c \
>> +		flatdevtree_conv.c marshal.c memranges.c kexec.c \
>>  		ns16550.c serial.c simple_alloc.c div64.S util.S \
>>  		gunzip_util.c elf_util.c $(zlib) devtree.c oflib.c ofconsole.c \
>>  		4xx.c ebony.c mv64x60.c mpsc.c mv64x60_i2c.c cuboot.c bamboo.c \
>> Index: kernel/arch/powerpc/boot/ops.h
>> ===================================================================
>> --- kernel.orig/arch/powerpc/boot/ops.h	2007-09-20 17:42:24.000000000 
>> -0500
>> +++ kernel/arch/powerpc/boot/ops.h	2007-09-20 17:49:04.000000000 -0500
>> @@ -81,7 +81,10 @@ struct loader_info {
>>  };
>>  extern struct loader_info loader_info;
>>
>> +struct boot_param_header;
>> +
>>  void start(void);
>> +void conv_flattree_inplace(struct boot_param_header *tree);
>>  int ft_init(void *dt_blob, unsigned int max_size, unsigned int 
>> max_find_device);
>>  int serial_console_init(void);
>>  int ns16550_console_init(void *devp, struct serial_console_data 
>> *scdp);
>> Index: kernel/arch/powerpc/boot/kexec.c
>> ===================================================================
>> --- kernel.orig/arch/powerpc/boot/kexec.c	2007-09-20 
>> 17:42:24.000000000 -0500
>> +++ kernel/arch/powerpc/boot/kexec.c	2007-09-20 17:49:04.000000000 
>> -0500
>> @@ -99,6 +99,7 @@ void kexec_platform_init(struct boot_par
>>  	move_slaves_up();
>>
>>  	setup_initial_heap();
>> +	conv_flattree_inplace(dt_blob);
>>  	init_flat_tree(dt_blob);
>>  	/*
>>  	 * drivers can malloc and read the tree, but not realloc later
>> _______________________________________________
>> Linuxppc-dev mailing list
>> Linuxppc-dev@ozlabs.org
>> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>>
>
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ 
> _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson
>

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[David Gibson]

On Mon, Sep 24, 2007 at 01:54:32AM -0500, Milton Miller wrote:
> 
> On Sep 23, 2007, at 10:36 PM, David Gibson wrote:
> 
> > On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
[snip]
> >> +#define MIN_VERSION 2
> >> +#define OUT_VERSION 16
> >
> > Should output version 17.  In any case, don't try to be so general -
> > just convert v123 (all basically the same) to latest (i.e. v17)
> > without all the #if nonsense.
> 
> Outputing v17 instead of 16 requires more words to be added to the 
> header, and the library does fine with v16.

For now.  libfdt will want v17.  Although it will (eventually) have
it's own v16->v17 conversion code.

>  Actually the v1 trees has 
> some other differences such as initrd addresses were kernel linear not 
> real, cpus were assigned logical numbers  ... so while the structure 
> didn't change except for the header field, the contents did.

!? what's your source for this.  v2 and v3 were absolutely supposed to
be backwards compatible with v1 which would not be the case with
silent semantic changes such as this.

>  Actually, 
> when converting v3 to v16 some of the code issn't needed, the ifs allow 
> the code size to be reduced.

Yes, but it never will be, because the only reason we'd include this
file is for converting old kexec-tools device trees which are v2 not
v3.

> >> +#define OUT_COMPAT 16
> >> +
> >> +#ifdef NO_CHECK
> >> +static int check_v123_tree(u32 *start, u32 *limit)
> >> +{
> >> +	return 0;
> >> +}
> >> +#else
> >> +/**
> >> + * check_v123_tree - check integrety of a version 1, 2, or 3 tree
> >> + * @start: the start of the device tree struct
> >> + * @limit: the end of the region for the struct
> >> + * structural checks on device_tree
> >> + */
> >> +static int check_v123_tree(u32 *start, u32 *limit)
> >
> > What is the point of this check?  If the device tree is corrupt, we're
> > stuffed anyway, so why bother?
> 
> Hence the ifdef NO_CHECK.   When developing, sometimes its nice to know 
> if its your input or your program.  These functions are destructive to 
> an improperlly formed tree, and in non-obvious ways.  When debugging, 
> it's not hard to hardcode console write or read the printf string 
> buffer with a hardware debugger to see error messages.  That said, it 
> could be removed.

Right.  Debugging code shouldn't pollute final patches.

[snip]
> >> +#if MIN_VERSION < 3 || OUT_VERSION > 16
> >> +/**
> >> + * move_nops_fwd - move nops in a v16 dt_struct to the beginning
> >> + * @start - device tree starting address
> >> + * @count - number of %OF_DT_NOP cells to move
> >> + */
> >> +static void move_nops_fwd(u32 *start, int count)
> >
> > What on earth is the point of this.  The NOPs are perfectly valid
> > scattered within the tree, why go to all this trouble to shuffle them
> > about.
> 
> And if you notice, there is a "how many to move" argument.  The point 
> of moving them to the front of the tree is the v17 device tree header 
> takes more space than the v3 one, and the v2 header is smaller than 
> both v17 and v16 header.  Since I am converting the tree in place, the 
> space has to come from somewhere.  Since we are pretty much guaranteed 
> to get several nops, this function moves them forward so they can be 
> overwritten.  In practice we move 1-3 NOPS from the early properties > 
> 8 bytes and early grandchild nodes (eg /cpus/PowerPC,xxx).

This is a hell of a lot of bother to go to for a few bytes.
Allocating a big-enough buffer for any reasonable devtree in BSS and
memmove()ing into there would be far simpler.

Remember this is a hack for horrid old device trees produced by
kexec-tools.  It simply doesn't justify large amounts of code to work
around.

[snip]
> >> +	if (tree->last_comp_version > 3)
> >> +		return;			/* don't know what to do */
> >
> > Rather, don't need to do anything.
> 
> If the tree is >= 16 then we don't presently need to do anything.   If 
> there is a theoritical v4 tree we don't know what to do.  And if output 
> version is 17 but input is 16 we don't know what to do either, because 
> there likely aren't nops in the tree to consume.  I suppose we could 
> preceed it with a check for version == OUTPUT_VERSION, but then I'm 
> sure I'd get pointless differentation :-).

There will never be a v4.  Or anything between v3 and v16.

Again this is far too general.  It's a hack to deal with kexec-tools
old trees.  Therefore it doesn't need to deal with any general old
tree, just the minimum to deal with trees as produced by old
kexec-tools.

[snip]
> >> +	/*
> >> +	 * move mem_rsvmap and dt_strings if they are before dt_struct
> >> +	 * onto our nops .  Adjust start addresses for the 3 sections.
> >> +	 */
> >
> > Hrm.  Do we really need to worry about this case.  You may be
> > producing v2 trees in kexec-tools, but do they actually have the
> > blocks out of order?  dtc certainly never produced them that way.
> 
> Out of order?  There has never been a spec as to the order of the 
> blocks, only the implicit assumption that they follow the device tree 
> header in a reasonably packed sequence.  booting-without-of says it 
> must be in ram,; the offsets are unsigned 32 bit quantities.

> As to the order, used, the first implemntation was the kernel which 
> writes memreserve, strings, then struct (both the openfirmware client 
> in prom_init and the iSeries procedural library in dt.c).  The second 
> implentation written is a procedural based library (similar to iseries, 
> never published, but still used internally) that starts with a 
> pre-built header and string table, builds the dt_struct as the 
> functions are called, and when finished copies the memreserve table and 
> fills in the dt_size, total size, and memreserve offset.

Hrm.  Yes, well, the iSeries tree is weird in a bunch of ways.

> fs2dt writes memreserve, struct, then strings.  Aparently the same as 
> dtc.  But yes, the strings can be before the struct, and the mem 
> reserve may or may not be when the strings are before the struct.

Again.  We don't need to deal with the general case here - just the
real case of trees produced by old kexec-tools.

[snip]

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

On Sep 24, 2007, at 10:46 PM, David Gibson wrote:
> On Mon, Sep 24, 2007 at 01:54:32AM -0500, Milton Miller wrote:
>> On Sep 23, 2007, at 10:36 PM, David Gibson wrote:
>>> On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
> [snip]
>>>> +#define MIN_VERSION 2
>>>> +#define OUT_VERSION 16
>>>
>>> Should output version 17.  In any case, don't try to be so general -
>>> just convert v123 (all basically the same) to latest (i.e. v17)
>>> without all the #if nonsense.
>>
>> Outputing v17 instead of 16 requires more words to be added to the
>> header, and the library does fine with v16.
>
> For now.  libfdt will want v17.  Although it will (eventually) have
> it's own v16->v17 conversion code.

If libfdt gets merged without supporting v16 input, then that will be a  
regression.  This code isn't pretending to address that.   The current  
flat tree code works with v16.

>>  Actually the v1 trees has
>> some other differences such as initrd addresses were kernel linear not
>> real, cpus were assigned logical numbers  ... so while the structure
>> didn't change except for the header field, the contents did.
>
> !? what's your source for this.  v2 and v3 were absolutely supposed to
> be backwards compatible with v1 which would not be the case with
> silent semantic changes such as this.

What's your souce for saying the were supposed to be backwards  
compatable?  That dtc fills out the struct header so?

My source is my involvment when v2 was defined (they were discovered  
while writing my device tree generation code):

The actual binary structure is compatable, just not the contents of the  
properties nor how any slave cpus wait (for some trees it doesn't  
matter).

http://git.kernel.org/?p=linux/kernel/git/horms/kexec-tools- 
testing.git;a=blob;f=kexec/arch/ppc64/fs2dt.c; 
hb=b84b87747a16f0afbef6f6802bb794a94f4961d9

And some more changes just before that:

http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git; 
a=history;f=arch/ppc64/kernel/prom_init.c; 
h=e570799a84cc5328e9f0fd44592cb0b828d8c13a; 
hb=4ae24c4e8a8f68950a7774ca1cdfe69bfe4e2ffc

So its mostly when the kernel generated and required v1 trees, it was  
ppc64 only and had these other content and handoff semantics.  If it  
were to get a v1 tree, it only copes for the boot cpu determination    
I'm not aware of any code other than the kernel that would actually  
generate a v1 tree (other than dtc, which always supporteed v2, and  
doesn't care about these differences).


>>  Actually,
>> when converting v3 to v16 some of the code issn't needed, the ifs  
>> allow
>> the code size to be reduced.
>
> Yes, but it never will be, because the only reason we'd include this
> file is for converting old kexec-tools device trees which are v2 not
> v3.
>
>>>> +#define OUT_COMPAT 16
>>>> +
>>>> +#ifdef NO_CHECK
>>>> +static int check_v123_tree(u32 *start, u32 *limit)
>>>> +{
>>>> +	return 0;
>>>> +}
>>>> +#else
>>>> +/**
>>>> + * check_v123_tree - check integrety of a version 1, 2, or 3 tree
>>>> + * @start: the start of the device tree struct
>>>> + * @limit: the end of the region for the struct
>>>> + * structural checks on device_tree
>>>> + */
>>>> +static int check_v123_tree(u32 *start, u32 *limit)
>>>
>>> What is the point of this check?  If the device tree is corrupt,  
>>> we're
>>> stuffed anyway, so why bother?
>>
>> Hence the ifdef NO_CHECK.   When developing, sometimes its nice to  
>> know
>> if its your input or your program.  These functions are destructive to
>> an improperlly formed tree, and in non-obvious ways.  When debugging,
>> it's not hard to hardcode console write or read the printf string
>> buffer with a hardware debugger to see error messages.  That said, it
>> could be removed.
>
> Right.  Debugging code shouldn't pollute final patches.

There debug code, and validation code.   I call this the later.  It  
validates that the information that should be redundant is in fact  
redundant before its removed.  The response of the kernel to the tree  
may be different if this check fails..

> [snip]
>>>> +#if MIN_VERSION < 3 || OUT_VERSION > 16
>>>> +/**
>>>> + * move_nops_fwd - move nops in a v16 dt_struct to the beginning
>>>> + * @start - device tree starting address
>>>> + * @count - number of %OF_DT_NOP cells to move
>>>> + */
>>>> +static void move_nops_fwd(u32 *start, int count)
>>>
>>> What on earth is the point of this.  The NOPs are perfectly valid
>>> scattered within the tree, why go to all this trouble to shuffle them
>>> about.
>>
>> And if you notice, there is a "how many to move" argument.  The point
>> of moving them to the front of the tree is the v17 device tree header
>> takes more space than the v3 one, and the v2 header is smaller than
>> both v17 and v16 header.  Since I am converting the tree in place, the
>> space has to come from somewhere.  Since we are pretty much guaranteed
>> to get several nops, this function moves them forward so they can be
>> overwritten.  In practice we move 1-3 NOPS from the early properties >
>> 8 bytes and early grandchild nodes (eg /cpus/PowerPC,xxx).
>
> This is a hell of a lot of bother to go to for a few bytes.
> Allocating a big-enough buffer for any reasonable devtree in BSS and
> memmove()ing into there would be far simpler.

How big is enough?  Who is going to do the research for the various  
firmware levels of the big machines to find out how big the tree is?

Sure there are other ways to get the tree to version 16, one way is to  
change what is generated.  We could update kexec-tools fs2dt to  
geneerate a v16 tree to support zImages.

A question for the list: does anyone else have firmware/environment  
that supplies a tree < v16?


> [snip]
>>>> +	if (tree->last_comp_version > 3)
>>>> +		return;			/* don't know what to do */
>>>
>>> Rather, don't need to do anything.
>>
>> If the tree is >= 16 then we don't presently need to do anything.   If
>> there is a theoritical v4 tree we don't know what to do.  And if  
>> output
>> version is 17 but input is 16 we don't know what to do either, because
>> there likely aren't nops in the tree to consume.  I suppose we could
>> preceed it with a check for version == OUTPUT_VERSION, but then I'm
>> sure I'd get pointless differentation :-).
>
> There will never be a v4.  Or anything between v3 and v16.

I could see a v4 defined as v3 + the extra header in v17, not that I am  
proposing that.  Someone decided to skip to v16 for a reason.   If we  
knew there would never be v4-v15, why did we skip them?

[too general]
> [snip]
>>>> +	/*
>>>> +	 * move mem_rsvmap and dt_strings if they are before dt_struct
>>>> +	 * onto our nops .  Adjust start addresses for the 3 sections.
>>>> +	 */
>>>
>>> Hrm.  Do we really need to worry about this case.  You may be
>>> producing v2 trees in kexec-tools, but do they actually have the
>>> blocks out of order?  dtc certainly never produced them that way.
>>
>> Out of order?  There has never been a spec as to the order of the
>> blocks, only the implicit assumption that they follow the device tree
>> header in a reasonably packed sequence.  booting-without-of says it
>> must be in ram,; the offsets are unsigned 32 bit quantities.
>
>> As to the order, used, the first implemntation was the kernel which
>> writes memreserve, strings, then struct (both the openfirmware client
>> in prom_init and the iSeries procedural library in dt.c).  The second
>> implentation written is a procedural based library (similar to  
>> iseries,
>> never published, but still used internally) that starts with a
>> pre-built header and string table, builds the dt_struct as the
>> functions are called, and when finished copies the memreserve table  
>> and
>> fills in the dt_size, total size, and memreserve offset.
>
> Hrm.  Yes, well, the iSeries tree is weird in a bunch of ways.

My point is its the same as the client interface code, and proves the  
point the order is not specified nor need it be.  And since the order  
is the same in platforms/iseries/dt.c and kernel/prom_init.c, I fail to  
see your point of calling the iseries code wierd.  It's building a  
device tree via the structured programming methods of calling  
functions, each of which creates a node and the properties for that  
node.  Doesn't sound that much different than how a ieee bytecode  
program would create the device tree and fill out properties, and I  
would expect any firmware who created the device tree from scratch (as  
oposed to editing a paritially built tree like the fixup hooks  
presently in the wrapper) would be structured similarly.   Since it  
doesn't ahve a dynamic insertion library, it must fill in all the  
properties of a node completely before its children, and must travel  
some depth first order when creating the tree.  I don't see any reason  
to call it wierd.

>
>> fs2dt writes memreserve, struct, then strings.  Aparently the same as
>> dtc.  But yes, the strings can be before the struct, and the mem
>> reserve may or may not be when the strings are before the struct.
>
> Again.  We don't need to deal with the general case here - just the
> real case of trees produced by old kexec-tools.

This function isn't conver_kexec_tree_inplacee.  While my justification  
for merging this code is it can be used as the target for kexec with  
kexec-tools, its not my only use.  I also have my internal library that  
generates a v3 tree with the pieces ordered differently.  If the code  
is changed from its general support to support only the tree generated  
by kexec-tools, then merging has little benifit to me and in fact it is  
harder (I have to maintain a diff to a changing target vs adding a file  
to the makefile).

I'll write a later reply that talks about what subsets of this series  
make sense to merge.

milton

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[David Gibson]

On Wed, Sep 26, 2007 at 11:19:47AM -0500, Milton Miller wrote:
> On Sep 24, 2007, at 10:46 PM, David Gibson wrote:
> > On Mon, Sep 24, 2007 at 01:54:32AM -0500, Milton Miller wrote:
> >> On Sep 23, 2007, at 10:36 PM, David Gibson wrote:
> >>> On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
> > [snip]
> >>>> +#define MIN_VERSION 2
> >>>> +#define OUT_VERSION 16
> >>>
> >>> Should output version 17.  In any case, don't try to be so general -
> >>> just convert v123 (all basically the same) to latest (i.e. v17)
> >>> without all the #if nonsense.
> >>
> >> Outputing v17 instead of 16 requires more words to be added to the
> >> header, and the library does fine with v16.
> >
> > For now.  libfdt will want v17.  Although it will (eventually) have
> > it's own v16->v17 conversion code.
> 
> If libfdt gets merged without supporting v16 input, then that will be a  
> regression.  This code isn't pretending to address that.   The current  
> flat tree code works with v16.

Hrm, true.  Since I'm working on merging libfdt right now, I guess
that moves up my schedule for getting the v16->v17 conversion code
working.

> 
> >>  Actually the v1 trees has
> >> some other differences such as initrd addresses were kernel linear not
> >> real, cpus were assigned logical numbers  ... so while the structure
> >> didn't change except for the header field, the contents did.
> >
> > !? what's your source for this.  v2 and v3 were absolutely supposed to
> > be backwards compatible with v1 which would not be the case with
> > silent semantic changes such as this.
> 
> What's your souce for saying the were supposed to be backwards  
> compatable?  That dtc fills out the struct header so?

Sitting next to BenH and knowing he always intended them to be so.

> My source is my involvment when v2 was defined (they were discovered  
> while writing my device tree generation code):
> 
> The actual binary structure is compatable, just not the contents of the  
> properties nor how any slave cpus wait (for some trees it doesn't  
> matter).
> 
> http://git.kernel.org/?p=linux/kernel/git/horms/kexec-tools- 
> testing.git;a=blob;f=kexec/arch/ppc64/fs2dt.c; 
> hb=b84b87747a16f0afbef6f6802bb794a94f4961d9

An old version of fs2dt is hardly definitive.  It could just be Plain
Wrong, nothing to do with the dt version.

> And some more changes just before that:
> 
> http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git; 
> a=history;f=arch/ppc64/kernel/prom_init.c; 
> h=e570799a84cc5328e9f0fd44592cb0b828d8c13a; 
> hb=4ae24c4e8a8f68950a7774ca1cdfe69bfe4e2ffc

I don't know what bit you're referring to in that batch of commits.

> So its mostly when the kernel generated and required v1 trees, it was  
> ppc64 only and had these other content and handoff semantics.  If it  
> were to get a v1 tree, it only copes for the boot cpu determination    
> I'm not aware of any code other than the kernel that would actually  
> generate a v1 tree (other than dtc, which always supporteed v2, and  
> doesn't care about these differences).

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

On Sep 26, 2007, at 9:45 PM, David Gibson wrote:

> On Wed, Sep 26, 2007 at 11:19:47AM -0500, Milton Miller wrote:
>> On Sep 24, 2007, at 10:46 PM, David Gibson wrote:
>>> On Mon, Sep 24, 2007 at 01:54:32AM -0500, Milton Miller wrote:
>>>> On Sep 23, 2007, at 10:36 PM, David Gibson wrote:
>>>>> On Fri, Sep 21, 2007 at 06:05:06PM -0500, Milton Miller wrote:
>>> [snip]
>>>>>> +#define MIN_VERSION 2
>>>>>> +#define OUT_VERSION 16z


>>>> Actually the v1 trees has
>>>> some other differences such as initrd addresses were kernel linear  
>>>> not
>>>> real, cpus were assigned logical numbers  ... so while the structure
>>>> didn't change except for the header field, the contents did.
>>>
>>> !? what's your source for this.  v2 and v3 were absolutely supposed  
>>> to
>>> be backwards compatible with v1 which would not be the case with
>>> silent semantic changes such as this.
>>
>> What's your souce for saying the were supposed to be backwards
>> compatable?  That dtc fills out the struct header so?
>
> Sitting next to BenH and knowing he always intended them to be so.
>
>> My source is my involvment when v2 was defined (they were discovered
>> while writing my device tree generation code):
>>
>> The actual binary structure is compatable, just not the contents of  
>> the
>> properties nor how any slave cpus wait (for some trees it doesn't
>> matter).
>>
>> http://git.kernel.org/?p=linux/kernel/git/horms/kexec-tools-
>> testing.git;a=blob;f=kexec/arch/ppc64/fs2dt.c;
>> hb=b84b87747a16f0afbef6f6802bb794a94f4961d9
>
> An old version of fs2dt is hardly definitive.  It could just be Plain
> Wrong, nothing to do with the dt version.

Sorry, copy and paste error.  I was tring to point out this changelog  
in 2.6.10:

http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git; 
a=commitdiff;h=e1b47549d1588ccea1fa5726eb430aae4e80f8ed


>
>> And some more changes just before that:
>>
>> http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git;
>> a=history;f=arch/ppc64/kernel/prom_init.c;
>> h=e570799a84cc5328e9f0fd44592cb0b828d8c13a;
>> hb=4ae24c4e8a8f68950a7774ca1cdfe69bfe4e2ffc
>
> I don't know what bit you're referring to in that batch of commits.

The following properties changed semantics and no heuristics are  
employed to check for the old vs new:
(1) tces changed from virtual to real
(2) cpus spin on physical (hw id) not logical (0-n)

Other changes in that series
(3) 0->klimit is not a memreserve in the tree (we now allow overlapping  
reserves, but not at the time)
(4) rtas properties are in a different location (but both could exist)

>
>> So its mostly when the kernel generated and required v1 trees, it was
>> ppc64 only and had these other content and handoff semantics.  If it
>> were to get a v1 tree, it only copes for the boot cpu determination
>> I'm not aware of any code other than the kernel that would actually
>> generate a v1 tree (other than dtc, which always supporteed v2, and
>> doesn't care about these differences).

So trying to boot 2.6.9 (2004-10-18 cutoff) kernel from a tree for  
2.6.10 would fail, and vice versa.  But a 2.6.10 kernel can boot a v1  
tree with properties, memreserves, and cpu ids that it expects, getting  
the boot-cpuid from the extra property in the tree.

Is that compatible?  If you only are talking about parsing the tree it  
is.

milton

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[David Gibson]

On Thu, Sep 27, 2007 at 10:44:27AM -0500, Milton Miller wrote:
> 
> On Sep 26, 2007, at 9:45 PM, David Gibson wrote:
[snip]
> >> What's your souce for saying the were supposed to be backwards
> >> compatable?  That dtc fills out the struct header so?
> >
> > Sitting next to BenH and knowing he always intended them to be so.
> >
> >> My source is my involvment when v2 was defined (they were discovered
> >> while writing my device tree generation code):
> >>
> >> The actual binary structure is compatable, just not the contents of  
> >> the
> >> properties nor how any slave cpus wait (for some trees it doesn't
> >> matter).
> >>
> >> http://git.kernel.org/?p=linux/kernel/git/horms/kexec-tools-
> >> testing.git;a=blob;f=kexec/arch/ppc64/fs2dt.c;
> >> hb=b84b87747a16f0afbef6f6802bb794a94f4961d9
> >
> > An old version of fs2dt is hardly definitive.  It could just be Plain
> > Wrong, nothing to do with the dt version.
> 
> Sorry, copy and paste error.  I was tring to point out this changelog  
> in 2.6.10:
> 
> http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git; 
> a=commitdiff;h=e1b47549d1588ccea1fa5726eb430aae4e80f8ed

Hrm, I see, yes that seems conclusive enough.  Yuck.

In which case, I think we should try to forget that v1 ever existed.
I don't think anyone ever generated v1 trees other than kernels which
also consumed them, and I doubt current kernels will correctly deal
with v1 trees in this form.

In any case, this is all rather beside the point.  My basic point is
that this bootwrapper stuff should not attempt to be a general
backwards compatibility layer for every broken early dt format that
ever existed.  In general we should try to make sure nothing ever uses
<v16 trees.  We want, here, to do the minimum we can get away with to
support the specific v2 trees produced by kexec-tools, as an interim
measure while kexec-tools itself is fixed to produce v17 trees (say,
by replacing fs2dt with dtc plus a libfdt based post-processing
program).

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[Milton Miller]

On Sep 27, 2007, at 9:40 PM, David Gibson wrote:
> On Thu, Sep 27, 2007 at 10:44:27AM -0500, Milton Miller wrote:
>> On Sep 26, 2007, at 9:45 PM, David Gibson wrote:
>>>> The actual binary structure is compatable, just not the contents of
>>>> the
>>>> properties nor how any slave cpus wait (for some trees it doesn't
>>>> matter).
>>
>> Sorry, copy and paste error.  I was tring to point out this changelog
>> in 2.6.10:
>>
>> http://git.kernel.org/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git;
>> a=commitdiff;h=e1b47549d1588ccea1fa5726eb430aae4e80f8ed
>
> Hrm, I see, yes that seems conclusive enough.  Yuck.
>
> In which case, I think we should try to forget that v1 ever existed.
> I don't think anyone ever generated v1 trees other than kernels which
> also consumed them,

I believe this to be true, unless you count telling dtc to do so.

> and I doubt current kernels will correctly deal
> with v1 trees in this form.

I'm quite certain of that.

> In any case, this is all rather beside the point.  My basic point is
> that this bootwrapper stuff should not attempt to be a general
> backwards compatibility layer for every broken early dt format that
> ever existed.

The only broken format was v1; I was submitting v2 :-).  Version 16 was  
"... to support a more compact representation, for use by embedded  
systems mostly"  
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git; 
a=commitdiff;h=34153fa3af45d84f3221d9b67ba2ab7e8a220d28

> In general we should try to make sure nothing ever uses
> <v16 trees.  We want, here, to do the minimum we can get away with to
> support the specific v2 trees produced by kexec-tools, as an interim
> measure

As I stated, that by itself isn't sufficient for my usage, as I have  
other v2 trees I need to deal with, at least until that generator gets  
updated.

> while kexec-tools itself is fixed to produce v17 trees (say,
> by replacing fs2dt with dtc plus a libfdt based post-processing
> program).

If you thought there were complaints trying to require an external  
utility to build zImage, just wait until you try to make kexec-tools  
not be self-contained.  Currently kexec doesn't even use a data  
directory, instead it builds the purgatory and other trampoline  
binaries into the kexec program.  Incorporating the post-processing and  
generation using libfdt (with a copy of the library in the source)  
could fly, if people don't care about kexec into kernels from 2.6.10 to  
2.6.14 when v16 was merged (about 1 year).

Actually, there is another approach: put this converter in kexec's  
purgatory or even the kexec program.  It can then run conditionally  
under command line flags to keep compatibility with the old kernels.   
If and when its is decided to only support >=v16 in kexec-tools it can  
be removed and we don't have this interim support code in the kernel  
forever (I'll handle my other usage out of tree).

milton

Re: [PATCH 8/15] bootwrapper: convert flatdevtree to version 16

[David Gibson]

On Fri, Sep 28, 2007 at 10:16:51AM -0500, Milton Miller wrote:
[snip]
> Actually, there is another approach: put this converter in kexec's  
> purgatory or even the kexec program.  It can then run conditionally  
> under command line flags to keep compatibility with the old kernels.   
> If and when its is decided to only support >=v16 in kexec-tools it can  
> be removed and we don't have this interim support code in the kernel  
> forever (I'll handle my other usage out of tree).

Hurrah!  Well, please do that then.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson