[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Ilya Yanok]

Hello everybody,

here is a set of changes we made to improve U-Boot JFFS2 code
performance. We still can't reach Linux's performance but improvements
are significant.

Any comments are welcome.

Regards, Ilya.

[U-Boot] [PATCH] jffs2: add sector_size field to part_info structure

[Ilya Yanok]

This patch adds sector_size field to part_info structure (used
by new JFFS2 code).

Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 common/cmd_jffs2.c          |   13 ++++++++++++-
 include/jffs2/load_kernel.h |    1 +
 2 files changed, 13 insertions(+), 1 deletions(-)

diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c
index c6920c9..e7f07bf 100644
--- a/common/cmd_jffs2.c
+++ b/common/cmd_jffs2.c
@@ -339,11 +339,13 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
 	extern flash_info_t flash_info[];
 	flash_info_t *flash;
 	int offset_aligned;
-	u32 end_offset;
+	u32 end_offset, sector_size = 0;
 	int i;
 
 	flash = &flash_info[id->num];
 
+	part->sector_size = 0;
+
 	offset_aligned = 0;
 	for (i = 0; i < flash->sector_count; i++) {
 		if ((flash->start[i] - flash->start[0]) == part->offset) {
@@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
 
 	end_offset = part->offset + part->size;
 	for (i = 0; i < flash->sector_count; i++) {
+		if (i) {
+			sector_size = flash->start[i] - flash->start[i-1];
+			if (part->sector_size < sector_size)
+				part->sector_size = sector_size;
+		}
 		if ((flash->start[i] - flash->start[0]) == end_offset)
 			return 0;
 	}
@@ -389,6 +396,8 @@ static int part_validate_nand(struct mtdids *id, struct part_info *part)
 
 	nand = &nand_info[id->num];
 
+	part->sector_size = nand->erasesize;
+
 	if ((unsigned long)(part->offset) % nand->erasesize) {
 		printf("%s%d: partition (%s) start offset alignment incorrect\n",
 				MTD_DEV_TYPE(id->type), id->num, part->name);
@@ -424,6 +433,8 @@ static int part_validate_onenand(struct mtdids *id, struct part_info *part)
 
 	mtd = &onenand_mtd;
 
+	part->sector_size = mtd->erasesize;
+
 	if ((unsigned long)(part->offset) % mtd->erasesize) {
 		printf("%s%d: partition (%s) start offset"
 			"alignment incorrect\n",
diff --git a/include/jffs2/load_kernel.h b/include/jffs2/load_kernel.h
index 551fd0c..c0442a2 100644
--- a/include/jffs2/load_kernel.h
+++ b/include/jffs2/load_kernel.h
@@ -50,6 +50,7 @@ struct part_info {
 	u32 offset;			/* offset within device */
 	void *jffs2_priv;		/* used internaly by jffs2 */
 	u32 mask_flags;			/* kernel MTD mask flags */
+	u32 sector_size;		/* size of sector */
 	struct mtd_device *dev;		/* parent device */
 };
 
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: add sector_size field to part_info structure

[Wolfgang Denk]

Dear Ilya,

In message <1223877234-720-2-git-send-email-yanok@emcraft.com> you wrote:
> This patch adds sector_size field to part_info structure (used
> by new JFFS2 code).
...
> @@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
>  
>  	end_offset = part->offset + part->size;
>  	for (i = 0; i < flash->sector_count; i++) {
> +		if (i) {
> +			sector_size = flash->start[i] - flash->start[i-1];
> +			if (part->sector_size < sector_size)
> +				part->sector_size = sector_size;
> +		}

What about the size of the last sector?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
God is real, unless declared integer.

[U-Boot] [PATCH] jffs2: add sector_size field to part_info structure

[Ilya Yanok]

Hi Wolfgang,

Wolfgang Denk wrote:
>> @@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
>>  
>>  	end_offset = part->offset + part->size;
>>  	for (i = 0; i < flash->sector_count; i++) {
>> +		if (i) {
>> +			sector_size = flash->start[i] - flash->start[i-1];
>> +			if (part->sector_size < sector_size)
>> +				part->sector_size = sector_size;
>> +		}
>>     
>
> What about the size of the last sector?
>   

Well, you are right, I've lost last sector... I'll fix this.

Regards, Ilya.

[U-Boot] [PATCH] jffs2: rewrite jffs2 scanning code based on Linux one

[Ilya Yanok]

Rewrites jffs2_1pass_build_lists() function in style of Linux's
jffs2_scan_medium() and jffs2_scan_eraseblock().
This includes:
 - Caching flash acceses
 - Smart dealing with free space

Signed-off-by: Alexey Neyman <avn@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 fs/jffs2/jffs2_1pass.c |  227 ++++++++++++++++++++++++++++++++++++------------
 1 files changed, 171 insertions(+), 56 deletions(-)

diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
index 57f5582..61a0459 100644
--- a/fs/jffs2/jffs2_1pass.c
+++ b/fs/jffs2/jffs2_1pass.c
@@ -389,6 +389,12 @@ static inline void *get_fl_mem_nor(u32 off)
 	return (void*)addr;
 }
 
+static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf)
+{
+	memcpy(ext_buf, get_fl_mem_nor(off), size);
+	return ext_buf;
+}
+
 static inline void *get_node_mem_nor(u32 off)
 {
 	return (void*)get_fl_mem_nor(off);
@@ -405,8 +411,11 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf)
 	struct mtdids *id = current_part->dev->id;
 
 #if defined(CONFIG_CMD_FLASH)
-	if (id->type == MTD_DEV_TYPE_NOR)
+	if (id->type == MTD_DEV_TYPE_NOR) {
+		if (ext_buf)
+			return get_fl_mem_nor_copy(off, size, ext_buf);
 		return get_fl_mem_nor(off);
+	}
 #endif
 
 #if defined(CONFIG_JFFS2_NAND) && defined(CONFIG_CMD_NAND)
@@ -480,9 +489,6 @@ static char *compr_names[] = {
 #endif
 };
 
-/* Spinning wheel */
-static char spinner[] = { '|', '/', '-', '\\' };
-
 /* Memory management */
 struct mem_block {
 	u32	index;
@@ -653,23 +659,6 @@ static int compare_dirents(struct b_node *new, struct b_node *old)
 }
 #endif
 
-static u32
-jffs2_scan_empty(u32 start_offset, struct part_info *part)
-{
-	char *max = (char *)(part->offset + part->size - sizeof(struct jffs2_raw_inode));
-	char *offset = (char *)(part->offset + start_offset);
-	u32 off;
-
-	while (offset < max &&
-	       *(u32*)get_fl_mem((u32)offset, sizeof(u32), &off) == 0xFFFFFFFF) {
-		offset += sizeof(u32);
-		/* return if spinning is due */
-		if (((u32)offset & ((1 << SPIN_BLKSIZE)-1)) == 0) break;
-	}
-
-	return (u32)offset - part->offset;
-}
-
 void
 jffs2_free_cache(struct part_info *part)
 {
@@ -768,6 +757,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 					put_fl_mem(jNode);
 					continue;
 				}
+				if (!data_crc(jNode)) {
+					put_fl_mem(jNode);
+					continue;
+				}
 
 				lDest = (uchar *) (dest + jNode->offset);
 #if 0
@@ -1271,17 +1264,33 @@ dump_dirents(struct b_lists *pL)
 }
 #endif
 
+#define min_t(type, x, y) ({                    \
+	type __min1 = (x);                      \
+	type __min2 = (y);                      \
+	__min1 < __min2 ? __min1: __min2; })
+
+#define DEFAULT_EMPTY_SCAN_SIZE	4096
+
+static inline uint32_t EMPTY_SCAN_SIZE(uint32_t sector_size)
+{
+	if (sector_size < DEFAULT_EMPTY_SCAN_SIZE)
+		return sector_size;
+	else
+		return DEFAULT_EMPTY_SCAN_SIZE;
+}
+
 static u32
 jffs2_1pass_build_lists(struct part_info * part)
 {
 	struct b_lists *pL;
 	struct jffs2_unknown_node *node;
-	u32 offset, oldoffset = 0;
-	u32 max = part->size - sizeof(struct jffs2_raw_inode);
-	u32 counter = 0;
+	u32 nr_sectors = part->size/part->sector_size;
+	u32 i;
 	u32 counter4 = 0;
 	u32 counterF = 0;
 	u32 counterN = 0;
+	u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE;
+	char *buf;
 
 	/* turn off the lcd.  Refreshing the lcd adds 50% overhead to the */
 	/* jffs2 list building enterprise nope.  in newer versions the overhead is */
@@ -1291,70 +1300,176 @@ jffs2_1pass_build_lists(struct part_info * part)
 	/* if we are building a list we need to refresh the cache. */
 	jffs_init_1pass_list(part);
 	pL = (struct b_lists *)part->jffs2_priv;
-	offset = 0;
+	buf = malloc(buf_size);
 	puts ("Scanning JFFS2 FS:   ");
 
 	/* start at the beginning of the partition */
-	while (offset < max) {
-		if ((oldoffset >> SPIN_BLKSIZE) != (offset >> SPIN_BLKSIZE)) {
-			printf("\b\b%c ", spinner[counter++ % sizeof(spinner)]);
-			oldoffset = offset;
-		}
+	for (i = 0; i < nr_sectors; i++) {
+		uint32_t sector_ofs = i * part->sector_size;
+		uint32_t buf_ofs = sector_ofs;
+		uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size);
+		uint32_t ofs, prevofs;
 
 		WATCHDOG_RESET();
+		get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
 
-		node = (struct jffs2_unknown_node *) get_node_mem((u32)part->offset + offset);
-		if (node->magic == JFFS2_MAGIC_BITMASK && hdr_crc(node)) {
+		/* We temporarily use 'ofs' as a pointer into the buffer/jeb */
+		ofs = 0;
+
+		/* Scan only 4KiB of 0xFF before declaring it's empty */
+		while (ofs < EMPTY_SCAN_SIZE(part->sector_size) &&
+				*(uint32_t *)(&buf[ofs]) == 0xFFFFFFFF)
+			ofs += 4;
+
+		if (ofs == EMPTY_SCAN_SIZE(part->sector_size))
+			continue;
+
+		ofs += sector_ofs;
+		prevofs = ofs - 1;
+
+	scan_more:
+		while (ofs < sector_ofs + part->sector_size) {
+			if (ofs == prevofs) {
+				printf("offset %08x already seen, skip\n", ofs);
+				ofs += 4;
+				counter4++;
+				continue;
+			}
+			prevofs = ofs;
+			if (sector_ofs + part->sector_size <
+					ofs + sizeof(*node))
+				break;
+			if (buf_ofs + buf_len < ofs + sizeof(*node)) {
+				buf_len = min_t(uint32_t, buf_size, sector_ofs
+						+ part->sector_size - ofs);
+				get_fl_mem((u32)part->offset + ofs, buf_len,
+					   buf);
+				buf_ofs = ofs;
+			}
+
+			node = (struct jffs2_unknown_node *)&buf[ofs-buf_ofs];
+
+			if (*(uint32_t *)(&buf[ofs-buf_ofs]) == 0xffffffff) {
+				uint32_t inbuf_ofs;
+				uint32_t empty_start, scan_end;
+
+				empty_start = ofs;
+				ofs += 4;
+				scan_end = min_t(uint32_t, EMPTY_SCAN_SIZE(
+							part->sector_size)/8,
+							buf_len);
+			more_empty:
+				inbuf_ofs = ofs - buf_ofs;
+				while (inbuf_ofs < scan_end) {
+					if (*(uint32_t *)(&buf[inbuf_ofs]) !=
+							0xffffffff)
+						goto scan_more;
+
+					inbuf_ofs += 4;
+					ofs += 4;
+				}
+				/* Ran off end. */
+
+				/* See how much more there is to read in this
+				 * eraseblock...
+				 */
+				buf_len = min_t(uint32_t, buf_size,
+						sector_ofs +
+						part->sector_size - ofs);
+				if (!buf_len) {
+					/* No more to read. Break out of main
+					 * loop without marking this range of
+					 * empty space as dirty (because it's
+					 * not)
+					 */
+					break;
+				}
+				scan_end = buf_len;
+				get_fl_mem((u32)part->offset + ofs, buf_len,
+					   buf);
+				buf_ofs = ofs;
+				goto more_empty;
+			}
+			if (node->magic != JFFS2_MAGIC_BITMASK ||
+					!hdr_crc(node)) {
+				ofs += 4;
+				counter4++;
+				continue;
+			}
+			if (ofs + node->totlen >
+					sector_ofs + part->sector_size) {
+				ofs += 4;
+				counter4++;
+				continue;
+			}
 			/* if its a fragment add it */
-			if (node->nodetype == JFFS2_NODETYPE_INODE &&
-				    inode_crc((struct jffs2_raw_inode *) node) &&
-				    data_crc((struct jffs2_raw_inode *) node)) {
+			switch (node->nodetype) {
+			case JFFS2_NODETYPE_INODE:
+				if (buf_ofs + buf_len < ofs + sizeof(struct
+							jffs2_raw_inode)) {
+					get_fl_mem((u32)part->offset + ofs,
+						   buf_len, buf);
+					buf_ofs = ofs;
+					node = (void *)buf;
+				}
+				if (!inode_crc((struct jffs2_raw_inode *) node))
+				       break;
+
 				if (insert_node(&pL->frag, (u32) part->offset +
-						offset) == NULL) {
-					put_fl_mem(node);
+						ofs) == NULL)
 					return 0;
+				break;
+			case JFFS2_NODETYPE_DIRENT:
+				if (buf_ofs + buf_len < ofs + sizeof(struct
+							jffs2_raw_dirent) +
+							((struct
+							 jffs2_raw_dirent *)
+							node)->nsize) {
+					get_fl_mem((u32)part->offset + ofs,
+						   buf_len, buf);
+					buf_ofs = ofs;
+					node = (void *)buf;
 				}
-			} else if (node->nodetype == JFFS2_NODETYPE_DIRENT &&
-				   dirent_crc((struct jffs2_raw_dirent *) node)  &&
-				   dirent_name_crc((struct jffs2_raw_dirent *) node)) {
+
+				if (!dirent_crc((struct jffs2_raw_dirent *)
+							node) ||
+						!dirent_name_crc(
+							(struct
+							 jffs2_raw_dirent *)
+							node))
+					break;
 				if (! (counterN%100))
 					puts ("\b\b.  ");
 				if (insert_node(&pL->dir, (u32) part->offset +
-						offset) == NULL) {
-					put_fl_mem(node);
+						ofs) == NULL)
 					return 0;
-				}
 				counterN++;
-			} else if (node->nodetype == JFFS2_NODETYPE_CLEANMARKER) {
+				break;
+			case JFFS2_NODETYPE_CLEANMARKER:
 				if (node->totlen != sizeof(struct jffs2_unknown_node))
 					printf("OOPS Cleanmarker has bad size "
 						"%d != %zu\n",
 						node->totlen,
 						sizeof(struct jffs2_unknown_node));
-			} else if (node->nodetype == JFFS2_NODETYPE_PADDING) {
+				break;
+			case JFFS2_NODETYPE_PADDING:
 				if (node->totlen < sizeof(struct jffs2_unknown_node))
 					printf("OOPS Padding has bad size "
 						"%d < %zu\n",
 						node->totlen,
 						sizeof(struct jffs2_unknown_node));
-			} else {
+				break;
+			default:
 				printf("Unknown node type: %x len %d offset 0x%x\n",
 					node->nodetype,
-					node->totlen, offset);
+					node->totlen, ofs);
 			}
-			offset += ((node->totlen + 3) & ~3);
+			ofs += ((node->totlen + 3) & ~3);
 			counterF++;
-		} else if (node->magic == JFFS2_EMPTY_BITMASK &&
-			   node->nodetype == JFFS2_EMPTY_BITMASK) {
-			offset = jffs2_scan_empty(offset, part);
-		} else {	/* if we know nothing, we just step and look. */
-			offset += 4;
-			counter4++;
 		}
-/*             printf("unknown node magic %4.4x %4.4x @ %lx\n", node->magic, node->nodetype, (unsigned long)node); */
-		put_fl_mem(node);
 	}
 
+	free(buf);
 	putstr("\b\b done.\r\n");		/* close off the dots */
 	/* turn the lcd back on. */
 	/* splash(); */
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: summary support

[Ilya Yanok]

This patch adds support for reading fs information from summary
node instead of scanning full eraseblock.

Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 fs/jffs2/jffs2_1pass.c |  187 +++++++++++++++++++++++++++++++++++++++++++++++-
 fs/jffs2/summary.h     |  163 +++++++++++++++++++++++++++++++++++++++++
 include/jffs2/jffs2.h  |   19 +++++
 3 files changed, 368 insertions(+), 1 deletions(-)
 create mode 100644 fs/jffs2/summary.h

diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
index 61a0459..58298c0 100644
--- a/fs/jffs2/jffs2_1pass.c
+++ b/fs/jffs2/jffs2_1pass.c
@@ -138,6 +138,8 @@
 # define DEBUGF(fmt,args...)
 #endif
 
+#include "summary.h"
+
 /* keeps pointer to currentlu processed partition */
 static struct part_info *current_part;
 
@@ -1202,6 +1204,132 @@ jffs2_1pass_rescan_needed(struct part_info *part)
 	return 0;
 }
 
+#define dbg_summary(...) do {} while (0);
+/* Process the stored summary information - helper function for
+ * jffs2_sum_scan_sumnode()
+ */
+
+static int jffs2_sum_process_sum_data(struct part_info *part, uint32_t offset,
+				struct jffs2_raw_summary *summary,
+				struct b_lists *pL)
+{
+	void *sp;
+	int i;
+
+	sp = summary->sum;
+
+	for (i = 0; i < summary->sum_num; i++) {
+		dbg_summary("processing summary index %d\n", i);
+
+		switch (((struct jffs2_sum_unknown_flash *)sp)->nodetype) {
+			case JFFS2_NODETYPE_INODE: {
+				struct jffs2_sum_inode_flash *spi;
+				spi = sp;
+
+				dbg_summary("Inode at 0x%08x-0x%08x\n",
+					    offset + spi->offset,
+					    offset + spi->offset + spi->totlen);
+
+				if (insert_node(&pL->frag, (u32) part->offset +
+						offset + spi->offset) == NULL)
+					return -1;
+
+				sp += JFFS2_SUMMARY_INODE_SIZE;
+
+				break;
+			}
+
+			case JFFS2_NODETYPE_DIRENT: {
+				struct jffs2_sum_dirent_flash *spd;
+				spd = sp;
+
+				dbg_summary("Dirent at 0x%08x-0x%08x\n",
+					    offset + spd->offset,
+					    offset + spd->offset + spd->totlen);
+
+				if (insert_node(&pL->dir, (u32) part->offset +
+						offset + spd->offset) == NULL)
+					return -1;
+
+				sp += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize);
+
+				break;
+			}
+			default : {
+				uint16_t nodetype =
+					((struct jffs2_sum_unknown_flash *)
+					 sp)->nodetype;
+				printf("Unsupported node type %x found in "
+						"summary!\n", nodetype);
+				break;
+			}
+		}
+	}
+	return 0;
+}
+
+/* Process the summary node - called from jffs2_scan_eraseblock() */
+int jffs2_sum_scan_sumnode(struct part_info *part, uint32_t offset,
+			   struct jffs2_raw_summary *summary, uint32_t sumsize,
+			   struct b_lists *pL)
+{
+	struct jffs2_unknown_node crcnode;
+	int ret, ofs;
+	uint32_t crc;
+
+	ofs = part->sector_size - sumsize;
+
+	dbg_summary("summary found for 0x%08x at 0x%08x (0x%x bytes)\n",
+		    offset, offset + ofs, sumsize);
+
+	/* OK, now check for node validity and CRC */
+	crcnode.magic = JFFS2_MAGIC_BITMASK;
+	crcnode.nodetype = JFFS2_NODETYPE_SUMMARY;
+	crcnode.totlen = summary->totlen;
+	crc = crc32_no_comp(0, (uchar *)&crcnode, sizeof(crcnode)-4);
+
+	if (summary->hdr_crc != crc) {
+		dbg_summary("Summary node header is corrupt (bad CRC or "
+				"no summary at all)\n");
+		goto crc_err;
+	}
+
+	if (summary->totlen != sumsize) {
+		dbg_summary("Summary node is corrupt (wrong erasesize?)\n");
+		goto crc_err;
+	}
+
+	crc = crc32_no_comp(0, (uchar *)summary,
+			sizeof(struct jffs2_raw_summary)-8);
+
+	if (summary->node_crc != crc) {
+		dbg_summary("Summary node is corrupt (bad CRC)\n");
+		goto crc_err;
+	}
+
+	crc = crc32_no_comp(0, (uchar *)summary->sum,
+			sumsize - sizeof(struct jffs2_raw_summary));
+
+	if (summary->sum_crc != crc) {
+		dbg_summary("Summary node data is corrupt (bad CRC)\n");
+		goto crc_err;
+	}
+
+	if (summary->cln_mkr)
+		dbg_summary("Summary : CLEANMARKER node \n");
+
+	ret = jffs2_sum_process_sum_data(part, offset, summary, pL);
+	if (ret)
+		return ret;		/* real error */
+
+	return 1;
+
+crc_err:
+	putstr("Summary node crc error, skipping summary information.\n");
+
+	return 0;
+}
+
 #ifdef DEBUG_FRAGMENTS
 static void
 dump_fragments(struct b_lists *pL)
@@ -1307,10 +1435,65 @@ jffs2_1pass_build_lists(struct part_info * part)
 	for (i = 0; i < nr_sectors; i++) {
 		uint32_t sector_ofs = i * part->sector_size;
 		uint32_t buf_ofs = sector_ofs;
-		uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size);
+		uint32_t buf_len;
 		uint32_t ofs, prevofs;
+		struct jffs2_sum_marker *sm;
+		void *sumptr = NULL;
+		uint32_t sumlen;
+		int ret;
 
 		WATCHDOG_RESET();
+
+		buf_len = sizeof(*sm);
+
+		/* Read as much as we want into the _end_ of the preallocated
+		 * buffer
+		 */
+		get_fl_mem(part->offset + sector_ofs + part->sector_size -
+				buf_len, buf_len, buf + buf_size - buf_len);
+
+		sm = (void *)buf + buf_size - sizeof(*sm);
+		if (sm->magic == JFFS2_SUM_MAGIC) {
+			sumlen = part->sector_size - sm->offset;
+			sumptr = buf + buf_size - sumlen;
+
+			/* Now, make sure the summary itself is available */
+			if (sumlen > buf_size) {
+				/* Need to kmalloc for this. */
+				sumptr = malloc(sumlen);
+				if (!sumptr) {
+					putstr("Can't get memory for summary "
+							"node!\n");
+					return 0;
+				}
+				memcpy(sumptr + sumlen - buf_len, buf +
+						buf_size - buf_len, buf_len);
+			}
+			if (buf_len < sumlen) {
+				/* Need to read more so that the entire summary
+				 * node is present
+				 */
+				get_fl_mem(part->offset + sector_ofs +
+						part->sector_size - sumlen,
+						sumlen - buf_len, sumptr);
+			}
+		}
+
+		if (sumptr) {
+			ret = jffs2_sum_scan_sumnode(part, sector_ofs, sumptr,
+					sumlen, pL);
+
+			if (buf_size && sumlen > buf_size)
+				free(sumptr);
+			if (ret < 0)
+				return 0;
+			if (ret)
+				continue;
+
+		}
+
+		buf_len = EMPTY_SCAN_SIZE(part->sector_size);
+
 		get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
 
 		/* We temporarily use 'ofs' as a pointer into the buffer/jeb */
@@ -1459,6 +1642,8 @@ jffs2_1pass_build_lists(struct part_info * part)
 						node->totlen,
 						sizeof(struct jffs2_unknown_node));
 				break;
+			case JFFS2_NODETYPE_SUMMARY:
+				break;
 			default:
 				printf("Unknown node type: %x len %d offset 0x%x\n",
 					node->nodetype,
diff --git a/fs/jffs2/summary.h b/fs/jffs2/summary.h
new file mode 100644
index 0000000..834933c
--- /dev/null
+++ b/fs/jffs2/summary.h
@@ -0,0 +1,163 @@
+/*
+ * JFFS2 -- Journalling Flash File System, Version 2.
+ *
+ * Copyright ?? 2004  Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ *		     Zoltan Sogor <weth@inf.u-szeged.hu>,
+ *		     Patrik Kluba <pajko@halom.u-szeged.hu>,
+ *		     University of Szeged, Hungary
+ *
+ * For licensing information, see the file 'LICENCE' in this directory.
+ *
+ */
+
+#ifndef JFFS2_SUMMARY_H
+#define JFFS2_SUMMARY_H
+
+#define BLK_STATE_ALLFF		0
+#define BLK_STATE_CLEAN		1
+#define BLK_STATE_PARTDIRTY	2
+#define BLK_STATE_CLEANMARKER	3
+#define BLK_STATE_ALLDIRTY	4
+#define BLK_STATE_BADBLOCK	5
+
+#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff
+#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash))
+#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x))
+#define JFFS2_SUMMARY_XATTR_SIZE (sizeof(struct jffs2_sum_xattr_flash))
+#define JFFS2_SUMMARY_XREF_SIZE (sizeof(struct jffs2_sum_xref_flash))
+
+/* Summary structures used on flash */
+
+struct jffs2_sum_unknown_flash
+{
+	__u16 nodetype;	/* node type */
+};
+
+struct jffs2_sum_inode_flash
+{
+	__u16 nodetype;	/* node type */
+	__u32 inode;		/* inode number */
+	__u32 version;	/* inode version */
+	__u32 offset;	/* offset on jeb */
+	__u32 totlen; 	/* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_flash
+{
+	__u16 nodetype;	/* == JFFS_NODETYPE_DIRENT */
+	__u32 totlen;	/* record length */
+	__u32 offset;	/* offset on jeb */
+	__u32 pino;		/* parent inode */
+	__u32 version;	/* dirent version */
+	__u32 ino; 		/* == zero for unlink */
+	uint8_t nsize;		/* dirent name size */
+	uint8_t type;		/* dirent type */
+	uint8_t name[0];	/* dirent name */
+} __attribute__((packed));
+
+struct jffs2_sum_xattr_flash
+{
+	__u16 nodetype;	/* == JFFS2_NODETYPE_XATR */
+	__u32 xid;		/* xattr identifier */
+	__u32 version;	/* version number */
+	__u32 offset;	/* offset on jeb */
+	__u32 totlen;	/* node length */
+} __attribute__((packed));
+
+struct jffs2_sum_xref_flash
+{
+	__u16 nodetype;	/* == JFFS2_NODETYPE_XREF */
+	__u32 offset;	/* offset on jeb */
+} __attribute__((packed));
+
+union jffs2_sum_flash
+{
+	struct jffs2_sum_unknown_flash u;
+	struct jffs2_sum_inode_flash i;
+	struct jffs2_sum_dirent_flash d;
+	struct jffs2_sum_xattr_flash x;
+	struct jffs2_sum_xref_flash r;
+};
+
+/* Summary structures used in the memory */
+
+struct jffs2_sum_unknown_mem
+{
+	union jffs2_sum_mem *next;
+	__u16 nodetype;	/* node type */
+};
+
+struct jffs2_sum_inode_mem
+{
+	union jffs2_sum_mem *next;
+	__u16 nodetype;	/* node type */
+	__u32 inode;		/* inode number */
+	__u32 version;	/* inode version */
+	__u32 offset;	/* offset on jeb */
+	__u32 totlen; 	/* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_mem
+{
+	union jffs2_sum_mem *next;
+	__u16 nodetype;	/* == JFFS_NODETYPE_DIRENT */
+	__u32 totlen;	/* record length */
+	__u32 offset;	/* ofset on jeb */
+	__u32 pino;		/* parent inode */
+	__u32 version;	/* dirent version */
+	__u32 ino; 		/* == zero for unlink */
+	uint8_t nsize;		/* dirent name size */
+	uint8_t type;		/* dirent type */
+	uint8_t name[0];	/* dirent name */
+} __attribute__((packed));
+
+struct jffs2_sum_xattr_mem
+{
+	union jffs2_sum_mem *next;
+	__u16 nodetype;
+	__u32 xid;
+	__u32 version;
+	__u32 offset;
+	__u32 totlen;
+} __attribute__((packed));
+
+struct jffs2_sum_xref_mem
+{
+	union jffs2_sum_mem *next;
+	__u16 nodetype;
+	__u32 offset;
+} __attribute__((packed));
+
+union jffs2_sum_mem
+{
+	struct jffs2_sum_unknown_mem u;
+	struct jffs2_sum_inode_mem i;
+	struct jffs2_sum_dirent_mem d;
+	struct jffs2_sum_xattr_mem x;
+	struct jffs2_sum_xref_mem r;
+};
+
+/* Summary related information stored in superblock */
+
+struct jffs2_summary
+{
+	uint32_t sum_size;      /* collected summary information for nextblock */
+	uint32_t sum_num;
+	uint32_t sum_padded;
+	union jffs2_sum_mem *sum_list_head;
+	union jffs2_sum_mem *sum_list_tail;
+
+	__u32 *sum_buf;	/* buffer for writing out summary */
+};
+
+/* Summary marker is stored at the end of every sumarized erase block */
+
+struct jffs2_sum_marker
+{
+	__u32 offset;	/* offset of the summary node in the jeb */
+	__u32 magic; 	/* == JFFS2_SUM_MAGIC */
+};
+
+#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_raw_summary) + sizeof(struct jffs2_sum_marker))
+
+#endif /* JFFS2_SUMMARY_H */
diff --git a/include/jffs2/jffs2.h b/include/jffs2/jffs2.h
index d142cd1..ed96bab 100644
--- a/include/jffs2/jffs2.h
+++ b/include/jffs2/jffs2.h
@@ -50,6 +50,9 @@
 #define JFFS2_EMPTY_BITMASK 0xffff
 #define JFFS2_DIRTY_BITMASK 0x0000
 
+/* Summary node MAGIC marker */
+#define JFFS2_SUM_MAGIC	0x02851885
+
 /* We only allow a single char for length, and 0xFF is empty flash so
    we don't want it confused with a real length. Hence max 254.
 */
@@ -89,6 +92,7 @@
 #define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2)
 #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
 #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
+#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
 
 /* Maybe later... */
 /*#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) */
@@ -166,9 +170,24 @@ struct jffs2_raw_inode
 /*	__u8 data[dsize]; */
 } __attribute__((packed));
 
+struct jffs2_raw_summary
+{
+	__u16 magic;
+	__u16 nodetype; 	/* = JFFS2_NODETYPE_SUMMARY */
+	__u32 totlen;
+	__u32 hdr_crc;
+	__u32 sum_num;	/* number of sum entries*/
+	__u32 cln_mkr;	/* clean marker size, 0 = no cleanmarker */
+	__u32 padded;	/* sum of the size of padding nodes */
+	__u32 sum_crc;	/* summary information crc */
+	__u32 node_crc; 	/* node crc */
+	__u32 sum[0]; 	/* inode summary info */
+};
+
 union jffs2_node_union {
 	struct jffs2_raw_inode i;
 	struct jffs2_raw_dirent d;
+	struct jffs2_raw_summary s;
 	struct jffs2_unknown_node u;
 } __attribute__((packed));
 
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: fix searching for latest version in jffs2_1pass_list_inodes()

[Ilya Yanok]

We need to update i_version inside cycle to find really latest version
inside jffs2_1pass_list_inodes(). With that fixed we can use isize inside
dump_inode() instead of calling expensive jffs2_1pass_read_inode().

Signed-off-by: Alexey Neyman <avn@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 fs/jffs2/jffs2_1pass.c |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
index 58298c0..94f8468 100644
--- a/fs/jffs2/jffs2_1pass.c
+++ b/fs/jffs2/jffs2_1pass.c
@@ -937,9 +937,7 @@ static inline u32 dump_inode(struct b_lists * pL, struct jffs2_raw_dirent *d, st
 	st.st_mtime = i->mtime;
 	st.st_mode = i->mode;
 	st.st_ino = i->ino;
-
-	/* neither dsize nor isize help us.. do it the long way */
-	st.st_size = jffs2_1pass_read_inode(pL, i->ino, NULL);
+	st.st_size = i->isize;
 
 	dump_stat(&st, fname);
 
@@ -973,6 +971,7 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino)
 				jNode = (struct jffs2_raw_inode *)
 					get_fl_mem(b2->offset, sizeof(ojNode), &ojNode);
 				if (jNode->ino == jDir->ino && jNode->version >= i_version) {
+					i_version = jNode->version;
 					if (i)
 						put_fl_mem(i);
 
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: add buffer to cache flash accesses

[Ilya Yanok]

With this patch JFFS2 code allocates memory buffer of max_totlen size
(size of the largest node, calculated during scan time) and uses it to
store entire node. Speeds up loading. If malloc fails we use old ways
to do things.

Signed-off-by: Alexey Neyman <avn@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 fs/jffs2/jffs2_1pass.c   |  121 ++++++++++++++++++++++++++++-----------------
 fs/jffs2/jffs2_private.h |    2 +-
 2 files changed, 76 insertions(+), 47 deletions(-)

diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
index 94f8468..9d9ac1a 100644
--- a/fs/jffs2/jffs2_1pass.c
+++ b/fs/jffs2/jffs2_1pass.c
@@ -247,7 +247,7 @@ static void *get_fl_mem_nand(u32 off, u32 size, void *ext_buf)
 	return buf;
 }
 
-static void *get_node_mem_nand(u32 off)
+static void *get_node_mem_nand(u32 off, void *ext_buf)
 {
 	struct jffs2_unknown_node node;
 	void *ret = NULL;
@@ -257,7 +257,7 @@ static void *get_node_mem_nand(u32 off)
 
 	if (!(ret = get_fl_mem_nand(off, node.magic ==
 			       JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node),
-			       NULL))) {
+			       ext_buf))) {
 		printf("off = %#x magic %#x type %#x node.totlen = %d\n",
 		       off, node.magic, node.nodetype, node.totlen);
 	}
@@ -346,7 +346,7 @@ static void *get_fl_mem_onenand(u32 off, u32 size, void *ext_buf)
 	return buf;
 }
 
-static void *get_node_mem_onenand(u32 off)
+static void *get_node_mem_onenand(u32 off, void *ext_buf)
 {
 	struct jffs2_unknown_node node;
 	void *ret = NULL;
@@ -356,7 +356,7 @@ static void *get_node_mem_onenand(u32 off)
 
 	ret = get_fl_mem_onenand(off, node.magic ==
 			JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node),
-			NULL);
+			ext_buf);
 	if (!ret) {
 		printf("off = %#x magic %#x type %#x node.totlen = %d\n",
 		       off, node.magic, node.nodetype, node.totlen);
@@ -379,7 +379,7 @@ static void put_fl_mem_onenand(void *buf)
  * NOR flash memory is mapped in processor's address space,
  * just return address.
  */
-static inline void *get_fl_mem_nor(u32 off)
+static inline void *get_fl_mem_nor(u32 off, u32 size, void *ext_buf)
 {
 	u32 addr = off;
 	struct mtdids *id = current_part->dev->id;
@@ -388,18 +388,22 @@ static inline void *get_fl_mem_nor(u32 off)
 	flash_info_t *flash = &flash_info[id->num];
 
 	addr += flash->start[0];
+	if (ext_buf) {
+		memcpy(ext_buf, (void *)addr, size);
+		return ext_buf;
+	}
 	return (void*)addr;
 }
 
-static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf)
+static inline void *get_node_mem_nor(u32 off, void *ext_buf)
 {
-	memcpy(ext_buf, get_fl_mem_nor(off), size);
-	return ext_buf;
-}
+	struct jffs2_unknown_node *pNode;
 
-static inline void *get_node_mem_nor(u32 off)
-{
-	return (void*)get_fl_mem_nor(off);
+	/* pNode will point directly to flash - don't provide external buffer
+	   and don't care about size */
+	pNode = get_fl_mem_nor(off, 0, NULL);
+	return (void *)get_fl_mem_nor(off, pNode->magic == JFFS2_MAGIC_BITMASK ?
+			pNode->totlen : sizeof(*pNode), ext_buf);
 }
 #endif
 
@@ -414,9 +418,7 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf)
 
 #if defined(CONFIG_CMD_FLASH)
 	if (id->type == MTD_DEV_TYPE_NOR) {
-		if (ext_buf)
-			return get_fl_mem_nor_copy(off, size, ext_buf);
-		return get_fl_mem_nor(off);
+		return get_fl_mem_nor(off, size, ext_buf);
 	}
 #endif
 
@@ -434,44 +436,48 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf)
 	return (void*)off;
 }
 
-static inline void *get_node_mem(u32 off)
+static inline void *get_node_mem(u32 off, void *ext_buf)
 {
 	struct mtdids *id = current_part->dev->id;
 
 #if defined(CONFIG_CMD_FLASH)
 	if (id->type == MTD_DEV_TYPE_NOR)
-		return get_node_mem_nor(off);
+		return get_node_mem_nor(off, ext_buf);
 #endif
 
 #if defined(CONFIG_JFFS2_NAND) && \
     defined(CONFIG_CMD_NAND)
 	if (id->type == MTD_DEV_TYPE_NAND)
-		return get_node_mem_nand(off);
+		return get_node_mem_nand(off, ext_buf);
 #endif
 
 #if defined(CONFIG_CMD_ONENAND)
 	if (id->type == MTD_DEV_TYPE_ONENAND)
-		return get_node_mem_onenand(off);
+		return get_node_mem_onenand(off, ext_buf);
 #endif
 
 	printf("get_node_mem: unknown device type, using raw offset!\n");
 	return (void*)off;
 }
 
-static inline void put_fl_mem(void *buf)
+static inline void put_fl_mem(void *buf, void *ext_buf)
 {
 #if defined(CONFIG_JFFS2_NAND) && \
     defined(CONFIG_CMD_NAND)
 	struct mtdids *id = current_part->dev->id;
 
-	if (id->type == MTD_DEV_TYPE_NAND)
+	/* If buf is the same as ext_buf, it was provided by the caller -
+	   we shouldn't free it then. */
+	if (id->type == MTD_DEV_TYPE_NAND && buf != ext_buf)
 		return put_fl_mem_nand(buf);
 #endif
 
 #if defined(CONFIG_CMD_ONENAND)
 	struct mtdids *id = current_part->dev->id;
 
-	if (id->type == MTD_DEV_TYPE_ONENAND)
+	/* If buf is the same as ext_buf, it was provided by the caller -
+	   we shouldn't free it then. */
+	if (id->type == MTD_DEV_TYPE_ONENAND && buf != ext_buf)
 		return put_fl_mem_onenand(buf);
 #endif
 }
@@ -670,6 +676,7 @@ jffs2_free_cache(struct part_info *part)
 		pL = (struct b_lists *)part->jffs2_priv;
 		free_nodes(&pL->frag);
 		free_nodes(&pL->dir);
+		free(pL->readbuf);
 		free(pL);
 	}
 }
@@ -716,7 +723,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 	 */
 	for (b = pL->frag.listHead; b != NULL; b = b->next) {
 		jNode = (struct jffs2_raw_inode *) get_fl_mem(b->offset,
-		        sizeof(struct jffs2_raw_inode), NULL);
+			sizeof(struct jffs2_raw_inode), pL->readbuf);
 		if ((inode == jNode->ino)) {
 			/* get actual file length from the newest node */
 			if (jNode->version >= latestVersion) {
@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 				latestVersion = jNode->version;
 			}
 		}
-		put_fl_mem(jNode);
+		if (pL->readbuf == NULL)
+			put_fl_mem(jNode);
 	}
 #endif
 
 	for (b = pL->frag.listHead; b != NULL; b = b->next) {
-		jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset);
+		jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset,
+								pL->readbuf);
 		if ((inode == jNode->ino)) {
 #if 0
 			putLabeledWord("\r\n\r\nread_inode: totlen = ", jNode->totlen);
@@ -756,11 +765,11 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 				src = ((uchar *) jNode) + sizeof(struct jffs2_raw_inode);
 				/* ignore data behind latest known EOF */
 				if (jNode->offset > totalSize) {
-					put_fl_mem(jNode);
+					put_fl_mem(jNode, pL->readbuf);
 					continue;
 				}
 				if (!data_crc(jNode)) {
-					put_fl_mem(jNode);
+					put_fl_mem(jNode, pL->readbuf);
 					continue;
 				}
 
@@ -801,7 +810,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 				default:
 					/* unknown */
 					putLabeledWord("UNKOWN COMPRESSION METHOD = ", jNode->compr);
-					put_fl_mem(jNode);
+					put_fl_mem(jNode, pL->readbuf);
 					return -1;
 					break;
 				}
@@ -813,7 +822,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 #endif
 		}
 		counter++;
-		put_fl_mem(jNode);
+		put_fl_mem(jNode, pL->readbuf);
 	}
 
 #if 0
@@ -839,12 +848,13 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino)
 	counter = 0;
 	/* we need to search all and return the inode with the highest version */
 	for(b = pL->dir.listHead; b; b = b->next, counter++) {
-		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset);
+		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset,
+								pL->readbuf);
 		if ((pino == jDir->pino) && (len == jDir->nsize) &&
 		    (jDir->ino) &&	/* 0 for unlink */
 		    (!strncmp((char *)jDir->name, name, len))) {	/* a match */
 			if (jDir->version < version) {
-				put_fl_mem(jDir);
+				put_fl_mem(jDir, pL->readbuf);
 				continue;
 			}
 
@@ -866,7 +876,7 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino)
 		putLabeledWord("b = ", (u32) b);
 		putLabeledWord("counter = ", counter);
 #endif
-		put_fl_mem(jDir);
+		put_fl_mem(jDir, pL->readbuf);
 	}
 	return inode;
 }
@@ -960,7 +970,8 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino)
 	struct jffs2_raw_dirent *jDir;
 
 	for (b = pL->dir.listHead; b; b = b->next) {
-		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset);
+		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset,
+								pL->readbuf);
 		if ((pino == jDir->pino) && (jDir->ino)) { /* ino=0 -> unlink */
 			u32 i_version = 0;
 			struct jffs2_raw_inode ojNode;
@@ -973,20 +984,23 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino)
 				if (jNode->ino == jDir->ino && jNode->version >= i_version) {
 					i_version = jNode->version;
 					if (i)
-						put_fl_mem(i);
+						put_fl_mem(i, NULL);
 
 					if (jDir->type == DT_LNK)
-						i = get_node_mem(b2->offset);
+						i = get_node_mem(b2->offset,
+								 NULL);
 					else
-						i = get_fl_mem(b2->offset, sizeof(*i), NULL);
+						i = get_fl_mem(b2->offset,
+							       sizeof(*i),
+							       NULL);
 				}
 				b2 = b2->next;
 			}
 
 			dump_inode(pL, jDir, i);
-			put_fl_mem(i);
+			put_fl_mem(i, NULL);
 		}
-		put_fl_mem(jDir);
+		put_fl_mem(jDir, pL->readbuf);
 	}
 	return pino;
 }
@@ -1064,10 +1078,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
 
 	/* we need to search all and return the inode with the highest version */
 	for(b = pL->dir.listHead; b; b = b->next) {
-		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset);
+		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset,
+								pL->readbuf);
 		if (ino == jDir->ino) {
 			if (jDir->version < version) {
-				put_fl_mem(jDir);
+				put_fl_mem(jDir, pL->readbuf);
 				continue;
 			}
 
@@ -1084,7 +1099,7 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
 			jDirFoundPino = jDir->pino;
 			version = jDir->version;
 		}
-		put_fl_mem(jDir);
+		put_fl_mem(jDir, pL->readbuf);
 	}
 	/* now we found the right entry again. (shoulda returned inode*) */
 	if (jDirFoundType != DT_LNK)
@@ -1093,7 +1108,8 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
 	/* it's a soft link so we follow it again. */
 	b2 = pL->frag.listHead;
 	while (b2) {
-		jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset);
+		jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset,
+								pL->readbuf);
 		if (jNode->ino == jDirFoundIno) {
 			src = (unsigned char *)jNode + sizeof(struct jffs2_raw_inode);
 
@@ -1105,11 +1121,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
 #endif
 			strncpy(tmp, (char *)src, jNode->dsize);
 			tmp[jNode->dsize] = '\0';
-			put_fl_mem(jNode);
+			put_fl_mem(jNode, pL->readbuf);
 			break;
 		}
 		b2 = b2->next;
-		put_fl_mem(jNode);
+		put_fl_mem(jNode, pL->readbuf);
 	}
 	/* ok so the name of the new file to find is in tmp */
 	/* if it starts with a slash it is root based else shared dirs */
@@ -1370,7 +1386,8 @@ dump_dirents(struct b_lists *pL)
 	putstr("\r\n\r\n******The directory Entries******\r\n");
 	b = pL->dir.listHead;
 	while (b) {
-		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset);
+		jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset,
+								pL->readbuf);
 		putstr("\r\n");
 		putnstr(jDir->name, jDir->nsize);
 		putLabeledWord("\r\n\tbuild_list: magic = ", jDir->magic);
@@ -1386,7 +1403,7 @@ dump_dirents(struct b_lists *pL)
 		putLabeledWord("\tbuild_list: name_crc = ", jDir->name_crc);
 		putLabeledWord("\tbuild_list: offset = ", b->offset);	/* FIXME: ? [RS] */
 		b = b->next;
-		put_fl_mem(jDir);
+		put_fl_mem(jDir, pL->readbuf);
 	}
 }
 #endif
@@ -1416,6 +1433,7 @@ jffs2_1pass_build_lists(struct part_info * part)
 	u32 counter4 = 0;
 	u32 counterF = 0;
 	u32 counterN = 0;
+	u32 max_totlen = 0;
 	u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE;
 	char *buf;
 
@@ -1600,6 +1618,8 @@ jffs2_1pass_build_lists(struct part_info * part)
 				if (insert_node(&pL->frag, (u32) part->offset +
 						ofs) == NULL)
 					return 0;
+				if (max_totlen < node->totlen)
+					max_totlen = node->totlen;
 				break;
 			case JFFS2_NODETYPE_DIRENT:
 				if (buf_ofs + buf_len < ofs + sizeof(struct
@@ -1625,6 +1645,8 @@ jffs2_1pass_build_lists(struct part_info * part)
 				if (insert_node(&pL->dir, (u32) part->offset +
 						ofs) == NULL)
 					return 0;
+				if (max_totlen < node->totlen)
+					max_totlen = node->totlen;
 				counterN++;
 				break;
 			case JFFS2_NODETYPE_CLEANMARKER:
@@ -1655,6 +1677,13 @@ jffs2_1pass_build_lists(struct part_info * part)
 
 	free(buf);
 	putstr("\b\b done.\r\n");		/* close off the dots */
+
+	/* We don't care if malloc failed - then each read operation will
+	 * allocate its own buffer as necessary (NAND) or will read directly
+	 * from flash (NOR).
+	 */
+	pL->readbuf = malloc(max_totlen);
+
 	/* turn the lcd back on. */
 	/* splash(); */
 
diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h
index 46ed644..b3fab1c 100644
--- a/fs/jffs2/jffs2_private.h
+++ b/fs/jffs2/jffs2_private.h
@@ -24,7 +24,7 @@ struct b_list {
 struct b_lists {
 	struct b_list dir;
 	struct b_list frag;
-
+	void *readbuf;
 };
 
 struct b_compr_info {
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: cache data_crc results

[Ilya Yanok]

As we moved data_crc() invocation from jffs2_1pass_build_lists() to
jffs2_1pass_read_inode() data_crc is going to be calculated on each
inode access. This patch adds caching of data_crc() results. There
is no significant improvement in speed (because of flash access
caching added in previous patch I think, crc in RAM is really fast)
but this patch impacts memory usage -- every b_node structure uses
12 bytes instead of 8.

Signed-off-by: Alexey Neyman <avn@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
---
 fs/jffs2/jffs2_1pass.c   |    5 ++++-
 fs/jffs2/jffs2_private.h |    1 +
 2 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
index 9d9ac1a..b41c299 100644
--- a/fs/jffs2/jffs2_1pass.c
+++ b/fs/jffs2/jffs2_1pass.c
@@ -768,7 +768,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
 					put_fl_mem(jNode, pL->readbuf);
 					continue;
 				}
-				if (!data_crc(jNode)) {
+				if (b->datacrc == CRC_UNKNOWN)
+					b->datacrc = data_crc(jNode) ?
+						CRC_OK : CRC_BAD;
+				if (b->datacrc == CRC_BAD) {
 					put_fl_mem(jNode, pL->readbuf);
 					continue;
 				}
diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h
index b3fab1c..7a9eda6 100644
--- a/fs/jffs2/jffs2_private.h
+++ b/fs/jffs2/jffs2_private.h
@@ -7,6 +7,7 @@
 struct b_node {
 	u32 offset;
 	struct b_node *next;
+	enum { CRC_UNKNOWN = 0, CRC_OK, CRC_BAD } datacrc;
 };
 
 struct b_list {
-- 
1.5.6.1

[U-Boot] [PATCH] jffs2: cache data_crc results

[Michael Lawnick]

Ilya Yanok said the following:
> As we moved data_crc() invocation from jffs2_1pass_build_lists() to
> jffs2_1pass_read_inode() data_crc is going to be calculated on each
> inode access. This patch adds caching of data_crc() results. There
> is no significant improvement in speed (because of flash access
> caching added in previous patch I think, crc in RAM is really fast)
> but this patch impacts memory usage -- every b_node structure uses
> 12 bytes instead of 8.
> 
> Signed-off-by: Alexey Neyman <avn@emcraft.com>
> Signed-off-by: Ilya Yanok <yanok@emcraft.com>
> ---
>  fs/jffs2/jffs2_1pass.c   |    5 ++++-
>  fs/jffs2/jffs2_private.h |    1 +
>  2 files changed, 5 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c
> index 9d9ac1a..b41c299 100644
> --- a/fs/jffs2/jffs2_1pass.c
> +++ b/fs/jffs2/jffs2_1pass.c
> @@ -768,7 +768,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest)
>  					put_fl_mem(jNode, pL->readbuf);
>  					continue;
>  				}
> -				if (!data_crc(jNode)) {
> +				if (b->datacrc == CRC_UNKNOWN)
> +					b->datacrc = data_crc(jNode) ?
> +						CRC_OK : CRC_BAD;
> +				if (b->datacrc == CRC_BAD) {
>  					put_fl_mem(jNode, pL->readbuf);
>  					continue;
>  				}
> diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h
> index b3fab1c..7a9eda6 100644
> --- a/fs/jffs2/jffs2_private.h
> +++ b/fs/jffs2/jffs2_private.h
> @@ -7,6 +7,7 @@
>  struct b_node {
>  	u32 offset;
>  	struct b_node *next;
> +	enum { CRC_UNKNOWN = 0, CRC_OK, CRC_BAD } datacrc;
>  };
>  
>  struct b_list {

Could be so kind to give us your source reference?
I can't find above code in u-boot 1.3.3 nor 1.3.4

The same seems to apply to the other patches you've sent yesterday :-(

I'm eagerly waiting for summary patch ;-)

Regards,
Michael

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Wolfgang Denk]

Dear Ilya,

In message <1223877234-720-1-git-send-email-yanok@emcraft.com> you wrote:
> 
> here is a set of changes we made to improve U-Boot JFFS2 code
> performance. We still can't reach Linux's performance but improvements
> are significant.
> 
> Any comments are welcome.

Are these patches independent of each  other,  or  are  all  of  them
needed,  and if so, is there any specific order in which they have to
be applied?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
You humans have that emotional need  to  express  gratitude.  "You're
welcome," I believe, is the correct response.
	-- Spock, "Bread and Circuses", stardate 4041.2

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Ilya Yanok]

Hi Wolfgang,

Wolfgang Denk wrote:
>> here is a set of changes we made to improve U-Boot JFFS2 code
>> performance. We still can't reach Linux's performance but improvements
>> are significant.
>>
>> Any comments are welcome.
>>     
>
> Are these patches independent of each  other,  or  are  all  of  them
> needed,  and if so, is there any specific order in which they have to
> be applied?
>   
Argh... I forgot to create patches with xxx/nnn notation...
Here is the order of patches:
0001-jffs2-add-sector_size-field-to-part_info-structure.patch
0002-jffs2-rewrite-jffs2-scanning-code-based-on-Linux-on.patch
0003-jffs2-summary-support.patch
0004-jffs2-fix-searching-for-latest-version-in-jffs2_1pa.patch
0005-jffs2-add-buffer-to-cache-flash-accesses.patch
0006-jffs2-cache-data_crc-results.patch

It's the order of changes in my tree and thus it's the simplest way to
apply this patches. But actual dependencies are as follows:
0001 and 0004 don't depend on anything
0002 depends on 0001
0003, 0005 and 0006 depend on 0002

Regards, Ilya.

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Michael Lawnick]

Ilya Yanok said the following:
> Hi Wolfgang,
> 
> Wolfgang Denk wrote:
>>> here is a set of changes we made to improve U-Boot JFFS2 code
>>> performance. We still can't reach Linux's performance but improvements
>>> are significant.
>>>
>>> Any comments are welcome.
>>>     
>>
>> Are these patches independent of each  other,  or  are  all  of  them
>> needed,  and if so, is there any specific order in which they have to
>> be applied?
>>   
> Argh... I forgot to create patches with xxx/nnn notation...
> Here is the order of patches:
> 0001-jffs2-add-sector_size-field-to-part_info-structure.patch
> 0002-jffs2-rewrite-jffs2-scanning-code-based-on-Linux-on.patch
> 0003-jffs2-summary-support.patch
> 0004-jffs2-fix-searching-for-latest-version-in-jffs2_1pa.patch
> 0005-jffs2-add-buffer-to-cache-flash-accesses.patch
> 0006-jffs2-cache-data_crc-results.patch
> 
> It's the order of changes in my tree and thus it's the simplest way to
> apply this patches. But actual dependencies are as follows:
> 0001 and 0004 don't depend on anything
> 0002 depends on 0001
> 0003, 0005 and 0006 depend on 0002
> 
> Regards, Ilya.

I have put these patches to our board (MPC8548, NOR Flash, MTD partition
size 48MB).

a) performance has improved (execution time of 'ls' reduced from ~16s to
~1.5s)

b) there is a bug in the patch
0005-jffs2-add-buffer-to-cache-flash-accesses:
The passage

@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32
inode, char *dest)
 				latestVersion = jNode->version;
 			}
 		}
-		put_fl_mem(jNode);
+		if (pL->readbuf == NULL)
+			put_fl_mem(jNode);
 	}
 #endif

should be

@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32
inode, char *dest)
 				latestVersion = jNode->version;
 			}
 		}
-		put_fl_mem(jNode);
+		if (pL->readbuf == NULL)
+			put_fl_mem(jNode, NULL);
 	}
 #endif

c) I found no improvement with CFG_JFFS2_SORT_FRAGMENTS set (the way I
detected b) ;-)

d) output of 'ls' is corrupted: no date and access right output,
directories in / are shown multiple times.


-- 

Michael Lawnick
Software Design Engineer

Lise-Meitner-Str. 7/1
89081 Ulm
Tel: +49 731 9533 2066

Michael.Lawnick.ext at nsn.com
http://www.nokiasiemensnetworks.com/global/

Think before you print

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Ilya Yanok]

Hello Michael,

Michael Lawnick wrote:
> I have put these patches to our board (MPC8548, NOR Flash, MTD partition
> size 48MB).
>
> a) performance has improved (execution time of 'ls' reduced from ~16s to
> ~1.5s)
>   

Sounds good. That is what we were expecting.

> b) there is a bug in the patch
> 0005-jffs2-add-buffer-to-cache-flash-accesses:
> The passage
>
> @@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32
> inode, char *dest)
>  				latestVersion = jNode->version;
>  			}
>  		}
> -		put_fl_mem(jNode);
> +		if (pL->readbuf == NULL)
> +			put_fl_mem(jNode);
>  	}
>  #endif
>
> should be
>
> @@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32
> inode, char *dest)
>  				latestVersion = jNode->version;
>  			}
>  		}
> -		put_fl_mem(jNode);
> +		if (pL->readbuf == NULL)
> +			put_fl_mem(jNode, NULL);
>  	}
>  #endif
>   

You are right. We haven't just tried to use CFG_JFFS2_SORT_FRAGMENTS...

> c) I found no improvement with CFG_JFFS2_SORT_FRAGMENTS set (the way I
> detected b) ;-)
>   

Same as above... We don't use CFG_JFFS2_SORT_FRAGMENTS... But it's very
sad result really... I have no idea what is wrong with it...

> d) output of 'ls' is corrupted: no date and access right output,
> directories in / are shown multiple times

Can you please be more specific on this? Is this with
CFG_JFFS2_SORT_FRAGMENTS enabled or not? How did you create your FS? I
can't reproduce any of described symptoms on our system...

The only thing I can tell you for now is that the first patch from my
posting is wrong, the following patch fixes the problem:

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Wolfgang Denk]

Dear Ilya,

In message <490BAA2A.6020806@emcraft.com> you wrote:
> Hello Michael,
> 
> Michael Lawnick wrote:
> > I have put these patches to our board (MPC8548, NOR Flash, MTD partition
> > size 48MB).
> >
> > a) performance has improved (execution time of 'ls' reduced from ~16s to
> > ~1.5s)
> >   
> 
> Sounds good. That is what we were expecting.

If you submit a patch, then please add such comments *below* the "---"
line.

> diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c
> index e7f07bf..2fc0b7d 100644
> --- a/common/cmd_jffs2.c
> +++ b/common/cmd_jffs2.c
> @@ -344,7 +344,9 @@ static int part_validate_nor(struct mtdids *id,
> struct part_info *part)
>  
>      flash = &flash_info[id->num];
>  
> -    part->sector_size = 0;
> +    /* size of last sector */
> +    part->sector_size = flash->size -
> +        (flash->start[flash->sector_count-1] - flash->start[0]);
>  
>      offset_aligned = 0;
>      for (i = 0; i < flash->sector_count; i++) {
> @@ -360,6 +362,7 @@ static int part_validate_nor(struct mtdids *id,
> struct part_info *part)
^^^^^^^^^^^^^^^^^^^^^^^^^^

Patch is correcpted because your mailer wrapped long lines.

Please fix your mailer setup and resubmit.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Cleverness and Style have no place in getting a project completed.
                                                  -- Tom Christiansen

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Ilya Yanok]

Hello Wolfgang,

Wolfgang Denk wrote:
>> Sounds good. That is what we were expecting.
>>     
>
> If you submit a patch, then please add such comments *below* the "---"
> line.
>   

That wasn't a patch submission actually... I've just wanted Michael to
test if this patch helps with his problem. I'll post the full updated
patch series soon.

>> diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c
>> index e7f07bf..2fc0b7d 100644
>> --- a/common/cmd_jffs2.c
>> +++ b/common/cmd_jffs2.c
>> @@ -344,7 +344,9 @@ static int part_validate_nor(struct mtdids *id,
>> struct part_info *part)
>>  
>>      flash = &flash_info[id->num];
>>  
>> -    part->sector_size = 0;
>> +    /* size of last sector */
>> +    part->sector_size = flash->size -
>> +        (flash->start[flash->sector_count-1] - flash->start[0]);
>>  
>>      offset_aligned = 0;
>>      for (i = 0; i < flash->sector_count; i++) {
>> @@ -360,6 +362,7 @@ static int part_validate_nor(struct mtdids *id,
>> struct part_info *part)
>>     
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Patch is correcpted because your mailer wrapped long lines.
>
> Please fix your mailer setup and resubmit.
>   
That's my fault... Usually I send patches with git-send-email... As I
said before I'll post correct patches in a short time.

Regards, Ilya.

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Michael Lawnick]

Ilya Yanok said the following:
> Hello Michael,
>
> Michael Lawnick wrote:
...
>> d) output of 'ls' is corrupted: no date and access right output,
>> directories in / are shown multiple times
>
> Can you please be more specific on this? Is this with
> CFG_JFFS2_SORT_FRAGMENTS enabled or not? How did you create your FS? I
> can't reproduce any of described symptoms on our system...
>
> The only thing I can tell you for now is that the first patch from my
> posting is wrong, the following patch fixes the problem:
>
...
I added your patch, it corrected date part of 'ls'.
For both versions (CFG_JFFS2_SORT_FRAGMENTS set or not set)
- device root based directories are printed out up to 9 times
- output of access rights is still corrupted, e.g.:
 d????-??-?        0 Tue Oct 21 11:08:17 2008 10
 -????-??-?   262144 Thu Jan 01 00:00:33 1970 u-boot.bin
 d????-??-?        0 Tue Oct 21 11:08:32 2008 1
 d????-??-?        0 Tue Oct 21 11:00:58 2008 2


The difference is:
*with*    CFG_JFFS2_SORT_FRAGMENTS time for ls is ~ 16s
*without* CFG_JFFS2_SORT_FRAGMENTS time for ls is ~ 2s

Setup scenario:
U-boot flashed, Linux booted and jffs mounted on erased flash --> in
background-process flash gets formatted.
create dir '10' in root of flash, copy 2 files with ~1.2MB each into it.
Run following script ~300 times:
 mkdir /flash/11
 cp /flash/10/* /flash/11
 rm -rf /flash/1
 mv /flash/2 /flash/1
 mv /flash/3 /flash/2
 mv /flash/4 /flash/3
 mv /flash/5 /flash/4
 mv /flash/6 /flash/5
 mv /flash/7 /flash/6
 mv /flash/8 /flash/7
 mv /flash/9 /flash/8
 mv /flash/10 /flash/9
 mv /flash/11 /flash/10
 sync

This should mix up jffs image enough to see what I observe.

-- 

Michael Lawnick
Software Design Engineer

Lise-Meitner-Str. 7/1
89081 Ulm
Tel: +49 731 9533 2066

Michael.Lawnick.ext at nsn.com
http://www.nokiasiemensnetworks.com/global/

Think before you print

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Michael Lawnick]

Michael Lawnick said the following:
...
> - output of access rights is still corrupted, e.g.:
>  d????-??-?        0 Tue Oct 21 11:08:17 2008 10
>  -????-??-?   262144 Thu Jan 01 00:00:33 1970 u-boot.bin
>  d????-??-?        0 Tue Oct 21 11:08:32 2008 1
>  d????-??-?        0 Tue Oct 21 11:00:58 2008 2
...

Ok, fixed this in my code:
in jffs2_1pass.c @@ mkmodestr(unsigned long mode, char *str)
you'll find declaration
    static const char *l = "xwr";

Although this is per se correct, it is not needed in context and screws
up 'ls' in my case: Our u-boot is not placed in flash were it is located
to on compilation time. static const char * points into flash space at
some not properly initialized flash space. After modifying to
    char *l = "xwr";
output of access rights is fine now:
 drwxr-xr-x        0 Tue Oct 21 11:08:17 2008 10
 -rwxr-xr-x   262144 Thu Jan 01 00:00:33 1970 u-boot.bin
 drwxr-xr-x        0 Tue Oct 21 11:08:32 2008 1
 drwxr-xr-x        0 Tue Oct 21 11:00:58 2008 2

(Still directories are listed multiple times)

Could we modify this declaration to my non-static version in the trunk?

-- 

Michael Lawnick
Software Design Engineer

Lise-Meitner-Str. 7/1
89081 Ulm
Tel: +49 731 9533 2066

Michael.Lawnick.ext at nsn.com
http://www.nokiasiemensnetworks.com/global/

Think before you print

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Joakim Tjernlund]

On Tue, 2008-11-04 at 13:31 +0100, Michael Lawnick wrote:
> Michael Lawnick said the following:
> ...
> > - output of access rights is still corrupted, e.g.:
> >  d????-??-?        0 Tue Oct 21 11:08:17 2008 10
> >  -????-??-?   262144 Thu Jan 01 00:00:33 1970 u-boot.bin
> >  d????-??-?        0 Tue Oct 21 11:08:32 2008 1
> >  d????-??-?        0 Tue Oct 21 11:00:58 2008 2
> ...
> 
> Ok, fixed this in my code:
> in jffs2_1pass.c @@ mkmodestr(unsigned long mode, char *str)
> you'll find declaration
>     static const char *l = "xwr";
> 
> Although this is per se correct, it is not needed in context and screws
> up 'ls' in my case: Our u-boot is not placed in flash were it is located
> to on compilation time. static const char * points into flash space at
> some not properly initialized flash space. After modifying to
>     char *l = "xwr";
> output of access rights is fine now:
>  drwxr-xr-x        0 Tue Oct 21 11:08:17 2008 10
>  -rwxr-xr-x   262144 Thu Jan 01 00:00:33 1970 u-boot.bin
>  drwxr-xr-x        0 Tue Oct 21 11:08:32 2008 1
>  drwxr-xr-x        0 Tue Oct 21 11:00:58 2008 2
> 
> (Still directories are listed multiple times)
> 
> Could we modify this declaration to my non-static version in the trunk?

Is this on PPC? If so it is due to missing fixup support. Grant Likely
once sent a patch fixing this. It got applied, but later reverted
because there was some problems that never got solved.

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Michael Lawnick]

Joakim Tjernlund said the following:
> On Tue, 2008-11-04 at 13:31 +0100, Michael Lawnick wrote:
>> Could we modify this declaration to my non-static version in the trunk?
> 
> Is this on PPC? If so it is due to missing fixup support. Grant Likely
> once sent a patch fixing this. It got applied, but later reverted
> because there was some problems that never got solved.

It is on PPC. IMHO such parts should be removed at least step by step,
where it is proven that the dynamic version works. I can't see any
benefit in the static solution here.
-- 

Michael Lawnick
Software Design Engineer

Lise-Meitner-Str. 7/1
89081 Ulm
Tel: +49 731 9533 2066

Michael.Lawnick.ext at nsn.com
http://www.nokiasiemensnetworks.com/global/

Think before you print

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Alexey Neyman]

Michael,

I tried to reproduce the bug you mentioned (multiple appearance of 
directories in ls output) using the script you provided, to no avail: 
ls correctly showed the top-level directory, both with and without 
CFG_JFFS2_SORT_FRAGMENTS option.

I am inclined to suspect the same cause as you mentioned below:

> Our u-boot is not placed in flash were it is located to on compilation
> time. static const char * points into flash space at some not properly
> initialized flash space.

This would affect all global and static variables, with unpredictable 
results. Could you try to reproduce this bug with u-boot placed at the 
address it was linked to?

As a side note, your fix for compilation with CFG_JFFS2_SORT_FRAGMENTS 
option turned on would be better as follows: replace

  if (pL->readbuf == NULL)
     put_fl_mem(jNode);

with

  put_fl_mem(jNode, pL->readbuf);

(put_fl_mem already performs this check)


Regards,
Alexey.

[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code

[Joakim Tjernlund]

On Mon, 2008-11-10 at 22:49 -0800, Alexey Neyman wrote:
> Michael,
> 
> I tried to reproduce the bug you mentioned (multiple appearance of 
> directories in ls output) using the script you provided, to no avail: 
> ls correctly showed the top-level directory, both with and without 
> CFG_JFFS2_SORT_FRAGMENTS option.
> 
> I am inclined to suspect the same cause as you mentioned below:
> 
> > Our u-boot is not placed in flash were it is located to on compilation
> > time. static const char * points into flash space at some not properly
> > initialized flash space.
> 
> This would affect all global and static variables, with unpredictable 
> results. Could you try to reproduce this bug with u-boot placed at the 
> address it was linked to?

You can't do that. What you need is to enable fixup support. Try undoing
this commit 1c3dd43338a077165e7e0309cb3994e65d2bdbf8