Large JFFS2 filesystem problem

Large JFFS2 filesystem problem

[Andy Hawkins]

Hi all.

A while back I posted a message about some problems we were having
mounting a large JFFS2 filesystem on a NAND device. We want to treat up
to 16 128MB devices as a single device, and have a filesystem on that
device (i.e a total of 2 Gig)

However, I've found that if I try to use more than two of these devices,
then I can no longer mount a JFFS2 filesystem on the device.

After some debugging, it appears that the kmalloc in jffs2_scan_medium
fails, trying to allocate 256K of memory. Elsewhere, there is a comment
that the maximum that kmalloc can return is 128K, so this is obviously
incorrect.

Is this a limitation in the size of a JFFS2 filesystem, or is this an
error? It seems to me that this code:

/* 
 * Check, if we have to concatenate physical blocks to larger virtual
blocks
 * to reduce the memorysize for c->blocks. (kmalloc allows max. 128K
allocation)
 */
c->sector_size = c->mtd->erasesize; 
blocks = c->flash_size / c->sector_size;
while ((blocks * sizeof (struct jffs2_eraseblock)) > (128 * 1024)) {
	blocks >>= 1;
	c->sector_size <<= 1;
}	

could be the source of the proble, because while it decreases the number
of blocks, it increases the sector size, and it is this sector size that
is used in the kmalloc line that fails.

Previously, someone recommended waiting for YAFFS2, as this is likely to
be more efficient with large filesystems. Does anyone have any idea as
to when this is likely to be available?

Many thanks.

Andy

Re: Large JFFS2 filesystem problem

[Thomas Gleixner]

On Tue, 2004-07-27 at 12:25, Andy Hawkins wrote:

>  * Check, if we have to concatenate physical blocks to larger virtual
> blocks
>  * to reduce the memorysize for c->blocks. (kmalloc allows max. 128K
> allocation)
>  */
> c->sector_size = c->mtd->erasesize; 
> blocks = c->flash_size / c->sector_size;
> while ((blocks * sizeof (struct jffs2_eraseblock)) > (128 * 1024)) {
> 	blocks >>= 1;
> 	c->sector_size <<= 1;
> }	
> 
> could be the source of the proble, because while it decreases the number
> of blocks, it increases the sector size, and it is this sector size that
> is used in the kmalloc line that fails.

Sure, the combination of sector size and number of sectors in relation
to the maximum allocation size of 128K is the limitation.

It should be easy to fix the scan code. All we have to do is to reduce
the chunk size which we read in one go. The functionality to do so is
already there and its simple to make it work

Can you try the following patch ?
Please let me know if it works.

Index: scan.c
===================================================================
RCS file: /home/cvs/mtd/fs/jffs2/scan.c,v
retrieving revision 1.110
diff -u -r1.110 scan.c
--- scan.c      17 Jun 2004 17:15:31 -0000      1.110
+++ scan.c      27 Jul 2004 12:15:52 -0000
@@ -104,6 +104,10 @@
                else
                        buf_size = PAGE_SIZE;
  
+               /* Respect kmalloc limitations */
+               if (buf_size > 128*1024)
+                       buf_size = 128*1024;
+
                D1(printk(KERN_DEBUG "Allocating readbuf of %d bytes\n",
buf_size));
                flashbuf = kmalloc(buf_size, GFP_KERNEL);
                if (!flashbuf)

> Previously, someone recommended waiting for YAFFS2, as this is likely to
> be more efficient with large filesystems. Does anyone have any idea as
> to when this is likely to be available?

Ask on the YAFFS mailing list.

tglx

Re: Large JFFS2 filesystem problem

[Andy Hawkins]

Hi,

On Tue, 2004-07-27 at 13:11, Thomas Gleixner wrote:
> It should be easy to fix the scan code. All we have to do is to reduce
> the chunk size which we read in one go. The functionality to do so is
> already there and its simple to make it work
> 
> Can you try the following patch ?
> Please let me know if it works.

[snip patch]

Apologies...I didn't read the code fully enough to realise that it would
use that block size in a loop to read the entire flash. I changed the
'kmalloc' to a 'vmalloc' and this appeared to work. Your patch also has
the desired effect.

> > Previously, someone recommended waiting for YAFFS2, as this is likely to
> > be more efficient with large filesystems. Does anyone have any idea as
> > to when this is likely to be available?
> 
> Ask on the YAFFS mailing list.

Ok, I'll do this.

Thanks.

Andy

Re: Large JFFS2 filesystem problem

[Thomas Gleixner]

On Tue, 2004-07-27 at 15:44, Andy Hawkins wrote:
> 'kmalloc' to a 'vmalloc' and this appeared to work. Your patch also has
> the desired effect.

Thanks, fixed in CVS

tglx

Re: Large JFFS2 filesystem problem

[Andy Hawkins]

On Tue, 2004-07-27 at 13:11, Thomas Gleixner wrote:
> Ask on the YAFFS mailing list.

I've tried to join this list, and am still waiting for the promised
'confirmation' e-mail.

Is some human intervention required?

Andy