jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Dragan Marinkovic]

[-- Attachment #1: Type: text/plain, Size: 2927 bytes --]

Hi,

I have little endian, MIPS embedded system in which rootfs is
stored on the USB stick (it has its own hardware wear leveling which
can't be turned off). The system is running 2.6.18 kernel. USB stick
has two partitions:

* RO squashfs partition
* RW jffs2 partition (to access the data on this partition, jffs2 and
block2mtd are used, block2mtd driver is the module loaded before jffs2
partition is mounted)

I was testing power loss scenario where power is lost during file update
on the jffs2 partition. After a few test runs I've encountered the
following kernel Oops while attempting to mount jffs2 partition:


CPU 0 Unable to handle kernel paging request at virtual address 0000000c,
epc == 802a4374, ra == 802a8788 Oops[#1]:
Cpu 0
$ 0   : 00000000 80000000 00000000 00008000
$ 4   : 00000000 00000004 00000004 00000000
$ 8   : af229674 878c5e94 38483592 00000000
$12   : 00000000 00000000 0000000a 00000000
$16   : 00000044 00000000 00000004 87814410
$20   : 87814e00 879429c0 865b07f8 865b07f8
$24   : 00000000 2acb8050
$28   : 878c4000 878c5da8 87814e00 802a8788
Hi    : 00000000
Lo    : 00001588
epc   : 802a4374 jffs2_truncate_fragtree+0xf8/0x14c     Not tainted
ra    : 802a8788 jffs2_do_read_inode_internal+0x58c/0xa34
Status: 9004fc03    KERNEL EXL IE
Cause : 00000008
BadVA : 0000000c
PrId  : 00002753
Modules linked in: block2mtd
Process jffs2_gcd_mtd0 (pid: 264, threadinfo=878c4000, task=811e3528)
Stack : 00000044 87814400 00000000 878c5e10 00000044 87814400 00000000 878c5e10
        878c5e50 802a8788 87814e00 87814400 878c5e08 878c5e0c 00000044 878c5e10
        878c5e50 00000000 384834b6 878c5e40 00000000 87814400 87814e00 00000000
        879429c0 00000000 00000044 87814400 87814400 87a24380 87814e00 00000000
        87814e2c 00000000 00000000 00000000 00000000 802a8eac 87de39b0 00000000
        ...
Call Trace:
[<802a4374>] jffs2_truncate_fragtree+0xf8/0x14c
[<802a8788>] jffs2_do_read_inode_internal+0x58c/0xa34
[<802a8eac>] jffs2_do_crccheck_inode+0x68/0x138
[<802acc04>] jffs2_garbage_collect_pass+0x2a4/0xa18
[<802b0184>] jffs2_garbage_collect_thread+0x150/0x17c
[<80104fc4>] kernel_thread_helper+0x10/0x18


Code: 5440fffe  8c840004  00801021 <8c43000c> 1060000c  8fbf0024  8c420014  3042
0fff  14400009

...

Digging a bit deeper, I found that jffs2_truncate_fragtree is not
handling the NULL as a return value from jffs2_lookup_node_frag . It looks
like that NULL return is perfectly legal. The patch to fix this is attached.
I'm not really sure if this is the right place to fix this problem but
it should be OK from the correctness prospective (can't use NULL
pointers).

More to it, I've noticed that if I remove the system power while file
on JFFS2 partition is being updated (it takes a bit of time to hit the
right moment), I'll completely lose the file (it just disappears). Any
ideas?

If anybody is interested I can post more info as well as the offending
jffs2 image.

Dragan

[-- Attachment #2: nodelist.c.patch --]
[-- Type: text/x-patch, Size: 797 bytes --]

--- linux-2.6.18/fs/jffs2/nodelist.c.orig	2006-11-18 18:27:43.530579992 -0800
+++ linux-2.6.18/fs/jffs2/nodelist.c	2006-11-18 18:28:43.089525656 -0800
@@ -57,6 +57,16 @@ void jffs2_add_fd_to_list(struct jffs2_s
 void jffs2_truncate_fragtree(struct jffs2_sb_info *c, struct rb_root *list, uint32_t size)
 {
 	struct jffs2_node_frag *frag = jffs2_lookup_node_frag(list, size);
+/*
+  DM: looks like it's legal for jffs2_lookup_node_frag to return
+  NULL but we were not checking for it (which leads to kernel oops and
+  our ultimate death when mounting jffs2 fs).
+*/
+        if( NULL == frag )
+        {
+            dbg_fragtree("Hm, jffs2_lookup_node_frag returned NULL (empty fragtree)!\n");
+            return;
+        }
 
 	dbg_fragtree("truncating fragtree to 0x%08x bytes\n", size);
 

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Artem Bityutskiy]

On Tue, 2006-11-21 at 23:35 -0800, Dragan Marinkovic wrote:
> Digging a bit deeper, I found that jffs2_truncate_fragtree is not
> handling the NULL as a return value from jffs2_lookup_node_frag . It looks
> like that NULL return is perfectly legal. The patch to fix this is attached.
> I'm not really sure if this is the right place to fix this problem but
> it should be OK from the correctness prospective (can't use NULL
> pointers).

The NULL is legal, but JFFS2 does handle it. Look at the latest git
version. Please, point which particular C operation in the
jffs2_truncate_fragtree() function causes the oops.

> More to it, I've noticed that if I remove the system power while file
> on JFFS2 partition is being updated (it takes a bit of time to hit the
> right moment), I'll completely lose the file (it just disappears). Any
> ideas?

What does "JFFS2 partition is being updated" mean?

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Dragan Marinkovic]

Thanks Artem, I'll take a look.

"JFFS2 partition is being updated" means that the file has been
written to at the moment of the power loss. What's interesting is that
if I'm writing 1MB file to the JFFS2 partition (which gives me enough
time to reliably pull the plug during write operation) the power loss
is always handled properly: the old version of the file is always
there when the power is back on.

To be more precise, I have trouble with the file size of about 11K
(this really isn't any magic number, basically smaller files). I have
/tmp dir. on tmpfs . I'm moving the file there, doing some
manipulation on it (sed, cat, etc.) and copy the file back to its
original location (JFFS2 fs). If I experience the power loss at the
moment of the file copy operation, the file just vanishes. The same is
true if the file is manipulated directly on the JFFS2 partition. I've
tried 'mv', 'cat' and 'cp' -- all with the same result.

The erase block size on the JFFS2 partition is set to 128K (131072
bytes) . I can post USB stick and kernel config specific details if
needed.

Thanks again,
Dragan

On 11/22/06, Artem Bityutskiy <dedekind@infradead.org> wrote:
> On Tue, 2006-11-21 at 23:35 -0800, Dragan Marinkovic wrote:
> > Digging a bit deeper, I found that jffs2_truncate_fragtree is not
> > handling the NULL as a return value from jffs2_lookup_node_frag . It looks
> > like that NULL return is perfectly legal. The patch to fix this is attached.
> > I'm not really sure if this is the right place to fix this problem but
> > it should be OK from the correctness prospective (can't use NULL
> > pointers).
>
> The NULL is legal, but JFFS2 does handle it. Look at the latest git
> version. Please, point which particular C operation in the
> jffs2_truncate_fragtree() function causes the oops.
>
> > More to it, I've noticed that if I remove the system power while file
> > on JFFS2 partition is being updated (it takes a bit of time to hit the
> > right moment), I'll completely lose the file (it just disappears). Any
> > ideas?
>
> What does "JFFS2 partition is being updated" mean?
>
> --
> Best regards,
> Artem Bityutskiy (Битюцкий Артём)
>
>

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Artem Bityutskiy]

On Wed, 2006-11-22 at 08:49 -0800, Dragan Marinkovic wrote:
> Thanks Artem, I'll take a look.
> 
> "JFFS2 partition is being updated" means that the file has been
> written to at the moment of the power loss. What's interesting is that
> if I'm writing 1MB file to the JFFS2 partition (which gives me enough
> time to reliably pull the plug during write operation) the power loss
> is always handled properly: the old version of the file is always
> there when the power is back on.
> 
> To be more precise, I have trouble with the file size of about 11K
> (this really isn't any magic number, basically smaller files). I have
> /tmp dir. on tmpfs . I'm moving the file there, doing some
> manipulation on it (sed, cat, etc.) and copy the file back to its
> original location (JFFS2 fs). If I experience the power loss at the
> moment of the file copy operation, the file just vanishes. The same is
> true if the file is manipulated directly on the JFFS2 partition. I've
> tried 'mv', 'cat' and 'cp' -- all with the same result.
> 
> The erase block size on the JFFS2 partition is set to 128K (131072
> bytes) . I can post USB stick and kernel config specific details if
> needed.

USB stick? How do you use JFFS2 on USB stick? Do you use block2mtd?

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Dragan Marinkovic]

On 11/22/06, Artem Bityutskiy <dedekind@infradead.org> wrote:
> On Wed, 2006-11-22 at 08:49 -0800, Dragan Marinkovic wrote:
> > Thanks Artem, I'll take a look.
> >
> > "JFFS2 partition is being updated" means that the file has been
> > written to at the moment of the power loss. What's interesting is that
> > if I'm writing 1MB file to the JFFS2 partition (which gives me enough
> > time to reliably pull the plug during write operation) the power loss
> > is always handled properly: the old version of the file is always
> > there when the power is back on.
> >
> > To be more precise, I have trouble with the file size of about 11K
> > (this really isn't any magic number, basically smaller files). I have
> > /tmp dir. on tmpfs . I'm moving the file there, doing some
> > manipulation on it (sed, cat, etc.) and copy the file back to its
> > original location (JFFS2 fs). If I experience the power loss at the
> > moment of the file copy operation, the file just vanishes. The same is
> > true if the file is manipulated directly on the JFFS2 partition. I've
> > tried 'mv', 'cat' and 'cp' -- all with the same result.
> >
> > The erase block size on the JFFS2 partition is set to 128K (131072
> > bytes) . I can post USB stick and kernel config specific details if
> > needed.
>
> USB stick? How do you use JFFS2 on USB stick? Do you use block2mtd?
>
> --
> Best regards,
> Artem Bityutskiy (Битюцкий Артём)
>
>

That is correct -- I'm using block2mtd .

Cheers,
Dragan

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Artem Bityutskiy]

On Wed, 2006-11-22 at 08:49 -0800, Dragan Marinkovic wrote:
> The erase block size on the JFFS2 partition is set to 128K (131072
> bytes) . I can post USB stick and kernel config specific details if
> needed.

This encouraged me to write this FAQ entry. If you use JFFS2 on top of
an USB stick, please read it.

file:///home/dedekind/opensource/git/mtd-www.git/faq/jffs2.html#L_stick_jffs2

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: jffs2_truncate_fragtree does not handle NULL return from jffs2_lookup_node_frag

[Artem Bityutskiy]

On Wed, 2006-11-22 at 19:19 +0200, Artem Bityutskiy wrote:
> file:///home/dedekind/opensource/git/mtd-www.git/faq/jffs2.html#L_stick_jffs2
Ops, apologies, this is the correct link:

http://www.linux-mtd.infradead.org/faq/jffs2.html#L_stick_jffs2

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)