JFFS2 on NAND reboot problems

JFFS2 on NAND reboot problems

[Chris Sperandeo]

I have pulled the latest from CVS server, applied the changes and retested.   
There seems to be no change in behaviour.  Any other suggestions?

Regards,

Chris

On Thursday 10 April 2003 02:20, Chris Sperandeo wrote:
> Hi,
>
> Wondering if anyone has experienced a bug described as follows.
>
> I am using the linux powerpc kernel 2.4.21-pre2.  We are using a March
> 09 2003 snapshot for MTD/JFFS2 from
> http://www.linux-mtd.infradead.org/.
>
> After rebooting the device we occasionally receive the following: ...
Could you please upgrade to latest CVS and try again ?

--
Thomas 
________________________________________________________________________
linutronix - competence in embedded & realtime linux http://www.linutronix.de
mail: tglx@linutronix.de

Hi,

Wondering if anyone has experienced a bug described as follows.

I am using the linux powerpc kernel 2.4.21-pre2.  We are using a March 09
2003 snapshot for MTD/JFFS2 from http://www.linux-mtd.infradead.org/.

After rebooting the device we occasionally receive the following:

*********************************
jffs2_get_inode_nodes(): CRC failed on node at 0x00d8a9c0: Read 0xffffffff,
calculated 0x4deb59e5

jffs2_get_inode_nodes(): Data CRC failed on node at 0x00d89a8c: Read
0xbaa193f5, calculated 0xcdd1fe6c Checked all inodes but still 0x1c20 bytes
of unchecked space? kernel BUG at gc.c:140!
Oops: Exception in kernel mode, sig: 4
NIP: C00A0420 XER: 00000000 LR: C00A0420 SP: C033FF10 REGS: c033fe60 TRAP:
0700
 Not tainted
MSR: 00009030 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 11
TASK = c033e000[8] 'jffs2_gcd_mtd1' Last syscall: -1
last math 00000000 last altivec 00000000
PLB0: bear= 0x00000000 acr=   0x00000000 besr=  0x00000000
PLB0 to OPB: bear= 0x00000000 besr0= 0x00000000 besr1= 0x00000000

GPR00: C00A0420 C033FF10 C033E000 00000018 00001030 00000001 00000020 C01F0000
GPR08: 00003DA8 C01A5D04 00000000 C033FE30 82004028 1001F598 00FE4100 007FFF2F
GPR16: 00000000 00000001 007FFF00 FFFFFFFF 00FDF920 00000000 00F9F548 00000002
GPR24: 00000000 C01AB4AC 00000000 FFFFFFFF 00000000 C035CDB8 C03CF0F8 
C03CF0C4 Call backtrace: C00A0420 C00A3790 C0006F00
**********************************

It appears that something has gone wrong with the garbage collection thread
(at least that is one of the theories).  Inserting a delay before actually
rebooting the device seems to remedy the problem somewhat but does not make
the problem dissappear.

Is it possible that the garbage collection thread is not given enough time to
do its works before the device is rebooted?

Any solution or suggestions on what avenues to a solution would be greatly
appreciated.

Regards,

Chris

Re: JFFS2 on NAND reboot problems

[Thomas Gleixner]

On Thursday 10 April 2003 21:26, Chris Sperandeo wrote:
> I have pulled the latest from CVS server, applied the changes and retested.
> There seems to be no change in behaviour.  Any other suggestions?

Can you send me a binary image of your flash ?

Turn on JFFS2 debugging to level 1 and log the debug messages on a serial  
console. Compress it and send it to me or put it anywhere for download. 

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: JFFS2 on NAND reboot problems

[matsunaga]

Hi
 
> Wondering if anyone has experienced a bug described as follows.
> 
> I am using the linux powerpc kernel 2.4.21-pre2.  We are using a March 09
> 2003 snapshot for MTD/JFFS2 from http://www.linux-mtd.infradead.org/.
> 
> After rebooting the device we occasionally receive the following:
> 
> *********************************
> jffs2_get_inode_nodes(): CRC failed on node at 0x00d8a9c0: Read 0xffffffff,
> calculated 0x4deb59e5
> 
> jffs2_get_inode_nodes(): Data CRC failed on node at 0x00d89a8c: Read
> 0xbaa193f5, calculated 0xcdd1fe6c Checked all inodes but still 0x1c20 bytes
> of unchecked space? kernel BUG at gc.c:140!
> Oops: Exception in kernel mode, sig: 4
> NIP: C00A0420 XER: 00000000 LR: C00A0420 SP: C033FF10 REGS: c033fe60 TRAP:
> 0700
>  Not tainted
> MSR: 00009030 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 11
> TASK = c033e000[8] 'jffs2_gcd_mtd1' Last syscall: -1
> last math 00000000 last altivec 00000000
> PLB0: bear= 0x00000000 acr=   0x00000000 besr=  0x00000000
> PLB0 to OPB: bear= 0x00000000 besr0= 0x00000000 besr1= 0x00000000
> 
> GPR00: C00A0420 C033FF10 C033E000 00000018 00001030 00000001 00000020 C01F0000
> GPR08: 00003DA8 C01A5D04 00000000 C033FE30 82004028 1001F598 00FE4100 007FFF2F
> GPR16: 00000000 00000001 007FFF00 FFFFFFFF 00FDF920 00000000 00F9F548 00000002
> GPR24: 00000000 C01AB4AC 00000000 FFFFFFFF 00000000 C035CDB8 C03CF0F8 
> C03CF0C4 Call backtrace: C00A0420 C00A3790 C0006F00
> **********************************
> 
> It appears that something has gone wrong with the garbage collection thread
> (at least that is one of the theories).  Inserting a delay before actually
> rebooting the device seems to remedy the problem somewhat but does not make
> the problem dissappear.
> 
> Is it possible that the garbage collection thread is not given enough time to
> do its works before the device is rebooted?
> 
> Any solution or suggestions on what avenues to a solution would be greatly
> appreciated.

It seems to me that the system was rebooted without wbuf flush.
If it is true, another problem I think is that mount process cannot detect the inode inconsistency.
This fs should be designed on the premise that power failure happens.
The last accessing file do not need to be saved, but fs does.

Sorry if this is solved on the latest code...

BR.
Matsunaga.

__________________________________________________
Do You Yahoo!?
Yahoo! BB is Broadband by Yahoo!  http://bb.yahoo.co.jp/

Re: JFFS2 on NAND reboot problems

[Thomas Gleixner]

On Thursday 17 April 2003 18:17, matsunaga wrote:
> It seems to me that the system was rebooted without wbuf flush.
> If it is true, another problem I think is that mount process cannot detect
> the inode inconsistency. This fs should be designed on the premise that
> power failure happens. The last accessing file do not need to be saved, but
> fs does.
> Sorry if this is solved on the latest code

It is designed this way. But we are neither perfect nor can we predict every 
10e12 possibilities to produce a hidden bug.

I cannot reproduce it here. 
That's why I asked for a bainary Image of the flash content, but I received 
none until now.

We can only solve problems, which are either reproducable or which can be 
reproduced by some user supplied pattern / image. 

We all do our very best to make this code stable and reliable, but we need the 
help from _USERS_ to detect insufficiencies. That's the way open source 
software works. If we don't get the requested feedback / help we can only 
say: You got the code. Feel free to fix it.

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: JFFS2 on NAND reboot problems

[matsunaga]

> On Thursday 17 April 2003 18:17, matsunaga wrote:
> > It seems to me that the system was rebooted without wbuf flush.
> > If it is true, another problem I think is that mount process cannot detect
> > the inode inconsistency. This fs should be designed on the premise that
> > power failure happens. The last accessing file do not need to be saved, but
> > fs does.
> > Sorry if this is solved on the latest code
> 
> It is designed this way. But we are neither perfect nor can we predict every 
> 10e12 possibilities to produce a hidden bug.
> 
> I cannot reproduce it here. 
> That's why I asked for a bainary Image of the flash content, but I received 
> none until now.

To tell the truth, I saw the same BUG() last month, but I had other things to fix and put it aside.
Now it is hard for me to reproduce it, too. I might be solved on the updated code.
Sending binary image could reveal confidential information, so he might hesitate to do it...
 
> We can only solve problems, which are either reproducable or which can be 
> reproduced by some user supplied pattern / image. 
> 
> We all do our very best to make this code stable and reliable, but we need the 
> help from _USERS_ to detect insufficiencies. That's the way open source 
> software works. If we don't get the requested feedback / help we can only 
> say: You got the code. Feel free to fix it.

I understand. We are all users and not guests.

P.S.
It is trivial, but I think gc.c line 125 ic is not necessary, or the name be changed.
__________________________________________________
Do You Yahoo!?
Yahoo! BB is Broadband by Yahoo!  http://bb.yahoo.co.jp/

Re: JFFS2 on NAND reboot problems

[Thomas Gleixner]

On Saturday 19 April 2003 11:46, Srinivasu.Vaduguri@nokia.com wrote:
> I see this problem often in our device.. I was not able to reproduce it
> till now.. But recently I found a way..
>
> If we have two directories 'a' and 'b' in /flash which is a JFFS2 file
> system.. and if we do an "rm -rf /flash/a" and reboot immediately or power
> cycle.. Then on reboot while mounting we get this bug in garbage
> collection.. This happens almost 95% of the times..

I tried about 25 times and had no chance to trigger the BUG().

Could you please provide a binary image of your FLASH partition, which 
triggers the BUG() ?
Reboot. Do not mount /flash and do a 
cat /dev/mtdX >flash:img
If this is NAND, you will get a bunch of "Reading without ECC is not 
recommended" warnings. Ignore them :)

Thanks.

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: JFFS2 on NAND reboot problems

[Thomas Gleixner]

Hi all !

Please get latest CVS code. The problem should be fixed now. Can you please 
check ?

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de