fix jffs2 data errors ?

fix jffs2 data errors ?

[Armin Schindler]

Hi,

together with latest CVS of jffs2 (one week old), I try to use
my rootfs on a jffs2 NAND-Flash partition.

It seems to work pretty good, but when I have e.g. reset/power-down,
on the next boot/rootfs-mount I get messages like:

 jffs2_scan_dirent_node(): Name CRC failed on node at 0x001b89dc: Read 0xffffffff,
 calculated 0xc04c39e5
 Empty flash at 0x001b8a10 ends at 0x001b8c00
 Empty flash at 0x001b93f8 ends at 0x001b9400

I thought, that such errors will be fixed, but every time I mount the partition,
these errors still exist.

Is the filesystem now unfixable or am I doing something wrong?

Thanks for any hint,
Armin

Re: fix jffs2 data errors ?

[John Stirling]

I get the same problems with a CVS snapshot about 4 weeks old. The
errors/warnings are reported on boot up which is disconcerting, but it
does seem to be working ok if you ignore the errors.


On Mon, 2004-01-26 at 14:26, Armin Schindler wrote:
> Hi,
> 
> together with latest CVS of jffs2 (one week old), I try to use
> my rootfs on a jffs2 NAND-Flash partition.
> 
> It seems to work pretty good, but when I have e.g. reset/power-down,
> on the next boot/rootfs-mount I get messages like:
> 
>  jffs2_scan_dirent_node(): Name CRC failed on node at 0x001b89dc: Read 0xffffffff,
>  calculated 0xc04c39e5
>  Empty flash at 0x001b8a10 ends at 0x001b8c00
>  Empty flash at 0x001b93f8 ends at 0x001b9400
> 
> I thought, that such errors will be fixed, but every time I mount the partition,
> these errors still exist.
> 
> Is the filesystem now unfixable or am I doing something wrong?
> 
> Thanks for any hint,
> Armin
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: fix jffs2 data errors ?

[David Woodhouse]

On Tue, 2004-01-27 at 11:23 +0000, John Stirling wrote:
> I get the same problems with a CVS snapshot about 4 weeks old. The
> errors/warnings are reported on boot up which is disconcerting, but it
> does seem to be working ok if you ignore the errors.

Please don't top-post. Place your reply below the quoted text, and
include _only_ those parts of the mail to which you're replying which
are _absolutely_ necessary for context. 

-- 
dwmw2

Re: fix jffs2 data errors ?

[Armin]

On 27 Jan 2004, John Stirling wrote:
> I get the same problems with a CVS snapshot about 4 weeks old. The
> errors/warnings are reported on boot up which is disconcerting, but it
> does seem to be working ok if you ignore the errors.

I'm not so sure. I did an ugly test by doing
 #cp file1 file2
and during this command I pushed the reset button.

After bootup and the expected error messages the file2 was
there in an expected size (about half of file1).

Now I tried to remove the "broken" file2, which worked, but
after umount/mount, the file2 was here again. So I was not
able to remove this file.

Armin

Re: fix jffs2 data errors ?

[David Woodhouse]

On Tue, 2004-01-27 at 12:42 +0100, Armin wrote:
> Now I tried to remove the "broken" file2, which worked, but
> after umount/mount, the file2 was here again. So I was not
> able to remove this file.

If this is reproduceable, please do so with CONFIG_JFFS2_FS_DEBUG=1,
logging all messages over a serial console. Show the output (you may
need to boot with 'debug' on the command line) while the file system is
mounted, the offending file unlinked, and then capture the output while
the file system is mounted a second time. If possible, take snapshots of
the flash before each mount attempt.

Thanks.

-- 
dwmw2

Re: fix jffs2 data errors ?

[Armin]

On Tue, 27 Jan 2004, David Woodhouse wrote:
> On Tue, 2004-01-27 at 12:42 +0100, Armin wrote:
> > Now I tried to remove the "broken" file2, which worked, but
> > after umount/mount, the file2 was here again. So I was not
> > able to remove this file.
>
> If this is reproduceable, please do so with CONFIG_JFFS2_FS_DEBUG=1,
> logging all messages over a serial console. Show the output (you may
> need to boot with 'debug' on the command line) while the file system is
> mounted, the offending file unlinked, and then capture the output while
> the file system is mounted a second time. If possible, take snapshots of
> the flash before each mount attempt.

I just updated my kernel with your latest changes in cvs and it looks very
good.

I cannot reproduce the problem any more, so it seems to work
with Howards fix!

Thanks for the quick help.

Armin

Re: fix jffs2 data errors ?

[David Woodhouse]

On Mon, 2004-01-26 at 15:26 +0100, Armin Schindler wrote:
> I thought, that such errors will be fixed, but every time I mount the partition,
> these errors still exist.

They'll persist until the garbage collection erases the block in
question. That'll happen naturally in time, although I suppose we
probably ought to prioritise it so the blocks we complain about are GC'd
sooner. 

It's harmless. I prefer to err on the side of verbosity than to miss
reporting things which really _could_ be a problem. 

-- 
dwmw2

Re: fix jffs2 data errors ?

[Armin]

On Tue, 27 Jan 2004, David Woodhouse wrote:
> On Mon, 2004-01-26 at 15:26 +0100, Armin Schindler wrote:
> > I thought, that such errors will be fixed, but every time I mount the partition,
> > these errors still exist.
>
> They'll persist until the garbage collection erases the block in
> question. That'll happen naturally in time, although I suppose we
> probably ought to prioritise it so the blocks we complain about are GC'd
> sooner.
>
> It's harmless. I prefer to err on the side of verbosity than to miss
> reporting things which really _could_ be a problem.

Okay, that makes sense.

What is the time the garbage collection starts or finished this?
I waited some time (about 10 minutes) and the messages still occured after
remount.

Armin

Re: fix jffs2 data errors ?

[Howard Gray]

On Mon, 2004-01-26 at 14:26, Armin Schindler wrote:
> Hi,
> 
> together with latest CVS of jffs2 (one week old), I try to use
> my rootfs on a jffs2 NAND-Flash partition.
> 
> It seems to work pretty good, but when I have e.g. reset/power-down,
> on the next boot/rootfs-mount I get messages like:
> 
>  jffs2_scan_dirent_node(): Name CRC failed on node at 0x001b89dc: Read 
0xffffffff,

Sorry, I missed this posting originally, otherwise I wouldn't have top posted. 
I have also seen the problems you mentioned. Please see my mail "Problem 
deleting files with NAND/JFFS2 and a possible solution" posted today. The 
"solution" appears to cure these problems too, at least for me.

On Tuesday 27 January 2004 12:47, Armin wrote:
> What is the time the garbage collection starts or finished this?
> I waited some time (about 10 minutes) and the messages still occured after
> remount.

The following is based on my observation, not from any detailed understanding 
of the code. I'm sure David Woodhouse will correct me if I'm wrong.

Garbage collection is started by the kupdated/bdflush daemon or when there are 
not enough free blocks to carry out another file operation or by the sync 
command. Or you can kick it by sending SIGHUP to the GC process. But in your 
case the GC won't clean up your bad blocks until some other file really needs 
the space, so it doesn't matter how long you wait. You could try filling up 
your FLASH with data and then erasing it again. That'll do it. 

If the GC is started by kupdated/bdflush or sync it will only really do 
something if it thinks it's necessary i.e.if some inodes are marked as dirty. 

Older versions of the GC were started by a 2 second timer but that seems to 
have been abondoned in favour of kupdated (but you can still find some 
refrences to it in the code, which foxed me for a while...).

P.S.
You can adjust the frequency with which bdflush runs by writing to /proc/sys/
vm/bdflush. See the kernel documentaion file sysctl/vm.txt for details.


--
Howard

Re: fix jffs2 data errors ?

[David Woodhouse]

On Tue, 2004-01-27 at 14:13 +0100, Howard Gray wrote:
> Sorry, I missed this posting originally, otherwise I wouldn't have top posted. 

You weren't top-posting, were you? By 'top-posting' I refer to the
abominable practice of writing a confusing upside-down mail, by placing
your own reply _above_ the mail to which you're replying. It's generally
accompanied by quoting a _complete_ copy of the original mail, rather
than only those parts which are needed for context. 

> I have also seen the problems you mentioned. Please see my mail "Problem 
> deleting files with NAND/JFFS2 and a possible solution" posted today. The 
> "solution" appears to cure these problems too, at least for me.

It should do, yes.

> Garbage collection is started by the kupdated/bdflush daemon or when
> there are not enough free blocks to carry out another file operation
> or by the sync command.

Also by the dedicated JFFS2 kernel thread, to make space for new writes
slightly ahead of time, before we actually have to garbage collect in
the context of a process which is attempting to write.

>  Or you can kick it by sending SIGHUP to the GC process. But in your 
> case the GC won't clean up your bad blocks until some other file really needs 
> the space, so it doesn't matter how long you wait. You could try filling up 
> your FLASH with data and then erasing it again. That'll do it. 

I'm thinking of prioritising the blocks we complain about, so they get
garbage collected sooner despite the fact that they haven't really got
much space wasted in them. It is purely a cosmetic issue though.

> Older versions of the GC were started by a 2 second timer but that seems to 
> have been abondoned in favour of kupdated (but you can still find some 
> refrences to it in the code, which foxed me for a while...).

Patches accepted :)

-- 
dwmw2

Re: fix jffs2 data errors ?

[David Woodhouse]

On Tue, 2004-01-27 at 12:47 +0100, Armin wrote:
> What is the time the garbage collection starts or finished this?
> I waited some time (about 10 minutes) and the messages still occured after
> remount.

The garbage collection may not erase the offending block immediately, or
even soon. Only when enough of the _rest_ of the nodes in the same
eraseblock are considered dirty will the block be likely to be selected
for garbage collection. 

-- 
dwmw2