JFFS2: checking CRCs twice!?

JFFS2: checking CRCs twice!?

[Steven Scholz]

Hi there,

IIUC than mounting a JFFS2 root fs takes so long (6 seconds on my embedded ARM9) 
cause the CRCs of all blocks/sectors are checked.

But then it looks to me that - shortly after boot - the GC thread doing the same 
again (in background.c), which takes another 6 scondes and slows down the 
application startup...

Or am I missing something?

-- 
Steven Scholz

imc Measurement & Control

Re: JFFS2: checking CRCs twice!?

[Ferenc Havasi]

Hi Steven,

> But then it looks to me that - shortly after boot - the GC thread doing 
> the same again (in background.c), which takes another 6 scondes and 
> slows down the application startup...

If I am right there are two kind of CRC: hdr_crc and node_crc. At 
mounting time only the hdr_crc is checked which depends on only the 
header. (using summary there is only one summary CRC check)

Later (when the data is readed or in the background by GC) the node_crc 
will also be checked. GC is running on a "niced" (10) priority, so it 
should not disturb any other process.

If I am right the CRC protects against unclean reboots and hardware 
corruptions. If you need we can modify the summary patch not to check 
node_crc for summarized nodes (where the unclean reboots is no problem), 
but if we do it you lose the ability to detect the hardware corruption 
(flash errors). Unfortunatelly NOR flash doesn't have any error detection.

Bye,
Ferenc

Re: JFFS2: checking CRCs twice!?

[Artem B. Bityuckiy]

> IIUC than mounting a JFFS2 root fs takes so long (6 seconds on my 
embedded ARM9)
> cause the CRCs of all blocks/sectors are checked.
Cause:
1. *Some* of CRCs are checked. Nodes has 3 CRCs: (1) Common header CRC, 
(2) Header CRC, (3) Data CRC. On mount, JFFS2 check only common header 
CRCs of inode nodes, and all CRCs of direntries. When checking in 
background, it checks other CRCs.
2. On mount, JFFS2 reads blocks fully. So multiply the number of used 
blocks on the block read time and add this number to your mount time. This 
should be fixed, IMHO, since we do not needed, for example, read inode 
node's data.
3. It builds different data structures - fragtree list, direntries list. 
It counts the inode link numbers. This also takes time, especially if you 
have slow CPU. Here is some room to improvement exist.

Seems that's all major activities.

>
> But then it looks to me that - shortly after boot - the GC thread doing 
the same
> again (in background.c), which takes another 6 scondes and slows down 
the
> application startup...
Not the same, it checks CRCs which were not checked during mount.

> Or am I missing something?

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: checking CRCs twice!?

[Artem B. Bityuckiy]

On Wed, 12 Jan 2005, Artem B. Bityuckiy wrote:

> > IIUC than mounting a JFFS2 root fs takes so long (6 seconds on my 
> embedded ARM9)
> > cause the CRCs of all blocks/sectors are checked.
> Cause:
> 1. *Some* of CRCs are checked. Nodes has 3 CRCs: (1) Common header CRC, 
> (2) Header CRC, (3) Data CRC. On mount, JFFS2 check only common header 
> CRCs of inode nodes, and all CRCs of direntries. When checking in 
> background, it checks other CRCs.
> 2. On mount, JFFS2 reads blocks fully. So multiply the number of used 
> blocks on the block read time and add this number to your mount time. This 
> should be fixed, IMHO, since we do not needed, for example, read inode 
> node's data.
> 3. It builds different data structures - fragtree list, direntries list. 
Sorry, I'm wrong - fragtree is not built during mount. Ony dirent list. 
And this is only needed to detect inodes nlink. 
> It counts the inode link numbers. This also takes time, especially if you 
> have slow CPU. Here is some room to improvement exist.
> 
> Seems that's all major activities.
> 
> >
> > But then it looks to me that - shortly after boot - the GC thread doing 
> the same
> > again (in background.c), which takes another 6 scondes and slows down 
> the
> > application startup...
> Not the same, it checks CRCs which were not checked during mount.
> 
> > Or am I missing something?
> 
> --
> Best Regards,
> Artem B. Bityuckiy,
> St.-Petersburg, Russia.
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: checking CRCs twice!?

[Steven Scholz]

Hi Ferenc,

Ferenc Havasi wrote:

> If I am right there are two kind of CRC: hdr_crc and node_crc. At 
> mounting time only the hdr_crc is checked which depends on only the 
> header. (using summary there is only one summary CRC check)
> 
> Later (when the data is readed or in the background by GC) the node_crc 
> will also be checked. GC is running on a "niced" (10) priority, so it 
> should not disturb any other process.
Well, it does. On small, slow systems.

> If I am right the CRC protects against unclean reboots and hardware 
> corruptions. If you need we can modify the summary patch not to check 
> node_crc for summarized nodes (where the unclean reboots is no problem), 
That would be nice.

> but if we do it you lose the ability to detect the hardware corruption 
> (flash errors). Unfortunatelly NOR flash doesn't have any error detection.
Well. I think when a sectors gots written correctly (and summary is correct), I 
have to assume that the sector stay intact. So no need for checks.

If a sector got broken just by reading it then the whole embedded system is 
broken and has to be replaced ...

-- 
Steven Scholz

Re: JFFS2: checking CRCs twice!?

[Artem B. Bityuckiy]

On Wed, 12 Jan 2005, Steven Scholz wrote:

> Hi Ferenc,
> 
> Ferenc Havasi wrote:
> 
> > If I am right there are two kind of CRC: hdr_crc and node_crc. At 
> > mounting time only the hdr_crc is checked which depends on only the 
> > header. (using summary there is only one summary CRC check)
> > 
> > Later (when the data is readed or in the background by GC) the node_crc 
> > will also be checked. GC is running on a "niced" (10) priority, so it 
> > should not disturb any other process.
> Well, it does. On small, slow systems.
> 
> > If I am right the CRC protects against unclean reboots and hardware 
> > corruptions. If you need we can modify the summary patch not to check 
> > node_crc for summarized nodes (where the unclean reboots is no problem), 
> That would be nice.
> 
> > but if we do it you lose the ability to detect the hardware corruption 
> > (flash errors). Unfortunatelly NOR flash doesn't have any error detection.
> Well. I think when a sectors gots written correctly (and summary is correct), I 
> have to assume that the sector stay intact. So no need for checks.
> 
> If a sector got broken just by reading it then the whole embedded system is 
> broken and has to be replaced ...
This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
and *report* media problems.

> 
> -- 
> Steven Scholz
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: checking CRCs twice!?

[Steven Scholz]

>>Well. I think when a sectors gots written correctly (and summary is correct), I 
>>have to assume that the sector stay intact. So no need for checks.
>>
>>If a sector got broken just by reading it then the whole embedded system is 
>>broken and has to be replaced ...
> 
> This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> and *report* media problems.
Fair enough.
But how will pay attention if the system is embedded into your microwave oven or 
washing maschine... ;-)

--
Steven

Re: JFFS2: checking CRCs twice!?

[Artem B. Bityuckiy]

On Wed, 12 Jan 2005, Steven Scholz wrote:

> 
> >>Well. I think when a sectors gots written correctly (and summary is correct), I 
> >>have to assume that the sector stay intact. So no need for checks.
> >>
> >>If a sector got broken just by reading it then the whole embedded system is 
> >>broken and has to be replaced ...
> > 
> > This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> > and *report* media problems.
> Fair enough.
> But how will pay attention if the system is embedded into your microwave oven or 
> washing maschine... ;-)
In this case this doesn't make sence.
But in general it does.

Possibly it makes sence to have this checking as an configuration option.

> 
> --
> Steven
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: checking CRCs twice!?

[Josh Boyer]

On Wed, 2005-01-12 at 09:21, Artem B. Bityuckiy wrote:
> On Wed, 12 Jan 2005, Steven Scholz wrote:
> 
> > 
> > >>Well. I think when a sectors gots written correctly (and summary is correct), I 
> > >>have to assume that the sector stay intact. So no need for checks.
> > >>
> > >>If a sector got broken just by reading it then the whole embedded system is 
> > >>broken and has to be replaced ...
> > > 
> > > This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> > > and *report* media problems.
> > Fair enough.
> > But how will pay attention if the system is embedded into your microwave oven or 
> > washing maschine... ;-)
> In this case this doesn't make sence.

Actually, it could make sense in a microwave oven.  They have a
mechanism to display errors to the users if they'd like by using the
clock.  It's all up to the overall design of the system.  You don't
necessarily need a human to parse through dmesg.

(And if your washing machine is sophisticated enough to need Linux, then
it probably at least had some LEDs you could flash a morse code message
with.)

josh

Re: JFFS2: checking CRCs twice!?

[Steven Scholz]

>>>This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
>>>and *report* media problems.
>>
>>Fair enough.
>>But who will pay attention if the system is embedded into your microwave oven or 
>>washing maschine... ;-)
> 
> In this case this doesn't make sence.
> But in general it does.
> 
> Possibly it makes sence to have this checking as an configuration option.
ON by default though.

--
Steven

Re: JFFS2: checking CRCs twice!?

[Jörn Engel]

On Wed, 12 January 2005 15:11:09 +0000, Artem B. Bityuckiy wrote:
> On Wed, 12 Jan 2005, Steven Scholz wrote:
> 
> > If a sector got broken just by reading it then the whole embedded system is 
> > broken and has to be replaced ...

Amen, brother!

> This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> and *report* media problems.

Well, more and more I come to the conclusion that media problems are a
non-issue.  At least, flashes are about as reliable as hard drives.

So, who can name a hd-oriented file system that does crc-checking of
it's date?

Jörn

-- 
Don't patch bad code, rewrite it.
-- Kernigham and Pike, according to Rusty

Re: JFFS2: checking CRCs twice!?

[Josh Boyer]

On Wed, 2005-01-12 at 12:26, Jörn Engel wrote:
> On Wed, 12 January 2005 15:11:09 +0000, Artem B. Bityuckiy wrote:
> > On Wed, 12 Jan 2005, Steven Scholz wrote:
> > 
> > > If a sector got broken just by reading it then the whole embedded system is 
> > > broken and has to be replaced ...
> 
> Amen, brother!
> 
> > This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> > and *report* media problems.
> 
> Well, more and more I come to the conclusion that media problems are a
> non-issue.  At least, flashes are about as reliable as hard drives.

I've seen them fail.  Multiple times.  And yes, the CRCs detected it and
helped me figure out that the hardware was borked.

You may be right that failures are rare, and when they do occur the
flash definitely needs to be replaced.  But I'd much rather debug a card
that had CRC errors than one that just had apps mysteriously getting
SIGILL because a bit flipped and the filesystem didn't bother to verify
the data before using it.

Unless you'd be willing to debug all those cards for me.  In that case,
I'm all for whatever you decide to do with CRCs :).  Until then, I'd
prefer that they stay.

josh

Re: JFFS2: checking CRCs twice!?

[Jörn Engel]

On Wed, 12 January 2005 12:43:30 -0600, Josh Boyer wrote:
> 
> I've seen them fail.  Multiple times.  And yes, the CRCs detected it and
> helped me figure out that the hardware was borked.

Ok, Empiria has spoken, I stand corrected.

> You may be right that failures are rare, and when they do occur the
> flash definitely needs to be replaced.  But I'd much rather debug a card
> that had CRC errors than one that just had apps mysteriously getting
> SIGILL because a bit flipped and the filesystem didn't bother to verify
> the data before using it.

Agreed.  So we still need checksums, and have the type of checksums
left to argue about.

Jörn

-- 
Public Domain  - Free as in Beer
General Public - Free as in Speech
BSD License    - Free as in Enterprise
Shared Source  - Free as in "Work will make you..."

Re: JFFS2: checking CRCs twice!?

[Steven Scholz]

Artem B. Bityuckiy wrote:

>>>This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
>>>and *report* media problems.
>>
>>Fair enough.
>>But who will pay attention if the system is embedded into your microwave oven or 
>>washing maschine... ;-)
> 
> In this case this doesn't make sence.
> But in general it does.
> 
> Possibly it makes sence to have this checking as an configuration option.

So will someone add this config option for JFFS2?
Or should it be done in SUMMARY patch?
Are there any plans to include this SUMMARY patch to the JFFS2 CVS?
Or will it stay seperated?

--
Steven

Re: JFFS2: checking CRCs twice!?

[Artem B. Bityuckiy]

On Fri, 14 Jan 2005, Steven Scholz wrote:

> Artem B. Bityuckiy wrote:
> 
> >>>This seems to be true for NOR. But  chrcking CRC helps at least *detect* 
> >>>and *report* media problems.
> >>
> >>Fair enough.
> >>But who will pay attention if the system is embedded into your microwave oven or 
> >>washing maschine... ;-)
> > 
> > In this case this doesn't make sence.
> > But in general it does.
> > 
> > Possibly it makes sence to have this checking as an configuration option.
> 
> So will someone add this config option for JFFS2?
> Or should it be done in SUMMARY patch?
> Are there any plans to include this SUMMARY patch to the JFFS2 CVS?
> Or will it stay seperated?
AFAIK, Summary will go to JFFS3.
I don't know for what Ferenc made "enable/disable summary" configuration 
option, I believe it must be enabled by default.

> 
> --
> Steven
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.