RE: JFFS3 & performance

Joakim Tjernlund
2004-12-16T22:07:07Z

> On Thu, 16 December 2004 22:06:21 +0100, Joakim Tjernlund wrote:
> >
> > There is still more one can do, look in lib/crc32.c in the kernel to see
> > what I mean. It is tricky to get everything right, but just unrolling it a bit will help.
>
> Sure. But I won't do it tonight anymore and the rest of the week is
> booked.

Same here :(

>
> In related news, I just had a look at adler32 code. It's pretty
> obvious why it runs so fast and why we shouldn't use it as a checksum:
> o It splits data into chunks of 5552 bytes.
> o Inside each chunk, it basically implements the two weak checksums
> from rsync. (And those really are weak. I once tried using them for
> a hash table and the result was horrible.)
> o At the end of each chunk, it implements crc16 on both weak
> checksums.
>
> With jffs3 node containing less than 5552 bytes each, the crc16
> operation, a simple modulo, is done but once. So in effect, we end up
> with the weak checksums from rsync, which cause tons of collisions in
> hash tables. Pretty nasty for a checksum.
>
> Istr reading something from one of the two authors that basically
> states the adler32 weakness for short files. But for long files, the
> checksum is getting strong enough and much faster. Too bad that our
> "files" are short.

I too rember reading about that. So a new optimzed crc16 is probably the best bet.
It should do better than crc32 once it is done.

hmm, I rember that the networking code has some crc/checksum algorithm too.
Maybe it is possible to use that?

Jocke