mkfs.ubifs gives corrupt images/crc errors

mkfs.ubifs gives corrupt images/crc errors

[Leo Barnes]

Hello!

The recent versions of mkfs.ubifs in the mtd-utils gives corrupt
images for me. I get the following error messages when trying to mount
the ubi-volume:

<3>[  711.246826] UBIFS error (pid 413): ubifs_check_node: bad CRC:
calculated 0x1f8f89ca, read 0xbb39064c
<3>[  711.247558] UBIFS error (pid 413): ubifs_check_node: bad node at LEB 0:0
<3>[  711.247924] UBIFS error (pid 413): ubifs_read_node: expected node type 6

I have identified it as being broken since (approximately) after
commit 004760f969175da7392dd02bb4fa14164ff46c1e (Wed Jun 16 09:47:27
2010 +0300). Two or three of the commits after this one refuse to
build with various make errors, and the next one that builds gives
corrupt images. It seems to have something to do with the CRC
calculation.

Best regards,
//Leo

Re: mkfs.ubifs gives corrupt images/crc errors

[Artem Bityutskiy]

On Tue, 2010-07-27 at 09:03 +0200, Leo Barnes wrote:
> Hello!
> 
> The recent versions of mkfs.ubifs in the mtd-utils gives corrupt
> images for me. I get the following error messages when trying to mount
> the ubi-volume:
> 
> <3>[  711.246826] UBIFS error (pid 413): ubifs_check_node: bad CRC:
> calculated 0x1f8f89ca, read 0xbb39064c
> <3>[  711.247558] UBIFS error (pid 413): ubifs_check_node: bad node at LEB 0:0
> <3>[  711.247924] UBIFS error (pid 413): ubifs_read_node: expected node type 6
> 
> I have identified it as being broken since (approximately) after
> commit 004760f969175da7392dd02bb4fa14164ff46c1e (Wed Jun 16 09:47:27
> 2010 +0300). Two or three of the commits after this one refuse to
> build with various make errors, and the next one that builds gives
> corrupt images. It seems to have something to do with the CRC
> calculation.

Could you please find the commit which moves CRC support out of
mtd-utils/mkfs.ubifs and investigate what changed please? I think you
will find the reason quite quickly.

I'm on holiday, cannot look at this now.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: mkfs.ubifs gives corrupt images/crc errors

[Leo Barnes]

> Could you please find the commit which moves CRC support out of
> mtd-utils/mkfs.ubifs and investigate what changed please? I think you
> will find the reason quite quickly.
>
> I'm on holiday, cannot look at this now.

I have now identified the first broken commit to be:

commit 2d40ed7b415bcb6a358c9f221160c4a419f01f19
Author: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Date:   Sat Jul 17 09:43:08 2010 +0300

    libs: make crc32 and fec to be libraries


When looking at what this commit does, I can't understand why it
breaks the CRC-calculation for mkfs.ubifs since all it seems to do is
to create a library archive out of the crc files (after renaming them)
and change the crc32() function from being static inline to normal. No
function calls in mkfs.ubifs are changed, and the new crc32() function
seems to do exactly the same thing as the old.

Has anyone else noticed this problem, or is my device more susceptible
to this for some odd reason?

Best regards,
//Leo

Re: mkfs.ubifs gives corrupt images/crc errors

[Joakim Tjernlund]

>
> > Could you please find the commit which moves CRC support out of
> > mtd-utils/mkfs.ubifs and investigate what changed please? I think you
> > will find the reason quite quickly.
> >
> > I'm on holiday, cannot look at this now.
>
> I have now identified the first broken commit to be:
>
> commit 2d40ed7b415bcb6a358c9f221160c4a419f01f19
> Author: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> Date:   Sat Jul 17 09:43:08 2010 +0300
>
>     libs: make crc32 and fec to be libraries
>
>
> When looking at what this commit does, I can't understand why it
> breaks the CRC-calculation for mkfs.ubifs since all it seems to do is
> to create a library archive out of the crc files (after renaming them)
> and change the crc32() function from being static inline to normal. No
> function calls in mkfs.ubifs are changed, and the new crc32() function
> seems to do exactly the same thing as the old.
>
> Has anyone else noticed this problem, or is my device more susceptible
> to this for some odd reason?

two ideas, either you need to clean out old object files
or mkfs.ubifs picks up another impl. of crc32

Re: mkfs.ubifs gives corrupt images/crc errors

[Artem Bityutskiy]

On Tue, 2010-07-27 at 09:03 +0200, Leo Barnes wrote:
> Hello!
> 
> The recent versions of mkfs.ubifs in the mtd-utils gives corrupt
> images for me. I get the following error messages when trying to mount
> the ubi-volume:
> 
> <3>[  711.246826] UBIFS error (pid 413): ubifs_check_node: bad CRC:
> calculated 0x1f8f89ca, read 0xbb39064c
> <3>[  711.247558] UBIFS error (pid 413): ubifs_check_node: bad node at LEB 0:0
> <3>[  711.247924] UBIFS error (pid 413): ubifs_read_node: expected node type 6
> 
> I have identified it as being broken since (approximately) after
> commit 004760f969175da7392dd02bb4fa14164ff46c1e (Wed Jun 16 09:47:27
> 2010 +0300). Two or three of the commits after this one refuse to
> build with various make errors, and the next one that builds gives
> corrupt images. It seems to have something to do with the CRC
> calculation.

I cannot reproduce this. Does this still happen for you? Does cloning
the mtd-utils git tree anew help?

Artem.

Re: mkfs.ubifs gives corrupt images/crc errors

[Leo Barnes]

> I cannot reproduce this. Does this still happen for you? Does cloning
> the mtd-utils git tree anew help?
>

Hello!

Sorry for not answering earlier, I was on vacation without internet
access. I have not tested the absolutely latest commits, but I have
now identified the problem.

In commit 2d40ed7b415bcb6a358c9f221160c4a419f01f19, the
crc-calculation function changed from being an inline function to a
linked function called crc32. There already exists a crc32 function
with the same arguments in ZLIB, which does not seem to give the same
checksums as the mtd-utils crc-function. After the mentioned commit,
mkfs.ubifs seems to use the ZLIB crc32-function instead of the
mtd-utils version which is what gives the faulty checksums.

I managed to fix this in two simple ways:
1. Change the order of linking in the makefile so that libmtd comes
before libz. This might possibly break ZLIB (I have not tried this
when using ZLIB).
2. Change the name of the mtd-utils crc32 function.

Changing the name of the function is probably the preferred option.

Best regards,
//Leo

Re: mkfs.ubifs gives corrupt images/crc errors

[Artem Bityutskiy]

On Thu, 2010-09-02 at 19:28 +0200, Leo Barnes wrote:
> Sorry for not answering earlier, I was on vacation without internet
> access. I have not tested the absolutely latest commits, but I have
> now identified the problem.
> 
> In commit 2d40ed7b415bcb6a358c9f221160c4a419f01f19, the
> crc-calculation function changed from being an inline function to a
> linked function called crc32. There already exists a crc32 function
> with the same arguments in ZLIB, which does not seem to give the same
> checksums as the mtd-utils crc-function. After the mentioned commit,
> mkfs.ubifs seems to use the ZLIB crc32-function instead of the
> mtd-utils version which is what gives the faulty checksums.
> 
> I managed to fix this in two simple ways:
> 1. Change the order of linking in the makefile so that libmtd comes
> before libz. This might possibly break ZLIB (I have not tried this
> when using ZLIB).
> 2. Change the name of the mtd-utils crc32 function.
> 
> Changing the name of the function is probably the preferred option.

Right, another guy reported the same issue a couple of days ago and I
fixed this.

-- 
Best Regards,
Artem Bityutskiy (Битюцкий Артём)