Latest mkfs.jffs2 is broken.

Latest mkfs.jffs2 is broken.

[H Hartley Sweeten]

Hello all,

Buildroot recently bumped the mtd-utils package from version 1.3.1 to 1.4.0.

With the mkfs.jffs2 utility in 1.3.1 I am able to create rootfs.jffs images that
work fine on my i.mx35 arm system.  But, with the 1.4.0 version all the images
do not work.

With the 1.4.0 built images I get messages like this during the kernel boot:

jffs2_scan_eraseblock(): Node at 0x00000000 {0x1985, 0x2003, 0x0000000c) has invalid CRC 0x813c6fd8 (calculated 0xe41eb0b1)
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x000c instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000008: 0x6fd8 instead
jffs2_scan_eraseblock(): Node at 0x0000000c {0x1985, 0xe001, 0x0000002b) has invalid CRC 0x1804b18f (calculated 0x7d266ee6)
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000010: 0x002b instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000014: 0xb18f instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000018: 0x0001 instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000020: 0x0002 instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000024: 0x1a8d instead
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000028: 0x0403 instead
Further such events for this erase block will not be printed
Old JFFS2 bitmask found at 0x00000078
You cannot use older JFFS2 filesystems with newer kernels
Empty flash at 0x0000a46c ends at 0x0000a470

And, if I check the images with jffs2dump I get stuff like:

Wrong hdr_crc  at  0x0001f808, 0x783f44d2 instead of 0x1d1d9bbb
Wrong bitmask  at  0x0001f80c, 0x07f3
Empty space found from 0x0001fffc to 0x00020000

Any ideas what could be wrong with the 1.4.0 release?

Thanks,
Hartley

Re: Latest mkfs.jffs2 is broken.

[Artem Bityutskiy]

On Thu, 2010-10-14 at 16:54 -0500, H Hartley Sweeten wrote:
> Hello all,
> 
> Buildroot recently bumped the mtd-utils package from version 1.3.1 to 1.4.0.
> 
> With the mkfs.jffs2 utility in 1.3.1 I am able to create rootfs.jffs images that
> work fine on my i.mx35 arm system.  But, with the 1.4.0 version all the images
> do not work.
> 
> With the 1.4.0 built images I get messages like this during the kernel boot:
> 
> jffs2_scan_eraseblock(): Node at 0x00000000 {0x1985, 0x2003, 0x0000000c) has invalid CRC 0x813c6fd8 (calculated 0xe41eb0b1)
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x000c instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000008: 0x6fd8 instead
> jffs2_scan_eraseblock(): Node at 0x0000000c {0x1985, 0xe001, 0x0000002b) has invalid CRC 0x1804b18f (calculated 0x7d266ee6)
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000010: 0x002b instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000014: 0xb18f instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000018: 0x0001 instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000020: 0x0002 instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000024: 0x1a8d instead
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000028: 0x0403 instead
> Further such events for this erase block will not be printed
> Old JFFS2 bitmask found at 0x00000078
> You cannot use older JFFS2 filesystems with newer kernels
> Empty flash at 0x0000a46c ends at 0x0000a470
> 
> And, if I check the images with jffs2dump I get stuff like:
> 
> Wrong hdr_crc  at  0x0001f808, 0x783f44d2 instead of 0x1d1d9bbb
> Wrong bitmask  at  0x0001f80c, 0x07f3
> Empty space found from 0x0001fffc to 0x00020000
> 
> Any ideas what could be wrong with the 1.4.0 release?

Could you please bisect the mtd-utils.git tree and find the offending
commit.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Latest mkfs.jffs2 is broken.

[Peter Korsgaard]

>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

Hi,

 Artem> On Thu, 2010-10-14 at 16:54 -0500, H Hartley Sweeten wrote:

 Artem> Could you please bisect the mtd-utils.git tree and find the offending
 Artem> commit.

With big endian atleast, jffs2dump complains about invalid hdr_crc on
1.4.0, but it's working on HEAD. A bisect shows that the commit fixing
it was Mike's:

commit b864c387e8f16273aad1453d6457d847e29c3d25
Author: Mike Frysinger <vapier@gentoo.org>
Date:   Wed Sep 22 22:32:56 2010 -0400

    mtd-utils: clean up zlib.h usage a bit
    
    Hide zlib's crc32 in compr_zlib.c and mkfs.ubifs/compr.c.
    
    jffs2reader.c and mkfs.jffs2.c don't actually use zlib, so punt the
    include from the file.
    
    mkfs.jffs2.c is implicitly using crc32 from zlib.h instead of the
    local mtd_crc32, so fix the local usage.  otherwise we get warnings
    about undefined crc32 because the file was redirecting the prototype.
    
    Signed-off-by: Mike Frysinger <vapier@gentoo.org>
    Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>

Hartley, that commit applies OK to 1.4.0, could you give it a try?

Artem: Time for a v1.4.1 release?

-- 
Bye, Peter Korsgaard

Re: Latest mkfs.jffs2 is broken.

[Artem Bityutskiy]

On Fri, 2010-10-15 at 10:35 +0200, Peter Korsgaard wrote:
> commit b864c387e8f16273aad1453d6457d847e29c3d25
> Author: Mike Frysinger <vapier@gentoo.org>
> Date:   Wed Sep 22 22:32:56 2010 -0400
> 
>     mtd-utils: clean up zlib.h usage a bit
>     
>     Hide zlib's crc32 in compr_zlib.c and mkfs.ubifs/compr.c.
>     
>     jffs2reader.c and mkfs.jffs2.c don't actually use zlib, so punt the
>     include from the file.
>     
>     mkfs.jffs2.c is implicitly using crc32 from zlib.h instead of the
>     local mtd_crc32, so fix the local usage.  otherwise we get warnings
>     about undefined crc32 because the file was redirecting the prototype.
>     
>     Signed-off-by: Mike Frysinger <vapier@gentoo.org>
>     Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> 
> Hartley, that commit applies OK to 1.4.0, could you give it a try?
> 
> Artem: Time for a v1.4.1 release?

Thanks. If it works, I'll back-port Mike's patch to 1.4.0 and tag it as
1.4.1. So 1.4.1 won't include tons of irrelevant commits.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Latest mkfs.jffs2 is broken.

[Peter Korsgaard]

>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

 >> Artem: Time for a v1.4.1 release?

 Artem> Thanks. If it works, I'll back-port Mike's patch to 1.4.0 and tag it as
 Artem> 1.4.1. So 1.4.1 won't include tons of irrelevant commits.

Great. An easy way to test:

git checkout v1.4.0
make WITHOUT_XATTR=1
./mkfs.jffs2 -e 0x20000 -b -s 0x1000 -d include -o test.jffs2
/usr/sbin/jffs2dump -b -c test.jffs2|head -n 4
Wrong hdr_crc  at  0x00000000, 0x954203f1 instead of 0xf060dc98
Wrong bitmask  at  0x00000004, 0x0000
Wrong hdr_crc  at  0x0000000c, 0xd10532a2 instead of 0xb427edcb
Wrong bitmask  at  0x00000010, 0x0000

git cherry-pick b864c387e8f1
make WITHOUT_XATTR=1
./mkfs.jffs2 -e 0x20000 -b -s 0x1000 -d include -o test.jffs2
/usr/sbin/jffs2dump -b -c test.jffs2|head -n 4
         Dirent     node at 0x0000000c, totlen 0x00000030, #pino     1, version     0, #ino         2, nsize        8, name common.h
         Inode      node at 0x0000003c, totlen 0x00000489, #ino      2, version     1, isize     2939, csize     1093, dsize     2939, offset        0
         Dirent     node at 0x000004c8, totlen 0x0000002f, #pino     1, version     1, #ino         3, nsize        7, name crc32.h
         Inode      node at 0x000004f8, totlen 0x0000011d, #ino      3, version     1, isize      294, csize      217, dsize      294, offset        0

-- 
Bye, Peter Korsgaard

Re: Latest mkfs.jffs2 is broken.

[Artem Bityutskiy]

On Fri, 2010-10-15 at 10:52 +0200, Peter Korsgaard wrote:
> >>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:
> 
>  >> Artem: Time for a v1.4.1 release?
> 
>  Artem> Thanks. If it works, I'll back-port Mike's patch to 1.4.0 and tag it as
>  Artem> 1.4.1. So 1.4.1 won't include tons of irrelevant commits.
> 
> Great. An easy way to test:

Done. You can find v1.4.1 tag, tarball, and announcement in the "news"
section of the web site.

-- 
Best Regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Latest mkfs.jffs2 is broken.

[Peter Korsgaard]

>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

Hi,

 >> Great. An easy way to test:

 Artem> Done. You can find v1.4.1 tag, tarball, and announcement in the "news"
 Artem> section of the web site.

Wow, a tarball right away - Thanks ;)

-- 
Bye, Peter Korsgaard

Re: Latest mkfs.jffs2 is broken.

[Peter Korsgaard]

>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

 Artem> On Fri, 2010-10-15 at 10:52 +0200, Peter Korsgaard wrote:
 >> >>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:
 >> 
 >> >> Artem: Time for a v1.4.1 release?
 >> 
 Artem> Thanks. If it works, I'll back-port Mike's patch to 1.4.0 and tag it as
 Artem> 1.4.1. So 1.4.1 won't include tons of irrelevant commits.
 >> 
 >> Great. An easy way to test:

 Artem> Done. You can find v1.4.1 tag, tarball, and announcement in the "news"
 Artem> section of the web site.

Thanks, but the 1.4.1 tarball is screwed - It contains a bunch of build
files which breaks the build if they aren't removed first.

diff -urpN mtd-utils-1.4.?|diffstat
 compr_zlib.c                       |    2 
 cscope.out                         |314340 +++++++++++++++++++++++++++++++++++++
 jffs2reader.c                      |    1 
 mkfs.jffs2.c                       |   49 
 mkfs.ubifs/compr.c                 |    1 
 tests/fs-tests/lib/tests.o         |binary
 tests/ubi-tests/.common.c.dep      |   38 
 tests/ubi-tests/.integ.c.dep       |   38 
 tests/ubi-tests/.io_basic.c.dep    |   33 
 tests/ubi-tests/.io_paral.c.dep    |   32 
 tests/ubi-tests/.io_read.c.dep     |   33 
 tests/ubi-tests/.io_update.c.dep   |   33 
 tests/ubi-tests/.mkvol_bad.c.dep   |   24 
 tests/ubi-tests/.mkvol_basic.c.dep |   24 
 tests/ubi-tests/.mkvol_paral.c.dep |   27 
 tests/ubi-tests/.rsvol.c.dep       |   33 
 tests/ubi-tests/.volrefcnt.c.dep   |   31 
 tests/ubi-tests/common.o           |binary
 tests/ubi-tests/integ              |binary
 tests/ubi-tests/io_basic           |binary
 tests/ubi-tests/io_paral           |binary
 tests/ubi-tests/io_read            |binary
 tests/ubi-tests/io_update          |binary
 tests/ubi-tests/libubi.a           |binary
 tests/ubi-tests/libubi.o           |binary
 tests/ubi-tests/mkvol_bad          |binary
 tests/ubi-tests/mkvol_basic        |binary
 tests/ubi-tests/mkvol_paral        |binary
 tests/ubi-tests/rsvol              |binary
 tests/ubi-tests/volrefcnt          |binary
 ubi-utils/cscope.out               |164861 +++++++++++++++++++
 ubi-utils/libmtd.a                 |binary
 32 files changed, 479573 insertions(+), 27 deletions(-)

Care to fix that?

-- 
Bye, Peter Korsgaard

Re: Latest mkfs.jffs2 is broken.

[Artem Bityutskiy]

On Mon, 2010-10-18 at 22:51 +0200, Peter Korsgaard wrote:
> Care to fix that?

Re-uploaded fixed tarball.

-- 
Best Regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Latest mkfs.jffs2 is broken.

[Peter Korsgaard]

>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

 Artem> On Mon, 2010-10-18 at 22:51 +0200, Peter Korsgaard wrote:
 >> Care to fix that?

 Artem> Re-uploaded fixed tarball.

Thanks!

-- 
Bye, Peter Korsgaard