SquashFS on ARM9

SquashFS on ARM9

[Dick Hollenbeck]

Dear Philip et. al,


I am using 2.6.30.5 on an ARM9 with squashfs.

The squashFS image is built with x86_64 form of

        mksquashfs version 4.1-CVS (2009/09/20)

a 64 bit linux program.

The image mounts into the 64 bit development system OK and I can read a 
particular file.  There are no problems on the development system, only 
the ARM9.

When mounted on ARM9 with said kernel, I get a read error reading that 
same particular file:

dmesg:

SQUASHFS error: zlib_inflate error, data probably corrupt
SQUASHFS error: squashfs_read_data failed to read block 0x50581
SQUASHFS error: Unable to read fragment cache entry [50581]
SQUASHFS error: Unable to read page, block 50581, size b6f3
SQUASHFS error: Unable to read fragment cache entry [50581]
SQUASHFS error: Unable to read page, block 50581, size b6f3
SQUASHFS error: Unable to read fragment cache entry [50581]
SQUASHFS error: Unable to read page, block 50581, size b6f3


That image resides on a CF card mounted with loop on the target ARM9.  
If I unmount the image, I can SSH copy it out off the flash back to the 
development system and do a binary compare on it there against the 
original and it is not corrupted.  Remember that the orginal mounts fine 
on the development system.  I think this means that the CF card is not 
corrupted.

Am I running into an alignment bug on the ARM9 or an endianess issue?  
(ARM9 is lsbyte first as far as I thought, same as the x86_64 Ubuntu 
Karmic development system.)

I checked the squashfs kernel block.c file against the latest mainline 
and they agree.

Please offer help.

Thanks,

Dick

Re: SquashFS on ARM9

[Dick Hollenbeck]

Dick Hollenbeck wrote:
>
> Dear Philip et. al,
>
>
> I am using 2.6.30.5 on an ARM9 with squashfs.
>
> The squashFS image is built with x86_64 form of
>
>        mksquashfs version 4.1-CVS (2009/09/20)
>
> 64 bit linux program.
 >  There are no problems on the development system, only the ARM9.



OK I got more news:

The image works OK if I build it with the 32 version of mksquashs on x86.

Amazing, there are differences in file format other than endian-ness?

So we have at least 3 forms of the file system image now?  I am 
disappointed.  One would have sufficed.  I don't even buy the need for 
LSbyte vs MSbyte being needed.  There should have been only one form of 
the FS image.

Ouch, this seems amateurish.   Can I assume it is the bitfields?   Get 
rid of the bitfields!

Use #defines and OR and AND them please.

In the mksquashfs program, simply create some ints

(1<<a)
(1<<b)

etc using #defines.


Andrew Morton warned you about this years ago.  You have the bitfields 
using defines, its just that you control where the bits are:

#define MBIT (1<<2)

struct s {
    int32_t val;
} r;


r.val |= MBIT;

How hard is that really?

For now I don't need anymore help.  Sorry if I burned down my bridge, I 
only intend to offer strategy to improve the filesystem.

Dick

Re: SquashFS on ARM9

[Phillip Lougher]

On Sun, Feb 7, 2010 at 7:18 AM, Dick Hollenbeck <dick@softplc.com> wrote:
> Dick Hollenbeck wrote:

> The image works OK if I build it with the 32 version of mksquashs on x86 vs x86_64 .
>
> Amazing, there are differences in file format other than endian-ness?

If you've found a difference in filesystems built by mksquashfs on x86
vs x86_64, then it is a *bug*.

>
> So we have at least 3 forms of the file system image now?  I am
> disappointed.  One would have sufficed.  I don't even buy the need for
> LSbyte vs MSbyte being needed.  There should have been only one form of the
> FS image.
>
> Ouch, this seems amateurish.   Can I assume it is the bitfields?   Get rid
> of the bitfields!
>

No need for insults, especially as it is based on an outdated
understanding.  Squashfs became fixed endian, and lost the bit-fields
in Squashfs 4.0.

> For now I don't need anymore help.  Sorry if I burned down my bridge, I only
> intend to offer strategy to improve the filesystem.
>

I like constructive feedback, excessively critical feedback is usually
self-defeating as it generally gets ignored.  By saying "Sorry if I
burned down my bridge" means you obviously understand this, and so it
is weird you bothered to send the email.

If you'd like the bug tracked down please raise it on the
squashfs.sourceforge.net bug tracker.  Include as much information as
you can including distribution, architecture, Mksquashfs options etc.
It would be best if you can provide a testcase set of source
directories that generate differing Squashfs filesystems on
x86/x86_64.

Phillip

Re: SquashFS on ARM9

[Dick Hollenbeck]

Phillip Lougher wrote:
> On Sun, Feb 7, 2010 at 7:18 AM, Dick Hollenbeck <dick@softplc.com> wrote:
>   
>> Dick Hollenbeck wrote:
>>     
>
>   
>> The image works OK if I build it with the 32 version of mksquashs on x86 vs x86_64 .
>>
>> Amazing, there are differences in file format other than endian-ness?
>>     
>
> If you've found a difference in filesystems built by mksquashfs on x86
> vs x86_64, then it is a *bug*.
>
>   
>> So we have at least 3 forms of the file system image now?  I am
>> disappointed.  One would have sufficed.  I don't even buy the need for
>> LSbyte vs MSbyte being needed.  There should have been only one form of the
>> FS image.
>>
>> Ouch, this seems amateurish.   Can I assume it is the bitfields?   Get rid
>> of the bitfields!
>>
>>     
>
> No need for insults, especially as it is based on an outdated
> understanding.  Squashfs became fixed endian, and lost the bit-fields
> in Squashfs 4.0.
>
>   
>> For now I don't need anymore help.  Sorry if I burned down my bridge, I only
>> intend to offer strategy to improve the filesystem.
>>
>>     
>
> I like constructive feedback, excessively critical feedback is usually
> self-defeating as it generally gets ignored.  By saying "Sorry if I
> burned down my bridge" means you obviously understand this, and so it
> is weird you bothered to send the email.
>
> If you'd like the bug tracked down please raise it on the
> squashfs.sourceforge.net bug tracker.  Include as much information as
> you can including distribution, architecture, Mksquashfs options etc.
> It would be best if you can provide a testcase set of source
> directories that generate differing Squashfs filesystems on
> x86/x86_64.
>
> Phillip
>   

My objection and criticism was based on my perception that *unnecessary 
risk had been taken* in using the bitfields.  I found no easy way to 
follow continuity of squashfs evolution, no mailing list, history doc, 
and made some assumptions based on an old LKML posting warning against 
bitfields.  I've been bit by bitfields myself in the past.  Typically 
human, my mistake was the same I was criticizing about, *taking 
unnecessary risk* when I made the assumptions.

I am sorry Phillip, your work is appreciated, your pay is too low, and 
you did not deserve this.

Dick