[ANN] Squashfs 3.0 released

[ANN] Squashfs 3.0 released

[Phillip Lougher]

Hi,

Squashfs 3.0 has finally been released.  Squashfs 3.0 is a major  
improvement to Squashfs, and it addresses most of the issues that  
that have been raised, particularly the 4GB filesystem and file  
limit.  It can be obtained from the usual address http:// 
squashfs.sourceforge.net.  There is still some work to be done, in  
particular NFS support which I'll add as soon as I get time.  After  
this I'll consider resubmitting patches to the LKML.

 From the changelog, the improvements are as follows:

         1. Filesystems are no longer limited to 4 GB.  In
            theory 2^64 or 4 exabytes is now supported.

         2. Files are no longer limited to 4 GB.  In theory the maximum
            file size is 4 exabytes.

         3. Metadata (inode table and directory tables) are no longer
            restricted to 16 Mbytes.

         4. Hardlinks are now suppported.

         5. Nlink counts are now supported.

         6. Readdir now returns '.' and '..' entries.

         7. Special support for files larger than 256 MB has been  
added to
            the Squashfs kernel code for faster read access.

         8. Inode numbers are now stored within the inode rather than  
being
            computed from inode location on disk (this is not so much an
            improvement, but a change forced by the previously listed
            improvements).

Phillip Lougher

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

On 17 Mar 2006, at 01:05, Andreas Dilger wrote:

> On Mar 17, 2006  00:45 +0000, Phillip Lougher wrote:
>> Squashfs 3.0 has finally been released.  Squashfs 3.0 is a major
>> improvement to Squashfs, and it addresses most of the issues that
>> that have been raised, particularly the 4GB filesystem and file
>> limit.
>
> Sometimes it is useful for the casual reader if you include a brief
> blurb about what exactly squashfs is... :-)
>
Ok, for those who are interested, old blurb  from the README follows:

"Squashfs is a highly compressed read-only filesystem for Linux.
It uses zlib compression to compress both files, inodes and directories.
Inodes in the system are very small and all blocks are packed to  
minimise
data overhead. Block sizes greater than 4K are supported up to a maximum
of 64K.

Squashfs is intended for general read-only filesystem use, for archival
use (i.e. in cases where a .tar.gz file may be used), and in constrained
block device/memory systems (e.g. embedded systems) where low  
overhead is
needed."

At the moment it tends to be used for embedded systems, and liveCDs.

Phillip

Re: [ANN] Squashfs 3.0 released

[Samuel Masham]

Hi Phillip,

On 17/03/06, Phillip Lougher <phillip@lougher.org.uk> wrote:
> and in constrained
> block device/memory systems (e.g. embedded systems) where low
> overhead is
> needed."
>
> At the moment it tends to be used for embedded systems, and liveCDs.

>From the embedded side here...

Have you any idea how the performance of version 3.0 stack up against 2.1?

You haven't updated the readme.performance file yet :)

thanks

Samuel

ps Looking forward to seeing squashfs in main line soon :)

Re: [ANN] Squashfs 3.0 released

[Jörn Engel]

On Fri, 17 March 2006 00:45:42 +0000, Phillip Lougher wrote:
> 
> Squashfs 3.0 has finally been released.  Squashfs 3.0 is a major  
> improvement to Squashfs, and it addresses most of the issues that  
> that have been raised, particularly the 4GB filesystem and file  
> limit.  It can be obtained from the usual address http:// 
> squashfs.sourceforge.net.  There is still some work to be done, in  
> particular NFS support which I'll add as soon as I get time.  After  
> this I'll consider resubmitting patches to the LKML.
> 
> From the changelog, the improvements are as follows:
> 
>         1. Filesystems are no longer limited to 4 GB.  In
>            theory 2^64 or 4 exabytes is now supported.
> 
>         2. Files are no longer limited to 4 GB.  In theory the maximum
>            file size is 4 exabytes.
> 
>         3. Metadata (inode table and directory tables) are no longer
>            restricted to 16 Mbytes.
> 
>         4. Hardlinks are now suppported.
> 
>         5. Nlink counts are now supported.
> 
>         6. Readdir now returns '.' and '..' entries.
> 
>         7. Special support for files larger than 256 MB has been  
> added to
>            the Squashfs kernel code for faster read access.
> 
>         8. Inode numbers are now stored within the inode rather than  
> being
>            computed from inode location on disk (this is not so much an
>            improvement, but a change forced by the previously listed
>            improvements).

Nice list of improvements.  The one still painfully missing is a
fixed-endianness disk format.  Would have been a good time to make an
incompatible change and decide on one or the other.

Jörn

-- 
And spam is a useful source of entropy for /dev/random too!
-- Jasmine Strong

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

On 17 Mar 2006, at 10:40, Jörn Engel wrote:
>
> Nice list of improvements.

Thanks.

> The one still painfully missing is a
> fixed-endianness disk format.

We had that argument last year.

Re: [ANN] Squashfs 3.0 released

[Jörn Engel]

On Fri, 17 March 2006 11:16:48 +0000, Phillip Lougher wrote:
> 
> >The one still painfully missing is a
> >fixed-endianness disk format.
> 
> We had that argument last year.

Yes, I remember.  What I don't remember is your opinion on the matter.
Did we reach some sort of conclusion?

Jörn

-- 
He who knows others is wise.
He who knows himself is enlightened.
-- Lao Tsu

Re: [ANN] Squashfs 3.0 released

[Xavier Bestel]

On Fri, 2006-03-17 at 01:45, Phillip Lougher wrote:
> Hi,
> 
> Squashfs 3.0 has finally been released.  Squashfs 3.0 is a major  
> improvement to Squashfs, and it addresses most of the issues that  
> that have been raised, particularly the 4GB filesystem and file  
> limit.  It can be obtained from the usual address http:// 
> squashfs.sourceforge.net.  There is still some work to be done, in  
> particular NFS support which I'll add as soon as I get time.  After  
> this I'll consider resubmitting patches to the LKML.
> 
>  From the changelog, the improvements are as follows:
> 
>          1. Filesystems are no longer limited to 4 GB.  In
>             theory 2^64 or 4 exabytes is now supported.

Isn't 2^64 16 exabytes ?

	Xav

Re: [ANN] Squashfs 3.0 released

[Jeff Garzik]

Jörn Engel wrote:
> On Fri, 17 March 2006 11:16:48 +0000, Phillip Lougher wrote:
> 
>>>The one still painfully missing is a
>>>fixed-endianness disk format.
>>
>>We had that argument last year.
> 
> 
> Yes, I remember.  What I don't remember is your opinion on the matter.
> Did we reach some sort of conclusion?

Fixed endian isn't necessarily a requirement.  Detectable endian is.  As 
long as (a) the filesystem mkfs notes the endian-ness and (b) the kernel 
filesystem code properly handles both types of endian, life is fine.

For SquashFS, though, I would think that fixed endian would be easy. 
Since it is byte-packed, just handle endian as you unpack.

	Jeff

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

On 17 Mar 2006, at 16:00, Jeff Garzik wrote:

> Jörn Engel wrote:
>> On Fri, 17 March 2006 11:16:48 +0000, Phillip Lougher wrote:
>>>> The one still painfully missing is a
>>>> fixed-endianness disk format.
>>>
>>> We had that argument last year.
>> Yes, I remember.  What I don't remember is your opinion on the  
>> matter.
>> Did we reach some sort of conclusion?
>
> Fixed endian isn't necessarily a requirement.  Detectable endian  
> is.  As long as (a) the filesystem mkfs notes the endian-ness and  
> (b) the kernel filesystem code properly handles both types of  
> endian, life is fine.
>
That's what is currently done.  There are two filesystem formats, big  
endian (donated by Squashfs magic of 'sqsh') and little endian  
(denoted by Squashfs magic of 'hsqs').  The kernel code detects the  
filesystem endianness and swaps if necessary.

> For SquashFS, though, I would think that fixed endian would be  
> easy. Since it is byte-packed, just handle endian as you unpack.
>
That's what is currently done.  The data is swapped if necessary at  
unpack time.  Afterwards no further swapping is performed.

Phillip

Re: [ANN] Squashfs 3.0 released

[Jeff Garzik]

Phillip Lougher wrote:
> On 17 Mar 2006, at 16:00, Jeff Garzik wrote:
>> Jörn Engel wrote:
>>>>> The one still painfully missing is a
>>>>> fixed-endianness disk format.

>> Fixed endian isn't necessarily a requirement.  Detectable endian  is.  
>> As long as (a) the filesystem mkfs notes the endian-ness and  (b) the 
>> kernel filesystem code properly handles both types of  endian, life is 
>> fine.
>>
> That's what is currently done.  There are two filesystem formats, big  
> endian (donated by Squashfs magic of 'sqsh') and little endian  (denoted 
> by Squashfs magic of 'hsqs').  The kernel code detects the  filesystem 
> endianness and swaps if necessary.

Well, then, I don't see a need to change anything.  As I said, 
[consistent and] detectable endian is the real requirement.  For 
SquashFS's users, I would think they would prefer the current situation 
(selectable endian) to fixed endian, because many SquashFS users need to 
squeeze every ounce of performance out of severely resource-constrained 
devices.

I have two routers, ADM5120-based Edimax and LinkSys WRT54G v5, both of 
which have a mere 2MB of flash, and both use SquashFS to maximize that 
space.  And both are el cheapo, slow embedded processors that run far 
slower than 300Mhz.  I look askance at anyone who wants to make an 
arbitrary filesystem design decision imposing tons of bytesex upon these 
lowly devices.

	Jeff

Re: [ANN] Squashfs 3.0 released

[Willy Tarreau]

Hi,

On Fri, Mar 17, 2006 at 12:25:44PM -0500, Jeff Garzik wrote:
> I have two routers, ADM5120-based Edimax and LinkSys WRT54G v5, both of 
> which have a mere 2MB of flash, and both use SquashFS to maximize that 
> space.  And both are el cheapo, slow embedded processors that run far 
> slower than 300Mhz.  I look askance at anyone who wants to make an 
> arbitrary filesystem design decision imposing tons of bytesex upon these 
> lowly devices.

100% agreed. I love squashfs because it's tiny and efficient, and I
would not want to see it getting heavy.

BTW, has someone tried to port LZMA to squashfs ? I tried so on
bzImage + initramfs, and got something like a 27% smaller image.
That would mean about 500 kB on a 2 MB image.

> 	Jeff

Willy

Re: [ANN] Squashfs 3.0 released

[Jan Engelhardt]

Hi,


nit:
Change the blob on http://squashfs.sf.net/ to mention 3.0 as most recent 
version (currently, 2.2r2 is still advertized).


Jan Engelhardt
-- 

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

On 17 Mar 2006, at 20:39, Willy Tarreau wrote:

> Hi,
>
> On Fri, Mar 17, 2006 at 12:25:44PM -0500, Jeff Garzik wrote:
>> I have two routers, ADM5120-based Edimax and LinkSys WRT54G v5,  
>> both of
>> which have a mere 2MB of flash, and both use SquashFS to maximize  
>> that
>> space.  And both are el cheapo, slow embedded processors that run far
>> slower than 300Mhz.  I look askance at anyone who wants to make an
>> arbitrary filesystem design decision imposing tons of bytesex upon  
>> these
>> lowly devices.
>
> 100% agreed. I love squashfs because it's tiny and efficient, and I
> would not want to see it getting heavy.
>

Thanks!  I'm determined to keep Squashfs as small/efficient as  
possible, as that's where I see its main advantages.

> BTW, has someone tried to port LZMA to squashfs ? I tried so on
> bzImage + initramfs, and got something like a 27% smaller image.
> That would mean about 500 kB on a 2 MB image.
>

There are quite a few third-party LZMA patches for Squashfs.  OpenWRT  
(for wifi routers) uses an LZMA patched Squashfs.

I've been asked about using LZMA for Squashfs quite a few times.   
The  major problem as I see it with LZMA is it is not supported by  
the Linux kernel, and I don't want to use anything that might prevent  
it being merged!  At the moment I'm happy with the current situation  
where there's third party patches available for those that want or  
need it.  Of course if the situation changed with LZMA going into the  
Linux kernel, then I'd see no problem in using it in Squashfs.

Phillip

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

On 17 Mar 2006, at 20:39, Willy Tarreau wrote:

> Hi,
>
> On Fri, Mar 17, 2006 at 12:25:44PM -0500, Jeff Garzik wrote:
>> I have two routers, ADM5120-based Edimax and LinkSys WRT54G v5,  
>> both of
>> which have a mere 2MB of flash, and both use SquashFS to maximize  
>> that
>> space.  And both are el cheapo, slow embedded processors that run far
>> slower than 300Mhz.  I look askance at anyone who wants to make an
>> arbitrary filesystem design decision imposing tons of bytesex upon  
>> these
>> lowly devices.
>
> 100% agreed. I love squashfs because it's tiny and efficient, and I
> would not want to see it getting heavy.
>

Thanks!  I'm determined to keep Squashfs as small/efficient as  
possible, as that's where I see its main advantages.

> BTW, has someone tried to port LZMA to squashfs ? I tried so on
> bzImage + initramfs, and got something like a 27% smaller image.
> That would mean about 500 kB on a 2 MB image.
>

There are quite a few third-party LZMA patches for Squashfs.  OpenWRT  
(for wifi routers) uses an LZMA patched Squashfs.

I've been asked about using LZMA for Squashfs quite a few times.   
The  major problem as I see it with LZMA is it is not supported by  
the Linux kernel, and I don't want to use anything that might prevent  
it being merged!  At the moment I'm happy with the current situation  
where there's third party patches available for those that want or  
need it.  Of course if the situation changed with LZMA going into the  
Linux kernel, then I'd see no problem in using it in Squashfs.

Phillip

Re: [ANN] Squashfs 3.0 released

[Pavel Machek]

On Fri 17-03-06 12:25:44, Jeff Garzik wrote:
> Phillip Lougher wrote:
> >On 17 Mar 2006, at 16:00, Jeff Garzik wrote:
> >>Jörn Engel wrote:
> >>>>>The one still painfully missing is a
> >>>>>fixed-endianness disk format.
> 
> >>Fixed endian isn't necessarily a requirement.  
> >>Detectable endian  is.  As long as (a) the filesystem 
> >>mkfs notes the endian-ness and  (b) the kernel 
> >>filesystem code properly handles both types of  endian, 
> >>life is fine.
> >>
> >That's what is currently done.  There are two filesystem 
> >formats, big  endian (donated by Squashfs magic of 
> >'sqsh') and little endian  (denoted by Squashfs magic of 
> >'hsqs').  The kernel code detects the  filesystem 
> >endianness and swaps if necessary.
> 
> Well, then, I don't see a need to change anything.  As I 
> said, [consistent and] detectable endian is the real 
> requirement.  For SquashFS's users, I would think they 
> would prefer the current situation (selectable endian) to 
> fixed endian, because many SquashFS users need to squeeze 
> every ounce of performance out of severely 
> resource-constrained devices.
> 
> I have two routers, ADM5120-based Edimax and LinkSys 
> WRT54G v5, both of which have a mere 2MB of flash, and 
> both use SquashFS to maximize that space.  And both are 
> el cheapo, slow embedded processors that run far slower 
> than 300Mhz.  I look askance at anyone who wants to make 
> an arbitrary filesystem design decision imposing tons of 
> bytesex upon these lowly devices.

gzip is already pretty expensive, I'd say. Is not byteswap lost in
noise?

-- 
Thanks, Sharp!

Re: [ANN] Squashfs 3.0 released

[Francesco Biscani]

Hi Phillip,

is there a way to use squashfs regularly as an ordinary user? I would love to 
use it as a replacement for tar archives, but as long as one needs to be root 
in order to mount the squashed fs it is quite impractical. Maybe a FUSE 
gateway would be a feasible solution?

BTW, shoot me in the face if this is a faq and has already been answered 
somewhere :)

  Francesco

-- 
Dr. Francesco Biscani
Dipartimento di Astronomia
Università di Padova
biscani@pd.astro.it

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

Pavel Machek wrote:
> On Fri 17-03-06 12:25:44, Jeff Garzik wrote:
>>I have two routers, ADM5120-based Edimax and LinkSys 
>>WRT54G v5, both of which have a mere 2MB of flash, and 
>>both use SquashFS to maximize that space.  And both are 
>>el cheapo, slow embedded processors that run far slower 
>>than 300Mhz.  I look askance at anyone who wants to make 
>>an arbitrary filesystem design decision imposing tons of 
>>bytesex upon these lowly devices.
> 
> 
> gzip is already pretty expensive, I'd say. Is not byteswap lost in
> noise?
>

Perhaps, but almost all the byteswap is performed on the metadata side, 
reading directories and inodes, where nearly every byte will need to be 
swapped.  As inodes are compacted and compressed in 8 KB blocks, and are 
on average 15 bytes in size, for each 8 KB decompress you're potentially 
doing 8192/15 inode byteswaps.  This is probably sufficent to affect 
directory search and lookup on a slow processor.

The data path is all gzip overhead (64K datablocks), there is no 
byteswap taking place except for the block size integer.  Therefore 
byteswap doesn't have any affect on reading data.

Phillip

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

Al Viro wrote:

>On Tue, Mar 21, 2006 at 04:01:51PM +0000, Phillip Lougher wrote:
>  
>
>>Perhaps, but almost all the byteswap is performed on the metadata side, 
>>reading directories and inodes, where nearly every byte will need to be 
>>swapped.  As inodes are compacted and compressed in 8 KB blocks, and are 
>>on average 15 bytes in size, for each 8 KB decompress you're potentially 
>>doing 8192/15 inode byteswaps.  This is probably sufficent to affect 
>>directory search and lookup on a slow processor.
>>    
>>
>
>Oh, please...  Conversion from known endianness to host-endian is considerably
>faster than checking flag + branch + two variants, not to mention being
>smaller.
>  
>
It's one flag check, and one set of swap code actually.  The point that 
was being made is it is better to avoid byte swapping if possible.

Re: [ANN] Squashfs 3.0 released

[Pavel Machek]

On Út 21-03-06 19:08:21, Phillip Lougher wrote:
> Al Viro wrote:
> 
> >On Tue, Mar 21, 2006 at 04:01:51PM +0000, Phillip Lougher wrote:
> > 
> >
> >>Perhaps, but almost all the byteswap is performed on the metadata side, 
> >>reading directories and inodes, where nearly every byte will need to be 
> >>swapped.  As inodes are compacted and compressed in 8 KB blocks, and are 
> >>on average 15 bytes in size, for each 8 KB decompress you're potentially 
> >>doing 8192/15 inode byteswaps.  This is probably sufficent to affect 
> >>directory search and lookup on a slow processor.
> >>   
> >>
> >
> >Oh, please...  Conversion from known endianness to host-endian is 
> >considerably
> >faster than checking flag + branch + two variants, not to mention being
> >smaller.
> > 
> >
> It's one flag check, and one set of swap code actually.  The point that 
> was being made is it is better to avoid byte swapping if possible.

Al is right. Unconditional swap is probably faster than
branch. Avoiding swaps is nice, but avoiding branches is probably more
important.

Can you try to benchmark it? I believe it is going to be lost in
noise, slow cpus or not.
								Pavel
-- 
Picture of sleeping (Linux) penguin wanted...

Re: [ANN] Squashfs 3.0 released

[Phillip Lougher]

Pavel Machek wrote:
> 
> Al is right. Unconditional swap is probably faster than
> branch. Avoiding swaps is nice, but avoiding branches is probably more
> important.

Quite possible.

> 
> Can you try to benchmark it? I believe it is going to be lost in
> noise, slow cpus or not.

Good idea, I'll try to benchmark it (on a slow CPU if I can find one :-) 
).  It will probably make no difference.

I don't want the lack of a fixed endianness on disk to become a problem. 
   I personally don't think the use of, or lack of a fixed endianness to 
be that important, but I'd prefer not to change the current situation 
and adopt a fixed format.  I use big endian systems almost exclusively, 
and I don't like the way fixed formats always tend to be little-endian.

Re: [ANN] Squashfs 3.0 released

[Al Viro]

On Tue, Mar 21, 2006 at 08:03:38PM +0000, Phillip Lougher wrote:
> Pavel Machek wrote:
> >
> >Al is right. Unconditional swap is probably faster than
> >branch. Avoiding swaps is nice, but avoiding branches is probably more
> >important.
> 
> Quite possible.
> 
> >
> >Can you try to benchmark it? I believe it is going to be lost in
> >noise, slow cpus or not.
> 
> Good idea, I'll try to benchmark it (on a slow CPU if I can find one :-) 
> ).  It will probably make no difference.
> 
> I don't want the lack of a fixed endianness on disk to become a problem. 
>   I personally don't think the use of, or lack of a fixed endianness to 
> be that important, but I'd prefer not to change the current situation 
> and adopt a fixed format.  I use big endian systems almost exclusively, 
> and I don't like the way fixed formats always tend to be little-endian.

You mean, like IP?  Or NFS?  Or XFS?  Or any number of other big-endian
data layouts?  Make it fixed to big-endian - no problem with that...

Re: [ANN] Squashfs 3.0 released

[Pavel Machek]

Hi!

> >Can you try to benchmark it? I believe it is going to be lost in
> >noise, slow cpus or not.
> 
> Good idea, I'll try to benchmark it (on a slow CPU if I can find one :-) 
> ).  It will probably make no difference.
> 
> I don't want the lack of a fixed endianness on disk to become a problem. 
>   I personally don't think the use of, or lack of a fixed endianness to 
> be that important, but I'd prefer not to change the current situation 
> and adopt a fixed format.  I use big endian systems almost exclusively, 
> and I don't like the way fixed formats always tend to be little-endian.

Fix it to big-endian, then. Network protocols are big-endian, anyway,
and PCs tend to be so fast that byteswap will be lost in cache misses,
anyway.

[Funny, it looks like all the big-endian machines are slow :-)))]

								Pavel
-- 
Picture of sleeping (Linux) penguin wanted...

Re: [ANN] Squashfs 3.0 released

[Al Viro]

On Tue, Mar 21, 2006 at 09:15:41PM +0100, Pavel Machek wrote:
> Hi!
> 
> > >Can you try to benchmark it? I believe it is going to be lost in
> > >noise, slow cpus or not.
> > 
> > Good idea, I'll try to benchmark it (on a slow CPU if I can find one :-) 
> > ).  It will probably make no difference.
> > 
> > I don't want the lack of a fixed endianness on disk to become a problem. 
> >   I personally don't think the use of, or lack of a fixed endianness to 
> > be that important, but I'd prefer not to change the current situation 
> > and adopt a fixed format.  I use big endian systems almost exclusively, 
> > and I don't like the way fixed formats always tend to be little-endian.
> 
> Fix it to big-endian, then. Network protocols are big-endian, anyway,
> and PCs tend to be so fast that byteswap will be lost in cache misses,
> anyway.

Note that "sometimes we swap" approach tends to create tons of bugs.
It's much easier to keep track of "this variable is host-endian, this
one is big-endian" and have appropriate conversions where needed.  Trying
to keep track of how many times we need to swap on this, this and that
codepath, OTOH, almost always ends up buggy.

Re: [ANN] Squashfs 3.0 released

[Andreas Dilger]

On Mar 21, 2006  20:03 +0000, Phillip Lougher wrote:
> I don't want the lack of a fixed endianness on disk to become a problem. 
>   I personally don't think the use of, or lack of a fixed endianness to 
> be that important, but I'd prefer not to change the current situation 
> and adopt a fixed format.  I use big endian systems almost exclusively, 
> and I don't like the way fixed formats always tend to be little-endian.

If you want to squeak every last ounce of performance out of the filesystem,
just have it declare two filesystem types - one for the little-endian, and
one for the bit endian.  Generate one of them via "sed" from the other, to
rename the functions, exports, etc, so they don't conflict.  Then, depending
on the superblock magic it will mount the right filesystem, depending on
endianness.  Since they are separate filesystems, normally only one module
or the other need to be loaded at a time, and there is no runtime overhead.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: [ANN] Squashfs 3.0 released

[Jan Engelhardt]

>
>You mean, like IP?  Or NFS?  Or XFS?  Or any number of other big-endian
>data layouts?  Make it fixed to big-endian - no problem with that...
>
And most machines are little endian. So statistically, the world swapped 
more than it would have to.



Jan Engelhardt
-- 

Re: [ANN] Squashfs 3.0 released

[Jeff Garzik]

Jan Engelhardt wrote:
>>You mean, like IP?  Or NFS?  Or XFS?  Or any number of other big-endian
>>data layouts?  Make it fixed to big-endian - no problem with that...
>>
> 
> And most machines are little endian. So statistically, the world swapped 
> more than it would have to.

No, rather, the more powerful machines do the swapping.

	Jeff

Re: [ANN] Squashfs 3.0 released

[Al Viro]

On Tue, Mar 21, 2006 at 11:01:58PM +0100, Jan Engelhardt wrote:
> >
> >You mean, like IP?  Or NFS?  Or XFS?  Or any number of other big-endian
> >data layouts?  Make it fixed to big-endian - no problem with that...
> >
> And most machines are little endian. So statistically, the world swapped 
> more than it would have to.

... and on most of them that costs how many cycles, again?  How much do
test and branch cost?

That is not to mention boxen with instructions along the lines of
load-and-swap/swap-and-store...

Re: [ANN] Squashfs 3.0 released

[Jörn Engel]

On Tue, 21 March 2006 14:28:53 -0700, Andreas Dilger wrote:
> On Mar 21, 2006  20:03 +0000, Phillip Lougher wrote:
> > I don't want the lack of a fixed endianness on disk to become a problem. 
> >   I personally don't think the use of, or lack of a fixed endianness to 
> > be that important, but I'd prefer not to change the current situation 
> > and adopt a fixed format.  I use big endian systems almost exclusively, 
> > and I don't like the way fixed formats always tend to be little-endian.
> 
> If you want to squeak every last ounce of performance out of the filesystem,
> just have it declare two filesystem types - one for the little-endian, and
> one for the bit endian.  Generate one of them via "sed" from the other, to
> rename the functions, exports, etc, so they don't conflict.  Then, depending
> on the superblock magic it will mount the right filesystem, depending on
> endianness.  Since they are separate filesystems, normally only one module
> or the other need to be loaded at a time, and there is no runtime overhead.

That would be an interesting idea for quite another purpose:
measurement.

So far, there has been a lack of numbers in this thread.  Al mentioned
that conditional branches can be more expensive and I usually trust
his words, but actual cold hard numbers would help more.

> 	"unlisted-recipients: no To-header on input <;, Jeff Garzik" <jeff@garzik.org>,

I fixed this up.  No idea what garbled the header.

Jörn

-- 
My second remark is that our intellectual powers are rather geared to
master static relations and that our powers to visualize processes
evolving in time are relatively poorly developed.
-- Edsger W. Dijkstra