Re: [RFC] Per-file compression

[Theodore Ts'o <tytso@mit.edu>]

On Sat, Apr 18, 2015 at 10:06:14AM +0200, Richard Weinberger wrote:
> On Sat, Apr 18, 2015 at 12:20 AM, Tom Marshall <tom@cyngn.com> wrote:
> > We are running into space issues when upgrading Android devices to the
> > latest version.  This issue is particularly troublesome for 64-bit devices
> > that initially shipped without 64-bit support, as the multi-lib files nearly
> > double the space requirements.  As it happens, many system files are quite
> > compressible -- namely shared libs and apk files.  So implementing
> > filesystem compression is a natural solution.

Tom, have you verified whether or not this approach actually is
sufficient to address the problem you are trying to solve?
Specifically, (a) can you get enough space compression that it will
make things fit?  Given that apk files are already compressed using
zip, I'm a bit surprised you would get enough of a compression savings
that it's going to allow to make things fit.  And secondly (b) after
you decompress the shared libraries, is there enough memory that the
resulting system functions well.  Often times these older Android
devices don't have a huge amount of memory, and so either with or
without the overhead of needing to decompress the entire file (which
can be solved with chunking, although this tends to reduce the
compression efficiency), is the system going to function well using
64-bit binaries that take up a lot more room than the older 32-bit
binaries?

> > So, I wrote a thing called 'zfile' that hooks into the VFS layer and
> > intercepts file_operations to do file (de)compression on the fly. When a
> > file is opened, it reads and decompresses the data into memory.  The file
> > may be read, written, and mmaped in the usual way.  If the contents are
> > changed, the data is compressed and written back.

Do you really need to be able to rewrite the files?  Life gets much
easier if you can assume that the compressed files are read-only;
especially once you start trying to use chunking, it *definitely*
becomes easier if you don't need to support modifying the compressed
files.  And in the case of the system partition, this might be a safe
assumption,yes?

					- Ted