[RFC] Per-file compression

[Tom Marshall <tom@cyngn.com>]

I'd like to get some thoughts on whether having transparent per-file 
compression implemented in the kernel would be desirable.

Details:

We are running into space issues when upgrading Android devices to the 
latest version.  This issue is particularly troublesome for 64-bit 
devices that initially shipped without 64-bit support, as the multi-lib 
files nearly double the space requirements.  As it happens, many system 
files are quite compressible -- namely shared libs and apk files.  So 
implementing filesystem compression is a natural solution.

After investigating several well known compression solutions, we did not 
find any that met our goals:

* Compression should be transparent to user space.  It would be possible 
to identify subsets of the filesystem that compress well and handle 
compression in user space.  However, doing this involves repeating 
similar work in different areas of code, and can never fully utilize 
compression across the entire filesystem.

* Compression must work with existing Android filesystems.  There are 
filesystems that have native compression.  However, switching to a new 
filesystem on a released device is not really desirable.

* Compression should be implemented as transparently as possible. cloop 
is filesystem independent, but it makes the underlying device 
read-only.  This could be mitigated somewhat with overlayfs, but that 
would be a rather complex solution.

* Compression should be independent of the filesystem.  e2compr may work 
for ext2, but it seems abandoned and does not address other filesystem 
types.

So, I wrote a thing called 'zfile' that hooks into the VFS layer and 
intercepts file_operations to do file (de)compression on the fly. When a 
file is opened, it reads and decompresses the data into memory.  The 
file may be read, written, and mmaped in the usual way.  If the contents 
are changed, the data is compressed and written back.  A working patch 
for an older kernel version may be found at: 
http://review.cyanogenmod.org/#/c/95220/

Note this is mostly just a prototype at the moment.  I'm fully aware 
that it has some bugs and limitations.  Pretty major ones, I'm sure.  
However, if this is something that is generally desirable, I would look 
forward to working with the VFS maintainers to make it merge worthy.

Thanks!