From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Elsayed Subject: Re: [RFC] Per-file compression Date: Sun, 19 Apr 2015 20:00:35 -0700 Message-ID: References: <55318733.4000503@cyngn.com> <20150418230912.GC25265@thunk.org> Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7Bit To: linux-fsdevel@vger.kernel.org Return-path: Received: from plane.gmane.org ([80.91.229.3]:43137 "EHLO plane.gmane.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751429AbbDTDAx (ORCPT ); Sun, 19 Apr 2015 23:00:53 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Yk1x4-0005DR-O2 for linux-fsdevel@vger.kernel.org; Mon, 20 Apr 2015 05:00:51 +0200 Received: from 66-87-139-42.pools.spcsdns.net ([66-87-139-42.pools.spcsdns.net]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 20 Apr 2015 05:00:50 +0200 Received: from eternaleye by 66-87-139-42.pools.spcsdns.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 20 Apr 2015 05:00:50 +0200 Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Theodore Ts'o wrote: > On Sat, Apr 18, 2015 at 10:06:14AM +0200, Richard Weinberger wrote: >> On Sat, Apr 18, 2015 at 12:20 AM, Tom Marshall wrote: >> > We are running into space issues when upgrading Android devices to the >> > latest version. This issue is particularly troublesome for 64-bit >> > devices that initially shipped without 64-bit support, as the multi-lib >> > files nearly >> > double the space requirements. As it happens, many system files are >> > quite >> > compressible -- namely shared libs and apk files. So implementing >> > filesystem compression is a natural solution. > > Tom, have you verified whether or not this approach actually is > sufficient to address the problem you are trying to solve? > Specifically, (a) can you get enough space compression that it will > make things fit? Given that apk files are already compressed using > zip, I'm a bit surprised you would get enough of a compression savings > that it's going to allow to make things fit. And secondly (b) after > you decompress the shared libraries, is there enough memory that the > resulting system functions well. Often times these older Android > devices don't have a huge amount of memory, and so either with or > without the overhead of needing to decompress the entire file (which > can be solved with chunking, although this tends to reduce the > compression efficiency), is the system going to function well using > 64-bit binaries that take up a lot more room than the older 32-bit > binaries? APK files may be using the zip _format_, but it's not uncommon for the compression level to be zero. Honestly, _that_ sounds like a saner way of addressing the issue to me, especially if files are compressed whole rather than chunked for random access - set the compression level to 9 or something, since whichever level you add the compression at you've already lost the (IMO of dubious importance) speed benefit zero-compression zip gives. >> > So, I wrote a thing called 'zfile' that hooks into the VFS layer and >> > intercepts file_operations to do file (de)compression on the fly. When >> > a >> > file is opened, it reads and decompresses the data into memory. The >> > file >> > may be read, written, and mmaped in the usual way. If the contents are >> > changed, the data is compressed and written back. > > Do you really need to be able to rewrite the files? Life gets much > easier if you can assume that the compressed files are read-only; > especially once you start trying to use chunking, it *definitely* > becomes easier if you don't need to support modifying the compressed > files. And in the case of the system partition, this might be a safe > assumption,yes? > > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" > in the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html