From mboxrd@z Thu Jan 1 00:00:00 1970 From: Richard Weinberger Subject: Re: [RFC] Per-file compression Date: Mon, 20 Apr 2015 18:51:03 +0200 Message-ID: <55352E77.4080100@nod.at> References: <55318733.4000503@cyngn.com> <20150418145828.GA13809@eden.sea.cyngn.com> <55327324.3030400@nod.at> <20150420145357.GA10981@boyd> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: Tom Marshall , linux-fsdevel To: Tyler Hicks Return-path: Received: from a.ns.miles-group.at ([95.130.255.143]:65275 "EHLO radon.swed.at" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753888AbbDTQvH (ORCPT ); Mon, 20 Apr 2015 12:51:07 -0400 In-Reply-To: <20150420145357.GA10981@boyd> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Am 20.04.2015 um 16:53 schrieb Tyler Hicks: > On 2015-04-18 17:07:16, Richard Weinberger wrote: >> Hi! >> >> Am 18.04.2015 um 16:58 schrieb Tom Marshall: >>> On Sat, Apr 18, 2015 at 01:41:09PM +0200, Richard Weinberger wrote: >>>> On Sat, Apr 18, 2015 at 12:20 AM, Tom Marshall wrote: >>>>> So, I wrote a thing called 'zfile' that hooks into the VFS layer and >>>>> intercepts file_operations to do file (de)compression on the fly. When a >>>>> file is opened, it reads and decompresses the data into memory. The file >>>>> may be read, written, and mmaped in the usual way. If the contents are >>>>> changed, the data is compressed and written back. A working patch for an >>>>> older kernel version may be found at: >>>>> http://review.cyanogenmod.org/#/c/95220/ >>>> >>>> So, I've extracted the patch from that website and gave a quick review. >>>> >>>> I'm pretty sure VFS folks will hate the VFS layering you do. >>> >>> This, I'm afraid, is the biggest obstacle to such a solution. I know that >>> OverlayFS has been merged, so filesystem stacking is acceptable. Perhaps >>> there would be a way to design a filesystem that stacks compression? >> >> That's why I said think of adding compression support to ecryptfs. > > I think adding compression support to eCryptfs is the wrong approach. > The "X is already a stacked filesystem so look into adding compression > support to it" logic also works when X=overlayfs. I doubt that Miklos > would be willing to accept such a feature. :) My thought was that compression is not far away from crypto an hence a lot of ecryptfs could be reused. > A stacked filesystem that implements compression should be fairly > simple. If it is not simple, it is too complicated to try to wedge into > an unrelated stacked filesystem. > > While it may be the quickest route to your end goal, it will overly > complicate eCryptfs. eCryptfs already has plenty of complexity around > the file offset since metadata may be stored in the first 8192 bytes of > the lower file, which offsets the entire file, and the end of the file > has to be padded. Mixing in compression would make things much worse. I assumed that you need also some meta data for compression. At least if you do it in a non-trivial way. > Also, eCryptfs has lots of cruft that is definitely not needed for > compression. As you're the maintainer of ecryptfs you know obviously better than I do. :) Tom, to make the story short, you'll have to experiment a bit. Thanks, //richard