From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Marshall Subject: Re: fs compression Date: Thu, 28 May 2015 17:18:31 -0700 Message-ID: <20150529001831.GA29447@eden.sea.cyngn.com> References: <20150514003721.GN15721@dastard> <20150516132403.GA2998@thunk.org> <20150516171326.GA24795@eden.sea.cyngn.com> <20150520174635.GA17651@eden.sea.cyngn.com> <20150520213641.GM2871@thunk.org> <20150520224630.GA10927@eden.sea.cyngn.com> <20150521042819.GA14709@eden.sea.cyngn.com> <5566129D.9040509@cyngn.com> <20150527233800.GB18540@thunk.org> <20150528205515.GA7450@eden.sea.cyngn.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jaegeuk Kim , linux-fsdevel@vger.kernel.org To: Theodore Ts'o Return-path: Received: from mail-pa0-f42.google.com ([209.85.220.42]:33937 "EHLO mail-pa0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753342AbbE2ASe (ORCPT ); Thu, 28 May 2015 20:18:34 -0400 Received: by pabru16 with SMTP id ru16so35142055pab.1 for ; Thu, 28 May 2015 17:18:34 -0700 (PDT) Content-Disposition: inline In-Reply-To: <20150528205515.GA7450@eden.sea.cyngn.com> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: So I've just gotten this all working. The last notable change I made was to inode size: I added an i_compressed_size member and then did some macro hackage to ensure that the fs implementation (eg. fs/ext4) sees the compressed size while everything else sees the uncompressed size. I'll be testing further tomorrow. On Thu, May 28, 2015 at 01:55:15PM -0700, Tom Marshall wrote: > On Wed, May 27, 2015 at 07:38:00PM -0400, Theodore Ts'o wrote: > > On Wed, May 27, 2015 at 11:53:17AM -0700, Tom Marshall wrote: > > > But one thing I'm wrestling with is how to be asynchronously notified when > > > the lower readpage/readpages complete. The two ideas that come to mind are > > > (1) plumbing a callback into mpage_end_io(), (2) allowing override of > > > mpage_end_io() with a custom function, (3) creating kernel threads analogous > > > to kblockd to wait for pending pages. > > > > Not all file systems use mpage_end_io(), so that's not a good > > solution. > > > > You can do something like > > > > wait_on_page_bit(page, PG_uptodate); > > > > ... although to be robust you will also need to wake up if PG_error is > > set (if there is an I/O error, PG_error is set instead of > > PG_uptodate). So that means you'd have to spin your own wait function > > using the waitqueue primitives and page_waitqueue(), using > > __wait_on_bit() as an initial model. > > My current thought is to use a workqueue and queue a work object for each > cluster read that is in flight. The work function does wait_on_page_locked > for each lower page. If/when all complete without error, the data is then > decompressed and the upper pages are entered. This is currently a work in > progress, it's not yet functional. > > So basically my questions here are: > > (1) Does this seem sane? > > (2) Is it ok to wait on !locked instead of waiting on (uptodate|error)? > > (3) Do I need to mark a pending cluster pending to prevent simultaneous > reads of the same cluster (especially since the cluster size will be > much larger than the lower block size)? > > Also note I've now realized that the logical conclusion to the wrapper inode > idea is a stacked filesystem. That's not the direction we are aiming for, > so instead I'm adding a "struct xcomp_inode_info" to ext4_inode_info, which > can then be passed back into the xcomp functions.