From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Marshall Subject: Re: fs compression Date: Thu, 28 May 2015 13:55:15 -0700 Message-ID: <20150528205515.GA7450@eden.sea.cyngn.com> References: <20150513064802.GA48682@jaegeuk-mac02.hsd1.ca.comcast.net> <20150514003721.GN15721@dastard> <20150516132403.GA2998@thunk.org> <20150516171326.GA24795@eden.sea.cyngn.com> <20150520174635.GA17651@eden.sea.cyngn.com> <20150520213641.GM2871@thunk.org> <20150520224630.GA10927@eden.sea.cyngn.com> <20150521042819.GA14709@eden.sea.cyngn.com> <5566129D.9040509@cyngn.com> <20150527233800.GB18540@thunk.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jaegeuk Kim , linux-fsdevel@vger.kernel.org To: Theodore Ts'o Return-path: Received: from mail-pa0-f44.google.com ([209.85.220.44]:32903 "EHLO mail-pa0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751812AbbE1UzT (ORCPT ); Thu, 28 May 2015 16:55:19 -0400 Received: by padbw4 with SMTP id bw4so31884569pad.0 for ; Thu, 28 May 2015 13:55:19 -0700 (PDT) Content-Disposition: inline In-Reply-To: <20150527233800.GB18540@thunk.org> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Wed, May 27, 2015 at 07:38:00PM -0400, Theodore Ts'o wrote: > On Wed, May 27, 2015 at 11:53:17AM -0700, Tom Marshall wrote: > > But one thing I'm wrestling with is how to be asynchronously notified when > > the lower readpage/readpages complete. The two ideas that come to mind are > > (1) plumbing a callback into mpage_end_io(), (2) allowing override of > > mpage_end_io() with a custom function, (3) creating kernel threads analogous > > to kblockd to wait for pending pages. > > Not all file systems use mpage_end_io(), so that's not a good > solution. > > You can do something like > > wait_on_page_bit(page, PG_uptodate); > > ... although to be robust you will also need to wake up if PG_error is > set (if there is an I/O error, PG_error is set instead of > PG_uptodate). So that means you'd have to spin your own wait function > using the waitqueue primitives and page_waitqueue(), using > __wait_on_bit() as an initial model. My current thought is to use a workqueue and queue a work object for each cluster read that is in flight. The work function does wait_on_page_locked for each lower page. If/when all complete without error, the data is then decompressed and the upper pages are entered. This is currently a work in progress, it's not yet functional. So basically my questions here are: (1) Does this seem sane? (2) Is it ok to wait on !locked instead of waiting on (uptodate|error)? (3) Do I need to mark a pending cluster pending to prevent simultaneous reads of the same cluster (especially since the cluster size will be much larger than the lower block size)? Also note I've now realized that the logical conclusion to the wrapper inode idea is a stacked filesystem. That's not the direction we are aiming for, so instead I'm adding a "struct xcomp_inode_info" to ext4_inode_info, which can then be passed back into the xcomp functions.