public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: Brian Foster <bfoster@redhat.com>
Cc: xfs@oss.sgi.com
Subject: Re: [PATCH RFC 00/18] xfs: sparse inode chunks
Date: Tue, 29 Jul 2014 10:26:50 +1000	[thread overview]
Message-ID: <20140729002650.GH26465@dastard> (raw)
In-Reply-To: <20140728121400.GB53501@bfoster.bfoster>

On Mon, Jul 28, 2014 at 08:14:01AM -0400, Brian Foster wrote:
> On Sat, Jul 26, 2014 at 10:03:35AM +1000, Dave Chinner wrote:
> > On Fri, Jul 25, 2014 at 12:30:57PM -0400, Brian Foster wrote:
> > > Hmm, I suppose that does create a new and interesting dynamic with
> > > regard to the feature bit (non-deterministic backwards compatibility).
> > > One could certainly value backwards compatibility over this particular
> > > feature, and there is currently no way to control it. I'll look into
> > > doing something with xfs_admin. In fact, I was thinking of adding
> > > something to tune the cluster size bit to get around the v5 scaling
> > > issue anyways.
> > 
> > What v5 scalability issue is that? I don't recall any outstanding
> > issues with inode cluster IO....
> > 
> 
> There's no scalability issue... I'm just referring to the fact that we
> scale the cluster size by the inode size increase factor on v5
> superblocks.
> 
> E.g., my free space fragmentation xfstests test started out with a fixed
> file size based on something close to the worst case with an
> implementation that used the allocation granularity of max(<holemask bit
> granularity>, <inodes per block>). Once I tied the implementation to the
> cluster size due to the aforementioned complexities, it became apparent
> the test was less effective with my chosen file size on v5 supers,
> particularly as the inode size increased. So from there I was
> considering a similar xfs_admin command a user could run to reduce the
> cluster size as a backstop should this limitation arise in the real
> world. We can start with doing something just to enable the feature as
> outlined above and revisit this then...

Right, but I'd suggest that the better long term solution to avoid
the limitations of inode cluster buffer alignment issues is to get
rid of inode clusters and inode buffers altogether. We only need
inode buffers for logging unlinked list modifications, so once we
log those as part of the inode core for for v5 filesystems then we
can do much more dynamic inode IO. That then frees us up to do fine
grained sparse inode allocation because we aren't limited by
in-memory buffering limitations.

http://xfs.org/index.php/Improving_inode_Caching#Food_For_Thought_.28Crazy_Ideas.29

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2014-07-29  0:30 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-07-24 14:22 [PATCH RFC 00/18] xfs: sparse inode chunks Brian Foster
2014-07-24 14:22 ` [PATCH 01/18] xfs: refactor xfs_inobt_insert() to eliminate loop and support variable count Brian Foster
2014-07-24 22:10   ` Dave Chinner
2014-07-28 16:03     ` Brian Foster
2014-07-28 23:32       ` Dave Chinner
2014-07-29 14:43         ` Brian Foster
2014-07-24 14:22 ` [PATCH 02/18] xfs: pass xfs_mount directly to xfs_ialloc_cluster_alignment() Brian Foster
2014-07-24 14:22 ` [PATCH 03/18] xfs: define sparse inode chunks v5 sb feature bit and helper function Brian Foster
2014-07-24 17:08   ` Mark Tinguely
2014-07-24 17:37     ` Brian Foster
2014-07-24 18:38       ` Mark Tinguely
2014-07-24 19:38         ` Brian Foster
2014-07-24 23:35   ` Dave Chinner
2014-07-24 14:22 ` [PATCH 04/18] xfs: introduce inode record hole mask for sparse inode chunks Brian Foster
2014-07-24 22:14   ` Dave Chinner
2014-07-28 16:16     ` Brian Foster
2014-08-07 15:18       ` Brian Foster
2014-07-24 14:22 ` [PATCH 05/18] xfs: create macros/helpers for dealing with " Brian Foster
2014-07-24 22:13   ` Dave Chinner
2014-07-24 14:22 ` [PATCH 06/18] xfs: pass inode count through ordered icreate log item Brian Foster
2014-07-24 14:22 ` [PATCH 07/18] xfs: handle sparse inode chunks in icreate log recovery Brian Foster
2014-07-24 14:22 ` [PATCH 08/18] xfs: create helper to manage record overlap for sparse inode chunks Brian Foster
2014-07-24 22:41   ` Dave Chinner
2014-07-28 16:19     ` Brian Foster
2014-07-29  0:07       ` Dave Chinner
2014-07-29 15:10         ` Brian Foster
2014-07-24 14:22 ` [PATCH 09/18] xfs: allocate sparse inode chunks on full chunk allocation failure Brian Foster
2014-07-24 14:23 ` [PATCH 10/18] xfs: set sparse inodes feature bit when a sparse chunk is allocated Brian Foster
2014-07-24 22:46   ` Dave Chinner
2014-07-28 16:23     ` Brian Foster
2014-07-24 14:23 ` [PATCH 11/18] xfs: reduce min. inode allocation space requirement for sparse inode chunks Brian Foster
2014-07-24 22:50   ` Dave Chinner
2014-07-24 14:23 ` [PATCH 12/18] xfs: helper to convert inobt record holemask to inode alloc. bitmap Brian Foster
2014-07-24 23:21   ` Dave Chinner
2014-07-24 14:23 ` [PATCH 13/18] xfs: filter out sparse regions from individual inode allocation Brian Foster
2014-07-24 14:23 ` [PATCH 14/18] xfs: update free inode record logic to support sparse inode records Brian Foster
2014-07-24 14:23 ` [PATCH 15/18] xfs: only free allocated regions of inode chunks Brian Foster
2014-07-24 23:24   ` Dave Chinner
2014-07-24 14:23 ` [PATCH 16/18] xfs: skip unallocated regions of inode chunks in xfs_ifree_cluster() Brian Foster
2014-07-24 14:23 ` [PATCH 17/18] xfs: use actual inode count for sparse records in bulkstat/inumbers Brian Foster
2014-07-24 23:29   ` Dave Chinner
2014-07-24 14:23 ` [PATCH 18/18] xfs: enable sparse inode chunks for v5 superblocks Brian Foster
2014-07-24 23:34   ` Dave Chinner
2014-07-24 16:28 ` [PATCH RFC 00/18] xfs: sparse inode chunks Brian Foster
2014-07-24 22:32 ` Dave Chinner
2014-07-25 16:30   ` Brian Foster
2014-07-26  0:03     ` Dave Chinner
2014-07-28 12:14       ` Brian Foster
2014-07-29  0:26         ` Dave Chinner [this message]
2014-07-29 15:25           ` Brian Foster

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140729002650.GH26465@dastard \
    --to=david@fromorbit.com \
    --cc=bfoster@redhat.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox