[GIT] Delayed Logging V3

[GIT] Delayed Logging V3

[Dave Chinner]

Hi Folks,

This is version 3 of the delayed logging series.

I won't repeat everything about what it is, just point you
here:

http://marc.info/?l=linux-xfs&m=126862777118946&w=2

for the description, and here:

git://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfs.git delayed-logging

for the current code. Note that this is a rebased branch, so you'll
need to pull it again into a new branch to review.

All previously known, reproducable issues have been fixed in this
release, and as such I consider it ready from a functional POV for
inclusion into the -dev tree as an experimental feature.

The new cleanups added in this version mean it toucheѕ more files
that the previous versions, but overall it should still be simpler
to review because I've collapsed many of the intermediate patches
into one "Introduce delayed logging core code" commit.

Version 3: 28 files changed, 2366 insertions(+), 506 deletions(-)
Version 2: 22 files changed, 2188 insertions(+), 377 deletions(-)
Version 1: 19 files changed, 2594 insertions(+), 580 deletions(-)

Changes for V3:
o changed buffer log item reference counted model to be consistent
  for both logging modes
o cleaned up XFS_BLI flags usage (new commit)
o separated out log ticket overrun printing cleanup (new commit)
o made sure "delaylog" option shows up in /proc/mounts
o collapsed many of the intermediate commits together to make it
  easier to review
o fixed inode buffer tagging issue that was causing shutdowns in log
  recovery in test 087 and 121

Changes for V2:
o 22 files changed, 2188 insertions(+), 377 deletions(-)
o fixed some memory leaks
o fixed ticket allocation for checkpoints to use KM_NOFS
o minor code cleanups
o performed stress and scalability testing


The following changes since commit 6ff75b78182c314112c1173edaab6c164c95d775:
  Christoph Hellwig (1):
        xfs: mark xfs_iomap_write_ helpers static

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfs.git delayed-logging

Dave Chinner (10):
      xfs: Improve scalability of busy extent tracking
      xfs: allow log ticket allocation to take allocation flags
      xfs: modify buffer item reference counting V2
      xfs: allow detection of inode allocation buffers in recovery
      xfs: clean up log ticket overrun debug output
      xfs: Delayed logging design documentation
      xfs: Introduce delayed logging core code
      xfs: forced unmounts need to push the CIL
      xfs: enable background pushing of the CIL
      xfs: allow detection of inode allocation buffers in recovery

 .../filesystems/xfs-delayed-logging-design.txt     |  819 ++++++++++++++++++++
 fs/xfs/Makefile                                    |    1 +
 fs/xfs/linux-2.6/xfs_buf.c                         |   11 +-
 fs/xfs/linux-2.6/xfs_quotaops.c                    |    1 +
 fs/xfs/linux-2.6/xfs_super.c                       |   12 +-
 fs/xfs/linux-2.6/xfs_trace.h                       |   80 ++-
 fs/xfs/quota/xfs_dquot.c                           |    6 +-
 fs/xfs/support/debug.c                             |    1 +
 fs/xfs/xfs_ag.h                                    |   21 +-
 fs/xfs/xfs_alloc.c                                 |  272 ++++---
 fs/xfs/xfs_alloc.h                                 |    5 +-
 fs/xfs/xfs_buf_item.c                              |  166 ++--
 fs/xfs/xfs_buf_item.h                              |   18 +-
 fs/xfs/xfs_error.c                                 |    2 +-
 fs/xfs/xfs_filestream.c                            |    1 +
 fs/xfs/xfs_log.c                                   |  107 ++-
 fs/xfs/xfs_log.h                                   |   12 +-
 fs/xfs/xfs_log_cil.c                               |  729 +++++++++++++++++
 fs/xfs/xfs_log_priv.h                              |  118 +++-
 fs/xfs/xfs_log_recover.c                           |   46 +-
 fs/xfs/xfs_log_recover.h                           |    2 +-
 fs/xfs/xfs_mount.h                                 |    1 +
 fs/xfs/xfs_trans.c                                 |  216 +++++-
 fs/xfs/xfs_trans.h                                 |   45 +-
 fs/xfs/xfs_trans_buf.c                             |   46 +-
 fs/xfs/xfs_trans_extfree.c                         |    1 +
 fs/xfs/xfs_trans_item.c                            |  114 +---
 fs/xfs/xfs_trans_priv.h                            |   19 +-
 28 files changed, 2366 insertions(+), 506 deletions(-)
 create mode 100644 Documentation/filesystems/xfs-delayed-logging-design.txt
 create mode 100644 fs/xfs/xfs_log_cil.c

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [GIT] Delayed Logging V3

[Christoph Hellwig]

On Tue, May 04, 2010 at 11:50:42AM +1000, Dave Chinner wrote:
> The new cleanups added in this version mean it touche?? more files
> that the previous versions, but overall it should still be simpler
> to review because I've collapsed many of the intermediate patches
> into one "Introduce delayed logging core code" commit.

Thanks.  Btw, a little procedural comment - you include the V2 markers
and changelogs in the commit message.  Normal procedure is to have
them below the -- or what it is marker with the diffstat so that they
only get picked up in the mail and not the final commits, and no V2 in
the subject line at all.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [GIT] Delayed Logging V3

[Dave Chinner]

On Tue, May 04, 2010 at 06:17:33AM -0400, Christoph Hellwig wrote:
> On Tue, May 04, 2010 at 11:50:42AM +1000, Dave Chinner wrote:
> > The new cleanups added in this version mean it touche?? more files
> > that the previous versions, but overall it should still be simpler
> > to review because I've collapsed many of the intermediate patches
> > into one "Introduce delayed logging core code" commit.
> 
> Thanks.  Btw, a little procedural comment - you include the V2 markers
> and changelogs in the commit message.  Normal procedure is to have
> them below the -- or what it is marker with the diffstat so that they
> only get picked up in the mail and not the final commits, and no V2 in
> the subject line at all.

Yeah, That's typical. The problem is that guilt seems to kill
anything I add to the patch headers below a "---" separator.  Hence
if i don't put it the in commit message it doesn't stick as I rebase
my working branches, and hence doesn't get included in the
patchbombs I send direct from the repository.

For stuff that is in a series that I'm tracking in a separate branch I
can probably just keep versioning changes in the series header (the
patch 0/N message), but for individual patches it's a bit harder.
I'll see what I can do to track this more easily in my workflow.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs