[Cluster-devel] [PATCH 00/19] gfs2-utils: Introduce extent allocation and speed up journal creation

[Andrew Price <anprice@redhat.com>]

Hi,

This patch set introduces extent allocation to libgfs2 and adds functions which
decouple file creation, allocation and writing so that mkfs.gfs2 can be
re-worked to write journals and resource groups sequentially.

With these patches, mkfs.gfs2 typically takes around 20% of the time that it
did before in my tests.  The main speed-up has been from the journal data
allocation functions not having to re-read and write a resource group for each
block allocated as it did before (this was a performance regression introduced
by previous memory footprint improvement patches, hence the significant perf
improvement).  Journals now each occupy an extent spanning an entire resource
group specifically sized for the journal, and the resource group headers are
written only once, after the journal blocks have been allocated in the
in-memory bitmaps. Resource groups are still only kept in memory for as long as
they are needed so peak memory usage should be largely unchanged.

One thing to note is that, with these patches, the root and master inodes are
no longer the first objects in the first resource group. The master inode is
written in the first free block after the journals and then the other metafs
structures are placed. The root directory inode is then finally created. This
is not a format change but it may cause some confusion after years of expecting
the root and master inodes to be at certain addresses so I thought it worth
mentioning.

Coverity and valgrind are happy about these patches and I've encountered no
problems after various tests which mount the fs.

Cheers,
Andy

Andrew Price (19):
  libgfs2: Keep a pointer to the sbd in lgfs2_rgrps_t
  libgfs2: Move bitmap buffers inside struct gfs2_bitmap
  libgfs2: Fix an impossible loop condition in gfs2_rgrp_read
  libgfs2: Introduce struct lgfs2_rbm
  libgfs2: Move struct _lgfs2_rgrps into rgrp.h
  libgfs2: Add functions for finding free extents
  tests: Add unit tests for the new extent search functions
  libgfs2: Ignore an empty rgrp plan if a length is specified
  libgfs2: Add back-pointer to rgrps in lgfs2_rgrp_t
  libgfs2: Const-ify the parameters of print functions
  libgfs2: Allow init_dinode to accept a preallocated bh
  libgfs2: Add extent allocation functions
  libgfs2: Add support for allocating entire rgrp headers
  libgfs2: Write file metadata sequentially
  libgfs2: Fix alignment in lgfs2_rgsize_for_data
  libgfs2: Handle non-zero bitmaps in lgfs2_rgrp_write
  libgfs2: Add a speedier journal data block writing function
  libgfs2: Create jindex directory separately from journals
  mkfs.gfs2: Improve journal creation performance

 .gitignore                  |   3 +-
 gfs2/convert/gfs2_convert.c |  49 +++--
 gfs2/edit/journal.c         |   6 +-
 gfs2/fsck/fs_recovery.c     |   2 +-
 gfs2/fsck/initialize.c      |  27 +--
 gfs2/fsck/metawalk.c        |  10 +-
 gfs2/fsck/pass5.c           |   9 +-
 gfs2/fsck/rgrepair.c        |  14 +-
 gfs2/fsck/util.c            |   2 +-
 gfs2/libgfs2/Makefile.am    |   2 +-
 gfs2/libgfs2/fs_bits.c      |  10 +-
 gfs2/libgfs2/fs_geometry.c  |   6 +-
 gfs2/libgfs2/fs_ops.c       | 184 ++++++++++++++---
 gfs2/libgfs2/libgfs2.h      |  50 +++--
 gfs2/libgfs2/ondisk.c       |  26 +--
 gfs2/libgfs2/rgrp.c         | 491 ++++++++++++++++++++++++++++++++++++--------
 gfs2/libgfs2/rgrp.h         |  50 +++++
 gfs2/libgfs2/structures.c   | 103 +++++++++-
 gfs2/mkfs/main_grow.c       |   4 +-
 gfs2/mkfs/main_mkfs.c       | 155 ++++++++++----
 tests/Makefile.am           |  33 ++-
 tests/check_rgrp.c          | 143 +++++++++++++
 tests/libgfs2.at            |   8 +-
 23 files changed, 1113 insertions(+), 274 deletions(-)
 create mode 100644 gfs2/libgfs2/rgrp.h
 create mode 100644 tests/check_rgrp.c

-- 
1.9.3