[Cluster-devel] [PATCH 00/19] gfs2-utils: Introduce extent allocation and speed up journal creation

[Steven Whitehouse <swhiteho@redhat.com>]

Hi,

Aside from the question on patch 6, the other patches all look good,

Steve.

On 02/09/14 13:07, Andrew Price wrote:
> Hi,
>
> This patch set introduces extent allocation to libgfs2 and adds functions which
> decouple file creation, allocation and writing so that mkfs.gfs2 can be
> re-worked to write journals and resource groups sequentially.
>
> With these patches, mkfs.gfs2 typically takes around 20% of the time that it
> did before in my tests.  The main speed-up has been from the journal data
> allocation functions not having to re-read and write a resource group for each
> block allocated as it did before (this was a performance regression introduced
> by previous memory footprint improvement patches, hence the significant perf
> improvement).  Journals now each occupy an extent spanning an entire resource
> group specifically sized for the journal, and the resource group headers are
> written only once, after the journal blocks have been allocated in the
> in-memory bitmaps. Resource groups are still only kept in memory for as long as
> they are needed so peak memory usage should be largely unchanged.
>
> One thing to note is that, with these patches, the root and master inodes are
> no longer the first objects in the first resource group. The master inode is
> written in the first free block after the journals and then the other metafs
> structures are placed. The root directory inode is then finally created. This
> is not a format change but it may cause some confusion after years of expecting
> the root and master inodes to be at certain addresses so I thought it worth
> mentioning.
>
> Coverity and valgrind are happy about these patches and I've encountered no
> problems after various tests which mount the fs.
>
> Cheers,
> Andy
>
> Andrew Price (19):
>    libgfs2: Keep a pointer to the sbd in lgfs2_rgrps_t
>    libgfs2: Move bitmap buffers inside struct gfs2_bitmap
>    libgfs2: Fix an impossible loop condition in gfs2_rgrp_read
>    libgfs2: Introduce struct lgfs2_rbm
>    libgfs2: Move struct _lgfs2_rgrps into rgrp.h
>    libgfs2: Add functions for finding free extents
>    tests: Add unit tests for the new extent search functions
>    libgfs2: Ignore an empty rgrp plan if a length is specified
>    libgfs2: Add back-pointer to rgrps in lgfs2_rgrp_t
>    libgfs2: Const-ify the parameters of print functions
>    libgfs2: Allow init_dinode to accept a preallocated bh
>    libgfs2: Add extent allocation functions
>    libgfs2: Add support for allocating entire rgrp headers
>    libgfs2: Write file metadata sequentially
>    libgfs2: Fix alignment in lgfs2_rgsize_for_data
>    libgfs2: Handle non-zero bitmaps in lgfs2_rgrp_write
>    libgfs2: Add a speedier journal data block writing function
>    libgfs2: Create jindex directory separately from journals
>    mkfs.gfs2: Improve journal creation performance
>
>   .gitignore                  |   3 +-
>   gfs2/convert/gfs2_convert.c |  49 +++--
>   gfs2/edit/journal.c         |   6 +-
>   gfs2/fsck/fs_recovery.c     |   2 +-
>   gfs2/fsck/initialize.c      |  27 +--
>   gfs2/fsck/metawalk.c        |  10 +-
>   gfs2/fsck/pass5.c           |   9 +-
>   gfs2/fsck/rgrepair.c        |  14 +-
>   gfs2/fsck/util.c            |   2 +-
>   gfs2/libgfs2/Makefile.am    |   2 +-
>   gfs2/libgfs2/fs_bits.c      |  10 +-
>   gfs2/libgfs2/fs_geometry.c  |   6 +-
>   gfs2/libgfs2/fs_ops.c       | 184 ++++++++++++++---
>   gfs2/libgfs2/libgfs2.h      |  50 +++--
>   gfs2/libgfs2/ondisk.c       |  26 +--
>   gfs2/libgfs2/rgrp.c         | 491 ++++++++++++++++++++++++++++++++++++--------
>   gfs2/libgfs2/rgrp.h         |  50 +++++
>   gfs2/libgfs2/structures.c   | 103 +++++++++-
>   gfs2/mkfs/main_grow.c       |   4 +-
>   gfs2/mkfs/main_mkfs.c       | 155 ++++++++++----
>   tests/Makefile.am           |  33 ++-
>   tests/check_rgrp.c          | 143 +++++++++++++
>   tests/libgfs2.at            |   8 +-
>   23 files changed, 1113 insertions(+), 274 deletions(-)
>   create mode 100644 gfs2/libgfs2/rgrp.h
>   create mode 100644 tests/check_rgrp.c
>