[Cluster-devel] [fsck.gfs2 v2 PATCH 00/40] fsck.gfs2: memory reduction patches

[Andrew Price <anprice@redhat.com>]

Hi Bob,

On 06/05/16 18:38, Bob Peterson wrote:
> On 15 April, I posted a set of 30 patches for fsck.gfs2 to reduce its
> memory requirements. Since that time, I've been testing and fixing
> problems. Here is version 2.
> ---
> Right now, fsck.gfs2 is a memory hog with regard to very large file
> systems. A file system of 250TB would require an absurd about of memory.
> That's because it uses memory for (1) rgrp bitmaps, (2) its own internal
> blockmap, (3) a directory tree, and (4) a tree of ALL inodes in the
> file system, for reconciling link counts.
>
> This series of 40 patches is designed to reduce the memory footprint
> of fsck.gfs2. It does so by almost entirely eliminating the inode
> tree in favor of two much smaller bitmaps. In order to do that, I
> rearranged the order in which things are done.
>
> In order to save on memory, some things will be slower, so this will
> cause a performance hit to fsck.gfs2. However, I'm confident that
> we can fix them and make it fast again. The slowness is caused because
> most bitmap manipulation is now done on the bitmaps in the resource
> groups, which means the code needs to search out the rgrp for these
> operations. This can be optimized to pass in the rgrp, since it will
> be known in most cases (for example, pass1 operates on one rgrp at a
> time, and chances are quite good that the rgrp needed will be that one.)
> These further optimizations will be done in later patches.
>
> I'm still testing these changes against my collection of gfs2 and gfs1
> metadata, but the results are good so far. So there may be changes.

I've read through the patches and the code looks good to me. It's 
difficult to comment on the design of the solution but nothing scary 
jumped out at me, and the test results look promising. There's some 
really good refactoring work in this set too.

> Signed-off-by: Bob Peterson <rpeterso@redhat.com>
> ---
> Bob Peterson (40):
>    fsck.gfs2: Move pass5 to immediately follow pass1
>    fsck.gfs2: Convert block_type to bitmap_type after pass1 and 5
>    fsck.gfs2: Change bitmap_type variables to int
>    fsck.gfs2: Use di_entries to determine if lost+found was created
>    fsck.gfs2: pass1b shouldn't complain about non-bitmap blocks
>    fsck.gfs2: Change all fsck_blockmap_set to fsck_bitmap_set
>    fsck.gfs2: Move set_ip_blockmap to pass1
>    fsck.gfs2: Remove unneeded parameter instree from set_ip_blockmap
>    fsck.gfs2: Move leaf repair to pass2
>    fsck.gfs2: Eliminate astate code
>    fsck.gfs2: Move reprocess code to pass1
>    fsck.gfs2: Separate out functions that may only be done after pass1
>    fsck.gfs2: Divest check_metatree from fsck_blockmap_set
>    fsck.gfs2: eliminate fsck_blockmap_set from check_eattr_entries
>    fsck.gfs2: Move blockmap stuff to pass1.c
>    fsck: make pass1 call bitmap reconciliation AKA pass5
>    fsck.gfs2: make blockmap global variable only to pass1
>    fsck.gfs2: Add wrapper function pass1_check_metatree
>    fsck.gfs2: pass counted_links into fix_link_count in pass4
>    fsck.gfs2: refactor pass4 function scan_inode_list
>    fsck.gfs2: More refactoring of pass4 function scan_inode_list
>    fsck.gfs2: Fix white space problems
>    fsck.gfs2: move link count info for directories to directory tree
>    fsck.gfs2: Use bitmaps instead of linked list for inodes w/nlink == 1
>    fsck.gfs2: Refactor check_n_fix_bitmap to make it more readable
>    fsck.gfs2: adjust rgrp inode count when fixing bitmap
>    fsck.gfs2: blocks cannot be UNLINKED in pass1b or after that
>    fsck.gfs2: Add error checks to get_next_leaf
>    fsck.gfs2: re-add a non-allocating repair_leaf to pass1
>    libgfs2: Allocate new GFS1 metadata as type 3, not type 1
>    fsck.gfs2: Undo partially done metadata records
>    fsck.gfs2: Eliminate redundant code in _fsck_bitmap_set
>    fsck.gfs2: Fix inode counting bug
>    fsck.gfs2: Adjust bitmap for lost+found after adding to dirtree
>    GFS2: Add initialization checks for GFS1 used metadata

s/GFS2/fsck.gfs2/ :)

Thanks,
Andy

>    fsck.gfs2: Use BLKST constants to make pass5 more clear
>    fsck.gfs2: Fix GFS1 "used meta" accounting bug
>    fsck.gfs2: pass1b is too noisy wrt gfs1 non-dinode metadata
>    fsck.gfs2: Fix rgrp dinode accounting bug
>    fsck.gfs2: Fix rgrp accounting in check_n_fix_bitmap
>
>   gfs2/fsck/Makefile.am         |   1 +
>   gfs2/fsck/afterpass1_common.c | 319 ++++++++++++++++
>   gfs2/fsck/afterpass1_common.h |  31 ++
>   gfs2/fsck/fsck.h              |  26 +-
>   gfs2/fsck/initialize.c        |  63 +++-
>   gfs2/fsck/inode_hash.c        |   3 +-
>   gfs2/fsck/link.c              | 139 ++++++-
>   gfs2/fsck/link.h              |   6 +-
>   gfs2/fsck/lost_n_found.c      |  17 +-
>   gfs2/fsck/main.c              |  38 +-
>   gfs2/fsck/metawalk.c          | 847 ++++++++----------------------------------
>   gfs2/fsck/metawalk.h          |  59 +--
>   gfs2/fsck/pass1.c             | 479 ++++++++++++++++++++++--
>   gfs2/fsck/pass1b.c            | 148 +++++---
>   gfs2/fsck/pass2.c             | 276 ++++++++++----
>   gfs2/fsck/pass3.c             |  52 +--
>   gfs2/fsck/pass4.c             | 339 ++++++++++-------
>   gfs2/fsck/pass5.c             |  60 +--
>   gfs2/fsck/util.c              | 171 +++------
>   gfs2/fsck/util.h              |  70 ++--
>   gfs2/libgfs2/fs_ops.c         |  33 +-
>   21 files changed, 1849 insertions(+), 1328 deletions(-)
>   create mode 100644 gfs2/fsck/afterpass1_common.c
>   create mode 100644 gfs2/fsck/afterpass1_common.h
>