All of lore.kernel.org
 help / color / mirror / Atom feed
From: Theodore Ts'o <tytso@mit.edu>
To: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: linux-ext4@vger.kernel.org
Subject: Re: [PATCH 06/35] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files
Date: Tue, 21 Apr 2015 12:33:00 -0400	[thread overview]
Message-ID: <20150421163300.GA14091@thunk.org> (raw)
In-Reply-To: <20150402023440.25243.95250.stgit@birch.djwong.org>

On Wed, Apr 01, 2015 at 07:34:40PM -0700, Darrick J. Wong wrote:
> Teach e2fsck to (re)construct extent trees.  This enables us to do
> either of the following: compress a highly sparse extent tree into
> fewer ETB blocks; or convert a ext3-style block mapped file to an
> extent file.  The reconstruction is performed during pass 1E or 3A,
> as detailed below.
> 
> For files that are already extent based, this algorithm will
> automatically run (pending user approval) if pass1 determines either
> (1) that a whole level of extent tree will fit into a higher level of
> the tree; (2) that the size of any level can be reduced by at least
> one ETB block; or (3) the extent tree is unnecessarily deep.  It will
> not run at all if errors are found and the user declines to fix the
> errors.
> 
> The option "-E bmap2extent" can be used to force e2fsck to convert all
> block map files to extent trees, and to rebuild all extent files'
> extent trees.  After conversion, files larger than 12 blocks should be
> defragmented to eliminate empty holes where a block lives.
> 
> The extent tree constructor is pretty dumb -- it creates a list of
> leaf extents (adjacent extents are collapsed), marks all indirect
> blocks / ETB blocks free, installs a new extent tree root in the
> inode, then loads the leaf extents into the tree.
> 
> v2: Account for extent tree block slack that we create when splitting
> a block, so that we don't repeatedly annoy the user to rebuild a tree
> that we can't optimize further.
> 
> v3: For any directory being rebuilt during pass 3A, defer any extent
> tree rebuilding until after the rehash.  It's quite possible that the
> act of compressing an aged directory will cause it to shrink far
> enough to enable us to knock a level off the dir's extent tree.
> 
> v4: Add a fixes_only option (and a E2FSCK_FIXES_ONLY environment
> variable) that disables optimization activities unless they are
> required to make the filesystem consistent.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

						- Ted

  reply	other threads:[~2015-04-21 16:33 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-02  2:34 [PATCH 00/35] e2fsprogs April 2015 patchbomb Darrick J. Wong
2015-04-02  2:34 ` [PATCH 01/35] e2fuzz: fuzz harder Darrick J. Wong
2015-04-21  1:47   ` Theodore Ts'o
2015-04-02  2:34 ` [PATCH 02/35] e2fsck: turn inline data symlink into a fast symlink when possible Darrick J. Wong
2015-04-21  1:47   ` Theodore Ts'o
2015-04-02  2:34 ` [PATCH 03/35] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
2015-04-21  3:03   ` Theodore Ts'o
2015-04-02  2:34 ` [PATCH 04/35] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
2015-04-21  3:03   ` Theodore Ts'o
2015-04-02  2:34 ` [PATCH 05/35] e2fsck: track directories to be rehashed with a bitmap Darrick J. Wong
2015-04-21  2:26   ` Theodore Ts'o
2015-04-21  4:43     ` Darrick J. Wong
2015-04-21 14:06       ` Theodore Ts'o
2015-04-02  2:34 ` [PATCH 06/35] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
2015-04-21 16:33   ` Theodore Ts'o [this message]
2015-04-02  2:34 ` [PATCH 07/35] e2fsck: convert block-mapped files to extents on bigalloc fs Darrick J. Wong
2015-04-21 14:36   ` Theodore Ts'o
2015-05-05 22:45     ` Darrick J. Wong
2015-04-02  2:34 ` [PATCH 08/35] tests: verify proper rebuilding of sparse extent trees and block map file conversion Darrick J. Wong
2015-04-21 14:47   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 09/35] e2fsck: abort on read error beyond end of FS Darrick J. Wong
2015-04-02  4:10   ` Andreas Dilger
     [not found]     ` <20150402060021.GP11031@birch.djwong.org>
     [not found]       ` <10D33B1F-52B7-4242-9A67-FB9E1CE75296@dilger.ca>
2015-04-06 18:57         ` Darrick J. Wong
2015-04-02  2:35 ` [PATCH 10/35] undo-io: add new calls to and speed up the undo io manager Darrick J. Wong
2015-04-02  4:06   ` Andreas Dilger
2015-04-21 15:00     ` Theodore Ts'o
2015-04-21 16:48       ` Theodore Ts'o
2015-04-22  2:47         ` Darrick J. Wong
2015-05-05 14:20   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 11/35] undo-io: be more flexible about setting block size Darrick J. Wong
2015-05-05 14:21   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 12/35] undo-io: use a bitmap to track what we've already written Darrick J. Wong
2015-05-05 14:21   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 13/35] e2undo: fix memory leaks and tweak the error messages somewhat Darrick J. Wong
2015-05-05 14:22   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 14/35] e2undo: ditch tdb file, write everything to a flat file Darrick J. Wong
2015-05-05 14:24   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 15/35] libext2fs: support atexit cleanups Darrick J. Wong
2015-05-05 14:31   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 16/35] e2fsck: optionally create an undo file Darrick J. Wong
2015-05-05 14:07   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 17/35] resize2fs: optionally create " Darrick J. Wong
2015-05-05 14:36   ` Theodore Ts'o
2015-04-02  2:35 ` [PATCH 18/35] tune2fs: " Darrick J. Wong
2015-05-05 14:36   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 19/35] mke2fs: " Darrick J. Wong
2015-05-05 14:37   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 20/35] debugfs: " Darrick J. Wong
2015-05-05 14:43   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 21/35] tests: test undo file creation in e2fsck/resize2fs/tune2fs/mke2fs Darrick J. Wong
2015-05-05 14:43   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 22/35] tests: test various features of the new e2undo format Darrick J. Wong
2015-05-05 14:44   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 23/35] copy-in: create hardlinks with the correct directory filetype Darrick J. Wong
2015-05-05 14:46   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 24/35] copy-in: for files, only iterate file blocks that are mapped Darrick J. Wong
2015-05-05 14:49   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 25/35] copyin: fix error handling Darrick J. Wong
2015-05-05 14:51   ` Theodore Ts'o
2015-04-02  2:36 ` [PATCH 26/35] mke2fs: add simple tests and re-alphabetize mke2fs manpage options Darrick J. Wong
2015-05-05 14:52   ` Theodore Ts'o
2015-04-02  2:37 ` [PATCH 27/35] contrib: script to create minified ext4 image from a directory Darrick J. Wong
2015-05-05 14:52   ` Theodore Ts'o
2015-04-02  2:37 ` [PATCH 28/35] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
2015-04-02  2:37 ` [PATCH 29/35] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
2015-04-02  2:37 ` [PATCH 30/35] libext2fs: add new hooks to support large allocations Darrick J. Wong
2015-04-02  2:37 ` [PATCH 31/35] libext2fs: implement fallocate Darrick J. Wong
2015-04-02  2:37 ` [PATCH 32/35] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
2015-04-02  2:37 ` [PATCH 33/35] debugfs: implement fallocate Darrick J. Wong
2015-04-02  2:37 ` [PATCH 34/35] tests: test debugfs punch command Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150421163300.GA14091@thunk.org \
    --to=tytso@mit.edu \
    --cc=darrick.wong@oracle.com \
    --cc=linux-ext4@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.