From: Theodore Ts'o <tytso@mit.edu>
To: Andreas Dilger <adilger@dilger.ca>
Cc: Ext4 Developers List <linux-ext4@vger.kernel.org>
Subject: Re: [PATCH 09/12] libext2: optimize ext2fs_new_block2()
Date: Tue, 21 Jan 2014 10:54:22 -0500 [thread overview]
Message-ID: <20140121155422.GB31769@thunk.org> (raw)
In-Reply-To: <85FC5E41-56C5-43A4-8514-CEADC36FE5C3@dilger.ca>
On Mon, Jan 20, 2014 at 02:52:43PM -0700, Andreas Dilger wrote:
> On Jan 19, 2014, at 10:54 PM, Theodore Ts'o <tytso@mit.edu> wrote:
> > If there are hundreds of thousands of blocks which are in use before
> > the first free block, it is much, MUCH faster to use
> > ext2fs_find_first_zero_block_bitmap2() instead of searching the
> > allocation bitmap bit by bit.
>
> Excellent. The libext2fs block allocator has typically been unusable
> for filesystems with allocated blocks, so it is good to see this.
Yes, all of these patches should fix the following e-mail message
question which you raised a while back:
------------
Date: Fri, 28 May 2010 13:25:45 -0600
From: Andreas Dilger <andreas.dilger@oracle.com>
To: "linux-ext4@vger.kernel.org development" <linux-ext4@vger.kernel.org>
Cc: Zhiqi Tao <zhiqi.tao@oracle.com>
Subject: Huge flex_bg count kills mke2fs
X-Mailer: Apple Mail (2.1078)
We're trying to put all of the static ext4 metadata at the beginning
of the disk to see whether this gives us a performance improvement
(avoid seeking during e2fsck, avoid free space fragmentation). This
is on a filesystem somewhat smaller than 16TB, and the flex_bg count
131072 would seem large enough to put all of groups into a single flex
group.
However, running the below command spins forever, apparently trying to
allocate the static metadata:
mke2fs -j -b 4096 -G 131072 -J size=1024 -i 65536 \
-O flex_bg,uninit_bg,extents -I 256 -F /dev/vgost0/lvost0 4294965248
...
-----------
(I found it a while today when I was doing some mail searching.)
> This could be further improved by skipping full groups entirely, but
> that doesn't detract from the benefits of this patch.
Actually, given that we're using a red-black tree for the block
bitmaps these days, it's actually much more efficient than checking
the block counts to see which block groups might be free. If the
first 4,000 block groups have been fully allocated, there will be a
single entry node in the red-black tree. :-)
- Ted
next prev parent reply other threads:[~2014-01-21 15:54 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-20 5:54 [PATCH 00/12] e2fsprogs mke2fs optimizations and new features Theodore Ts'o
2014-01-20 5:54 ` [PATCH 01/12] libext2fs: fix off-by-one bug in ext2fs_extent_insert() Theodore Ts'o
2014-01-20 5:54 ` [PATCH 02/12] libext2fs: clean up generic handling of ext2fs_find_first_{set,zero}_*() Theodore Ts'o
2014-01-20 5:54 ` [PATCH 03/12] libext2fs: build tst_bitmaps with rep invariants checking enabled Theodore Ts'o
2014-01-20 5:54 ` [PATCH 04/12] libext: optimize find_first_set() for bitarray-based bitmaps Theodore Ts'o
2014-01-20 5:54 ` [PATCH 05/12] libext2fs: optimize find_first_{zero,set}() for red-black tree based bitmaps Theodore Ts'o
2014-01-20 5:54 ` [PATCH 06/12] libext2fs: further clean up and rename check_block_uninit Theodore Ts'o
2014-01-20 20:17 ` Darrick J. Wong
2014-01-20 5:54 ` [PATCH 07/12] libext2fs: add ext2fs_block_alloc_stats_range() Theodore Ts'o
2014-02-13 21:50 ` Darrick J. Wong
2014-01-20 5:54 ` [PATCH 08/12] libext2fs: optimize ext2fs_allocate_group_table() Theodore Ts'o
2014-01-20 5:54 ` [PATCH 09/12] libext2: optimize ext2fs_new_block2() Theodore Ts'o
2014-01-20 21:52 ` Andreas Dilger
2014-01-21 15:54 ` Theodore Ts'o [this message]
2014-01-20 5:54 ` [PATCH 10/12] mke2fs: optimize fix_cluster_bg_counts() Theodore Ts'o
2014-01-20 5:54 ` [PATCH 11/12] Add support for new compat feature "sparse_super2" Theodore Ts'o
2014-01-20 21:49 ` Andreas Dilger
[not found] ` <alpine.DEB.2.10.1402051559390.16807@tglase.lan.tarent.de>
2014-02-05 16:17 ` Theodore Ts'o
2014-01-20 5:54 ` [PATCH 12/12] mke2fs: allow metadata blocks to be at the beginning of the file system Theodore Ts'o
2014-01-20 16:30 ` Theodore Ts'o
2014-01-20 23:25 ` Andreas Dilger
2014-01-21 6:23 ` Theodore Ts'o
2014-01-23 21:28 ` Andreas Dilger
2014-01-20 16:30 ` [PATCH 00/12] e2fsprogs mke2fs optimizations and new features Theodore Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140121155422.GB31769@thunk.org \
--to=tytso@mit.edu \
--cc=adilger@dilger.ca \
--cc=linux-ext4@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).