From: "Theodore Y. Ts'o" <tytso@mit.edu>
To: Li Dongyang <dongyangli@ddn.com>
Cc: "linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>,
"adilger@dilger.ca" <adilger@dilger.ca>
Subject: Re: [PATCH v3 4/5] mke2fs: set overhead in super block
Date: Sun, 26 Jan 2020 23:15:49 -0500 [thread overview]
Message-ID: <20200127041549.GC115399@mit.edu> (raw)
In-Reply-To: <20191120043448.249988-4-dongyangli@ddn.com>
On Wed, Nov 20, 2019 at 04:35:27AM +0000, Li Dongyang wrote:
> If overhead is not recorded in the super block, it is caculated
> during mount in kernel, for bigalloc file systems the it takes
> O(groups**2) in time.
> For a 1PB deivce with 32K cluste size it takes ~12 mins to
> mount, with most of the time spent on figuring out overhead.
>
> While we can not improve the overhead algorithm in kernel
> due to the nature of bigalloc, we can work out the overhead
> during mke2fs and set it in the super block, avoiding calculating
> it every time when it mounts.
>
> Overhead is s_first_data_block plus internal journal blocks plus
> the block and inode bitmaps, inode table, super block backups and
> group descriptor blocks for every group. This patch introduces
> ext2fs_count_used_clusters(), which calculates the clusters used
> in the block bitmap for the given range.
>
> When bad blocks are involved, it gets tricky because the blocks
> counted as overhead and the bad blocks can end up in the same
> allocation cluster. In this case we will unmark the bad blocks from
> the block bitmap, convert to cluster bitmap and get the overhead,
> then mark the bad blocks back in the cluster bitmap.
>
> Reset the overhead to zero when resizing, we can not simplly count
> the used blocks as overhead like we do when mke2fs. The overhead
> can be calculated by kernel side during mount.
>
> Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Applied, but I had to fix up a number of spelling errors in the commit
description, and fix up a large number of test failures caused by this
change.
- Ted
next prev parent reply other threads:[~2020-01-27 6:29 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-20 4:35 [PATCH v3 1/5] libext2fs: optimize ext2fs_convert_subcluster_bitmap() Li Dongyang
2019-11-20 4:35 ` [PATCH v3 2/5] mke2fs: fix setting bad blocks in the block bitmap Li Dongyang
2019-12-31 3:47 ` Theodore Y. Ts'o
2019-11-20 4:35 ` [PATCH v3 3/5] ext2fs: rename "s_overhead_blocks" to "s_overhead_clusters" Li Dongyang
2020-01-27 3:05 ` Theodore Y. Ts'o
2019-11-20 4:35 ` [PATCH v3 4/5] mke2fs: set overhead in super block Li Dongyang
2020-01-27 4:15 ` Theodore Y. Ts'o [this message]
2019-11-20 4:35 ` [PATCH v3 5/5] ext4: record " Li Dongyang
2020-01-27 2:56 ` [PATCH v3 1/5] libext2fs: optimize ext2fs_convert_subcluster_bitmap() Theodore Y. Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200127041549.GC115399@mit.edu \
--to=tytso@mit.edu \
--cc=adilger@dilger.ca \
--cc=dongyangli@ddn.com \
--cc=linux-ext4@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.