From: Coly Li <i@coly.li>
To: Theodore Tso <tytso@MIT.EDU>
Cc: Andreas Dilger <adilger@whamcloud.com>,
linux-ext4 development <linux-ext4@vger.kernel.org>,
Alex Zhuravlev <bzzz@whamcloud.com>, Tao Ma <tm@tao.ma>,
"hao.bigrat@gmail.com" <hao.bigrat@gmail.com>
Subject: Re: bigalloc and max file size
Date: Mon, 31 Oct 2011 17:35:16 +0800 [thread overview]
Message-ID: <4EAE6BD4.9080705@coly.li> (raw)
In-Reply-To: <F1D09DA1-3E1E-4D31-9F26-4AADAAF7A91D@mit.edu>
On 2011年10月31日 03:49, Theodore Tso Wrote:
>
> On Oct 30, 2011, at 1:37 AM, Coly Li wrote:
>
>> Forgive me if this is out of topic.
>> In our test, allocating directories W/ bigalloc and W/O inline-data may occupy most of disk space. By now Ext4
>> inline-data is not merged yet, I just wondering how Google uses bigalloc without inline-data patch set ?
>
> It depends on how many directories you have (i.e, how deep your directory structure is) and how many small files you have in the file system as to whether bigalloc w/o inline-data has an acceptable overhead or not.
[snip]
> I'm not against your patch set, however; I just haven't had time to look at them, at all (nor the secure delete patch set, etc.) . Between organizing the kernel summit, the kernel.org compromise, and some high priority bugs at $WORK, things have just been too busy. Sorry for that; I'll get to them after the merge window and post-merge bug fixing is under control.
Hi Ted,
In our test, bigalloc without inline-data dose not work very well with deep directory structure, e.g. Hadoop or Squid,
because small directories occupies all disk space. That's why I asked the question. Thanks for your patient reply, it
makes sense for me :-)
Back to our topic, Ext4 doesn't have too much on-disk incompatible flag-bits now. If we get current bigalloc code merged
now, we have to use another incompatible bit when we merge cluster/chunk based extent patch set. Further more, we
observe performance regression without cluster-based-extent on file system umount (as Tao mentioned in this thread).
IMHO, without inline-data and cluster-based-extent, current bigalloc code is a little bit inperfect for many users.
Bigalloc is a very useful feature, can we consider making it better before getting merged ?
Thanks.
--
Coly Li
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-10-31 9:25 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-10-26 23:36 bigalloc and max file size Andreas Dilger
2011-10-27 1:05 ` Tao Ma
2011-10-27 6:35 ` Theodore Tso
[not found] ` <EB03FF23-73BC-4FDC-B991-5EB3FEEB8DAE@whamcloud.com>
2011-10-27 11:48 ` Theodore Tso
[not found] ` <97D9C5CC-0F22-4BC7-BDFA-7781D33CA7F3@whamcloud.com>
2011-10-27 21:42 ` Theodore Tso
2011-10-28 3:31 ` Tao Ma
2011-10-31 10:15 ` Theodore Tso
2011-10-31 10:27 ` Tao Ma
2011-10-31 18:53 ` Sunil Mushran
2011-10-31 19:09 ` Andreas Dilger
2011-10-31 20:00 ` Ted Ts'o
2011-11-01 4:06 ` Tao Ma
2011-10-30 5:37 ` Coly Li
2011-10-30 19:49 ` Theodore Tso
2011-10-31 9:35 ` Coly Li [this message]
2011-10-31 10:22 ` Theodore Tso
2011-10-31 16:08 ` Andreas Dilger
2011-10-31 16:22 ` Ted Ts'o
2011-10-31 17:39 ` Coly Li
2011-10-31 19:38 ` Ted Ts'o
2011-11-01 1:10 ` Coly Li
2011-11-01 11:47 ` Theodore Tso
2011-11-01 12:22 ` Coly Li
2011-10-31 16:34 ` Andreas Dilger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4EAE6BD4.9080705@coly.li \
--to=i@coly.li \
--cc=adilger@whamcloud.com \
--cc=bzzz@whamcloud.com \
--cc=hao.bigrat@gmail.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tm@tao.ma \
--cc=tytso@MIT.EDU \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).