From: Chris Mason <chris.mason@oracle.com>
To: Theodore Tso <tytso@MIT.EDU>
Cc: Jacek Luczak <difrost.kernel@gmail.com>,
linux-ext4@vger.kernel.org,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-btrfs@vger.kernel.org
Subject: Re: getdents - ext4 vs btrfs performance
Date: Thu, 1 Mar 2012 09:38:59 -0500 [thread overview]
Message-ID: <20120301143859.GX5054@shiny> (raw)
In-Reply-To: <E15A491F-62E2-4518-939F-07B3AB4C3E65@mit.edu>
On Wed, Feb 29, 2012 at 11:44:31PM -0500, Theodore Tso wrote:
> You might try sorting the entries returned by readdir by inode number=
before you stat them. This is a long-standing weakness in ext3/ext4=
, and it has to do with how we added hashed tree indexes to directories=
in (a) a backwards compatible way, that (b) was POSIX compliant with r=
espect to adding and removing directory entries concurrently with readi=
ng all of the directory entries using readdir.
>=20
> You might try compiling spd_readdir from the e2fsprogs source tree (i=
n the contrib directory):
>=20
> http://git.kernel.org/?p=3Dfs/ext2/e2fsprogs.git;a=3Dblob;f=3Dcontrib=
/spd_readdir.c;h=3Df89832cd7146a6f5313162255f057c5a754a4b84;hb=3Dd9a5d3=
7535794842358e1cfe4faa4a89804ed209
>=20
> =E2=80=A6 and then using that as a LD_PRELOAD, and see how that chang=
es things.
>=20
> The short version is that we can't easily do this in the kernel since=
it's a problem that primarily shows up with very big directories, and =
using non-swappable kernel memory to store all of the directory entries=
and then sort them so they can be returned in inode number just isn't =
practical. It is something which can be easily done in userspace, tho=
ugh, and a number of programs (including mutt for its Maildir support) =
does do, and it helps greatly for workloads where you are calling readd=
ir() followed by something that needs to access the inode (i.e., stat, =
unlink, etc.)
>=20
=46or reading the files, the acp program I sent him tries to do somethi=
ng
similar. I had forgotten about spd_readdir though, we should consider
hacking that into cp and tar.
One interesting note is the page cache used to help here. Picture two
tests:
A) time tar cf /dev/zero /home
and
cp -a /home /new_dir_in_new_fs
unmount / flush caches
B) time tar cf /dev/zero /new_dir_in_new_fs
On ext, The time for B used to be much faster than the time for A
because the files would get written back to disk in roughly htree order=
=2E
Based on Jacek's data, that isn't true anymore.
-chris
next prev parent reply other threads:[~2012-03-01 14:38 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-29 13:52 getdents - ext4 vs btrfs performance Jacek Luczak
2012-02-29 13:55 ` Jacek Luczak
2012-02-29 14:07 ` Jacek Luczak
2012-02-29 14:21 ` Jacek Luczak
2012-02-29 14:42 ` Chris Mason
2012-02-29 14:55 ` Jacek Luczak
2012-03-01 13:35 ` Jacek Luczak
2012-03-01 13:50 ` Hillf Danton
2012-03-01 14:03 ` Jacek Luczak
2012-03-01 14:18 ` Chris Mason
2012-03-01 14:43 ` Jacek Luczak
2012-03-01 14:51 ` Chris Mason
2012-03-01 14:57 ` Jacek Luczak
2012-03-01 18:42 ` Ted Ts'o
2012-03-02 9:51 ` Jacek Luczak
2012-03-01 4:44 ` Theodore Tso
2012-03-01 14:38 ` Chris Mason [this message]
2012-03-02 10:05 ` Jacek Luczak
2012-03-02 14:00 ` Chris Mason
2012-03-02 14:16 ` Jacek Luczak
2012-03-02 14:26 ` Chris Mason
2012-03-02 19:32 ` Ted Ts'o
2012-03-02 19:50 ` Chris Mason
2012-03-05 13:10 ` Jan Kara
2012-03-03 22:41 ` Jacek Luczak
2012-03-04 10:25 ` Jacek Luczak
2012-03-05 11:32 ` Jacek Luczak
2012-03-06 0:37 ` Chris Mason
2012-03-08 17:02 ` Phillip Susi
2012-03-09 11:29 ` Lukas Czerner
2012-03-09 14:34 ` Chris Mason
2012-03-10 0:09 ` Andreas Dilger
2012-03-10 4:48 ` Ted Ts'o
2012-03-11 10:30 ` Andreas Dilger
2012-03-11 16:13 ` Ted Ts'o
2012-03-15 10:42 ` Jacek Luczak
2012-03-18 20:56 ` Ted Ts'o
2012-03-13 19:05 ` Phillip Susi
2012-03-13 19:53 ` Ted Ts'o
2012-03-13 20:22 ` Phillip Susi
2012-03-13 21:33 ` Ted Ts'o
2012-03-14 2:48 ` Yongqiang Yang
2012-03-14 2:51 ` Ted Ts'o
2012-03-14 14:17 ` Zach Brown
2012-03-14 16:48 ` Ted Ts'o
2012-03-14 17:37 ` Zach Brown
2012-03-14 8:12 ` Lukas Czerner
2012-03-14 9:29 ` Yongqiang Yang
2012-03-14 9:38 ` Lukas Czerner
2012-03-14 12:50 ` Ted Ts'o
2012-03-14 14:34 ` Lukas Czerner
2012-03-14 17:02 ` Ted Ts'o
2012-03-14 19:17 ` Chris Mason
2012-03-14 14:28 ` Phillip Susi
2012-03-14 16:54 ` Ted Ts'o
2012-03-10 3:52 ` Ted Ts'o
2012-03-15 7:59 ` Jacek Luczak
-- strict thread matches above, loose matches on Subject: below --
2012-02-29 13:31 Jacek Luczak
2012-02-29 13:51 ` Chris Mason
2012-02-29 14:00 ` Lukas Czerner
2012-02-29 14:05 ` Chris Mason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120301143859.GX5054@shiny \
--to=chris.mason@oracle.com \
--cc=difrost.kernel@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@MIT.EDU \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).