From: Chris Mason <chris.mason@oracle.com>
To: Theodore Tso <tytso@MIT.EDU>
Cc: Jacek Luczak <difrost.kernel@gmail.com>,
linux-ext4@vger.kernel.org,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-btrfs@vger.kernel.org
Subject: Re: getdents - ext4 vs btrfs performance
Date: Thu, 1 Mar 2012 09:38:59 -0500 [thread overview]
Message-ID: <20120301143859.GX5054@shiny> (raw)
In-Reply-To: <E15A491F-62E2-4518-939F-07B3AB4C3E65@mit.edu>
On Wed, Feb 29, 2012 at 11:44:31PM -0500, Theodore Tso wrote:
> You might try sorting the entries returned by readdir by inode number before you stat them. This is a long-standing weakness in ext3/ext4, and it has to do with how we added hashed tree indexes to directories in (a) a backwards compatible way, that (b) was POSIX compliant with respect to adding and removing directory entries concurrently with reading all of the directory entries using readdir.
>
> You might try compiling spd_readdir from the e2fsprogs source tree (in the contrib directory):
>
> http://git.kernel.org/?p=fs/ext2/e2fsprogs.git;a=blob;f=contrib/spd_readdir.c;h=f89832cd7146a6f5313162255f057c5a754a4b84;hb=d9a5d37535794842358e1cfe4faa4a89804ed209
>
> … and then using that as a LD_PRELOAD, and see how that changes things.
>
> The short version is that we can't easily do this in the kernel since it's a problem that primarily shows up with very big directories, and using non-swappable kernel memory to store all of the directory entries and then sort them so they can be returned in inode number just isn't practical. It is something which can be easily done in userspace, though, and a number of programs (including mutt for its Maildir support) does do, and it helps greatly for workloads where you are calling readdir() followed by something that needs to access the inode (i.e., stat, unlink, etc.)
>
For reading the files, the acp program I sent him tries to do something
similar. I had forgotten about spd_readdir though, we should consider
hacking that into cp and tar.
One interesting note is the page cache used to help here. Picture two
tests:
A) time tar cf /dev/zero /home
and
cp -a /home /new_dir_in_new_fs
unmount / flush caches
B) time tar cf /dev/zero /new_dir_in_new_fs
On ext, The time for B used to be much faster than the time for A
because the files would get written back to disk in roughly htree order.
Based on Jacek's data, that isn't true anymore.
-chris
next prev parent reply other threads:[~2012-03-01 14:38 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-29 13:52 getdents - ext4 vs btrfs performance Jacek Luczak
2012-02-29 13:55 ` Jacek Luczak
2012-02-29 14:07 ` Jacek Luczak
2012-02-29 14:21 ` Jacek Luczak
2012-02-29 14:42 ` Chris Mason
2012-02-29 14:55 ` Jacek Luczak
2012-03-01 13:35 ` Jacek Luczak
2012-03-01 13:50 ` Hillf Danton
2012-03-01 14:03 ` Jacek Luczak
2012-03-01 14:18 ` Chris Mason
2012-03-01 14:43 ` Jacek Luczak
2012-03-01 14:51 ` Chris Mason
2012-03-01 14:57 ` Jacek Luczak
2012-03-01 18:42 ` Ted Ts'o
2012-03-02 9:51 ` Jacek Luczak
2012-03-01 4:44 ` Theodore Tso
2012-03-01 14:38 ` Chris Mason [this message]
2012-03-02 10:05 ` Jacek Luczak
2012-03-02 14:00 ` Chris Mason
2012-03-02 14:16 ` Jacek Luczak
2012-03-02 14:26 ` Chris Mason
2012-03-02 19:32 ` Ted Ts'o
2012-03-02 19:50 ` Chris Mason
2012-03-05 13:10 ` Jan Kara
2012-03-03 22:41 ` Jacek Luczak
2012-03-04 10:25 ` Jacek Luczak
2012-03-05 11:32 ` Jacek Luczak
2012-03-06 0:37 ` Chris Mason
2012-03-08 17:02 ` Phillip Susi
2012-03-09 11:29 ` Lukas Czerner
2012-03-09 14:34 ` Chris Mason
2012-03-10 0:09 ` Andreas Dilger
2012-03-10 4:48 ` Ted Ts'o
2012-03-11 10:30 ` Andreas Dilger
2012-03-11 16:13 ` Ted Ts'o
2012-03-15 10:42 ` Jacek Luczak
2012-03-18 20:56 ` Ted Ts'o
2012-03-13 19:05 ` Phillip Susi
2012-03-13 19:53 ` Ted Ts'o
2012-03-13 20:22 ` Phillip Susi
2012-03-13 21:33 ` Ted Ts'o
2012-03-14 2:48 ` Yongqiang Yang
2012-03-14 2:51 ` Ted Ts'o
2012-03-14 14:17 ` Zach Brown
2012-03-14 16:48 ` Ted Ts'o
2012-03-14 17:37 ` Zach Brown
2012-03-14 8:12 ` Lukas Czerner
2012-03-14 9:29 ` Yongqiang Yang
2012-03-14 9:38 ` Lukas Czerner
2012-03-14 12:50 ` Ted Ts'o
2012-03-14 14:34 ` Lukas Czerner
2012-03-14 17:02 ` Ted Ts'o
2012-03-14 19:17 ` Chris Mason
2012-03-14 14:28 ` Phillip Susi
2012-03-14 16:54 ` Ted Ts'o
2012-03-10 3:52 ` Ted Ts'o
2012-03-15 7:59 ` Jacek Luczak
-- strict thread matches above, loose matches on Subject: below --
2012-02-29 13:31 Jacek Luczak
2012-02-29 13:51 ` Chris Mason
2012-02-29 14:00 ` Lukas Czerner
2012-02-29 14:05 ` Chris Mason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120301143859.GX5054@shiny \
--to=chris.mason@oracle.com \
--cc=difrost.kernel@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@MIT.EDU \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).