From: Andreas Dilger <adilger@sun.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: "Theodore Ts'o" <tytso@MIT.EDU>,
linux-kernel@vger.kernel.org, alex@clusterfs.com,
adilger@clusterfs.com, aneesh.kumar@linux.vnet.ibm.com,
sandeen@redhat.com,
"linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>
Subject: Re: [PATCH 41/49] ext4: Add multi block allocator for ext4
Date: Wed, 23 Jan 2008 16:20:23 -0700 [thread overview]
Message-ID: <20080123232023.GA18891@webber.adilger.int> (raw)
In-Reply-To: <20080123140727.f47e9c9d.akpm@linux-foundation.org>
On Jan 23, 2008 14:07 -0800, Andrew Morton wrote:
> > +#define mb_correct_addr_and_bit(bit, addr) \
> > +{ \
> > + bit += ((unsigned long) addr & 3UL) << 3; \
> > + addr = (void *) ((unsigned long) addr & ~3UL); \
> > +}
>
> Why do these exist?
They seem to be a holdover from when mballoc stored the buddy bitmaps
on disk. That no longer happens (to avoid bitmap vs. buddy consistency
problems), so I suspect they can be removed.
I can't comment on many of the other issues because Alex wrote most
of the code.
> Gosh what a lot of code. Is it faster?
Yes, and also importantly it uses a lot less CPU to do a given amount
of allocation, which is critical in our environments where there is
very high disk bandwidth on a single node and CPU becomes the limiting
factor of the IO speed. This of course also helps any write-intensive
environment where the CPU is doing something "useful".
Some older test results include:
https://ols2006.108.redhat.com/2007/Reprints/mathur-Reprint.pdf (Section 7)
In the linux-ext4 thread "compilebench numbers for ext4":
http://www.mail-archive.com/linux-ext4@vger.kernel.org/msg03834.html
http://oss.oracle.com/~mason/compilebench/ext4/ext-create-compare.png
http://oss.oracle.com/~mason/compilebench/ext4/ext-compile-compare.png
http://oss.oracle.com/~mason/compilebench/ext4/ext-read-compare.png
http://oss.oracle.com/~mason/compilebench/ext4/ext-rm-compare.png
note the ext-read-compare.png graph shows lower read performance, but
a couple of bugs in mballoc were since fixed to have ext4 allocate more
contiguous extents.
In the old linux-ext4 thread "[RFC] delayed allocation testing on node zefir"
http://www.mail-archive.com/linux-ext4@vger.kernel.org/msg00587.html
: dd2048rw
: REAL UTIME STIME READ WRITTEN DETAILS
EXT3 : 58.46 23 1491 2572 2097292 17 extents
EXT4 : 44.56 19 1018 12 2097244 19 extents
REISERFS: 56.80 26 1370 2952 2097336 457 extents
JFS : 45.77 22 984 0 2097216 1 extents
XFS : 50.97 20 1394 0 2100825 7 extents
: kernuntar
: REAL UTIME STIME READ WRITTEN DETAILS
EXT3 : 56.99 5037 651 68 252016
EXT4 : 55.03 5034 553 36 249884
REISERFS: 52.55 4996 854 64 238068
JFS : 70.15 5057 630 496 288116
XFS : 72.84 5052 953 132 316798
: kernstat
: REAL UTIME STIME READ WRITTEN DETAILS
EXT3 : 2.83 8 15 5892 0
EXT4 : 0.51 9 10 5892 0
REISERFS: 0.81 7 49 2696 0
JFS : 6.19 11 49 12552 0
XFS : 2.09 9 61 6504 0
: kerncat
: REAL UTIME STIME READ WRITTEN DETAILS
EXT3 : 9.48 25 213 241624 0
EXT4 : 6.29 27 197 238560 0
REISERFS: 14.69 33 230 234744 0
JFS : 23.51 23 231 244596 0
XFS : 18.24 36 254 238548 0
: kernrm
: REAL UTIME STIME READ WRITTEN DETAILS
EXT3 : 4.82 4 108 9628 4672
EXT4 : 1.61 5 110 6536 4632
REISERFS: 3.15 8 276 2768 236
JFS : 33.90 7 168 14400 33048
XFS : 20.03 8 296 6632 86160
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
next prev parent reply other threads:[~2008-01-23 23:21 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-01-22 3:01 ext4 merge plans for 2.6.25 Theodore Ts'o
2008-01-22 3:01 ` [PATCH 01/49] ext4: Support large blocksize up to PAGESIZE Theodore Ts'o
2008-01-22 3:01 ` [PATCH 02/49] ext4: Avoid rec_len overflow with 64KB block size Theodore Ts'o
2008-01-22 3:01 ` [PATCH 03/49] ext4: Introduce ext4_lblk_t Theodore Ts'o
2008-01-22 3:01 ` [PATCH 04/49] ext4 extents: remove unneeded casts Theodore Ts'o
2008-01-22 3:01 ` [PATCH 05/49] ext4: add ext4_group_t, and change all group variables to this type Theodore Ts'o
2008-01-22 3:01 ` [PATCH 06/49] ext4: fixes block group number being set to a negative value Theodore Ts'o
2008-01-22 3:01 ` [PATCH 07/49] ext4: Introduce ext4_update_*_feature Theodore Ts'o
2008-01-22 3:01 ` [PATCH 08/49] ext4: Fix sparse warnings Theodore Ts'o
2008-01-22 3:01 ` [PATCH 09/49] ext4: Rename i_file_acl to i_file_acl_lo Theodore Ts'o
2008-01-22 3:01 ` [PATCH 10/49] ext4: Rename i_dir_acl to i_size_high Theodore Ts'o
2008-01-22 3:01 ` [PATCH 11/49] ext4: Add support for 48 bit inode i_blocks Theodore Ts'o
2008-01-22 3:01 ` [PATCH 12/49] ext4: Support large files Theodore Ts'o
2008-01-22 3:01 ` [PATCH 13/49] ext4: different maxbytes functions for bitmap & extent files Theodore Ts'o
2008-01-22 3:01 ` [PATCH 14/49] ext4: export iov_shorten from kernel for ext4's use Theodore Ts'o
2008-01-22 3:01 ` [PATCH 15/49] ext4: store maxbytes for bitmapped files and return EFBIG as appropriate Theodore Ts'o
2008-01-22 3:01 ` [PATCH 16/49] ext2: Fix the max file size for ext2 file system Theodore Ts'o
2008-01-22 3:01 ` [PATCH 17/49] ext3: Fix the max file size for ext3 " Theodore Ts'o
2008-01-22 3:01 ` [PATCH 18/49] ext4: sync up block group descriptor with e2fsprogs Theodore Ts'o
2008-01-22 3:01 ` [PATCH 19/49] ext4: Return after ext4_error in case of failures Theodore Ts'o
2008-01-22 3:01 ` [PATCH 20/49] ext4/super.c: fix #ifdef's (CONFIG_EXT4_* -> CONFIG_EXT4DEV_*) Theodore Ts'o
2008-01-22 3:02 ` [PATCH 21/49] ext4: fix oops on corrupted ext4 mount Theodore Ts'o
2008-01-22 3:02 ` [PATCH 22/49] ext4: Change the default behaviour on error Theodore Ts'o
2008-01-22 3:02 ` [PATCH 23/49] Add buffer head related helper functions Theodore Ts'o
2008-01-22 3:02 ` [PATCH 24/49] ext4: add block bitmap validation Theodore Ts'o
2008-01-22 3:02 ` [PATCH 25/49] jbd2: Remove printk from J_ASSERT to preserve registers during BUG Theodore Ts'o
2008-01-22 3:02 ` [PATCH 26/49] jbd2: Fix assertion failure in fs/jbd2/checkpoint.c Theodore Ts'o
2008-01-22 3:02 ` [PATCH 27/49] ext4: Check for the correct error return from Theodore Ts'o
2008-01-22 3:02 ` [PATCH 28/49] ext4: remove unused code from ext4_find_entry() Theodore Ts'o
2008-01-22 3:02 ` [PATCH 29/49] ext4: Make ext4_get_blocks_wrap take the truncate_mutex early Theodore Ts'o
2008-01-22 3:02 ` [PATCH 30/49] ext4: Convert truncate_mutex to read write semaphore Theodore Ts'o
2008-01-22 3:02 ` [PATCH 31/49] ext4: Take read lock during overwrite case Theodore Ts'o
2008-01-22 3:02 ` [PATCH 32/49] jbd2: jbd2 stats through procfs Theodore Ts'o
2008-01-22 3:02 ` [PATCH 33/49] ext4: Add the journal checksum feature Theodore Ts'o
2008-01-22 3:02 ` [PATCH 34/49] vfs: Add 64 bit i_version support Theodore Ts'o
2008-01-22 3:02 ` [PATCH 35/49] ext4: Add inode version support in ext4 Theodore Ts'o
2008-01-22 3:02 ` [PATCH 36/49] ext4: Add EXT4_IOC_MIGRATE ioctl Theodore Ts'o
2008-01-22 3:02 ` [PATCH 37/49] ext4: Fix ext4_show_options to show the correct mount options Theodore Ts'o
2008-01-22 3:02 ` [PATCH 38/49] ext4: fix up EXT4FS_DEBUG builds Theodore Ts'o
2008-01-22 3:02 ` [PATCH 39/49] ext4: Add ext4_find_next_bit() Theodore Ts'o
2008-01-22 3:02 ` [PATCH 40/49] ext4: Add new functions for searching extent tree Theodore Ts'o
2008-01-22 3:02 ` [PATCH 41/49] ext4: Add multi block allocator for ext4 Theodore Ts'o
2008-01-22 3:02 ` [PATCH 42/49] ext4: Enable the multiblock allocator by default Theodore Ts'o
2008-01-22 3:02 ` [PATCH 43/49] ext4: Check for return value from sb_set_blocksize Theodore Ts'o
2008-01-22 3:02 ` [PATCH 44/49] ext4: fix uniniatilized extent splitting error Theodore Ts'o
2008-01-22 3:02 ` [PATCH 45/49] ext4: Use the ext4_ext_actual_len() helper function Theodore Ts'o
2008-01-22 3:02 ` [PATCH 46/49] jbd2: add lockdep support Theodore Ts'o
2008-01-22 3:02 ` [PATCH 47/49] jbd2: Mark jbd2 slabs as SLAB_TEMPORARY Theodore Ts'o
2008-01-22 3:02 ` [PATCH 48/49] jbd2: Use round-jiffies() function for the "5 second" ext4/jbd2 wakeup Theodore Ts'o
2008-01-22 3:02 ` [PATCH 49/49] jbd2: sparse pointer use of zero as null Theodore Ts'o
2008-01-23 22:07 ` [PATCH 41/49] ext4: Add multi block allocator for ext4 Andrew Morton
2008-01-23 23:20 ` Andreas Dilger [this message]
2008-01-24 7:56 ` Aneesh Kumar K.V
2008-01-24 9:04 ` Aneesh Kumar K.V
2008-01-24 14:53 ` Aneesh Kumar K.V
2008-01-28 18:45 ` Eric Sandeen
2008-01-23 22:07 ` [PATCH 36/49] ext4: Add EXT4_IOC_MIGRATE ioctl Andrew Morton
2008-01-24 5:55 ` Aneesh Kumar K.V
2008-01-26 4:15 ` Theodore Tso
2008-01-26 8:42 ` Aneesh Kumar K.V
2008-01-23 22:07 ` [PATCH 33/49] ext4: Add the journal checksum feature Andrew Morton
2008-01-23 22:40 ` Andreas Dilger
2008-01-24 21:24 ` Mingming Cao
2008-02-01 20:50 ` Girish Shilamkar
2008-01-23 22:06 ` [PATCH 30/49] ext4: Convert truncate_mutex to read write semaphore Andrew Morton
2008-01-24 5:29 ` Aneesh Kumar K.V
2008-01-24 13:00 ` Andy Whitcroft
2008-01-23 22:06 ` [PATCH 24/49] ext4: add block bitmap validation Andrew Morton
2008-01-26 13:26 ` Theodore Tso
2008-01-23 22:06 ` [PATCH 23/49] Add buffer head related helper functions Andrew Morton
2008-01-24 5:22 ` Aneesh Kumar K.V
2008-01-24 8:53 ` Andrew Morton
2008-01-23 12:43 ` ext4 merge plans for 2.6.25 Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080123232023.GA18891@webber.adilger.int \
--to=adilger@sun.com \
--cc=adilger@clusterfs.com \
--cc=akpm@linux-foundation.org \
--cc=alex@clusterfs.com \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=sandeen@redhat.com \
--cc=tytso@MIT.EDU \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox