linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC] delayed allocation, mballoc, etc
@ 2006-12-01  0:15 Alex Tomas
  2006-12-07 17:18 ` Valerie Clement
  0 siblings, 1 reply; 5+ messages in thread
From: Alex Tomas @ 2006-12-01  0:15 UTC (permalink / raw)
  To: linux-ext4


Good day,

I'd like to ask the community to discuss and review few things
I've been working on. we propose set of patches with intention
to improve performance of ext4:

 * locality groups

   to achieve good performance writing many small files
   we need to allocate them closely each to other. the
   simplest way could be to allocate all small files using
   next block after the previous small file. and this would
   work well for a single-job case. for multi-job case (few
   untar's, for example) this would break job locality and
   cause performance penaly in subsequent access. locality
   groups idea may help here: let's group all files by some
   property. pgid, for example. now, every time the kernel
   ask filesystem to flush dirty pages, we flush inodes from
   1st group, then from 2nd and go on. this one we can form
   large contiguous allocations (for a whole group) achieving
   good throughput and preserve quite good locality.

 * scalable block reservation

   this is required to protect from -ENOSPC when pages enter
   pagecache w/o space allocation (delayed allocation). it
   also should scale well on high-end SMP as every cpu has
   one "pool" of block. when pool is empty, the filesystem
   rebalance free blocks between all cpus

 * mballoc v4

   multiblock allocator. it's supposed to be ablo to allocate
   many blocks at once saving cpu.

   with the following changes since v2 published before:

     a) per-inode preallocation
 
        every regular inode may have few preallocated chunks
        assigned to specific logical offset. it's intended to
        help applications like IOR and p2p

     b) per-locality-group preallocation

        a locality group may have few preallocated chunks

     c) buddy structures aren't stored on a disk, instead
        they are regenerated from on-disk bitmaps on demand

     d) has stride option to align requests (useful for arrays)

 * delayed allocation

   not that many changes have been done since the previous
   publication: few bugfixes and tweaks, adopted to new mballoc

as usual, there are tons of things yet to be done/fixed/tweaked.
I'm trying to keep them uptodate in TODOs.

few tests have been done. I'm sending the numbers (as well as
the patches) in the subsequent mails. please, have a look.

all the series can be found at
    ftp://ftp.clusterfs.com/pub/people/alex/2.6.19-rc6/

to enable the features, ext4 should be mounted with options:
   extents,mballoc,delalloc

any comments and questions are very welcome.

thanks, Alex

PS. I'd like to give thanks to CFS for help. especially to
    Peter Braam and Andreas Dilger who feed me with ideas.

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: [RFC] delayed allocation, mballoc, etc
@ 2006-12-27 11:09 sho
  2006-12-27 11:16 ` Alex Tomas
  0 siblings, 1 reply; 5+ messages in thread
From: sho @ 2006-12-27 11:09 UTC (permalink / raw)
  To: alex; +Cc: linux-ext4

Hi Alex

I found a bug on linux-2.6.19-rc6 with Alex's patches.

With no files on the device, doing the following system call:
1. open with O_CREAT
	fd = open("test_file", O_RDWR|O_CREAT, 0777)
2. ftruncate (length is not aligned with blocksize)
	ftruncate(fd, 200)
3. write out the same block
	write(fd, write_buf, 100)

As a result, panic occurred at the following code:
  ext4_wb_commit_write()
          BUG_ON(EXT4_I(inode)->i_locality_group == NULL);

I tracked down the scenario of causing this panic, which is as below:
1. i_locality_group is set to NULL when a file is created at first

2. Given a length which is not aligned with blocksize to ftruncate,
   PG_dirty flag is set in _set_page_dirty_nobuffers() after zeroing
   out halfway part of the block on ftruncate
   	ext4_wb_block_truncate_page()
        	kaddr = kmap_atomic(page, KM_USER0);
        	memset(kaddr + offset, 0, length);
        	flush_dcache_page(page);
        	kunmap_atomic(kaddr, KM_USER0);
        	SetPageUptodate(page);s
        	_set_page_dirty_nobuffers(page);

3. With PG_dirty flag set, i_locality_group is not set in
   ext4_lg_page_enter_inode()
     ext4_wb_commit_write()
		if (__set_page_dirty_nobuffers(page))
			ext4_lg_page_enter_inode(inode, page,
				PageMappedToDisk(page));

4. i_locality_group set to NULL causes BUG_ON

I tried the attached patch where ext4_lg_page_enter_inode()
is necessarily called.  It seems to me that the problem does not occur
with this patch, how about your comment?

diff -upNr -X linux-2.6.19-rc6/Documentation/dontdiff linux-2.6.19-rc6/fs/ext4/writeback.c linux-2.6.19-rc6-tmp/fs/ext4/writeback.c
--- linux-2.6.19-rc6/fs/ext4/writeback.c        2006-12-22 19:16:17.000000000 +0900
+++ linux-2.6.19-rc6-tmp/fs/ext4/writeback.c   2006-12-22 19:15:45.000000000 +0900
@@ -968,10 +968,8 @@ int ext4_wb_commit_write(struct file *fi
 
-       if (__set_page_dirty_nobuffers(page)) {
-                __set_page_dirty_nobuffers(page);
-               ext4_lg_page_enter_inode(inode, page, PageMappedToDisk(page));
-       }
+       __set_page_dirty_nobuffers(page);
+       ext4_lg_page_enter_inode(inode, page, PageMappedToDisk(page));


Cheers, Takashi

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2006-12-27 11:17 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-12-01  0:15 [RFC] delayed allocation, mballoc, etc Alex Tomas
2006-12-07 17:18 ` Valerie Clement
2006-12-07 17:26   ` Alex Tomas
  -- strict thread matches above, loose matches on Subject: below --
2006-12-27 11:09 sho
2006-12-27 11:16 ` Alex Tomas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).