public inbox for linux-ext4@vger.kernel.org
 help / color / mirror / Atom feed
From: Mingming Cao <cmm@us.ibm.com>
To: tytso <tytso@mit.edu>
Cc: linux-ext4@vger.kernel.org,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
	Andreas Dilger <adilger@sun.com>
Subject: [PATCH 5/6 ]Ext4 journal credits reservation fixes
Date: Tue, 12 Aug 2008 09:35:38 -0700	[thread overview]
Message-ID: <1218558938.6766.55.camel@mingming-laptop> (raw)
In-Reply-To: <1218558190.6766.37.camel@mingming-laptop>

Ext4: journal credit fix the delalloc writepages

From: Mingming Cao <cmm@us.ibm.com>

Previous delalloc writepages implementation start a new transaction outside
a loop call of get_block() to do the block allocation. Due to lack of information 
of how many blocks to be allocated, the estimate of the journal credits is very
 Conservative and caused many issues.

With the reworked delayed allocation, a new transaction is created for
each get_block(), thus we don't need to guess how many credits for the multiple
chunk of allocation. Start every transaction with credits for insert a single 
extent is enough. But we still need to consider the journalled mode, where
it need to account for the number of data blocks.  So we guess max number of
data blocks for each allocation.  Due to the current VFS implementation
writepages() could only flush PAGEVEC of pages at a time, the max block
allocation is limited and calculated based on that, an the total number
of reserved delalloc datablocks, whichever is smaller.

Signed-off-by: Mingming Cao <cmm@us.ibm.com>
---
 fs/ext4/inode.c |   39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

Index: linux-2.6.27-rc1/fs/ext4/inode.c
===================================================================
--- linux-2.6.27-rc1.orig/fs/ext4/inode.c	2008-08-12 08:15:59.000000000 -0700
+++ linux-2.6.27-rc1/fs/ext4/inode.c	2008-08-12 08:30:41.000000000 -0700
@@ -2210,17 +2210,28 @@ static int ext4_da_writepage(struct page
 }
 
 /*
- * For now just follow the DIO way to estimate the max credits
- * needed to write out EXT4_MAX_WRITEBACK_PAGES.
- * todo: need to calculate the max credits need for
- * extent based files, currently the DIO credits is based on
- * indirect-blocks mapping way.
- *
- * Probably should have a generic way to calculate credits
- * for DIO, writepages, and truncate
+ * This is called via ext4_da_writepages() to
+ * calulate the total number of credits to reserve to fit
+ * a single extent allocation into a single transaction,
+ * ext4_da_writpeages() will loop calling this before
+ * the block allocation.
+ *
+ * The page vector size limited the max number of pages could
+ * be writeout at a time. Based on this, the max blocks to pass to
+ * get_block is calculated
  */
-#define EXT4_MAX_WRITEBACK_PAGES      DIO_MAX_BLOCKS
-#define EXT4_MAX_WRITEBACK_CREDITS    25
+
+#define		EXT4_MAX_WRITEPAGES_SIZE	PAGEVEC_SIZE
+static int ext4_writepages_trans_blocks(struct inode *inode)
+{
+	int bpp = ext4_journal_blocks_per_page(inode);
+	int max_blocks = EXT4_MAX_WRITEPAGES_SIZE * bpp;
+
+	if (max_blocks > EXT4_I(inode)->i_reserved_data_blocks)
+		max_blocks =  EXT4_I(inode)->i_reserved_data_blocks;
+
+	return ext4_data_trans_blocks(inode, max_blocks);
+}
 
 static int ext4_da_writepages(struct address_space *mapping,
                                 struct writeback_control *wbc)
@@ -2262,7 +2273,7 @@ restart_loop:
 		 * by delalloc
 		 */
 		BUG_ON(ext4_should_journal_data(inode));
-		needed_blocks = EXT4_DATA_TRANS_BLOCKS(inode->i_sb);
+		needed_blocks = ext4_writepages_trans_blocks(inode);
 
 		/* start a new transaction*/
 		handle = ext4_journal_start(inode, needed_blocks);
@@ -4449,11 +4460,9 @@ static int ext4_writeblocks_trans_credit
  * the modification of a single pages into a single transaction,
  * which may include multile chunk of block allocations.
  *
- * This could be called via ext4_write_begin() or later
- * ext4_da_writepages() in delalyed allocation case.
+ * This could be called via ext4_write_begin()
  *
- * In both case it's possible that we could allocating multiple
- * chunks of blocks. We need to consider the worse case, when
+ * We need to consider the worse case, when
  * one new block per extent.
  */
 int ext4_writepage_trans_blocks(struct inode *inode)



  parent reply	other threads:[~2008-08-12 16:36 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-07-21  4:28 Crash and stack trace for jbd2 under ext4 Shehjar Tikoo
2008-07-21  8:20 ` Aneesh Kumar K.V
2008-07-23  0:49   ` Mingming Cao
2008-07-23  0:51   ` [RFC]Ext4: journal credits reservation fixes for DIO, fallocate and delalloc writepages Mingming Cao
2008-07-23  1:18     ` Andreas Dilger
2008-07-23 18:19       ` Theodore Tso
2008-07-25 19:38       ` Mingming Cao
2008-07-23  7:42     ` Aneesh Kumar K.V
2008-07-24  2:07       ` Andreas Dilger
2008-07-25 19:26       ` Mingming Cao
2008-07-28 16:11         ` Aneesh Kumar K.V
2008-07-28 19:07           ` Mingming Cao
2008-07-29  6:24             ` Aneesh Kumar K.V
2008-07-26  0:42       ` [PATCH v2]Ext4: " Mingming Cao
2008-07-30  1:58         ` [PATCH v3]Ext4: " Mingming Cao
2008-07-30 11:29           ` Frédéric Bohé
2008-07-31 18:07             ` Mingming Cao
2008-08-01  5:49               ` Theodore Tso
2008-08-01 10:51                 ` Frédéric Bohé
2008-08-01 18:08                   ` Mingming Cao
2008-08-01 18:03                 ` Mingming Cao
2008-08-01 19:10                   ` Theodore Tso
2008-08-02  0:03                     ` Theodore Tso
2008-08-04 11:23                       ` Frédéric Bohé
2008-08-04 13:20                         ` Theodore Tso
2008-07-30 11:36           ` Andreas Dilger
2008-07-30 12:16           ` Aneesh Kumar K.V
2008-08-01 19:29           ` Theodore Tso
2008-08-02  0:22             ` Theodore Tso
2008-08-12 16:23         ` [PATCH 0/6 ]Ext4 journal credits reservation fixes Mingming Cao
2008-08-12 16:25           ` [PATCH 1/6 ]Ext4 credits caclulation cleanup and fix that for nonextent writepage Mingming Cao
2008-08-13  8:31             ` Aneesh Kumar K.V
2008-08-14  0:30               ` Mingming Cao
2008-08-13 10:19             ` Aneesh Kumar K.V
2008-08-14  1:02               ` Mingming Cao
2008-08-16  0:37             ` [PATCH 1/6 V2 " Mingming Cao
2008-08-12 16:27           ` [PATCH 2/6 ]Ext4: journal credits reservation fixes for extent file writepage Mingming Cao
2008-08-13  8:37             ` Aneesh Kumar K.V
2008-08-14  0:26               ` Mingming Cao
2008-08-14  8:28                 ` Aneesh Kumar K.V
2008-08-16  0:38             ` [PATCH 2/6 V2]Ext4: " Mingming Cao
2008-08-16  4:25               ` Aneesh Kumar K.V
2008-08-12 16:29           ` [PATCH 3/6 ]Ext4: journal credits reservation fixes for DIO, fallocate Mingming Cao
2008-08-13  8:53             ` Aneesh Kumar K.V
2008-08-13 10:14               ` Aneesh Kumar K.V
2008-08-14  0:50               ` Mingming Cao
2008-08-16  0:39             ` [PATCH 3/6 V2 " Mingming Cao
2008-08-12 16:32           ` [PATCH 4/6 ]ext4: Rework the ext4_da_writepages Mingming Cao
2008-08-16  0:43             ` [PATCH 4/6 V2]ext4: " Mingming Cao
2008-08-12 16:35           ` Mingming Cao [this message]
2008-08-13  9:46             ` [PATCH 5/6 ]Ext4 journal credits reservation fixes Aneesh Kumar K.V
2008-08-14  1:01               ` Mingming Cao
2008-08-14  8:40                 ` Aneesh Kumar K.V
2008-08-16  0:40             ` [PATCH 5/6 V2]Ext4 journal credits fixes for delalloc writepages Mingming Cao
2008-08-16  4:23               ` Aneesh Kumar K.V
2008-08-12 16:37           ` [PATCH 6/6 ]Ext4 journal credits reservation fixes for defrag Mingming Cao
2008-08-16  0:45             ` [PATCH 6/6 V2]Ext4 " Mingming Cao
2008-08-16 15:55               ` Theodore Tso
2008-08-15 17:33           ` [PATCH 0/6 ]Ext4 journal credits reservation fixes Aneesh Kumar K.V
2008-08-15 19:02             ` Mingming Cao
2008-08-16  0:34               ` Mingming Cao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1218558938.6766.55.camel@mingming-laptop \
    --to=cmm@us.ibm.com \
    --cc=adilger@sun.com \
    --cc=aneesh.kumar@linux.vnet.ibm.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox