linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Liu Bo <liubo2009@cn.fujitsu.com>
To: <linux-btrfs@vger.kernel.org>
Cc: <dave@jikos.cz>, <chris.mason@oracle.com>, <josef@redhat.com>
Subject: [GIT PULL v4] Btrfs: improve write ahead log with sub transaction
Date: Thu, 30 Jun 2011 15:36:20 +0800	[thread overview]
Message-ID: <1309419392-17770-1-git-send-email-liubo2009@cn.fujitsu.com> (raw)

I've been working to try to improve the write-ahead log's performance,
and I found that the bottleneck addresses in the checksum items,
especially when we want to make a random write on a large file, e.g a 4G file.

Then a idea for this suggested by Chris is to use sub transaction ids and just
to log the part of inode that had changed since either the last log commit or
the last transaction commit.  And as we also push the sub transid into the btree
blocks, we'll get much faster tree walks.  As a result, we abandon the original
brute force approach, which is "to delete all items of the inode in log",
to making sure we get the most uptodate copies of everything, and instead
we manage to "find and merge", i.e. finding extents in the log tree and merging
in the new extents from the file.

This patchset puts the above idea into code, and although the code is now more
complex, it brings us a great deal of performance improvement:

in my sysbench "write + fsync" test:

        451.01Kb/sec -> 4.3621Mb/sec

Also, I've run the "synctest", and it works well with both directory and file.

v1->v2, rebase.
v2->v3, thanks to Chris, we worked together to solve 2 bugs, and after that it
worked as expected.
v3->v4, thanks to Josef, we simplify several codes.
 
Liu Bo (12):
  Btrfs: introduce sub transaction stuff
  Btrfs: update block generation if should_cow_block fails
  Btrfs: modify btrfs_drop_extents API
  Btrfs: introduce first sub trans
  Btrfs: still update inode trans stuff when size remains unchanged
  Btrfs: improve log with sub transaction
  Btrfs: add checksum check for log
  Btrfs: fix a bug of log check
  Btrfs: kick off useless code
  Btrfs: use the right generation number to read log_root_tree
  Btrfs: do not iput inode when inode is still in log
  Revert "Btrfs: do not flush csum items of unchanged file data during
    treelog"

 fs/btrfs/btrfs_inode.h |   12 ++-
 fs/btrfs/ctree.c       |   69 +++++++++++----
 fs/btrfs/ctree.h       |    5 +-
 fs/btrfs/disk-io.c     |   12 ++--
 fs/btrfs/extent-tree.c |   10 ++-
 fs/btrfs/file.c        |   22 ++---
 fs/btrfs/inode.c       |   39 ++++++---
 fs/btrfs/ioctl.c       |    6 +-
 fs/btrfs/relocation.c  |    6 +-
 fs/btrfs/transaction.c |   13 ++-
 fs/btrfs/transaction.h |   19 ++++-
 fs/btrfs/tree-defrag.c |    2 +-
 fs/btrfs/tree-log.c    |  225 ++++++++++++++++++++++++++++++++----------------
 13 files changed, 293 insertions(+), 147 deletions(-)


             reply	other threads:[~2011-06-30  7:36 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-06-30  7:36 Liu Bo [this message]
2011-06-30  7:36 ` [PATCH 01/12] Btrfs: introduce sub transaction stuff Liu Bo
2011-06-30  7:36 ` [PATCH 02/12] Btrfs: update block generation if should_cow_block fails Liu Bo
2011-06-30  7:36 ` [PATCH 03/12] Btrfs: modify btrfs_drop_extents API Liu Bo
2011-06-30  7:36 ` [PATCH 04/12] Btrfs: introduce first sub trans Liu Bo
2011-06-30  7:36 ` [PATCH 05/12] Btrfs: still update inode trans stuff when size remains unchanged Liu Bo
2011-06-30  7:36 ` [PATCH 06/12] Btrfs: improve log with sub transaction Liu Bo
2011-06-30  7:36 ` [PATCH 07/12] Btrfs: add checksum check for log Liu Bo
2011-06-30  7:36 ` [PATCH 08/12] Btrfs: fix a bug of log check Liu Bo
2011-06-30  7:36 ` [PATCH 09/12] Btrfs: kick off useless code Liu Bo
2011-06-30  7:36 ` [PATCH 10/12] Btrfs: use the right generation number to read log_root_tree Liu Bo
2011-06-30  7:36 ` [PATCH 11/12] Btrfs: do not iput inode when inode is still in log Liu Bo
2011-06-30  7:36 ` [PATCH 12/12] Revert "Btrfs: do not flush csum items of unchanged file data during treelog" Liu Bo
2011-06-30  9:27 ` [GIT PULL v4] Btrfs: improve write ahead log with sub transaction liubo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1309419392-17770-1-git-send-email-liubo2009@cn.fujitsu.com \
    --to=liubo2009@cn.fujitsu.com \
    --cc=chris.mason@oracle.com \
    --cc=dave@jikos.cz \
    --cc=josef@redhat.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).