From: Liu Bo <liubo2009@cn.fujitsu.com>
To: <linux-btrfs@vger.kernel.org>
Cc: <dave@jikos.cz>, <chris.mason@oracle.com>, <josef@redhat.com>
Subject: [GIT PULL v4] Btrfs: improve write ahead log with sub transaction
Date: Thu, 30 Jun 2011 15:36:20 +0800 [thread overview]
Message-ID: <1309419392-17770-1-git-send-email-liubo2009@cn.fujitsu.com> (raw)
I've been working to try to improve the write-ahead log's performance,
and I found that the bottleneck addresses in the checksum items,
especially when we want to make a random write on a large file, e.g a 4G file.
Then a idea for this suggested by Chris is to use sub transaction ids and just
to log the part of inode that had changed since either the last log commit or
the last transaction commit. And as we also push the sub transid into the btree
blocks, we'll get much faster tree walks. As a result, we abandon the original
brute force approach, which is "to delete all items of the inode in log",
to making sure we get the most uptodate copies of everything, and instead
we manage to "find and merge", i.e. finding extents in the log tree and merging
in the new extents from the file.
This patchset puts the above idea into code, and although the code is now more
complex, it brings us a great deal of performance improvement:
in my sysbench "write + fsync" test:
451.01Kb/sec -> 4.3621Mb/sec
Also, I've run the "synctest", and it works well with both directory and file.
v1->v2, rebase.
v2->v3, thanks to Chris, we worked together to solve 2 bugs, and after that it
worked as expected.
v3->v4, thanks to Josef, we simplify several codes.
Liu Bo (12):
Btrfs: introduce sub transaction stuff
Btrfs: update block generation if should_cow_block fails
Btrfs: modify btrfs_drop_extents API
Btrfs: introduce first sub trans
Btrfs: still update inode trans stuff when size remains unchanged
Btrfs: improve log with sub transaction
Btrfs: add checksum check for log
Btrfs: fix a bug of log check
Btrfs: kick off useless code
Btrfs: use the right generation number to read log_root_tree
Btrfs: do not iput inode when inode is still in log
Revert "Btrfs: do not flush csum items of unchanged file data during
treelog"
fs/btrfs/btrfs_inode.h | 12 ++-
fs/btrfs/ctree.c | 69 +++++++++++----
fs/btrfs/ctree.h | 5 +-
fs/btrfs/disk-io.c | 12 ++--
fs/btrfs/extent-tree.c | 10 ++-
fs/btrfs/file.c | 22 ++---
fs/btrfs/inode.c | 39 ++++++---
fs/btrfs/ioctl.c | 6 +-
fs/btrfs/relocation.c | 6 +-
fs/btrfs/transaction.c | 13 ++-
fs/btrfs/transaction.h | 19 ++++-
fs/btrfs/tree-defrag.c | 2 +-
fs/btrfs/tree-log.c | 225 ++++++++++++++++++++++++++++++++----------------
13 files changed, 293 insertions(+), 147 deletions(-)
next reply other threads:[~2011-06-30 7:36 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-06-30 7:36 Liu Bo [this message]
2011-06-30 7:36 ` [PATCH 01/12] Btrfs: introduce sub transaction stuff Liu Bo
2011-06-30 7:36 ` [PATCH 02/12] Btrfs: update block generation if should_cow_block fails Liu Bo
2011-06-30 7:36 ` [PATCH 03/12] Btrfs: modify btrfs_drop_extents API Liu Bo
2011-06-30 7:36 ` [PATCH 04/12] Btrfs: introduce first sub trans Liu Bo
2011-06-30 7:36 ` [PATCH 05/12] Btrfs: still update inode trans stuff when size remains unchanged Liu Bo
2011-06-30 7:36 ` [PATCH 06/12] Btrfs: improve log with sub transaction Liu Bo
2011-06-30 7:36 ` [PATCH 07/12] Btrfs: add checksum check for log Liu Bo
2011-06-30 7:36 ` [PATCH 08/12] Btrfs: fix a bug of log check Liu Bo
2011-06-30 7:36 ` [PATCH 09/12] Btrfs: kick off useless code Liu Bo
2011-06-30 7:36 ` [PATCH 10/12] Btrfs: use the right generation number to read log_root_tree Liu Bo
2011-06-30 7:36 ` [PATCH 11/12] Btrfs: do not iput inode when inode is still in log Liu Bo
2011-06-30 7:36 ` [PATCH 12/12] Revert "Btrfs: do not flush csum items of unchanged file data during treelog" Liu Bo
2011-06-30 9:27 ` [GIT PULL v4] Btrfs: improve write ahead log with sub transaction liubo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1309419392-17770-1-git-send-email-liubo2009@cn.fujitsu.com \
--to=liubo2009@cn.fujitsu.com \
--cc=chris.mason@oracle.com \
--cc=dave@jikos.cz \
--cc=josef@redhat.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).