From: Dave Chinner <david@fromorbit.com>
To: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Subject: [PATCH 0/17] fs: Inode cache scalability
Date: Wed, 29 Sep 2010 22:18:32 +1000 [thread overview]
Message-ID: <1285762729-17928-1-git-send-email-david@fromorbit.com> (raw)
This patch set is derived from Nick Piggin's VFS scalability tree.
There doesn't appear to be any push to get that tree into shape for
.37, so this is an attempt to start the process of finer grained
review of the series for upstream inclusion. I'm hitting VFS lock
contention problems with XFS on 8-16p machines now, so I need to get
this stuff moving.
This patch set is just the basic inode_lock breakup patches plus a
few more simple changes to the inode code. It stops short of
introducing RCU inode freeing because those changes are not
completely baked yet. It also stops short of changing the way inodes
are tracked for writeback because I'd like not to spend my week
after -rc1 is released fixing writeback again....
As a result, the full inode handling improvements of Nick's patch
set are not realised with this short series. However, my own testing
indicates that the amount of lock traffic and contention is down by
an order of magnitude on an 8-way box for parallel inode create and
unlink workloads, so there is still significant improvements in
scalability from just this patch set.
I've only ported the patches so far, without changing anything
significant other than the comit descriptions. One thing that has
stood out as I've done this is that the ordering of the patches is
not ideal, and some things (like the inode counters) are modified
multiple times through the patch set. I'm quite happy to
reorder/rework the series to fix these problems if that is desired.
Basically I'm trying to get this patchset ready for .37 (merge
window is not really that far off now), and I'm aiming to have the
rest of the inode changes (RCU freeing, writeback, etc) ready for
.38. I may even look to some of the dcache changes for .38 depending
on how much I can get tested and reviewed in that time frame.
Comments are welcome.
The current patchset is also available at the following location.
The following changes since commit b30a3f6257ed2105259b404d419b4964e363928c:
Linux 2.6.36-rc5 (2010-09-20 16:56:53 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfsdev.git inode-scale
Eric Dumazet (2):
fs: inode per-cpu last_ino allocator
fs: Convert nr_inodes to a per-cpu counter
Nick Piggin (15):
kernel: add bl_list
fs: icache lock s_inodes list
fs: icache lock inode hash
fs: icache lock i_state
fs: icache lock i_count
fs: icache lock lru/writeback lists
fs: icache atomic inodes_stat
fs: icache protect inode state
fs: Make last_ino, iunique independent of inode_lock
fs: icache remove inode_lock
fs: Factor inode hash operations into functions
fs: Introduce per-bucket inode hash locks
fs: Implement lazy LRU updates for inodes.
fs: Inode counters do not need to be atomic.
fs: Clean up inode reference counting
Documentation/filesystems/Locking | 2 +-
Documentation/filesystems/porting | 10 +-
Documentation/filesystems/vfs.txt | 2 +-
arch/powerpc/platforms/cell/spufs/file.c | 2 +-
drivers/staging/pohmelfs/inode.c | 14 +-
fs/9p/vfs_inode.c | 4 +-
fs/affs/inode.c | 2 +-
fs/afs/dir.c | 4 +-
fs/anon_inodes.c | 2 +-
fs/bfs/dir.c | 2 +-
fs/block_dev.c | 7 +-
fs/btrfs/inode.c | 23 +-
fs/buffer.c | 2 +-
fs/ceph/mds_client.c | 2 +-
fs/cifs/inode.c | 2 +-
fs/coda/dir.c | 2 +-
fs/drop_caches.c | 19 +-
fs/exofs/inode.c | 10 +-
fs/exofs/namei.c | 2 +-
fs/ext2/namei.c | 2 +-
fs/ext3/ialloc.c | 4 +-
fs/ext3/namei.c | 2 +-
fs/ext4/ialloc.c | 4 +-
fs/ext4/namei.c | 2 +-
fs/fs-writeback.c | 156 +++++---
fs/gfs2/ops_inode.c | 2 +-
fs/hfs/hfs_fs.h | 2 +-
fs/hfs/inode.c | 2 +-
fs/hfsplus/dir.c | 2 +-
fs/hfsplus/hfsplus_fs.h | 2 +-
fs/hfsplus/inode.c | 2 +-
fs/hpfs/inode.c | 2 +-
fs/inode.c | 603 ++++++++++++++++++++----------
fs/jffs2/dir.c | 4 +-
fs/jfs/jfs_txnmgr.c | 2 +-
fs/jfs/namei.c | 2 +-
fs/libfs.c | 2 +-
fs/locks.c | 2 +-
fs/logfs/dir.c | 2 +-
fs/logfs/inode.c | 2 +-
fs/logfs/readwrite.c | 6 +-
fs/minix/namei.c | 2 +-
fs/namei.c | 2 +-
fs/nfs/dir.c | 2 +-
fs/nfs/getroot.c | 4 +-
fs/nfs/inode.c | 4 +-
fs/nfs/nfs4state.c | 2 +-
fs/nfs/write.c | 2 +-
fs/nilfs2/gcdat.c | 1 +
fs/nilfs2/gcinode.c | 22 +-
fs/nilfs2/mdt.c | 2 +-
fs/nilfs2/namei.c | 2 +-
fs/nilfs2/segment.c | 2 +-
fs/nilfs2/the_nilfs.h | 2 +-
fs/notify/inode_mark.c | 46 ++-
fs/notify/mark.c | 1 -
fs/notify/vfsmount_mark.c | 1 -
fs/ntfs/inode.c | 4 +-
fs/ntfs/super.c | 2 +-
fs/ocfs2/inode.c | 2 +-
fs/ocfs2/namei.c | 2 +-
fs/quota/dquot.c | 36 +-
fs/reiserfs/namei.c | 2 +-
fs/reiserfs/stree.c | 2 +-
fs/reiserfs/xattr.c | 2 +-
fs/sysv/namei.c | 2 +-
fs/ubifs/dir.c | 2 +-
fs/ubifs/super.c | 2 +-
fs/udf/namei.c | 2 +-
fs/ufs/namei.c | 2 +-
fs/xfs/linux-2.6/xfs_iops.c | 2 +-
fs/xfs/linux-2.6/xfs_trace.h | 2 +-
fs/xfs/xfs_inode.h | 4 +-
include/linux/fs.h | 54 ++-
include/linux/list_bl.h | 127 +++++++
include/linux/rculist_bl.h | 128 +++++++
include/linux/writeback.h | 4 +-
ipc/mqueue.c | 2 +-
kernel/futex.c | 2 +-
kernel/sysctl.c | 4 +-
mm/backing-dev.c | 8 +-
mm/filemap.c | 6 +-
mm/rmap.c | 6 +-
mm/shmem.c | 6 +-
net/socket.c | 2 +-
85 files changed, 1001 insertions(+), 437 deletions(-)
create mode 100644 include/linux/list_bl.h
create mode 100644 include/linux/rculist_bl.h
next reply other threads:[~2010-09-29 12:19 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-09-29 12:18 Dave Chinner [this message]
2010-09-29 12:18 ` [PATCH 01/17] kernel: add bl_list Dave Chinner
2010-09-30 4:52 ` Andrew Morton
2010-10-16 7:55 ` Nick Piggin
2010-10-16 16:28 ` Christoph Hellwig
2010-10-01 5:48 ` Christoph Hellwig
2010-09-29 12:18 ` [PATCH 02/17] fs: icache lock s_inodes list Dave Chinner
2010-10-01 5:49 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-10-16 16:12 ` Christoph Hellwig
2010-10-16 17:09 ` Nick Piggin
2010-10-17 0:42 ` Christoph Hellwig
2010-10-17 2:03 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 03/17] fs: icache lock inode hash Dave Chinner
2010-09-30 4:52 ` Andrew Morton
2010-09-30 6:13 ` Dave Chinner
2010-10-01 6:06 ` Christoph Hellwig
2010-10-16 7:57 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 04/17] fs: icache lock i_state Dave Chinner
2010-10-01 5:54 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 05/17] fs: icache lock i_count Dave Chinner
2010-09-30 4:52 ` Andrew Morton
2010-10-01 5:55 ` Christoph Hellwig
2010-10-01 6:04 ` Andrew Morton
2010-10-01 6:16 ` Christoph Hellwig
2010-10-01 6:23 ` Andrew Morton
2010-09-29 12:18 ` [PATCH 06/17] fs: icache lock lru/writeback lists Dave Chinner
2010-09-30 4:52 ` Andrew Morton
2010-09-30 6:16 ` Dave Chinner
2010-10-16 7:55 ` Nick Piggin
2010-10-01 6:01 ` Christoph Hellwig
2010-10-05 22:30 ` Dave Chinner
2010-09-29 12:18 ` [PATCH 07/17] fs: icache atomic inodes_stat Dave Chinner
2010-09-30 4:52 ` Andrew Morton
2010-09-30 6:20 ` Dave Chinner
2010-09-30 6:37 ` Andrew Morton
2010-10-16 7:56 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 08/17] fs: icache protect inode state Dave Chinner
2010-10-01 6:02 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 09/17] fs: Make last_ino, iunique independent of inode_lock Dave Chinner
2010-09-30 4:53 ` Andrew Morton
2010-10-01 6:08 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 10/17] fs: icache remove inode_lock Dave Chinner
2010-09-29 12:18 ` [PATCH 11/17] fs: Factor inode hash operations into functions Dave Chinner
2010-10-01 6:06 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 12/17] fs: Introduce per-bucket inode hash locks Dave Chinner
2010-09-30 1:52 ` Christoph Hellwig
2010-09-30 2:43 ` Dave Chinner
2010-10-16 7:55 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 13/17] fs: Implement lazy LRU updates for inodes Dave Chinner
2010-09-30 2:05 ` Christoph Hellwig
2010-10-16 7:54 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 14/17] fs: Inode counters do not need to be atomic Dave Chinner
2010-09-29 12:18 ` [PATCH 15/17] fs: inode per-cpu last_ino allocator Dave Chinner
2010-09-30 2:07 ` Christoph Hellwig
2010-10-06 6:29 ` Dave Chinner
2010-10-06 8:51 ` Christoph Hellwig
2010-09-30 4:53 ` Andrew Morton
2010-09-30 5:36 ` Eric Dumazet
2010-09-30 7:53 ` Eric Dumazet
2010-09-30 8:14 ` Andrew Morton
2010-09-30 10:22 ` [PATCH] " Eric Dumazet
2010-09-30 16:45 ` Andrew Morton
2010-09-30 17:28 ` Eric Dumazet
2010-09-30 17:39 ` Andrew Morton
2010-09-30 18:05 ` Eric Dumazet
2010-10-01 6:12 ` Christoph Hellwig
2010-10-01 6:45 ` Eric Dumazet
2010-10-16 6:36 ` Nick Piggin
2010-10-16 6:40 ` Nick Piggin
2010-09-29 12:18 ` [PATCH 16/17] fs: Convert nr_inodes to a per-cpu counter Dave Chinner
2010-09-30 2:12 ` Christoph Hellwig
2010-09-30 4:53 ` Andrew Morton
2010-09-30 6:10 ` Dave Chinner
2010-10-16 7:55 ` Nick Piggin
2010-10-16 8:29 ` Eric Dumazet
2010-10-16 9:07 ` Andrew Morton
2010-10-16 9:31 ` Eric Dumazet
2010-10-16 14:19 ` [PATCH] percpu_counter : add percpu_counter_add_fast() Eric Dumazet
2010-10-18 15:24 ` Christoph Lameter
2010-10-18 15:39 ` Eric Dumazet
2010-10-18 16:12 ` Christoph Lameter
2010-10-21 22:37 ` Andrew Morton
2010-10-21 23:10 ` Christoph Lameter
2010-10-22 0:45 ` Andrew Morton
2010-10-22 1:55 ` Andrew Morton
2010-10-22 1:58 ` Nick Piggin
2010-10-22 2:14 ` Andrew Morton
2010-10-22 4:12 ` Eric Dumazet
2010-10-21 22:43 ` Andrew Morton
2010-10-21 22:58 ` Eric Dumazet
2010-10-21 23:18 ` Andrew Morton
2010-10-21 23:22 ` Eric Dumazet
2010-10-21 22:31 ` [PATCH 16/17] fs: Convert nr_inodes to a per-cpu counter Andrew Morton
2010-10-21 22:58 ` Eric Dumazet
2010-10-02 16:02 ` Christoph Hellwig
2010-09-29 12:18 ` [PATCH 17/17] fs: Clean up inode reference counting Dave Chinner
2010-09-30 2:15 ` Christoph Hellwig
2010-10-16 7:55 ` Nick Piggin
2010-10-16 16:14 ` Christoph Hellwig
2010-10-16 17:09 ` Nick Piggin
2010-09-30 4:53 ` Andrew Morton
2010-09-29 23:57 ` [PATCH 0/17] fs: Inode cache scalability Christoph Hellwig
2010-09-30 0:24 ` Dave Chinner
2010-09-30 2:21 ` Christoph Hellwig
2010-10-02 23:10 ` Carlos Carvalho
2010-10-04 7:22 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1285762729-17928-1-git-send-email-david@fromorbit.com \
--to=david@fromorbit.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).