* [RFC] [PATCH 0/2] OrangeFS in-kernel lookup and attribute cache
@ 2016-07-05 19:13 Martin Brandenburg
2016-07-05 19:13 ` [PATCH 1/2] orangefs: Use d_time to avoid excessive lookups Martin Brandenburg
2016-07-05 19:13 ` [PATCH 2/2] orangefs: Cache getattr results Martin Brandenburg
0 siblings, 2 replies; 3+ messages in thread
From: Martin Brandenburg @ 2016-07-05 19:13 UTC (permalink / raw)
To: linux-fsdevel; +Cc: Martin Brandenburg
This patch introduces caching for OrangeFS server lookups and getattrs.
The userspace OrangeFS client has done this in the past. We see a
significant (3 to 4 times) performance improvement with these patches
at a cost of consistency across the cluster, which OrangeFS doesn't
really guarantee anyway.
Martin Brandenburg (2):
orangefs: Use d_time to avoid excessive lookups
orangefs: Cache getattr results.
fs/orangefs/dcache.c | 4 ++++
fs/orangefs/inode.c | 6 +++---
fs/orangefs/namei.c | 12 ++++++++++++
fs/orangefs/orangefs-kernel.h | 4 +++-
fs/orangefs/orangefs-utils.c | 38 +++++++++++++++++++++-----------------
fs/orangefs/protocol.h | 8 --------
6 files changed, 43 insertions(+), 29 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 1/2] orangefs: Use d_time to avoid excessive lookups
2016-07-05 19:13 [RFC] [PATCH 0/2] OrangeFS in-kernel lookup and attribute cache Martin Brandenburg
@ 2016-07-05 19:13 ` Martin Brandenburg
2016-07-05 19:13 ` [PATCH 2/2] orangefs: Cache getattr results Martin Brandenburg
1 sibling, 0 replies; 3+ messages in thread
From: Martin Brandenburg @ 2016-07-05 19:13 UTC (permalink / raw)
To: linux-fsdevel; +Cc: Martin Brandenburg
From: Martin Brandenburg <martin@omnibond.com>
Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
fs/orangefs/dcache.c | 4 ++++
fs/orangefs/namei.c | 5 +++++
2 files changed, 9 insertions(+)
diff --git a/fs/orangefs/dcache.c b/fs/orangefs/dcache.c
index 5dfc4f3..dcb55bb 100644
--- a/fs/orangefs/dcache.c
+++ b/fs/orangefs/dcache.c
@@ -73,6 +73,7 @@ static int orangefs_revalidate_lookup(struct dentry *dentry)
}
}
+ dentry->d_time = jiffies + HZ;
ret = 1;
out_release_op:
op_release(new_op);
@@ -94,6 +95,9 @@ static int orangefs_d_revalidate(struct dentry *dentry, unsigned int flags)
{
int ret;
+ if (dentry->d_time > jiffies)
+ return 1;
+
if (flags & LOOKUP_RCU)
return -ECHILD;
diff --git a/fs/orangefs/namei.c b/fs/orangefs/namei.c
index 5a60c50..5e32e0e 100644
--- a/fs/orangefs/namei.c
+++ b/fs/orangefs/namei.c
@@ -72,6 +72,7 @@ static int orangefs_create(struct inode *dir,
d_instantiate(dentry, inode);
unlock_new_inode(inode);
+ dentry->d_time = jiffies + HZ;
gossip_debug(GOSSIP_NAME_DEBUG,
"%s: dentry instantiated for %s\n",
@@ -181,6 +182,8 @@ static struct dentry *orangefs_lookup(struct inode *dir, struct dentry *dentry,
goto out;
}
+ dentry->d_time = jiffies + HZ;
+
inode = orangefs_iget(dir->i_sb, &new_op->downcall.resp.lookup.refn);
if (IS_ERR(inode)) {
gossip_debug(GOSSIP_NAME_DEBUG,
@@ -316,6 +319,7 @@ static int orangefs_symlink(struct inode *dir,
d_instantiate(dentry, inode);
unlock_new_inode(inode);
+ dentry->d_time = jiffies + HZ;
gossip_debug(GOSSIP_NAME_DEBUG,
"Inode (Symlink) %pU -> %s\n",
@@ -378,6 +382,7 @@ static int orangefs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode
d_instantiate(dentry, inode);
unlock_new_inode(inode);
+ dentry->d_time = jiffies + HZ;
gossip_debug(GOSSIP_NAME_DEBUG,
"Inode (Directory) %pU -> %s\n",
--
2.7.4
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH 2/2] orangefs: Cache getattr results.
2016-07-05 19:13 [RFC] [PATCH 0/2] OrangeFS in-kernel lookup and attribute cache Martin Brandenburg
2016-07-05 19:13 ` [PATCH 1/2] orangefs: Use d_time to avoid excessive lookups Martin Brandenburg
@ 2016-07-05 19:13 ` Martin Brandenburg
1 sibling, 0 replies; 3+ messages in thread
From: Martin Brandenburg @ 2016-07-05 19:13 UTC (permalink / raw)
To: linux-fsdevel; +Cc: Martin Brandenburg
From: Martin Brandenburg <martin@omnibond.com>
The userspace component attempts to do this, but this will prevent
us from even needing to go into userspace to satisfy certain getattr
requests.
Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
fs/orangefs/inode.c | 6 +++---
fs/orangefs/namei.c | 7 +++++++
fs/orangefs/orangefs-kernel.h | 4 +++-
fs/orangefs/orangefs-utils.c | 38 +++++++++++++++++++++-----------------
fs/orangefs/protocol.h | 8 --------
5 files changed, 34 insertions(+), 29 deletions(-)
diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c
index 85640e9..6bf7a19 100644
--- a/fs/orangefs/inode.c
+++ b/fs/orangefs/inode.c
@@ -265,7 +265,7 @@ int orangefs_getattr(struct vfsmount *mnt,
"orangefs_getattr: called on %s\n",
dentry->d_name.name);
- ret = orangefs_inode_getattr(inode, 0, 1);
+ ret = orangefs_inode_getattr(inode, 0, 0);
if (ret == 0) {
generic_fillattr(inode, kstat);
@@ -387,7 +387,7 @@ struct inode *orangefs_iget(struct super_block *sb, struct orangefs_object_kref
if (!inode || !(inode->i_state & I_NEW))
return inode;
- error = orangefs_inode_getattr(inode, 1, 0);
+ error = orangefs_inode_getattr(inode, 1, 1);
if (error) {
iget_failed(inode);
return ERR_PTR(error);
@@ -432,7 +432,7 @@ struct inode *orangefs_new_inode(struct super_block *sb, struct inode *dir,
orangefs_set_inode(inode, ref);
inode->i_ino = hash; /* needed for stat etc */
- error = orangefs_inode_getattr(inode, 1, 0);
+ error = orangefs_inode_getattr(inode, 1, 1);
if (error)
goto out_iput;
diff --git a/fs/orangefs/namei.c b/fs/orangefs/namei.c
index 5e32e0e..ffe6377 100644
--- a/fs/orangefs/namei.c
+++ b/fs/orangefs/namei.c
@@ -73,6 +73,7 @@ static int orangefs_create(struct inode *dir,
d_instantiate(dentry, inode);
unlock_new_inode(inode);
dentry->d_time = jiffies + HZ;
+ ORANGEFS_I(inode)->getattr_time = 0;
gossip_debug(GOSSIP_NAME_DEBUG,
"%s: dentry instantiated for %s\n",
@@ -192,6 +193,8 @@ static struct dentry *orangefs_lookup(struct inode *dir, struct dentry *dentry,
goto out;
}
+ ORANGEFS_I(inode)->getattr_time = 0;
+
gossip_debug(GOSSIP_NAME_DEBUG,
"%s:%s:%d "
"Found good inode [%lu] with count [%d]\n",
@@ -320,6 +323,7 @@ static int orangefs_symlink(struct inode *dir,
d_instantiate(dentry, inode);
unlock_new_inode(inode);
dentry->d_time = jiffies + HZ;
+ ORANGEFS_I(inode)->getattr_time = 0;
gossip_debug(GOSSIP_NAME_DEBUG,
"Inode (Symlink) %pU -> %s\n",
@@ -383,6 +387,7 @@ static int orangefs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode
d_instantiate(dentry, inode);
unlock_new_inode(inode);
dentry->d_time = jiffies + HZ;
+ ORANGEFS_I(inode)->getattr_time = 0;
gossip_debug(GOSSIP_NAME_DEBUG,
"Inode (Directory) %pU -> %s\n",
@@ -417,6 +422,8 @@ static int orangefs_rename(struct inode *old_dir,
new_dentry->d_name.name,
d_count(new_dentry));
+ ORANGEFS_I(new_dentry->d_parent->d_inode)->getattr_time = 0;
+
new_op = op_alloc(ORANGEFS_VFS_OP_RENAME);
if (!new_op)
return -EINVAL;
diff --git a/fs/orangefs/orangefs-kernel.h b/fs/orangefs/orangefs-kernel.h
index a9925e2..f6f58b7 100644
--- a/fs/orangefs/orangefs-kernel.h
+++ b/fs/orangefs/orangefs-kernel.h
@@ -257,6 +257,8 @@ struct orangefs_inode_s {
* with this object
*/
unsigned long pinode_flags;
+
+ unsigned long getattr_time;
};
#define P_ATIME_FLAG 0
@@ -540,7 +542,7 @@ int orangefs_inode_setxattr(struct inode *inode,
size_t size,
int flags);
-int orangefs_inode_getattr(struct inode *inode, int new, int size);
+int orangefs_inode_getattr(struct inode *inode, int new, int bypass);
int orangefs_inode_check_changed(struct inode *inode);
diff --git a/fs/orangefs/orangefs-utils.c b/fs/orangefs/orangefs-utils.c
index 2d129b5..1eccc0c 100644
--- a/fs/orangefs/orangefs-utils.c
+++ b/fs/orangefs/orangefs-utils.c
@@ -251,7 +251,7 @@ static int orangefs_inode_is_stale(struct inode *inode, int new,
return 0;
}
-int orangefs_inode_getattr(struct inode *inode, int new, int size)
+int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
{
struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode);
struct orangefs_kernel_op_s *new_op;
@@ -261,12 +261,16 @@ int orangefs_inode_getattr(struct inode *inode, int new, int size)
gossip_debug(GOSSIP_UTILS_DEBUG, "%s: called on inode %pU\n", __func__,
get_khandle_from_ino(inode));
+ if (!new && !bypass) {
+ if (orangefs_inode->getattr_time > jiffies)
+ return 0;
+ }
+
new_op = op_alloc(ORANGEFS_VFS_OP_GETATTR);
if (!new_op)
return -ENOMEM;
new_op->upcall.req.getattr.refn = orangefs_inode->refn;
- new_op->upcall.req.getattr.mask = size ?
- ORANGEFS_ATTR_SYS_ALL_NOHINT : ORANGEFS_ATTR_SYS_ALL_NOHINT_NOSIZE;
+ new_op->upcall.req.getattr.mask = ORANGEFS_ATTR_SYS_ALL_NOHINT;
ret = service_operation(new_op, __func__,
get_interruptible_flag(inode));
@@ -287,20 +291,18 @@ int orangefs_inode_getattr(struct inode *inode, int new, int size)
case S_IFREG:
inode->i_flags = orangefs_inode_flags(&new_op->
downcall.resp.getattr.attributes);
- if (size) {
- inode_size = (loff_t)new_op->
- downcall.resp.getattr.attributes.size;
- rounded_up_size =
- (inode_size + (4096 - (inode_size % 4096)));
- inode->i_size = inode_size;
- orangefs_inode->blksize =
- new_op->downcall.resp.getattr.attributes.blksize;
- spin_lock(&inode->i_lock);
- inode->i_bytes = inode_size;
- inode->i_blocks =
- (unsigned long)(rounded_up_size / 512);
- spin_unlock(&inode->i_lock);
- }
+ inode_size = (loff_t)new_op->
+ downcall.resp.getattr.attributes.size;
+ rounded_up_size =
+ (inode_size + (4096 - (inode_size % 4096)));
+ inode->i_size = inode_size;
+ orangefs_inode->blksize =
+ new_op->downcall.resp.getattr.attributes.blksize;
+ spin_lock(&inode->i_lock);
+ inode->i_bytes = inode_size;
+ inode->i_blocks =
+ (unsigned long)(rounded_up_size / 512);
+ spin_unlock(&inode->i_lock);
break;
case S_IFDIR:
inode->i_size = PAGE_SIZE;
@@ -345,6 +347,7 @@ int orangefs_inode_getattr(struct inode *inode, int new, int size)
inode->i_mode = type | (is_root_handle(inode) ? S_ISVTX : 0) |
orangefs_inode_perms(&new_op->downcall.resp.getattr.attributes);
+ orangefs_inode->getattr_time = jiffies + HZ;
ret = 0;
out:
op_release(new_op);
@@ -418,6 +421,7 @@ int orangefs_inode_setattr(struct inode *inode, struct iattr *iattr)
ClearMtimeFlag(orangefs_inode);
ClearCtimeFlag(orangefs_inode);
ClearModeFlag(orangefs_inode);
+ orangefs_inode->getattr_time = 0;
}
return ret;
diff --git a/fs/orangefs/protocol.h b/fs/orangefs/protocol.h
index 1efc6f8..3d7418c 100644
--- a/fs/orangefs/protocol.h
+++ b/fs/orangefs/protocol.h
@@ -207,14 +207,6 @@ typedef __s64 ORANGEFS_offset;
ORANGEFS_ATTR_SYS_DIRENT_COUNT | \
ORANGEFS_ATTR_SYS_BLKSIZE)
-#define ORANGEFS_ATTR_SYS_ALL_NOHINT_NOSIZE \
- (ORANGEFS_ATTR_SYS_COMMON_ALL | \
- ORANGEFS_ATTR_SYS_LNK_TARGET | \
- ORANGEFS_ATTR_SYS_DFILE_COUNT | \
- ORANGEFS_ATTR_SYS_MIRROR_COPIES_COUNT | \
- ORANGEFS_ATTR_SYS_DIRENT_COUNT | \
- ORANGEFS_ATTR_SYS_BLKSIZE)
-
#define ORANGEFS_XATTR_REPLACE 0x2
#define ORANGEFS_XATTR_CREATE 0x1
#define ORANGEFS_MAX_SERVER_ADDR_LEN 256
--
2.7.4
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-07-05 19:13 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-05 19:13 [RFC] [PATCH 0/2] OrangeFS in-kernel lookup and attribute cache Martin Brandenburg
2016-07-05 19:13 ` [PATCH 1/2] orangefs: Use d_time to avoid excessive lookups Martin Brandenburg
2016-07-05 19:13 ` [PATCH 2/2] orangefs: Cache getattr results Martin Brandenburg
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).