* [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs
2013-02-21 18:28 Jeff Layton
@ 2013-02-21 18:28 ` Jeff Layton
0 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-02-21 18:28 UTC (permalink / raw)
To: bfields; +Cc: linux-nfs
For presenting statistics relating to duplicate reply cache.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/cache.h | 1 +
fs/nfsd/nfscache.c | 37 +++++++++++++++++++++++++++++++++----
fs/nfsd/nfsctl.c | 9 +++++++++
3 files changed, 43 insertions(+), 4 deletions(-)
diff --git a/fs/nfsd/cache.h b/fs/nfsd/cache.h
index 87fd141..d5c5b3e 100644
--- a/fs/nfsd/cache.h
+++ b/fs/nfsd/cache.h
@@ -82,6 +82,7 @@ int nfsd_reply_cache_init(void);
void nfsd_reply_cache_shutdown(void);
int nfsd_cache_lookup(struct svc_rqst *);
void nfsd_cache_update(struct svc_rqst *, int, __be32 *);
+int nfsd_reply_cache_stats_open(struct inode *, struct file *);
#ifdef CONFIG_NFSD_V4
void nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index efed50e..a5ac9ab 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -23,12 +23,17 @@
static struct hlist_head * cache_hash;
static struct list_head lru_head;
static struct kmem_cache *drc_slab;
-static unsigned int num_drc_entries;
-static unsigned int max_drc_entries;
-static unsigned int drc_mem_usage;
-static unsigned int csum_misses;
/*
+ * Stats on the duplicate reply cache code. All of these and the "rc" fields
+ * in nfsdstats are protected by the cache_lock
+ */
+static unsigned int num_drc_entries; /* number of entries */
+static unsigned int max_drc_entries; /* max number of entries */
+static unsigned int drc_mem_usage; /* memory used by cache */
+static unsigned int csum_misses; /* cache misses due only to
+ csum comparison failures */
+/*
* Calculate the hash index from an XID.
*/
static inline u32 request_hash(u32 xid)
@@ -545,3 +550,27 @@ nfsd_cache_append(struct svc_rqst *rqstp, struct kvec *data)
vec->iov_len += data->iov_len;
return 1;
}
+
+/*
+ * Note that fields may be added, removed or reordered in the future. Programs
+ * scraping this file for info should test the labels to ensure they're
+ * getting the correct field.
+ */
+static int nfsd_reply_cache_stats_show(struct seq_file *m, void *v)
+{
+ spin_lock(&cache_lock);
+ seq_printf(m, "max entries: %u\n", max_drc_entries);
+ seq_printf(m, "num entries: %u\n", num_drc_entries);
+ seq_printf(m, "mem usage: %u\n", drc_mem_usage);
+ seq_printf(m, "cache hits: %u\n", nfsdstats.rchits);
+ seq_printf(m, "cache misses: %u\n", nfsdstats.rcmisses);
+ seq_printf(m, "not cached: %u\n", nfsdstats.rcnocache);
+ seq_printf(m, "checksum misses: %u\n", csum_misses);
+ spin_unlock(&cache_lock);
+ return 0;
+}
+
+int nfsd_reply_cache_stats_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, nfsd_reply_cache_stats_show, NULL);
+}
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 8ead2c2..c7238a5 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -35,6 +35,7 @@ enum {
NFSD_Threads,
NFSD_Pool_Threads,
NFSD_Pool_Stats,
+ NFSD_Reply_Cache_Stats,
NFSD_Versions,
NFSD_Ports,
NFSD_MaxBlkSize,
@@ -212,6 +213,13 @@ static const struct file_operations pool_stats_operations = {
.owner = THIS_MODULE,
};
+static struct file_operations reply_cache_stats_operations = {
+ .open = nfsd_reply_cache_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
/*----------------------------------------------------------------------------*/
/*
* payload - write methods
@@ -1047,6 +1055,7 @@ static int nfsd_fill_super(struct super_block * sb, void * data, int silent)
[NFSD_Threads] = {"threads", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Pool_Threads] = {"pool_threads", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Pool_Stats] = {"pool_stats", &pool_stats_operations, S_IRUGO},
+ [NFSD_Reply_Cache_Stats] = {"reply_cache_stats", &reply_cache_stats_operations, S_IRUGO},
[NFSD_Versions] = {"versions", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Ports] = {"portlist", &transaction_ops, S_IWUSR|S_IRUGO},
[NFSD_MaxBlkSize] = {"max_block_size", &transaction_ops, S_IWUSR|S_IRUGO},
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance
@ 2013-03-04 18:56 Jeff Layton
2013-03-04 18:56 ` [PATCH v2 1/5] nfsd: break out comparator into separate function Jeff Layton
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
This patchset adds some stats tracking to the DRC to hopefully allow
us to make better-informed decisions in the future on performance
improvements.
Some of these I've already posted, but reposting them now as a set
to allow Bruce to more easily merge them. I think these would be good
for 3.9, but they're obviously not critical.
Jeff Layton (5):
nfsd: break out comparator into separate function
nfsd: track memory utilization in the DRC
nfsd: add new reply_cache_stats file in nfsdfs
nfsd: keep track of the max and average time to search the cache
nfsd: keep stats on worst hash balancing seen so far
fs/nfsd/cache.h | 1 +
fs/nfsd/nfscache.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++-------
fs/nfsd/nfsctl.c | 9 ++++
3 files changed, 137 insertions(+), 17 deletions(-)
--
1.7.11.7
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/5] nfsd: break out comparator into separate function
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
@ 2013-03-04 18:56 ` Jeff Layton
2013-03-04 18:56 ` [PATCH v2 2/5] nfsd: track memory utilization in the DRC Jeff Layton
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
Break out the function that compares the rqstp and checksum against a
reply cache entry. While we're at it, track the efficacy of the checksum
over the NFS data by tracking the cases where we would have incorrectly
matched a DRC entry if we had not tracked it or the length.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/nfscache.c | 44 ++++++++++++++++++++++++++++++++++----------
1 file changed, 34 insertions(+), 10 deletions(-)
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index 62c1ee1..3fba32f 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -23,9 +23,21 @@
static struct hlist_head * cache_hash;
static struct list_head lru_head;
static struct kmem_cache *drc_slab;
+
+/*
+ * Stats and other tracking of on the duplicate reply cache. All of these and
+ * the "rc" fields in nfsdstats are protected by the cache_lock
+ */
+
+/* total number of entries */
static unsigned int num_drc_entries;
+
+/* max number of entries allowed in the cache */
static unsigned int max_drc_entries;
+/* cache misses due only to checksum comparison failures */
+static unsigned int payload_misses;
+
/*
* Calculate the hash index from an XID.
*/
@@ -272,6 +284,26 @@ nfsd_cache_csum(struct svc_rqst *rqstp)
return csum;
}
+static bool
+nfsd_cache_match(struct svc_rqst *rqstp, __wsum csum, struct svc_cacherep *rp)
+{
+ /* Check RPC header info first */
+ if (rqstp->rq_xid != rp->c_xid || rqstp->rq_proc != rp->c_proc ||
+ rqstp->rq_prot != rp->c_prot || rqstp->rq_vers != rp->c_vers ||
+ rqstp->rq_arg.len != rp->c_len ||
+ !rpc_cmp_addr(svc_addr(rqstp), (struct sockaddr *)&rp->c_addr) ||
+ rpc_get_port(svc_addr(rqstp)) != rpc_get_port((struct sockaddr *)&rp->c_addr))
+ return false;
+
+ /* compare checksum of NFS data */
+ if (csum != rp->c_csum) {
+ ++payload_misses;
+ return false;
+ }
+
+ return true;
+}
+
/*
* Search the request hash for an entry that matches the given rqstp.
* Must be called with cache_lock held. Returns the found entry or
@@ -282,18 +314,10 @@ nfsd_cache_search(struct svc_rqst *rqstp, __wsum csum)
{
struct svc_cacherep *rp;
struct hlist_head *rh;
- __be32 xid = rqstp->rq_xid;
- u32 proto = rqstp->rq_prot,
- vers = rqstp->rq_vers,
- proc = rqstp->rq_proc;
- rh = &cache_hash[request_hash(xid)];
+ rh = &cache_hash[request_hash(rqstp->rq_xid)];
hlist_for_each_entry(rp, rh, c_hash) {
- if (xid == rp->c_xid && proc == rp->c_proc &&
- proto == rp->c_prot && vers == rp->c_vers &&
- rqstp->rq_arg.len == rp->c_len && csum == rp->c_csum &&
- rpc_cmp_addr(svc_addr(rqstp), (struct sockaddr *)&rp->c_addr) &&
- rpc_get_port(svc_addr(rqstp)) == rpc_get_port((struct sockaddr *)&rp->c_addr))
+ if (nfsd_cache_match(rqstp, csum, rp))
return rp;
}
return NULL;
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 2/5] nfsd: track memory utilization in the DRC
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
2013-03-04 18:56 ` [PATCH v2 1/5] nfsd: break out comparator into separate function Jeff Layton
@ 2013-03-04 18:56 ` Jeff Layton
2013-03-04 18:56 ` [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs Jeff Layton
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/nfscache.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index 3fba32f..bcbbe6b 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -38,6 +38,9 @@ static unsigned int max_drc_entries;
/* cache misses due only to checksum comparison failures */
static unsigned int payload_misses;
+/* amount of memory (in bytes) currently consumed by the DRC */
+static unsigned int drc_mem_usage;
+
/*
* Calculate the hash index from an XID.
*/
@@ -112,11 +115,14 @@ nfsd_reply_cache_alloc(void)
static void
nfsd_reply_cache_free_locked(struct svc_cacherep *rp)
{
- if (rp->c_type == RC_REPLBUFF)
+ if (rp->c_type == RC_REPLBUFF && rp->c_replvec.iov_base) {
+ drc_mem_usage -= rp->c_replvec.iov_len;
kfree(rp->c_replvec.iov_base);
+ }
hlist_del(&rp->c_hash);
list_del(&rp->c_lru);
--num_drc_entries;
+ drc_mem_usage -= sizeof(*rp);
kmem_cache_free(drc_slab, rp);
}
@@ -377,6 +383,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)
return RC_DOIT;
}
spin_lock(&cache_lock);
+ drc_mem_usage += sizeof(*rp);
++num_drc_entries;
/*
@@ -417,6 +424,7 @@ setup_entry:
/* release any buffer */
if (rp->c_type == RC_REPLBUFF) {
+ drc_mem_usage -= rp->c_replvec.iov_len;
kfree(rp->c_replvec.iov_base);
rp->c_replvec.iov_base = NULL;
}
@@ -485,6 +493,7 @@ nfsd_cache_update(struct svc_rqst *rqstp, int cachetype, __be32 *statp)
struct svc_cacherep *rp = rqstp->rq_cacherep;
struct kvec *resv = &rqstp->rq_res.head[0], *cachv;
int len;
+ size_t bufsize = 0;
if (!rp)
return;
@@ -506,19 +515,21 @@ nfsd_cache_update(struct svc_rqst *rqstp, int cachetype, __be32 *statp)
break;
case RC_REPLBUFF:
cachv = &rp->c_replvec;
- cachv->iov_base = kmalloc(len << 2, GFP_KERNEL);
+ bufsize = len << 2;
+ cachv->iov_base = kmalloc(bufsize, GFP_KERNEL);
if (!cachv->iov_base) {
nfsd_reply_cache_free(rp);
return;
}
- cachv->iov_len = len << 2;
- memcpy(cachv->iov_base, statp, len << 2);
+ cachv->iov_len = bufsize;
+ memcpy(cachv->iov_base, statp, bufsize);
break;
case RC_NOCACHE:
nfsd_reply_cache_free(rp);
return;
}
spin_lock(&cache_lock);
+ drc_mem_usage += bufsize;
lru_put_end(rp);
rp->c_secure = rqstp->rq_secure;
rp->c_type = cachetype;
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
2013-03-04 18:56 ` [PATCH v2 1/5] nfsd: break out comparator into separate function Jeff Layton
2013-03-04 18:56 ` [PATCH v2 2/5] nfsd: track memory utilization in the DRC Jeff Layton
@ 2013-03-04 18:56 ` Jeff Layton
2013-03-04 18:56 ` [PATCH v2 4/5] nfsd: keep track of the max and average time to search the cache Jeff Layton
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
For presenting statistics relating to duplicate reply cache.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/cache.h | 1 +
fs/nfsd/nfscache.c | 25 +++++++++++++++++++++++++
fs/nfsd/nfsctl.c | 9 +++++++++
3 files changed, 35 insertions(+)
diff --git a/fs/nfsd/cache.h b/fs/nfsd/cache.h
index 87fd141..d5c5b3e 100644
--- a/fs/nfsd/cache.h
+++ b/fs/nfsd/cache.h
@@ -82,6 +82,7 @@ int nfsd_reply_cache_init(void);
void nfsd_reply_cache_shutdown(void);
int nfsd_cache_lookup(struct svc_rqst *);
void nfsd_cache_update(struct svc_rqst *, int, __be32 *);
+int nfsd_reply_cache_stats_open(struct inode *, struct file *);
#ifdef CONFIG_NFSD_V4
void nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp);
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index bcbbe6b..24ac996 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -557,3 +557,28 @@ nfsd_cache_append(struct svc_rqst *rqstp, struct kvec *data)
vec->iov_len += data->iov_len;
return 1;
}
+
+/*
+ * Note that fields may be added, removed or reordered in the future. Programs
+ * scraping this file for info should test the labels to ensure they're
+ * getting the correct field.
+ */
+static int nfsd_reply_cache_stats_show(struct seq_file *m, void *v)
+{
+ spin_lock(&cache_lock);
+ seq_printf(m, "max entries: %u\n", max_drc_entries);
+ seq_printf(m, "num entries: %u\n", num_drc_entries);
+ seq_printf(m, "hash buckets: %u\n", HASHSIZE);
+ seq_printf(m, "mem usage: %u\n", drc_mem_usage);
+ seq_printf(m, "cache hits: %u\n", nfsdstats.rchits);
+ seq_printf(m, "cache misses: %u\n", nfsdstats.rcmisses);
+ seq_printf(m, "not cached: %u\n", nfsdstats.rcnocache);
+ seq_printf(m, "payload misses: %u\n", payload_misses);
+ spin_unlock(&cache_lock);
+ return 0;
+}
+
+int nfsd_reply_cache_stats_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, nfsd_reply_cache_stats_show, NULL);
+}
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 13a21c8..2baaa5a 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -35,6 +35,7 @@ enum {
NFSD_Threads,
NFSD_Pool_Threads,
NFSD_Pool_Stats,
+ NFSD_Reply_Cache_Stats,
NFSD_Versions,
NFSD_Ports,
NFSD_MaxBlkSize,
@@ -212,6 +213,13 @@ static const struct file_operations pool_stats_operations = {
.owner = THIS_MODULE,
};
+static struct file_operations reply_cache_stats_operations = {
+ .open = nfsd_reply_cache_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
/*----------------------------------------------------------------------------*/
/*
* payload - write methods
@@ -1047,6 +1055,7 @@ static int nfsd_fill_super(struct super_block * sb, void * data, int silent)
[NFSD_Threads] = {"threads", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Pool_Threads] = {"pool_threads", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Pool_Stats] = {"pool_stats", &pool_stats_operations, S_IRUGO},
+ [NFSD_Reply_Cache_Stats] = {"reply_cache_stats", &reply_cache_stats_operations, S_IRUGO},
[NFSD_Versions] = {"versions", &transaction_ops, S_IWUSR|S_IRUSR},
[NFSD_Ports] = {"portlist", &transaction_ops, S_IWUSR|S_IRUGO},
[NFSD_MaxBlkSize] = {"max_block_size", &transaction_ops, S_IWUSR|S_IRUGO},
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 4/5] nfsd: keep track of the max and average time to search the cache
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
` (2 preceding siblings ...)
2013-03-04 18:56 ` [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs Jeff Layton
@ 2013-03-04 18:56 ` Jeff Layton
2013-03-04 18:56 ` [PATCH v2 5/5] nfsd: keep stats on worst hash balancing seen so far Jeff Layton
2013-03-05 22:20 ` [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance J. Bruce Fields
5 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
Since we're moderately concerned about the potential increase in hash
chain lengths, keep a running average of the time and the max time to
search.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/nfscache.c | 37 +++++++++++++++++++++++++++++++++----
1 file changed, 33 insertions(+), 4 deletions(-)
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index 24ac996..7640e80 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -41,6 +41,15 @@ static unsigned int payload_misses;
/* amount of memory (in bytes) currently consumed by the DRC */
static unsigned int drc_mem_usage;
+/* total time spent searching the cache */
+static s64 time_in_search;
+
+/* total number of times we searched the cache */
+static u64 num_searches;
+
+/* longest time we spent doing a single search */
+static s64 max_search_time;
+
/*
* Calculate the hash index from an XID.
*/
@@ -318,15 +327,25 @@ nfsd_cache_match(struct svc_rqst *rqstp, __wsum csum, struct svc_cacherep *rp)
static struct svc_cacherep *
nfsd_cache_search(struct svc_rqst *rqstp, __wsum csum)
{
- struct svc_cacherep *rp;
+ struct svc_cacherep *rp, *ret = NULL;
struct hlist_head *rh;
+ ktime_t start;
+ s64 delta;
+ start = ktime_get();
rh = &cache_hash[request_hash(rqstp->rq_xid)];
hlist_for_each_entry(rp, rh, c_hash) {
- if (nfsd_cache_match(rqstp, csum, rp))
- return rp;
+ if (nfsd_cache_match(rqstp, csum, rp)) {
+ ret = rp;
+ break;
+ }
}
- return NULL;
+ delta = ktime_to_ns(ktime_sub(ktime_get(), start));
+ time_in_search += delta;
+ max_search_time = max(max_search_time, delta);
+ ++num_searches;
+
+ return ret;
}
/*
@@ -558,6 +577,14 @@ nfsd_cache_append(struct svc_rqst *rqstp, struct kvec *data)
return 1;
}
+static s64
+avg_search_time(void)
+{
+ if (!num_searches)
+ return 0;
+ return time_in_search / num_searches;
+}
+
/*
* Note that fields may be added, removed or reordered in the future. Programs
* scraping this file for info should test the labels to ensure they're
@@ -574,6 +601,8 @@ static int nfsd_reply_cache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "cache misses: %u\n", nfsdstats.rcmisses);
seq_printf(m, "not cached: %u\n", nfsdstats.rcnocache);
seq_printf(m, "payload misses: %u\n", payload_misses);
+ seq_printf(m, "avg search time: %lldns\n", avg_search_time());
+ seq_printf(m, "max search time: %lldns\n", max_search_time);
spin_unlock(&cache_lock);
return 0;
}
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 5/5] nfsd: keep stats on worst hash balancing seen so far
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
` (3 preceding siblings ...)
2013-03-04 18:56 ` [PATCH v2 4/5] nfsd: keep track of the max and average time to search the cache Jeff Layton
@ 2013-03-04 18:56 ` Jeff Layton
2013-03-05 22:20 ` [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance J. Bruce Fields
5 siblings, 0 replies; 8+ messages in thread
From: Jeff Layton @ 2013-03-04 18:56 UTC (permalink / raw)
To: bfields; +Cc: chuck.lever, linux-nfs
The typical case with the DRC is a cache miss, so if we keep track of
the max number of entries that we've ever walked over in a search, then
we should have a reasonable estimate of the longest hash chain that
we've ever seen.
With that, we'll also keep track of the total size of the cache when we
see the longest chain. In the case of a tie, we prefer to track the
smallest total cache size in order to properly gauge the worst-case
ratio of max vs. avg chain length.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
fs/nfsd/nfscache.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index 7640e80..3af1767 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -50,6 +50,12 @@ static u64 num_searches;
/* longest time we spent doing a single search */
static s64 max_search_time;
+/* longest hash chain seen */
+static unsigned int longest_chain;
+
+/* size of cache when we saw the longest hash chain */
+static unsigned int longest_chain_cachesize;
+
/*
* Calculate the hash index from an XID.
*/
@@ -330,11 +336,13 @@ nfsd_cache_search(struct svc_rqst *rqstp, __wsum csum)
struct svc_cacherep *rp, *ret = NULL;
struct hlist_head *rh;
ktime_t start;
+ unsigned int entries = 0;
s64 delta;
start = ktime_get();
rh = &cache_hash[request_hash(rqstp->rq_xid)];
hlist_for_each_entry(rp, rh, c_hash) {
+ ++entries;
if (nfsd_cache_match(rqstp, csum, rp)) {
ret = rp;
break;
@@ -343,6 +351,17 @@ nfsd_cache_search(struct svc_rqst *rqstp, __wsum csum)
delta = ktime_to_ns(ktime_sub(ktime_get(), start));
time_in_search += delta;
max_search_time = max(max_search_time, delta);
+
+ /* tally hash chain length stats */
+ if (entries >= longest_chain) {
+ longest_chain = entries;
+ if (!longest_chain_cachesize)
+ longest_chain_cachesize = num_drc_entries;
+ else
+ longest_chain_cachesize = min(longest_chain_cachesize,
+ num_drc_entries);
+ }
+
++num_searches;
return ret;
@@ -603,6 +622,8 @@ static int nfsd_reply_cache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "payload misses: %u\n", payload_misses);
seq_printf(m, "avg search time: %lldns\n", avg_search_time());
seq_printf(m, "max search time: %lldns\n", max_search_time);
+ seq_printf(m, "longest chain len: %u\n", longest_chain);
+ seq_printf(m, "cachesize at longest: %u\n", longest_chain_cachesize);
spin_unlock(&cache_lock);
return 0;
}
--
1.7.11.7
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
` (4 preceding siblings ...)
2013-03-04 18:56 ` [PATCH v2 5/5] nfsd: keep stats on worst hash balancing seen so far Jeff Layton
@ 2013-03-05 22:20 ` J. Bruce Fields
5 siblings, 0 replies; 8+ messages in thread
From: J. Bruce Fields @ 2013-03-05 22:20 UTC (permalink / raw)
To: Jeff Layton; +Cc: chuck.lever, linux-nfs
On Mon, Mar 04, 2013 at 01:56:35PM -0500, Jeff Layton wrote:
> This patchset adds some stats tracking to the DRC to hopefully allow
> us to make better-informed decisions in the future on performance
> improvements.
>
> Some of these I've already posted, but reposting them now as a set
> to allow Bruce to more easily merge them. I think these would be good
> for 3.9, but they're obviously not critical.
OK, thanks. I'll queue them up for 3.10.
--b.
>
> Jeff Layton (5):
> nfsd: break out comparator into separate function
> nfsd: track memory utilization in the DRC
> nfsd: add new reply_cache_stats file in nfsdfs
> nfsd: keep track of the max and average time to search the cache
> nfsd: keep stats on worst hash balancing seen so far
>
> fs/nfsd/cache.h | 1 +
> fs/nfsd/nfscache.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++-------
> fs/nfsd/nfsctl.c | 9 ++++
> 3 files changed, 137 insertions(+), 17 deletions(-)
>
> --
> 1.7.11.7
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-03-05 22:20 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-03-04 18:56 [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance Jeff Layton
2013-03-04 18:56 ` [PATCH v2 1/5] nfsd: break out comparator into separate function Jeff Layton
2013-03-04 18:56 ` [PATCH v2 2/5] nfsd: track memory utilization in the DRC Jeff Layton
2013-03-04 18:56 ` [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs Jeff Layton
2013-03-04 18:56 ` [PATCH v2 4/5] nfsd: keep track of the max and average time to search the cache Jeff Layton
2013-03-04 18:56 ` [PATCH v2 5/5] nfsd: keep stats on worst hash balancing seen so far Jeff Layton
2013-03-05 22:20 ` [PATCH v2 0/5] nfsd: add a reply_cache_stats file to nfsd for tracking DRC performance J. Bruce Fields
-- strict thread matches above, loose matches on Subject: below --
2013-02-21 18:28 Jeff Layton
2013-02-21 18:28 ` [PATCH v2 3/5] nfsd: add new reply_cache_stats file in nfsdfs Jeff Layton
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).