git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Luciano Rocha <luciano@eurotux.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Pieter de Bie <pdebie@ai.rug.nl>, git@vger.kernel.org
Subject: [PATCH 01/02/RFC] implement a stat cache
Date: Sun, 20 Apr 2008 12:13:46 +0100	[thread overview]
Message-ID: <20080420111346.GA13411@bit.office.eurotux.com> (raw)
In-Reply-To: <alpine.LFD.1.10.0804191515120.2779@woody.linux-foundation.org>

An implementation of stat(2) and lstat(2) caching. Both the return code
and returned information are cached.

Signed-off-by: Luciano Rocha <strange@nsk.no-ip.org>
---
On Sat, Apr 19, 2008 at 03:39:37PM -0700, Linus Torvalds wrote:
> Yeah. I didn't look any further, but we do a total of *nine* 'lstat()' 
> calls for each file we know about that is dirty, and *seven* when they are 
> clean. Plus maybe a few more.

That's a lot. Why not use a stat cache?

With these changes, my git status . in WebKit changes from 28.215s to
15.414s.

 stat-cache.c |   69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 stat-cache.h |    9 +++++++
 2 files changed, 78 insertions(+), 0 deletions(-)
 create mode 100644 stat-cache.c
 create mode 100644 stat-cache.h

diff --git a/stat-cache.c b/stat-cache.c
new file mode 100644
index 0000000..6a33cec
--- /dev/null
+++ b/stat-cache.c
@@ -0,0 +1,69 @@
+/*
+ * Cache (l)stat operations
+ */
+
+#include "stat-cache.h"
+#include "hash.h"
+#include "path-list.h";
+
+static struct hash_table stat_cache;
+static struct hash_table lstat_cache;
+
+struct stat_result {
+	struct stat st;
+	int ret;
+};
+
+/* based on hash_name from read_cache.c */
+static unsigned int hash_path(const char *path)
+{
+	unsigned int hash = 0x123;
+
+	while (*path)
+		hash = hash*101 + *path++;
+	return hash;
+}
+
+/* cache is HASH->PATH-LIST->(return code, struct stat) */
+static int cached_stat(int (*f)(const char *, struct stat *),
+		struct hash_table *ht, const char *path, struct stat *buf)
+{
+	unsigned int hash;
+	struct path_list *list;
+	struct path_list_item *cached;
+	struct stat_result *result;
+
+	hash = hash_path(path);
+
+	list = lookup_hash(hash, ht);
+
+	if (!list) {
+		list = xcalloc(1, sizeof *list);
+		list->strdup_paths = 1;
+		insert_hash(hash, list, ht);
+	}
+
+	cached = path_list_lookup(path, list);
+
+	if (cached) {
+		result = cached->util;
+	} else {
+		result = xmalloc(sizeof *result);
+		result->ret = f(path, &result->st);
+		path_list_insert(path, list)->util = result;
+	}
+
+	if (result->ret == 0)
+		memcpy(buf, &result->st, sizeof *buf);
+	return result->ret;
+}
+
+int cstat(const char *path, struct stat *buf)
+{
+	return cached_stat(stat, &stat_cache, path, buf);
+}
+
+int clstat(const char *path, struct stat *buf)
+{
+	return cached_stat(lstat, &lstat_cache, path, buf);
+}
diff --git a/stat-cache.h b/stat-cache.h
new file mode 100644
index 0000000..754348f
--- /dev/null
+++ b/stat-cache.h
@@ -0,0 +1,9 @@
+#ifndef STAT_CACHE_H
+#define STAT_CACHE_H
+
+#include "git-compat-util.h"
+
+int cstat(const char *path, struct stat *buf);
+int clstat(const char *path, struct stat *buf);
+
+#endif /* STAT_CACHE_H */
-- 
1.5.5.76.gbb45.dirty

  parent reply	other threads:[~2008-04-20 11:14 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-04-19 19:28 Git performance on OS X Pieter de Bie
2008-04-19 21:22 ` Linus Torvalds
2008-04-19 21:29   ` Linus Torvalds
2008-04-19 22:08     ` Pieter de Bie
2008-04-20 16:17     ` David Kastrup
2008-04-19 21:54 ` Linus Torvalds
2008-04-19 22:00   ` Pieter de Bie
2008-04-19 22:39     ` Linus Torvalds
2008-04-20  4:14       ` Junio C Hamano
2008-04-20 11:13       ` Luciano Rocha [this message]
2008-04-20 11:15         ` [PATCH 02/02/RFC] make use of the stat cache Luciano Rocha
2008-04-20 11:18         ` [PATCH 01/02/RFC] implement a " Luciano Rocha
2008-04-20 16:03         ` Linus Torvalds
2008-04-20 22:04           ` Luciano Rocha
2008-04-20 22:29             ` Linus Torvalds
2008-04-20 23:07               ` Linus Torvalds
2008-04-21  0:53                 ` Dmitry Potapov
2008-04-21  8:41                   ` Johan Herland
2008-04-21  1:21                 ` Junio C Hamano
2008-04-21  3:15                   ` Linus Torvalds
2008-04-21  3:20                     ` Linus Torvalds
2008-04-21 18:27                     ` Junio C Hamano
2008-04-21 19:09                       ` Linus Torvalds
2008-04-21 20:06                         ` Junio C Hamano
2008-04-21 10:04               ` David Kastrup
2008-04-19 22:44     ` Git performance on OS X Jakub Narebski
2008-04-19 22:50       ` Linus Torvalds
2008-04-19 22:54         ` Linus Torvalds
2008-04-19 23:10           ` Pieter de Bie
2008-04-19 23:26             ` Linus Torvalds
2008-04-19 23:35               ` Roman Shaposhnik
2008-04-19 23:57                 ` Pieter de Bie
2008-04-20  0:06                 ` Linus Torvalds
2008-04-20  0:21                   ` Roman Shaposhnik
2008-04-19 23:56               ` Pieter de Bie
2008-04-20  0:31                 ` Linus Torvalds
2008-04-20  1:23                   ` Dmitry Potapov
2008-04-20 16:22                   ` David Kastrup
2008-04-19 23:04         ` Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20080420111346.GA13411@bit.office.eurotux.com \
    --to=luciano@eurotux.com \
    --cc=git@vger.kernel.org \
    --cc=pdebie@ai.rug.nl \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).