From: Jakub Narebski <jnareb@gmail.com>
To: git@vger.kernel.org
Cc: Petr Baudis <pasky@ucw.cz>,
John 'Warthog9' Hawley <warthog9@eaglescrag.net>,
John 'Warthog9' Hawley <warthog9@kernel.org>,
Jakub Narebski <jnareb@gmail.com>
Subject: [RFC PATCHv4 18/17] gitweb/lib - Add clear() and size() methods to caching interface
Date: Thu, 24 Jun 2010 09:56:52 +0200 [thread overview]
Message-ID: <1277366213-28381-1-git-send-email-jnareb@gmail.com> (raw)
In-Reply-To: <1276531710-22945-1-git-send-email-jnareb@gmail.com>
Add ->size() method, which following Cache::Cache interface returns
estimated total size of all entries in whole cache (in the namsepace
assiciated with give cache instance). Note that ->get_size($key)
returns size of a single entry!
Add ->clear() method, which removes all entries from the namespace
associated with given cache instance. For safety it requires
namespace to be set to true value, which means that it cannot be
empty; therefore default namespace is changed to 'gitweb'.
The ->clear() method should be fairly safe, because it first renames
directory (which should be atomic), and only then removes it
(following code from CGI::Driver::File).
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This is new patch, implementing methods that allow for rudimentary
gitweb cache administration. For now it is only safe removing whole
cache, in the future it could include removing only stale (expired)
entries.
This is "backend" patch.
gitweb/lib/GitwebCache/SimpleFileCache.pm | 51 ++++++++++++++++++++++++++--
t/t9503/test_cache_interface.pl | 1 -
2 files changed, 47 insertions(+), 5 deletions(-)
diff --git a/gitweb/lib/GitwebCache/SimpleFileCache.pm b/gitweb/lib/GitwebCache/SimpleFileCache.pm
index 9eb4896..457bbe7 100644
--- a/gitweb/lib/GitwebCache/SimpleFileCache.pm
+++ b/gitweb/lib/GitwebCache/SimpleFileCache.pm
@@ -20,8 +20,9 @@ package GitwebCache::SimpleFileCache;
use strict;
use warnings;
-use File::Path qw(mkpath);
-use File::Temp qw(tempfile);
+use File::Path qw(mkpath rmtree);
+use File::Temp qw(tempfile mktemp);
+use File::Find qw(find);
use Digest::MD5 qw(md5_hex);
# by default, the cache nests all entries on the filesystem single
@@ -37,7 +38,7 @@ our $DEFAULT_CACHE_ROOT = "cache";
# by default we don't use cache namespace (empty namespace);
# empty namespace does not allow for simple implementation of clear() method.
#
-our $DEFAULT_NAMESPACE = '';
+our $DEFAULT_NAMESPACE = "gitweb";
# ......................................................................
# constructor
@@ -202,6 +203,9 @@ sub path_to_namespace {
return $self->{'path_to_namespace'};
}
+# $path = $cache->path_to_key($key);
+# $path = $cache->path_to_key($key, \$dir);
+#
# Take an human readable key, and return file path.
# Puts dirname of file path in second argument, if it is provided.
sub path_to_key {
@@ -321,7 +325,7 @@ sub get_size {
}
# ......................................................................
-# interface methods
+# interface methods dealing with single item
# Removing and expiring
@@ -412,6 +416,45 @@ sub compute {
return $data;
}
+# ......................................................................
+# interface methods dealing with whole namespace
+
+# $cache->clear();
+#
+# Remove all entries from the namespace.
+# Namespace must be defined and not empty.
+sub clear {
+ my $self = shift;
+
+ return unless $self->get_namespace();
+
+ my $namespace_dir = $self->path_to_namespace();
+ return if !-d $namespace_dir;
+
+ my $renamed_dir = mktemp($namespace_dir . '.XXXX');
+ rename($namespace_dir, $renamed_dir);
+ rmtree($renamed_dir);
+ die "Couldn't remove '$renamed_dir' directory"
+ if -d $renamed_dir;
+}
+
+# $size = $cache->size();
+#
+# Size of whole names (or whole cache if namespace empty)
+sub size {
+ my $self = shift;
+
+ my $namespace_dir = $self->path_to_namespace();
+ return if !-d $namespace_dir;
+
+ my $total_size = 0;
+ my $add_size = sub { $total_size += -s $_ };
+
+ File::Find::find({ wanted => $add_size, no_chdir => 1 }, $namespace_dir);
+
+ return $total_size;
+}
+
1;
__END__
# end of package GitwebCache::SimpleFileCache;
diff --git a/t/t9503/test_cache_interface.pl b/t/t9503/test_cache_interface.pl
index a84faf9..0a18f77 100755
--- a/t/t9503/test_cache_interface.pl
+++ b/t/t9503/test_cache_interface.pl
@@ -35,7 +35,6 @@ SKIP: {
unless ($GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT &&
$GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH);
- is($cache->get_namespace(), '', "default namespace is ''");
cmp_ok($cache->get_root(), 'eq', $GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT,
"default cache root is '$GitwebCache::SimpleFileCache::DEFAULT_CACHE_ROOT'");
cmp_ok($cache->get_depth(), '==', $GitwebCache::SimpleFileCache::DEFAULT_CACHE_DEPTH,
--
1.7.0.1
next prev parent reply other threads:[~2010-06-24 7:57 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-06-14 16:08 [RFC PATCHv4 00/17] gitweb: Simple file based output caching Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 01/17] gitweb: Return or exit after done serving request Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 02/17] gitweb: Fix typo in hash key name in %opts in git_header_html Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 03/17] gitweb: Prepare for splitting gitweb Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 04/17] gitweb/lib - Very simple file based cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 05/17] gitweb/lib - Stat-based cache expiration Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 06/17] gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 07/17] gitweb/lib - Simple select(FH) based output capture Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 08/17] gitweb/lib - Alternate ways of capturing output Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 09/17] gitweb/lib - Cache captured output (using get/set) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 10/17] gitweb: Add optional output caching (from gitweb/cache.pm) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 10/18] gitweb/lib - Cache captured output (using get/set) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 11/17] gitweb/lib - Adaptive cache expiration time Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 12/17] gitweb/lib - Use CHI compatibile (compute method) caching interface Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 13/17] gitweb/lib - Use locking to avoid 'cache miss stampede' problem Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 14/17] gitweb/lib - Serve stale data when waiting for filling cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 15/17] gitweb/lib - Regenerate (refresh) cache in background Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 16/17] gitweb: Show appropriate "Generating..." page when regenerating cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 17/17] gitweb: Add startup delay to activity indicator for cache Jakub Narebski
2010-06-24 7:56 ` Jakub Narebski [this message]
2010-06-24 7:56 ` [RFC PATCHv4 19/17] gitweb: Add beginnings of cache administration page (WIP) Jakub Narebski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1277366213-28381-1-git-send-email-jnareb@gmail.com \
--to=jnareb@gmail.com \
--cc=git@vger.kernel.org \
--cc=pasky@ucw.cz \
--cc=warthog9@eaglescrag.net \
--cc=warthog9@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).