git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jakub Narebski <jnareb@gmail.com>
To: git@vger.kernel.org
Cc: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>,
	Petr Baudis <pasky@ucw.cz>,
	Christian Couder <chriscool@tuxfamily.org>,
	John 'Warthog9' Hawley <warthog9@eaglescrag.net>,
	John 'Warthog9' Hawley <warthog9@kernel.org>,
	Jakub Narebski <jnareb@gmail.com>
Subject: [RFC PATCHv4 09/17] gitweb/lib - Cache captured output (using get/set)
Date: Mon, 14 Jun 2010 18:08:21 +0200	[thread overview]
Message-ID: <1276531710-22945-10-git-send-email-jnareb@gmail.com> (raw)
In-Reply-To: <1276531710-22945-1-git-send-email-jnareb@gmail.com>

Add GitwebCache::CacheOutput package, which contains cache_output
subroutine, and (re)exports capture_stop from GitwebCache::Capture.
The cache_output gets data from cache and prints it, or captures
output of provided subroutine (code reference), saves it to cache and
prints it.  It currently uses Cache::Cache compatibile (get, set)
interface to cache.  Capturing is currently (not configurable) done
using GitwebCache::Capture::SelectFH introduced in previous commit,
but any class derived from GitwebCache::Capture (like provided example
GitwebCache::Capture::TiedCapture and GitwebCache::Capture::PerlIO)
would work.

Gitweb would use cache_output to get page from cache, or to generate
page and save it to cache.  The capture_stop would be used in
die_error subroutine, because error pages would not be cached.

It is assumed that data is saved to cache _converted_, and should
therefore be read from cache and printed to STDOUT in ':raw' (binary)
mode.


Add t9503/test_cache_output.pl test, run as external test in
t9503-gitweb-caching.  It checks that cache_output behaves correctly,
namely that it saves and restores action output in cache, and that it
prints generated output or cached output.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
 gitweb/lib/GitwebCache/CacheOutput.pm |   62 +++++++++++++++++++++++++++++++
 t/t9503-gitweb-caching.sh             |    3 +
 t/t9503/test_cache_output.pl          |   66 +++++++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+), 0 deletions(-)
 create mode 100644 gitweb/lib/GitwebCache/CacheOutput.pm
 create mode 100755 t/t9503/test_cache_output.pl

diff --git a/gitweb/lib/GitwebCache/CacheOutput.pm b/gitweb/lib/GitwebCache/CacheOutput.pm
new file mode 100644
index 0000000..8195f0b
--- /dev/null
+++ b/gitweb/lib/GitwebCache/CacheOutput.pm
@@ -0,0 +1,62 @@
+# gitweb - simple web interface to track changes in git repositories
+#
+# (C) 2010, Jakub Narebski <jnareb@gmail.com>
+# (C) 2006, John 'Warthog9' Hawley <warthog19@eaglescrag.net>
+#
+# This program is licensed under the GPLv2
+
+#
+# Capturing and caching (gitweb) output
+#
+
+# Capture output, save it in cache and print it, or retrieve it from
+# cache and print it.
+
+package GitwebCache::CacheOutput;
+
+use strict;
+use warnings;
+
+use GitwebCache::SimpleFileCache;
+use GitwebCache::Capture::SelectFH qw(:all);
+
+use Exporter qw(import);
+our @EXPORT      = qw(cache_output capture_stop);
+our %EXPORT_TAGS = (all => [ @EXPORT ]);
+
+# cache_output($cache, $key, $action_code);
+#
+# Attempts to get $key from $cache; if successful, prints the value.
+# Otherwise, calls $action_code, capture its output and use
+# the captured output as the new value for $key in $cache,
+# then print captured output.
+our $CAPTURE_CLASS = 'GitwebCache::Capture::SelectFH';
+
+sub cache_output {
+	my ($cache, $key, $code) = @_;
+
+	my $data = $cache->get($key);
+
+	# capture and cache output, if there was nothing in the cache
+	if (!defined $data) {
+		my $capture = $CAPTURE_CLASS;
+		setup_capture($capture);
+
+		# do not use 'capture_block' prototype
+		$data = &capture_block($code);
+		$cache->set($key, $data) if defined $data;
+	}
+
+	# print cached data
+	if (defined $data) {
+		# select() instead of STDOUT is here for t9503 test:
+		binmode select(), ':raw';
+		print $data;
+	}
+
+	return $data;
+}
+
+1;
+__END__
+# end of package GitwebCache::CacheOutput;
diff --git a/t/t9503-gitweb-caching.sh b/t/t9503-gitweb-caching.sh
index 73b3f5a..0afcc0c 100755
--- a/t/t9503-gitweb-caching.sh
+++ b/t/t9503-gitweb-caching.sh
@@ -32,4 +32,7 @@ test_external 'GitwebCache::SimpleFileCache Perl API (in gitweb/cache.pm)' \
 test_external 'GitwebCache::Capture Perl API (in gitweb/cache.pm)' \
 	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_capture_interface.pl
 
+test_external 'GitwebCache::CacheOutput Perl API (in gitweb/cache.pm)' \
+	"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_output.pl
+
 test_done
diff --git a/t/t9503/test_cache_output.pl b/t/t9503/test_cache_output.pl
new file mode 100755
index 0000000..96eedb5
--- /dev/null
+++ b/t/t9503/test_cache_output.pl
@@ -0,0 +1,66 @@
+#!/usr/bin/perl
+use lib (split(/:/, $ENV{GITPERLLIB}));
+
+use warnings;
+use strict;
+
+use Test::More;
+
+# test source version
+use lib "$ENV{TEST_DIRECTORY}/../gitweb/lib";
+
+# ....................................................................
+
+# prototypes must be known at compile time, otherwise they do not work
+BEGIN { use_ok('GitwebCache::CacheOutput'); }
+
+# Test setting up $cache and $capture
+my $cache   = new_ok('GitwebCache::SimpleFileCache'   => [], 'The $cache  ');
+my $capture = new_ok('GitwebCache::Capture::SelectFH' => [], 'The $capture');
+
+# ......................................................................
+
+# Prepare for testing cache_output
+my $key = 'Key';
+my $action_output = <<'EOF';
+# This is data to be cached and shown
+EOF
+my $cached_output = <<"EOF";
+$action_output# (version recovered from cache)
+EOF
+sub action {
+	print $action_output;
+}
+
+# Catch output printed by cache_fetch
+# (only for 'print <sth>' and 'printf <sth>')
+sub capture_output_of_cache_output {
+	my $test_data = '';
+
+	open my $test_data_fh, '>', \$test_data;
+	my $oldfh = select($test_data_fh);
+
+	cache_output($cache, $key, \&action);
+
+	select($oldfh);
+	close $test_data_fh;
+
+	return $test_data;
+}
+
+# clean state
+$cache->set_expires_in(-1);
+$cache->remove($key);
+my $test_data;
+
+# first time (if there is no cache) generates cache entry
+$test_data = capture_output_of_cache_output();
+is($test_data, $action_output,        'action output is printed (generated)');
+is($cache->get($key), $action_output, 'action output is saved in cache (generated)');
+
+# second time (if cache is set/valid) reads from cache
+$cache->set($key, $cached_output);
+$test_data = capture_output_of_cache_output();
+is($test_data, $cached_output,        'action output is printed (from cache)');
+
+done_testing();
-- 
1.7.0.1

  parent reply	other threads:[~2010-06-14 16:10 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-06-14 16:08 [RFC PATCHv4 00/17] gitweb: Simple file based output caching Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 01/17] gitweb: Return or exit after done serving request Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 02/17] gitweb: Fix typo in hash key name in %opts in git_header_html Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 03/17] gitweb: Prepare for splitting gitweb Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 04/17] gitweb/lib - Very simple file based cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 05/17] gitweb/lib - Stat-based cache expiration Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 06/17] gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 07/17] gitweb/lib - Simple select(FH) based output capture Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 08/17] gitweb/lib - Alternate ways of capturing output Jakub Narebski
2010-06-14 16:08 ` Jakub Narebski [this message]
2010-06-14 16:08 ` [RFC PATCHv4 10/17] gitweb: Add optional output caching (from gitweb/cache.pm) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 10/18] gitweb/lib - Cache captured output (using get/set) Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 11/17] gitweb/lib - Adaptive cache expiration time Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 12/17] gitweb/lib - Use CHI compatibile (compute method) caching interface Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 13/17] gitweb/lib - Use locking to avoid 'cache miss stampede' problem Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 14/17] gitweb/lib - Serve stale data when waiting for filling cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 15/17] gitweb/lib - Regenerate (refresh) cache in background Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 16/17] gitweb: Show appropriate "Generating..." page when regenerating cache Jakub Narebski
2010-06-14 16:08 ` [RFC PATCHv4 17/17] gitweb: Add startup delay to activity indicator for cache Jakub Narebski
2010-06-24  7:56 ` [RFC PATCHv4 18/17] gitweb/lib - Add clear() and size() methods to caching interface Jakub Narebski
2010-06-24  7:56 ` [RFC PATCHv4 19/17] gitweb: Add beginnings of cache administration page (WIP) Jakub Narebski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1276531710-22945-10-git-send-email-jnareb@gmail.com \
    --to=jnareb@gmail.com \
    --cc=chriscool@tuxfamily.org \
    --cc=git@vger.kernel.org \
    --cc=pasky@ucw.cz \
    --cc=pavan.sss1991@gmail.com \
    --cc=warthog9@eaglescrag.net \
    --cc=warthog9@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).