[PATCH RFC] git-remote-mediawiki: push-by-rev

[PATCH RFC] git-remote-mediawiki: push-by-rev

[Célestin Matte]

From: Célestin Matte <celestin.matte@ensimag.fr>

This patch intends to introduce the by_rev strategy for the push command, as
already available for the fetch one.
This uses subroutines used by the fetch-by-rev strategy. I'm not sure it's
actually complete: can it be that simple? However, I tested on a local wiki and
it seemed to work perfectly. 
Should I add associate tests?

Célestin Matte (1):
  git-remote-mediawiki: push-by-rev

 contrib/mw-to-git/git-remote-mediawiki.perl | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

-- 
1.8.3.1.522.gd761f2b.dirty

[PATCH RFC] git-remote-mediawiki: push-by-rev

[Célestin Matte]

From: Célestin Matte <celestin.matte@ensimag.fr>

Add the push-by-rev option
This allows one to look for changes by revision instead of by page.
The result is a much faster push on little-activity wikis. Indeed, instead of
sending one request by page to check that the remote revision is our local
latest revision, we only send one request for every new local revision.

Signed-off-by: Célestin Matte <celestin.matte@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
---
 contrib/mw-to-git/git-remote-mediawiki.perl | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl
index 9ff45fd..fa49882 100755
--- a/contrib/mw-to-git/git-remote-mediawiki.perl
+++ b/contrib/mw-to-git/git-remote-mediawiki.perl
@@ -102,6 +102,15 @@ if (!$fetch_strategy) {
 	$fetch_strategy = 'by_page';
 }
 
+my $push_strategy = run_git("config --get remote.${remotename}.pushStrategy");
+if (!$push_strategy) {
+	$push_strategy = run_git('config --get mediawiki.pushStrategy');
+}
+chomp($push_strategy);
+if (!$push_strategy) {
+	$push_strategy = 'by_page';
+}
+
 # Remember the timestamp corresponding to a revision id.
 my %basetimestamps;
 
@@ -512,7 +521,7 @@ sub get_last_local_revision {
 # Get the last remote revision without taking in account which pages are
 # tracked or not. This function makes a single request to the wiki thus
 # avoid a loop onto all tracked pages. This is useful for the fetch-by-rev
-# option.
+# and the push-by-rev options.
 sub get_last_global_remote_rev {
 	mw_connect_maybe();
 
@@ -1160,8 +1169,19 @@ sub mw_push_revision {
 	my $local = shift;
 	my $remote = shift; # actually, this has to be "refs/heads/master" at this point.
 	my $last_local_revid = get_last_local_revision();
+	my $last_remote_revid;
 	print {*STDERR} ".\n"; # Finish sentence started by get_last_local_revision()
-	my $last_remote_revid = get_last_remote_revision();
+	if ($push_strategy eq 'by_page') {
+		print {*STDERR} "Pushing export data by pages...\n";
+		$last_remote_revid = get_last_remote_revision();
+	} elsif ($push_strategy eq 'by_rev') {
+		print {*STDERR} "Pushing export data by revs...\n";
+		$last_remote_revid = get_last_global_remote_rev();
+	} else {
+		print {*STDERR} qq(fatal: invalid push strategy "${push_strategy}".\n);
+		print {*STDERR} "Check your configuration variables remote.${remotename}.pushStrategy and mediawiki.pushStrategy\n";
+		exit 1;
+	}
 	my $mw_revision = $last_remote_revid;
 
 	# Get sha1 of commit pointed by local HEAD
-- 
1.8.3.1.522.gd761f2b.dirty

Re: [PATCH RFC] git-remote-mediawiki: push-by-rev

[Matthieu Moy]

Célestin Matte <celestin.matte@ensimag.fr> writes:

> This uses subroutines used by the fetch-by-rev strategy. I'm not sure it's
> actually complete: can it be that simple?

The function says:

  # Get the last remote revision without taking in account which pages are
  # tracked or not. This function makes a single request to the wiki thus
  # avoid a loop onto all tracked pages. This is useful for the fetch-by-rev
  # option.
  sub get_last_global_remote_rev {
  ...

So I don't think this would work when you track only a subset of pages.
If an untracked page has been modified, then the global last revision
has increased and you'll get a non-fast forward. If you try to pull,
it'll tell you there's nothing to import.

You'd have to iterate through revisions between the one given by
get_last_global_remote_rev and the last one the local repo knows about,
and remove ones touching untracked pages from the list.

> However, I tested on a local wiki and it seemed to work perfectly.
> Should I add associate tests?

See what t/t9364-pull-by-rev.sh does. You can set pushStrategy too in
the same file (this won't test all push/pull combinations, but push and
pull should be independant enough, so testing both unset and both set
should be enough).

Not sure we have enough test for push with a subset of pages though.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/