Git development
 help / color / mirror / Atom feed
* [PATCH v7] gitweb: add test suite with Test::WWW::Mechanize::CGI
From: Lea Wiemann @ 2008-06-24  2:18 UTC (permalink / raw)
  To: git; +Cc: Lea Wiemann, Jakub Narebski
In-Reply-To: <4860556F.5060206@gmail.com>

This test uses Test::WWW::Mechanize::CGI to check gitweb's output.  It
also uses HTML::Lint, XML::Parser, and Archive::Tar (if present, each)
to validate the HTML/XML/tgz output, and checks all links on the
tested pages if --long-tests is given.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>
---
Changes since v6:

- Removed GITPERL again.

- Added some basic tests for the page contents of blob and blob_plain
  views.

- Incorporated Jakub's suggestions (see parent posts); in particular
  blame and feed views *do* get spidered now in --long-tests mode, but
  the spider tests are marked as TODO and thus don't cause the test
  suite to fail.

- Some minor (internal) changes: @files only contains blobs, not
  trees; improved the logic that avoids checking pages twice; die if
  test.pl is run directly rather than from the shell script.

 t/t9503-gitweb-Mechanize.sh |  132 +++++++++++++++
 t/t9503/test.pl             |  378 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 510 insertions(+), 0 deletions(-)
 create mode 100755 t/t9503-gitweb-Mechanize.sh
 create mode 100755 t/t9503/test.pl

diff --git a/t/t9503-gitweb-Mechanize.sh b/t/t9503-gitweb-Mechanize.sh
new file mode 100755
index 0000000..6f7168e
--- /dev/null
+++ b/t/t9503-gitweb-Mechanize.sh
@@ -0,0 +1,132 @@
+#!/bin/sh
+#
+# Copyright (c) 2008 Jakub Narebski
+# Copyright (c) 2008 Lea Wiemann
+#
+
+# This test supports the --long-tests option.
+
+# This test only runs on Perl 5.8 and later versions, since
+# Test::WWW::Mechanize::CGI requires Perl 5.8.
+
+test_description='gitweb tests (using WWW::Mechanize)
+
+This test uses Test::WWW::Mechanize::CGI to test gitweb.'
+
+# helper functions
+
+safe_chmod () {
+	chmod "$1" "$2" &&
+	if [ "$(git config --get core.filemode)" = false ]
+	then
+		git update-index --chmod="$1" "$2"
+	fi
+}
+
+. ./test-lib.sh
+
+# check if test can be run
+perl -MEncode -e 'decode_utf8("", Encode::FB_CROAK)' >/dev/null 2>&1 || {
+	test_expect_success \
+		'skipping gitweb tests, perl version is too old' :
+	test_done
+	exit
+}
+
+perl -MTest::WWW::Mechanize::CGI -e '' >/dev/null 2>&1 || {
+	test_expect_success \
+		'skipping gitweb tests, Test::WWW::Mechanize::CGI not found' :
+	test_done
+	exit
+}
+
+# set up test repository
+test_expect_success 'set up test repository' '
+
+	echo "Not an empty file." > file &&
+	git add file &&
+	test_tick && git commit -a -m "Initial commit." &&
+	git branch b &&
+
+	echo "New file" > new_file &&
+	git add new_file &&
+	test_tick && git commit -a -m "File added." &&
+
+	safe_chmod +x new_file &&
+	test_tick && git commit -a -m "Mode changed." &&
+
+	git mv new_file renamed_file &&
+	test_tick && git commit -a -m "File renamed." &&
+
+	rm renamed_file &&
+	ln -s file renamed_file &&
+	test_tick && git commit -a -m "File to symlink." &&
+	git tag with-symlink &&
+
+	git rm renamed_file &&
+	rm -f renamed_file &&
+	test_tick && git commit -a -m "File removed." &&
+
+	cp file file2 &&
+	git add file2 &&
+	test_tick && git commit -a -m "File copied." &&
+
+	echo "New line" >> file2 &&
+	safe_chmod +x file2 &&
+	test_tick && git commit -a -m "Mode change and modification." &&
+
+	git checkout b &&
+	echo "Branch" >> b &&
+	git add b &&
+	test_tick && git commit -a -m "On branch" &&
+	git checkout master &&
+	test_tick && git pull . b
+'
+
+# set up empty repository
+# TODO!
+
+# set up repositories for gitweb
+# TODO!
+
+# set up gitweb configuration
+safe_pwd="$(perl -MPOSIX=getcwd -e 'print quotemeta(getcwd)')"
+cat >gitweb_config.perl <<EOF
+# gitweb configuration for tests
+
+our \$version = "current";
+our \$GIT = "$GIT_EXEC_PATH/git";
+our \$projectroot = "$safe_pwd";
+our \$project_maxdepth = 8;
+our \$home_link_str = "projects";
+our \$site_name = "[localhost]";
+our \$site_header = "";
+our \$site_footer = "";
+our \$home_text = "indextext.html";
+our @stylesheets = ("file:///$safe_pwd/../../gitweb/gitweb.css");
+our \$logo = "file:///$safe_pwd/../../gitweb/git-logo.png";
+our \$favicon = "file:///$safe_pwd/../../gitweb/git-favicon.png";
+our \$projects_list = "";
+our \$export_ok = "";
+our \$strict_export = "";
+our %feature;
+\$feature{'blame'}{'default'} = [1];
+
+1;
+__END__
+EOF
+
+cat >.git/description <<EOF
+$0 test repository
+EOF
+
+GITWEB_CONFIG="$(pwd)/gitweb_config.perl"
+export GITWEB_CONFIG
+
+# run tests
+
+test_external \
+	'test gitweb output' \
+	perl ../t9503/test.pl
+
+test_done
diff --git a/t/t9503/test.pl b/t/t9503/test.pl
new file mode 100755
index 0000000..872dbfd
--- /dev/null
+++ b/t/t9503/test.pl
@@ -0,0 +1,378 @@
+#!/usr/bin/perl
+use lib (split(/:/, $ENV{GITPERLLIB}));
+
+# This test supports the --long-tests option.
+
+use warnings;
+use strict;
+
+use Cwd qw(abs_path);
+use File::Spec;
+use File::Temp;
+
+use Test::More qw(no_plan);
+use Test::WWW::Mechanize::CGI;
+
+die "this must be run by calling the t/t*.sh shell script(s)\n"
+    if Cwd->cwd !~ /trash directory$/;
+
+our $long_tests = $ENV{GIT_TEST_LONG}; # "our" so we can use "local $long_tests"
+
+eval { require Archive::Tar; };
+my $archive_tar_installed = !$@
+    or diag('Archive::Tar is not installed; no tests for valid snapshots');
+
+eval { require HTML::Lint; };
+my $html_lint_installed = !$@
+    or diag('HTML::Lint is not installed; no HTML validation tests');
+
+eval { require XML::Parser; };
+my $xml_parser_installed = !$@
+    or diag('XML::Parser is not installed; no tests for well-formed XML');
+
+sub rev_parse {
+	my $name = shift;
+	chomp(my $hash = `git rev-parse $name 2> /dev/null`);
+	$hash or undef;
+}
+
+sub get_type {
+	my $name = shift;
+	chomp(my $type = `git cat-file -t $name 2> /dev/null`);
+	$type or undef;
+}
+
+my @revisions = split /\s/, `git-rev-list --first-parent HEAD`;
+chomp(my @heads = map { (split('/', $_))[2] } `git-for-each-ref --sort=-committerdate refs/heads`);
+chomp(my @tags = map { (split('/', $_))[2] } `git-for-each-ref --sort=-committerdate refs/tags`);
+chomp(my @root_entries = `git-ls-tree --name-only HEAD`);
+my @files = grep { get_type("HEAD:$_") eq 'blob' } @root_entries or die;
+
+my $gitweb = abs_path(File::Spec->catfile('..', '..', 'gitweb', 'gitweb.cgi'));
+
+# This subroutine was copied (and modified to work with blanks in the
+# application path) from WWW::Mechanize::CGI 0.3, which is licensed
+# 'under the same terms as perl itself' and thus GPL compatible.
+# http://rt.cpan.org/Ticket/Display.html?id=36654
+my $cgi = sub {
+	# Use exec, not the shell, to support embedded whitespace in
+	# the path to gitweb.cgi.
+	my $status = system $gitweb $gitweb;
+	my $value  = $status >> 8;
+
+	croak( qq/Failed to execute application '$gitweb'. Reason: '$!'/ )
+	    if ( $status == -1 );
+	croak( qq/Application '$gitweb' exited with value: $value/ )
+	    if ( $value > 0 );
+};
+
+my $mech = new Test::WWW::Mechanize::CGI;
+$mech->cgi($cgi);
+# On some systems(?) it's necessary to have %ENV here, otherwise the
+# CGI process won't get *any* of the current environment variables
+# (not even PATH, etc.)
+$mech->env(%ENV,
+	   GITWEB_CONFIG => $ENV{'GITWEB_CONFIG'},
+	   SCRIPT_FILENAME => $gitweb,
+	   $mech->env);
+
+# import config, predeclaring config variables
+our $site_name;
+require_ok($ENV{'GITWEB_CONFIG'})
+	or diag('Could not load gitweb config; some tests would fail');
+
+# Perform non-recursive checks on the current page, but do not check
+# the status code.
+my %verified_uris;
+sub _verify_page {
+	my ($uri, $fragment) = split '#', $mech->uri;
+	$mech->content_like(qr/(name|id)="$fragment"/,
+			    "[auto] fragment #$fragment exists ($uri)")
+	    if $fragment;
+
+	return 1 if $verified_uris{$uri};
+	$verified_uris{$uri} = 1;
+
+	# Internal errors yield 200 but cause gitweb.cgi to exit with
+	# non-zero exit code, which Mechanize::CGI translates to 500,
+	# so we don't really need to check for "Software error" here,
+	# provided that the test cases always check the status code.
+	#$mech->content_lacks('<h1>Software error:</h1>') or return 0;
+
+	# Validate.  This is fast, so we can do it even without
+	# $long_tests.
+	$mech->html_lint_ok('[auto] validate HTML') or return 0
+	    if $html_lint_installed && $mech->is_html;
+	my $content_type = $mech->response->header('Content-Type')
+	    or die "$uri does not have a Content-Type header";
+	if ($xml_parser_installed && $content_type =~ /xml/) {
+		eval { XML::Parser->new->parse($mech->content); };
+		ok(!$@, "[auto] check for XML well-formedness ($uri)") or diag($@);
+	}
+	if ($archive_tar_installed && $uri =~ /sf=tgz/) {
+		my $snapshot_file = File::Temp->new;
+		print $snapshot_file $mech->content;
+		close $snapshot_file;
+		my $t = Archive::Tar->new;
+		$t->read($snapshot_file->filename, 1);
+		ok($t->get_files, "[auto] valid tgz snapshot ($uri)");
+	}
+	# WebService::Validator::Feed::W3C would be nice to
+	# use, but it doesn't support direct input (as opposed
+	# to URIs) long enough for our feeds.
+
+	return 1;
+}
+
+# Verify and spider the current page, the latter only if --long-tests
+# (-l) is given.  Do not check the status code of the current page.
+my %spidered_uris;  # pages whose links have been checked
+my %status_checked_uris;  # verified pages whose status is known to be 2xx
+sub check_page {
+	_verify_page or return 0;
+	my $orig_url = $mech->uri;
+	if ($long_tests && !$spidered_uris{$mech->uri} ) {
+		$spidered_uris{$mech->uri} = 1;
+		for my $url (map { $_->url_abs } $mech->followable_links) {
+			if (!$status_checked_uris{$url}) {
+				$status_checked_uris{$url} = 1;
+				$mech->get_ok($url, "[auto] check link ($url)")
+				    or diag("broken link to $url on $orig_url");
+				_verify_page;
+				$mech->back;
+			}
+		}
+	}
+	return 1;
+}
+
+my $baseurl = "http://localhost";
+my ($params, $url, $pagedesc, $status);
+
+# test_page ( <params>, <page_description>, <expected_status> )
+# Example:
+# if (test_page('?p=.git;a=summary', 'repository summary')) {
+#     $mech->...;
+#     $mech->...;
+# }
+#
+# Test that the page can be opened, call _verify_page on it, and
+# return true if there was no test failure.  Also set the global
+# variables $params, $pagedesc, and $url for use in the if block.
+# Optionally pass a third parameter $status to test the HTTP status
+# code of the page (useful for error pages).  You can also pass a full
+# URL instead of just parameters as the first parameter.
+sub test_page {
+	($params, $pagedesc, $status) = @_;
+	$pagedesc = $pagedesc ? " -- $pagedesc" : '';
+	if($params =~ /^$baseurl/) {
+		$url = "$params";
+	} else {
+		$url = "$baseurl$params";
+	}
+	if ($status) {
+		$mech->get($url);
+	} else {
+		$mech->get_ok($url, "get $url$pagedesc") or return 0;
+	}
+	check_page or return 0;
+	if ($status) {
+		return is($mech->status, $status, "getting $url$pagedesc -- yields $status");
+	} else {
+		return 1;
+	}
+}
+
+# follow_link ( \%parms, $pagedesc )
+# Example: follow_link( { text => 'commit' }, 'first commit link')
+# Like test_page, but does not support status code testing.
+sub follow_link {
+	(my $parms, $pagedesc) = @_;
+	unless ($mech->follow_link_ok($parms, "follow link: $pagedesc")) {
+		diag("cannot follow link ($pagedesc) on page " . $mech->uri);
+		return 0;
+	}
+	$url = $mech->uri;
+	return check_page;
+}
+
+if (test_page '', 'project list (implicit)') {
+	$mech->title_like(qr!$site_name!,
+		"title contains $site_name");
+	$mech->content_contains('./t9503-gitweb-Mechanize.sh test repository', 
+		'lists test repository (by description)');
+}
+
+# Test repository summary: implicit, implicit with pathinfo, explicit.
+for my $sumparams ('?p=.git', '/.git', '?p=.git;a=summary') {
+	if (test_page $sumparams, 'repository summary') {
+		$mech->title_like(qr!$site_name.*\.git/summary!,
+				  "title contains $site_name and \".git/summary\"");
+	}
+}
+
+# Search form (on summary page).
+$mech->get_ok('?p=.git', 'get repository summary');
+if ($mech->submit_form_ok( { form_number => 1,
+			     fields => { 's' => 'Initial' }
+			   }, "submit search form (default)")) {
+	check_page;
+	$mech->content_contains('Initial commit',
+				'content contains searched commit');
+}
+
+test_page('?p=non-existent.git', 'non-existent project', 404);
+test_page('?p=.git;a=commit;h=non-existent', 'non-existent commit', 404);
+
+
+# Summary view
+
+# Check short log.  To do: Extract into separate test_short_log
+# function since the short log occurs on several pages.
+$mech->get_ok('?p=.git', 'get repository summary');
+for my $revision (@revisions[0..2]) {
+	for my $link_text qw( commit commitdiff tree snapshot ) {
+		ok($mech->find_link(url_abs_regex => qr/h=$revision/, text => $link_text), "$link_text link for $revision");
+	}
+}
+# Check that branches and tags are highlighted in green and yellow in
+# the shortlog.  We assume here that we are on master, so it should be
+# at the top.
+$mech->content_like(qr{<span [^>]*class="head"[^>]*>master</span>},
+		    'master branch is highlighted in shortlog');
+$mech->content_like(qr{<span [^>]*class="tag"[^>]*>$tags[0]</span>},
+		    "$tags[0] (most recent tag) is highlighted in shortlog");
+
+# Check heads.  (This should be extracted as well.)
+for my $head (@heads) {
+	for my $link_text qw( shortlog log tree ) {
+		ok($mech->find_link(url_abs_regex => qr{h=refs/heads/$head}, text => $link_text), "$link_text link for head '$head'");
+	}
+}
+
+# Check tags (assume we only have tags referring to commits).
+for my $tag (@tags) {
+	my $commit = rev_parse("$tag^{commit}");
+	ok($mech->find_link(url_abs_regex => qr{h=refs/tags/$tag}, text => 'shortlog'),
+	   "shortlog link for tag '$tag'");
+	ok($mech->find_link(url_abs_regex => qr{h=refs/tags/$tag}, text => 'log'),
+	   "log link for tag '$tag'");
+	ok($mech->find_link(url_abs_regex => qr{h=$commit}, text => 'commit'),
+	   "commit link for tag '$tag'");
+	# To do: Test tag link for tag objects.
+	# Why don't we have tree + snapshot links?
+}
+
+
+# RSS/Atom/OPML view
+# Simply retrieve and verify well-formedness, but don't spider.
+$mech->get_ok('?p=.git;a=atom', 'Atom feed') and _verify_page;
+$mech->get_ok('?p=.git;a=rss', 'RSS feed') and _verify_page;
+TODO: {
+	# Now spider -- but there are broken links.
+	# http://mid.gmane.org/485EB333.5070108@gmail.com
+	local $TODO = "fix broken links in Atom/RSS feeds";
+	test_page('?p=.git;a=atom', 'Atom feed');
+	test_page('?p=.git;a=rss', 'RSS feed');
+}
+test_page('?a=opml', 'OPML outline');
+
+
+# Commit view
+if (test_page('?p=.git;a=commit;h=master')) {
+	ok($mech->find_link(url_abs_regex => qr/a=tree/),
+	   "tree link on commit page ($url)");
+	$mech->content_like(qr/A U Thor/, "author mentioned on commit page ($url)");
+}
+
+
+# Commitdiff view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text_regex => qr/file added/i }, 'commit with added file') &&
+    follow_link( { text => 'commitdiff' }, 'commitdiff')) {
+	$mech->content_like(qr/new file with mode/, "commitdiff has diffstat ($url)");
+	$mech->content_like(qr/new file mode/, "commitdiff has diff ($url)");
+}
+
+
+# Tree view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text => 'tree' }, 'first tree link on page')) {
+	for my $file (@files) {
+		my $file_hash = rev_parse("HEAD:$file");
+		ok($mech->find_link(text => $file), "'$file' listed (and linked) in tree view ($url)");
+		for my $link_text qw( blob blame history raw ) {
+			my $link = $mech->find_link(url_abs_regex => qr/[^a-z]f=$file(;|$)/,
+						    text => $link_text);
+			ok($link, "'$file' file has $link_text link in tree view ($url)");
+		}
+	}
+	# To do: write tests for subtrees.  (Need to set up subtrees
+	# in t9503-gitweb-Mechanize.sh.)
+}
+
+
+# Blame view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text => 'tree' }, 'first tree link on page')) {
+	for my $blame_link ($mech->find_all_links(text => 'blame')) {
+		my $url = $blame_link->url;
+		$mech->get_ok($url, "get $url -- blame link on tree view")
+		    and _verify_page;
+		$mech->content_like(qr/A U Thor/,
+				    "author mentioned on blame page");
+		TODO: {
+			# Now spider -- but there are broken links.
+			# http://mid.gmane.org/485EC621.7090101@gmail.com
+			local $TODO = "fix broken links in certain blame views";
+			check_page;
+		}
+	}
+}
+
+
+# History view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text => 'tree' }, 'first tree link on page')) {
+	for my $history_link ($mech->find_all_links(text => 'history')) {
+		test_page($history_link->url, "follow history link from tree view");
+		# To do: Expand.
+	}
+}
+
+
+# Blob view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text => 'tree' }, 'first tree link on page')) {
+	for my $file (@files) {
+		if (follow_link( { text => $file, url_abs_regex => qr/a=blob/ },
+				 "\"$file\" (blob) entry on tree view")) {
+			chomp(my $first_line_regex = (`cat "$file"`)[0]);
+			$first_line_regex =~ s/ / |&nbsp;/g;
+			# Hope that the first line doesn't contain any
+			# HTML-escapable character.
+			$mech->content_like(qr/$first_line_regex/,
+					    "blob view contains first line of file ($url)");
+			$mech->back;
+		}
+	}
+}
+
+
+# Raw (blob_plain) view
+if ($mech->get_ok('?p=.git', 'get repository summary') &&
+    follow_link( { text => 'tree' }, 'first tree link on page')) {
+	for my $file (@files) {
+		if (follow_link( { text => 'raw', url_abs_regex => qr/[^a-z]f=$file(;|$)/ },
+				 "raw (blob_plain) entry for \"$file\" in tree view")) {
+			chomp(my $last_line = (`cat "$file"`)[-1]);
+			$mech->content_contains(
+				$last_line, "blob_plain view contains last line of file");
+			$mech->back;
+		}
+	}
+}
+
+
+1;
+__END__
-- 
1.5.6.109.g0c85.dirty

^ permalink raw reply related

* Re: [PATCH v2] git-svn: make rebuild respect rewriteRoot option
From: Joey Hess @ 2008-06-24  2:11 UTC (permalink / raw)
  To: Git mailing list
In-Reply-To: <20080624021736.2d272d9c@neuron>

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

Jan Krüger wrote:
> Suppose someone fetches git-svn-ified commits from another repo and then
> attempts to use 'git-svn init --rewrite-root=foo bar'. Using git svn rebase
> after that will fail badly:
> 
>  * For each commit tried by working_head_info, rebuild is called indirectly.
>  * rebuild will iterate over all commits and skip all of them because the
>    URL does not match. Because of that no rev_map file is generated at all.
>  * Thus, rebuild will run once for every commit. This takes ages.
>  * In the end there still isn't any rev_map file and thus working_head_info
>    fails.

Very timely, I had just been beating my head against this problem.
Your patch works great for me.

-- 
see shy jo

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply

* Re: [PATCH 2/3] implement some resilience against pack corruptions
From: Nicolas Pitre @ 2008-06-24  2:20 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Junio C Hamano, git
In-Reply-To: <20080624013331.GR11793@spearce.org>

On Mon, 23 Jun 2008, Shawn O. Pearce wrote:

> Nicolas Pitre <nico@cam.org> wrote:
> > We should be able to fall back to loose objects or alternative packs when
> > a pack becomes corrupted.  This is especially true when an object exists
> > in one pack only as a delta but its base object is corrupted.  Currently
> > there is no way to retrieve the former object even if the later is
> > available in another pack or loose.
> 
> Dang, nice timing Nico.  We were bitten by something like this at
> day-job a couple of weeks back.  Adding this sort of support went
> onto my internal todo-list.  I'm glad you beat me to it.  :)
>  
> > Yes I've been bitten by the corruption described above...
> 
> But sorry to hear about this.

Well, this is fixed now.  So the theory worked.  ;-)


Nicolas

^ permalink raw reply

* [PATCH/RFC] Created git-basis and modified git-bundle to accept --stdin.
From: Adam Brewster @ 2008-06-24  2:21 UTC (permalink / raw)
  To: git
In-Reply-To: <1214273297-8257-2-git-send-email-adambrewster@gmail.com>

Git-basis is a perl script that remembers bases for use by git-bundle.
Code from rev-parse was borrowed to allow git-bundle to handle --stdin.

Signed-off-by: Adam Brewster <adambrewster@gmail.com>
---
Git-bundle is a great tool to move repositories between computers on
different networks, but in order to create thin packs, it needs to be
given a list of objects that are available on the remote end.  I don't
like any of the options listed in the git-bundle documentation, so I did
this.

Git-basis is a script that maintains lists of objects that exist in
various places.

With this patch, the command:

git-basis my-basis | git-bundle create my-bundle --all --stdin

will prepare a bundle containing everything except that which is a part
of my-basis.

Then you can add the objects in the bundle to the basis, so they won't
get included in the next pack like this:

git-basis --update my-basis < my-bundle

I'm sure that my implementation is crap, but I think this is a useful
idea.  Anybody agree?  Disagree?

 bundle.c  |   22 +++++++++++++++++-
 git-basis |   71 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+), 2 deletions(-)
 create mode 100755 git-basis

diff --git a/bundle.c b/bundle.c
index 0ba5df1..0af12d7 100644
--- a/bundle.c
+++ b/bundle.c
@@ -227,8 +227,26 @@ int create_bundle(struct bundle_header *header,
const char *path,

       /* write references */
       argc = setup_revisions(argc, argv, &revs, NULL);
-       if (argc > 1)
-               return error("unrecognized argument: %s'", argv[1]);
+
+       for (i = 1; i < argc; i++) {
+               if ( !strcmp(argv[i], "--stdin") ) {
+                       char line[1000];
+                               while (fgets(line, sizeof(line),
stdin) != NULL) {
+                               int len = strlen(line);
+                               if (len && line[len - 1] == '\n')
+                                       line[--len] = 0;
+                               if (!len)
+                                       break;
+                               if (line[0] == '-')
+                                       die("options not supported in
--stdin mode");
+                               if (handle_revision_arg(line, &revs, 0, 1))
+                                       die("bad revision '%s'", line);
+                       }
+                       continue;
+               }
+
+               return error("unrecognized argument: %s'", argv[i]);
+       }

       for (i = 0; i < revs.pending.nr; i++) {
               struct object_array_entry *e = revs.pending.objects + i;
diff --git a/git-basis b/git-basis
new file mode 100755
index 0000000..891635c
--- /dev/null
+++ b/git-basis
@@ -0,0 +1,71 @@
+#!/usr/bin/perl
+
+use strict;
+
+use Git;
+
+my $r = Git->repository();
+my $d = $r->repo_path();
+
+if ( ! -d "$d/bases" ) {
+    system( "mkdir '$d/bases'" );
+}
+
+if ( $#ARGV == -1 ) {
+    print "usage: git-basis [--update] basis1...\n";
+    exit;
+} elsif ( $ARGV[0] eq '--update' ) {
+    shift @ARGV;
+
+    my %new = ();
+    while (<STDIN>) {
+       if (!/^^?([a-z0-9]{40})/) {next;}
+       $new{$1} = 1;
+    }
+
+    foreach my $f (@ARGV) {
+       my %these = ();
+       open F, "<$d/bases/$f" || die "Can't open bases/$f: $!";
+       while (<F>) {
+           if (!/^([a-z0-9]{40})/) {next;}
+           $these{$1} = 1;
+       }
+       close F;
+       open F, ">>$d/bases/$f" || die "Can't open bases/$f: $!";
+       print F "\#" . `date`;
+       foreach my $b (keys %new) {
+           if (exists($these{$b})) {next;}
+           print F "$b\n";
+       }
+       close F;
+    }
+} else {
+    my $n = 0;
+    my %basis = ();
+
+    my $f = shift @ARGV;
+    open F, "<$d/bases/$f" || die "Can't open bases/$f: $!";
+    while (<F>) {
+       if (!/^([a-z0-9]{40})/) {next;}
+       $basis{$1} = $n;
+    }
+    close F;
+
+    foreach $f (@ARGV) {
+       open F, "<$d/bases/$f" || die "Can't open bases/$f: $!";
+       while (<F>) {
+           if (!/^([a-z0-9]{40})/) {next;}
+           if (!exists($basis{$1})) {next;}
+
+           if ($basis{$1} == $n) {$basis{$1}++;}
+           else {delete $basis{$1};}
+       }
+       close F;
+       $n++;
+    }
+
+    foreach my $b (keys %basis) {
+       if ( $basis{$b} != $n ) {next;}
+       print "^$b\n";
+    }
+}
--
1.5.5.1.211.g65ea3.dirty

^ permalink raw reply related

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  3:18 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.1.10.0806232213360.2979@xanadu.home>

-- Nicolas Pitre wrote:
>> or we expect "running without ever collecting the log"
>> to be a valid mode of operation... which I doubt is the case.
>
> Why not?

Is see the hole I left in my logic, so let me restate.

... or we expect "human parsing of the the log" is a valid common
user-interface for non-git developers.

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  3:18 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.1.10.0806232213360.2979@xanadu.home>

-- Nicolas Pitre wrote:
>> or we expect "running without ever collecting the log"
>> to be a valid mode of operation... which I doubt is the case.
>
> Why not?

Is see the hole I left in my logic, so let me restate.

... or we expect "human parsing of the the log" is a valid common
user-interface for non-git developers.

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Nicolas Pitre @ 2008-06-24  4:03 UTC (permalink / raw)
  To: David Jeske; +Cc: git
In-Reply-To: <willow-jeske-01l5e9cgFEDjCh3F@videotron.ca>

On Tue, 24 Jun 2008, David Jeske wrote:

> -- Nicolas Pitre wrote:
> >> or we expect "running without ever collecting the log"
> >> to be a valid mode of operation... which I doubt is the case.
> >
> > Why not?
> 
> Is see the hole I left in my logic, so let me restate.
> 
> ... or we expect "human parsing of the the log" is a valid common
> user-interface for non-git developers.

The reflog is one of the primary user interface for all git users. 
Please just try:

	git reflog

and see for yourself.

And if you want more details, then just try:

	git log -g

You may even try any combination of flags in addition to -g with
'git log'.

I hope you'll feel much safer then.


Nicolas

^ permalink raw reply

* Re: [PATCH v6] gitweb: add test suite with Test::WWW::Mechanize::CGI
From: Junio C Hamano @ 2008-06-24  4:20 UTC (permalink / raw)
  To: Lea Wiemann; +Cc: Jakub Narebski, git
In-Reply-To: <485FE3F7.4040102@gmail.com>

Lea Wiemann <lewiemann@gmail.com> writes:

>> How it is different from PERL_PATH?
>
> Right, I didn't think of that.  PERL_PATH isn't available in the tests
> though, it's only used internally by the Makefile to generate (among
> other things) gitweb.cgi.  This means that while we can control under
> which Perl version gitweb.cgi runs, we cannot control under which Perl
> version the test suite runs (at least without $PATH trickery).  Does
> this bother us?
>
> If yes, I'd suggest we keep GITPERL but rename it to GIT_TEST_PERL,
> because that's what it's about.  If not, I'll rip it out and simply
> call 'perl' in the test shell script, whatever version it may be.

That sounds wrong, as the point of tests would be to make sure the stuff
you are going to install would work with what you thought will be used
from the system.

If "isn't available in the tests" is the problem, is it possible to make
it available?  We are passing down SHELL_PATH from primary Makefile to t/
and you should be able to do the same for Perl path...

About the Test::WWW:Mechanize::CGI thing, how widely available is it?  I
do not think it is packaged for Debian nor Ubuntu, for example.

^ permalink raw reply

* Re: [PATCH 2/3] implement some resilience against pack corruptions
From: Junio C Hamano @ 2008-06-24  4:49 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.1.10.0806232122180.2979@xanadu.home>

Nicolas Pitre <nico@cam.org> writes:

> We should be able to fall back to loose objects or alternative packs when
> a pack becomes corrupted.  This is especially true when an object exists
> in one pack only as a delta but its base object is corrupted.  Currently
> there is no way to retrieve the former object even if the later is
> available in another pack or loose.

All three patches applied, but with the following squashed in.  Thanks

 sha1_file.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index a5cef94..9330bc4 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1418,8 +1418,8 @@ const char *packed_object_info_detail(struct packed_git *p,
 		case OBJ_OFS_DELTA:
 			obj_offset = get_delta_base(p, &w_curs, &curpos, type, obj_offset);
 			if (!obj_offset)
-				die("pack %s contains bad delta base reference",
-				    p->pack_name, type);
+				die("pack %s contains bad delta base reference of type %s",
+				    p->pack_name, typename(type));
 			if (*delta_chain_length == 0) {
 				revidx = find_pack_revindex(p, obj_offset);
 				hashcpy(base_sha1, nth_packed_object_sha1(p, revidx->nr));

^ permalink raw reply related

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  4:40 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.1.10.0806232356340.2979@xanadu.home>

-- Nicolas Pitre wrote:
> I hope you'll feel much safer then.

I moved a branch around and then deleted it, and I don't see any record in the
reflog of where it was, or that it ever was.

Am I missing something about how branches are used? I see some language in "git
tag" about how attempts are made to assure that others can't move around
semi-immutable tags during push, but I don't see any such language about
branches. What prevents someone from accidentally deleting an old branch that
nobody is watching, but is important to the history and then not noticing as gc
silently deletes the old deltas?

I've had need to pull out versions several years old multiple times in my
career, so this is the kind of thing I'm thinking about.

git config --global gc.reflogexpire            "10 years"'
git config --global gc.reflogexpireunreachable "10 years"

Makes me feel safer that the data will be in there, but even with the reflog
and access to the repository, I doubt I could FIND the place an old branch was
supposed to be if it was inadvertently deleted in a 2-million line source tree.
Am I just looking in the wrong places?

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  4:40 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: git
In-Reply-To: <alpine.LFD.1.10.0806232356340.2979@xanadu.home>

-- Nicolas Pitre wrote:
> I hope you'll feel much safer then.

I moved a branch around and then deleted it, and I don't see any record in the
reflog of where it was, or that it ever was.

Am I missing something about how branches are used? I see some language in "git
tag" about how attempts are made to assure that others can't move around
semi-immutable tags during push, but I don't see any such language about
branches. What prevents someone from accidentally deleting an old branch that
nobody is watching, but is important to the history and then not noticing as gc
silently deletes the old deltas?

I've had need to pull out versions several years old multiple times in my
career, so this is the kind of thing I'm thinking about.

git config --global gc.reflogexpire            "10 years"'
git config --global gc.reflogexpireunreachable "10 years"

Makes me feel safer that the data will be in there, but even with the reflog
and access to the repository, I doubt I could FIND the place an old branch was
supposed to be if it was inadvertently deleted in a 2-million line source tree.
Am I just looking in the wrong places?

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Teemu Likonen @ 2008-06-24  4:59 UTC (permalink / raw)
  To: David Jeske; +Cc: git

David Jeske wrote (2008-06-24 01:47 -0000):

> As a new user, I'm finding git difficult to trust, because there are
> operations which are destructive by default and capable of
> inadvertently throwing hours or days of work into the bit bucket.

I'm also quite new and I actually feel safe using git, and it's because
of reflog. No matter what I do (except manual reflog expire) I can see
where I was before with command

  git log --walk-reflogs

and get everything back. I have needed it a couple of times. So the
safety net is there, one just has to learn to trust it. :-)

Git is much safer than the standard Unix tools like rm, mv and cp.

^ permalink raw reply

* Re: apply --recount, was Re: [PATCH v4] git-add--interactive: manual hunk editing mode
From: Jeff King @ 2008-06-24  5:09 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Thomas Rast, git
In-Reply-To: <alpine.DEB.1.00.0806232204550.6440@racer>

On Mon, Jun 23, 2008 at 10:16:08PM +0100, Johannes Schindelin wrote:

> I think it was not a flaw, but something to be worried about: 
> 
> http://mid.gmane.org/7v4p87zcv6.fsf@gitster.siamese.dyndns.org
> 
> To spare you following that link: Junio wanted to reuse "git apply 
> --recount" to apply mboxes, where a separator "^-- $" to the signature is 
> quite common, and could be mistaken for a "-" line of a hunk.

Ah, OK. Thanks for the recap, that makes sense to me now.

> However, I think that this issue should not concern us _now_.  As long as 
> --recount is only to be used in "add -i" and "add -e", I think the patch 
> is good as is:

I agree. We can worry about the other when and if somebody decides to
use it that way (but maybe Junio is already planning to do so...).

Thomas, do you want to just re-submit the "--recount" patches when you
re-submit your patch?

-Peff

^ permalink raw reply

* Re: Importing non-version controlled bits and pieces to Git
From: Karl Hasselström @ 2008-06-24  5:12 UTC (permalink / raw)
  To: Peter Karlsson; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0806231543580.31319@ds9.cixit.se>

On 2008-06-23 15:46:11 +0100, Peter Karlsson wrote:

> Peter Karlsson:
>
> > That might work. My problem is the non-linearity of the data I
> > want to import. But I might be able to massage the import-tars
> > output before I feed it to git-fast-import to describe the history
> > I need it to.
>
> I ended up making a script that converted a description file that
> described the hierarchy of versions and generated a fast-import file
> from it. Good to have so that I could move stuff around a bit to
> make up a version tree that looked somewhat reasonable (a lot of
> cross-directory copying has been going on here, and no-one knows
> which version is which).

A bit too late now, but ... another thing you could have done is make
a straight linear import à la import-tars, and then modified the
parentage with grafts until it looked good; and then finally use
filter-branch to make the grafts part of the "real" history.

The advantage of this approach is that you can edit the parentage
almost interactively, which should be the best approach given that you
have a small number of commits and have to guess their relationships.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Avery Pennarun @ 2008-06-24  5:20 UTC (permalink / raw)
  To: David Jeske; +Cc: Nicolas Pitre, git
In-Reply-To: <1978205964779154253@unknownmsgid>

On 6/24/08, David Jeske <jeske@google.com> wrote:
> I moved a branch around and then deleted it, and I don't see any record in the
>  reflog of where it was, or that it ever was.
>
>  Am I missing something about how branches are used? I see some language in "git
>  tag" about how attempts are made to assure that others can't move around
>  semi-immutable tags during push, but I don't see any such language about
>  branches. What prevents someone from accidentally deleting an old branch that
>  nobody is watching, but is important to the history and then not noticing as gc
>  silently deletes the old deltas?
>
>  I've had need to pull out versions several years old multiple times in my
>  career, so this is the kind of thing I'm thinking about.

git branches are actually a very different concept from branches in,
say, subversion.

In subversion, a branch is normally created so that you can do
parallel development, and then you merge whole batches of changes
(with 'svn merge') from one branch into another.  When you do this,
you create a single new commit in the destination branch that contains
*all* the changes.  So if you want to look back in history to see who
did which part of the change for what reason, you have to go back to
the branch you merged *from*.  Thus, it's very important in subversion
that old branches never disappear.

git's philosophy is different.  Branches are really just "temporary
tags".  A merge operation doesn't just copy data from one branch to
another: it actually joins the two histories together, so you can then
trace back through the exact history of the merged branches, commit by
commit.  "git log" will show each checkin to *either* branch
individually, instead of just one big "merge" checkin.

The end result is that even if you delete the source branch after
doing a merge, nothing is actually lost.  Thus, there's no reason for
git to try to make branches impossible to lose, as they are in svn.
In the event that you really needed that branch pointer, it's in the
reflog, as a few people have pointed out.

Another way to think of it is that svn's concept of a "branch" is
actually the "reflog" in git.  (svn records which data a particular
branch name points to over time, just like git's reflog does.)  git
branches are something else entirely; a git branch always points at
only a single commit, and has no history of its own.

Does that help?  Perhaps it only confuses the issue :)

Have fun,

Avery

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Jan Krüger @ 2008-06-24  5:24 UTC (permalink / raw)
  To: David Jeske; +Cc: git
In-Reply-To: <willow-jeske-01l5g9o4FEDjCXMB>

Hi David,

"David Jeske" <jeske@google.com> wrote:
> I moved a branch around and then deleted it, and I don't see any
> record in the reflog of where it was, or that it ever was.

If a branch you're trying to delete is not part (or, more correctly,
an ancestor) of your current branch, you'll get a warning that you have
to explicitly bypass by using -D rather than -d.

Still, after deleting the branch, its old tip will very likely show up
in the reflog for HEAD (at the point you last worked on the branch),
even if the branch name won't show up anywhere. After locating the
commit in there it's a simple case of git checkout -b whatever
HEAD@{123} to get back that branch.

> What prevents someone from accidentally deleting an old branch that
> nobody is watching, but is important to the history and then not
> noticing as gc silently deletes the old deltas?

One thing to keep in mind is that deleting your branch locally won't
rid you of remote copies of it, so anything that you considered worth
sharing will probably survive even if you accidentally bypassed Git's
warning about deleting branches.

Best,
Jan

^ permalink raw reply

* Re: [RFC] Re: Convert 'git blame' to parse_options()
From: Jeff King @ 2008-06-24  5:35 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Johannes Schindelin, Pierre Habouzit, Git Mailing List,
	Junio C Hamano
In-Reply-To: <alpine.LFD.1.10.0806231307070.2926@woody.linux-foundation.org>

On Mon, Jun 23, 2008 at 01:12:28PM -0700, Linus Torvalds wrote:

> > in your options table. And if setup_revisions takes options that affect
> > things that _aren't_ in that struct, then they probably ought to be.
> 
> The world isn't like "it ought to be". It is like it is.
> 
> If you keep on dreaming about how things "ought to be", you'll never 
> confront the things as they are. The fact is, turning the existing very 
> simple argument parsers into using "parse_options()" is just _hard_.
> 
> Then you say that it all "ought to be" in those options already.

Did you even read my mail? All I said "ought to be" is that revision
options "ought to be" affecting things in the struct. As in, when I
convert this to parse_options, if there are any cases which don't fall
into that pattern then I "ought to be" able to convert them as part of
the code cleanup. And looking over the revision parsing code, it looks
like everything falls into that case already.

So if you have something FACTUAL to say that contradicts that point, by
all means do it. You might save me some work if you find a showstopper
in my approach.

> Sure. Do we have some invisible sky wizard to make it so?

Well, given that I already said I would work on it, I guess that makes
me the sky wizard. Bow down before my wizardly might!

That being said, it looks like Pierre is working on a slightly different
approach, and I am happy to sit back and see how that pans out for now.
So perhaps he is the sky wizard.

> I tried to do just _one_ program. Trust me, I'm not going to even bother 
> trying to do another unless parse_options() is made more palatable to do 
> in small pieces. 

Maybe we are fundamentally talking across each other. I'm not talking
about some specifics for converting a bunch of programs, or some
problems with parse_options. I'm suggesting that the main "two stage"
option parsing generally has the revision parsing as the "second stage".
Converting that one thing to parse_options will help with converting
many programs (e.g., a failed conversion of shortlog that others and I
worked on).

You seem to have a bunch of _other_ problems with parse_options. And
that is fine, but they have nothing whatsoever to do with anything I've
said. So don't "sky wizard" _me_ about those problems. ;P

-Peff

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Teemu Likonen @ 2008-06-24  5:43 UTC (permalink / raw)
  To: David Jeske; +Cc: git
In-Reply-To: <e80d075a0806232201o3933d154he2b570986604c30a@mail.gmail.com>

David Jeske wrote (2008-06-23 22:01 -0700):

> On Mon, Jun 23, 2008 at 9:59 PM, Teemu Likonen <tlikonen@iki.fi>
> wrote:
> 
> > I'm also quite new and I actually feel safe using git, and it's
> > because of reflog. No matter what I do (except manual reflog expire)
> > I can see where I was before with command
> >
> >  git log --walk-reflogs
> 
> 
> Perhaps I'm misunderstanding how to read it, but how do you tell where
> a branch was from the reflog if you inadvertantly moved it?

Perhaps I'm misunderstanding what you mean but I try to explain. In git
branches are nothing but named pointers to certain commit. If you "move
a branch" you actually rename the pointer, nothing more. With command

  git log --walk-reflogs --all

you can see everything in your reflog. When branches are moved (i.e.,
renamed) in "git log --walk-reflogs" output it shows like this:

  commit 269f10bca2273c1cf77831d5e23c6e0361217697
  Reflog: refs/heads/master@{2008-06-24 08:15:56 +0300} (Teemu Likonen <tlikonen@iki.fi>)
  Reflog message: Branch: renamed refs/heads/master to refs/heads/testbranch
  Author: Teemu Likonen <tlikonen@iki.fi>
  Date:   2008-03-25 19:10:40 +0200
  
  [commit message]

See the "Reflog message" field above. It tells what happened. The
"Reflog" field tells when it happened. If I later remove this
"testbranch" I can browse the reflog and create this branch (i.e.
a pointer) again with command "git branch testbranch 269f10bca". The
269f10bca comes from the commit ID of the above log item.

^ permalink raw reply

* [PATCH] clone: create intermediate directories of destination repo
From: Jeff King @ 2008-06-24  5:50 UTC (permalink / raw)
  To: Brandon Casey; +Cc: Junio C Hamano, Daniel Barkalow, zuh, git
In-Reply-To: <0YbSdnKH0_SP30-YMV0Y1aH4SO46yc6x_2EA1VGwPtV_CltGk-7wdg@cipher.nrlssc.navy.mil>

The shell version used to use "mkdir -p" to create the repo
path, but the C version just calls "mkdir". Let's replicate
the old behavior. In this case we can simply create the
directories leading up to the git dir. If it's a bare repo,
then that is everything that init_db wants ahead of time. If
it isn't bare, then the worktree contains the git dir, so we
create the worktree.

We can reuse safe_create_leading_directories, but we need to
make a copy of our const buffer to do so. Since
merge-recursive uses the same pattern, we can factor this
out into a global function. This has two other cleanup
advantages for merge-recursive:

  1. mkdir_p wasn't a very good name. "mkdir -p foo/bar" actually
     creates bar, but this function just creates the leading
     directories.

  2. mkdir_p took a mode argument, but it was completely
     ignored.
---
On Mon, Jun 23, 2008 at 04:03:47PM -0500, Brandon Casey wrote:

> There is a mkdir_p() in builtin-merge-recursive.c which calls
> safe_create_leading_directories() in sha1_file.c
> 
> Can these functions be reused?

Thanks, I hadn't noticed those for some reason. This version uses the
existing code, and correctly handles both the bare and worktree cases.

The big difference is that safe_create_leading_directories will do an
adjust_shared_perm on the result. I don't think that should be a
problem, but it is a difference.

 builtin-clone.c           |    5 +++--
 builtin-merge-recursive.c |   13 ++-----------
 cache.h                   |    1 +
 sha1_file.c               |    9 +++++++++
 t/t5601-clone.sh          |   14 ++++++++++++++
 5 files changed, 29 insertions(+), 13 deletions(-)

diff --git a/builtin-clone.c b/builtin-clone.c
index 7190952..e951911 100644
--- a/builtin-clone.c
+++ b/builtin-clone.c
@@ -398,10 +398,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 		git_dir = xstrdup(mkpath("%s/.git", dir));
 	}
 
+	if (safe_create_leading_directories_const(git_dir) < 0)
+		die("could not create leading directories of '%s'", git_dir);
+
 	if (!option_bare) {
 		junk_work_tree = work_tree;
-		if (mkdir(work_tree, 0755))
-			die("could not create work tree dir '%s'.", work_tree);
 		set_git_work_tree(work_tree);
 	}
 	junk_git_dir = git_dir;
diff --git a/builtin-merge-recursive.c b/builtin-merge-recursive.c
index 4aa28a1..43bf6aa 100644
--- a/builtin-merge-recursive.c
+++ b/builtin-merge-recursive.c
@@ -481,15 +481,6 @@ static char *unique_path(const char *path, const char *branch)
 	return newpath;
 }
 
-static int mkdir_p(const char *path, unsigned long mode)
-{
-	/* path points to cache entries, so xstrdup before messing with it */
-	char *buf = xstrdup(path);
-	int result = safe_create_leading_directories(buf);
-	free(buf);
-	return result;
-}
-
 static void flush_buffer(int fd, const char *buf, unsigned long size)
 {
 	while (size > 0) {
@@ -512,7 +503,7 @@ static int make_room_for_path(const char *path)
 	int status;
 	const char *msg = "failed to create path '%s'%s";
 
-	status = mkdir_p(path, 0777);
+	status = safe_create_leading_directories_const(path);
 	if (status) {
 		if (status == -3) {
 			/* something else exists */
@@ -583,7 +574,7 @@ static void update_file_flags(const unsigned char *sha,
 			close(fd);
 		} else if (S_ISLNK(mode)) {
 			char *lnk = xmemdupz(buf, size);
-			mkdir_p(path, 0777);
+			safe_create_leading_directories_const(path);
 			unlink(path);
 			symlink(lnk, path);
 			free(lnk);
diff --git a/cache.h b/cache.h
index d12ee7d..49f20dd 100644
--- a/cache.h
+++ b/cache.h
@@ -517,6 +517,7 @@ enum sharedrepo {
 int git_config_perm(const char *var, const char *value);
 int adjust_shared_perm(const char *path);
 int safe_create_leading_directories(char *path);
+int safe_create_leading_directories_const(const char *path);
 char *enter_repo(char *path, int strict);
 static inline int is_absolute_path(const char *path)
 {
diff --git a/sha1_file.c b/sha1_file.c
index 191f814..71be0c9 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -116,6 +116,15 @@ int safe_create_leading_directories(char *path)
 	return 0;
 }
 
+int safe_create_leading_directories_const(const char *path)
+{
+	/* path points to cache entries, so xstrdup before messing with it */
+	char *buf = xstrdup(path);
+	int result = safe_create_leading_directories(buf);
+	free(buf);
+	return result;
+}
+
 char *sha1_to_hex(const unsigned char *sha1)
 {
 	static int bufno;
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index 593d1a3..c2c83f0 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -30,4 +30,18 @@ test_expect_success 'clone checks out files' '
 
 '
 
+test_expect_success 'clone creates intermediate directories (non-bare)' '
+
+	git clone src long/path/to/dst &&
+	test -f long/path/to/dst/file
+
+'
+
+test_expect_success 'clone creates intermediate directories (bare)' '
+
+	git clone --bare src long/path/to/bare/dst &&
+	test -f long/path/to/bare/dst/config
+
+'
+
 test_done
-- 
1.5.6.50.gae760.dirty

^ permalink raw reply related

* Re: Code Formatting vs Trailing Whitespaces
From: Robin Rosenberg @ 2008-06-24  6:01 UTC (permalink / raw)
  To: Florian Köberle; +Cc: git, Shawn O. Pearce, Marek Zawirski
In-Reply-To: <486012F2.6070006@web.de>

måndagen den 23 juni 2008 23.17.38 skrev Florian Köberle:
> Hi
> 
> | This one matches: https://bugs.eclipse.org/bugs/show_bug.cgi?id=49619
> Thank you. I voted for that feature, maybe it helps to draw attention to
> this feature wish.
> 
> | When I have both options active, it adds the whitespace when
> formatting, but
> | removes it when saving in (version 3.3.2).
> Strange, I use version 3.3.2 too. I configured it in the workspace settings.
> Maybe you did in the project settings?

Indeed.

-- robin

^ permalink raw reply

* [PATCH] help: check early if we have a command, if not try a documentation topic
From: Christian Couder @ 2008-06-24  6:46 UTC (permalink / raw)
  To: Junio Hamano, Pieter de Bie, Jakub Narebski; +Cc: git

Before this patch, something like "git help tutorial" did not work,
people had to use "git help gittutorial" which is not very intuitive.

This patch uses the "is_git_command" function to test early if the
argument passed to "git help" is a git command, and if this is not the
case then we prefix the argument with "git" instead of "git-".

This way, things like "git help tutorial" or "git help glossary" will
work fine.

The little downside of this patch is that the "is_git_command" is a
little bit slow.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 help.c |   23 +++++++++++++++--------
 1 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/help.c b/help.c
index 8aff94c..ef2e6b7 100644
--- a/help.c
+++ b/help.c
@@ -527,20 +527,27 @@ static int is_git_command(const char *s)
 		is_in_cmdlist(&other_cmds, s);
 }
 
+static const char *add_prefix(const char *prefix, const char *cmd)
+{
+	size_t pre_len = strlen(prefix);
+	size_t cmd_len = strlen(cmd);
+	char *p = xmalloc(pre_len + cmd_len + 1);
+	strcpy(p, prefix);
+	strcpy(p + pre_len, cmd);
+	p[pre_len + cmd_len] = 0;
+	return p;
+}
+
 static const char *cmd_to_page(const char *git_cmd)
 {
 	if (!git_cmd)
 		return "git";
 	else if (!prefixcmp(git_cmd, "git"))
 		return git_cmd;
-	else {
-		int page_len = strlen(git_cmd) + 4;
-		char *p = xmalloc(page_len + 1);
-		strcpy(p, "git-");
-		strcpy(p + 4, git_cmd);
-		p[page_len] = 0;
-		return p;
-	}
+	else if (is_git_command(git_cmd))
+		return add_prefix("git-", git_cmd);
+	else
+		return add_prefix("git", git_cmd);
 }
 
 static void setup_man_path(void)
-- 
1.5.6.109.g4f927.dirty

^ permalink raw reply related

* Re: Importing non-version controlled bits and pieces to Git
From: Peter Karlsson @ 2008-06-24  6:46 UTC (permalink / raw)
  To: Karl Hasselström; +Cc: Git Mailing List
In-Reply-To: <20080624051204.GA1760@diana.vm.bytemark.co.uk>

Karl Hasselström:

> A bit too late now, but ...

Not really, I'm on to the next project to import :-)

> another thing you could have done is make a straight linear import à
> la import-tars, and then modified the parentage with grafts until it
> looked good; and then finally use filter-branch to make the grafts
> part of the "real" history.

Yeah, I was considering this approach, but then I was half-way
designing the script in my head, plus that the import is so quick that
it was easy enough to re-write the configuration file and re-import to
try something else. I love that aspect of Git -- the preparing of the
file to import takes longer than the actual git-fast-import call...

> The advantage of this approach is that you can edit the parentage
> almost interactively, which should be the best approach given that
> you have a small number of commits and have to guess their
> relationships.

"Almost" being the magic word :-) I achieved more or less the same by
re-importing. Plus that the job of preparing tarballs of everything
would have been some work as well, since I had to choose which files to
import from each directory (and do CRLF transform).

-- 
\\// Peter - http://www.softwolves.pp.se/

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  6:35 UTC (permalink / raw)
  To: Avery Pennarun; +Cc: Nicolas Pitre, git
In-Reply-To: <32541b130806232220r292d691cn5bf5f9976126aa29@mail.gmail.com>

Thanks for all the helpful responses...

-- Avery Pennarun wrote:
> git's philosophy is different. Branches are really just "temporary
> tags". A merge operation doesn't just copy data from one branch to
> another: it actually joins the two histories together, so you can then
> trace back through the exact history of the merged branches, commit by
> commit. "git log" will show each checkin to *either* branch
> individually, instead of just one big "merge" checkin.

If branches are "temporary tags" how do I see the actual code they had working
in their branch before they merged it?

I'm reading about rebase, and it sounds like something I would want to forever
disallow on my git repository, because it looks like it rewrites history and
makes it impossible to get to the state of the tree they actually had working
before the merge. However, something you say below both clarifies and confuses
this.

Am I understanding this wrong?

> The end result is that even if you delete the source branch after
> doing a merge, nothing is actually lost.

..and what if you never merge? That branch-pointer points to useful information
about a development attempt, but it was never merged. (imagine a different
development path was taken) They never created a tag because it's not clear
when that work was "done" (unlike a release, which is much more well
understood). What prevents someone from deleting the branch-pointer or moving
it to a different part of the tree, causing that set of changes to be a
dangling ref lost in a sea of refs. Later when someone goes back looking for
it, how would they ever find it in a sea of tens of thousands of checkins?

> Thus, there's no reason for git to try to make branches impossible
> to lose, as they are in svn.

Before I set the GC times to "100 years", there was a HUGE reason for git to
make those branch-pointers impossible to lose, because by default if you lose
them git actually garbage collects them and throws the diffs away after 90
days!

> Another way to think of it is that svn's concept of a "branch" is
> actually the "reflog" in git. (svn records which data a particular
> branch name points to over time, just like git's reflog does.) git
> branches are something else entirely; a git branch always points at
> only a single commit, and has no history of its own.

That's sort of helpful, and sort of confusing. I think of git's branches as
"branch pointers to the head of a linked-list of states of the tree".

As long as you keep those refs without deleting them, and you keep that branch
pointer to the head, you can walk back through the history of that branch. If
multiple developers are working in the branch (and not using rebase, and not
garbage collecting), can't you even go track down the working state of their
local clients while they were working before they merged?

If I'm understanding all that right, it's exactly the kind of functionality I
want -- the ability to reproduce the state of all working history, exactly as
it was when the code was actually working in someone's client a long time ago,
before they merged it to the mainline. Except the standard model seems to be to
let the system "garbage collect" all that history, and toss it away as
unimportant -- and in some cases it seems to even provide developers with ways
to more aggressively assure garbage collection makes it disappear.

Am I expecting too much out of git? It doesn't really feel like a source
control system for an organization that wants to save everything, forever, even
when those people and trees and home directories disappear. It feels like a
distributed patch manager that is much more automatic than sending around
diffs, but isn't overly concerned with providing access to old history. (which,
duh, is no surprise given that's what I expect it's doing for linux kernel)

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: David Jeske @ 2008-06-24  6:35 UTC (permalink / raw)
  To: Avery Pennarun; +Cc: Nicolas Pitre, git
In-Reply-To: <32541b130806232220r292d691cn5bf5f9976126aa29@mail.gmail.com>

Thanks for all the helpful responses...

-- Avery Pennarun wrote:
> git's philosophy is different. Branches are really just "temporary
> tags". A merge operation doesn't just copy data from one branch to
> another: it actually joins the two histories together, so you can then
> trace back through the exact history of the merged branches, commit by
> commit. "git log" will show each checkin to *either* branch
> individually, instead of just one big "merge" checkin.

If branches are "temporary tags" how do I see the actual code they had working
in their branch before they merged it?

I'm reading about rebase, and it sounds like something I would want to forever
disallow on my git repository, because it looks like it rewrites history and
makes it impossible to get to the state of the tree they actually had working
before the merge. However, something you say below both clarifies and confuses
this.

Am I understanding this wrong?

> The end result is that even if you delete the source branch after
> doing a merge, nothing is actually lost.

..and what if you never merge? That branch-pointer points to useful information
about a development attempt, but it was never merged. (imagine a different
development path was taken) They never created a tag because it's not clear
when that work was "done" (unlike a release, which is much more well
understood). What prevents someone from deleting the branch-pointer or moving
it to a different part of the tree, causing that set of changes to be a
dangling ref lost in a sea of refs. Later when someone goes back looking for
it, how would they ever find it in a sea of tens of thousands of checkins?

> Thus, there's no reason for git to try to make branches impossible
> to lose, as they are in svn.

Before I set the GC times to "100 years", there was a HUGE reason for git to
make those branch-pointers impossible to lose, because by default if you lose
them git actually garbage collects them and throws the diffs away after 90
days!

> Another way to think of it is that svn's concept of a "branch" is
> actually the "reflog" in git. (svn records which data a particular
> branch name points to over time, just like git's reflog does.) git
> branches are something else entirely; a git branch always points at
> only a single commit, and has no history of its own.

That's sort of helpful, and sort of confusing. I think of git's branches as
"branch pointers to the head of a linked-list of states of the tree".

As long as you keep those refs without deleting them, and you keep that branch
pointer to the head, you can walk back through the history of that branch. If
multiple developers are working in the branch (and not using rebase, and not
garbage collecting), can't you even go track down the working state of their
local clients while they were working before they merged?

If I'm understanding all that right, it's exactly the kind of functionality I
want -- the ability to reproduce the state of all working history, exactly as
it was when the code was actually working in someone's client a long time ago,
before they merged it to the mainline. Except the standard model seems to be to
let the system "garbage collect" all that history, and toss it away as
unimportant -- and in some cases it seems to even provide developers with ways
to more aggressively assure garbage collection makes it disappear.

Am I expecting too much out of git? It doesn't really feel like a source
control system for an organization that wants to save everything, forever, even
when those people and trees and home directories disappear. It feels like a
distributed patch manager that is much more automatic than sending around
diffs, but isn't overly concerned with providing access to old history. (which,
duh, is no surprise given that's what I expect it's doing for linux kernel)

^ permalink raw reply

* Re: why is git destructive by default? (i suggest it not be!)
From: Jeff King @ 2008-06-24  7:24 UTC (permalink / raw)
  To: David Jeske; +Cc: Avery Pennarun, Nicolas Pitre, git
In-Reply-To: <willow-jeske-01l5izRzFEDjCdyL>

On Tue, Jun 24, 2008 at 06:35:16AM -0000, David Jeske wrote:

> If branches are "temporary tags" how do I see the actual code they had
> working in their branch before they merged it?

You look at the shape of the history. But if it is really an important
event for you to say "this was the state right before some merge of
interest", then by all means, tag it with a real tag. Or don't delete
the branch.

Have you tried running gitk on the kernel or git repositories?

> I'm reading about rebase, and it sounds like something I would want to
> forever disallow on my git repository, because it looks like it
> rewrites history and makes it impossible to get to the state of the
> tree they actually had working before the merge. However, something
> you say below both clarifies and confuses this.

It does throw away the state before the rebase (well, there is no longer
a pointer to it; it is still recoverable via the reflog). But for most
push/pull collaboration, you probably want to be using merge. Rebase is
more useful for people who are more accustomed to a patch-based
workflow.

> > The end result is that even if you delete the source branch after
> > doing a merge, nothing is actually lost.
> 
> ..and what if you never merge? That branch-pointer points to useful
> information about a development attempt, but it was never merged.
> (imagine a different development path was taken) They never created a
> tag because it's not clear when that work was "done" (unlike a
> release, which is much more well understood). What prevents someone
> from deleting the branch-pointer or moving it to a different part of
> the tree, causing that set of changes to be a dangling ref lost in a
> sea of refs. Later when someone goes back looking for it, how would
> they ever find it in a sea of tens of thousands of checkins?

If it's not merged, then don't delete the branch pointer! And "git
branch -d" will even refuse to do the deletion, unless you force it with
"git branch -D".

And keep in mind that when you clone repos, you clone the branch
pointer. So if you have a centralized server that your developers push
and pull from, a stray "git branch -D" from one developer _doesn't_ ruin
it for the rest of them. All that does is delete the branch from their
local repo, but it still exists in the central repo and for all of the
other developers. But it's not clear to me what sort of developer
topology you're interested in.

> Before I set the GC times to "100 years", there was a HUGE reason for git to
> make those branch-pointers impossible to lose, because by default if you lose
> them git actually garbage collects them and throws the diffs away after 90
> days!

I think most people are comfortable with "if I have an unmerged branch,
it stays forever. If I accidentally delete my branch, I have 30 days to
pull the tip out of my reflog". Sure, it's _possible_ to lose work. But
you could also accidentally "rm -rf" your .git directory. If you want an
extra layer of protection, push your work periodically to a backup repo.

> That's sort of helpful, and sort of confusing. I think of git's branches as
> "branch pointers to the head of a linked-list of states of the tree".

More or less true (they aren't linked-list, but arbitrary DAGs --
commits can have more than one parent (i.e., a merge) and can have many
children (i.e., many people build off in different directions from one
spot)).

> If I'm understanding all that right, it's exactly the kind of
> functionality I want -- the ability to reproduce the state of all
> working history, exactly as it was when the code was actually working
> in someone's client a long time ago, before they merged it to the
> mainline. Except the standard model seems to be to let the system
> "garbage collect" all that history, and toss it away as unimportant --
> and in some cases it seems to even provide developers with ways to
> more aggressively assure garbage collection makes it disappear.

I think you are confusing two aspects of history.

There is the commit DAG, which says "at some time T, the files were at
some state S, and the commit message by author A was M". And those
commits form a chain so you can see how the state of the files
progressed. And anything that is reachable through that history will
always be kept by git, and you can always go back to any point.

But we also give particular names to some points, like "this is tag
v1.0" or "this is the head of the experimental line of development". We
call those refs.  Git remembers those names until you ask it not to (by
deleting the ref).  And there is a history to those names, like
"experimental was at some commit C1. Then somebody committed and it was
at C2. And then they did a git-reset and it was at C3". And that history
is encapsulated in the reflog, and is purely local to each repository
(since git is distributed, it makes no sense to talk about "where the
experimental name pointed" without talking about a specific repo).

And the ref history is what gets garbage collected. Most people are fine
with that, because they care about the actual commit history, and the
reflog is just a convenient way of saying "oops, what was happening
yesterday?" But if you really care, then by all means, set the reflog
expiration much higher.

> Am I expecting too much out of git? It doesn't really feel like a
> source control system for an organization that wants to save
> everything, forever, even when those people and trees and home
> directories disappear. It feels like a distributed patch manager that
> is much more automatic than sending around diffs, but isn't overly
> concerned with providing access to old history. (which, duh, is no
> surprise given that's what I expect it's doing for linux kernel)

Git _will_ remember content forever, _if_ you put into git. So if you
are saying "git won't remember work that employee X did after he is
gone", that isn't true. X's work will be part of the commit DAG and will
be a part of everybody's repo. If you are saying "I blew away employee
X's home directory, and he had a git repo in it, why didn't git save
that data?" then the problem is that you deleted the repo! If you are
concerned about that situation, have employee X push his work to a repo
that doesn't get deleted.

-Peff

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox