* archimport documentation updates and tidyups @ 2005-09-11 9:26 martin 2005-09-11 9:26 ` [PATCH 1/5] archimport documentation update martin 0 siblings, 1 reply; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git This patch series covers the Documentation for the archimport script, plus the remaining TODOs that I consider important. With these patches I consider archimport done, modulo bugs and interesting user requests. [PATCH 1/5] archimport documentation update [PATCH 2/5] archimport documentation tidyup [PATCH 3/5] archimport - update in-script doco, options tidyup [PATCH 4/5] archimport - use GIT_DIR instead of hardcoded ".git" [PATCH 5/5] archimport - better handling of temp dirs ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/5] archimport documentation update 2005-09-11 9:26 archimport documentation updates and tidyups martin @ 2005-09-11 9:26 ` martin 2005-09-11 9:26 ` [PATCH 2/5] archimport documentation tidyup martin 2005-09-11 14:58 ` [PATCH 1/5] archimport documentation update A Large Angry SCM 0 siblings, 2 replies; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git; +Cc: Martin Langhoff Updated and expanded the command description, and added a reference of the command line options. Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> --- Documentation/git-archimport.txt | 63 +++++++++++++++++++++++++++++++------- 1 files changed, 51 insertions(+), 12 deletions(-) b3ee96f94c9ec4378d18a13074d1bbeebb0858b5 diff --git a/Documentation/git-archimport.txt b/Documentation/git-archimport.txt --- a/Documentation/git-archimport.txt +++ b/Documentation/git-archimport.txt @@ -8,24 +8,63 @@ git-archimport - Import an arch reposito SYNOPSIS -------- -`git-archimport` [--option...] <args> +`git-archimport` [ -h ] [ -v ] [ -T ] [ -t tempdir ] + <archive/branch> [ <archive/branch> ] DESCRIPTION ----------- -The script expects you to provide the key roots where it can start the -import from an 'initial import' or 'tag' type of Arch commit. It will -then follow all the branching and tagging within the provided roots. - -It will die if it sees branches that have different roots. - +Imports a project from one or more Arch repositories. It will follow branches +and repositories within the namespaces defined by the <archive/branch> +parameters suppplied. If it cannot find the remote branch a merge comes from +it will just import it as a regular merge. If it can find it, it will perform +a merge whenever possible. + +The script expects you to provide the key roots where it can start the import +from an 'initial import' or 'tag' type of Arch commit. It will follow and import +new branches within the provided roots. + +It expects to be dealing with one project only. If it sees +branches that have different roots, it will refuse to run. In that case, edit your +<archive/branch> parameters to define clearly the scope of the import. + +`git-archimport` uses `tla` extensively in the background to access the Arch repository. +Make sure you have a recent version of `tla` available in the path. `tla` must +know about the repositories you pass to `git-archimport`. + +For the initial import `git-archimport` expects to find itself in an empty +directory. To follow the development of a project that uses Arch, rerun +`git-archimport` with the same parameters as the initial import to perform incremental imports. +of a project using Arch. + +Patch merge data from Arch is used to mark merges in GIT as well. GIT +does not care much about tracking patches, and only considers a merge when a +branch incorporates all the commits since the point they forked. The end result +is that GIT will have a good idea of how far branches have diverged. So the +import process does lose some patch-trading metadata. + +Fortunately, when you try and merge branches imported from Arch, +GIT will find a good merge base, and it has a good chance of identifying +patches that have been traded out-of-sequence between the branches. OPTIONS ------- ---option:: - Some option not yet documented. -<args>...:: - Some argument not yet documented. +-h:: + Display usage. + +-v:: + Verbose output. + +-T:: + Many tags. Will create a tag for every commit, reflecting the commit + name in the Arch repository. + +-t <tmpdir>:: + Override the default tempdir. + + +<archive/branch>:: + Archive/branch identifier in a format that `tla log` understands. Author @@ -34,7 +73,7 @@ Written by Martin Langhoff <martin@catal Documentation -------------- -Documentation by Junio C Hamano and the git-list <git@vger.kernel.org>. +Documentation by Junio C Hamano, Martin Langhoff and the git-list <git@vger.kernel.org>. GIT --- ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/5] archimport documentation tidyup 2005-09-11 9:26 ` [PATCH 1/5] archimport documentation update martin @ 2005-09-11 9:26 ` martin 2005-09-11 9:26 ` [PATCH 3/5] archimport - update in-script doco, options tidyup martin 2005-09-11 14:58 ` [PATCH 1/5] archimport documentation update A Large Angry SCM 1 sibling, 1 reply; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git; +Cc: Martin Langhoff New "merges" headline, clarified some parts that were not easy to understand. Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> --- Documentation/git-archimport.txt | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) da341e8530b0d852c9b6910778df6b41cdee3e54 diff --git a/Documentation/git-archimport.txt b/Documentation/git-archimport.txt --- a/Documentation/git-archimport.txt +++ b/Documentation/git-archimport.txt @@ -3,7 +3,7 @@ git-archimport(1) NAME ---- -git-archimport - Import an arch repository into git +git-archimport - Import an Arch repository into GIT SYNOPSIS @@ -16,8 +16,8 @@ DESCRIPTION Imports a project from one or more Arch repositories. It will follow branches and repositories within the namespaces defined by the <archive/branch> parameters suppplied. If it cannot find the remote branch a merge comes from -it will just import it as a regular merge. If it can find it, it will perform -a merge whenever possible. +it will just import it as a regular commit. If it can find it, it will mark it +as a merge whenever possible (see discussion below). The script expects you to provide the key roots where it can start the import from an 'initial import' or 'tag' type of Arch commit. It will follow and import @@ -34,8 +34,9 @@ know about the repositories you pass to For the initial import `git-archimport` expects to find itself in an empty directory. To follow the development of a project that uses Arch, rerun `git-archimport` with the same parameters as the initial import to perform incremental imports. -of a project using Arch. +MERGES +------ Patch merge data from Arch is used to mark merges in GIT as well. GIT does not care much about tracking patches, and only considers a merge when a branch incorporates all the commits since the point they forked. The end result ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/5] archimport - update in-script doco, options tidyup 2005-09-11 9:26 ` [PATCH 2/5] archimport documentation tidyup martin @ 2005-09-11 9:26 ` martin 2005-09-11 9:26 ` [PATCH 4/5] archimport - use GIT_DIR instead of hardcoded ".git" martin 0 siblings, 1 reply; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git; +Cc: Martin Langhoff Updated the usage/help message to match asciidoc documentation. The perldoc documentation now includes the first paragraph from the asciidoc documentation and points users to the manpage. Updated TODO section. Removed some redundant options from the getopt() invocation. Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> --- git-archimport.perl | 33 ++++++++++++++++----------------- 1 files changed, 16 insertions(+), 17 deletions(-) 8d625fb42f58a6b4b2af135efe76726e8f0579e0 diff --git a/git-archimport.perl b/git-archimport.perl --- a/git-archimport.perl +++ b/git-archimport.perl @@ -6,23 +6,27 @@ # The basic idea is to walk the output of tla abrowse, # fetch the changesets and apply them. # + =head1 Invocation - git-archimport -i <archive>/<branch> [<archive>/<branch>] - [ <archive>/<branch> ] + git-archimport [ -h ] [ -v ] [ -T ] [ -t tempdir ] <archive>/<branch> [ <archive>/<branch> ] - The script expects you to provide the key roots where it can start the - import from an 'initial import' or 'tag' type of Arch commit. It will - then follow all the branching and tagging within the provided roots. +Imports a project from one or more Arch repositories. It will follow branches +and repositories within the namespaces defined by the <archive/branch> +parameters suppplied. If it cannot find the remote branch a merge comes from +it will just import it as a regular commit. If it can find it, it will mark it +as a merge whenever possible. - It will die if it sees branches that have different roots. +See man (1) git-archimport for more details. -=head2 TODO +=head1 TODO - - keep track of merged patches, and mark a git merge when it happens - - smarter rules to parse the archive history "up" and "down" - - be able to continue an import where we left off + - create tag objects instead of ref tags - audit shell-escaping of filenames + - better handling of temp directories + - use GIT_DIR instead of hardcoded ".git" + - hide our private tags somewhere smarter + - find a way to make "cat *patches | patch" safe even when patchfiles are missing newlines =head1 Devel tricks @@ -54,14 +58,13 @@ our($opt_h,$opt_v, $opt_T, sub usage() { print STDERR <<END; Usage: ${\basename $0} # fetch/update GIT from Arch - [ -h ] [ -v ] [ -T ] - [ -C GIT_repository ] [ -t tempdir ] + [ -h ] [ -v ] [ -T ] [ -t tempdir ] repository/arch-branch [ repository/arch-branch] ... END exit(1); } -getopts("ThviC:t:") or usage(); +getopts("Thvt:") or usage(); usage if $opt_h; @ARGV >= 1 or usage(); @@ -71,10 +74,6 @@ my $tmp = $opt_t; $tmp ||= '/tmp'; $tmp .= '/git-archimport/'; -my $git_tree = $opt_C; -$git_tree ||= "."; - - my @psets = (); # the collection my %psets = (); # the collection, by name ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 4/5] archimport - use GIT_DIR instead of hardcoded ".git" 2005-09-11 9:26 ` [PATCH 3/5] archimport - update in-script doco, options tidyup martin @ 2005-09-11 9:26 ` martin 2005-09-11 9:26 ` [PATCH 5/5] archimport - better handling of temp dirs martin 0 siblings, 1 reply; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git; +Cc: Martin Langhoff Use GIT_DIR from the environment instead of a hardcoded '.git' string. Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> --- git-archimport.perl | 42 ++++++++++++++++++++++-------------------- 1 files changed, 22 insertions(+), 20 deletions(-) 600258cfefc38939f09f267090ba400d80ab14c3 diff --git a/git-archimport.perl b/git-archimport.perl --- a/git-archimport.perl +++ b/git-archimport.perl @@ -24,7 +24,6 @@ See man (1) git-archimport for more deta - create tag objects instead of ref tags - audit shell-escaping of filenames - better handling of temp directories - - use GIT_DIR instead of hardcoded ".git" - hide our private tags somewhere smarter - find a way to make "cat *patches | patch" safe even when patchfiles are missing newlines @@ -52,6 +51,9 @@ use IPC::Open2; $SIG{'PIPE'}="IGNORE"; $ENV{'TZ'}="UTC"; +my $git_dir = $ENV{"GIT_DIR"} || ".git"; +$ENV{"GIT_DIR"} = $git_dir; + our($opt_h,$opt_v, $opt_T, $opt_C,$opt_t); @@ -179,7 +181,7 @@ foreach my $root (@arch_roots) { ## and put an initial import ## or a full tag my $import = 0; -unless (-d '.git') { # initial import +unless (-d $git_dir) { # initial import if ($psets[0]{type} eq 'i' || $psets[0]{type} eq 't') { print "Starting import from $psets[0]{id}\n"; `git-init-db`; @@ -190,11 +192,11 @@ unless (-d '.git') { # initial import } } else { # progressing an import # load the rptags - opendir(DIR, ".git/archimport/tags") + opendir(DIR, "$git_dir/archimport/tags") || die "can't opendir: $!"; while (my $file = readdir(DIR)) { # skip non-interesting-files - next unless -f ".git/archimport/tags/$file"; + next unless -f "$git_dir/archimport/tags/$file"; next if $file =~ m/--base-0$/; # don't care for base-0 my $sha = ptag($file); chomp $sha; @@ -238,7 +240,7 @@ foreach my $ps (@psets) { } unless ($import) { # skip for import - if ( -e ".git/refs/heads/$ps->{branch}") { + if ( -e "$git_dir/refs/heads/$ps->{branch}") { # we know about this branch `git checkout $ps->{branch}`; } else { @@ -291,7 +293,7 @@ foreach my $ps (@psets) { # imports don't give us good info # on added files. Shame on them if ($ps->{type} eq 'i' || $ps->{type} eq 't') { - `find . -type f -print0 | grep -zv '^./.git' | xargs -0 -l100 git-update-index --add`; + `find . -type f -print0 | grep -zv '^./$git_dir' | xargs -0 -l100 git-update-index --add`; `git-ls-files --deleted -z | xargs --no-run-if-empty -0 -l100 git-update-index --remove`; } @@ -355,8 +357,8 @@ foreach my $ps (@psets) { # Who's your daddy? # my @par; - if ( -e ".git/refs/heads/$ps->{branch}") { - if (open HEAD, "<.git/refs/heads/$ps->{branch}") { + if ( -e "$git_dir/refs/heads/$ps->{branch}") { + if (open HEAD, "<$git_dir/refs/heads/$ps->{branch}") { my $p = <HEAD>; close HEAD; chomp $p; @@ -403,11 +405,11 @@ foreach my $ps (@psets) { # # Update the branch # - open HEAD, ">.git/refs/heads/$ps->{branch}"; + open HEAD, ">$git_dir/refs/heads/$ps->{branch}"; print HEAD $commitid; close HEAD; - unlink ('.git/HEAD'); - symlink("refs/heads/$ps->{branch}",".git/HEAD"); + unlink ("$git_dir/HEAD"); + symlink("refs/heads/$ps->{branch}","$git_dir/HEAD"); # tag accordingly ptag($ps->{id}, $commitid); # private tag @@ -436,7 +438,7 @@ sub apply_import { `tla get -s --no-pristine -A $ps->{repo} $ps->{id} $tmp/import`; die "Cannot get import: $!" if $?; - `rsync -v --archive --delete --exclude '.git' --exclude '.arch-ids' --exclude '{arch}' $tmp/import/* ./`; + `rsync -v --archive --delete --exclude '$git_dir' --exclude '.arch-ids' --exclude '{arch}' $tmp/import/* ./`; die "Cannot rsync import:$!" if $?; `rm -fr $tmp/import`; @@ -482,7 +484,7 @@ sub apply_cset { } # bring in new files - `rsync --archive --exclude '.git' --exclude '.arch-ids' --exclude '{arch}' $tmp/changeset/new-files-archive/* ./`; + `rsync --archive --exclude '$git_dir' --exclude '.arch-ids' --exclude '{arch}' $tmp/changeset/new-files-archive/* ./`; # deleted files are hinted from the commitlog processing @@ -577,7 +579,7 @@ sub tag { $tag = shell_quote($tag); if ($commit) { - open(C,">.git/refs/tags/$tag") + open(C,">$git_dir/refs/tags/$tag") or die "Cannot create tag $tag: $!\n"; print C "$commit\n" or die "Cannot write tag $tag: $!\n"; @@ -585,7 +587,7 @@ sub tag { or die "Cannot write tag $tag: $!\n"; print " * Created tag ' $tag' on '$commit'\n" if $opt_v; } else { # read - open(C,"<.git/refs/tags/$tag") + open(C,"<$git_dir/refs/tags/$tag") or die "Cannot read tag $tag: $!\n"; $commit = <C>; chomp $commit; @@ -603,12 +605,12 @@ sub ptag { $tag =~ s|/|--|g; $tag = shell_quote($tag); - unless (-d '.git/archimport/tags') { - mkpath('.git/archimport/tags'); + unless (-d "$git_dir/archimport/tags") { + mkpath("$git_dir/archimport/tags"); } if ($commit) { # write - open(C,">.git/archimport/tags/$tag") + open(C,">$git_dir/archimport/tags/$tag") or die "Cannot create tag $tag: $!\n"; print C "$commit\n" or die "Cannot write tag $tag: $!\n"; @@ -618,10 +620,10 @@ sub ptag { unless $tag =~ m/--base-0$/; } else { # read # if the tag isn't there, return 0 - unless ( -s ".git/archimport/tags/$tag") { + unless ( -s "$git_dir/archimport/tags/$tag") { return 0; } - open(C,"<.git/archimport/tags/$tag") + open(C,"<$git_dir/archimport/tags/$tag") or die "Cannot read tag $tag: $!\n"; $commit = <C>; chomp $commit; ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 5/5] archimport - better handling of temp dirs 2005-09-11 9:26 ` [PATCH 4/5] archimport - use GIT_DIR instead of hardcoded ".git" martin @ 2005-09-11 9:26 ` martin 0 siblings, 0 replies; 7+ messages in thread From: martin @ 2005-09-11 9:26 UTC (permalink / raw) To: git; +Cc: Martin Langhoff Switched from backwards hard-coded tmp directory creation to using File::Temp::tempdir() to create the directory inside $TMP_PATH or what the user has provided via the -t parameter. Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> --- git-archimport.perl | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) 67d3cf04c6a2353835d52a4d1ae85c7fb91bad36 diff --git a/git-archimport.perl b/git-archimport.perl --- a/git-archimport.perl +++ b/git-archimport.perl @@ -23,7 +23,6 @@ See man (1) git-archimport for more deta - create tag objects instead of ref tags - audit shell-escaping of filenames - - better handling of temp directories - hide our private tags somewhere smarter - find a way to make "cat *patches | patch" safe even when patchfiles are missing newlines @@ -37,7 +36,7 @@ use strict; use warnings; use Getopt::Std; use File::Spec; -use File::Temp qw(tempfile); +use File::Temp qw(tempfile tempdir); use File::Path qw(mkpath); use File::Basename qw(basename dirname); use String::ShellQuote; @@ -72,9 +71,10 @@ usage if $opt_h; @ARGV >= 1 or usage(); my @arch_roots = @ARGV; -my $tmp = $opt_t; -$tmp ||= '/tmp'; -$tmp .= '/git-archimport/'; +my ($tmpdir, $tmpdirname) = tempdir('git-archimport-XXXXXX', TMPDIR => 1, CLEANUP => 1); +my $tmp = $opt_t || 1; +$tmp = tempdir('git-archimport-XXXXXX', TMPDIR => 1, CLEANUP => 1); +$opt_v && print "+ Using $tmp as temporary directory\n"; my @psets = (); # the collection my %psets = (); # the collection, by name ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/5] archimport documentation update 2005-09-11 9:26 ` [PATCH 1/5] archimport documentation update martin 2005-09-11 9:26 ` [PATCH 2/5] archimport documentation tidyup martin @ 2005-09-11 14:58 ` A Large Angry SCM 1 sibling, 0 replies; 7+ messages in thread From: A Large Angry SCM @ 2005-09-11 14:58 UTC (permalink / raw) To: martin; +Cc: git martin@catalyst.net.nz wrote: > Updated and expanded the command description, and added a reference of the > command line options. > > Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> > [...] Thanks Martin! ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2005-09-11 14:58 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2005-09-11 9:26 archimport documentation updates and tidyups martin 2005-09-11 9:26 ` [PATCH 1/5] archimport documentation update martin 2005-09-11 9:26 ` [PATCH 2/5] archimport documentation tidyup martin 2005-09-11 9:26 ` [PATCH 3/5] archimport - update in-script doco, options tidyup martin 2005-09-11 9:26 ` [PATCH 4/5] archimport - use GIT_DIR instead of hardcoded ".git" martin 2005-09-11 9:26 ` [PATCH 5/5] archimport - better handling of temp dirs martin 2005-09-11 14:58 ` [PATCH 1/5] archimport documentation update A Large Angry SCM
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).