Re: Merging submodules

["H.Merijn Brand" <h.m.brand@xs4all.nl>]

On Fri, 1 Aug 2008 13:34:42 +0200, "Santi Béjar" <sbejar@gmail.com>
wrote:

> On Fri, Aug 1, 2008 at 12:35, H.Merijn Brand <h.m.brand@xs4all.nl> wrote:
> 
> >
> > After we join/merge these into the super-project, we're going to remove
> > the sup-repos, so all new commits will be made in the super-repo.
> 
> If you'll remove the subrepos, the best thing would be to rewrite the
> history on those subrepos just moving all the path to the
> corresponding subfolder (with git-filter-branch, and you have exactly
> what you need at the end of the example section in the manpage). And
> then just do a normal merge. Or, even, you could try to create a
> project with everything there in the correct order and location, I
> don't know if git-filter-branch or git-fast-import/export (you have
> some examples in git.git in contrib/fast-import) can do it, but if you
> get it, please, post it here because it can be useful for others).

I gave up, even thought I'm not satisfied with the result: Merging with
subtrees works, but completely ruins my view over the history, cause
gitk shows the branches as a whole, and not `merged' by date. I did
change the merging process to merge the repo with the least recent
change date first, so that it would show up at the bottom and the
newest on top. This is workable, but far from perfect.

The fast-export/fast import with renaming started out fine, but
fast-import does not merge. For the latter I (tried to) use the
following approach:

--8<---
#!/pro/bin/perl

use strict;
use warnings;

sub usage
{
    my $err = shift and select STDERR;
    print "usage: $0 dir|repo.tar\n";
    @_ and print join "\n", @_, "";
    exit $err;
    } # usage

-d "new" and die "Dir new already exists\n";

use Cwd;
use File::Find;
use Getopt::Long qw(:config bundling nopermute);
my $opt_v = 1;
GetOptions (
    "help|?"    => sub { usage (0); },

    "v:2"       => \$opt_v,
    ) or usage (1);

@ARGV == 1 or usage (1);

my $tmp_archive = "/tmp/git-join-$$.tgz";
END { unlink $tmp_archive };

my $archive = shift;
if (-d $archive) {
    my $cwd = getcwd;
    my @dir;
    chdir $archive or die "$archive: $!\n";
    find (sub {
        $_ eq ".git" && -d $_ and push @dir, $File::Find::name;
        }, ".");
    qx{ tar czf $tmp_archive @dir };
    $archive = $tmp_archive;
    chdir $cwd;
    }

sub pr_time
{
    my @d = @_;
    sprintf "%4d-%02d-%02d %02d:%02d:%02d", 1900 + $d[5], ++$d[4], @d[3,2,1,0];
    } # pr_time

-f $archive && -s _ or usage (1, "Archive is not a file");

my @cmd =
    $archive =~ m/\.tar$/               ? qw( tar xf  )         :
    $archive =~ m/\.t(ar\.)?gz$/        ? qw( tar xzf )         :
    $archive =~ m/\.t(ar\.)?bz2?$/      ? qw( tar xhf )         :
    usage (1, "$archive is not a recognized archive type");

print STDERR "Creating merge environment\n";
mkdir "new", 0777;
chdir "new" or die "Canot use folder new\n";

print STDERR "Recovering original repo's\n";
system @cmd, $archive;

my %modules;
find (sub {
    (my $f = $File::Find::name) =~ s{^\./}{};
    $f =~ s{/\.git$}{};
    $_ eq ".git" && -d $_ && !$modules{$f} or return;
    print "Found $f\n";
    system "git-log '--pretty=format:%ct' | head -1";
    chomp ($modules{$f} = `git-log '--pretty=format:%ct' | head -1`);
    }, ".");
my @modules = sort { $modules{$a} <=> $modules{$b} } keys %modules;

sub git
{
    system "git", @_;
    } # git

my $top = getcwd;

print STDERR "Initializing new repo\n";
git "init";

foreach my $mod (@modules) {
    print STDERR "Merging ", pr_time (localtime $modules{$mod}), " $mod ...\n";
    chdir $mod;

    git "checkout", "-f";
    git "filter-branch", "--index-filter",
        qq{git ls-files -s | sed "s-\t-&$mod/-" | }.
         q{GIT_INDEX_FILE=$GIT_INDEX_FILE.new }.
         q{git update-index --index-info && }.
         q{mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE}, "HEAD";
    git "fast-export", "--all", ">/tmp/git-export-$$.bin";

    chdir $top;
    git "fast-import", "</tmp/git-export-$$.bin";
    }

print STDERR "Done\n";
-->8---

Which looks perfect after the first import, but fails on every next

Using --force for fast-import removes every previous import, so it is
useless for this process.

I do have to work with the repo, and that is more important than having
a perfect repo.

-- 
H.Merijn Brand          Amsterdam Perl Mongers  http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, SuSE 10.1, 10.2, and 10.3, AIX 5.2, and Cygwin.
http://mirrors.develooper.com/hpux/           http://www.test-smoke.org/
http://qa.perl.org      http://www.goldmark.org/jeff/stupid-disclaimers/