* finding similar blobs (was: Re: $Revision$ keyword replacement?
From: Junio C Hamano @ 2005-11-22 22:42 UTC (permalink / raw)
To: Santi Béjar; +Cc: git
In-Reply-To: <871x18h9ee.fsf@ifae.es>
Santi Béjar <sbejar@gmail.com> writes:
> How similar this file is to the file $path in these commit(s)?
Very cute.
> tmp=`mktemp -t -d git-find-sim.XXXXXXX`
> ...
> git update-index --add $file || exit 1
> tree=`git-write-tree`
Are you going through all this trouble just to avoid a blob and
a tree object left dangling after you are done? Or is there
something else going on?
> rev_arg=`GIT_DIR=$GIT_DIR_ORIG git-rev-parse --default HEAD --revs-only "$@"`
> revs=`GIT_DIR=$GIT_DIR_ORIG git-rev-list $rev_arg`
> for i in $revs; do
> git diff-tree --name-status $i -C $tree | grep $file |
> sed "s/^/$i:/"
> done
Perhaps
GIT_DIR=$GIT_DIR_ORIG git-rev-list $rev_arg |
while read one
git diff-tree --name-status -r $one -C $tree | grep $file |
sed "s/^/$one:/"
done
just in case the similar file you will discover is hidden in a
subdirectory?
^ permalink raw reply
* [PATCH] Add git-graft-ripple, a tool for permanently grafting history into a tree.
From: Ryan Anderson @ 2005-11-22 20:50 UTC (permalink / raw)
To: Junio C Hamano, git; +Cc: Ryan Anderson
Enhancements over the original example:
o Each newly created commit A' references A, and (A^1)' (The first try
referenced A^1 and (A^1)' but not A)
o Support for incrementally rewriting history is present.
Signed-off-by: Ryan Anderson <ryan@michonline.com>
---
Documentation/git-graft-ripple | 57 +++++++++++++++++
Makefile | 3 +
git-graft-ripple.perl | 137 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 196 insertions(+), 1 deletions(-)
create mode 100644 Documentation/git-graft-ripple
create mode 100755 git-graft-ripple.perl
applies-to: 92551f12af8cd77f821d066a53c4dd301d1b41f5
9bfbdd948e6c0c9a8f0383b355e5cbbb42e0a772
diff --git a/Documentation/git-graft-ripple b/Documentation/git-graft-ripple
new file mode 100644
index 0000000..2fa38cb
--- /dev/null
+++ b/Documentation/git-graft-ripple
@@ -0,0 +1,57 @@
+git-graft-ripple(1)
+===============
+
+NAME
+----
+git-graft-ripple - Rewrite history with an additional commitish as a parent.
+
+
+SYNOPSIS
+--------
+'git-log <git repository> <branch committish> <graft SHA1> [start commitish]'
+
+DESCRIPTION
+-----------
+
+git-graft-ripple rewrites a series of commit objects by adding a new parent to
+the most ancestral commit, and passing the changed history back up through the
+revision history.
+
+Proceeding in reverse merge order a new commit is created using the following
+algorithim:
+
+ If this is the first commit reached, append the graft object name to
+ the parents.
+
+ Replace each parent object name with the object name generated by this
+ algorithm earlier.
+
+ Add the original object name to the parents.
+
+If a start commitish is given, the output of git-rev-list is restricted using
+it. This allows for incremental grafting to occur.
+
+Note: No branches are modified by this operation.
+
+Files
+-----
+
+GIT_DIR/info/ripple-status contains a mapping between the original object name
+and the object name generated by this algorithm. When doing incremental
+grafting, this file is necessary to continue to rename objects. If this is not
+present, git-graft-ripple will leave the old object name in the list of
+parents, which should still allow most merges to occur cleanly.
+
+
+Author
+------
+Written by Ryan Anderson <ryan@michonline.com>
+
+Documentation
+--------------
+Documentation by Ryan Anderson.
+
+GIT
+---
+Part of the gitlink:git[7] suite
+
diff --git a/Makefile b/Makefile
index 1109dc6..984be32 100644
--- a/Makefile
+++ b/Makefile
@@ -97,7 +97,8 @@ SCRIPT_SH = \
SCRIPT_PERL = \
git-archimport.perl git-cvsimport.perl git-relink.perl \
git-shortlog.perl git-fmt-merge-msg.perl \
- git-svnimport.perl git-mv.perl git-cvsexportcommit.perl
+ git-svnimport.perl git-mv.perl git-cvsexportcommit.perl \
+ git-graft-ripple.perl
SCRIPT_PYTHON = \
git-merge-recursive.py
diff --git a/git-graft-ripple.perl b/git-graft-ripple.perl
new file mode 100755
index 0000000..b8ec990
--- /dev/null
+++ b/git-graft-ripple.perl
@@ -0,0 +1,137 @@
+#!/usr/bin/perl
+# Copyright 2005 Ryan Anderson <ryan@michonline.com>
+#
+# GPL v2 (See COPYING)
+#
+# This file is licensed under the GPL v2, or a later version
+# at the discretion of Linus Torvalds.
+
+use strict;
+use warnings;
+
+use IPC::Open2;
+
+sub git_commit_tree {
+ my ($tree,$comments,@parents) = @_;
+
+ my @cparents;
+ foreach my $p (@parents) {
+ push @cparents,"-p",$p;
+ }
+
+ my $pid = open2(*Reader, *Writer,
+ "git-commit-tree",$tree,@cparents);
+ print Writer $comments;
+ close(Writer);
+ my $commit = <Reader>;
+
+ waitpid $pid, 0;
+ close(Reader);
+
+ chomp $commit;
+
+ return $commit;
+}
+
+my ($gittree,$branch,$graft,$start) = @ARGV;
+chdir($gittree)
+ or die sprintf("Failed to chdir to '%s': %s\n", $ARGV[0], $!);
+
+my $range = defined $start ? sprintf("%s..%s",$start,$branch) : $branch;
+
+#printf("Running git-rev-list --parents --merge-order %s\n",$range);
+open(GRL,"-|","git-rev-list","--parents", "--merge-order", $range)
+ or die "Failed to run git-rev-list: " . $!;
+
+my %csets;
+my @revs;
+while(<GRL>) {
+ chomp;
+ my ($commit,@parents) = split;
+ $csets{$commit}{parents} = \@parents;
+ push @revs, $commit;
+# printf("Investigating %s\n", $commit);
+
+ open(GCF,"-|","git-cat-file","commit",$commit)
+ or die "Failed to open git-cat-file: " . $!;
+
+ my $in_comments = 0;
+ while(<GCF>) {
+ chomp;
+ if ($in_comments) {
+ $csets{$commit}{comments} .= $_ . "\n";
+
+ } elsif (m/^tree (.+)$/) {
+ $csets{$commit}{tree} = $1;
+ #printf("tree = %s\n",$1);
+
+ } elsif (m/^parent (.+)$/) {
+ # Do nothing, we already got
+ # the parents from rev-list.
+
+ } elsif (m/^(author|committer) (.*) <(.*)> (.*)$/) {
+ #printf("%s = %s <%s> at %s\n",$1, $2,$3,$4);
+ @{$csets{$commit}{$1}}{qw(name email datetime)}
+ = ($2,$3,$4);
+
+ } elsif (length == 0) {
+ $in_comments = 1;
+ $csets{$commit}{comments} = "";
+ next;
+ }
+ }
+ close(GCF);
+
+}
+close(GRL);
+
+@revs = reverse @revs;
+shift @revs if defined $start;
+push @{$csets{$revs[0]}{parents}},$graft;
+
+my %newcsets;
+if (-f ".git/info/ripple-status") {
+ open(RSTATUS,"<",".git/info/ripple-status")
+ or die "Failed to restore status from .git/info/ripple-status: " . $!;
+
+ while (<RSTATUS>) {
+ chomp;
+ my ($k,$newcset) = split;
+ $newcsets{$k} = $newcset;
+ }
+ close(RSTATUS);
+}
+
+
+foreach my $old (@revs) {
+ printf("\nProcessing commit %s\n",$old);
+ $ENV{GIT_AUTHOR_EMAIL} = $csets{$old}{author}{email};
+ $ENV{GIT_AUTHOR_NAME} = $csets{$old}{author}{name};
+ $ENV{GIT_AUTHOR_DATE} = $csets{$old}{author}{datetime};
+ $ENV{GIT_COMMITTER_DATE} = $csets{$old}{committer}{datetime};
+ $ENV{GIT_COMMITTER_EMAIL} = $csets{$old}{committer}{email};
+ $ENV{GIT_COMMITTER_NAME} = $csets{$old}{committer}{name};
+
+ my @parents = ($old);
+ foreach my $p (@{$csets{$old}{parents}}) {
+ if (exists $newcsets{$p}) {
+ push @parents, $newcsets{$p};
+ printf("\t%s => %s\n",
+ $newcsets{$p},$p);
+ } else {
+ push @parents, $p;
+ }
+ }
+ my $commit = git_commit_tree($csets{$old}{tree},
+ $csets{$old}{comments},@parents);
+ $newcsets{$old} = $commit;
+ printf("\tNew commit: %s\n",$newcsets{$old});
+}
+
+open(RSTATUS,">",".git/info/ripple-status")
+ or die "Failed to save status in .git/info/ripple-status: " . $!;
+
+foreach my $k (keys %newcsets) {
+ printf RSTATUS "%s\t%s\n", $k, $newcsets{$k};
+}
+close(RSTATUS);
---
0.99.9.GIT
^ permalink raw reply related
* Re: [PATCH] speedup allocation in pack-redundant.c
From: Junio C Hamano @ 2005-11-22 20:41 UTC (permalink / raw)
To: Alex Riesen; +Cc: Lukas Sandström, git
In-Reply-To: <81b0412b0511220656l528436b1xea80ee18965e4dda@mail.gmail.com>
Alex Riesen <raa.lkml@gmail.com> writes:
> Subject: [PATCH] speedup allocation in pack-redundant.c
>
> Reuse discarded nodes of llists
>
> Signed-off-by: Alex Riesen <ariesen@harmanbecker.com>
I think making allocation/deallocation to the central place is a
good cleanup, but I am not sure about the free-nodes reusing.
Does this make difference in real life? If so, it might be
worth doing the slab-like allocation, since free-nodes are very
small structure and malloc overhead is not ignorable there.
^ permalink raw reply
* Re: 2.6.15-rc2 tag
From: Jeff Garzik @ 2005-11-22 20:38 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Linus Torvalds, git
In-Reply-To: <20051122201726.GB8738@fieldses.org>
On Tue, Nov 22, 2005 at 03:17:26PM -0500, J. Bruce Fields wrote:
> Yup, thanks. Is there any reason to ever use http to get to your
> repository?
Linus's? For you? Probably not. But for others:
1) Corporate firewalls, which only allow HTTP transit, without
jumping through hoops.
2) Far easier to find HTTP hosting than rsync or git hosting.
Jeff
^ permalink raw reply
* Re: 2.6.15-rc2 tag
From: Junio C Hamano @ 2005-11-22 20:33 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0511221200340.13959@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Tue, 22 Nov 2005, J. Bruce Fields wrote:
>>
>> I'm still not getting a 2.6.15-rc2 tag with either git-clone or
>> git-fetch --tags. Any ideas? Is the problem with me or with the
>> repository?
>
> Try using
>
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
>
> instead. Does that help?
It should. And I think git:// should be encouraged over
http://, but some people (including me at work) can only come
from http://, so...
> I don't understand how http:// works (or doesn't), so..
The difference between git-ls-remote against your repository via
git:// and http:// would tell you. Your info/refs in your repo
is stale; git http transport is designed to cope with webservers
without dirindex, and only uses info/refs to find out what refs
you have in that repository. It does not fall back on "wget -r"
behaviour simply because it is a mess (you get all sorts of
useless links by doing dirindex in refs/ directory, like Name,
Last modified, Size, Parent Directory).
Mind enabling hooks/post-update on the master so that
update-server-info is run after you push into it?
^ permalink raw reply
* Re: 2.6.15-rc2 tag
From: J. Bruce Fields @ 2005-11-22 20:17 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0511221200340.13959@g5.osdl.org>
On Tue, Nov 22, 2005 at 12:01:08PM -0800, Linus Torvalds wrote:
> On Tue, 22 Nov 2005, J. Bruce Fields wrote:
> >
> > I'm still not getting a 2.6.15-rc2 tag with either git-clone or
> > git-fetch --tags. Any ideas? Is the problem with me or with the
> > repository?
>
> Try using
>
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
>
> instead. Does that help?
Yup, thanks. Is there any reason to ever use http to get to your
repository?
> I don't understand how http:// works (or doesn't), so..
>From a quick look in ethereal, it seems to be fetching
http://kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/info/refs
which has a list of tags that doesn't include 2.6.15-rc2. What
generates that file? Is it broken?
--b.
^ permalink raw reply
* Re: Get rid of .git/branches/ and .git/remotes/?
From: Adrien Beau @ 2005-11-22 20:13 UTC (permalink / raw)
To: git
In-Reply-To: <20051122191234.GA9040@puritan.petwork>
On 11/22/05, Nikolai Weibull <mailing-lists.git@rawuncut.elitemail.org> wrote:
>
> who is the target audience of git? I get the
> feeling that most users will be programmers, so that's kind of a
> non-argument (even though I agree with your standpoint).
If Git is successful, then there will also be a lot of bleeding-edge
users, people who want to be able to:
* Get the latest and greatest (and build it and run it)
* Browse the repository (with gitk or gitweb)
* Maybe (very occasionnally) create a simple patch
This is already the case with CVS, plenty of people know (or are
instructed to do) cvs checkout, cvs update, and nothing more. That's
not much, but they're CVS users, nevertheless.
^ permalink raw reply
* Re: 2.6.15-rc2 tag
From: Linus Torvalds @ 2005-11-22 20:01 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: git
In-Reply-To: <20051122193418.GC5628@fieldses.org>
On Tue, 22 Nov 2005, J. Bruce Fields wrote:
>
> I'm still not getting a 2.6.15-rc2 tag with either git-clone or
> git-fetch --tags. Any ideas? Is the problem with me or with the
> repository?
Try using
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
instead. Does that help?
I don't understand how http:// works (or doesn't), so..
Linus
^ permalink raw reply
* Re: Question about handling of heterogeneous repositories
From: Andreas Ericsson @ 2005-11-22 19:40 UTC (permalink / raw)
Cc: git
In-Reply-To: <81b0412b0511220850w429d2f36lafe9de7ce19ce8f@mail.gmail.com>
Alex Riesen wrote:
> Hi,
>
> it is sometimes the case that a project consists of parts which are
> unrelated to each other, and only thing in common between them is that
> they all are used in that particular project. For example a program
> uses some library and the developer(s) of that program would like to
> have the source of that library somewhere close. Well, for this simple
> example one could just use two repositories, laid close to each other
> in a directory, like project/lib and project/prog.
> Now, if I make the example a bit more complex and say, that the
> developers of the program are the developers in that project and
> change everything under project/ directory, including
> project/library/. They are also good people and ready to give the
> changes to the library upstream.
>
> How do they achieve that, without sending project/ and project/program/?
>
> For everyone who have an experience with ClearCase or Perforce (I'm
> sorry for mentioning it) it is what the "mappings" are often used for:
> a project is build together from different parts, which can be worked
> on separately.
>
> I'm trying to introduce git at work, but have to prepare myself for
> possible questions first, and this is one of them :)
>
We do like this;
core
core/gui
core/lib
$ cat .gitignore
gui
lib
This is also nice because it lets the gui maintainers have the gui as
the root with the core and lib parts as subdirectories. Everyone has
their own responsibility checked out at top-level with other pieces
below it. It's easy enough to script a pull of all repos so everyone's
up to sync and everybody's happy.
It would certainly be nicer to have git ignore directories that have the
".git" directory (so long as it's not the top of the repo, that is), but
I haven't had the energy to fix that when there's already a solution
that's simple enough and quite adequate.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: 2.6.15-rc2 tag
From: J. Bruce Fields @ 2005-11-22 19:34 UTC (permalink / raw)
To: git
In-Reply-To: <20051121212549.GA23213@fieldses.org>
I'm still not getting a 2.6.15-rc2 tag with either git-clone or
git-fetch --tags. Any ideas? Is the problem with me or with the
repository?
--b.
On Mon, Nov 21, 2005 at 04:25:49PM -0500, J. Bruce Fields wrote:
> Help! I'm confused.
>
> git-fetch --tags http://kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
>
> isn't getting me a 2.6.15-rc2 tag. So maybe there's some delay? There
> does, however, appear to be a file
>
> http://kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/tags/v2.6.15-rc2
>
> Its contents are 7305b5cb045e2c71250b5b7472771ed2620bc514 which isn't
> anything I can find anywhere.
>
> gitweb, on the other hand:
>
> http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=tag;h=7305b5cb045e2c71250b5b7472771ed2620bc514
>
> shows 3bedff1d73b86e0cf52634efb447e9ada08f2cc6 as the tagged commit,
> which is something I do have. What don't I understand?
>
> --b.
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Get rid of .git/branches/ and .git/remotes/?
From: Andreas Ericsson @ 2005-11-22 19:30 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Josef Weidendorfer, git
In-Reply-To: <Pine.LNX.4.63.0511221854120.27872@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin wrote:
>
> git-config-set --get remote.origin.url
>
So... "git-config-set" is used for both getting and setting? Why not
just "git-config --set" and "git-config --get" to make things a bit less
confusing?
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Get rid of .git/branches/ and .git/remotes/?
From: Nikolai Weibull @ 2005-11-22 19:12 UTC (permalink / raw)
To: git
In-Reply-To: <4382DFDA.6040306@op5.se>
Andreas Ericsson wrote:
> Excellent error messages aren't good enough. It's ok for Python, since
> that's a programming language. We can expect infinitely more from
> programmers than we can from users.
A semi-related question: who is the target audience of git? I get the
feeling that most users will be programmers, so that's kind of a
non-argument (even though I agree with your standpoint).
Furthermore, does it really matter what format .git/config has now that
we have git-config-set? Shouldn't all access go through that command,
so that we can change to some other format (YAML, XML, STUPIDABBR) if we
so desire without breaking anything?
Finally, a plain-text easy-to-edit format is great, and that's a good
enough argument not to use indentation (as has already been pointed out,
indentation is not always what it seems).
nikolai
--
Nikolai Weibull: now available free of charge at http://bitwi.se/!
Born in Chicago, IL USA; currently residing in Gothenburg, Sweden.
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
^ permalink raw reply
* Re: [QUESTION] Access to a huge GIT repository.
From: Franck @ 2005-11-22 19:10 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v1x18fw8j.fsf@assigned-by-dhcp.cox.net>
2005/11/22, Junio C Hamano <junkio@cox.net>:
> Were you around on this list, around the beginning of this
> month?
nope, I subscribed to the list one week later.
> The thread that starts here may be of interest:
>
> http://marc.theaimsgroup.com/?l=git&m=113089701819420
>
> I think (although I do not exactly know how the "lite"
> repository was constructed) what you are doing is similar to
> what I called "shallow clone" there in that thread.
Indeed. What I'm doing to get my "shallow clone" is basicaly remove
every ref/branchs I don't want, then run git-prune, git-repack -a -d.
But the result should be the same as your "shallow clone" method.
> At the end
> of the thread I think I listed what you can and cannot do in
> such an incomplete repository.
>
Yes. But the big difference is that, in my case, the shallow copy is
used as a public repository, whereas in your case the shallow copy is
used as a working repository. Anyway, Ralf (the mips arch maintainer)
is going to create a new pruned repository that should resolve my
problem. I'm going to wait for it instead of trying to make my own
"broken" repository.
Thanks
--
Franck
^ permalink raw reply
* Re: Diffs "from" working directory
From: Linus Torvalds @ 2005-11-22 18:32 UTC (permalink / raw)
To: Chuck Lever; +Cc: Catalin Marinas, git
In-Reply-To: <43835D8E.60109@citi.umich.edu>
On Tue, 22 Nov 2005, Chuck Lever wrote:
>
> then perhaps the problem is that the "stg mail" tool should place the author
> in the From: field automatically? (ie change the tool, or permanently modify
> the default template that comes with StGIT to do this, as Catalin suggested
> earlier).
>
> that seems a little twisty to me; you're overloading the SMTP header field
> instead of explicitly specifying patch authorship. seems like a layering
> violation.
No, I only use the actual SMTP header field if the _body_ of the email
doesn't contain the "From:".
So there's really two different "From:" lines: there's the SMTP header
one, which is just a default fallback one, and there's the first non-empty
line of the email body itself, which is the preferred one. No layering
violation, just two different layers that have the same format for the
line.
See "The Perfect Patch" by Andrew, and bullet (4): Attribution:
http://www.zip.com.au/~akpm/linux/patches/stuff/tpp.txt
To quote:
'If someone else wrote the patch, they should be credited (and blamed)
for it. To communicate this, add a line:
From: John Doe <jdoe@wherever.com>
as the very first line of the email. Downstream tools will pick this
up and jdoe will get the git "Author" line.'
and I'd be even more anal about it: I would seriously suggest to people
that they just _always_ add the "From:" line at the head of the email,
even if it just is exactly the same as what will be in the SMTP header.
Why? Simple. It makes is less likely that somebody who just forwards the
patch will forget to add that line for you. So you are really helping
people out - and making sure the attribution stays correct - by adding
that extra "From:" line at the top of your email body, even if it is
"unnecessary" in the sense that it's also in your SMTP header.
Linus
^ permalink raw reply
* Re: auto-packing on kernel.org? please?
From: Chuck Lever @ 2005-11-22 18:18 UTC (permalink / raw)
To: Linus Torvalds
Cc: Carl Baldwin, H. Peter Anvin, Git Mailing List, Catalin Marinas
In-Reply-To: <Pine.LNX.4.64.0511212134330.13959@g5.osdl.org>
[-- Attachment #1: Type: text/plain, Size: 2357 bytes --]
Linus Torvalds wrote:
>
> On Tue, 22 Nov 2005, Chuck Lever wrote:
>
>>there are some things repacking does that breaks StGIT, though.
>>
>>git repack -d
>>
>>seems to remove old commits that StGIT was still depending on.
>
>
> If that is true, then "git-fsck-cache" probably also reports errors on a
> StGIT repository. No? Basically, it implies that the tool doesn't know how
> to find all the "heads".
indeed. this is one area where StGIT is "not safe" to use with other
porcelains. these raw GIT commands can show a bunch of confusing
"dangling references" type errors, or actually modify the index in ways
that eliminate StGIT-related commits that aren't currently attached to
any ancestry. (i think Catalin mentioned these are related to the
unapplied patches in a stack, but there could be others; see below).
> The preferred way would be to just list the references somewhere under
> .git/refs/stgit, in which case fsck and repack should pick them up
> automatically (so clearly stgit doesn't do that right now ;).
that could be an extremely large number of commits on a large repository
with a lot of patches that have been worked on over a long period. so
whatever mechanism is created to do this needs to scale well in the
number of commits.
> It also implies that doing a "git prune" will do horribly bad things to a
> stgit repo, since it would remove all the objects that it thinks aren't
> reachable..
yup. been there, done that. lucky for me i have an excellent hourly
backup scheme.
>>git repack -a -n
>>
>>seems to work fine with StGIT,
>
>
> Well, it "works", but not "fine". Since it doesn't know about the stgit
> objects, it won't ever pack them.
ah!
> But maybe that's what stgit wants (since they are "temporary"), but it
> does mean that if you see a big advantage from packing, you might be
> losing some of it.
actually, those commits aren't all that "temporary". the
history/revision feature i'm working on would like to maintain all the
commits ever done to an StGIT patch.
the only time you can throw away such commits is when the patch is
deleted or when it is finally committed to the repository via "stg
commit". otherwise, keeping these commits in a pack would be quite a
good thing.
maybe the first thing to do is to get a basic understanding of an StGIT
commit's lifetime.
[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 439 bytes --]
begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Linux NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763-4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668-1089
x-mozilla-html:FALSE
url:http://www.monkey.org/~cel/
version:2.1
end:vcard
^ permalink raw reply
* Re: auto-packing on kernel.org? please?
From: Linus Torvalds @ 2005-11-22 17:58 UTC (permalink / raw)
To: Carl Baldwin; +Cc: H. Peter Anvin, Git Mailing List
In-Reply-To: <20051122172558.GA1935@hpsvcnb.fc.hp.com>
On Tue, 22 Nov 2005, Carl Baldwin wrote:
> On Mon, Nov 21, 2005 at 11:24:11AM -0800, Linus Torvalds wrote:
> > NOTE! Since that email, "git repack" has gotten a "local" option (-l),
> > which is very useful if the repositories have pointers to alternates.
> >
> > So do
> >
> > git repack -l
> >
> > instead, to get much better packs (and "-a -d" for the full case, of
> > course).
>
> I'm assuming that this option will have no effect on a repository with
> no alternates file.
Correct.
The only thing it does is that when it looks up an object, if it's not in
our _own_ ".git/objects/" dir, it won't pack it.
Actually, that's not entirely true. It isn't smart enough to know where
every object exists, so it only knows about remote _packs_. So what
happens is that if you do
git repack -l -a -d
it will create a pack-file that contains _all_ unpacked objects (whether
local or not) and all objects that are in local packs (because of the
"-a"), but not any objects that are in "alternate packs".
Which is actually exactly what you want, if you are in the situation that
kernel.org is, and you have people who point their alternates to mine:
when I repack my objects, they'll use my packs, but other than that,
they'll prefer to use their own packs over any unpacked objects.
> > So right now, repacking a broken archive can actually break it even more.
>
> Interesting.
Well, with the latest git repack script, that should no longer be true.
> > NOTE! Your "git verify-pack" wouldn't even catch this: the _pack_ is fine,
> > it's just incomplete.
>
> In my opinion, git repack did the right thing in creating the pack even
> if it is more broken. Starting with a broken repository was the real
> problem. git repack shouldn't need to worry too much about it.
Well, "git repack" did the wrong thing in that it never _noticed_, and it
then removed all old packs - even though those old packs contained objects
that we hadn't repacked because of the broken repository.
Of course, _usually_ a broken repository is just that - broken. The way
you fix a broken repo is to find a non-broken one, and clone that.
However, sometimes what you can do (if you literally just lost a few
objects) is to find a non-broken repo, and make that the _alternates_, in
which case you may be able to save any work you had in the broken one
(assuming you only lost objects that were available somewhere else).
> Looking at it from the nervous repository admin's point of view I think
> he would want to make sure that the repository is good to begin with.
Doing an fsck is certainly always a good idea. I do a "shallow" fsck
usually several times a day ("shallow" means that it doesn't fsck packs,
only new objects that I have aquired since the last repacking), and I do a
full fsck a couple of times a week.
I don't actually know why I do that, though. I don't think I've really
_ever_ had a broken repo since some very early days, except for the cases
where I break things on purpose (like remove an object to check whether
"git repack" does the right thing or not). I'm just used to it, and the
shallow fsck takes a fraction of a second, so I tend to do it after each
pull.
So I really think that an admin has to be more than "nervous" to worry
about it. He has to be really anal.
(Now, doing a repack and a fsck every week or so might be good, and
automatic shallow fsck's daily is probably a great idea too. After all, it
_is_ checking checksums, so if you worry about security and want to make
sure that nobody is trying to break in and do bad things to your repo, a
regular fsck is a good thing even if you're not otherwise worried about
corruption).
> *NOTE* There is one question that I feel remains unanswered. Is it
> possible to split up the repack -a and repack -d so that the nervous
> repository owner can insert a git verify-pack in the middle.
They are already split up inside "git-repack", so we could add a hook
there, I guess. See the git-repack.sh file, and notice how it does the
"remove_redundant" part only after it has created the new pack-file and
done a "sync".
> I don't mean to say that I don't trust git repack to do the right thing.
> Fundamentally, I just think that I shouldn't depend on it to do the
> right thing in order to avoid corruption in my repository.
That's good. However, as the previous failure of git repack showed, to
some degree the more likely failure mode is actually that the pack
generated by "git repack" is perfectly fine, but it's not _complete_. Say
we have a bug in git repack, for example.
Another case where it's not complete is when you have deleted a branch.
"git repack -a -d" will effectively do a "git prune" wrt objects that are
no longer reachable, and that were in the old packs.
So I'd actually suggest a slightly different approach. When-ever you
remove old objects (whether it's "git prune" or "git prune-packed" or "git
repack -a -d"), you might want to have an option that doesn't actually
_remove_ them, but just moves them into ".git/attic" or something like
that.
Then you can clean up the attic after doing your weekly full fsck or
something. And it has the advantage that if somebody has deleted a branch,
and notices later that maybe he wanted that branch back, you can "unprune"
all the objects, run "git-fsck-objects --full" to find any dangling
commits, and you'll have all your branches back.
So in many ways it would perhaps be nicer to have that kind of "safe
remove" option to the pruning commands?
Linus
^ permalink raw reply
* Re: Diffs "from" working directory
From: Chuck Lever @ 2005-11-22 18:03 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Catalin Marinas, git
In-Reply-To: <Pine.LNX.4.64.0511212124160.13959@g5.osdl.org>
[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]
Linus Torvalds wrote:
>
> On Tue, 22 Nov 2005, Chuck Lever wrote:
>
>>for some reason i was under the impression that it would parse the
>>Signed-off-by: fields in the patch description, and take the first one as the
>>patch author.
>
>
> The first sign-off really isn't necessarily the author.
>
> It might be a company sign-off (many companies don't want any random
> engineer to send out patches), but much more commonly it's a trivial patch
> that somebody else signs off on, even if the original patcher didn't (see
> case (b) in the sign-off-rules: you can sign of on somebody elses work if
> you know it's under the GPL).
heh. in fact that is what my company (NetApp) requires.
> So the fact that there was a sign-off procedure doesn't automatically mean
> that the author will be the first sign-off person, although in _practice_
> that obviously would likely always be the most common case by far.
>
> (Another reason is that some people actually add the sign-offs above
> previous ones. It happens, although if I notice, I try to point it out).
>
> So authorship really is totally separate from sign-off, and all _my_ tools
> take the authorship from the first "From:" line at the top of the message
> body or from the email itself.
then perhaps the problem is that the "stg mail" tool should place the
author in the From: field automatically? (ie change the tool, or
permanently modify the default template that comes with StGIT to do
this, as Catalin suggested earlier).
that seems a little twisty to me; you're overloading the SMTP header
field instead of explicitly specifying patch authorship. seems like a
layering violation.
[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 439 bytes --]
begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Linux NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763-4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668-1089
x-mozilla-html:FALSE
url:http://www.monkey.org/~cel/
version:2.1
end:vcard
^ permalink raw reply
* Re: Get rid of .git/branches/ and .git/remotes/?
From: Johannes Schindelin @ 2005-11-22 17:56 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: git
In-Reply-To: <200511221831.03954.Josef.Weidendorfer@gmx.de>
Hi,
On Tue, 22 Nov 2005, Josef Weidendorfer wrote:
> [remote:origin]
> url = master.kernel.org:/pub/scm/git/git.git
> pull = master:origin
>
> For this, the parser could return
> remote.url = master.kernel.org:/pub/scm/git/git.git for origin
> remote.pull = master:origin for origin
>
> By introducing such a scheme, multi-value vars would fit perfectly
> for use for .git/remotes or .git/branches.
I don't know if you missed it, but hierarchical config keys were
introduced by b17e659dd4007cb1d3eb5ac32b524c0c5ab59601:
[remote.origin]
url = master.kernel.org:/pub/scm/git/git.git
pull = master:origin
is parsable by
git-config-set --get remote.origin.url
now.
Hth,
Dscho
^ permalink raw reply
* Re: [PATCH 4/6] Add check_repo_format check for all major operations.
From: Junio C Hamano @ 2005-11-22 17:46 UTC (permalink / raw)
To: Martin Atukunda; +Cc: git
In-Reply-To: <200511221555.24572.matlads@dsmagic.com>
Martin Atukunda <matlads@dsmagic.com> writes:
>> that call in the current series yet. Even if you had, this does
>> not feel quite right to me.
>
> would something like the following apply in this case: (totally untested :)
Yes, although that is exactly what I said "this does not quite
feel right" ;-). Is it hard to arrange things so that a process
does exactly one check_repo_format() during its lifetime?
Two more issues I've been thinking about:
1. The core.repositoryformatversion scheme assumes and relies on
that at least .git/config would stay forward compatible. But
if you cover unconditionally the three main entry points, I
suspect "git-var" or "git-config-set --get" would stop
working in a wrong repository. I think the scripts and
Porcelains need a way to check if we are on a repository from
the correct vintage before running other low-level commands,
so one of these commands probably needs to be made to work
without check_repo_format() dying. Or we could introduce a
new command 'git-check-repo-format' for this specific
purpose, and special case only that one.
2. Some commands are read-only and are handy for problem
diagnosis (e.g. cat-file), so it _might_ make sense to allow
them to attempt running in a newer repository (which may well
fail due to repository format difference). I am not sure
about the merit of doing that outweighs the complexity,
though. What you did covers _everything_ uniformly, and
certainly is simpler, easier to explain, and nicer.
^ permalink raw reply
* $Revision$ keyword replacement?
From: Santi Béjar @ 2005-11-22 17:36 UTC (permalink / raw)
To: Git Mailing List
[-- Attachment #1: Type: text/plain, Size: 887 bytes --]
Hello:
I'm looking for a $Revision$ replacement. I know, in git all that
matters are the contents. So the equivalent in git could be:
How similar this file is to the file $path in these commit(s)?
What I use now is an ugly shell script (as you might known
I'm not a C programer), see the attatchment.
So I get:
311bf741d16ee5d0166639a4fafd9194e504aad3:R094 introduccio.tex sim.tex
b764ee049289f5157238596cccc7a4e15f23896c:R094 introduccio.tex sim.tex
f293e1ab1dfb7ff60cb269ed5b18f9222099a752:R098 introduccio.tex sim.tex
f14c4b2d6799354d376e2026a5b7a85ccce48284:R098 introduccio.tex sim.tex
183af55606310218ca0e9a735dad0827acea7910:R098 introduccio.tex sim.tex
73fd2531cce3db0ca804577fb29f87f413fc4563:R096 introduccio.tex sim.tex
Do you know any other (or better :) way to do it?
Thanks
Santi
[-- Attachment #2: git-find-sim --]
[-- Type: text/plain, Size: 1052 bytes --]
#!/bin/sh
##
## find-sim finds the similarity of one file with the files in the
## selected commits
##
## The arguments are:
## $1 - file to compare
## $n n>2 - arguments for git-rev-list to select the commits
##
. git-sh-setup || die "Not a git archive."
[ ! -e $1 ] && exit 1
file=$1
shift
case $GIT_DIR in
/*)
GIT_DIR_ORIG=$GIT_DIR;;
*)
GIT_DIR_ORIG=$PWD/$GIT_DIR
esac
tmp=`mktemp -t -d git-find-sim.XXXXXXX`
GIT_DIR_TEMP=$tmp
mkdir -p $GIT_DIR_TEMP/objects/info
[ -e $GIT_DIR_ORIG/objects/info/alternates ] &&
cp $GIT_DIR_ORIG/objects/info/alternates $GIT_DIR_TEMP/objects/info/alternates
echo $GIT_DIR_ORIG/objects >> $GIT_DIR_TEMP/objects/info/alternates
export GIT_DIR=$GIT_DIR_TEMP
trap "rm -rf $tmp" 0 1 2 3 15
git update-index --add $file || exit 1
tree=`git-write-tree`
rev_arg=`GIT_DIR=$GIT_DIR_ORIG git-rev-parse --default HEAD --revs-only "$@"`
revs=`GIT_DIR=$GIT_DIR_ORIG git-rev-list $rev_arg`
for i in $revs; do
git diff-tree --name-status $i -C $tree | grep $file |
sed "s/^/$i:/"
done
^ permalink raw reply
* Re: Get rid of .git/branches/ and .git/remotes/?
From: Josef Weidendorfer @ 2005-11-22 17:31 UTC (permalink / raw)
To: git
In-Reply-To: <200511210026.30280.Josef.Weidendorfer@gmx.de>
On Monday 21 November 2005 00:26, Josef Weidendorfer wrote:
> On Sunday 20 November 2005 19:09, Linus Torvalds wrote:
> > [branches.origin]
> > url = master.kernel.org:/pub/scm/git/git.git
> > pull = master:origin
>
> Two things:
> * base names are case insensitive. Repository shortcuts are case
> sensitive (and head names, too)
> * to get rid of .git/branches/XXX, XXX has to be allowed as
> base name. But XXX can contain anything a head name can (including ".").
>
> Not really a problem. Use the ' for ' syntax:
>
> [remotes]
> url = master.kernel.org:/pub/scm/git/git.git for origin
> pull = master:origin for origin
>
> Not really nice. We can not have "for" as head name.
What about another syntax for multivalue vars?
Proposed syntax:
[remote:origin]
url = master.kernel.org:/pub/scm/git/git.git
pull = master:origin
For this, the parser could return
remote.url = master.kernel.org:/pub/scm/git/git.git for origin
remote.pull = master:origin for origin
By introducing such a scheme, multi-value vars would fit perfectly
for use for .git/remotes or .git/branches.
Josef
^ permalink raw reply
* Re: auto-packing on kernel.org? please?
From: Carl Baldwin @ 2005-11-22 17:25 UTC (permalink / raw)
To: Linus Torvalds; +Cc: H. Peter Anvin, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0511211110480.13959@g5.osdl.org>
On Mon, Nov 21, 2005 at 11:24:11AM -0800, Linus Torvalds wrote:
> NOTE! Since that email, "git repack" has gotten a "local" option (-l),
> which is very useful if the repositories have pointers to alternates.
>
> So do
>
> git repack -l
>
> instead, to get much better packs (and "-a -d" for the full case, of
> course).
I'm assuming that this option will have no effect on a repository with
no alternates file.
> Other that than, the old email suggestion should still be fine.
[snip]
> You can certainly do that if you are nervous. It might even be a good
> idea: just for fun, I just did
>
> git clone -l git git-clone
> cd git-clone
>
> # pick an object at random
> rm .git/objects/f7/c3d39fe3db6da3a307da385a7a1cb563ed15f7
>
> git repack -a -d
>
> and it said:
>
> error: Could not read f7c3d39fe3db6da3a307da385a7a1cb563ed15f7
> fatal: bad tree object f7c3d39fe3db6da3a307da385a7a1cb563ed15f7
>
> but then it created the pack _anyway_, and said:
>
> Packing 27 objects
> Pack pack-13bfca704078175c1c1c59964553b14f7b952651 created.
>
> and happily removed all the old ones.
>
> So right now, repacking a broken archive can actually break it even more.
Interesting.
> NOTE! Your "git verify-pack" wouldn't even catch this: the _pack_ is fine,
> it's just incomplete.
In my opinion, git repack did the right thing in creating the pack even
if it is more broken. Starting with a broken repository was the real
problem. git repack shouldn't need to worry too much about it.
Looking at it from the nervous repository admin's point of view I think
he would want to make sure that the repository is good to begin with. I
think this should be left up to the repository owner and maybe not git
repack. Although, the check that you do following this is probably a
good idea.
> Of course, this only happens if the repository was broken to begin with,
> so arguably it's not that bad. But it does show that git-repack should be
> more careful and return an error more aggressively.
>
> Can anybody tell me how to do that sanely? Right now we do
>
> ..
> name=$(git-rev-list --objects $rev_list $(git-rev-parse $rev_parse) |
> git-pack-objects --non-empty $pack_objects .tmp-pack) ||
> exit 1
> ..
>
> and the thing is, the "git-pack-objects" thing is happy, it's the
> "git-rev-list" that fails. So because the last command in the pipeline
> returns ok, we think it all is ok..
>
> (This is one of the reasons I much prefer working in C over working in
> shell: it may be twenty times more lines, but when you have a problem, the
> fix is always obvious..)
>
> Anyway, with that fixed, a "git repack" in many ways would be a mini-fsck,
> so it should be very safe in general. Modulo any other bugs like the
> above.
>
> Linus
*NOTE* There is one question that I feel remains unanswered. Is it
possible to split up the repack -a and repack -d so that the nervous
repository owner can insert a git verify-pack in the middle.
I'm not nearly this nervous about repositories that I keep for myself
but I have ownership of some repositories on which many people may
depend. I will feel better if I can verify the pack separately from
git-repack before I do the (potentially destructive) -d to remove old
packs.
I don't mean to say that I don't trust git repack to do the right thing.
Fundamentally, I just think that I shouldn't depend on it to do the
right thing in order to avoid corruption in my repository.
Carl
PS I love that the git object store is designed so that object files
never *need* to be removed, renamed, modified or otherwise touched in
any way after being written to disk. I think this makes git inherently
extremely safe from corruption unlike many other older repository
designs. The only thing that breaks this inherent safety is the desire
to pack repositories to avoid bloat.
That is why I want to be a little paranoid when I do the repacking. I
want to maintain some inherent safety in the process that I use to pack
them. This kind of inherent safety is much more valuable then even the
highest quality code written to actually do the packing.
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Carl Baldwin Systems VLSI Laboratory
Hewlett Packard Company
MS 88 work: 970 898-1523
3404 E. Harmony Rd. work: Carl.N.Baldwin@hp.com
Fort Collins, CO 80525 home: Carl@ecBaldwin.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
^ permalink raw reply
* Re: auto-packing on kernel.org? please?
From: Linus Torvalds @ 2005-11-22 17:05 UTC (permalink / raw)
To: Catalin Marinas
Cc: Chuck Lever, Carl Baldwin, H. Peter Anvin, Git Mailing List
In-Reply-To: <b0943d9e0511220613h5978a600l@mail.gmail.com>
On Tue, 22 Nov 2005, Catalin Marinas wrote:
>
> > The preferred way would be to just list the references somewhere under
> > .git/refs/stgit, in which case fsck and repack should pick them up
> > automatically (so clearly stgit doesn't do that right now ;).
>
> I thought about adding .git/refs/patches/<branch>/* files
> corresponding to the every StGIT patch. Are the above git commands
> looking at all depths in the .git/refs/ directory?
Yes. Or at least they're supposed to. If they are not, it's a bug
regardless, and we'll fix it.
> The 'git repack -a' command would include the applied patches in the
> newly created pack but leave out the unapplied ones. It would be even
> better to leave all of them out since the StGIT patches are frequently
> changed but an independent mechanism for this would complicate GIT -
> 'git repack' shouldn't pack any of the objects found in
> .git/refs/patches/, even if they are reachable via .git/refs/heads/*
> (and maybe call the patches directory something like
> .git/refs/unpackable or volatile).
If we have some default location (and .git/refs/patches/ sounds good), we
can make git do the right thing - find them for git-fsck-objects, and
ignore them for git-repack.
Linus
^ permalink raw reply
* Re: [QUESTION] Access to a huge GIT repository.
From: Junio C Hamano @ 2005-11-22 17:06 UTC (permalink / raw)
To: Franck; +Cc: git
In-Reply-To: <cda58cb80511220240u45267b18o@mail.gmail.com>
Franck <vagabon.xyz@gmail.com> writes:
> It seems that the "lite" repository can't be used as a working
> repository.
>...
> Do you have any clues ?
Were you around on this list, around the beginning of this
month? The thread that starts here may be of interest:
http://marc.theaimsgroup.com/?l=git&m=113089701819420
I think (although I do not exactly know how the "lite"
repository was constructed) what you are doing is similar to
what I called "shallow clone" there in that thread. At the end
of the thread I think I listed what you can and cannot do in
such an incomplete repository.
^ permalink raw reply
* Re: strange cg-add problem
From: Junio C Hamano @ 2005-11-22 16:55 UTC (permalink / raw)
To: Nico -telmich- Schottelius; +Cc: git
In-Reply-To: <20051122123058.GB19989@schottelius.org>
Nico -telmich- Schottelius <nico-linux-git@schottelius.org> writes:
> srwali01:/home/server/backup/db# ~/db-dump.sh
> /home/server/backup/db//2005-11-22/13:26 does not exist or is not a regular file
> cg-add: warning: not all files could be added
> cg-commit: Nothing to commit
I do not use cogito, but I suspect it would work if you change
your script to avoid double slash between db and 2005-11-22.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox