* Re: Importing Mozilla CVS into git
From: Martin Langhoff @ 2006-06-02 4:16 UTC (permalink / raw)
To: Jon Smirl; +Cc: Keith Packard, git
In-Reply-To: <9e4733910606012100s7ace4721le6fbfbcaadfb6c43@mail.gmail.com>
On 6/2/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> It is going to have to be native Windows to move some of the
> developers over. They are true blue MS types that won't touch anything
> close to Unix so cygwin is out.
As others have pointed out, you have git-cvsserver which emulates a
CVS server on top of GIT, so it can be used with (almost any) CVS
client. They will be 2nd class citizens however...
cheers,
martin
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Pavel Roskin @ 2006-06-02 4:39 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Jon Smirl, Keith Packard, git
In-Reply-To: <20060602041107.GA5429@spearce.org>
On Fri, 2006-06-02 at 00:11 -0400, Shawn Pearce wrote:
> GIT today requires not only a decent UNIX shell but also, GNU tools,
> Perl and Python. Porting to Solaris has recently had some more
> effort put into it to remove some of the GNU tool dependencies but
> perhaps one of the most important features (git-merge-recursive)
> is a Python script.
The great thing about git is that it's modular. A single utility can be
replaced and retested in the same environment, without having to rewrite
the rest of the scripts. A dedicated programmer with good C and Python
skills could rewrite git-merge-recursive.py in C in 2 days, I believe.
Add a few days of bug fixing, of course.
Dependency on Cygwin, Perl and Python is too much. Windows is becoming
a legacy system in some circles, and it may run on legacy hardware. Yet
it's irreplaceable as a testing platform for many projects.
I really need to rewrite git-clean in C, since it doesn't handle
embedded newlines properly.
--
Regards,
Pavel Roskin
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Shawn Pearce @ 2006-06-02 4:44 UTC (permalink / raw)
To: Pavel Roskin; +Cc: Jon Smirl, Keith Packard, git
In-Reply-To: <1149223164.2443.33.camel@dv>
Pavel Roskin <proski@gnu.org> wrote:
> On Fri, 2006-06-02 at 00:11 -0400, Shawn Pearce wrote:
>
> > GIT today requires not only a decent UNIX shell but also, GNU tools,
> > Perl and Python. Porting to Solaris has recently had some more
> > effort put into it to remove some of the GNU tool dependencies but
> > perhaps one of the most important features (git-merge-recursive)
> > is a Python script.
>
> The great thing about git is that it's modular. A single utility can be
> replaced and retested in the same environment, without having to rewrite
> the rest of the scripts. A dedicated programmer with good C and Python
> skills could rewrite git-merge-recursive.py in C in 2 days, I believe.
> Add a few days of bug fixing, of course.
Heh. Funny you should mention that. I was just thinking a few
minutes ago about working on that exact change...
> Dependency on Cygwin, Perl and Python is too much. Windows is becoming
> a legacy system in some circles, and it may run on legacy hardware. Yet
> it's irreplaceable as a testing platform for many projects.
Its already legacy to me. Heck its 2006 and my work desktop still
says something about 2000 when I login. :-)
--
Shawn.
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Jon Smirl @ 2006-06-02 4:44 UTC (permalink / raw)
To: Pavel Roskin; +Cc: Shawn Pearce, Keith Packard, git
In-Reply-To: <1149223164.2443.33.camel@dv>
On 6/2/06, Pavel Roskin <proski@gnu.org> wrote:
> Dependency on Cygwin, Perl and Python is too much. Windows is becoming
> a legacy system in some circles, and it may run on legacy hardware. Yet
> it's irreplaceable as a testing platform for many projects.
80% of Mozilla commiters are running Windows. Some are OS bilingual
but many are not.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: More information on git for Cygwin x86-64
From: Alex Riesen @ 2006-06-02 6:37 UTC (permalink / raw)
To: Tim O'Callaghan, git
In-Reply-To: <20060601234410.GA3408@Zangband>
On 6/2/06, Tim O'Callaghan <timo@dspsrv.com> wrote:
> After some investigation, it looks like the problem is to do with the
> index.lock file. I think it may have something to do with with the
> rename function in index.c.
Does setting "NO_MMAP = YesPlease" in config.mak helps you?
The problem this option solves has to do with stupid open
file behaviour in windows.
^ permalink raw reply
* Re: HTTP questions
From: Junio C Hamano @ 2006-06-02 6:38 UTC (permalink / raw)
To: Nick Hengeveld; +Cc: git
In-Reply-To: <20060601232437.GD12261@reactrix.com>
Nick Hengeveld <nickh@reactrix.com> writes:
> - "git push" seems to pass --thin by default to http-push, which
> subsequently barfs because that's not a valid http-push option.
> Should it be? Should it be silently ignored? Should git-push not
> default to --thin when pushing with HTTP transport?
The way I understand http-push works is that it does not use
packed transfer, so I presume even if we give --thin as a hint
it cannot even take advantage of it. Probably we should stop
passing --thin to http transport (git native one uses it as a
cue that it can generate baseless deltas in the resulting pack).
> - when I clone, http-fetch outputs a whole bunch of
> "error: Could not read ..." messages - is that expected?
The clone over http seems to be severely broken in "next" right
now. The one in "master" seems to be OK. bisecting suggests
the breakage is coming from the tree parser rewrite.
bisect points at 136f2e548a34f1f504b0f062f87ddf33e8d6e83b.
^ permalink raw reply
* Re: What's in git.git (part #2)
From: Junio C Hamano @ 2006-06-02 6:40 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20060602023545.GA5039@spearce.org>
Shawn Pearce <spearce@spearce.org> writes:
> I find it useful to track what I've sent to you just in case I
> screw up some ref somewhere. I like knowing that if I perform a
> bad update-ref call (which I'm prone to do sometimes) that I can
> recover quickly as the log exists.
I find it interesting to be able to say:
$ git log next@{yesterday}..next
I often find myself getting curious to see:
$ git reflog next
Wed May 31 14:23:58 2006 -0700
62b693a... Merge branch 'master' into next
Wed May 31 14:26:39 2006 -0700
422dfaf... Merge branch 'lt/tree-2' into next
Wed May 31 15:14:58 2006 -0700
100c25f... Merge branch 'ff/svnimport' into next
Wed May 31 15:23:54 2006 -0700
a25963b... Merge branch 'jc/fmt-patch' into next
...
The latter is probably not so useful in practice -- I suspect
that I would want to see such a list only while I am interested
in how well reflog works.
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Junio C Hamano @ 2006-06-02 6:40 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0606011809401.5498@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> You're much better off using "gitk --all" if you want to see the result,
> the "show-branch" this is really broken. It is using the old algorithm
> that we used to use for "git-rev-tree", and got rid of about a year ago
> there in favour of git-rev-list ;)
Are you sure about it? My recollection is it uses the
merge-base logic, naturally enhanced for multiple heads.
And enhancing it to support more than one int wide bitmap should
not be too difficult, although looking at the output would be
very taxing for human eye, so I do not know if it is worth it.
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Johannes Schindelin @ 2006-06-02 7:46 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Pavel Roskin, Jon Smirl, Keith Packard, git
In-Reply-To: <20060602044404.GB5429@spearce.org>
Hi,
On Fri, 2 Jun 2006, Shawn Pearce wrote:
> Pavel Roskin <proski@gnu.org> wrote:
> > On Fri, 2006-06-02 at 00:11 -0400, Shawn Pearce wrote:
> >
> > > GIT today requires not only a decent UNIX shell but also, GNU tools,
> > > Perl and Python. Porting to Solaris has recently had some more
> > > effort put into it to remove some of the GNU tool dependencies but
> > > perhaps one of the most important features (git-merge-recursive)
> > > is a Python script.
> >
> > The great thing about git is that it's modular. A single utility can be
> > replaced and retested in the same environment, without having to rewrite
> > the rest of the scripts. A dedicated programmer with good C and Python
> > skills could rewrite git-merge-recursive.py in C in 2 days, I believe.
> > Add a few days of bug fixing, of course.
>
> Heh. Funny you should mention that. I was just thinking a few
> minutes ago about working on that exact change...
I thought about this a couple of weeks ago. I recalled to have read
something about the principle: if there is more than one merge-base
candidate, it starts by merging the merge-base candidates until there is
only one 'virtual' merge-base candidate.
However, looking at the code I fainted. Sure, a lot should be way easier
in C, because the functions are already there, _but_ it seemed too much
work for one afternoon nevertheless (and I did not have more time to
spare).
Ciao,
Dscho
^ permalink raw reply
* Re: git reset --hard not removing some files
From: Jakub Narebski @ 2006-06-02 8:16 UTC (permalink / raw)
To: git
In-Reply-To: <BAYC1-PASMTP02F2FF1D978A880AFDDA1DAE900@CEZ.ICE>
Sean wrote:
> On Thu, 01 Jun 2006 19:21:19 +0200
> Jakub Narebski <jnareb@gmail.com> wrote:
>
>> Do you think it is _frequently_ asked question, worth adding
>> to http://git.or.cz/gitwiki/GitFaq ?
>
> Not sure if it needs to be in the FAQ at this point, but i guess it couldn't
> hurt either.
Added. See: http://git.or.cz/gitwiki/GitFaq#reset-hard-leaving-files
Feel free to add examples, correct information, rework it etc.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: git reset --hard not removing some files
From: Martin Waitz @ 2006-06-02 9:37 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0606010918060.5498@g5.osdl.org>
[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]
hoi :)
On Thu, Jun 01, 2006 at 09:21:38AM -0700, Linus Torvalds wrote:
> Those files were _never_ tracked.
You are right, I used the wrong demonstration to show my problem.
I had one problem like this in the linux-kernel and tried to
reproduce them in the git repository but did it wrong...
GIT reset seems to have a problem when a file is tracked and ignored
at the same time.
This fails:
diff --git a/t/t7101-reset.sh b/t/t7101-reset.sh
index a919140..865e0f6 100755
--- a/t/t7101-reset.sh
+++ b/t/t7101-reset.sh
@@ -21,10 +21,12 @@ test_expect_success \
cp ../../COPYING path1/COPYING &&
cp ../../COPYING COPYING &&
cp ../../COPYING path0/COPYING-TOO &&
+ echo COPYING > .gitignore &&
git-add path1/path2/COPYING &&
git-add path1/COPYING &&
git-add COPYING &&
git-add path0/COPYING-TOO &&
+ git-add .gitignore &&
git-commit -m change -a'
test_expect_success \
This hit me as the Linux kernel .gitignore includes ".*" which matches
all the .gitignore files and so they are not removed when you go
back to 2.6.13. But with the new git checks regarding files in
the working dir, git then refuses to pull in newer Linux versions
because it would overwrite ".gitignore".
--
Martin Waitz
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply related
* Re: git reset --hard not removing some files
From: Sean @ 2006-06-02 10:08 UTC (permalink / raw)
To: Martin Waitz; +Cc: torvalds, git
In-Reply-To: <20060602093736.GL14325@admingilde.org>
On Fri, 2 Jun 2006 11:37:36 +0200
Martin Waitz <tali@admingilde.org> wrote:
Hi Martin,
> GIT reset seems to have a problem when a file is tracked and ignored
> at the same time.
I tried to reproduce this problem, but couldn't.
> This fails:
>
> diff --git a/t/t7101-reset.sh b/t/t7101-reset.sh
> index a919140..865e0f6 100755
> --- a/t/t7101-reset.sh
> +++ b/t/t7101-reset.sh
> @@ -21,10 +21,12 @@ test_expect_success \
> cp ../../COPYING path1/COPYING &&
> cp ../../COPYING COPYING &&
> cp ../../COPYING path0/COPYING-TOO &&
> + echo COPYING > .gitignore &&
> git-add path1/path2/COPYING &&
> git-add path1/COPYING &&
> git-add COPYING &&
> git-add path0/COPYING-TOO &&
> + git-add .gitignore &&
> git-commit -m change -a'
>
> test_expect_success \
This fails because git-add refuses to mark an ignored file as tracked.
So in your altered test above the COPYING file is never tracked.
git-reset then does the proper thing and refuses to remove it. So it's
still not demonstrating the case you mention below.
> This hit me as the Linux kernel .gitignore includes ".*" which matches
> all the .gitignore files and so they are not removed when you go
> back to 2.6.13. But with the new git checks regarding files in
> the working dir, git then refuses to pull in newer Linux versions
> because it would overwrite ".gitignore".
Strange.. this little test shows that tracked files that are ignored
do get removed when resetting --hard...
$ mkdir tryit ; cd tryit
$ git init-db
defaulting to local storage area
$ touch one ; git add one
$ git commit -a -m"Initial"
Committing initial tree 5fcffbd6e4c5c5b8d81f5e9314b20e338e3ffff5
$ touch .gitignore
$ git add .gitignore
$ git commit -a -m"now we have gitignore"
$ echo '.*' >> .gitignore
$ git commit -a -m'now git ignores all .* files'
$ ls -l .gitignore
-rw-r--r-- 1 sean sean 3 Jun 2 06:01 .gitignore
$ git reset --hard master~2
$ ls -l .gitignore
ls: .gitignore: No such file or directory
Sean
^ permalink raw reply
* Re: [PATCH] A Perforce importer for git.
From: Alex Riesen @ 2006-06-02 13:09 UTC (permalink / raw)
To: Sean; +Cc: git
In-Reply-To: <BAYC1-PASMTP117796558F3B42B2C18DBCAE980@CEZ.ICE>
On 5/25/06, Sean <seanlkml@sympatico.ca> wrote:
>
> Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca>
> ---
> Documentation/git-p4import.txt | 165 ++++++++++++++++++
> git-p4import.py | 357 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 522 insertions(+), 0 deletions(-)
BTW, can I suggest to import the _currently_ synced state?
The reason is that because of the way how Perforce is done
its working directories (views, aka clients) are often a
horrible mix of occasionally synced files to some random
versions. No one actually uses "p4 sync" for whole project here
where I work, because it is absolutely useless at this level
(updated files have abolutely no relevance at head revision,
which is what you get by syncing without strictly specifying
a revision). So a working state is stitched together from
a lot of "mappings": perforce path to local path -> revision.
That state can be actually worked on (up until you have
to commit something, that is not possible except on head).
That is why I actually can't sync before importing, but have
to use the method Junio described for CVS:
p4 sync files...#revisions
git add new-files...
git commit -a
(simplified. There are problems with case-sensitivity and that
stupid read-only attribute perfoce uses as sync marker).
^ permalink raw reply
* [PATCH] format-patch: resurrect extra headers from config
From: Johannes Schindelin @ 2006-06-02 13:21 UTC (permalink / raw)
To: git, junkio
Once again, if you have
[format]
headers = "Origamization: EvilEmpire\n"
format-patch will add these headers just after the "Subject:" line.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
---
builtin-log.c | 20 ++++++++++++++++++++
log-tree.c | 8 +++++---
revision.h | 1 +
3 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/builtin-log.c b/builtin-log.c
index ac4822d..6612f4c 100644
--- a/builtin-log.c
+++ b/builtin-log.c
@@ -85,6 +85,23 @@ static int istitlechar(char c)
(c >= '0' && c <= '9') || c == '.' || c == '_';
}
+static char *extra_headers = NULL;
+static int extra_headers_size = 0;
+
+static int git_format_config(const char *var, const char *value)
+{
+ if (!strcmp(var, "format.headers")) {
+ int len = strlen(value);
+ extra_headers_size += len + 1;
+ extra_headers = realloc(extra_headers, extra_headers_size);
+ extra_headers[extra_headers_size - len - 1] = 0;
+ strcat(extra_headers, value);
+ return 0;
+ }
+ return git_default_config(var, value);
+}
+
+
static FILE *realstdout = NULL;
static char *output_directory = NULL;
@@ -162,6 +179,9 @@ int cmd_format_patch(int argc, const cha
rev.ignore_merges = 1;
rev.diffopt.output_format = DIFF_FORMAT_PATCH;
+ git_config(git_format_config);
+ rev.extra_headers = extra_headers;
+
/*
* Parse the arguments before setup_revisions(), or something
* like "git fmt-patch -o a123 HEAD^.." may fail; a123 is
diff --git a/log-tree.c b/log-tree.c
index e86e16b..ebb49f2 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -51,7 +51,7 @@ void show_log(struct rev_info *opt, stru
int abbrev_commit = opt->abbrev_commit ? opt->abbrev : 40;
const char *extra;
int len;
- char *subject = NULL, *after_subject = NULL;
+ const char *subject = NULL, *extra_headers = opt->extra_headers;
opt->loginfo = NULL;
if (!opt->verbose_header) {
@@ -100,6 +100,7 @@ void show_log(struct rev_info *opt, stru
static char subject_buffer[1024];
static char buffer[1024];
snprintf(subject_buffer, sizeof(subject_buffer) - 1,
+ "%s"
"MIME-Version: 1.0\n"
"Content-Type: multipart/mixed;\n"
" boundary=\"%s%s\"\n"
@@ -110,9 +111,10 @@ void show_log(struct rev_info *opt, stru
"Content-Type: text/plain; "
"charset=UTF-8; format=fixed\n"
"Content-Transfer-Encoding: 8bit\n\n",
+ extra_headers ? extra_headers : "",
mime_boundary_leader, opt->mime_boundary,
mime_boundary_leader, opt->mime_boundary);
- after_subject = subject_buffer;
+ extra_headers = subject_buffer;
snprintf(buffer, sizeof(buffer) - 1,
"--%s%s\n"
@@ -141,7 +143,7 @@ void show_log(struct rev_info *opt, stru
/*
* And then the pretty-printed message itself
*/
- len = pretty_print_commit(opt->commit_format, commit, ~0u, this_header, sizeof(this_header), abbrev, subject, after_subject);
+ len = pretty_print_commit(opt->commit_format, commit, ~0u, this_header, sizeof(this_header), abbrev, subject, extra_headers);
if (opt->add_signoff)
len = append_signoff(this_header, sizeof(this_header), len,
diff --git a/revision.h b/revision.h
index 75796bc..7d85b0f 100644
--- a/revision.h
+++ b/revision.h
@@ -61,6 +61,7 @@ struct rev_info {
int nr, total;
const char *mime_boundary;
const char *add_signoff;
+ const char *extra_headers;
/* special limits */
int max_count;
--
1.3.3.g83f2b-dirty
^ permalink raw reply related
* Re: [PATCH] A Perforce importer for git.
From: Sean @ 2006-06-02 13:43 UTC (permalink / raw)
To: Alex Riesen; +Cc: git
In-Reply-To: <81b0412b0606020609o24ee34b4p7b1bcd013136d49a@mail.gmail.com>
On Fri, 2 Jun 2006 15:09:45 +0200
"Alex Riesen" <raa.lkml@gmail.com> wrote:
> BTW, can I suggest to import the _currently_ synced state?
>
> The reason is that because of the way how Perforce is done
> its working directories (views, aka clients) are often a
> horrible mix of occasionally synced files to some random
> versions. No one actually uses "p4 sync" for whole project here
> where I work, because it is absolutely useless at this level
> (updated files have abolutely no relevance at head revision,
> which is what you get by syncing without strictly specifying
> a revision). So a working state is stitched together from
> a lot of "mappings": perforce path to local path -> revision.
> That state can be actually worked on (up until you have
> to commit something, that is not possible except on head).
Hey Alex,
I'd happily change the script to accommodate your needs if it's at all
reasonable. As you've no doubt gathered, the script is currently very
branch centric. It needs a mapping view from each p4 branch into the
local git directory. Then it does a sync //p4/mapping@{revision} for
each revision along each mapped branch and commits it into git.
Note that it never syncs specifically to head, it explicitly asks for
each revision along the branch and tt doesn't know anything about your
working state, it can only import commits.
Are you looking for the ability to create a single git branch which has
the history of the combined view of your stitched together client mappings
rather than the independent branches held by the server?
Sean
^ permalink raw reply
* Re: git reset --hard not removing some files
From: Martin Waitz @ 2006-06-02 14:17 UTC (permalink / raw)
To: Sean; +Cc: torvalds, git
In-Reply-To: <20060602060820.8eebe391.seanlkml@sympatico.ca>
[-- Attachment #1: Type: text/plain, Size: 326 bytes --]
hoi :)
On Fri, Jun 02, 2006 at 06:08:20AM -0400, Sean wrote:
> Strange.. this little test shows that tracked files that are ignored
> do get removed when resetting --hard...
ok, git must be behaving correctly then, I don't know what problem I ran into.
Must have been some kind of user error...
--
Martin Waitz
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: git reset --hard not removing some files
From: Junio C Hamano @ 2006-06-02 14:57 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git, Martin Waitz
In-Reply-To: <Pine.LNX.4.64.0606010918060.5498@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Thu, 1 Jun 2006, Martin Waitz wrote:
>>
>> I have the following problem:
>
> It's not a problem, it's a feature.
>...
> Those files were _never_ tracked.
I would agree in the reproduction recipe Martin gave there is no
problem but feature, but at the same time I suspect the recent
"reset --hard simplification" has introduced a true regression.
$ mkdir test || exit
$ cd test || exit
$ git init-db
defaulting to local storage area
$ echo init >file0
$ echo init >file1
$ git add file0 file1
$ git commit -m initial
Committing initial tree de84dc367842fdbbb3acad3b3ed252f8c984296f
$ git branch side
$ rm -f file1
$ echo second >file2
$ git add file2
$ git commit -a -m 'master adds file2 and deletes file1'
$ git checkout side
$ echo modified >file1
$ git commit -a -m 'side edits file1'
$ git checkout master
$ ls
file0 file2
$ git pull . side
Trying really trivial in-index merge...
fatal: Merge requires file-level merging
Nope.
Merging HEAD with 7934c9c383f611cf2b9895a46cf95b815569beef
Merging:
b684570dc1141552d0da950a18f2d84a3ffadbc1 master adds file2 and deletes file1
7934c9c383f611cf2b9895a46cf95b815569beef side edits file1
found 1 common ancestor(s):
598d6491f72b6057ca87683d43cf64b08d90ddaf initial
CONFLICT (delete/modify): file1 deleted in HEAD and modified in 7934c9c383f611cf2b9895a46cf95b815569beef. Version 7934c9c383f611cf2b9895a46cf95b815569beef of file1 left in tree.
Automatic merge failed; fix conflicts and then commit the result.
$ git ls-files -u
100644 b1b716105590454bfc4c0247f193a04088f39c7f 1 file1
100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 3 file1
$ ls
file0 file1 file2
$ git reset --hard
$ ls
file0 file1 file2
We used to remove file1 from the working tree in this case. One
of the most important reason to use "git reset --hard" is to
recover from a conflicted, failed merge.
Leaving file1 in the working tree around in this case has
unpleasant consequences. After the above:
$ git checkout side
fatal: Untracked working tree file 'file1' would be overwritten by merge.
^ permalink raw reply
* [PATCH] handle concurrent pruning of packed objects
From: Jeff King @ 2006-06-02 15:32 UTC (permalink / raw)
To: git
This patch causes read_sha1_file and sha1_object_info to re-examine the
list of packs if an object cannot be found. It works by re-running
prepare_packed_git() after an object fails to be found.
It does not attempt to clean up the old pack list. Old packs which are in
use can continue to be used (until unused by lru selection). New packs
are placed at the front of the list and will thus be examined before old
packs.
Signed-off-by: Jeff King <peff@peff.net>
---
This is a repost, since there was no response last time. Linus
indicated this approach was reasonable; see:
<Pine.LNX.4.64.0605300752430.5623@g5.osdl.org>
I tested this by making a simple repo with three commits: a, b, and c.
I ran git diff-tree --stdin, and then did the following:
1. fed 'a b' to diff-tree
2. ran git repack -a -d
3. fed 'b c' to diff-tree
Vanilla git complains about the lack of object, whereas this version
provides the correct diff-tree output.
sha1_file.c | 24 ++++++++++++++++++------
1 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index f77c189..696e53f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -626,12 +626,12 @@ static void prepare_packed_git_one(char
closedir(dir);
}
+static int prepare_packed_git_run_once = 0;
void prepare_packed_git(void)
{
- static int run_once = 0;
struct alternate_object_database *alt;
- if (run_once)
+ if (prepare_packed_git_run_once)
return;
prepare_packed_git_one(get_object_directory(), 1);
prepare_alt_odb();
@@ -640,7 +640,13 @@ void prepare_packed_git(void)
prepare_packed_git_one(alt->base, 0);
alt->name[-1] = '/';
}
- run_once = 1;
+ prepare_packed_git_run_once = 1;
+}
+
+static void reprepare_packed_git(void)
+{
+ prepare_packed_git_run_once = 0;
+ prepare_packed_git();
}
int check_sha1_signature(const unsigned char *sha1, void *map, unsigned long size, const char *type)
@@ -1212,9 +1218,12 @@ int sha1_object_info(const unsigned char
if (!map) {
struct pack_entry e;
- if (!find_pack_entry(sha1, &e))
- return error("unable to find %s", sha1_to_hex(sha1));
- return packed_object_info(&e, type, sizep);
+ if (find_pack_entry(sha1, &e))
+ return packed_object_info(&e, type, sizep);
+ reprepare_packed_git();
+ if (find_pack_entry(sha1, &e))
+ return packed_object_info(&e, type, sizep);
+ return error("unable to find %s", sha1_to_hex(sha1));
}
if (unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0)
status = error("unable to unpack %s header",
@@ -1256,6 +1265,9 @@ void * read_sha1_file(const unsigned cha
munmap(map, mapsize);
return buf;
}
+ reprepare_packed_git();
+ if (find_pack_entry(sha1, &e))
+ return read_packed_sha1(sha1, type, size);
return NULL;
}
--
1.3.3.g331f
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
----- End forwarded message -----
^ permalink raw reply related
* git-http-fetch segfault backtrace
From: Becky Bruce @ 2006-06-02 15:47 UTC (permalink / raw)
To: git
I've been having problems with git-http-fetch segfaulting for several
git versions now. I've been watching the list and have seen other
complaints about similar problems, but the issue I'm hitting seems to
be slightly different. Any insight anyone has here is greatly
appreciated. I've included what information I have, and if there's
any other information or experiments anyone would like me to try,
I'll be happy to give it a go.
Some information:
- the git version is top of tree as of 1pm 6/1. I pulled it from
http://www.kernel.org/pub/scm/git/git.git
- I am running on an i686 Linux machine: uname -a returns
Linux machinename 2.4.21-32.EL #1 Fri Apr 15 21:29:19 EDT 2005 i686
i686 i386 GNU/Linux
- my curl version is
curl 7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a
ipv6 zlib/1.1.4. git with the same curl version appears to work fine
on x86_64 systems here.
- As far as I can tell, this problem seems to have appeared sometime
after git 1.2.4
I compiled git without optimizations and ran git-http-fetch via gdb
to see what's going on. Here's the backtrace:
(gdb) run -v -a -w heads/html heads/html http://www.kernel.org/pub/
scm/git/git.git/
Starting program: /tmp/install/bin/git-http-fetch -v -a -w heads/html
heads/html http://www.kernel.org/pub/scm/git/git.git/
walk bb8fb05ed082c81af81f9eecf356f993e2ef83b7
walk f31d9f5bcd0a1c236d8277df39c74927917ffb8f
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
Program received signal SIGSEGV, Segmentation fault.
0x00eed07c in memcpy () from /lib/tls/libc.so.6
(gdb) bt
#0 0x00eed07c in memcpy () from /lib/tls/libc.so.6
#1 0x0804a6db in fread_buffer (ptr=0x9323a01, eltsize=1,
nmemb=16384, buffer=0xbfffb830)
at http.c:38
#2 0x0012c438 in Curl_getinfo () from /usr/lib/libcurl.so.2
#3 0x0012ce49 in Curl_readwrite () from /usr/lib/libcurl.so.2
#4 0x001314f2 in curl_multi_perform () from /usr/lib/libcurl.so.2
#5 0x0804afb9 in get_active_slot () at http.c:304
#6 0x0804d41d in fetch_indices (repo=0x9309498) at http-fetch.c:910
#7 0x0804d666 in fetch_pack (repo=0x9309498, sha1=0x9312390 "Ѥ???r#
\032??&?\aC??:?+'[?\005\b")
at http-fetch.c:977
#8 0x0804dd41 in fetch (sha1=0x9312390 "Ѥ???r#\032??&?\aC??:?+'[?\005
\b") at http-fetch.c:1129
#9 0x0804a4a9 in loop () at fetch.c:166
#10 0x0804a64f in pull (target=0xbfffefe1 "heads/html") at fetch.c:218
#11 0x0804e2c9 in main (argc=7, argv=0xbfffcae4) at http-fetch.c:1269
I also did some poking around, and it looks like the call to
fread_buffer that causes the problem (it's the 3rd call, in this
case) has a potentially bogus buffer->posn (at least, it seems a
little ridiculous to me).... Note that the previous call to fread
resulted in a 0 size.
Starting program: /tmp/install/bin/git-http-fetch -v -a -w heads/html
heads/html http://www.kernel.org/pub/scm/git/git.git/
[/_TOOLS_/plat/local-/ppc-/login]
Installing oss-/1.0.0
walk bb8fb05ed082c81af81f9eecf356f993e2ef83b7
walk f31d9f5bcd0a1c236d8277df39c74927917ffb8f
Getting alternates list for http://www.kernel.org/pub/scm/git/git.git/
Getting pack list for http://www.kernel.org/pub/scm/git/git.git/
Breakpoint 1, fread_buffer (ptr=0x914fa01, eltsize=1, nmemb=16384,
buffer=0xbfffb9a0)
at http.c:35
35 size_t size = eltsize * nmemb;
(gdb) step
36 if (size > buffer->size - buffer->posn)
(gdb) step
37 size = buffer->size - buffer->posn;
(gdb) p/x size
$1 = 0x4000
(gdb) p/x buffer->size
$2 = 0x5e
(gdb) p/x buffer->posn
$3 = 0x0
(gdb) c
Continuing.
Breakpoint 1, fread_buffer (ptr=0x914fa01, eltsize=1, nmemb=16384,
buffer=0xbfffb9a0)
at http.c:35
35 size_t size = eltsize * nmemb;
(gdb) step
36 if (size > buffer->size - buffer->posn)
(gdb) p/x size
$4 = 0x4000
(gdb) p/x buffer->size
$5 = 0x5e
(gdb) p/x buffer->posn
$6 = 0x5e
(gdb) c
Continuing.
Breakpoint 1, fread_buffer (ptr=0x914fa01, eltsize=1, nmemb=16384,
buffer=0xbfffb9a0)
at http.c:35
35 size_t size = eltsize * nmemb;
(gdb) step
36 if (size > buffer->size - buffer->posn)
(gdb) p/x size
$7 = 0x4000
(gdb) p/x buffer->size
$8 = 0x7
(gdb) p/x buffer->posn
$9 = 0xbfffcc54
(gdb)
We thought perhaps the problem was related to CURL MULTI, but we set
http.maxrequests to 1, we get the same behavior. FYI - if you turn
off USE_CURL_MULTI, git no longer builds. There is stuff that uses
the curlm variable that is not inside the ifdefs.
Thanks,
B
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Linus Torvalds @ 2006-06-02 15:53 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vac8wdpr5.fsf@assigned-by-dhcp.cox.net>
On Thu, 1 Jun 2006, Junio C Hamano wrote:
> > You're much better off using "gitk --all" if you want to see the result,
> > the "show-branch" this is really broken. It is using the old algorithm
> > that we used to use for "git-rev-tree", and got rid of about a year ago
> > there in favour of git-rev-list ;)
>
> Are you sure about it? My recollection is it uses the
> merge-base logic, naturally enhanced for multiple heads.
Well, it's all the same algorithm, where just the bit usage differs.
git-rev-tree is slightly closer, if only because the original
git-merge-base only did two heads, if I recall correctly (while
git-rev-tree did 16 - the ability of git-show-branch to do 29 came from
just using all the free bits rather than the high bits like rev-tree did)
> And enhancing it to support more than one int wide bitmap should
> not be too difficult, although looking at the output would be
> very taxing for human eye, so I do not know if it is worth it.
Yeah, I don't think there is any reason to really support it. If you have
more than a few heads, you really do need the graphical version to see
what is going on, and git-show-branch doesn't buy you anything.
Linus
^ permalink raw reply
* Re: [PATCH] handle concurrent pruning of packed objects
From: Junio C Hamano @ 2006-06-02 15:53 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20060602153223.GA4223@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> This is a repost, since there was no response last time. Linus
> indicated this approach was reasonable; see:
> <Pine.LNX.4.64.0605300752430.5623@g5.osdl.org>
I haven't forgotten about it, but I have been sick.
I am uncertain about not re-examining the packs it originally
thought it had. By prepending the new ones (and the same old
surviving ones) at the beginning you are effectively hiding the
old packs, which sounds reasonable in the usual case.
Also I suspect this might have funny interaction with the case
where there are hand-added packs (see how verify-pack does it).
We do not silently "fix" missing object problems we discover
there.
^ permalink raw reply
* Re: Importing Mozilla CVS into git
From: Junio C Hamano @ 2006-06-02 16:00 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0606020849390.5498@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> Yeah, I don't think there is any reason to really support it. If you have
> more than a few heads, you really do need the graphical version to see
> what is going on, and git-show-branch doesn't buy you anything.
The real reason it uses a bit per given ref is what it wants to
show is different from what gitk shows. It wants to show which
ones are reachable from which head on each commit -- in gitk the
user has to follow the line to find it out.
However, to track 300 branches, you would need a terminal with
360 columns or so, _and_ you have to count columns to see if a
given commit is reachable from the ref you are interested in, so
it is not useful at all in practice to do more than a handful
refs at a time.
^ permalink raw reply
* Re: [PATCH] handle concurrent pruning of packed objects
From: Junio C Hamano @ 2006-06-02 16:03 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <7vwtbzblkf.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> writes:
> Also I suspect this might have funny interaction with the case
> where there are hand-added packs (see how verify-pack does it).
> We do not silently "fix" missing object problems we discover
> there.
The last sentence should read 'We do not want to silently...'.
Sorry.
^ permalink raw reply
* Re: [PATCH] handle concurrent pruning of packed objects
From: Jeff King @ 2006-06-02 16:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwtbzblkf.fsf@assigned-by-dhcp.cox.net>
On Fri, Jun 02, 2006 at 08:53:52AM -0700, Junio C Hamano wrote:
> I am uncertain about not re-examining the packs it originally
> thought it had. By prepending the new ones (and the same old
> surviving ones) at the beginning you are effectively hiding the
> old packs, which sounds reasonable in the usual case.
That shouldn't make a difference for correctness, even if the old packs
are still there. If you have an object in two packs, then it doesn't
matter which one you pull it from. The main impacts I can think of are:
1. The old pack may already be mapped, and it would be more efficient
to use it. However, the new pack will be mapped on first use, so it
will be used from then on.
2. The pack list can grow without bound. However, for this to matter,
you'd have to do many prunes during the course of a single git
command.
> Also I suspect this might have funny interaction with the case
> where there are hand-added packs (see how verify-pack does it).
> We do not silently "fix" missing object problems we discover
> there.
I will take a look at this.
-Peff
^ permalink raw reply
* Re: [PATCH] handle concurrent pruning of packed objects
From: Junio C Hamano @ 2006-06-02 16:10 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20060602160456.GA8957@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> That shouldn't make a difference for correctness, even if the old packs
> are still there. If you have an object in two packs, then it doesn't
> matter which one you pull it from. The main impacts I can think of are:
> 1. The old pack may already be mapped, and it would be more efficient
> to use it. However, the new pack will be mapped on first use, so it
> will be used from then on.
> 2. The pack list can grow without bound. However, for this to matter,
> you'd have to do many prunes during the course of a single git
> command.
I agree 100% on "shouldn't" part. What I wonder is if everybody
works correctly if we mmap the same file (all available .idx are
mapped all the time, and we map .pack LRU) twice. But I realize
we have NO_MMAP configuration for unfortunate platforms to work
it around so that wouldn't be a big deal.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox