* Re: Horrible re-packing?
From: Linus Torvalds @ 2006-06-05 18:44 UTC (permalink / raw)
To: Junio C Hamano, Nicolas Pitre; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606050951120.5498@g5.osdl.org>
On Mon, 5 Jun 2006, Linus Torvalds wrote:
>
> Whaah! That nice 6.33MB pack-file exploded to 14.5MB!
>
> And it's possibly broken by the fact that we've been renaming things
> lately (ie the "rev-list.c" -> "builtin-rev-list.c" thing ends up not
> finding things)
No, it's even simpler.
The breakage is entirely mine, and due to the tree-walking conversion of
the "process_tree()" function.
In that function, we used to have a local "const char *name" that
_shadowed_ the incoming _argument_ with the same type, and the
tree-walking conversion did not notice that the inner "name" should have
been converted to "entry.path" - so it used the outer-level "name".
Gaah. We should probably use -Wshadow or something, which would hopefully
have warned about the re-use of the same variable name in two different
scopes.
Regardless, this fixes it.
Linus
---
diff --git a/builtin-rev-list.c b/builtin-rev-list.c
index 17c04b9..e885624 100644
--- a/builtin-rev-list.c
+++ b/builtin-rev-list.c
@@ -135,9 +135,9 @@ static struct object_list **process_tree
while (tree_entry(&desc, &entry)) {
if (S_ISDIR(entry.mode))
- p = process_tree(lookup_tree(entry.sha1), p, &me, name);
+ p = process_tree(lookup_tree(entry.sha1), p, &me, entry.path);
else
- p = process_blob(lookup_blob(entry.sha1), p, &me, name);
+ p = process_blob(lookup_blob(entry.sha1), p, &me, entry.path);
}
free(tree->buffer);
tree->buffer = NULL;
^ permalink raw reply related
* Re: Horrible re-packing?
From: Linus Torvalds @ 2006-06-05 19:03 UTC (permalink / raw)
To: Junio C Hamano, Nicolas Pitre; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051140530.5498@g5.osdl.org>
On this same thread..
This trivial patch not only simplifies the name hashing, it actually
improves packing for both git and the kernel.
The git archive pack shrinks from 6824090->6622627 bytes (a 3%
improvement), and the kernel pack shrinks from 108756213 to 108219021 (a
mere 0.5% improvement, but still, it's an improvement from making the
hashing much simpler!)
I think the hash function with its comment is self-explanatory:
/*
* This effectively just creates a sortable number from the
* last sixteen non-whitespace characters. Last characters
* count "most", so things that end in ".c" sort together.
*/
while ((c = *name++) != 0) {
if (isspace(c))
continue;
hash = (hash >> 2) + (c << 24);
}
return hash;
ie we just create a 32-bit hash, where we "age" previous characters by two
bits, so the last characters in a filename count most. So when we then
compare the hashes in the sort routine, filenames that end the same way
sort the same way.
It takes the subdirectory into account (unless the filename is > 16
characters), but files with the same name within the same subdirectory
will obviously sort closer than files in different subdirectories.
And, incidentally (which is why I tried the hash change in the first
place, of course) builtin-rev-list.c will sort fairly close to rev-list.c.
And no, it's not a "good hash" in the sense of being secure or unique, but
that's not what we're looking for. The whole "hash" thing is misnamed
here. It's not so much a hash as a "sorting number".
Comments?
Linus
----
pack-objects.c | 41 +++++++++++------------------------------
1 files changed, 11 insertions(+), 30 deletions(-)
diff --git a/pack-objects.c b/pack-objects.c
index 3590cd5..ae49fba 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -473,38 +473,19 @@ #define DIRBITS 12
static unsigned name_hash(struct name_path *path, const char *name)
{
- struct name_path *p = path;
- const char *n = name + strlen(name);
- unsigned hash = 0, name_hash = 0, name_done = 0;
-
- if (n != name && n[-1] == '\n')
- n--;
- while (name <= --n) {
- unsigned char c = *n;
- if (c == '/' && !name_done) {
- name_hash = hash;
- name_done = 1;
- hash = 0;
- }
- hash = hash * 11 + c;
- }
- if (!name_done) {
- name_hash = hash;
- hash = 0;
- }
- for (p = path; p; p = p->up) {
- hash = hash * 11 + '/';
- n = p->elem + p->len;
- while (p->elem <= --n) {
- unsigned char c = *n;
- hash = hash * 11 + c;
- }
- }
+ unsigned char c;
+ unsigned hash = 0;
+
/*
- * Make sure "Makefile" and "t/Makefile" are hashed separately
- * but close enough.
+ * This effectively just creates a sortable number from the
+ * last sixteen non-whitespace characters. Last characters
+ * count "most", so things that end in ".c" sort together.
*/
- hash = (name_hash<<DIRBITS) | (hash & ((1U<<DIRBITS )-1));
+ while ((c = *name++) != 0) {
+ if (isspace(c))
+ continue;
+ hash = (hash >> 2) + (c << 24);
+ }
return hash;
}
^ permalink raw reply related
* Re: Horrible re-packing?
From: Junio C Hamano @ 2006-06-05 19:37 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Nicolas Pitre, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051155000.5498@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> I think the hash function with its comment is self-explanatory:
>
> /*
> * This effectively just creates a sortable number from the
> * last sixteen non-whitespace characters. Last characters
> * count "most", so things that end in ".c" sort together.
> */
> while ((c = *name++) != 0) {
> if (isspace(c))
> continue;
> hash = (hash >> 2) + (c << 24);
> }
> return hash;
>
> ie we just create a 32-bit hash, where we "age" previous characters by two
> bits, so the last characters in a filename count most. So when we then
> compare the hashes in the sort routine, filenames that end the same way
> sort the same way.
IIRC, sometimes this function is called with path and name split
and sometimes with full path in name, depending on who calls you
(the latter happens for rev-list --object generated names, and
the former is for objects we extract ourselves from the --thin
base tree, or something like that). I suspect your patch may
break paths whose filename after the last slash is shorter than
16 bytes.
^ permalink raw reply
* Re: [PATCH] Fix git_setup_directory_gently when GIT_DIR is set
From: Junio C Hamano @ 2006-06-05 19:45 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, junkio
In-Reply-To: <Pine.LNX.4.63.0606051943540.29608@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> When calling git_setup_directory_gently, and GIT_DIR was set, it just
> ignored the variable nongit_ok.
Hmph. Is this really a breakage? That is, gently() is meant
for a case where you do not know if you even find a git
repository and tell it not to complain because you are prepared
for the case where you are not in a git repository.
If the environment has an incorrect GIT_DIR, I think that falls
into a different category. It is more like "the user or calling
script says we have GIT_DIR there but it is corrupt and
unusable".
I do not have a strong opinion on this, though. If you have
two commands in your script, the first of which does gently()
with such an environment, your change may allow that first
command to succeed, but if the second command does not say
nongit_ok, it would die() there anyway.
^ permalink raw reply
* Fix typo in tutorial-2.txt
From: Linus Torvalds @ 2006-06-05 19:47 UTC (permalink / raw)
To: J. Bruce Fields, Junio C Hamano, Git Mailing List
This should be obvious enough.
I didn't actually _test_ the tutorial, but if the old command worked,
something is really wrong!
Signed-off-by: Linus "Duh!" Torvalds <torvalds@osdl.org>
---
Documentation/tutorial-2.txt | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/Documentation/tutorial-2.txt b/Documentation/tutorial-2.txt
index 82c6922..894ca5e 100644
--- a/Documentation/tutorial-2.txt
+++ b/Documentation/tutorial-2.txt
@@ -136,7 +136,7 @@ The "tree" object here refers to the new
------------------------------------------------
$ git ls-tree d0492b36
100644 blob a0423896973644771497bdc03eb99d5281615b51 file.txt
-$ git cat-file commit a0423896
+$ git cat-file blob a0423896
hello world!
------------------------------------------------
^ permalink raw reply related
* Re: Horrible re-packing?
From: Linus Torvalds @ 2006-06-05 19:57 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nicolas Pitre, Git Mailing List
In-Reply-To: <7vy7wb4cmy.fsf@assigned-by-dhcp.cox.net>
On Mon, 5 Jun 2006, Junio C Hamano wrote:
>
> IIRC, sometimes this function is called with path and name split
> and sometimes with full path in name
Yeah, I was pretty confused by the whole hashing thing. Are you sure that
complexity is needed, it seems a bit overkill.
Linus
^ permalink raw reply
* [PATCH] builtin-push: don't pass --thin to HTTP transport
From: Nick Hengeveld @ 2006-06-05 20:02 UTC (permalink / raw)
To: git
git-http-push does not currently use packs to transfer objects.
Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
---
builtin-push.c | 20 +++++++++++---------
1 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/builtin-push.c b/builtin-push.c
index e530022..66b9407 100644
--- a/builtin-push.c
+++ b/builtin-push.c
@@ -214,7 +214,7 @@ static int do_push(const char *repo)
{
const char *uri[MAX_URI];
int i, n;
- int remote;
+ int common_argc;
const char **argv;
int argc;
@@ -231,23 +231,25 @@ static int do_push(const char *repo)
argv[argc++] = "--force";
if (execute)
argv[argc++] = execute;
- if (thin)
- argv[argc++] = "--thin";
- remote = argc;
- argv[argc++] = "dummy-remote";
- while (refspec_nr--)
- argv[argc++] = *refspec++;
- argv[argc] = NULL;
+ common_argc = argc;
for (i = 0; i < n; i++) {
int error;
+ int dest_argc = common_argc;
+ int dest_refspec_nr = refspec_nr;
+ const char **dest_refspec = refspec;
const char *dest = uri[i];
const char *sender = "git-send-pack";
if (!strncmp(dest, "http://", 7) ||
!strncmp(dest, "https://", 8))
sender = "git-http-push";
+ else if (thin)
+ argv[dest_argc++] = "--thin";
argv[0] = sender;
- argv[remote] = dest;
+ argv[dest_argc++] = dest;
+ while (dest_refspec_nr--)
+ argv[dest_argc++] = *dest_refspec++;
+ argv[dest_argc] = NULL;
error = run_command_v(argc, argv);
if (!error)
continue;
--
1.3.3.g423a-dirty
^ permalink raw reply related
* Re: Using pickaxe to track changed symbol CR4_FEATURES_ADDR
From: Junio C Hamano @ 2006-06-05 20:03 UTC (permalink / raw)
To: Thomas Glanzmann; +Cc: git
In-Reply-To: <20060605102627.GB24346@cip.informatik.uni-erlangen.de>
Thomas Glanzmann <sithglan@stud.uni-erlangen.de> writes:
> I am looking for the symbol CR4_FEATURES_ADDR which must be gone in one
> of the last kernel revision. Now how I do use pickaxe to track any
> changes that involve my missing symbol? Or is there a better way to
> track that change down?
None of the major recent versions seem to have the symbol.
: gitster; git grep -e CR4_FEATURES_ADDR \
v2.6.12-rc2 v2.6.12 v2.6.13 v2.6.14 v2.6.15 \
v2.6.16
and I did not get any google hits for "CR4_FEATURES_ADDR". Are
you spelling it right?
^ permalink raw reply
* Re: Gitk feature - show nearby tags
From: Junio C Hamano @ 2006-06-05 20:08 UTC (permalink / raw)
To: Jonas Fonseca; +Cc: Paul Mackerras, git
In-Reply-To: <7v3bek7589.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> writes:
> Jonas Fonseca <fonseca@diku.dk> writes:
>
> Whichever was the latest when I wrote the message. I see you
> have added a handful commits on it since then.
>
>>> - Press UP or DOWN and I can move the highlight to
>>> neighbouring commits. This is wonderful, but the lower
>>> pane does not follow this -- it keeps showing the original
>>> commit, and I have to say ENTER again.
>>
>> .. this unnecessary.
>
> Maybe I am misusing it then.
Sorry, I *was* misusing it. I was doing PageUp/PageDown. Duh.
I typed uparrow and downarrow while in the split view and it was
doing what I wanted to do.
^ permalink raw reply
* Re: Using pickaxe to track changed symbol CR4_FEATURES_ADDR
From: Randy.Dunlap @ 2006-06-05 20:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: sithglan, git
In-Reply-To: <7v8xob4bft.fsf@assigned-by-dhcp.cox.net>
On Mon, 05 Jun 2006 13:03:34 -0700 Junio C Hamano wrote:
> Thomas Glanzmann <sithglan@stud.uni-erlangen.de> writes:
>
> > I am looking for the symbol CR4_FEATURES_ADDR which must be gone in one
> > of the last kernel revision. Now how I do use pickaxe to track any
> > changes that involve my missing symbol? Or is there a better way to
> > track that change down?
>
> None of the major recent versions seem to have the symbol.
>
> : gitster; git grep -e CR4_FEATURES_ADDR \
> v2.6.12-rc2 v2.6.12 v2.6.13 v2.6.14 v2.6.15 \
> v2.6.16
>
> and I did not get any google hits for "CR4_FEATURES_ADDR". Are
> you spelling it right?
include/asm-i386/processor.h has names like:
/*
* Intel CPU features in CR4
*/
#define X86_CR4_VME 0x0001 /* enable vm86 extensions */
#define X86_CR4_PVI 0x0002 /* virtual interrupts flag enable */
#define X86_CR4_TSD 0x0004 /* disable time stamp at ipl 3 */
#define X86_CR4_DE 0x0008 /* enable debugging extensions */
#define X86_CR4_PSE 0x0010 /* enable page size extensions */
#define X86_CR4_PAE 0x0020 /* enable physical address extensions */
#define X86_CR4_MCE 0x0040 /* Machine check enable */
#define X86_CR4_PGE 0x0080 /* enable global pages */
#define X86_CR4_PCE 0x0100 /* enable performance counters at ipl 3 */
#define X86_CR4_OSFXSR 0x0200 /* enable fast FPU save and restore */
#define X86_CR4_OSXMMEXCPT 0x0400 /* enable unmasked SSE exceptions */
extern unsigned long mmu_cr4_features;
static inline void set_in_cr4 (unsigned long mask)
{
unsigned cr4;
mmu_cr4_features |= mask;
cr4 = read_cr4();
cr4 |= mask;
write_cr4(cr4);
}
static inline void clear_in_cr4 (unsigned long mask)
{
unsigned cr4;
mmu_cr4_features &= ~mask;
cr4 = read_cr4();
cr4 &= ~mask;
write_cr4(cr4);
}
but nothing exactly like you asked about.
---
~Randy
^ permalink raw reply
* Re: Using pickaxe to track changed symbol CR4_FEATURES_ADDR
From: Thomas Glanzmann @ 2006-06-05 20:16 UTC (permalink / raw)
To: Randy.Dunlap; +Cc: Junio C Hamano, git
In-Reply-To: <20060605131151.b8878c7c.rdunlap@xenotime.net>
Hello Randy,
I confused it. CR4_FEATURES_ADDR was in the glue code of parallels
binary only modules, but it sounded like a a MMU definition. Thanks a
lot. That was why my pickaxe did not work out in the first place.
However I have a workaround for the damn build problem.
See http://wwwcip.informatik.uni-erlangen.de/~sithglan/parallels/
Thanks,
Thomas
^ permalink raw reply
* Re: Gitk feature - show nearby tags
From: Jonas Fonseca @ 2006-06-05 20:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Paul Mackerras, git
In-Reply-To: <7v3bek7589.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote Sun, Jun 04, 2006:
> Jonas Fonseca <fonseca@diku.dk> writes:
>
> >> - I want to see the neighbouring commits, but UP or DOWN
> >> does not do what I naïvely expect. It scrolls the lower
> >> pane. I say TAB to go up.
> >
> > I wonder what tig version you are using. If you are using the tig
> > version from my git repo this should also be working to your
> > expectation, making ...
>
> Whichever was the latest when I wrote the message. I see you
> have added a handful commits on it since then.
Yes, I added your Makefile patch. Thanks for that one.
> >> - Press UP or DOWN and I can move the highlight to
> >> neighbouring commits. This is wonderful, but the lower
> >> pane does not follow this -- it keeps showing the original
> >> commit, and I have to say ENTER again.
> >
> > .. this unnecessary.
>
> Maybe I am misusing it then.
I admit the basic user controls might still have some rough spots.
Here is my take on what you are trying to do:
- When you open tig, UP/DOWN will move the cursor line.
- When you press ENTER on the commit you would like to see, it will
open the split view and move focus to the diff view. So if you press
Enter again it will start scrolling the diff view. This is much like
the way Mutt works.
- If you press UP/DOWN (while the diff view is focused) you will move
to previous or next commit in the main view (the one-line log view)
and load it in the diff view. That is, there is no reason to switch
to the main view unless you want to navigate to a commit without
repeatedly reloading the diff view which is clearly not what you are
requesting.
Is this not what you requested? That you can "see neighbouring commit"
by pressing UP and DOWN? But without having to press ENTER again.
Now, I've experienced some problems with ncurses and key detection but
only for Insert/Delete and Home/End. If UP/DOWN scrolls the diff view
something is terrible wrong.
> I like viewing the list in the upper and diff/log in the lower
> at the same time, and that is the primary reason I liked tig, so
> moving around in the commit list view and not seeing the
> diff/log updated in sync was major dissapointment at least for
> me.
Ok, well I can just make it optional, if you want the split view always
to be in sync, even while moving when the main view is in focus.
--
Jonas Fonseca
^ permalink raw reply
* Re: [PATCH] A Perforce importer for git.
From: Alex Riesen @ 2006-06-05 21:00 UTC (permalink / raw)
To: Sean; +Cc: git
In-Reply-To: <20060604100430.cb2789dd.seanlkml@sympatico.ca>
Sean, Sun, Jun 04, 2006 16:04:30 +0200:
> > I'm rather looking for a ability to manage a single branch where
> > import "sync" events appear as a merge of changes to the files
> > involved in the sync. I just haven't figured out yet how to "break" a
> > Perforce change into changes to single files and import that broken up
> > commit into git as a merge.
>
> Ahh, so what you're really asking is for a way to maintain the perforce
> merge history within git. Whereas the current p4import script just
> shows a linear history without any merges.
I assume that by "perforce merge" you understand the set of revisions
in the working directory. That's what you get in a typical corporate
environment with hundreds libraries and source files somehow stitched
together in a hope it'd work. It does, surprisingly often. There is
also another "merge" in perforce workflow - plain text merge of many
files, done manually in working directory and checked in afterwards.
I believe it wouldn't be possible to get a history of this merge,
because there is just no information about the merge anywhere.
> The problem is that Perforce doesn't merge at the commit level. It
> allows changes from other branches to be pulled one file at a time and
> from any rev level.
Right. Awkward.
> Now, even if you break those changes into one git commit per file per
> revision level (yuck!), you still couldn't use them to record Perforce
> merges. Git would still merge the entire history of such commits from
> the other branch whenever you tried to merge just one.
I think it's worse: you can't merge (as in git) anything because of
that salad from local (working) and remote (p4 server-side) pathnames.
> AFAICS, the best you could do would be to create cherry-picks, plucking
> just the commits from the other branch you want. However at that point
> you're not getting a git merge anyway and it doesn't seem to be any
> benefit beyond what the importer already does. Well, the importer
> _could_ make a comment in the commit message describing where each
> file change originated (ie. from which branch/rev). Would that help?
Don't think so, even if this is surely better detailed this way. I
still wont have the ability to merge branches. Maybe if every change
to every file gets it own commit one can use that information to
either cherry-pick the changes or fix the pathnames and apply that
patch? And a P4 change could be represented as a git-merge.
Like this:
P4:
Change 213412
a/foo.c #3
b/bar.c #6
Git:
+--commit abcdef ----+
| a/foo.c +3 lines |
base-+ commit deffff (merge, represents Change 213412)
| commit abcccd |
+--b/bar.c -3 lines -+
Now I can cherry-pick (or just copy) commit "abcdef" or commit
"abcccd", and still can find out what that "Change 213412" was all
about.
^ permalink raw reply
* Re: Gitk feature - show nearby tags
From: Jonas Fonseca @ 2006-06-05 21:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Paul Mackerras, git
In-Reply-To: <7vejy48wp5.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote Sun, Jun 04, 2006:
> I do not necessarily think an ascii-art is needed, nor an
> appropriate way to present it to the curses user.
With certain limitations, I think it could be useful for some
epositories. Optionally, of course.
> When the user wants to "view" a commit, you could show from which
> branch heads and from which tags the commit is reachable, and perhaps
> which tag is the latest among the ones reachable from that commit, as
> part of the commit detail information you display on the lower pane
> (log/diff view).
Thanks for recapping, I've added this to the TODO file.
--
Jonas Fonseca
^ permalink raw reply
* Re: Horrible re-packing?
From: Olivier Galibert @ 2006-06-05 21:14 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Nicolas Pitre, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051155000.5498@g5.osdl.org>
On Mon, Jun 05, 2006 at 12:03:31PM -0700, Linus Torvalds wrote:
> Comments?
Why don't you just sort the full path+filename with a strcmp variant
that starts by the end of the string for comparison? May at least be
simpler to understand.
OG.
^ permalink raw reply
* Re: Horrible re-packing?
From: Nicolas Pitre @ 2006-06-05 21:20 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051155000.5498@g5.osdl.org>
On Mon, 5 Jun 2006, Linus Torvalds wrote:
>
>
> On this same thread..
>
> This trivial patch not only simplifies the name hashing, it actually
> improves packing for both git and the kernel.
>
> The git archive pack shrinks from 6824090->6622627 bytes (a 3%
> improvement), and the kernel pack shrinks from 108756213 to 108219021 (a
> mere 0.5% improvement, but still, it's an improvement from making the
> hashing much simpler!)
OK here's the scoop. I still have a sample repo (I forget who it was
from) that used to exhibit a big packing size regression which was fixed
a while ago. I tend to test new packing strategies on that repo as well
since it has rather interesting characteristics that makes it pretty
sensitive to changes to name hashing and size filtering heuristics.
Before this hashing patch (including the rev-list fix):
$ git repack -a -f
Generating pack...
Done counting 46391 objects.
Deltifying 46391 objects.
100% (46391/46391) done
Writing 46391 objects.
100% (46391/46391) done
Total 46391, written 46391 (delta 7457), reused 38934 (delta 0)
Pack pack-7f766f5af5547554bacb28c0294bd562589dc5e7 created.
$ ll .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
-rw-rw-r-- 1 nico nico 39486095 Jun 5 16:28 .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
Now with this patch applied:
$ git repack -a -f
Generating pack...
Done counting 46391 objects.
Deltifying 46391 objects.
100% (46391/46391) done
Writing 46391 objects.
100% (46391/46391) done
Total 46391, written 46391 (delta 9920), reused 36447 (delta 0)
Pack pack-7f766f5af5547554bacb28c0294bd562589dc5e7 created.
$ ll .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
-rw-rw-r-- 1 nico nico 16150417 Jun 5 16:31 .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
In other words, the pack shrunk to less than half the size of the
previous one !
And yes fsck-objects still pass (I was doubtful at first).
Nicolas
^ permalink raw reply
* Re: Horrible re-packing?
From: Nicolas Pitre @ 2006-06-05 21:22 UTC (permalink / raw)
To: Olivier Galibert; +Cc: Linus Torvalds, Junio C Hamano, Git Mailing List
In-Reply-To: <20060605211436.GA58708@dspnet.fr.eu.org>
On Mon, 5 Jun 2006, Olivier Galibert wrote:
> On Mon, Jun 05, 2006 at 12:03:31PM -0700, Linus Torvalds wrote:
> > Comments?
>
> Why don't you just sort the full path+filename with a strcmp variant
> that starts by the end of the string for comparison? May at least be
> simpler to understand.
Much more expensive for both memory usage and CPU cycles.
Nicolas
^ permalink raw reply
* Re: Horrible re-packing?
From: Linus Torvalds @ 2006-06-05 21:27 UTC (permalink / raw)
To: Olivier Galibert; +Cc: Junio C Hamano, Nicolas Pitre, Git Mailing List
In-Reply-To: <20060605211436.GA58708@dspnet.fr.eu.org>
On Mon, 5 Jun 2006, Olivier Galibert wrote:
>
> Why don't you just sort the full path+filename with a strcmp variant
> that starts by the end of the string for comparison? May at least be
> simpler to understand.
That's actually what I was going to do, but we don't save the whole name,
just the sorting number.
(This is actually an area where saving space is important - we can easily
be working with hundreds of thousands or millions of objects, and we don't
want to keep the name of each of them around).
So the suggested hash sort is designed exactly to end up approximating
that ascii sort-from-end-of-string.
Linus
^ permalink raw reply
* Re: Horrible re-packing?
From: Linus Torvalds @ 2006-06-05 21:40 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051637490.24152@localhost.localdomain>
On Mon, 5 Jun 2006, Nicolas Pitre wrote:
>
> In other words, the pack shrunk to less than half the size of the
> previous one !
Ok, that's a bit more extreme than expected.
It's obviously great news, and says that the approach of sorting by
"reversed name" is a great heuristic, but at the same time it makes me
worry a bit that this thing that is supposed to be a heuristic ends up
being _so_ important from a pack size standpoint. I was happier when it
was more about saving a couple of percent.
Now, your repo may be a strange case, and it just happens to fit the
suggested hash, but on the other hand it's nice to see three totally
different repositories that all improve, albeit with wildly different
numbers.
I'm wondering if we could have some "incremental optimizer" thing that
would take a potentially badly packed archive, and just start looking for
better delta chain possibilities? That way we would still try to get a
good initial pack with some heuristic, but we could have people run the
incremental improver every once in a while looking for good deltas that it
missed due to the project not fitting the heuristics..
The fact that we normally do incremental repacking (and "-f" is unusual)
is obviously one thing that makes us less susceptible to bad patterns (and
is also what allows us to run the incremental optimizer - any good delta
choice will automatically percolate into subsequent versions, including
packs that have been cloned).
So the packing strategy itself seems to be very stable (and partly _due_
to the "optimization" to re-use earlier pack choices), but we currently
lack the thing that fixes up any initial bad assumptions in case they
happen.
Linus
^ permalink raw reply
* Re: [PATCH] Fix git_setup_directory_gently when GIT_DIR is set
From: Johannes Schindelin @ 2006-06-05 22:57 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vodx74ca9.fsf@assigned-by-dhcp.cox.net>
Hi,
On Mon, 5 Jun 2006, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > When calling git_setup_directory_gently, and GIT_DIR was set, it just
> > ignored the variable nongit_ok.
>
> Hmph. Is this really a breakage? That is, gently() is meant
> for a case where you do not know if you even find a git
> repository and tell it not to complain because you are prepared
> for the case where you are not in a git repository.
Yes, it is a breakage: in git-clone, line 212, we explicitely set GIT_DIR
(to the not-yet-existing repository path), and call git-init-db. Now, with
the alias thing we need to get the config if it exists, so we _got_ to
call gently(). Boom.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Fix git_setup_directory_gently when GIT_DIR is set
From: Junio C Hamano @ 2006-06-05 23:10 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0606060053440.25344@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi,
>
> On Mon, 5 Jun 2006, Junio C Hamano wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>> > When calling git_setup_directory_gently, and GIT_DIR was set, it just
>> > ignored the variable nongit_ok.
>>
>> Hmph. Is this really a breakage? That is, gently() is meant
>> for a case where you do not know if you even find a git
>> repository and tell it not to complain because you are prepared
>> for the case where you are not in a git repository.
>
> Yes, it is a breakage: in git-clone, line 212, we explicitely set GIT_DIR
> (to the not-yet-existing repository path), and call git-init-db. Now, with
> the alias thing we need to get the config if it exists, so we _got_ to
> call gently(). Boom.
Hmph. Would it be a bug in clone that does not create GIT_DIR
then?
^ permalink raw reply
* Re: Horrible re-packing?
From: Nicolas Pitre @ 2006-06-05 23:13 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051432270.5498@g5.osdl.org>
On Mon, 5 Jun 2006, Linus Torvalds wrote:
>
>
> On Mon, 5 Jun 2006, Nicolas Pitre wrote:
> >
> > In other words, the pack shrunk to less than half the size of the
> > previous one !
>
> Ok, that's a bit more extreme than expected.
>
> It's obviously great news, and says that the approach of sorting by
> "reversed name" is a great heuristic, but at the same time it makes me
> worry a bit that this thing that is supposed to be a heuristic ends up
> being _so_ important from a pack size standpoint. I was happier when it
> was more about saving a couple of percent.
Well... this is the repository that exhibited a repack regression a
while ago, going from something like ~40MB to ~160MB when Junio
initially added the directory in the name hash. No other popular
repositories had that problem.
Which is why I said this repo is particularly sensitive to heuristic
changes. So I wouldn't worry too much about your proposed patch making
it too great in this case. It certainly didn't cause any (significant)
regression overall which is what matters.
We already have surprizing results when combining two heuristics
together although if used separately they do worse. So trying to have
fallback/incremental heuristics is going to make things simply too
complicated for when it breaks. Better experiment with new ideas and
adopt them when they do a better job universally.
... which your proposed hashing change does.
Nicolas
^ permalink raw reply
* Re: Fix typo in tutorial-2.txt
From: Johannes Schindelin @ 2006-06-05 23:15 UTC (permalink / raw)
To: Linus Torvalds; +Cc: J. Bruce Fields, Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051245470.5498@g5.osdl.org>
Hi,
On Mon, 5 Jun 2006, Linus Torvalds wrote:
> I didn't actually _test_ the tutorial, but if the old command worked,
> something is really wrong!
Maybe some new gitter wants to get dirty hands? We used to have a test
which just replayed the commands in the tutorial, to make sure that new
users would not hit a regression or a syntax change...
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Fix git_setup_directory_gently when GIT_DIR is set
From: Johannes Schindelin @ 2006-06-05 23:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vk67v2o85.fsf@assigned-by-dhcp.cox.net>
Hi,
On Mon, 5 Jun 2006, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > Hi,
> >
> > On Mon, 5 Jun 2006, Junio C Hamano wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >>
> >> > When calling git_setup_directory_gently, and GIT_DIR was set, it just
> >> > ignored the variable nongit_ok.
> >>
> >> Hmph. Is this really a breakage? That is, gently() is meant
> >> for a case where you do not know if you even find a git
> >> repository and tell it not to complain because you are prepared
> >> for the case where you are not in a git repository.
> >
> > Yes, it is a breakage: in git-clone, line 212, we explicitely set GIT_DIR
> > (to the not-yet-existing repository path), and call git-init-db. Now, with
> > the alias thing we need to get the config if it exists, so we _got_ to
> > call gently(). Boom.
>
> Hmph. Would it be a bug in clone that does not create GIT_DIR
> then?
I don't think so. The whole point in calling git-init-db is to create
that. GIT_DIR is set so that the otherwise nice work-in-a-subdirectory
does not kick in. Imagine for example:
git-clone ./. victim
(taken straight out of t5400). If GIT_DIR was not set, git-init-db (which
reads repositoryformat from the config if that exists, right?) would find
.git/ in git/t/trash, and _not_ create git/t/trash/victim/.git/.
Ciao,
Dscho
^ permalink raw reply
* Re: Fix typo in tutorial-2.txt
From: J. Bruce Fields @ 2006-06-05 23:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0606051245470.5498@g5.osdl.org>
On Mon, Jun 05, 2006 at 12:47:49PM -0700, Linus Torvalds wrote:
>
> This should be obvious enough.
>
> I didn't actually _test_ the tutorial, but if the old command worked,
> something is really wrong!
Aie, sorry, thanks.--b.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox