* Re: [Census] So who uses git?
From: J. Bruce Fields @ 2006-01-31 20:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jon Loeliger, git
In-Reply-To: <7vd5i8w2nc.fsf@assigned-by-dhcp.cox.net>
On Tue, Jan 31, 2006 at 12:41:59PM -0800, Junio C Hamano wrote:
> On the tutorial front, maybe we could start teaching people to
> always use "commit -a", and not tell them about update-index nor
> "commit paths.." at all. Have them do "hello world", review
> changes since the last commit with "git diff", and make commit
> with "git commit -a". Next tell them about index, and after
> they understand index, finally tell them "commit paths..." is
> there merely to reduce typing.
Yeah, I think that's approximately what you get right now if you read
tutorial.txt followed by core-tutorial.txt, though the two currently may
not really work together well as sequels.
So I'm inclined to start by revising the two to make them read well as
sequels, then maybe moving some of core-tutorial.txt into the earlier
tutorial.txt. By the time we're done the two might end up being one
document. Or they might still be two, but with the split being more
clearly beginning/advanced instead of high-level/low-level.
Feedback from people who'd actually worked through the two would
obviously be useful.
--b.
^ permalink raw reply
* Re: [Census] So who uses git?
From: Sam Ravnborg @ 2006-01-31 20:56 UTC (permalink / raw)
To: Keith Packard
Cc: Linus Torvalds, keithp, Johannes Schindelin, Carl Baldwin,
Junio C Hamano, Martin Langhoff, Git Mailing List
In-Reply-To: <1138734110.18852.26.camel@evo.keithp.com>
> As a newly initiated user, this would have been a more gentle
> introduction to the system. But, it would be hard to make it entirely
> invisible given the current interfaces. I'm not sure if obscuring the
> presense of the index is a great plan; it's already hard enough to
> figure out how it works.
I have found myself using a mixture of cogito and git commands lately.
Part of it being that my finger type something like:
rm `git ls-files -m`
cg-restore
and I have not convinced them about git reset --hard
But the primary thing is cg-commit
I give you a list of files modified which can be edited and
it have saved me a couple of times commiting to much.
And I get vi fired up so no need to fiddle with command line argumetns.
Sam
^ permalink raw reply
* Re: [PATCH] Set errno to EEXIST if mkdir returns EACCES or EPERM
From: Alex Riesen @ 2006-01-30 23:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vvew14frk.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano, Mon, Jan 30, 2006 21:33:51 +0100:
> and have current callers of mkdir() use it, regardless of the
> platform. It may not worth saving mkdir_errno, though.
errno may be worth saving. I find the process of finding
"why-the-f$%^-did-the-windows-broke-again" really tedious:
1. find the application which failed (grep for the die message)
2. put "%s", strerror(errno) in the die
3. retest
4. find out errno is 0 (success)
5. remove close(fd), munmap, whatever before the die
6. retest
7. repeat
> Then everybody but entry.c one would say force=0, and entry.c
> one passes force appropriately using the condition it uses in
> its current if() statement.
I like it :)
^ permalink raw reply
* Re: [RFC] shallow clone
From: Junio C Hamano @ 2006-01-31 20:59 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <7v8xsxa70o.fsf@assigned-by-dhcp.cox.net>
This is whacky, but another completely different strategy is to
introduce remote alternates.
If you can allow objects/info/alternates to name a repository
that is not on the local disk, we can set the original remote
repository we "clone" from as one of the alternates, and teach
read_sha1_file() to locally cache objects we read from remote
alternates.
After such a "shallow clone", the user may want to prime the
cache by something like:
$ git-rev-list --objects v2.6.14..master |
git-pack-objects --stdout >/dev/null
before going offline. Obviously you can keep the resulting pack
instead of leaving things loose.
I am not seriously advocating this yet -- adding calls to http
and git transfer machinery in read_sha1_file(), which is as low
level as you can go, is not something I have guts to do at the
moment.
^ permalink raw reply
* Re: [Census] So who uses git?
From: Alex Riesen @ 2006-01-30 22:51 UTC (permalink / raw)
To: Radoslaw Szkodzinski
Cc: Keith Packard, Junio C Hamano, cworth, Martin Langhoff,
Linus Torvalds, Git Mailing List
In-Reply-To: <43DCA495.9040301@gorzow.mm.pl>
Radoslaw Szkodzinski, Sun, Jan 29, 2006 12:18:45 +0100:
> > Fortunately, there are very few people involved with any specific piece
> > of the X.org distribution; there's really only one or two people
> > actively developing the X.org core server, so that part of the migration
> > will be easy. Our users will be stuck, but there aren't many of them
> > either, and git makes just sucking the current bits pretty easy.
>
> Not under Windows (bleh), but it's support for Cygwin is getting better
> and better.
>
I use git in cygwin for a project with more then 17k files (almost 6M lines).
It's real slow on ntfs (on 3.2Mhz PIV!), PITA on fat, and has some hiccups now
and then (of the kind: "windows unexpectedly does not have feature X, which
everything else has" or "windows broke a 20-year-old feature Y").
But its more intuitive and more powerful than any alternatives here (Perforce,
SVN and CVS come to mind).
^ permalink raw reply
* Re: [Census] So who uses git?
From: Radoslaw Szkodzinski @ 2006-01-31 21:02 UTC (permalink / raw)
To: Junio C Hamano
Cc: Greg KH, Keith Packard, cworth, Martin Langhoff, Linus Torvalds,
Git Mailing List
In-Reply-To: <7v8xsww2kt.fsf@assigned-by-dhcp.cox.net>
[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]
Junio C Hamano wrote:
> Radoslaw Szkodzinski <astralstorm@gorzow.mm.pl> writes:
>
>> Radoslaw Szkodzinski wrote:
>>> Cloning without -l option is much slower - some minutes vs below a minute.
>>> I could have time(8)d it, but it's no use.
>>>
>> Make that time(1)d.
>>
>> Results for the kernel follow. Disc cache has been preheated with find.
>
> While you are at it, "git clone -l -s -n" might be more interesting.
>
>
Sure:
time git clone -l -s -n linux-2.6.git linux-2.6.git.lsn
real 0m0.458s
user 0m0.020s
sys 0m0.027s
Speed demon. I'd use it, but I often need a checkout anyway, so...
time git clone -l -s linux-2.6.git linux-2.6.git.ls
real 0m35.752s
user 0m2.661s
sys 0m2.374s
Not really better than git clone -l and relies on the tools more.
However, it should make for easier repacking and pruning. I'll keep it.
--
GPG Key id: 0xD1F10BA2
Fingerprint: 96E2 304A B9C4 949A 10A0 9105 9543 0453 D1F1 0BA2
AstralStorm
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply
* Re: [PATCH 3/3] Use struct tree in diff-tree
From: Junio C Hamano @ 2006-01-31 21:20 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Daniel Barkalow, git
In-Reply-To: <Pine.LNX.4.64.0601310846380.7301@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Sun, 29 Jan 2006, Daniel Barkalow wrote:
>>
>> It had been open-coding a tree parser. This updates the programs that
>> call diff_tree() to send it the struct tree instead of a buffer and
>> size.
>
> Please don't.
> ...
> Junio, please don't apply this.
>
> Linus
I haven't, and I won't. From my gut feeling I did not even
place it in "pu". After timing it myself and then looking at
the code I agree with your analysis.
The one to git-tar-tree I've already applied, mostly because I
was not careful enough and especially I did not care enough
about performance of that program. On my slow machine the tip
of kernel before you came back takes 9.2 seconds wallclock as
opposed to 8.7 seconds to tar up, so the patch degrades the
performance by about 5%. Maybe we would want to revert that one
as well.
^ permalink raw reply
* Re: [Census] So who uses git?
From: Linus Torvalds @ 2006-01-31 21:25 UTC (permalink / raw)
To: Alex Riesen
Cc: Radoslaw Szkodzinski, Keith Packard, Junio C Hamano, cworth,
Martin Langhoff, Git Mailing List
In-Reply-To: <20060130225107.GA3857@limbo.home>
On Mon, 30 Jan 2006, Alex Riesen wrote:
>
> I use git in cygwin for a project with more then 17k files (almost 6M lines).
> It's real slow on ntfs (on 3.2Mhz PIV!)
One thing that git does rely on is a fast "lstat()" system call. The index
file means that we almost never need to read the contents of a file to
compare, but git _does_ check that files haven't been modified, and doing
an "lstat()" on every single file it knows about is the way to do that.
Now, I suspect that you simply can't do basic filename lookups much faster
than Linux does them. The Linux VFS layer name caching reigns supreme: the
dentries are just incredibly powerful, and the reason Linux kicks ass on
many benchmarks.
And yes, git was designed for it. git is _really_ fast on Linux, but any
operating system that is so stupid that it has to call down to the
low-level filesystem for filename lookup (which is most of them, and from
what I have heard, the NT VFS layer is worse than most) will take a lot
longer.
This is sadly not something I think you can possibly avoid. Git is
literally being as fast as is humanly possible without doing explicit
locking. You _can_ avoid the "lstat()" calls if you are willing to always
explicitly mark files that you have changed (so that the SCM can stat just
_those_ files and ignore all the others), but I personally much prefer
being able to use any random tools on the files without having to prepare
them some way.
So we could speed it up on cygwin (and yes, it would speed git up a lot
even on Linux, but since the cached lstat() case is so fast anyway, I
doubt a lot of Linux users care - the biggest win would be on a cold-cache
tree). But it would require that you explicitly _mark_ the files you edit
some way.
Btw, BK wanted that, and it wasn't _too_ painful. You had to do
bk edit
to mark a file as being ready to be dirtied, and as a helper command you
would use
bk editor
which would first do the "bk edit" thing and then start up your favourite
editor (the usual ${EDITOR:${VISUAL:vi}} rules applied) on it, and it
worked fine. We _could_ do the same in git.
I'd just prefer not to.
For small projects (or big projects with fairly few files), it really
shouldn't matter. Your 17k files example is hopefully fairly rare..
> But its more intuitive and more powerful than any alternatives here (Perforce,
> SVN and CVS come to mind).
Good to know.
Linus
^ permalink raw reply
* Re: [PATCH] Fix HTTP request result processing after slot reuse
From: Junio C Hamano @ 2006-01-31 21:39 UTC (permalink / raw)
To: Nick Hengeveld; +Cc: git
In-Reply-To: <20060131190655.GE3873@reactrix.com>
Nick Hengeveld <nickh@reactrix.com> writes:
> Add a way to store the results of an HTTP request when a slot finishes
> so the results can be processed after the slot has been reused.
>
> Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
I do not claim I understand this part of the code well, but it
feels awkward to see that a problem caused by a structure being
reused can be solved by having a pointer from that structure
that points at a static location.
These static variables are probably correct, provided if
fetch_index, fetch_indices and friends do not recurse into
themselves, but it just gives me this funny feeling...
Care to enlighten me, please?
^ permalink raw reply
* Re: [PATCH 3/3] Use struct tree in diff-tree
From: Daniel Barkalow @ 2006-01-31 21:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, git
In-Reply-To: <7virs0uma8.fsf@assigned-by-dhcp.cox.net>
On Tue, 31 Jan 2006, Junio C Hamano wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
>
> > On Sun, 29 Jan 2006, Daniel Barkalow wrote:
> >>
> >> It had been open-coding a tree parser. This updates the programs that
> >> call diff_tree() to send it the struct tree instead of a buffer and
> >> size.
> >
> > Please don't.
> > ...
> > Junio, please don't apply this.
> >
> > Linus
>
> I haven't, and I won't. From my gut feeling I did not even
> place it in "pu". After timing it myself and then looking at
> the code I agree with your analysis.
>
> The one to git-tar-tree I've already applied, mostly because I
> was not careful enough and especially I did not care enough
> about performance of that program. On my slow machine the tip
> of kernel before you came back takes 9.2 seconds wallclock as
> opposed to 8.7 seconds to tar up, so the patch degrades the
> performance by about 5%. Maybe we would want to revert that one
> as well.
Probably a better solution is to move the tree parser from tree-diff.c
into tree.c, provide a clear API in tree.h, and use that in tar-tree.
There's definitely no need to have another parser beyond the one
currently in tree-diff and the struct tree one.
I think I should be able to change the struct tree API slightly to make it
an iterator instead of a linked list, and get the efficiency benefits of
the tree-diff parser while still having a nice API.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: [Census] So who uses git?
From: J. Bruce Fields @ 2006-01-31 21:52 UTC (permalink / raw)
To: Linus Torvalds
Cc: Alex Riesen, Radoslaw Szkodzinski, Keith Packard, Junio C Hamano,
cworth, Martin Langhoff, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0601311314030.7301@g5.osdl.org>
On Tue, Jan 31, 2006 at 01:25:08PM -0800, Linus Torvalds wrote:
> So we could speed it up on cygwin (and yes, it would speed git up a lot
> even on Linux, but since the cached lstat() case is so fast anyway, I
> doubt a lot of Linux users care - the biggest win would be on a cold-cache
> tree). But it would require that you explicitly _mark_ the files you edit
> some way.
You couldn't depend on a combination of lstat's and some kind of
filesystem change notifications?
--b.
^ permalink raw reply
* Bottlenecks in git merge
From: Peter Eriksen @ 2006-01-31 21:33 UTC (permalink / raw)
To: Git Mailing List
Hello,
In connection with Ian Molton's question about merge have I played a
little with 'git merge' on the kernel sources. What I find is that a
merge can take quite some time, but I'm not sure where that time exactly
goes to. Here are the times I got:
Recursive (default): 4m22.282s
Resolve (-s resolve): 3m23.548s
What is taking so long?
Regards,
Peter
==============================>8==================
#!/bin/sh
# Run from linux-2.6
change_readme() {
sed -i "s/are much better/are better/" README
}
git checkout -f master
git branch -d test
git checkout -b test v2.6.12
change_readme
git commit -a -m "Work, work, work"
time git merge $STRATEGY "Merging happily." HEAD v2.6.15
^ permalink raw reply
* Re: [Census] So who uses git?
From: Alex Riesen @ 2006-01-31 22:01 UTC (permalink / raw)
To: Linus Torvalds
Cc: Radoslaw Szkodzinski, Keith Packard, Junio C Hamano, cworth,
Martin Langhoff, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0601311314030.7301@g5.osdl.org>
Linus Torvalds, Tue, Jan 31, 2006 22:25:08 +0100:
> > I use git in cygwin for a project with more then 17k files (almost
> > 6M lines). It's real slow on ntfs (on 3.2Mhz PIV!)
> ...
> So we could speed it up on cygwin (and yes, it would speed git up a lot
> even on Linux, but since the cached lstat() case is so fast anyway, I
> doubt a lot of Linux users care - the biggest win would be on a cold-cache
> tree). But it would require that you explicitly _mark_ the files you edit
> some way.
I'd hate to have to do that. The project in question is just stuffed
up beyond all reason, windows' VFS is a sorry piece of junk, and I
care much more about how comfortable the tool is.
> ...
> For small projects (or big projects with fairly few files), it really
> shouldn't matter. Your 17k files example is hopefully fairly rare..
I'd say it is fairly common. It's what driven by paranoia and
suffering from chronic undereducation projects in big companies
usually end up with. Frequently right from the start...
^ permalink raw reply
* Re: [PATCH 3/3] Use struct tree in diff-tree
From: Linus Torvalds @ 2006-01-31 22:07 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Daniel Barkalow, git
In-Reply-To: <7virs0uma8.fsf@assigned-by-dhcp.cox.net>
On Tue, 31 Jan 2006, Junio C Hamano wrote:
>
> The one to git-tar-tree I've already applied, mostly because I
> was not careful enough and especially I did not care enough
> about performance of that program. On my slow machine the tip
> of kernel before you came back takes 9.2 seconds wallclock as
> opposed to 8.7 seconds to tar up, so the patch degrades the
> performance by about 5%. Maybe we would want to revert that one
> as well.
Hmm. Rather than revert it outright, it might be better to make it use the
nicer parsing functions and "struct tree_desc".
It shouldn't look _that_ different from the "struct tree" version: instead
of doing
item = item->next;
it would do
update_tree_entry(tree);
instead.
Give me a minute, I'll send you patches.
Linus
^ permalink raw reply
* [PATCH 1/2] Make the "struct tree_desc" operations available to others
From: Linus Torvalds @ 2006-01-31 22:10 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
We have operations to "extract" and "update" a "struct tree_desc", but we
only used them in tree-diff.c and they were static to that file.
But other tree traversal functions can use them to their advantage
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
This is just infrastructure. No real changes, except for the renaming of
the "extract()" function to make it have a more palatable name when it's
globally visible.
diff --git a/diff.h b/diff.h
index 9a0169c..32134d7 100644
--- a/diff.h
+++ b/diff.h
@@ -13,6 +13,9 @@ struct tree_desc {
unsigned long size;
};
+extern void update_tree_entry(struct tree_desc *);
+extern const unsigned char *tree_entry_extract(struct tree_desc *, const char **, unsigned int *);
+
struct diff_options;
typedef void (*change_fn_t)(struct diff_options *options,
diff --git a/tree-diff.c b/tree-diff.c
index 382092b..d978428 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -9,7 +9,7 @@ static int nr_paths = 0;
static const char **paths = NULL;
static int *pathlens = NULL;
-static void update_tree_entry(struct tree_desc *desc)
+void update_tree_entry(struct tree_desc *desc)
{
void *buf = desc->buf;
unsigned long size = desc->size;
@@ -21,7 +21,7 @@ static void update_tree_entry(struct tre
desc->size = size - len;
}
-static const unsigned char *extract(struct tree_desc *desc, const char **pathp, unsigned int *modep)
+const unsigned char *tree_entry_extract(struct tree_desc *desc, const char **pathp, unsigned int *modep)
{
void *tree = desc->buf;
unsigned long size = desc->size;
@@ -56,8 +56,8 @@ static int compare_tree_entry(struct tre
const unsigned char *sha1, *sha2;
int cmp, pathlen1, pathlen2;
- sha1 = extract(t1, &path1, &mode1);
- sha2 = extract(t2, &path2, &mode2);
+ sha1 = tree_entry_extract(t1, &path1, &mode1);
+ sha2 = tree_entry_extract(t2, &path2, &mode2);
pathlen1 = strlen(path1);
pathlen2 = strlen(path2);
@@ -109,7 +109,7 @@ static int interesting(struct tree_desc
if (!nr_paths)
return 1;
- (void)extract(desc, &path, &mode);
+ (void)tree_entry_extract(desc, &path, &mode);
pathlen = strlen(path);
baselen = strlen(base);
@@ -167,7 +167,7 @@ static int show_entry(struct diff_option
{
unsigned mode;
const char *path;
- const unsigned char *sha1 = extract(desc, &path, &mode);
+ const unsigned char *sha1 = tree_entry_extract(desc, &path, &mode);
if (opt->recursive && S_ISDIR(mode)) {
char type[20];
^ permalink raw reply related
* [PATCH 2/2] Make git-tar-tree use the tree_desc abstractions
From: Linus Torvalds @ 2006-01-31 22:13 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0601311407460.7301@g5.osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
This is a first try at making git-tar-tree use the "struct tree_desc"
infrastructure. I actually did test it by creating a tar-file of Linux
v2.6.15, and it _seemed_ to work, and it passes all the git testsuite, but
hey, that was all very superficial.
It was a pretty straightforward change from "struct tree" to "struct
tree_desc", though, so it should all be fine. Daniel had already cleaned
up the code by his original conversion.
diff --git a/tar-tree.c b/tar-tree.c
index d36baed..e85a1ed 100644
--- a/tar-tree.c
+++ b/tar-tree.c
@@ -3,7 +3,7 @@
*/
#include <time.h>
#include "cache.h"
-#include "tree.h"
+#include "diff.h"
#include "commit.h"
#define RECORDSIZE (512)
@@ -336,37 +336,38 @@ static void write_header(const unsigned
write_if_needed();
}
-static void traverse_tree(struct tree *tree,
+static void traverse_tree(struct tree_desc *tree,
struct path_prefix *prefix)
{
struct path_prefix this_prefix;
- struct tree_entry_list *item;
this_prefix.prev = prefix;
- parse_tree(tree);
- item = tree->entries;
-
- while (item) {
+ while (tree->size) {
+ const char *name;
+ const unsigned char *sha1;
+ unsigned mode;
void *eltbuf;
char elttype[20];
unsigned long eltsize;
- eltbuf = read_sha1_file(item->item.any->sha1,
- elttype, &eltsize);
+ sha1 = tree_entry_extract(tree, &name, &mode);
+ update_tree_entry(tree);
+
+ eltbuf = read_sha1_file(sha1, elttype, &eltsize);
if (!eltbuf)
- die("cannot read %s",
- sha1_to_hex(item->item.any->sha1));
- write_header(item->item.any->sha1, TYPEFLAG_AUTO, basedir,
- prefix, item->name,
- item->mode, eltbuf, eltsize);
- if (item->directory) {
- this_prefix.name = item->name;
- traverse_tree(item->item.tree, &this_prefix);
- } else if (!item->symlink) {
+ die("cannot read %s", sha1_to_hex(sha1));
+ write_header(sha1, TYPEFLAG_AUTO, basedir,
+ prefix, name, mode, eltbuf, eltsize);
+ if (S_ISDIR(mode)) {
+ struct tree_desc subtree;
+ subtree.buf = eltbuf;
+ subtree.size = eltsize;
+ this_prefix.name = name;
+ traverse_tree(&subtree, &this_prefix);
+ } else if (!S_ISLNK(mode)) {
write_blocked(eltbuf, eltsize);
}
free(eltbuf);
- item = item->next;
}
}
@@ -374,7 +375,7 @@ int main(int argc, char **argv)
{
unsigned char sha1[20];
struct commit *commit;
- struct tree *tree;
+ struct tree_desc tree;
setup_git_directory();
@@ -395,8 +396,8 @@ int main(int argc, char **argv)
write_global_extended_header(commit->object.sha1);
archive_time = commit->date;
}
- tree = parse_tree_indirect(sha1);
- if (!tree)
+ tree.buf = read_object_with_reference(sha1, "tree", &tree.size, NULL);
+ if (!tree.buf)
die("not a reference to a tag, commit or tree object: %s",
sha1_to_hex(sha1));
if (!archive_time)
@@ -404,7 +405,7 @@ int main(int argc, char **argv)
if (basedir)
write_header((unsigned char *)"0", TYPEFLAG_DIR, NULL, NULL,
basedir, 040777, NULL, 0);
- traverse_tree(tree, NULL);
+ traverse_tree(&tree, NULL);
write_trailer();
return 0;
}
^ permalink raw reply related
* Re: [Census] So who uses git?
From: Junio C Hamano @ 2006-01-31 22:21 UTC (permalink / raw)
To: Sam Ravnborg; +Cc: git
In-Reply-To: <29639.194.237.142.10.1138741008.squirrel@194.237.142.10>
"Sam Ravnborg" <sam@ravnborg.org> writes:
> But the primary thing is cg-commit
> I give you a list of files modified which can be edited and
> it have saved me a couple of times commiting to much.
> And I get vi fired up so no need to fiddle with command line argumetns.
[this is what I sent in a separate message but I goofed up the
destination headers and the message did not appear on the list,
so I am reprinting.]
I have always felt "git commit paths..." was a mistake; it
encourages partial commits by individual developers.
By "partial commit", I mean a commit that does not exactly match
the state of the working tree when the commit is made. There
are two kinds of "partial commits". Good ones and bad ones.
Being able to make partial commits is handy for people whose
primary role is to integrate many changes from trusted
developers rather than testing each and every commit as a whole
(read: Linus and subsystem maintainers). Integrators' job may
include testing what have been merged as a whole by a compile
and reboot cycle as the final "wrap-up" step, but the most
important role they play is to sanity check the changes from
architectural perspective.
For that workflow to work effectively, however, the changes fed
by individual developers to the integrators have to be clean and
well tested. A partial commit records something that never
existed in any working tree as a whole, so by definition it is
an untested change. You would risk "sorry I forgot to commit
the changes to these paths but without them it does not even
compile", and end up wasting integrators' time.
The integrators make commits out of their working trees using
git-merge and git-apply to record changes made by others after
reviewing them. These commands ignore unconflicting local
changes (but notices conflicting ones to operate correctly), and
allow them to make partial commits. This is a good thing;
otherwise they would have to reset their own changes in their
working tree, only to do merges and to accept patches. However,
people playing the integrator role rarely have reason to use
"git commit paths..." while merging from others to make such a
partial commit. Only after they resolve conflicts by hand,
perhaps. But that happens far less often than careless
individual developers making partial commits of bad kind using
the same "git commit paths..." command.
This is the reason why I feel "git commit paths..." is a bad
feature. It helps to make bad partial commits, without having
to do much with making good partial commits.
Many SCMs may have the ability to do "commit paths...", but that
does not change the fact that it encourages carelessness for
individual developers, which is especially bad in a distributed
development workflow like the Linux kernel style [*1*].
But that was not my change ;-).
[Foornote]
*1* It could be argued that being able to do partial commit is a
good thing in other SCM systems where there is no equivalent to
our "index" file. It is one way for the developer to snapshot
their work-in-progress state where they might later come back to
if the approach they are currently pursuing does not pan out.
But for that, we have index file we can "check into" without
committing.
^ permalink raw reply
* Re: [Census] So who uses git?
From: Joel Becker @ 2006-01-31 22:55 UTC (permalink / raw)
To: Linus Torvalds
Cc: Keith Packard, Johannes Schindelin, Carl Baldwin, Junio C Hamano,
Martin Langhoff, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0601311110210.7301@g5.osdl.org>
On Tue, Jan 31, 2006 at 11:21:52AM -0800, Linus Torvalds wrote:
> Now, I do agree. I don't actually like hiding the index too much.
> Understanding the index is _invaluable_ whenever you're doing a merge with
> conflicts, and understanding what tools are available to you to resolve
> those conflicts.
This is precisely the experience I've had explaining GIT to
folks moving to it. The simplest workflow (clone; hack one file, commit
one file) is so similar to CVS/Subversion/Anything that it's immediately
understood. But when pull, push, merge, and any non-linear history are
discussed, I have to describe the index and the commit/tree layout.
Once I do, they get it.
> So I'm actually of the "revel in the index" camp (as could probably be
> guessed by the original tutorial).
I'm going to second this, from a real-world "explain it to
others" standpoint.
Joel
--
"Every day I get up and look through the Forbes list of the richest
people in America. If I'm not there, I go to work."
- Robert Orben
Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127
^ permalink raw reply
* Re: Bottlenecks in git merge
From: Junio C Hamano @ 2006-01-31 23:06 UTC (permalink / raw)
To: Peter Eriksen; +Cc: Git Mailing List
In-Reply-To: <20060131213314.GA32131@ebar091.ebar.dtu.dk>
"Peter Eriksen" <s022018@student.dtu.dk> writes:
> Recursive (default): 4m22.282s
> Resolve (-s resolve): 3m23.548s
>
> What is taking so long?
I am actually surprised that recursive is not much slower than
resolve. I expected to see bigger difference for a merge like
this.
> git checkout -b test v2.6.12
> change_readme
> git commit -a -m "Work, work, work"
> time git merge $STRATEGY "Merging happily." HEAD v2.6.15
You are merging a variant of v2.6.12 and v2.6.15. Each of these
two official revisions has roughly 18,000 files, and they differ
at 10,723 files among them.
With an up-to-date index that has small changes from v2.6.12,
merging these two revisions using read-tree -m to do the trivial
merge (the part that comes before recursive/resolve) leaves
about 850 files to be resolved in the working tree. For these
files, you need to do an equivalent of merge-one-file to merge
the differences (in this particular case, most of them are
"removed in one but unchanged in the other" kind). In addition,
you have to checkout the result of the merge, which means you
need to update at least 10,723 files.
I suspect that it might make things quite faster if we resolved
case 8 and 10 (see either Documentation/technical/trivial-merge
or t/t1000) in index for this particular case, but it has
correctness issues. A merge strategy may want to say "This file
was removed by the other branch while it stayed on our branch;
but this is not a remove but actually a rename", and do
something different from what merge-one-file does, and resolving
these cases in index closes the door for that possibility.
^ permalink raw reply
* Re: [Census] So who uses git?
From: Daniel Barkalow @ 2006-01-31 23:16 UTC (permalink / raw)
To: Johannes Schindelin
Cc: Carl Baldwin, Junio C Hamano, Keith Packard, Martin Langhoff,
Linus Torvalds, Git Mailing List
In-Reply-To: <Pine.LNX.4.63.0601311127250.25248@wbgn013.biozentrum.uni-wuerzburg.de>
On Tue, 31 Jan 2006, Johannes Schindelin wrote:
> Hi,
>
> On Mon, 30 Jan 2006, Carl Baldwin wrote:
>
> > In general, I think it is grasping the reason for the index file and how
> > git commands like git-commit and git-diff interact with it.
>
> IMHO this is the one big showstopper. I had problems explaining the
> concept myself.
>
> For example, I had a hard time explaining to a friend why a git-add'ed
> file is committed when saying "git commit some_other_file", but not
> another (modified) file. Very unintuitive.
I sort of suspect that "git commit some_other_file" should really read
HEAD into a temporary index, update "some_other_file" in that (and the
main index), and commit it. The concept of the index isn't hard (it's the
preparation you've made so far towards a commit), and plain "git commit"
makes sense with it; "git commit -a" also makes sense, since committing
all changes is pretty clear. The surprising thing is that "git commit path
..." means "everything I've already mentioned, plus path..." not just
"path ...", and it's particularly surprising because people only tend to
specify paths when they've done something they don't want to commit.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: Bottlenecks in git merge
From: Petr Baudis @ 2006-01-31 23:27 UTC (permalink / raw)
To: Git Mailing List
In-Reply-To: <20060131213314.GA32131@ebar091.ebar.dtu.dk>
Hello,
Dear diary, on Tue, Jan 31, 2006 at 10:33:14PM CET, I got a letter
where Peter Eriksen <s022018@student.dtu.dk> said that...
> In connection with Ian Molton's question about merge have I played a
> little with 'git merge' on the kernel sources. What I find is that a
> merge can take quite some time, but I'm not sure where that time exactly
> goes to. Here are the times I got:
>
> Recursive (default): 4m22.282s
> Resolve (-s resolve): 3m23.548s
>
>
> What is taking so long?
it is difficult to benchmark for me since everything required for the
merge (that is, both all the objects and the whole working tree) just
won't fit into my caches (or Linux at least won't let it stay there for
long enough). I ended up repeatedly calling the subcommands, but that
obviously is not a real world usage pattern. Proportionally, the
significant eaters of time for cg-merge (similar to -s resolve) are:
git-merge-base --- 1s cached, 10s to 20s uncached
git-read-tree -m --- 1s cached, 10s or more uncached
git-read-tree -m -u --- 1m50s w/ heavy disk activity, but big part of it
is writing blocks
git-merge-index -a \
-o /bin/true --- 1s cached
git-merge-index -a \
-o ~/cogito/cg-Xmergefile
--- 1m27s with some disk activity (44s user, 20s sys)
cg-Xmergefile is very similar to
git-merge-one-file
Note that the time spent by git-read-tree here is just checking out the
new file versions, which is inevitable. ;-)
The real killer here is therefore git-merge-one-file. Most frequent hits
here are probably of the added-in-one case, resulting in two more
fork()s, reloading the index like mad all the time.
Comparing cg-merge to git-merge, one difference is that git-merge tries
to do kind of "trivial" merge first (apparently even if -s was passed to
it), the point of which kind of escapes me if you are using the resolve
strategy, but which causes two git-update-index calls - even one can
take good half a minute or more if your cache is cold.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe. -- Douglas Adams
^ permalink raw reply
* Re: Bottlenecks in git merge
From: Petr Baudis @ 2006-01-31 23:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Peter Eriksen, Git Mailing List
In-Reply-To: <7vk6cgq9ny.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Wed, Feb 01, 2006 at 12:06:57AM CET, I got a letter
where Junio C Hamano <junkio@cox.net> said that...
> "Peter Eriksen" <s022018@student.dtu.dk> writes:
>
> > Recursive (default): 4m22.282s
> > Resolve (-s resolve): 3m23.548s
> >
> > What is taking so long?
>
> I am actually surprised that recursive is not much slower than
> resolve. I expected to see bigger difference for a merge like
> this.
>
> > git checkout -b test v2.6.12
> > change_readme
> > git commit -a -m "Work, work, work"
> > time git merge $STRATEGY "Merging happily." HEAD v2.6.15
>
> You are merging a variant of v2.6.12 and v2.6.15. Each of these
> two official revisions has roughly 18,000 files, and they differ
> at 10,723 files among them.
>
> With an up-to-date index that has small changes from v2.6.12,
> merging these two revisions using read-tree -m to do the trivial
> merge (the part that comes before recursive/resolve) leaves
> about 850 files to be resolved in the working tree. For these
> files, you need to do an equivalent of merge-one-file to merge
> the differences (in this particular case, most of them are
> "removed in one but unchanged in the other" kind). In addition,
> you have to checkout the result of the merge, which means you
> need to update at least 10,723 files.
>
> I suspect that it might make things quite faster if we resolved
> case 8 and 10 (see either Documentation/technical/trivial-merge
> or t/t1000) in index for this particular case, but it has
> correctness issues. A merge strategy may want to say "This file
> was removed by the other branch while it stayed on our branch;
> but this is not a remove but actually a rename", and do
> something different from what merge-one-file does, and resolving
> these cases in index closes the door for that possibility.
What about letting the file-handler actually tell merge-index what to
do? merge-index could make a fifo at fd 3 for it (we might fork a
special buffering process for it to avoid PIPE_BUF issues) and let it
write there a sequence of lines like:
path\0{add|remove|update} {workingcopy|<sha1> <mode>}
That would avoid many in-file-handler forks and especially perpetual
reloading and rewriting of the index file, which _seems_ to be the main
time waster according to my somewhat fuzzy benchmarks.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe. -- Douglas Adams
^ permalink raw reply
* Re: [Census] So who uses git?
From: Petr Baudis @ 2006-01-31 23:36 UTC (permalink / raw)
To: Daniel Barkalow
Cc: Johannes Schindelin, Carl Baldwin, Junio C Hamano, Keith Packard,
Martin Langhoff, Linus Torvalds, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0601311750270.25300@iabervon.org>
Dear diary, on Wed, Feb 01, 2006 at 12:16:26AM CET, I got a letter
where Daniel Barkalow <barkalow@iabervon.org> said that...
> On Tue, 31 Jan 2006, Johannes Schindelin wrote:
>
> > Hi,
> >
> > On Mon, 30 Jan 2006, Carl Baldwin wrote:
> >
> > > In general, I think it is grasping the reason for the index file and how
> > > git commands like git-commit and git-diff interact with it.
> >
> > IMHO this is the one big showstopper. I had problems explaining the
> > concept myself.
> >
> > For example, I had a hard time explaining to a friend why a git-add'ed
> > file is committed when saying "git commit some_other_file", but not
> > another (modified) file. Very unintuitive.
>
> I sort of suspect that "git commit some_other_file" should really read
> HEAD into a temporary index, update "some_other_file" in that (and the
> main index), and commit it.
FWIW, this is also what cg-commit does.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe. -- Douglas Adams
^ permalink raw reply
* [PATCH] Allow diff and index commands to be interrupted
From: Petr Baudis @ 2006-01-31 23:40 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
So far, e.g. git-update-index --refresh was basically uninterruptable
by ctrl-c, since it hooked the SIGINT handler, but that handler would
only unlink the lockfile but not actually quit. This makes it propagate
the signal to the default handler.
Note that I expected it to work without resetting the signal handler to
SIG_DFL, but without that it ended in an infinite loop of tgkill()s -
is my glibc violating SUS or what?
Signed-off-by: Petr Baudis <pasky@suse.cz>
---
diff.c | 2 ++
index.c | 2 ++
2 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/diff.c b/diff.c
index 8ae6dbc..ec51e7d 100644
--- a/diff.c
+++ b/diff.c
@@ -555,6 +555,8 @@ static void remove_tempfile(void)
static void remove_tempfile_on_signal(int signo)
{
remove_tempfile();
+ signal(SIGINT, SIG_DFL);
+ raise(signo);
}
/* An external diff command takes:
diff --git a/index.c b/index.c
index ad0eafe..f92b960 100644
--- a/index.c
+++ b/index.c
@@ -18,6 +18,8 @@ static void remove_lock_file(void)
static void remove_lock_file_on_signal(int signo)
{
remove_lock_file();
+ signal(SIGINT, SIG_DFL);
+ raise(signo);
}
int hold_index_file_for_update(struct cache_file *cf, const char *path)
^ permalink raw reply related
* Re: Bottlenecks in git merge
From: Linus Torvalds @ 2006-01-31 23:45 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Peter Eriksen, Git Mailing List
In-Reply-To: <7vk6cgq9ny.fsf@assigned-by-dhcp.cox.net>
On Tue, 31 Jan 2006, Junio C Hamano wrote:
>
> I am actually surprised that recursive is not much slower than
> resolve. I expected to see bigger difference for a merge like
> this.
Well, if most of the cost is just the trivial single-file merges and the
fact that we have to update a ton of files from an old version (and it
probably is), the difference between the trivial and the recursive merge
is not going to be huge.
> With an up-to-date index that has small changes from v2.6.12,
> merging these two revisions using read-tree -m to do the trivial
> merge (the part that comes before recursive/resolve) leaves
> about 850 files to be resolved in the working tree. For these
> files, you need to do an equivalent of merge-one-file to merge
> the differences (in this particular case, most of them are
> "removed in one but unchanged in the other" kind). In addition,
> you have to checkout the result of the merge, which means you
> need to update at least 10,723 files.
It would be interesting to see how big the "resolve 850 files" part is vs
the "check out 10k+ files" is.
In particular, if the "resolve 850 files" is a noticeable portion of it,
then the right thing to do may be to just re-write git-merge-one-file.sh
in C. Right now, almost _all_ of the expense of that thing is just the
shell interpreter startup. The actual actions it does are usually fairly
cheap.
(yes, a real three-way merge is more expensive, but I suspect that even
that isn't much more expensive than starting up an invocation of "bash".
The other actions that merge-one-file does are _really_ trivial).
In fact, we could hardcode the "git-merge-one-file" behaviour inside
"git-merge-index".
Now, that won't help "recursive" (which doesn't use git-merge-one-file at
all, and does it all by hand), but it would be an interesting test to
make, becuase if it makes the simpler "-s resolve" merge even faster, then
we know that this is likely a large portion of the time.
Then, somebody would have to consider what to do about
git-merge-recursive. For example, if the _common_ case is "modified in
both, but differently", and they merge cleanly, maybe the recursive merge
could handle those separately and fast with a special "git-merge-one-file"
invocation (just to cut down the number of files that it needs to think
more about).
Linus
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox