* Re: [RFC] Submodules in GIT
From: Robin Rosenberg @ 2006-12-03 12:22 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200612030231.05900.jnareb@gmail.com>
söndag 03 december 2006 02:31 skrev Jakub Narebski:
> Actually, rm -rf * is not needed for "git reset --hard" or
> "git checkout -f" to succeed.
True, but git reset --hard isn't always enough and rm -rf * is the good ol'
way of resetting things. Typically this comes from make being upset (or too
content) with something.
^ permalink raw reply
* Re: [EGIT PATCH] Convert author and comment on demand.
From: Robin Rosenberg @ 2006-12-03 12:18 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20061203021655.GA26668@spearce.org>
söndag 03 december 2006 03:16 skrev Shawn Pearce:
> Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> > This sppeds up reading commits a lot by only store the byte
> > array data when reading commits.
>
> Thanks. I was working on something similar but did not have
> a chance to finish it. I've applied your patch instead.
>
> > + try {
> > + BufferedReader br=new BufferedReader(new InputStreamReader(new
> > ByteArrayInputStream(raw)));
> > + String n=br.readLine();
>
> Something's wrong with your mail client... the patch was mangled.
Not really, only the user. KMail by default wraps lines and I didn't turn that
off before sending. Usually I send patches just using stg mail, but this time
I used git-format-patch and kmail. Generatig attachements may be better (next
time).
^ permalink raw reply
* Re: Thoughts about memory requirements in traversals [Was: Re: [RFC] Submodules in GIT]
From: Josef Weidendorfer @ 2006-12-03 11:47 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <ekubag$6km$2@sea.gmane.org>
On Sunday 03 December 2006 12:10, Jakub Narebski wrote:
> > You simply sequentially renumber any object SHA, starting from 0
> > in the order you see them. You can do two renumberings, one for
> > the objects contained in the original pack (1), and one for the
> > external ones (2). Put these new numbers (with a bit distinguishing
> > (1) and (2)) as replacement into commit/tree objects.
> > At the end, you have the new offsets for objects in (1). Put
> > redirection tables for (1) [new number -> new offset]
> > and (2) [other new number->SHA1 of external object] at the end
> > of the new pack.
>
> Doesn't this idea clash with the object and delta reusing for repack?
In general, yes: you modify object content by encoding, and if you want
to reuse the objects without decompression, and need to keep the info
to be able to decode, ie. the redirection table.
This gets problematic if you want to join multiple packs, or fetch objects
and want to reuse the compressed representation, as the object renumbering
is only local to one pack, and numbers can be reused between packs.
So this idea is probably for archival packs only.
^ permalink raw reply
* Re: Thoughts about memory requirements in traversals [Was: Re: [RFC] Submodules in GIT]
From: Jakub Narebski @ 2006-12-03 11:10 UTC (permalink / raw)
To: git
In-Reply-To: <200612030421.18662.Josef.Weidendorfer@gmx.de>
Josef Weidendorfer wrote:
> On Sunday 03 December 2006 03:46, Shawn Pearce wrote:
>> Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
>>> Thinking even one step further:
>>> Would it make sense to define an encoding format for the content of
>>> commit and tree objects inside of packs, where the SHA1 is replaced by
>>> the offset of the object in this pack?
>>> As exactly the SHA1 is the least compressable thing, this could promise
>>> quite a benefit.
>> [...]
>>
>> This means that when we start to write out a commit we need to know
>> the offset to the tree that commit references. But git-pack-objects
>> sorts object by type: commit, tree, blob (I forget where tags go,
>> but they aren't important in this context). So generally *all*
>> commits appear before the first tree. So when we write out the first
>> commit we need to know exactly how many bytes every commit will need
>> (compressed mind you) in this pack so we can determine the position
>> of the first tree. Now do this for every commit and every tree
>> that those commits use... yes, its a lot of work to precompute
>> and store all offsets before you even write out the first byte.
>
> Yes, it looks like a hen-and-egg problem, but IMHO you can
> handle it nicely with another redirection, i.e. a table you build
> up while repacking the file, and storing this table at the end.
>
> You simply sequentially renumber any object SHA, starting from 0
> in the order you see them. You can do two renumberings, one for
> the objects contained in the original pack (1), and one for the
> external ones (2). Put these new numbers (with a bit distinguishing
> (1) and (2)) as replacement into commit/tree objects.
> At the end, you have the new offsets for objects in (1). Put
> redirection tables for (1) [new number -> new offset]
> and (2) [other new number->SHA1 of external object] at the end
> of the new pack.
> This way, you effectivly have removed all incompressable SHAs from
> the pack file aside from one entry in the redirection tables for
> each external object.
>
> The only problem I see is how to decode the objects, i.e. how to
> get the original SHA1 from an offset: we can not recalculate the
> SHA1 from the object content as we changed the content itself.
> But there should be a way to store the SHA1 in front of the object
> somehow, perhaps it is already given by the current format?
>
> Am I missing something here?
Doesn't this idea clash with the object and delta reusing for repack?
Hmmm... perhaps with the two indirect tables it wouldn't, only
the tables would need to be recalculated... or perhaps it would because
of offset clashes.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Jakub Narebski @ 2006-12-03 11:00 UTC (permalink / raw)
To: git
In-Reply-To: <200612030224.50592.robin.rosenberg.lists@dewire.com>
Robin Rosenberg wrote:
> lördag 02 december 2006 21:16 skrev Jakub Narebski:
>>
>> The problem with submodule as separate git repository is that if you
>> move submodule (subproject) somewhere else in the repository (or just
>> rename it), you have to update alternates file... and this happens not
>> only on move itself, but also on checkout and reset. But that can be
>> managed by having in alternates all possible places the submodule ends
>> into. I don't know if it is truly a problem.
>
> A nasty problem with separate repositories for submodules is that when you
> screw up and git complains about everything you try do do, you previously
> could do rm -rf *; git reset --hard and retry whatever you were trying to do.
> With separate repositories your submodules will be resting in /dev/null,
> unless you're very, very careful.
The solution to this concern could be having GIT_DIR for submodule
outside it's working area, for example somewhere in GIT_DIR of
the supermodule, and use either symlink or (to be coded) .gitlink
symbolic reference to GIT_DIR file. Disadvantage of that is that it
moves troubles with moving subproject (although there are no troubles
with simple subproject directory renaming) from alternates file to
GIT_DIR link representation in submodule.
As Linus said, there are advantages to having submodule repository use
separate object database (clone and other operation scaling, index size),
and I think they outweight the troubles with moving/renaming the directory
submodule resides in.
P.S. I think that the problem with bad performance of too large index
is similar to the problems filesystems have with directories with large
number of files; some filesystems solved this problem.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: [PATCH] Make git-commit cleverer - have it figure out whether it needs -a automatically
From: Andy Parkins @ 2006-12-03 9:57 UTC (permalink / raw)
To: git
In-Reply-To: <7vbqmmzplm.fsf@assigned-by-dhcp.cox.net>
On Saturday 2006, December 02 08:09, Junio C Hamano wrote:
> Aside from the "working tree not matching index" safety valve I
> asked for, there is a more important case that "when the index
> matches HEAD" is not a safe enough check for this 'cleverness'.
As I pointed out, that safety valve made the whole thing a no-op.
> So that is _another_ exception you must handle.
>
> But I think the problem with this 'cleverer' commit runs
> deeper.
>
> Notice that you needed to say "The main idea is this cleverness,
> but foo pointed out this special case, and bar pointed out
> another, and we fixed all of these known ones and now this is
> good, let's apply it." in your proposed commit log message? You
> should smell fishiness in that kind of reasoning.
I don't believe so. In deep parts of a program cleverness is always a bad
idea. It just obfuscates functionality. However, this "cleverness" is about
making things easier for a human. Humans have a notoriously illogical set of
requirements for doing the right thing. As an example I'd hold up git's
clever date specification code; a computer would be perfectly happy to accept
epoch time, but instead humans like to be able to say "three weeks ago this
Thursday at five to four".
This patch is merely to make git-commit do something sensible when it would
normally do nothing. I was sure there would be more exceptions to when it
should activate; as git-commit is already a mess of "useful" extra switches.
To blame this patch for the fact that git-commit does a lot of things in
slightly different ways hardly seems fair.
As usual though: I don't mind, I'm not some huge proponent of this
functionality - /I/ get along fine with git-commit
> I really think the users would be much better off with
> consistent behaviour that is easy and simple to describe than a
> complex magic that does the right thing 99.9% of the time,
> because you either understand the complex magic or constantly in
> fear of the tool that can work against you 0.01% of the time.
I suspect that this patch is the least of git-commit's problems in that
department. "Easy to describe" is certainly not the case already, otherwise
none of this discussion (or patch) would ever have started.
Andy
--
Dr Andrew Parkins, M Eng (Hons), AMIEE
^ permalink raw reply
* [PATCH] [checkout-index] Give names to stages
From: Luben Tuikov @ 2006-12-03 9:49 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 371 bytes --]
One can now say "git-checkout-index --stage=ours ..." or
"git-checkout-index --stage=theirs ...", instead of having
to remember the corresponding number assigned to each stage.
Signed-off-by: Luben Tuikov <ltuikov@yahoo.com>
---
builtin-checkout-index.c | 11 ++++++++++-
1 files changed, 10 insertions(+), 1 deletions(-)
I find this quite helpful when resolving.
[-- Attachment #2: 1207600725-p1.txt --]
[-- Type: text/plain, Size: 808 bytes --]
diff --git a/builtin-checkout-index.c b/builtin-checkout-index.c
index b097c88..d54a290 100644
--- a/builtin-checkout-index.c
+++ b/builtin-checkout-index.c
@@ -236,8 +236,17 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
int ch = arg[8];
if ('1' <= ch && ch <= '3')
checkout_stage = arg[8] - '0';
+ else if (!strncmp(arg+8, "ancestor", 8))
+ checkout_stage = 1;
+ else if (!strncmp(arg+8, "ours", 4))
+ checkout_stage = 2;
+ else if (!strncmp(arg+8, "HEAD", 4))
+ checkout_stage = 2;
+ else if (!strncmp(arg+8, "theirs", 6))
+ checkout_stage = 3;
else
- die("stage should be between 1 and 3 or all");
+ die("stage should be "
+ "(1|ancestor)|(2|ours|HEAD)|(3|theirs)");
}
continue;
}
--
1.4.4.1.gc87e
^ permalink raw reply related
* Re: [RFC] Submodules in GIT
From: Andy Parkins @ 2006-12-03 9:42 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Git Mailing List
In-Reply-To: <200612021255.59972.Josef.Weidendorfer@gmx.de>
On Saturday 2006, December 02 11:55, Josef Weidendorfer wrote:
> > > 100644 blob 08602f522183dc43787616f37cba9b8af4e3dade xdiff-interface.c
> > > 100644 blob 1346908bea31319aabeabdfd955e2ea9aab37456 xdiff-interface.h
> > > 040000 tree 959dd5d97e665998eb26c764d3a889ae7903d9c2 xdiff
> > > 050000 link 0215ffb08ce99e2bb59eca114a99499a4d06e704 xyzzy
> > >
> > > where that 050000 is the new magic type (I picked one out of my *ss:
> > > it's not a valid type for a file mode, so it's a godo choice, but it
> > > could be anythign that cannot conflict with a real file), which just
> > > specifies the "link" part. The SHA1 is the SHA1 of the commit, and the
> > > "xyzzy" is obviously just the name within the directory of the
> > > submodule.
> >
> > Can I argue that the hash in that object should actually be to a real
> > object in the supermodule repository rather than a link?
>
> That is the thing we already are discussing here :-)
> IMHO, submodule IDs make a lot of sense, and this needs to specify the
> submodule ID at every link. Which would force us to use seperate objects.
I wasn't really going as deep as a submodule ID. Just moving the submodule
commit hash from the supermodule tree, to a supermodule "link" object. What
goes in that object is a separate problem I believe.
The primary reason I think it's a good idea is that it is consistent with
every other hash in the tree. It seems to be inconsistent to say
blob objects have a hash that points to an object in this repo
tree objects have a hash that points to an object in this repo
link objects have a hash the points to an object in a different repo
> However, I am not speaking about some separation issue, but more about a
> design decision. For fetching/pulling/merging, you want be able to
> distinguish submodules not only by the commit id into the submodule:
> multiple
> submodules could link into the same DAG (but different branches) of another
> repository which would make unique fetching/pulling/merge decisions
> difficult, especially when you think about the possibility that the
> relative root path of a submodule in a supermodule should be able to change
> at any supermodule commit.
I can't say I've understood what you mean here. There is no difference in
facilities if there is a link object in the local repository as well. It's
merely an extra layer of indirection. Apart from the tiny cost of
dereferencing that link object, there is no disadvantage.
Andy
--
Dr Andrew Parkins, M Eng (Hons), AMIEE
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Torgil Svensson @ 2006-12-03 9:19 UTC (permalink / raw)
To: Linus Torvalds, sf-gmane, sf, git, Martin Waitz
In-Reply-To: <Pine.LNX.4.64.0612021114270.3476@woody.osdl.org>
On 12/2/06, Linus Torvalds <torvalds@osdl.org> wrote:
>
> In other words, I don't think people expect or want something hugely more
> complicated than the CVS/modules kind of file.
What about the case when you want _everything_, do you then have to
know the names of all submodules, present and past?
If you have an old irrelevant submodule in the history that happens to
have the same name as one of them you are interested in, do you get
this as well?
During a debugging session it might be convenient to do a "all but X"
kind of fetch if you have a project dependent on several small modules
and one of them is the big black sheep.
For simple cases, I think it's sufficient to have the "everyone or
no-one" option. If git enforces sending submodules one by one and
requires the fetching side to specify links explicitly couldn't the
selection be left to the user to decide with "hooks" or plumbing?
Default hook could implement a simple white- or black-list.
^ permalink raw reply
* Re: [PATCH v2] make 'git add' a first class user friendly interface to the index
From: Alan Chandler @ 2006-12-03 9:16 UTC (permalink / raw)
To: git; +Cc: Nicolas Pitre, Junio C Hamano
In-Reply-To: <Pine.LNX.4.64.0612030028290.2630@xanadu.home>
On Sunday 03 December 2006 05:33, Nicolas Pitre wrote:
...
> +But here's a twist. If you do 'git commit <file1> <file2> ...' then only
> +the changes belonging to those explicitly specified files will be
> +committed, entirely bypassing the current "added" changes. Those "added"
> +changes will still remain available for a subsequent commit though.
> +
> +But for instance it is best to only remember 'git add' + 'git commit'
> +and/or 'git commit -a'.
> +
The "But for instance" seems a strange way of saying that.
How about
However, for normal usage you only have to remember 'git add' + 'git commit'
and/or 'git commit -a'.
--
Alan Chandler
^ permalink raw reply
* Re: [RFC] Submodules in GIT
From: Martin Waitz @ 2006-12-03 6:29 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Linus Torvalds, sf, git
In-Reply-To: <200612030155.09630.Josef.Weidendorfer@gmx.de>
[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]
hoi :)
On Sun, Dec 03, 2006 at 01:55:08AM +0100, Josef Weidendorfer wrote:
> On Saturday 02 December 2006 21:24, Martin Waitz wrote:
> > On Sat, Dec 02, 2006 at 12:17:44AM +0100, Josef Weidendorfer wrote:
> > > After some thinking, a submodule namespace even is important for checking
> > > out only parts of a supermodule, exactly because the root of a submodule
> > > potentially can change at every commit.
> >
> > have you ever thought about the idea that the location may be an
> > important thing to consider for your decision.
>
> Which decision, for what? Sorry, I do not understand.
to check out, or not to check out.
> What should such a general partial tree support look like? I suppose you
> want to configure paths which should not be checked out. As long as you
> say that a given submodule always has to exist at a given path, you are
> right: then, you can say: "Please, do not check out this submodule" which
> is the same as "Do not check out this path".
You could say something like "do not check out anything below "test/".
If then some submodule moves from "test/foo" to "build/foo", it will be
checked out, because this module is now not only used for testing, but
is needed for building in the new version of the supermodule.
--
Martin Waitz
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* [PATCH v2] make 'git add' a first class user friendly interface to the index
From: Nicolas Pitre @ 2006-12-03 5:33 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0612011444310.9647@xanadu.home>
I personally think this is going to make the GIT experience lot more
enjoyable for everybody. This brings the power of the index up front
using a proper mental model without talking about the index at all. See
for example how all the technical discussion has been evacuated from the
git-add man page.
Any content to be committed must be added together. Whether that
content comes from new files or modified files doesn't matter. You just
need to "add" it, either with git-add, or by providing git-commit with
-a (for already known files only of course). No need for a separate
command to distinguish new vs modified files please. That would only
screw the mental model everybody should have when using GIT.
Signed-off-by: Nicolas Pitre <nico@cam.org>
---
Documentation/git-add.txt | 55 +++++++++++++++++++++++--------------------
Documentation/tutorial.txt | 46 ++++++++++++++++++++++++++++++++----
builtin-add.c | 6 ++--
wt-status.c | 2 +-
4 files changed, 73 insertions(+), 36 deletions(-)
diff --git a/Documentation/git-add.txt b/Documentation/git-add.txt
index 6342ea3..411adad 100644
--- a/Documentation/git-add.txt
+++ b/Documentation/git-add.txt
@@ -3,7 +3,7 @@ git-add(1)
NAME
----
-git-add - Add files to the index file
+git-add - Add file content to the changeset to be committed next
SYNOPSIS
--------
@@ -11,16 +11,32 @@ SYNOPSIS
DESCRIPTION
-----------
-A simple wrapper for git-update-index to add files to the index,
-for people used to do "cvs add".
-
-It only adds non-ignored files, to add ignored files use
+All the changed file content to be committed together in a single set
+of changes must be "added" together with the 'add' command before using
+the 'commit' command.
+
+This is not only for adding new files. Even modified files must be
+added to the set of changes about to be committed. This command can
+be performed multiple times before a commit. The 'git status' command
+can be used to obtain a summary of what is included for the next commit.
+
+Note: don't forget to 'add' a file again if you modified it after the
+first 'add' and before 'commit'. Otherwise only the previous added
+state of that file will be committed. This is because git tracks
+content, so what you're really 'add'ing to the commit is the *content*
+of the file in the state it is in when you 'add' it. Of course there are
+legitimate usage cases for not updating an already added file content
+in order to commit a previous file state if you really know what
+you're doing.
+
+This command only adds non-ignored files, to add ignored files use
"git update-index --add".
+
OPTIONS
-------
<file>...::
- Files to add to the index (see gitlink:git-ls-files[1]).
+ Files to add content from.
-n::
Don't actually add the file(s), just show if they exist.
@@ -34,27 +50,12 @@ OPTIONS
for command-line options).
-DISCUSSION
-----------
-
-The list of <file> given to the command is fed to `git-ls-files`
-command to list files that are not registered in the index and
-are not ignored/excluded by `$GIT_DIR/info/exclude` file or
-`.gitignore` file in each directory. This means two things:
-
-. You can put the name of a directory on the command line, and
- the command will add all files in it and its subdirectories;
-
-. Giving the name of a file that is already in index does not
- run `git-update-index` on that path.
-
-
EXAMPLES
--------
git-add Documentation/\\*.txt::
- Adds all `\*.txt` files that are not in the index under
- `Documentation` directory and its subdirectories.
+ Adds content from all `\*.txt` files under `Documentation`
+ directory and its subdirectories.
+
Note that the asterisk `\*` is quoted from the shell in this
example; this lets the command to include the files from
@@ -62,15 +63,17 @@ subdirectories of `Documentation/` directory.
git-add git-*.sh::
- Adds all git-*.sh scripts that are not in the index.
+ Considers adding content from all git-*.sh scripts.
Because this example lets shell expand the asterisk
(i.e. you are listing the files explicitly), it does not
- add `subdir/git-foo.sh` to the index.
+ consider `subdir/git-foo.sh`.
See Also
--------
gitlink:git-rm[1]
-gitlink:git-ls-files[1]
+gitlink:git-mv[1]
+gitlink:git-commit[1]
+gitlink:git-update-index[1]
Author
------
diff --git a/Documentation/tutorial.txt b/Documentation/tutorial.txt
index fe4491d..0069fc3 100644
--- a/Documentation/tutorial.txt
+++ b/Documentation/tutorial.txt
@@ -87,14 +87,48 @@ thorough description. Tools that turn commits into email, for
example, use the first line on the Subject line and the rest of the
commit in the body.
-To add a new file, first create the file, then
-------------------------------------------------
-$ git add path/to/new/file
-------------------------------------------------
+Git tracks content not files
+----------------------------
+
+With git you have to explicitly "add" all the changed _content_ you
+want to commit together. This can be done in a few different ways:
+
+1) By using 'git add <file_spec>...'
+
+ This can be performed multiple times before a commit. Note that this
+ is not only for adding new files. Even modified files must be
+ added to the set of changes about to be committed. The "git status"
+ command gives you a summary of what is included so far for the
+ next commit. When done you should use the 'git commit' command to
+ make it real.
+
+ Note: don't forget to 'add' a file again if you modified it after the
+ first 'add' and before 'commit'. Otherwise only the previous added
+ state of that file will be committed. This is because git tracks
+ content, so what you're really 'add'ing to the commit is the *content*
+ of the file in the state it is in when you 'add' it.
+
+2) By using 'git commit -a' directly
+
+ This is a quick way to automatically 'add' the content from all files
+ that were modified since the previous commit, and perform the actual
+ commit without having to separately 'add' them beforehand. This will
+ not add content from new files i.e. files that were never added before.
+ Those files still have to be added explicitly before performing a
+ commit.
+
+But here's a twist. If you do 'git commit <file1> <file2> ...' then only
+the changes belonging to those explicitly specified files will be
+committed, entirely bypassing the current "added" changes. Those "added"
+changes will still remain available for a subsequent commit though.
+
+But for instance it is best to only remember 'git add' + 'git commit'
+and/or 'git commit -a'.
+
-then commit as usual. No special command is required when removing a
-file; just remove it, then tell `commit` about the file as usual.
+Viewing the changelog
+---------------------
At any point you can view the history of your changes using
diff --git a/builtin-add.c b/builtin-add.c
index febb75e..b3f9206 100644
--- a/builtin-add.c
+++ b/builtin-add.c
@@ -94,9 +94,6 @@ int cmd_add(int argc, const char **argv, const char *prefix)
newfd = hold_lock_file_for_update(&lock_file, get_index_file(), 1);
- if (read_cache() < 0)
- die("index file corrupt");
-
for (i = 1; i < argc; i++) {
const char *arg = argv[i];
@@ -131,6 +128,9 @@ int cmd_add(int argc, const char **argv, const char *prefix)
return 0;
}
+ if (read_cache() < 0)
+ die("index file corrupt");
+
for (i = 0; i < dir.nr; i++)
add_file_to_index(dir.entries[i]->name, verbose);
diff --git a/wt-status.c b/wt-status.c
index de1be5b..4b8b570 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -163,7 +163,7 @@ static void wt_status_print_changed_cb(struct diff_queue_struct *q,
int i;
if (q->nr)
wt_status_print_header("Changed but not updated",
- "use git-update-index to mark for commit");
+ "use git-add on files to include for commit");
for (i = 0; i < q->nr; i++)
wt_status_print_filepair(WT_STATUS_CHANGED, q->queue[i]);
if (q->nr)
--
1.4.4.1.gc419-dirty
^ permalink raw reply related
* Re: [PATCH] make 'git add' a first class user friendly interface to the index
From: Nicolas Pitre @ 2006-12-03 5:03 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vpsb36yem.fsf@assigned-by-dhcp.cox.net>
On Fri, 1 Dec 2006, Junio C Hamano wrote:
> > +Contrary to other SCMs, with GIT you have to explicitly "add" all the
> > +changed file content you want to commit together to form a changeset
> > +with the 'add' command before using the 'commit' command.
>
> ... "before a new commit is made"; it is not an offence to leave
> local changes outside the index. Staging such changes to all
> files is done using the "-a" flag and that is done "before a new
> commit is made", but not "before using the 'commit' command" --
> it is done at the same time.
Sorry but I don't think this is a good idea to tell that. At least not
here. Opening all the possibilities too fast at once is a good way to
create distrust. Let's focus on what the user needs to know about the
add command only.
The newbie that becomes not so newbie aftera while will deduce that he
actually _can_ leave local changes outside the index and he'll go "wow,
that is cool!" especially if he deduce this by himself. And that
deduction will happen in time while using the tool when the opportunity
for leaving local changes outside the index arises which is a much
better way to grasp the power of the index than by just being told about
it.
AS to the commit -a ... I think it is better to refer to the commit man
page once it has been refactored with the writeup you posted yourself
and simply direct the user with "You may also have a look at the
git-commit documentation for alternative ways to add content to a
commit."
> > +This is not only for adding new files. Even modified files must be
> > +added to the set of changes about to be committed. This command can
> > +be performed multiple times before a commit. The 'git status' command
> > +will give you a summary of what is included for the next commit.
> > +
> > +Note: don't forget to 'add' a file again if you modified it after the
> > +first 'add' and before 'commit'. Otherwise only the previous added
> > +state of that file will be committed. This is because git tracks
> > +content, so what you're really 'add'ing to the commit is the *content*
> > +of the file in the state it is in when you 'add' it. Of course there are
> > +legitimate usage cases for not updating an already added file content
> > +in order to commit a previous file state, but in this case you better
> > +know what you're doing.
>
> May be we could hint the reader that a faster-to-type
> alternative exists here. Perhaps...
Perhaps not.
> > +GIt tracks content not files
>
> s/I/i/
Yup
> > +But here's a twist. If you do 'git commit <file1> <file2> ...' then only
> > +the changes belonging to those explicitly specified files will be
> > +committed, entirely bypassing the current "added" changes. Those "added"
> > +changes will still remain available for a subsequent commit though.
> > +
> > +There is a twist about that twist: if you do 'git commit -i <file>...'
> > +then the commit will consider changes to those specified files _including_
> > +all "added" changes so far.
> > +
>
> I think there is another twist more deserving of mention than -i twist.
Actually I removed the -i twist entirely. It is simply too much for the
context of the tutorial and it is of no advantage for a newbie to even
know that -i exists just yet.
> If you jump the index using --only, what is committed with that
> commit becomes part of what is staged for the commit after that,
> and in order to prevent data loss, we disallow this sequence:
>
> $ git checkout
> $ edit foo
> $ git add foo ;# your new add to update the existing entry.
> $ edit foo
> $ git commit foo
>
> If we did not have the second edit (the behaviour is the same if
> we did not have "git add foo" there), this commit:
>
> * commits the changes to 'foo' (not because you staged it
> earlier with 'git add', but only because you said "commit
> foo" to invoke the '--only' semantics), obviously;
>
> * updates 'foo' in the index to what was committed.
>
> So if we allowed the above sequence to succeed, we would commit
> the result of the second edit, and after the commit, the index
> would have the result of the second edit. We would lose the
> state the user wanted to keep in the index while this commit
> jumped the index, and that is why we disallow it.
Great. This is perfectly fine behavior. But I think this definitely
doesn't belong in the tutorial. the probability for a newbie to perform
the above sequence is rather low, and even then the explanation belongs
in the failure message not in the tutorial. It can be as short as
"Please see git-commit man page and look for xyz for explanation about
this failure" if the inline explanation would be too long.
> > +But for instance it is best to only remember 'git add' + 'git commit'
> > +and/or 'git commit -a'.
> > +
> > +No special command is required when removing a file; just remove it,
> > +then tell `commit` about the file as usual.
>
> I wonder if this sequence should do the same as "git rm -f foo":
>
> $ /bin/rm foo
> $ git add foo
Well I think Linus' suggestions about git-rm are really sane. When
git-rm has been updated then it could be mentioned here, along with
git-mv. In the mean time I simply removed that paragraph.
^ permalink raw reply
* jgit performance update
From: Shawn Pearce @ 2006-12-03 4:59 UTC (permalink / raw)
To: git
With the help of Robin Rosenberg I've been able to make jgit's log
operation run (on average) within a few milliseconds of core Git.
Walking the 50,000 most recent commits from the Mozilla trunk[1]:
$ time git rev-list --max-count=50000 HEAD >/dev/null
core Git: 1.882s (average)
jgit: 1.932s (average)
(times are with hot cache and from repeated executions)
I think that is actually pretty good given that jgit is written
in Java using a fairly object-oriented design and has to deal with
some of the limitations of the language.
One of the biggest annoyances has been the fact that although Java
1.4 offers a way to mmap a file into the process, the overhead to
access that data seems to be far higher than just reading the file
content into a very large byte array, especially if we are going
to access that file content multiple times. So jgit performs worse
than core Git early on while it copies everything from the OS buffer
cache into the Java process, but then performs reasonably well once
the internal cache is hot. On the other hand using the mmap call
reduces early latency but hurts the access times so much that we're
talking closer to 3s average read times for the same log operation.
Anyway, jgit is now hopefully fast enough that we can start to build
some real functionality on top of it, and not need to wait several
minutes for answers from those features while debugging them. :)
**1** This is the pack file from Jon Smirl's import attempt.
--
^ permalink raw reply
* Re: [PATCH] make 'git add' a first class user friendly interface to the index
From: Nicolas Pitre @ 2006-12-03 4:34 UTC (permalink / raw)
To: Alan Chandler; +Cc: git, Carl Worth, Junio C Hamano
In-Reply-To: <200612020828.57989.alan@chandlerfamily.org.uk>
On Sat, 2 Dec 2006, Alan Chandler wrote:
> So I am with you that we need to effective teach
>
> git add <filename> #add content of filename to the SCM
> #edit <filename>
> git commit -a #commit current state of all tracked content
>
> first, and then move on to teach selective commiting
I think that's pretty much what my patch to the tutorial does.
The tutorial talks about:
1) git init-db
2) git add .
3) git commit
4) modifying some files then git diff
5) git commit file1 file2, or git commit -a
Then goes the discussion about what git add does and why. It is quite
early in the tutorial and making it earlier would be a bit premature.
Let's have the user make his first simple commit while blindly
following the instructions before going with the
actual usage model. At that point,since we just mentioned "git commit
file1 file2" or "git commit -a" will the user be in the proper mindset
to wonder why not using plain "git commit"... and incidentally the whole
explanation is there to follow immediately.
I'm reworking my patch with suggestions that have been posted so let's
hope it'll be even clearer.
^ permalink raw reply
* Re: [PATCH] make 'git add' a first class user friendly interface to the index
From: Nicolas Pitre @ 2006-12-03 4:22 UTC (permalink / raw)
To: Carl Worth; +Cc: Alan Chandler, git, Junio C Hamano
In-Reply-To: <87psb22qgu.wl%cworth@cworth.org>
On Sat, 2 Dec 2006, Carl Worth wrote:
> If git's model imposes the requirement, "we should first teach one
> thing, then move on to teach a subsequent thing", it would be just
> that much nicer if the commands themselves could help us do that,
> (because the default would do the thing they would need first, and
> then the user has to explicitly do _something_ else to get the
> subsequent thing).
Since I've been thinking about this issue I've come to the conclusion
that making commit -a the default for commit is not a good thing.
Why? Because we really want newbies to be tricked into using the index.
And teaching about the different ways to update the index in the
tutorial right after the first commit example is IMHO the best thing to
do.
Making commit -a the default would make it possible for newbies to
get along for a long while ignoring the usage model of git and that is
bad.
I think the idea is really to make "git commit" with a clean index more
clueful to the user. Right now it only says "use git-update-index to
mark for commit" which is really not that helpful, and actually the
point of failure with the example newbie problem you pointed out.
There is a compromise to reach. Sure the _user_ needs a proper model to
use the tool without being bothered with technical implementation or
architecture details. But we still need newbies to get into the git
model nevertheless, and having a default for git-commit geared towards
making it bump free for new users is not the way to go I think. The
"nothing to commit" message needs to be way more helpful with better
guidance and the git-commit default behavior should be overcome.
^ permalink raw reply
* Re: git-svn and empty directories in svn
From: Steven Grimm @ 2006-12-03 4:08 UTC (permalink / raw)
To: Eric Wong; +Cc: git
In-Reply-To: <20061203014756.GE1369@localdomain>
Eric Wong wrote:
> Since git-svn misses some other stuff (many property settings,
> externals) I'll be working on an internal logging format that can help
> track those things. It'd be nice to have a command like git svn
> checkout which works like git checkout; but empty directories are
> created.
Presumably once the submodule support is worked out, svn externals could
be represented as git-svn-managed submodules, yes?
In fact, I'd go so far as to say it should be a design goal of the
submodule support: you should be able to indicate somehow that a
submodule is a git clone of some non-git resource, and anything that
iterates through the submodules (e.g. to freshen them from their
respective origins) should know how to run git-svn or whatever so it's
all seamlessly integrated. I suppose that's a special case of making
git-svn and friends more tightly integrated with git in general; if the
git "push" and "fetch" commands know to run git-svn instead of talking
to a remote git repository, then it might Just Work for submodules.
Independent of the supermodule being managed by git-svn, the "my
software depends on externally-managed code" problem that submodules are
attempting to address would be solved a lot more comprehensively if the
remote code base could be an svn repository and git knew enough to run
git-svn as appropriate to keep it fresh. (Not just svn, of course; any
foreign CM system that has an equivalent of git-svn should work.) It'd
be pretty cool to have a supermodule that tied together some
native-git-managed code, a couple of external svn repositories, and a
CVS tree or two, all under a single umbrella with the details
automatically taken care of by default.
^ permalink raw reply
* Re: [PATCH] make 'git add' a first class user friendly interface to the index
From: Nicolas Pitre @ 2006-12-03 4:04 UTC (permalink / raw)
To: Alan Chandler; +Cc: git, Carl Worth, Junio C Hamano
In-Reply-To: <200612021805.09143.alan@chandlerfamily.org.uk>
On Sat, 2 Dec 2006, Alan Chandler wrote:
> The argument I was _trying_ to make was that we should teach the second
> conceptual model not the first one AND stick with just the git add command
> (in response to your (Carl's) statement earlier in the thread that there
> needs to be two separate commands) . My if statements were to illustrate
> that there are two fundamental ways of looking at this, not lots of ifs that
> newbies would have to consider. We should up-front (in the tutorial, in
> appropriate man pages) use the one conceptual model (and I also like Junio's
> argument that git should take an aggressive stance of this is how the
> conceptual model is rather than the "contrary to ..." approach).
Agreed.
^ permalink raw reply
* Re: On removing files and "git-rm is pointless"
From: Nicolas Pitre @ 2006-12-03 3:50 UTC (permalink / raw)
To: Sam Vilain; +Cc: Linus Torvalds, Carl Worth, git
In-Reply-To: <4571DB40.6020800@vilain.net>
On Sun, 3 Dec 2006, Sam Vilain wrote:
> Linus Torvalds wrote:
> > I'd like it more if it defaulted to actually removing the file, preferably
> > refusing to with an error message if the file didn't match the index.
>
> index, or HEAD version? Otherwise you can "update-index"; "rm" without
> seeing something wrong is happening.
Well tough then.
I think what Linus is proposing makes tons of sense.
If you do git rm by mistake then you can always do git checkout on that
file to get it back.
If you modified it so it doesn't match the index then git rm won't do
anything by default so you have a chance to think a bit more.
If you updated the index, didn't commit anything but then do git rm then
you certainly wanted to really rm the file.
If not then just feel the pain of your stupidity and start again from
the latest version.
^ permalink raw reply
* Re: Thoughts about memory requirements in traversals [Was: Re: [RFC] Submodules in GIT]
From: Josef Weidendorfer @ 2006-12-03 3:21 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Linus Torvalds, Martin Waitz, sf, git
In-Reply-To: <20061203024655.GD26668@spearce.org>
On Sunday 03 December 2006 03:46, Shawn Pearce wrote:
> Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> > Thinking even one step further:
> > Would it make sense to define an encoding format for the content of
> > commit and tree objects inside of packs, where the SHA1 is replaced by the
> > offset of the object in this pack?
> > As exactly the SHA1 is the least compressable thing, this could promise
> > quite a benefit.
> [...]
>
> This means that when we start to write out a commit we need to know
> the offset to the tree that commit references. But git-pack-objects
> sorts object by type: commit, tree, blob (I forget where tags go,
> but they aren't important in this context). So generally *all*
> commits appear before the first tree. So when we write out the first
> commit we need to know exactly how many bytes every commit will need
> (compressed mind you) in this pack so we can determine the position
> of the first tree. Now do this for every commit and every tree
> that those commits use... yes, its a lot of work to precompute
> and store all offsets before you even write out the first byte.
Yes, it looks like a hen-and-egg problem, but IMHO you can
handle it nicely with another redirection, i.e. a table you build
up while repacking the file, and storing this table at the end.
You simply sequentially renumber any object SHA, starting from 0
in the order you see them. You can do two renumberings, one for
the objects contained in the original pack (1), and one for the
external ones (2). Put these new numbers (with a bit distinguishing
(1) and (2)) as replacement into commit/tree objects.
At the end, you have the new offsets for objects in (1). Put
redirection tables for (1) [new number -> new offset]
and (2) [other new number->SHA1 of external object] at the end
of the new pack.
This way, you effectivly have removed all incompressable SHAs from
the pack file aside from one entry in the redirection tables for
each external object.
The only problem I see is how to decode the objects, i.e. how to
get the original SHA1 from an offset: we can not recalculate the
SHA1 from the object content as we changed the content itself.
But there should be a way to store the SHA1 in front of the object
somehow, perhaps it is already given by the current format?
Am I missing something here?
^ permalink raw reply
* Re: Thoughts about memory requirements in traversals [Was: Re: [RFC] Submodules in GIT]
From: Shawn Pearce @ 2006-12-03 2:46 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Linus Torvalds, Martin Waitz, sf, git
In-Reply-To: <200612030307.26429.Josef.Weidendorfer@gmx.de>
Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> Thinking even one step further:
> Would it make sense to define an encoding format for the content of
> commit and tree objects inside of packs, where the SHA1 is replaced by the
> offset of the object in this pack?
> As exactly the SHA1 is the least compressable thing, this could promise
> quite a benefit.
I actually had the same idea the other day. I discarded it after
thinking about it for a minute. Here's the problem:
Lets say we do this for the tree and parent IDs in a commit, because
these are the most commonly needed part of a commit during revision
traversal. So we want to put the offset to the tree and the offset
to each parent at the front of the commit somehow to make them very
cheap to access.
This means that when we start to write out a commit we need to know
the offset to the tree that commit references. But git-pack-objects
sorts object by type: commit, tree, blob (I forget where tags go,
but they aren't important in this context). So generally *all*
commits appear before the first tree. So when we write out the first
commit we need to know exactly how many bytes every commit will need
(compressed mind you) in this pack so we can determine the position
of the first tree. Now do this for every commit and every tree
that those commits use... yes, its a lot of work to precompute
and store all offsets before you even write out the first byte.
Its even worse with parent commits because ancestors tend to appear
behind the commit (newest->oldest) so that "git log" can benefit
from OS read-ahead. So you also have to keep track of your parent
commmit offsets. Not pretty.
Extending that idea to tree objects (store the offset of the entry)
makes the issue even uglier.
Oh, and packs aren't entirely self-contained. A pack is only self
contained in the sense that no object in the pack deltafies against
an object outside of the pack[1]. However by design an object
(e.g. a commit or a tree) can reference an object which is either
loose or which is in another pack. This is especially important
for every large projects where not every commit/tree/tag/blob will
fit into one 4 giB file.
**1** Except in the case of thin packs, which are used only on the
network and only to save bandwidth.
> AFAIK, we currently only use these offsets for referencing objects in
> delta chains.
Yes, that's a recent feature to reference a delta base.
--
^ permalink raw reply
* Re: [PATCH] Oops, get the commit first, then the tree.
From: Shawn Pearce @ 2006-12-03 2:34 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <200612030145.45567.robin.rosenberg@dewire.com>
Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> This made the eclipse plugin unable to connect to Git.
...
> --- a/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
> +++ b/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
> @@ -168,7 +168,8 @@ public class Repository {
> }
>
> public Tree mapTree(final String revstr) throws IOException {
> - final ObjectId id = resolve(revstr);
> + Commit commit = mapCommit(revstr);
> + final ObjectId id = commit.getTreeId();
> return id != null ? mapTree(id) : null;
I don't want to apply the fix above because mapTree(String) should
be able to take an arbitrary SHA1 expression and return the Tree
object corresponding to that. Your change only permits commits to
be resolved; but if the revstr resolved to a tag we'd need to peel
back the tag to get to the tree. Ideally that should be done in
mapTree(ObjectId), based on the type of thing ObjectId points to.
I think the bug was caused by a recent breakage in Repository.mapTree
(42691339). I fixed it in 020e0ee but neglected to push it out.
Actually I'm surprised mapTree(ObjectId) even works before 020e0ee;
I just don't see how its right...
--
^ permalink raw reply
* Re: Thoughts about memory requirements in traversals [Was: Re: [RFC] Submodules in GIT]
From: Linus Torvalds @ 2006-12-03 2:25 UTC (permalink / raw)
To: Josef Weidendorfer; +Cc: Martin Waitz, sf, git
In-Reply-To: <200612030307.26429.Josef.Weidendorfer@gmx.de>
On Sun, 3 Dec 2006, Josef Weidendorfer wrote:
>
> Thinking about this...
> You have to make very sure to always update the caching layer containing
> the backlinks on every addition of a further object. You can do this
> because you always reached this new object by some other object, which
> exactly is the backpointer.
You're missing the big issue.
The issue is that a cache like that would ABSOLUTELY SUCK.
You could speed up the non-common operations with it, but:
- any changes would become a LOT more expensive to do, because they all
need to update every single object they add (ie a "commit" would now
have to add backpointers TO EVERY SINGLE BLOB).
Imagine what this does to something like the kernel, where a commit
reaches 22,000 files!
You can do it at a finer granularity (ie do just the direct backlinks
and only do the "tree->blob" and "tree->tree" things rather than the
full commit reachability, but it's still going to be MUCH more painful
than what we do now.
- the cache would be a lot bigger than the current pack-files, and it
would be fragile as hell to boot. Because it needs to get rewritten for
every operation, it gets corrupted much more easily, and that's
ignoring things like race conditions, so it would now need a ton of
locking that git simply doesn't do at all.
- everything would basically slow down.
- you couldn't do shared object databases AT ALL, because backpointers
wouldn't work. The whole _reason_ you can share object databases is the
same reason we can't have backpointers: objects are immutable and never
change depending on circustances.
The _only_ downside of the current situation is literally the 24 or 28
bytes per object that we look at. For most operations, we don't even look
at that many objects, so it's really the worst-case things.
> In fact, this "cache" can be created with a usual object traversal
> (which has the original memory requirement), but as long as we do
> not add objects to the database, further traversals would only need
> a fraction of memory.
Right. If the project is totally read-only, the cache would work well.
For real development, it would SUCK. It would make things like "git reset"
very expensive indeed, for example (you'd have to unwind the whole cache:
either regenerating it - which would take minutes - or being very careful
indeed and being able to always remove objects properly and keeping track
of them 100%).
IOW, it's nasty nasty nasty. And it doesn't really even help anything but
a case that we actually already handle really well (I spent a lot of
effort on making the memory footprint minimal).
But it does mean that you do NOT want to traverse a hundred different
project "as if" they were one. That's really the only thing it means.
And since you can do submodules as independent projects, and you SHOULD do
them that way for tons of other reasons _anyway_, even that isn't a reason
to screw up all the _wonderful_ properties of the git object database.
So what I'm trying to say is that the immutable non-backpointer nature of
the git database is what makes it so WONDERFUL. It's efficient, it's
dense, it's stable, and it allows us all the clever things we do. But it
means that we do end up alway spending 28 bytes per object, and we can
never throw those 28 bytes away during a single "traversal" run.
^ permalink raw reply
* Re: EGIT unpackedreadr problem
From: Shawn Pearce @ 2006-12-03 2:25 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <200612030200.13857.robin.rosenberg@dewire.com>
Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> Subject: [PATCH] Utility to show a log.
Thanks. This is somewhat handy so I've applied the utility program.
Like your other patch to Commit class this one had some whitespace
issues. In the future could you apply the Eclipse formatter (using
the Java Conventions style) to the source before genererating a diff?
> Not sure what's going on. The UnpackedReader "looks" like it does
> the equivalent of what it did before,but apparently it isn't.
Yea... I broke the legacy header parsing in 42691339.
The breakage actually showed up in one of the unit tests
(test009_CreateCommitOldFormat) but I didn't notice it before
pushing the changes out as I failed to run the entire unit test
suite first. Whoops.
I found it and fixed it in f70704f9. The issue was we were copying
the remaining data from the Inflater ontop of the data we had
already decompressed out for the header; this meant that we lost the
leading bytes of each object. I just neglated to push the fix out.
Double whoops.
--
^ permalink raw reply
* Re: [EGIT PATCH] Convert author and comment on demand.
From: Shawn Pearce @ 2006-12-03 2:16 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <200612030145.09576.robin.rosenberg@dewire.com>
Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> This sppeds up reading commits a lot by only store the byte
> array data when reading commits.
Thanks. I was working on something similar but did not have
a chance to finish it. I've applied your patch instead.
> + try {
> + BufferedReader br=new BufferedReader(new InputStreamReader(new
> ByteArrayInputStream(raw)));
> + String n=br.readLine();
Something's wrong with your mail client... the patch was mangled.
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox