Importing from tarballs; add, rm, update-index?

Importing from tarballs; add, rm, update-index?

[Chris Riddoch]

Hi, folks.

I've got a very, very old codebase I'm trying to wrap my head around.
It was apparently once tracked with RCS, but the ,v files are long
gone and all that remains are a series of tarballs on an FTP site
containing alpha, beta, and final releases of various versions of the
project.  There's a logical progression, but between each there are
new files, deleted files, and lots of changed files.  gitk will at
least help me make sense of the actual changes.  I've got part of a
shell script to automate this process.

Here's the problem.

I have tried to follow the debate on git add, rm, commit -a, etc.  But
I can't figure out how to simply say, take the full state of the
working directory, and make the index directly reflect that state.
Additions, removals, and differences alike.  One step, preferably.

My 2 cents on recent UI changes: I'm quite uncertain about the correct
(future 1.5.0) way of handling this kind of change to the index.  To
elaborate:

First, specifying extra files after 'git commit' bypasses the index.
If I remove foo.txt, and want to make a new commit reflecting only
that removal, would 'git commit foo.txt' do what I mean?  Apparently
so; I had to test it to find out, though.  It is a little surprising.

Using the 'add' command to really mean 'record the state of this file'
is confusing.  It makes me think of CVS's add (predictably), which
makes note of a new file and I think, "Well, I read on the list that
this is just another kind of 'adding content to the index,' right?"
Okay, I can accept that.  But using the 'git add .' idiom makes me
wonder whether the subdirectories are also supposed to be added, or
whether I need to worry about a recursive option.  Oddly, 'git pull .'
doesn't make me blink.  But then I need to remember to use 'git add'
to keep track of most changes in the index, new files and edits alike.
 I suspect a newbie coming from CVS might even use the word "re-add"
in their head to understand 'add'.

But this makes 'git rm' quite confusing.  After thinking I finally
understand 'git add', I think 'git rm' would mean "record the new
(nonexistent) state of this file in the index."  And then I'm
surprised to discover the file in my working directory actually
removed.  No, I guess I needed --cached.  Oops.  Wait a minute.
Cache?  The same cache mentioned in glossary.txt as being "Obsolete
for index?"

If rm is the opposite of add, shouldn't it work on the index by
default?  Hmm... "adding content to the index."  This file being
removed is new information that needs to be added to the index, like a
modification.  Removing a file is a kind of modification, after all.
I think this was the logic behind someone's suggestion of 'git add
--remove' which is quite ridiculous but follows somewhat naturally
from the idea that 'git add' is supposed to add information about
changes in the working directory to the index.

At this point, I think the behavior of 'git add' is as ridiculous as
that of 'git rm'.  Recent changes to 'git add' and 'git rm' make me
believe we're indecisive about whether we want people to think in
terms of the index or not.  I get the impression we're leaning towards
encouraging awareness of the index, because it's so helpful with
merges.

You know what I think we need?  A good command for updating the index.
 It could even be both porcelain and plumbing, if it had the right
error-checking.

I suggest calling it something like update-index.  ;)

Unless I'm completely misunderstanding git?

-- 
epistemological humility
  Chris Riddoch

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Chris Riddoch <riddochc@gmail.com> wrote:
> I've got a very, very old codebase I'm trying to wrap my head around.
> It was apparently once tracked with RCS, but the ,v files are long
> gone and all that remains are a series of tarballs on an FTP site
> containing alpha, beta, and final releases of various versions of the
> project.  There's a logical progression, but between each there are
> new files, deleted files, and lots of changed files.  gitk will at
> least help me make sense of the actual changes.  I've got part of a
> shell script to automate this process.
> 
> Here's the problem.
> 
> I have tried to follow the debate on git add, rm, commit -a, etc.  But
> I can't figure out how to simply say, take the full state of the
> working directory, and make the index directly reflect that state.
> Additions, removals, and differences alike.  One step, preferably.

Hum... something like the following (completely untested!) should do the
trick:

  cd /basedir
  mkdir codebase; cd codebase; git init-db
  for version in 1.0 1.1 1.1a 1.1b 2.0.0 ...; do
     cd /basedir
     tar xf tarball-$version.tar
     mv codebase-$version/* codebase    # Take care to move everything!
     cd codebase
     git add .
     git commit -a -m "Updated to $version"
     rm -rf *                  # Delete everything except for git stuff
  done

Files that don't change will be recorded as is (git tracks contents).
    
Comments?
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Johannes Sixt]

"Horst H. von Brand" wrote:
> Hum... something like the following (completely untested!) should do the
> trick:
> 
>   cd /basedir
>   mkdir codebase; cd codebase; git init-db
>   for version in 1.0 1.1 1.1a 1.1b 2.0.0 ...; do
>      cd /basedir
>      tar xf tarball-$version.tar
>      mv codebase-$version/* codebase    # Take care to move everything!
>      cd codebase
>      git add .
>      git commit -a -m "Updated to $version"
>      rm -rf *                  # Delete everything except for git stuff
>   done

You can let GIT_DIR point somewhere outside the extracted directory.

If the dates of your tarballs are meaningful, you can use something like
this:

export GIT_AUTHOR_DATE="$(date --reference=foo-1.1.tar +%s) -07:00"

   
-- Hannes

Re: Importing from tarballs; add, rm, update-index?

[Morten Welinder]

I did this with a large pile of tarballs, see

    http://blogs.gnome.org/view/mortenw/2007/01/07/0

for the script.

Bottom line: git is very, very good at compressing a pile of related tarballs.
In fact, about 10x better than gzip.

(Not really surprising, of course.)

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Johannes Sixt <J.Sixt@eudaptics.com> wrote:
> "Horst H. von Brand" wrote:
> > Hum... something like the following (completely untested!) should do the
> > trick:
> > 
> >   cd /basedir
> >   mkdir codebase; cd codebase; git init-db
> >   for version in 1.0 1.1 1.1a 1.1b 2.0.0 ...; do
> >      cd /basedir
> >      tar xf tarball-$version.tar
> >      mv codebase-$version/* codebase    # Take care to move everything!
> >      cd codebase
> >      git add .
> >      git commit -a -m "Updated to $version"
> >      rm -rf *                  # Delete everything except for git stuff
> >   done
> 
> You can let GIT_DIR point somewhere outside the extracted directory.

OK! Then it would be:

   cd /basedir
   mkdir codebase/.git; cd codebase; git init-db
   for version in 1.0 1.1 1.1a 1.1b 2.0.0 ...; do
      cd /basedir
      tar xf tarball-$version.tar
      cd codebase-$version
      git --git-dir=/basedir/codebase/.git add .
      git --git-dir=/basedir/codebase/.git \
          commit -a -m "Updated to $version"    # Spice up to taste
   done

And the "add" looks strangely misplaced... but it /does/ its job.

> If the dates of your tarballs are meaningful, you can use something like
> this:
> 
> export GIT_AUTHOR_DATE="$(date --reference=foo-1.1.tar +%s) -07:00"

Better use it to make up a commit message. Could use inline docs for this.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

"Chris Riddoch" <riddochc@gmail.com> writes:

> I suggest calling it something like update-index.  ;)

Exactly.

People new to git need to learn that the next commit is prepared
not in the working tree but in a separate entity (staging area),
and there are ways to update what it consists of.  If that
concept is new for people coming from other SCM, renaming
"index" to "staging area" only reduces potential confusion
(because "index" is too generic word that can mean anything) --
it does not remove the need to learn the new concept.

And we have called that staging area "the index", and the act of
updating what it consists of has been called "updating the
index" for a long time.  The primary command to do so has been
"git-update-index" plumbing, but we added some sugarcoating and
now "git-add" and "git-rm" (also "git-merge", "git-am" and
friends "updates the index" for automatable cases) are two most
visible ways for the users.

The logical name for the operation, if we _were_ to have only
one command for "updating the index", is "git-update" or
"git-update-index".  I do not have anything against "git stage"
but I simply do not see the point, other than "git update" would
imply something entirely different to people coming from other
SCM so we would want to avoid the word, and "git update-index"
is too long to type every day.

In any case, there are valid reasons that update-index has --add
and --remove options to force callers to specifically say "Yes I
know I am talking about unusual things, please".  If we _were_
to do a single command (be it "git-update" or "git-stage"), it
needs the same --add/--remove safety, which makes "it's too long
to type every day -- let's have a single Porcelain-ish" argument
somewhat moot.  We can have "git-add" and "git-rm" instead.

And indeed we do.  That's where we currently stand.

> First, specifying extra files after 'git commit' bypasses the index.

Which I happen to think is a misfeature.

> If I remove foo.txt, and want to make a new commit reflecting only
> that removal, 

If you try latest 'git status' in that situation, it would say
foo.txt is deleted but not updated and suggests to use git-add or
git-rm to include it in the commit you are creating.

> ...  But then I need to remember to use 'git add'
> to keep track of most changes in the index, new files and edits alike.

"git commit -a" is your friend.  I think new people should be
taught that form first, and then "git status" output, and then
what "git status" suggests them to do (which is "git add" or
"git rm").

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On 2007-01-12, Junio C Hamano <junkio@cox.net> wrote:
> "Chris Riddoch" <riddochc@gmail.com> writes:
>
>> I suggest calling it something like update-index.  ;)
>
> Exactly.
>
> People new to git need to learn that the next commit is prepared
> not in the working tree but in a separate entity (staging area),
> and there are ways to update what it consists of.  If that
> concept is new for people coming from other SCM, renaming
> "index" to "staging area" only reduces potential confusion
> (because "index" is too generic word that can mean anything) --
> it does not remove the need to learn the new concept.
>
> And we have called that staging area "the index", and the act of
> updating what it consists of has been called "updating the
> index" for a long time.  The primary command to do so has been
> "git-update-index" plumbing, but we added some sugarcoating and
> now "git-add" and "git-rm" (also "git-merge", "git-am" and
> friends "updates the index" for automatable cases) are two most
> visible ways for the users.
>
> The logical name for the operation, if we _were_ to have only
> one command for "updating the index", is "git-update" or
> "git-update-index".  I do not have anything against "git stage"
> but I simply do not see the point, other than "git update" would
> imply something entirely different to people coming from other
> SCM so we would want to avoid the word, and "git update-index"
> is too long to type every day.

I already proposed it but I like "git refresh". It describes precisely
whats happening - the previous content of the file known to git is
refreshed. AFAIK "update" isn't such a good choice because people from
other VCS would expect something else(think e.g. of cvs update).
>
> In any case, there are valid reasons that update-index has --add
> and --remove options to force callers to specifically say "Yes I
> know I am talking about unusual things, please".  If we _were_
> to do a single command (be it "git-update" or "git-stage"), it
> needs the same --add/--remove safety, which makes "it's too long
> to type every day -- let's have a single Porcelain-ish" argument
> somewhat moot.  We can have "git-add" and "git-rm" instead.
>
> And indeed we do.  That's where we currently stand.
>

Me doesn't really like the new semantics of "git-add", because it does
two seperate things - it adds new files and it refreshes the content of
previously known files. These two are, at least for me, two seperate
operations. And reading "git add" only suggests to me the first - adding
new files.

-Peter

Re: Importing from tarballs; add, rm, update-index?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2156 bytes --]

On Fri, 12 Jan 2007 10:47:56 -0800, Junio C Hamano wrote:
> People new to git need to learn that the next commit is prepared
> not in the working tree but in a separate entity (staging area),

Why's that? Git really provides two separate notions here.

1. Prepare content in staging area, then commit it.

	Working with this notion, the commands a user might use are
	"git update-index", (and more recently "git add"), "git
	commit", and "git commit -a".

2. Prepare content in working tree, and commit from there

	Working with this notion, the commands a user might use are
	"git add" (to add a new file---not the new staging add),
	"git commit files...", and "git commit -a".

> "git-update-index".  I do not have anything against "git stage"
> but I simply do not see the point, other than "git update" would
> imply something entirely different to people coming from other
> SCM so we would want to avoid the word, and "git update-index"
> is too long to type every day.

Yes, those would be the only reasons for the rename. If those aren't
important then just declare "update-index" the porcelain command.

> > First, specifying extra files after 'git commit' bypasses the index.
>
> Which I happen to think is a misfeature.

But this feature _exists_ in git, and it's also _extremely_ useful.

It also just happens to provide an alternate concept by which someone
can understand "commit -a", (not that "commit -a" shows up under both
of the conceptual notions I described above).

I still don't understand how people argue so strongly against the
conceptual notion of "committing content from working tree" when git
does provide this functionality, and it also just happens to exactly
match what a lot of people want to do anyway.

The index is still there, and will still benefit the user during merge
conflicts, but there's no requirement for the user to think of the
index as a staging area to get that benefit. And there's particularly
no need for the user to be forced to conceptualize every commit as
first being staged before committed.

Where's the actual benefit to the user in only promoting the "stage
content, then commit" model?

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

Peter Baumann <Peter.B.Baumann@stud.informatik.uni-erlangen.de>
writes:

> Me doesn't really like the new semantics of "git-add", because it does
> two seperate things - it adds new files and it refreshes the content of
> previously known files.

http://thread.gmane.org/gmane.comp.version-control.git/32452/focus=32792

Re: Importing from tarballs; add, rm, update-index?

[Jakub Narebski]

[Cc: git@vger.kernel.org]

Chris Riddoch wrote:

> I've got a very, very old codebase I'm trying to wrap my head around.
> It was apparently once tracked with RCS, but the ,v files are long
> gone and all that remains are a series of tarballs on an FTP site
> containing alpha, beta, and final releases of various versions of the
> project.  There's a logical progression, but between each there are
> new files, deleted files, and lots of changed files.  gitk will at
> least help me make sense of the actual changes.  I've got part of a
> shell script to automate this process.

http://git.or.cz/gitwiki/GitFaq#import-tar

(because this kind usage is/should be rare, using plumbing here is not an
error, I think).
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On Fri, Jan 12, 2007 at 11:43:32AM -0800, Junio C Hamano wrote:
> Peter Baumann <Peter.B.Baumann@stud.informatik.uni-erlangen.de>
> writes:
> 
> > Me doesn't really like the new semantics of "git-add", because it does
> > two seperate things - it adds new files and it refreshes the content of
> > previously known files.
> 
> http://thread.gmane.org/gmane.comp.version-control.git/32452/focus=32792
> 

[For the readers conveniance I added the above mail below.]

Yes. I fully second Linus opinion. But I think there should be a difference in
adding completly new content to the index (number of entries in the index grows)
or replacing content in the index.

That's what I'm aiming for.

-Peter

?> 
?> If the "create file; git add; edit file; git commit" confusion isn't
?> blisteringly obvious to the git maintainers then I think I have to
?> give up here.
?> 
?> And this isn't just CVS-induced brain damage.
?
?I'm sorry, but you are wrong.
?
? It really _is_ CVS-induced brain damage, and I'm trying to teach you. You 
? can give up, but that's really "refuse to see the damage that systems like 
? RCS and CVS has done to the world"
? 
? The fundamental brain damage that CVS (and RCS, and SVN, and just about 
? anything else) has had is thinking that "filenames" (and sometimes this is 
? "fixed" to be "file ID's") are somehow special, and a totally separate 
? thing from "file contents".
? 
? Really. It's a BUG. It's a deficiency in CVS and friends. And it's a 
? deficiency that you have gotten so used to that you don't even see that 
? it's simply obviously NOT TRUE.
? 
? You _cannot_ have a filename without the contents of that filename. That 
? whole concept doesn't make sense, except in the twisted AND WRONG mental 
? model of "files have identities even without content".
? 
? The whole point of git is that it is about "project state" and the history 
? that binds those states together. People have kind of come to accept that, 
? and a lot of people realize what it means, but I don't think you've really 
? accepted what it means for something as simple as a "git add" command.
? 
? Again, totally ignore the index. Imagine that it doesn't exist. Imagine 
? that you never actually learnt about it, and that none of the 
? documentation ever mentions it, and just ask yourself:
? 
? 	"What does 'adding a file' really mean?"
? 
? I mean _really_. It cannot be about the "filename", because a filename 
? simply doesn't have any meaning alone. Remember what git is all about.
? 
? No, when you do a "git add", YOU DO NOT TALK ABOUT FILENAMES AT ALL.
? 
? 	NOT EVEN CLOSE!
? 
? No. Git is, and has always been, all about tracking project content. The 
? fact that CVS is crap, and thinks that "filenames" are special (and this 
? causes major problems when you do renames), and the fact that SVN is crap, 
? and things that "file identities" are special (and this causes major 
? problems when you split a file or when two files join) is very much about 
? THEIR F*CKING IDIOTIC FUNDAMENTAL BRAINDAMAGE!
? 
? So take five minutes to really think about that. Take an hour. Take a 
? week. Ponder it.
? 
? What does it mean to "add" something to a project? It has _nothing_ to do 
? with "filenames". Yeah, the filename obviously exists, but it's not 
? something that exists on its own. You add the ONLY thing that git tracks. 
? 
? You add CONTENT.
? 
? When you do "git add file.c" you aren't adding a filename to the list of 
? files that git knows about. Not even CLOSE. No. You are really adding 
? _content_ to the project you are tracking. You haven't bound it to a 
? commit yet, but it's there. It's there both conceptually, and very much in 
? a real technical sense too (you've literally added the git object that 
? that file describes to the object database - the "commit" and "tree" 
? objects to tie it all together is just waiting to be added, but they 
? really just expose it - the actual file object has already been created 
? when you do "git add".)
? 
? So yes, you very much ARE talking about CVS braindamage. The reason why
? 
? 	git add file.c
? 	echo New line >> file.c
? 	git commit
? 
? commits the _old_ content, is very much because git is ALL ABOUT THE 
? CONTENT. It has _never_ been about filenames. And it _shouldn't_ be about 
? filenames, because that would be BUGGY AND BROKEN.
? 
? Sorry for shouting, but as long as you think "git add" adds a filename, 
? you're just not getting it. And I think it's really sad that you don't 
? even seem to understand that yes, this _is_ braindamage that has been 
? forced upon you by decades of mental rape done by bad source control 
? systems.
? 
? Please. File identities are _bad_ in the SVN kind of setting. The CVS kind 
? of "filename == file identity" is even _worse_, but it's still exactly the 
? same disease. It's the disease of thinking that metadata is somehow 
? "different" from real data, and that "files" have identities that are 
? somehow separate from the data they contain.
? 
? Face it, git is consistent, and if it acted the way you seem to expect it 
? to act, it would actually be a BUG. Exactly because you cannot and MUST 
? NOT think that "filename" is something that has meaning without "file 
? content" (or "file type" and "file permissions" - they all go together).
? 
? And notice? NONE OF THIS HAS ANYTHING AT ALL TO DO WITH 'INDEX'!! The 
? explanation above is not "this is how the index works". It's a much more 
? fundamnetal issue of getting the right mental model, where the only thing 
? that matters is contents.
? 
? So even without an index, "git add" should work the way it works, once you 
? can just let go of the broken model that is CVS.
? 
? Please. Join me, Luke. The power of the git side is stronger. I am your 
? father. 
? 
?			Linus

Re: Importing from tarballs; add, rm, update-index?

[Johannes Schindelin]

Hi,

On Fri, 12 Jan 2007, Junio C Hamano wrote:

> "Chris Riddoch" <riddochc@gmail.com> writes:
>
> > First, specifying extra files after 'git commit' bypasses the index.
> 
> Which I happen to think is a misfeature.

You probably should not teach people about that feature right away, 
because it has huge potential of shooting-yourself-in-your-own-foot.

But darn it, it's _useful_.

Very often I happen to find a subtle bug in the middle of my work. Which 
basically means that I have a dirty working tree, a dirty index, and I 
_need_ to commit something completely different. Usually it is a 
one-liner, which I don't even have to test in isolation, but with my dirty 
working tree. And usually it is in a file which was non-dirty until I 
put in a fix, so committing specific files -- bypassing the current index 
-- _is_ useful.

Ciao,
Dscho

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> You probably should not teach people about that feature right away, 
> because it has huge potential of shooting-yourself-in-your-own-foot.
>
> But darn it, it's _useful_.

I am not saying it is not useful.  It is just I do not think
people should be taught about before understanding what it
means.

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

Peter Baumann <waste.manager@gmx.de> writes:

> Yes. I fully second Linus opinion. But I think there should be
> a difference in adding completly new content to the index
> (number of entries in the index grows) or replacing content in
> the index.

Huh?

> ? So take five minutes to really think about that. Take an hour. Take a 
> ? week. Ponder it.

I'd second this ;-).

> ? What does it mean to "add" something to a project? It has _nothing_ to do 
> ? with "filenames". Yeah, the filename obviously exists, but it's not 
> ? something that exists on its own. You add the ONLY thing that git tracks. 
> ? 
> ? You add CONTENT.
> ? 
> ? When you do "git add file.c" you aren't adding a filename to the list of 
> ? files that git knows about. Not even CLOSE. No. You are really adding 
> ? _content_ to the project you are tracking.

Read this again, please.  Ponder it if you may.

> ? So even without an index, "git add" should work the way it works, once you 
> ? can just let go of the broken model that is CVS.
> ? 
> ? Please. Join me, Luke. The power of the git side is stronger. I am your 
> ? father. 
> ? 
> ?			Linus

And probably I am your uncle ;-).

Re: Importing from tarballs; add, rm, update-index?

[Brian Gernhardt]

On Jan 12, 2007, at 2:43 PM, Junio C Hamano wrote:

> Peter Baumann <Peter.B.Baumann@stud.informatik.uni-erlangen.de>
> writes:
>
>> Me doesn't really like the new semantics of "git-add", because it  
>> does
>> two seperate things - it adds new files and it refreshes the  
>> content of
>> previously known files.
>
> http://thread.gmane.org/gmane.comp.version-control.git/32452/ 
> focus=32792

Should this be added to Documentation/rants/filename- 
braindamage.txt?  ;-)

~~ Brian

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On Fri, Jan 12, 2007 at 04:48:09PM -0800, Junio C Hamano wrote:
> Peter Baumann <waste.manager@gmx.de> writes:
> 
> > Yes. I fully second Linus opinion. But I think there should be
> > a difference in adding completly new content to the index
> > (number of entries in the index grows) or replacing content in
> > the index.
> 
> Huh?
> 
> > ? So take five minutes to really think about that. Take an hour. Take a 
> > ? week. Ponder it.
> 
> I'd second this ;-).
> 
> > ? What does it mean to "add" something to a project? It has _nothing_ to do 
> > ? with "filenames". Yeah, the filename obviously exists, but it's not 
> > ? something that exists on its own. You add the ONLY thing that git tracks. 
> > ? 
> > ? You add CONTENT.
> > ? 
> > ? When you do "git add file.c" you aren't adding a filename to the list of 
> > ? files that git knows about. Not even CLOSE. No. You are really adding 
> > ? _content_ to the project you are tracking.
> 
> Read this again, please.  Ponder it if you may.
> 

Yes. I am adding content. And not a file. But at least to me, it makes a
*BIG* difference if I'm adding totally new content (reserving one more
bucket where to place to content) or just replacing the content *in* one
of those already reserved buckets. And that has nothing to do with
files (or at least the silly me can't grok it).

> > ? So even without an index, "git add" should work the way it works, once you 
> > ? can just let go of the broken model that is CVS.
> > ? 
> > ? Please. Join me, Luke. The power of the git side is stronger. I am your 
> > ? father. 
> > ? 
> > ?			Linus
> 
> And probably I am your uncle ;-).
> 

You are welcome :-)

-Peter

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On 2007-01-13, Brian Gernhardt <benji@silverinsanity.com> wrote:
>
> On Jan 12, 2007, at 2:43 PM, Junio C Hamano wrote:
>
>> Peter Baumann <Peter.B.Baumann@stud.informatik.uni-erlangen.de>
>> writes:
>>
>>> Me doesn't really like the new semantics of "git-add", because it  
>>> does
>>> two seperate things - it adds new files and it refreshes the  
>>> content of
>>> previously known files.
>>
>> http://thread.gmane.org/gmane.comp.version-control.git/32452/ 
>> focus=32792
>
> Should this be added to Documentation/rants/filename- 
> braindamage.txt?  ;-)
>
> ~~ Brian

Ok. Obviously I should't have said that it add "files" (gr, silly me).
What I meant was it adds the content of files. But there is a difference
in adding and replacing content.

-Peter

Re: Importing from tarballs; add, rm, update-index?

[Johannes Schindelin]

Hi,

On Sat, 13 Jan 2007, Peter Baumann wrote:

> On Fri, Jan 12, 2007 at 04:48:09PM -0800, Junio C Hamano wrote:
> > Peter Baumann <waste.manager@gmx.de> writes:
> > 
> > > ? What does it mean to "add" something to a project? It has 
> > > _nothing_ to do ? with "filenames". Yeah, the filename obviously 
> > > exists, but it's not ? something that exists on its own. You add the 
> > > ONLY thing that git tracks. ? ? You add CONTENT. ? ? When you do 
> > > "git add file.c" you aren't adding a filename to the list of ? files 
> > > that git knows about. Not even CLOSE. No. You are really adding ? 
> > > _content_ to the project you are tracking.
> > 
> > Read this again, please.  Ponder it if you may.
> > 
> 
> Yes. I am adding content. And not a file. But at least to me, it makes a
> *BIG* difference if I'm adding totally new content (reserving one more
> bucket where to place to content) or just replacing the content *in* one
> of those already reserved buckets.

Bzzzzt! Nope. "Reserved buckets" as you use it is nothing else than a 
file.

> And that has nothing to do with files (or at least the silly me can't 
> grok it).

Content: a byte stream with a label (so you can find it again). Of 
_course_ you don't want the byte stream vanish in a big black hole, so you 
_have_ to name it.

But git-add actually does two things: it adds a (completely new) object, 
which just holds the byte stream, being named by its content (the hash). 

But when committing, the _existing_ tree object is "updated", by writing a 
_new_ tree object. So, it is not an "updating" in the sense of "editing", 
rather "updating" as in copy-on-write.

So no, there are no "reserved buckets". You are very much _adding_ 
new information.

Another way to look at it: in git, you never "take away" anything. You 
only add things. Even if you remove a file from your working tree, and 
want to commit the change, it means that you _add_ information: The 
information that this file is no longer in the current revision. But the 
commit references the old revision (indeed, the _whole_ ancestry!), in 
which the file _was_ present, so you literally _added_ something _on top_ 
of the old revision.

Hth,
Dscho

Re: Importing from tarballs; add, rm, update-index?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 3730 bytes --]

On Fri, 12 Jan 2007 16:48:09 -0800, Junio C Hamano wrote:
> Peter Baumann <waste.manager@gmx.de> writes:
>
> > Yes. I fully second Linus opinion. But I think there should be
> > a difference in adding completly new content to the index
> > (number of entries in the index grows) or replacing content in
> > the index.
>
> Huh?

Here's an easy way to see the difference that Peter is trying to point
out, (and it really has nothing to do with whether "git add" for a new
file should add the content of that file to the index---that's a
totally separate issue that Linus was talking about in that other
message).

Just look at "commit -a" and how its documented right now. Currently
it's documented as doing an automatic "add" to all known files. That
descriptions is unsatisfactory for two reasons:

1. "commit -a" will also commit the removal of files---which requires
   an index modification that "git add" cannot do

2. "add" can cause an entirely new path (with content, Vader!) to be
   added to the index. So the user has to carefully separate out this
   behavior of "add" to properly understand what "commit -a" is
   doing. The documentation tries to help here with "known files", but
   the talk of an "automatic 'add'" that never adds any new paths
   really goes against the primary functionality of "git add".

   I say "primary functionality" because the 'commit -a' workflow,
   (which we've all agreed should be the thing that is taught first),
   requires users to use 'git add' when adding a new path to the
   index, but never requires the user to use the 'update the index'
   sense of 'git add', (instead, the user just needs to _learn_ this
   sense to understand commit -a).

So there's lots of room for potential confusion there, and we've got
evidence of that confusion in the messages that started this an other
recent threads about how to remove files.

I like the idea of adding a porcelain command for update-index, and
it's nice to try to describe "commit -a" in terms of the new porcelain
command. But, to make that really work, I think that porcelain for
update-index should really match the semantics needed by "commit
-a". That is, it should never add new paths to the index, but it
should update content for existing paths, and it should remove paths
from the index when files have been removed from the working tree.

Let's call this new command "refresh", just to experiment with another
name. If it existed, then "commit -a" could be described as simply
doing "refresh" on all files, (with no need to have a notion of
"tracked files", nor any extra language about file removal). That is,
"commit -a" could be understood as something like:

	git refresh -a
	git commit

(or maybe "git refresh .; git commit" if one prefers that, but I think
it'd be nice to carry the -a option over to the new porcelain).

Also, this would even make it possible to provide an accurate
index-based description of "commit paths...". Namely, something like:

	commit paths...

	This command starts with a new index initialized from the
	contents of the current commit (HEAD). It then performs the
	following commands:

		git refresh paths...
		git commit

	[Some extra language needed here about restoring into the
	index other changes that were "skipped over".]

So, someone might like to have that kind of description somewhere in
the technical documentation of git. (I'd still prefer to see "commit
paths..." documented as simply "commits the working-tree content of
all specified paths").

Anyway, did I succeed in pointing out why some of us think that the
"add a new path (with content) to the index" and the "update content
for existing path" really shouldn't be mixed up in the same "add"
command?

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On Sat, Jan 13, 2007 at 12:17:18PM +0100, Johannes Schindelin wrote:
> Hi,
> 
> On Sat, 13 Jan 2007, Peter Baumann wrote:
> 
> > On Fri, Jan 12, 2007 at 04:48:09PM -0800, Junio C Hamano wrote:
> > > Peter Baumann <waste.manager@gmx.de> writes:
> > > 
> > > > ? What does it mean to "add" something to a project? It has 
> > > > _nothing_ to do ? with "filenames". Yeah, the filename obviously 
> > > > exists, but it's not ? something that exists on its own. You add the 
> > > > ONLY thing that git tracks. ? ? You add CONTENT. ? ? When you do 
> > > > "git add file.c" you aren't adding a filename to the list of ? files 
> > > > that git knows about. Not even CLOSE. No. You are really adding ? 
> > > > _content_ to the project you are tracking.
> > > 
> > > Read this again, please.  Ponder it if you may.
> > > 
> > 
> > Yes. I am adding content. And not a file. But at least to me, it makes a
> > *BIG* difference if I'm adding totally new content (reserving one more
> > bucket where to place to content) or just replacing the content *in* one
> > of those already reserved buckets.
> 
> Bzzzzt! Nope. "Reserved buckets" as you use it is nothing else than a 
> file.
> 
> > And that has nothing to do with files (or at least the silly me can't 
> > grok it).
> 
> Content: a byte stream with a label (so you can find it again). Of 
> _course_ you don't want the byte stream vanish in a big black hole, so you 
> _have_ to name it.
> 
> But git-add actually does two things: it adds a (completely new) object, 
> which just holds the byte stream, being named by its content (the hash). 
> 
OK. Now we are talking clear. I don't like git-add to do two things (see
below)

> But when committing, the _existing_ tree object is "updated", by writing a 
> _new_ tree object. So, it is not an "updating" in the sense of "editing", 
> rather "updating" as in copy-on-write.
> 
> So no, there are no "reserved buckets". You are very much _adding_ 
> new information.
> 

I unterstand the concept of the index, I even glanced at the code. The
index holds for every file... err..  content a struct cache_entry (my
above mentioned "buckets"). If I add a new cache_entry, I enlarge my
later committed tree by at least one new file (adding a new file could
also add an entry for a directory in the final tree object).

I would much more like the idea of having one command (git-add) which
would enlarge the index by a new struct cache_entry and another which
replaces a previously added cache_entry. Because I'd like to control
how my final tree looks like and add least to me adding a file entry to
a tree is something different than updating a file entry (at least
mentally).

I'd favour the following model:

 git-add: register the content of a previously unkown file to git
    (there was now struct chache_entry in the index previously which
     described a file with the same name)

 git-rm: remove a struct cache_entry from the index and after some
    safty checks remove the file, too. (see in the mailinglist archive;
    there was much talk about git-rm and its semantic)

 git-refresh (or git-stage or git-update or what-ever you call it):
    replace the cache_entry by a new one

And this all about content; the content which would represent my next
tree object. Because developers don't think of "add" if they want to
remove a file from the commit. If the power users liked to have only one
command, wich does remove, add and update then lets not call it add.
Better make this commmand the above mentioned git-refresh which would do
"the right thing" if called with a new file/removed file

Im simply think its confusing to call the described command git-add, as
we have it now. It's at least *very* confusing for new starters.

-Peter

> Another way to look at it: in git, you never "take away" anything. You 
> only add things. Even if you remove a file from your working tree, and 
> want to commit the change, it means that you _add_ information: The 
> information that this file is no longer in the current revision. But the 
> commit references the old revision (indeed, the _whole_ ancestry!), in 
> which the file _was_ present, so you literally _added_ something _on top_ 
> of the old revision.
> 
> Hth,
> Dscho
> 

Re: Importing from tarballs; add, rm, update-index?

[Julian Phillips]

On Sat, 13 Jan 2007, Peter Baumann wrote:

> I'd favour the following model:
>
> git-add: register the content of a previously unkown file to git
>    (there was now struct chache_entry in the index previously which
>     described a file with the same name)
>
> git-rm: remove a struct cache_entry from the index and after some
>    safty checks remove the file, too. (see in the mailinglist archive;
>    there was much talk about git-rm and its semantic)
>
> git-refresh (or git-stage or git-update or what-ever you call it):
>    replace the cache_entry by a new one
>
> And this all about content; the content which would represent my next
> tree object. Because developers don't think of "add" if they want to
> remove a file from the commit. If the power users liked to have only one
> command, wich does remove, add and update then lets not call it add.
> Better make this commmand the above mentioned git-refresh which would do
> "the right thing" if called with a new file/removed file
>
> Im simply think its confusing to call the described command git-add, as
> we have it now. It's at least *very* confusing for new starters.

Personally I actually find having a single add command to be the simplest 
conceptual model ...

I think of it like this:

* start off with current content (index matches HEAD matches working tree)
* I do some stuff (add files, edit files, delete files)
* I add my changes to the index
* I do more stuff
* I add my changes to the index
* I do more stuff
* I realise that the latest changes are actually different, so I commit 
the index, and then keep going, or I add the changes and commit.

The only thing I find slightly confusing is that the staging area for the 
next commit is called the index.

(But then maybe I've been reading this list too long - though I have only 
actually starting playing with git recently)

-- 
Julian

  ---
Power, like a desolating pestilence,
Pollutes whate'er it touches...
 		-- Percy Bysshe Shelley

Re: Importing from tarballs; add, rm, update-index?

[Brian Gernhardt]

On Jan 13, 2007, at 4:36 AM, Peter Baumann wrote:

> On 2007-01-13, Brian Gernhardt <benji@silverinsanity.com> wrote:
>>
>> On Jan 12, 2007, at 2:43 PM, Junio C Hamano wrote:
>>
>>> Peter Baumann <Peter.B.Baumann@stud.informatik.uni-erlangen.de>
>>> writes:
>>>
>>>> Me doesn't really like the new semantics of "git-add", because it
>>>> does
>>>> two seperate things - it adds new files and it refreshes the
>>>> content of
>>>> previously known files.
>>>
>>> http://thread.gmane.org/gmane.comp.version-control.git/32452/
>>> focus=32792
>>
>> Should this be added to Documentation/rants/filename-
>> braindamage.txt?  ;-)
>>
>> ~~ Brian
>
> Ok. Obviously I should't have said that it add "files" (gr, silly me).
> What I meant was it adds the content of files. But there is a  
> difference
> in adding and replacing content.

I was referring to adding Linus' rant...  And maybe several others.   
I tend to find his rants at least slightly amusing, highly  
informative, and I tend to end up agreeing.  I have very little  
opinion on your complaint so long as the system works consistently.   
"git commit -a" is still my most common workflow.  I've used git-add  
(and prior to that git-update-index) from time to time when I fix  
bugs that need to be separate from my current work, but far far more  
common is "I finished this chunk of functionality, add all the  
changes I did to make it happen".

~~ Brian

Re: Importing from tarballs; add, rm, update-index?

[Brian Gernhardt]

On Jan 13, 2007, at 11:09 AM, Carl Worth wrote:

> Also, this would even make it possible to provide an accurate
> index-based description of "commit paths...". Namely, something like:
>
> 	commit paths...
>
> 	This command starts with a new index initialized from the
> 	contents of the current commit (HEAD). It then performs the
> 	following commands:
>
> 		git refresh paths...
> 		git commit
>
> 	[Some extra language needed here about restoring into the
> 	index other changes that were "skipped over".]
>
> So, someone might like to have that kind of description somewhere in
> the technical documentation of git. (I'd still prefer to see "commit
> paths..." documented as simply "commits the working-tree content of
> all specified paths").

I fail to see why this description can't be used with s/refresh/ 
add/.  I also don't think it's a very clear description because of  
the "starting with a new index" and the hand-waving involved in  
"restoring into the index other changes".

I can somewhat understand the desire to split git-add (although I  
don't share it).  But I don't see the need for it to be a new git- 
refresh, since that functionality already exists as git-update- 
index.  Is using git-add to add to the index a conceptual problem or  
is it causing actual problems in people's usage of git?  If it's an  
issue of teaching new users, I _think_ that could be resolved very  
simply as "git add adds content to the index" when we explain the  
index as the staging area for a new commit and wean people off of  
"git commit -a".  A short discussion of "tracking content vs. files"  
is probably also a good idea.  (I honestly haven't read the tutorials  
in a long long time, so this may already be in there.)

~~ Brian

Re: Importing from tarballs; add, rm, update-index?

[Peter Baumann]

On Sat, Jan 13, 2007 at 08:09:35AM -0800, Carl Worth wrote:
> On Fri, 12 Jan 2007 16:48:09 -0800, Junio C Hamano wrote:
> > Peter Baumann <waste.manager@gmx.de> writes:
> >
> > > Yes. I fully second Linus opinion. But I think there should be
> > > a difference in adding completly new content to the index
> > > (number of entries in the index grows) or replacing content in
> > > the index.
> >
> > Huh?
> 
> Here's an easy way to see the difference that Peter is trying to point
> out, (and it really has nothing to do with whether "git add" for a new
> file should add the content of that file to the index---that's a
> totally separate issue that Linus was talking about in that other
> message).
> 
> Just look at "commit -a" and how its documented right now. Currently
> it's documented as doing an automatic "add" to all known files. That
> descriptions is unsatisfactory for two reasons:
> 
> 1. "commit -a" will also commit the removal of files---which requires
>    an index modification that "git add" cannot do
> 
> 2. "add" can cause an entirely new path (with content, Vader!) to be
>    added to the index. So the user has to carefully separate out this
>    behavior of "add" to properly understand what "commit -a" is
>    doing. The documentation tries to help here with "known files", but
>    the talk of an "automatic 'add'" that never adds any new paths
>    really goes against the primary functionality of "git add".
> 
>    I say "primary functionality" because the 'commit -a' workflow,
>    (which we've all agreed should be the thing that is taught first),
>    requires users to use 'git add' when adding a new path to the
>    index, but never requires the user to use the 'update the index'
>    sense of 'git add', (instead, the user just needs to _learn_ this
>    sense to understand commit -a).
> 
> So there's lots of room for potential confusion there, and we've got
> evidence of that confusion in the messages that started this an other
> recent threads about how to remove files.
> 
> I like the idea of adding a porcelain command for update-index, and
> it's nice to try to describe "commit -a" in terms of the new porcelain
> command. But, to make that really work, I think that porcelain for
> update-index should really match the semantics needed by "commit
> -a". That is, it should never add new paths to the index, but it
> should update content for existing paths, and it should remove paths
> >from the index when files have been removed from the working tree.
> 
> Let's call this new command "refresh", just to experiment with another
> name. If it existed, then "commit -a" could be described as simply
> doing "refresh" on all files, (with no need to have a notion of
> "tracked files", nor any extra language about file removal). That is,
> "commit -a" could be understood as something like:
> 
> 	git refresh -a
> 	git commit
> 
> (or maybe "git refresh .; git commit" if one prefers that, but I think
> it'd be nice to carry the -a option over to the new porcelain).
> 
> Also, this would even make it possible to provide an accurate
> index-based description of "commit paths...". Namely, something like:
> 
> 	commit paths...
> 
> 	This command starts with a new index initialized from the
> 	contents of the current commit (HEAD). It then performs the
> 	following commands:
> 
> 		git refresh paths...
> 		git commit
> 
> 	[Some extra language needed here about restoring into the
> 	index other changes that were "skipped over".]
> 
> So, someone might like to have that kind of description somewhere in
> the technical documentation of git. (I'd still prefer to see "commit
> paths..." documented as simply "commits the working-tree content of
> all specified paths").
> 
> Anyway, did I succeed in pointing out why some of us think that the
> "add a new path (with content) to the index" and the "update content
> for existing path" really shouldn't be mixed up in the same "add"
> command?
> 

Yes. At least for me :-)

-Peter

> -Carl

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

Peter Baumann <waste.manager@gmx.de> writes:

> Yes. I am adding content. And not a file. But at least to me, it makes a
> *BIG* difference if I'm adding totally new content (reserving one more
> bucket where to place to content) or just replacing the content *in* one
> of those already reserved buckets. And that has nothing to do with
> files (or at least the silly me can't grok it).

To put this silly naming argument to the rest for now, because I
am not going to change add/rm nor introduce refresh, at least
during this round, so keeping this thread alive would just waste
everybody's time doing mental masturbation.

Physically the index is represented as a list of <blob object
name, pathname> tuples.  When we say "git tracks contents",
however, we look at it as if content of each blob object is just
a bytestream labeled with the pathname.

When you say "git-update-index file" (without --add/--remove),
what happens is "UPDATE" (in SQL sense).  The part of the data
recorded in the current index that is labeled with "file" is
replaced with what is from the working tree.

The --add option changes the "UPDATE" to "INSERT OR REPLACE".
It allows contents that are labelled with a pathname that does
not yet exist in the index.  What --remove does is to allow it
to also "DELETE".

So there _is_ a distinction between adding new pathname and
updating the contents at the low level.

However, if you look at the way 'git-update-index' is used in
the Porcelain-ish scripts (now you would need to go back and
examine a bit older versions of git, since many commands have
been rewritten in C to become built-in and we use update-index
in much fewer places in today's version), we almost always used
update-index with --add when talking about the set of paths the
end user talks about ('am' and 'applypatch' uses --index-info;
this is also "INSERT OR REPLACE" operation primarily).  The
places we did not, we knew we were only dealing with known set
of paths taken from the current index, so they also could have
had --add without any ill effects.  

In other words, there was not much need for "UPDATE only, please
do not INSERT" in practice.

That's primarily why the higher level interface git-add / git-mv
does not expose that distinction; git-add will do "INSERT OR
REPLACE".  git-rm will do "DELETE", and there will be no higher
level to only do "UPDATE".

Re: Importing from tarballs; add, rm, update-index?

[Alan Chandler]

> I was referring to adding Linus' rant...  And maybe several others.
> I tend to find his rants at least slightly amusing, highly
> informative, and I tend to end up agreeing.  I have very little
> opinion on your complaint so long as the system works consistently.
> "git commit -a" is still my most common workflow.  I've used git-add
> (and prior to that git-update-index) from time to time when I fix
> bugs that need to be separate from my current work, but far far more
> common is "I finished this chunk of functionality, add all the
> changes I did to make it happen".

I think the fact that this thread has come alive again implies we didn't 
bottom it last time through.

One thing, in particular, has been bugging me with the hide the index 
concept - that is, its still necessary to git add files for them to be 
picked up with git commit -a

My (albeit limited) experience with using git is at home coding a java 
application for my web site using eclipse.  During the application 
development when I am initially coding the application, or when I am 
doing a major update that adds new pages to my site then I have to 
remember to git add files.  My immediate instinct is do do commands of 
the form

git add 
Javasource/uk/org/chandlerfamily/appname/tapestry/pages/subdir/xxx.java

and
git add Webcontent/subdir/xxx.html

which even with bash completion is a pain to enter.

(although that is probably harder than it needs to be - can't I just do 
git add . ?)

I don't know whether we have had the debate here - if we have done it 
would have been in the very very early days, but subject to 
the .gitignore rules what would be the implications of a git commit -a 
that automatically adds any files within the directory (and 
subdirectories) in which it is issued.

Then I think you don't even have to get into what is git add all about 
until you get to the "use the index" stage.

I am (at the moment - but I am good at changing my mind) in the side of 
giit add for both adding new paths and updating content.  This is 
purely  pragmatic - don't have to remember which one I am trying to do.

-- 
Alan Chandler
http://www.chandlerfamily.org.uk

Re: Importing from tarballs; add, rm, update-index?

[Nicolas Pitre]

On Sat, 13 Jan 2007, Carl Worth wrote:

> Anyway, did I succeed in pointing out why some of us think that the
> "add a new path (with content) to the index" and the "update content
> for existing path" really shouldn't be mixed up in the same "add"
> command?

Not really.

The fact is that there is no strong reason why they shouldn't.  But 
there are good reasons why they should.  The most important one being 
that the user doesn't need to bother deciding which one of the two 
commands should be used in any given situation.  And because a single 
command can cover two _technically_ different cases transparently is a 
pretty good reason for not imposing this technical issue to the user.

But remember that git-update-index is still and will always be available 
to you for fancier and more fine grained control if/when you have a 
special need for it.  This is not the case for the vast majority of 
users though and the primary user interface should reflect that.


Nicolas

Re: Importing from tarballs; add, rm, update-index?

[Brian Gernhardt]

On Jan 13, 2007, at 1:15 PM, Alan Chandler wrote:

> (although that is probably harder than it needs to be - can't I  
> just do
> git add . ?)

Yes.  It sounds very much like you want to simply do "git add . ; git  
commit -a".  But making that the default for "commit -a" would be  
obnoxious for many other people.

I know that fairly often I begin adding a chunk of new code and  
realize the changes I made to the existing code should logically be a  
different commit.  Having "git commit -a" ignore the new files (and  
any new object/log/debug/etc files I haven't added to .gitignore)  
makes things so much simpler.

A more through version ("git commit --everything"?) that also adds  
files would be fine, but don't muck up the existing -a, please.

~~ Brian

Re: Importing from tarballs; add, rm, update-index?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

On Sat, 13 Jan 2007 13:54:20 -0500 (EST), Nicolas Pitre wrote:
> The fact is that there is no strong reason why they shouldn't.  But
> there are good reasons why they should.  The most important one being
> that the user doesn't need to bother deciding which one of the two
> commands should be used in any given situation.  And because a single
> command can cover two _technically_ different cases transparently is a
> pretty good reason for not imposing this technical issue to the user.

But that same reasoning could be extended to say there shouldn't be
separate "add" and "rm", because a single command can transparently
cover these two technically different cases transparently, (that would
be update-index without --add and --remove safety checks). But nobody
has been proposing that that would be a good direction to go.

So there are at least three cases one could identify for updating
content into the index:

1. Adding content for a path that didn't previously exist in the index

2. Updating content for a path that does already exist in the index

3. Removing a path and its content from the index

As things stand currently, git's providing a first-class operation
("git add") that provides (1) and (2) and another operation ("git rm")
for (3).

However, "commit -a" is implicitly performing operations from (2) and
(3).

So the documentation of "commit -a" being implemented with "add" just
plain doesn't make sense---and this is causing confusion.

I'd love to see _something_ get accepted to resolve that
confusion. What I was proposing was a command that did (1) and another
that did (2) or (3), (and "commit -a" could then be documented as
using this command).

But there are probably other ways to fix the problem.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

Carl Worth <cworth@cworth.org> writes:

> So the documentation of "commit -a" being implemented with "add" just
> plain doesn't make sense---and this is causing confusion.

diff --git a/Documentation/git-commit.txt b/Documentation/git-commit.txt
index cb081cd..b4528d7 100644
--- a/Documentation/git-commit.txt
+++ b/Documentation/git-commit.txt
@@ -32,7 +32,8 @@ methods:
 
 4. by using the -a switch with the 'commit' command to automatically "add"
    changes from all known files i.e. files that have already been committed
-   before, and perform the actual commit.
+   before, and to automatically "rm" files that have been
+   removed from the working tree, and perform the actual commit.
 
 The gitlink:git-status[1] command can be used to obtain a
 summary of what is included by any of the above for the next

Re: Importing from tarballs; add, rm, update-index?

[Nicolas Pitre]

On Sat, 13 Jan 2007, Carl Worth wrote:

> On Sat, 13 Jan 2007 13:54:20 -0500 (EST), Nicolas Pitre wrote:
> > The fact is that there is no strong reason why they shouldn't.  But
> > there are good reasons why they should.  The most important one being
> > that the user doesn't need to bother deciding which one of the two
> > commands should be used in any given situation.  And because a single
> > command can cover two _technically_ different cases transparently is a
> > pretty good reason for not imposing this technical issue to the user.
> 
> But that same reasoning could be extended to say there shouldn't be
> separate "add" and "rm", because a single command can transparently
> cover these two technically different cases transparently, (that would
> be update-index without --add and --remove safety checks). But nobody
> has been proposing that that would be a good direction to go.

If nobody has been proposing that then it must not be a good direction 
to go indeed.  This is however not the case for 'add' handling both new 
and existing files which I believe most people like.

> So there are at least three cases one could identify for updating
> content into the index:
> 
> 1. Adding content for a path that didn't previously exist in the index
> 
> 2. Updating content for a path that does already exist in the index
> 
> 3. Removing a path and its content from the index
> 
> As things stand currently, git's providing a first-class operation
> ("git add") that provides (1) and (2) and another operation ("git rm")
> for (3).
> 
> However, "commit -a" is implicitly performing operations from (2) and
> (3).
> 
> So the documentation of "commit -a" being implemented with "add" just
> plain doesn't make sense---and this is causing confusion.

OK if that is what your grip is about let's fix the documentation then.

> I'd love to see _something_ get accepted to resolve that
> confusion. What I was proposing was a command that did (1) and another
> that did (2) or (3), (and "commit -a" could then be documented as
> using this command).

No. The command set is sane.  Many people like it, and it does the work 
fine already, better than it used to.

You don't fix bad documentation by suiting the command set to it.  
You fix the bad documentation instead.

So what about this:

diff --git a/Documentation/git-commit.txt b/Documentation/git-commit.txt
index cb081cd..96917d4 100644
--- a/Documentation/git-commit.txt
+++ b/Documentation/git-commit.txt
@@ -32,7 +32,7 @@ methods:
 
 4. by using the -a switch with the 'commit' command to automatically "add"
    changes from all known files i.e. files that have already been committed
-   before, and perform the actual commit.
+   before, and/or "rm" missing known files, then perform the actual commit.
 
 The gitlink:git-status[1] command can be used to obtain a
 summary of what is included by any of the above for the next

Nicolas

Re: Importing from tarballs; add, rm, update-index?

[Shawn O. Pearce]

Brian Gernhardt <benji@silverinsanity.com> wrote:
> Yes.  It sounds very much like you want to simply do "git add . ; git  
> commit -a".  But making that the default for "commit -a" would be  
> obnoxious for many other people.

I find it annoying that "commit -a" isn't implemented in terms of
"git add .".  Mainly because I'll make a number of changes in Eclipse
then go back and do "commit -a" and only days later discover that
I have untracked files in my working directory which should have
been added to the commit several days ago.

Although despite the fact that I always have my .gitignore setup 
properly, every once in a while I'll change something to produce
a new file that Git should really ignore, and I'll forget to put
it into .gitignore.  Having some sort of "commit -a" which adds
that new file would be an issue.
 
> A more through version ("git commit --everything"?) that also adds  
> files would be fine, but don't muck up the existing -a, please.

Yes, breaking -a may be a problem.

-- 
Shawn.

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Alan Chandler <alan@chandlerfamily.org.uk> wrote:

[...]

> My (albeit limited) experience with using git is at home coding a java 
> application for my web site using eclipse.  During the application 
> development when I am initially coding the application, or when I am 
> doing a major update that adds new pages to my site then I have to 
> remember to git add files.  My immediate instinct is do do commands of 
> the form
> 
> git add 
> Javasource/uk/org/chandlerfamily/appname/tapestry/pages/subdir/xxx.java
> 
> and
> git add Webcontent/subdir/xxx.html
> 
> which even with bash completion is a pain to enter.
> 
> (although that is probably harder than it needs to be - can't I just do 
> git add . ?)

That's different...

> I don't know whether we have had the debate here - if we have done it 
> would have been in the very very early days, but subject to 
> the .gitignore rules what would be the implications of a git commit -a 
> that automatically adds any files within the directory (and 
> subdirectories) in which it is issued.

Please don't. It would add e.g. the .class and .o and all ~ files, and all
other junk you have lying around (test cases, test run output, ...). It
isn't /that/ much more work after creating a new file to record its
existence...

> I am (at the moment - but I am good at changing my mind) in the side of 
> giit add for both adding new paths and updating content.  This is 
> purely  pragmatic - don't have to remember which one I am trying to do.

Me too. It's just a bit weird to use it to register the fact that a file is
gone.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Shawn O. Pearce]

"Horst H. von Brand" <vonbrand@inf.utfsm.cl> wrote:
> Please don't. It would add e.g. the .class and .o and all ~ files, and all
> other junk you have lying around (test cases, test run output, ...). It
> isn't /that/ much more work after creating a new file to record its
> existence...

  echo \*.class >>.gitignore
  echo \*.o >>.gitignore
  git add .gitignore

doesn't work for some reason?
 
-- 
Shawn.

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Shawn O. Pearce <spearce@spearce.org> wrote:
> "Horst H. von Brand" <vonbrand@inf.utfsm.cl> wrote:
> > Please don't. It would add e.g. the .class and .o and all ~ files, and all
> > other junk you have lying around (test cases, test run output, ...). It
> > isn't /that/ much more work after creating a new file to record its
> > existence...
> 
>   echo \*.class >>.gitignore
>   echo \*.o >>.gitignore
>   git add .gitignore
> 
> doesn't work for some reason?

Because each time I build a bit of scaffolding, or have a new test output,
or... I'd have to add them to .gitignore. I'd very much prefer tracking
real contents and not junk.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Johannes Schindelin]

Hi,

On Sat, 13 Jan 2007, Shawn O. Pearce wrote:

> Brian Gernhardt <benji@silverinsanity.com> wrote:
> > Yes.  It sounds very much like you want to simply do "git add . ; git  
> > commit -a".  But making that the default for "commit -a" would be  
> > obnoxious for many other people.
> 
> I find it annoying that "commit -a" isn't implemented in terms of
> "git add .".  Mainly because I'll make a number of changes in Eclipse
> then go back and do "commit -a" and only days later discover that
> I have untracked files in my working directory which should have
> been added to the commit several days ago.

You mean, you _ignored_ the text "git commit -a" gives you? It really 
shows you the output of "git status", exactly so you know what you 
committed, and sometimes more importantly, what you didn't.

I mean, "git commit" spends a lot of time getting that information, so you 
better use it.

Ciao,
Dscho

Re: Importing from tarballs; add, rm, update-index?

[Shawn O. Pearce]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> You mean, you _ignored_ the text "git commit -a" gives you? It really 
> shows you the output of "git status", exactly so you know what you 
> committed, and sometimes more importantly, what you didn't.

Because I'm a moron and forgot what files I had created recently.
Consequently I don't see them missing from the output of git commit.
Consequently I think the commit is OK.  :-)

-- 
Shawn.

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Shawn O. Pearce <spearce@spearce.org> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > You mean, you _ignored_ the text "git commit -a" gives you? It really 
> > shows you the output of "git status", exactly so you know what you 
> > committed, and sometimes more importantly, what you didn't.

> Because I'm a moron and forgot what files I had created recently.
> Consequently I don't see them missing from the output of git commit.
> Consequently I think the commit is OK.  :-)

That can't be fixed by software... better get the habit of "git add"ing new
files ASAP.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>> You mean, you _ignored_ the text "git commit -a" gives you? It really 
>> shows you the output of "git status", exactly so you know what you 
>> committed, and sometimes more importantly, what you didn't.
>
> Because I'm a moron and forgot what files I had created recently.
> Consequently I don't see them missing from the output of git commit.
> Consequently I think the commit is OK.  :-)

I think Johannes is refering you to the "Untracked files"
section.

Re: Importing from tarballs; add, rm, update-index?

[Shawn O. Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> 
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >> You mean, you _ignored_ the text "git commit -a" gives you? It really 
> >> shows you the output of "git status", exactly so you know what you 
> >> committed, and sometimes more importantly, what you didn't.
> >
> > Because I'm a moron and forgot what files I had created recently.
> > Consequently I don't see them missing from the output of git commit.
> > Consequently I think the commit is OK.  :-)
> 
> I think Johannes is refering you to the "Untracked files"
> section.

I commit often directly from the command line.

But yes, you're right.  If I actually started my editor its right
there in the commit message template.  Easy enough to quit without
writing the file, add the new files, and restart the commit.

-- 
Shawn.

Re: Importing from tarballs; add, rm, update-index?

[Daniel Barkalow]

On Sun, 14 Jan 2007, Shawn O. Pearce wrote:

> I commit often directly from the command line.
> 
> But yes, you're right.  If I actually started my editor its right
> there in the commit message template.  Easy enough to quit without
> writing the file, add the new files, and restart the commit.

An config option to prohibit committing with untracked files should be 
easy to add. If your workflow is such that incorrect commits are sometimes 
generated given either policy, the system should ask you which you mean.

	-Daniel
*This .sig left intentionally blank*

Re: Importing from tarballs; add, rm, update-index?

[Horst H. von Brand]

Daniel Barkalow <barkalow@iabervon.org> wrote:
> An config option to prohibit committing with untracked files should be 
> easy to add.

Right. And that will annoy the heck out of people who have random litter
left behind, so their fingers will go "git clean; git commit -a" and then
"OOoops!!!". If they can't read the commit message template in the first
place, or train their fingers to "git add" new files immediately...

>              If your workflow is such that incorrect commits are sometimes 
> generated given either policy, the system should ask you which you mean.

Leave it alone. At least it will work the same always. Consistency is good.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                    Fono: +56 32 2654431
Universidad Tecnica Federico Santa Maria             +56 32 2654239
Casilla 110-V, Valparaiso, Chile               Fax:  +56 32 2797513

Re: Importing from tarballs; add, rm, update-index?

[Jakub Narebski]

Daniel Barkalow wrote:

> On Sun, 14 Jan 2007, Shawn O. Pearce wrote:
> 
>> I commit often directly from the command line.
>> 
>> But yes, you're right.  If I actually started my editor its right
>> there in the commit message template.  Easy enough to quit without
>> writing the file, add the new files, and restart the commit.
> 
> An config option to prohibit committing with untracked files should be 
> easy to add. If your workflow is such that incorrect commits are sometimes 
> generated given either policy, the system should ask you which you mean.

Not a config option. Pre-commit hook should be enough.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Importing from tarballs; add, rm, update-index?

[Daniel Barkalow]

On Mon, 15 Jan 2007, Horst H. von Brand wrote:

> Daniel Barkalow <barkalow@iabervon.org> wrote:
> > An config option to prohibit committing with untracked files should be 
> > easy to add.
> 
> Right. And that will annoy the heck out of people who have random litter
> left behind, so their fingers will go "git clean; git commit -a" and then
> "OOoops!!!". If they can't read the commit message template in the first
> place, or train their fingers to "git add" new files immediately...

Or they can avoid enabling the config option if it's not actually helpful 
to them. I don't think it should be the default behavior, but I think it 
should be available to people who tend to make the mistake Shawn 
described. For that matter, it would be nice to have an option for 
filename patterns that shouldn't be left untracked. I end up with plenty 
of junk, but none of it is *.c or *.h, unless the file is one I put in 
.gitignore.

Actually, a "git ignore" command that adds things to .gitignore (like 'for 
i in "$*"; do echo $i >> .gitignore; done; git-update-index --add 
.gitignore') would probably also be helpful. All of the files in my 
directories are one of (1) Things everybody wants to ignore, because 
they're build system output or common backup file patterns, (2) Things 
that should be tracked, because they're source, and (3) Things that 
shouldn't be tracked in the project, but which I want to hang on to, like 
interesting debugging output.

	-Daniel
*This .sig left intentionally blank*

Re: Importing from tarballs; add, rm, update-index?

[Christian MICHON]

This is an interesting idea... so I went through the <<silly>> exercise of
downloading 117 tar bzipped versions of the kernel 2.6 (from 2.6.0 to
2.6.19.2, without the release candidates). Quite an interesting
benchmark.

Then I started importing them, through a similar script, creating a
branch for each version (for easy checkout). It takes around
1.5 hours to complete.

Results:
======

initial point: 4.3Gb of tar.bz2 files

intermediate: ~800Mb with ~120000 git objects

final point: 100Mb with prune-packed objects
(so it's about 2.5x the average tar.bz2 size of just a single kernel)

--
Christian