files are disappearing in git

files are disappearing in git

[Nico -telmich- Schottelius]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

Hello!

I've the problem that some files (a directory with 3 files) is simply 'away':

- We added it once
- In current tree it's away
- pasky aided me in irc to find the commit where it is gone with git bisect
--> very nice tool
- the commit, after which the directory was gone did NOT modify this directory
- though the directory is gone

What should I do know to find out what's the reason git 'forgot' that directory?

Nico

P.S.: I cannot put the .git directory online, it's a closed source project
      for a special customer with customer data in it.

-- 
Latest project: cinit-0.2.1 (http://linux.schottelius.org/cinit/)
Open Source nutures open minds and free, creative developers.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: files are disappearing in git

[Linus Torvalds]

On Wed, 23 Nov 2005, Nico -telmich- Schottelius wrote:
> 
> I've the problem that some files (a directory with 3 files) is simply 'away':
> 
> - We added it once
> - In current tree it's away
> - pasky aided me in irc to find the commit where it is gone with git bisect
> --> very nice tool
> - the commit, after which the directory was gone did NOT modify this directory
> - though the directory is gone
> 
> What should I do know to find out what's the reason git 'forgot' that directory?

I bet somebody just messed up the index before that commit. So the commit 
probably _did_ modify the directory, though some incorrect patching or 
some scripting bug.

If you can guess at all what went on at around that commit (what else 
happened there?) that might help. Was it a merge? If those three files 
existed in the base commit of the merge but not in one of the branches, 
then the merge would have assumed that the branch just deleted it.

Is the tree public so that we can look at it and perhaps make a guess from 
what happened around it?

		Linus

Re: files are disappearing in git

[Nico -telmich- Schottelius]

[-- Attachment #1: Type: text/plain, Size: 2720 bytes --]

Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:
> On Wed, 23 Nov 2005, Nico -telmich- Schottelius wrote:
> > 
> > I've the problem that some files (a directory with 3 files) is simply 'away':
> > 
> > - We added it once
> > - In current tree it's away
> > - pasky aided me in irc to find the commit where it is gone with git bisect
> > --> very nice tool
> > - the commit, after which the directory was gone did NOT modify this directory
> > - though the directory is gone
> > 
> > What should I do know to find out what's the reason git 'forgot' that directory?
> 
> I bet somebody just messed up the index before that commit.

What would be the best things to corrupt the index?
Our developers here do the following each day:

- cg-update
- <work>
- cg-commit
- cg-update + merge if there are changes
- cg-push origin
--> this goes to NFS mounted /home (in which developers also work)
[9:34] klapperwachstum:walderlift% cat .git/branches/origin 
/home/server/git/walderlift.git


> So the commit 
> probably _did_ modify the directory, though some incorrect patching or 
> some scripting bug.

Hrm, perhaps this is true, though I am not aware _how_ they (two people
besides me) could even manage to do that. Our developers are
Kylix developers under Linux, who are raelly limited to the above
cg-commands and using gitweb for seeing changes.

> If you can guess at all what went on at around that commit (what else 
> happened there?) that might help.

In this commit where specified three files changed. The commit text is not related
to the directory which is missing.

Files changed are

Code/Components/Utilities/LW1Calendar.pas
Code/lw1/Client/Terminierung/FeinTerminierungForm.pas
Code/lw1/Client/Terminierung/FeinTerminierungForm.xfm

and files missing after this release are

Code/Spikes/Statistik/

If I select "history" of one of the files at the point it still existed in the tree,
it has only one entry, where it was added.

> Was it a merge?

No, we also assumed that first, but git bisect showed that it is not.

> Is the tree public so that we can look at it and perhaps make a guess from 
> what happened around it?

I am sorry it is not. Perhaps I can convince my boss to allow access to it for
some git developers, so someone could debug it. Thought, every information
found in the tree would have to be treated confidental.

Do you have some other hints on how to debug that? Perhaps some conistency checking
tool?

Or perhaps should I put that git directory under cvs? ;-)

Nico

-- 
Latest project: cinit-0.2.1 (http://linux.schottelius.org/cinit/)
Open Source nutures open minds and free, creative developers.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: files are disappearing in git

[Junio C Hamano]

Nico -telmich- Schottelius <nico-linux-git@schottelius.org> writes:

> Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:
>> On Wed, 23 Nov 2005, Nico -telmich- Schottelius wrote:
>>...
>> I bet somebody just messed up the index before that commit.
>
> What would be the best things to corrupt the index?
> Our developers here do the following each day:
>
> - cg-update
> - <work>
> - cg-commit
> - cg-update + merge if there are changes
> - cg-push origin

Was any of the CG:F line changed/removed by the developer during
cg-commit?

Does any of the files under Code/Spikes/Statistik/ have funny
characters in their pathnames?

Re: files are disappearing in git

[Nico -telmich- Schottelius]

[-- Attachment #1: Type: text/plain, Size: 1229 bytes --]

Junio C Hamano [Thu, Nov 24, 2005 at 01:15:13AM -0800]:
> Nico -telmich- Schottelius <nico-linux-git@schottelius.org> writes:
> 
> > Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:
> >> On Wed, 23 Nov 2005, Nico -telmich- Schottelius wrote:
> >>...
> >> I bet somebody just messed up the index before that commit.
> >
> > What would be the best things to corrupt the index?
> > Our developers here do the following each day:
> >
> > - cg-update
> > - <work>
> > - cg-commit
> > - cg-update + merge if there are changes
> > - cg-push origin
> 
> Was any of the CG:F line changed/removed by the developer during
> cg-commit?

He said it could be possible. In fact the situation that we delete those lines
are not so seldem.

> Does any of the files under Code/Spikes/Statistik/ have funny
> characters in their pathnames?

No:

Code/Spikes/Statistik/Project.dpr   [new file with mode: 0644]    blob
Code/Spikes/Statistik/Statistik.pas    [new file with mode: 0644]    blob
Code/Spikes/Statistik/Statistik.xfm    [new file with mode: 0644]    blob

Nico

-- 
Latest project: cinit-0.2.1 (http://linux.schottelius.org/cinit/)
Open Source nutures open minds and free, creative developers.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: files are disappearing in git

[Ryan Anderson]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

Nico -telmich- Schottelius wrote:
> Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:
 >>Is the tree public so that we can look at it and perhaps make a guess
from
>>what happened around it?
> 
> 
> I am sorry it is not. Perhaps I can convince my boss to allow access to it for
> some git developers, so someone could debug it. Thought, every information
> found in the tree would have to be treated confidental.
> 
> Do you have some other hints on how to debug that? Perhaps some conistency checking
> tool?
> 
> Or perhaps should I put that git directory under cvs? ;-)

Is there anything in the directory structure that would be confidential?

Can you maybe provide the output of "git-whatchanged" (with no
parameters) or maybe

	git-whatchanged $commit1..$commit2 path/

Where $commit1 is a few commits before the problem, and $commit2 is a
few after it, and path/ is a path above the problem path?

You might add -m to that command line, too.

If the output of one of those commands, which will be commit objects +
tree differences, is less problematic, perhaps that would be easier to
share?




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: files are disappearing in git

[Nico -telmich- Schottelius]

[-- Attachment #1.1: Type: text/plain, Size: 2135 bytes --]

Ryan Anderson [Thu, Nov 24, 2005 at 08:54:34PM -0500]:
> Nico -telmich- Schottelius wrote:
> > Linus Torvalds [Wed, Nov 23, 2005 at 09:20:28AM -0800]:
>  >>Is the tree public so that we can look at it and perhaps make a guess
> from
> >>what happened around it?
> > 
> > 
> > I am sorry it is not. Perhaps I can convince my boss to allow access to it for
> > some git developers, so someone could debug it. Thought, every information
> > found in the tree would have to be treated confidental.
> > 
> > Do you have some other hints on how to debug that? Perhaps some conistency checking
> > tool?
> > 
> > Or perhaps should I put that git directory under cvs? ;-)
> 
> Is there anything in the directory structure that would be confidential?

The problem is that this is a project developed for a single company in
switzerland which uses it to manage elevators. We are right before to release
it and switch the customer away from MS Access to our program. It is currently
not clear, which license applies after the customer uses it, because he is interested
that some people should not use the software.

As soon as this is clearified the software may even become GPLed, but currently
I may not publish it public.

> Can you maybe provide the output of "git-whatchanged" (with no
> parameters) or maybe
> 
> 	git-whatchanged $commit1..$commit2 path/

If I do it as you described, I get the usage of git-rev-list:

[11:01] srsyg01:walderlift% git-whatchanged 4cc7ff0cb20df228e4db467114c1c67b0933592a..cfd54ffd2aa8e44ff5c341030e9c5f22fbbc7f9f Code/Spikes/        
usage: git-rev-list [OPTION] <commit-id>... [ -- paths... ]

I junh did git-whatchanged and stripped it manually. I included the initial
commit until the commit when it is gone. See attached file.

> If the output of one of those commands, which will be commit objects +
> tree differences, is less problematic, perhaps that would be easier to
> share?

I think this could be done.

Nico

-- 
Latest project: cinit-0.2.1 (http://linux.schottelius.org/cinit/)
Open Source nutures open minds and free, creative developers.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: git-log --]
[-- Type: text/plain; charset=utf-8, Size: 4470 bytes --]

diff-tree 056c65efea28066b7b241240f0d8421d3204b624 (from 7a9b0a3d72bc4f5417f3aecb9ca41c7dd5389ca9)
Author: Hansjörg Hassler <hhassler@srsyg03.(none)>
Date:   Thu Nov 17 14:05:51 2005 +0100

    Bug 195 - Tagesarbeitsplan: Nur eine Wartung pro Seite
    
    Description:     	 Opened: 2005-05-01 20:23
    
    .. dies passiert manchmal. Wundi weiss angeblich warum.
    
    ------- Comment #1 From Daniel Wunderlin 2005-05-09 14:28 -------
    
    Nicht reproduzierbar, aber lösbar: einfach Zeitbereich neu in
    Monatsterminierung generieren lassen
    
    ------- Comment #2 From Fabian Hernandez 2005-11-01 14:06 -------
    
    der bug ist plötzlich wieder aufgetreten.
    
    1.0 - Beta(Build 29.10.2005 15:00:07)
    lw26
    
    ------- Comment #3 From Sven Leser 2005-11-02 06:38 -------
    
    *** Bug 348 has been marked as a duplicate of this bug. ***
    
    ------- Comment #4 From Hansjoerg Hassler 2005-11-17 13:02 -------
    
    Dieses kahm durch verunreinigungen der DB bei Updatvorgängen der
    Monatsterminierung.
    Das ganze Anzeigen/Speichern der Daten wurde bei der Monatsterminierung neu
    überarbeitet. nun werden Tagesdaten korrekt abgespeichert was eine
    Voraussetzung ist damit beim Erstellen eines Tagesplanes die Darstellung stimmt
    (Seitenumbrüche)
    
    CGarbeitsplan: Nur eine Wartung pro Seite

:100644 100644 b876a90c52d4b48130450c0af5a11467b14d8410 175e435835deab5e36dc25c17c398f74c55dae9c M	Code/Components/Utilities/LW1Calendar.pas
:100644 100644 ab01a0ad119927394e0a0d108251626480cffaba 5c0b5587aff550af0eb8c584826cf877e0e22738 M	Code/lw1/Client/Terminierung/FeinTerminierungForm.pas
:100644 100644 b5e2bd6d55ae7a2d4a70c02c6d021d8d00b9db11 0b7551631f6c425b61daa60495803d35616844eb M	Code/lw1/Client/Terminierung/FeinTerminierungForm.xfm

diff-tree dcd70e89dc7b2280260628fd699aa906c319c68f (from 10ad8267d77d78b615cf2560fb030dcc8b85c9ad)
Author: Peter Portmann <peterp@srsyg03.(none)>
Date:   Thu Nov 17 10:24:56 2005 +0100

    Bug 359 - Defektmeldungen ueberarbeiten
    Das Suchen nach den Defektmeldungen ist nun eingebaut. Das Ändern des
    Statuses ist noch nicht möglich. Zusätzlich wurden noch die neuen
    Berechtigungen eingefügt.

:100644 100644 0b3b339507a4199f1484d4981bdf4ee31770ec4f 1e8a27a3dfbac681a842e4ac3b17caeba687cd28 M	Code/lw1/Client/Constants.pas
:100644 100644 0047fba049e1fb4a40a0a6385c9acb7f4fb7afbb 7381ca88f1741441bb4f8c3012f1655a1e43b438 M	Code/lw1/Client/DefektMeldung/DefektSearchForm.pas
:100644 100644 4a42ca39f4c74aedc32ac284240184284a05e8a0 d5234e556b2f7c671c79bb133c08b1f49ab1b93b M	Code/lw1/Client/DefektMeldung/DefektSearchForm.xfm
:100644 100644 cb318c4231eb760ff0e6e8f0741b4eb0d68d46f3 0ae36dc7fc3d7740e798b41d72452bab99e450ae M	Code/lw1/Client/MainForm.pas
:100644 100644 6d61131a11188762ac0f342afb820bc7aedb7079 e713a756a93da1d97face30ee7298a062b66e4a7 M	Code/lw1/Client/MainForm.xfm

diff-tree 10ad8267d77d78b615cf2560fb030dcc8b85c9ad (from f1680a99ba34062c23060da1339898779f9a7a69)
Author: Peter Portmann <peterp@srsyg03.(none)>
Date:   Thu Nov 17 10:20:41 2005 +0100

    Datenbank auf lw29 umgestellt.

:100644 100644 fc46e27179ae8b3fb5b8c590988b7b5c905509e4 c3a5e848d4d43111f63ad86d5a6773ae81ca75e3 M	Code/lw1/Client/MainDataModule.xfm
:100644 100644 f1018c16eea62a15b3166c193a63dd52bd7cc5d8 048f85089d3e924848a9a29a5ee4eb36afcaafa3 M	Code/lw1/Client/lw1.conf

diff-tree f1680a99ba34062c23060da1339898779f9a7a69 (from 9e9b91166bfca448a8f1ddeb4580d73ac8ea0986)
Author: Peter Portmann <peterp@srsyg03.(none)>
Date:   Thu Nov 17 10:19:25 2005 +0100

    Neues Icon für lw1. Danke an Fabian Hernandez!

:000000 100644 0000000000000000000000000000000000000000 24f8254c06ea5efc89944a3a9f52d2c79c841c0b A	Code/lw1/Client/Pics/icon/lw1-icon.png

diff-tree 9e9b91166bfca448a8f1ddeb4580d73ac8ea0986 (from b4d9c2fc8df035a340e675fa405ad942b032830b)
Author: Peter Portmann <peterp@srsyg03.(none)>
Date:   Tue Nov 15 11:31:08 2005 +0100

    Neuer Spike: Statistik fÃuer Auswertung.

:000000 100644 0000000000000000000000000000000000000000 5c8c47d855ee0f78a7fb81c4a49224b1607a88b7 A	Code/Spikes/Statistik/Project.dpr
:000000 100644 0000000000000000000000000000000000000000 7bd2de9b7142999bc6bbadde06e73267d1b5ee0e A	Code/Spikes/Statistik/Statistik.pas
:000000 100644 0000000000000000000000000000000000000000 a0fb45e2b906bdac7929cfd8174053015ab629b4 A	Code/Spikes/Statistik/Statistik.xfm

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: files are disappearing in git

[Linus Torvalds]

Ok, 

  Nico gave me private access to the tree, so I quickly cloned it and 
started bisecting it to figure out where the problem was. I haven't looked 
at the source code, and all the commit messages seem to be in German 
(which I can kind of understand if I work at it, but not very well), but 
it definitely turns out none of that matters.

The problem is a bad merge. And in fact, that merge lost _more_ than just 
the three files under Code/Spikes/Statistik/, it also lost a file called 
Code/lw1/Client/Pics/icon/lw1-icon.png.

I don't quite see _how_ it lost them. The merge in question is a totally 
trivial in-index merge, and when I re-do it, I don't lose those files. In 
this case, all the lost files were from the "other branch" of the merge, 
and they were new to that branch. IOW, in git-merge-one-file parlance, it 
is that trivial "added in one" case.

Pasky - do you know of any historical cogito problems like this?

Nico: the files were added in commit

 - 9e9b91166bfca448a8f1ddeb4580d73ac8ea0986:
	Neuer Spike: Statistik ...

and they stayed around in that branch until commit (which is still good):

 - dcd70e89dc7b2280260628fd699aa906c319c68f:
	Bug 359 - Defektmeldungen ...

but they disappeared when the that branch was merged with commit

 - 056c65efea28066b7b241240f0d8421d3204b624
	Bug 195 - Tagesarbeitsplan: ...

and the result is the bad merge commit:

 - 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c:
	Merge with /home/server/git/walderlift.git

After that merge, those new files are gone - they were never added by the 
merge.

Quite frankly, this is clearly a bug, and it looks very much like a really 
serious bug in cg-merge. It's not in the core git: even old versions of 
git should never have been able to generate that broken merge.

But when I try to do that same merge with current cogito, I can't make it 
break _either_. In all cases, regardless of how I do the merge, when I do 
(bad-merge is the merge in your tree, "cg-test" in this case is the merge 
I did using cogito):

	git-diff-tree -r bad-merge cg-test

I get

	:000000 100644 0000000000000000000000000000000000000000 5c8c47d855ee0f78a7fb81c4a49224b1607a88b7 A      Code/Spikes/Statistik/Project.dpr
	:000000 100644 0000000000000000000000000000000000000000 7bd2de9b7142999bc6bbadde06e73267d1b5ee0e A      Code/Spikes/Statistik/Statistik.pas
	:000000 100644 0000000000000000000000000000000000000000 a0fb45e2b906bdac7929cfd8174053015ab629b4 A      Code/Spikes/Statistik/Statistik.xfm
	:000000 100644 0000000000000000000000000000000000000000 24f8254c06ea5efc89944a3a9f52d2c79c841c0b A      Code/lw1/Client/Pics/icon/lw1-icon.png

ie the "bad merge" thing (commit 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c 
in your tree) will not have those four files, and the good merge (whether 
I use the git "recursive" or git "resolve" merge, or use "cg-merge" to do 
it) will always have those four files.

So I do not see how that bad commit happened, especially since it's even 
a fairly recent commit (the date of the merge is 2005-11-17).

I wonder if there's a really old and broken version of cogito somewhere 
around. If so, it's on "srsyg03".

I also went through every single merge in your tree, and apart from two 
merges that had merge conflicts and were fixed up by hand (and that I thus 
didn't check), all other merges looked ok. So as far as I can tell it's 
only that 228b94dd0a7aa1516eb867674cdf8c7c7b2bfd4c merge that is bad.

(And the two commits that needed manual merging _look_ fine. No lost files 
at least, except for one of the merges losing "bitte_bitte_loesch_mich", 
which judging by it's name _should_ be lost ;)

Finally: Nico, I've deleted the trees on my machine, and you can remove my 
ssh key. I don't think I can tell you anything more.

			Linus

Re: files are disappearing in git

[Petr Baudis]

Dear diary, on Fri, Nov 25, 2005 at 08:12:00PM CET, I got a letter
where Linus Torvalds <torvalds@osdl.org> said that...
> The problem is a bad merge. And in fact, that merge lost _more_ than just 
> the three files under Code/Spikes/Statistik/, it also lost a file called 
> Code/lw1/Client/Pics/icon/lw1-icon.png.
> 
> I don't quite see _how_ it lost them. The merge in question is a totally 
> trivial in-index merge, and when I re-do it, I don't lose those files. In 
> this case, all the lost files were from the "other branch" of the merge, 
> and they were new to that branch. IOW, in git-merge-one-file parlance, it 
> is that trivial "added in one" case.
> 
> Pasky - do you know of any historical cogito problems like this?

Puzzling. In the past, cg-merge might get confused by merging on top of
tree with some uncommitted changes (either deleting them, or possibly
intermixing them with the merge - not sure about the latter now), but a
quick scan of Cogito changes over the last few months shows no other
bugs, and I can't remember any major bugfixes besides this either.

Any idea what Cogito version (roughly) could have been used to perform
the merge, Nico?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: files are disappearing in git

[Ryan Anderson]

On Fri, Nov 25, 2005 at 11:12:00AM -0800, Linus Torvalds wrote:
> 
> Ok, 
> 
>   Nico gave me private access to the tree, so I quickly cloned it and 
> started bisecting it to figure out where the problem was. I haven't looked 
> at the source code, and all the commit messages seem to be in German 
> (which I can kind of understand if I work at it, but not very well), but 
> it definitely turns out none of that matters.
> 
> The problem is a bad merge. And in fact, that merge lost _more_ than just 
> the three files under Code/Spikes/Statistik/, it also lost a file called 
> Code/lw1/Client/Pics/icon/lw1-icon.png.
> 
> I don't quite see _how_ it lost them. The merge in question is a totally 
> trivial in-index merge, and when I re-do it, I don't lose those files. In 
> this case, all the lost files were from the "other branch" of the merge, 
> and they were new to that branch. IOW, in git-merge-one-file parlance, it 
> is that trivial "added in one" case.

Can something like this sequence do it?

	git-init-db
	git add file1 ; git commit -m "1"
	(cd .. ; git clone tree1 tree2 )
	(cd .. ; git clone tree1 tree3 )
	git add file2 ; git commit -m "2"
	git push ../tree2/
	git add file3 ; git commit -m "3"

	cd ../tree3/
	git add file4
	git commit -m "4"

	cd ../tree2/
	git pull ../tree3/

The key point being that the merge is done in a tree that has its index
out of sync with its HEAD (git push ../tree2/ .... git pull ../tree3/ )

I think that's the situation where I've personally managed to lose
and/or revert some changes.

-- 

Ryan Anderson
  sometimes Pug Majere

Re: files are disappearing in git

[Junio C Hamano]

Ryan Anderson <ryan@michonline.com> writes:

> Can something like this sequence do it?
>
> ...
>
> The key point being that the merge is done in a tree that has its index
> out of sync with its HEAD (git push ../tree2/ .... git pull ../tree3/ )
>
> I think that's the situation where I've personally managed to lose
> and/or revert some changes.

The moral of the story is not to push into the checked-out
branch of a non-naked repository, or you need to do what you are
doing if you choose to do so.  It might have made sense if index
file recorded which tree object its contents came from, but I am
not sure what tools is responsible for recording that if we
choose to extend index file to store that information.  Probably
read-tree is, but it should not do that unconditionally.

In any case, the workflow presented by Nick is that individual
developers use cg-* commands in CVS like workflow where the only
place you push into is the central repository and everybody
pulls from there, so I do not think the above example applies.

Re: files are disappearing in git

[Linus Torvalds]

On Fri, 25 Nov 2005, Ryan Anderson wrote:
> 
> Can something like this sequence do it?

Nope, I don't think that should matter. Also, that doesn't seem to match 
what Nico & co are doing, but that's hard to tell.. 

A merge result should be totally independent of the index file(s) 
involved.

A dirty index file can cause a merge to _fail_, in that git may refuse to 
do the merge at all because of the index not matching the original branch, 
but a successful automated merge should never have any dependencies on 
what happens to be in the index at the time the merge was done.

So you can think of a merge as being totally specified by the trees 
involved, unless we have some bug, of course. I can't think of any.

Now, what _can_ happen (I think) is that if a merge is a failure (and 
there, a dirty index can certainly be the _cause_ of that failure), then 
when you fix it up and commit, there's some mix-up at _that_ stage.

For example, let's say that you had a dirty tree or something, and then 
the merge failed, and you didn't see anything wrong, so you just end up 
doing a "git commit". At _that_ point, what you had in the index matters 
very much, of course, since the index is what will be committed.

> I think that's the situation where I've personally managed to lose
> and/or revert some changes.

Hmm.. Can you elaborate?

(Side note: all my commentary is purely about the "raw git" interfaces. I 
don't know what cogito may do on top of it).

		Linus

Re: files are disappearing in git

[Petr Baudis]

Dear diary, on Fri, Nov 25, 2005 at 11:11:22PM CET, I got a letter
where Linus Torvalds <torvalds@osdl.org> said that...
> For example, let's say that you had a dirty tree or something, and then 
> the merge failed, and you didn't see anything wrong, so you just end up 
> doing a "git commit". At _that_ point, what you had in the index matters 
> very much, of course, since the index is what will be committed.
> 
> > I think that's the situation where I've personally managed to lose
> > and/or revert some changes.
> 
> Hmm.. Can you elaborate?
> 
> (Side note: all my commentary is purely about the "raw git" interfaces. I 
> don't know what cogito may do on top of it).

Note that Cogito is now (v0.15.1 or later) supposed to handle merges on
top of trees with local changes fine - it will either error out, or in
case the merge is done on unrelated files it will temporarily ignore
your local changes and cg-commit won't mix them up (even if you do some
conflict fixups).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: files are disappearing in git

[Nico -telmich- Schottelius]

[-- Attachment #1: Type: text/plain, Size: 2655 bytes --]

Good evening,


Linus Torvalds [Fri, Nov 25, 2005 at 11:12:00AM -0800]:
>   Nico gave me private access to the tree, so I quickly cloned it and 
> started bisecting it to figure out where the problem was. I haven't looked 
> at the source code, and all the commit messages seem to be in German 
> (which I can kind of understand if I work at it, but not very well), but 
> it definitely turns out none of that matters.

Well, there's still a broken username, but that's our problem with
iso8859-1 in passwd.

> The problem is a bad merge. And in fact, that merge lost _more_ than just 
> the three files under Code/Spikes/Statistik/, it also lost a file called 
> Code/lw1/Client/Pics/icon/lw1-icon.png.

Good to know, we didn't that we lost it until now :)

> Pasky - do you know of any historical cogito problems like this?

For information:

[22:07] srsyg03:packages% ls -l /usr/packages 
insgesamt 28
drwxr-xr-x  4 root root 4096 Nov 10 15:52 cogito-/
drwxr-xr-x  4 root root 4096 Okt 14 15:15 cogito-0.15/
drwxr-xr-x  4 root root 4096 Nov 23 13:31 cogito-73874dddeec2d0a8e5cd343eec762d98314def63/
drwxr-xr-x  4 root root 4096 Okt 14 15:15 cvsps-2.1/
drwxr-xr-x  4 root root 4096 Nov 10 15:52 git-/
drwxr-xr-x  4 root root 4096 Okt 17 14:09 git-20051016.git/
drwxr-xr-x  4 root root 4096 Nov 23 13:31 git-c61642185d411e5e3350566a68483e358ca392b9/

At the time of 2005-11-17 we'll have used a cogito and git version,
which was from 2005-11-10.

> [...] 
> So I do not see how that bad commit happened, especially since it's even 
> a fairly recent commit (the date of the merge is 2005-11-17).
> 
> I wonder if there's a really old and broken version of cogito somewhere 
> around. If so, it's on "srsyg03".

Above you see the versions of which were/are installed. I always
link the latest binaries to /usr/local/bin/, so our developers do not
need to care about what version we have.

> (And the two commits that needed manual merging _look_ fine. No lost files 
> at least, except for one of the merges losing "bitte_bitte_loesch_mich", 
> which judging by it's name _should_ be lost ;)

Those were some lessons in our house. Our developers had to test
if everything works, including deleting :)

> Finally: Nico, I've deleted the trees on my machine, and you can remove my 
> ssh key. I don't think I can tell you anything more.

Ok. Thanks for your help! I think we'll add those four files
back to git on monday and continue to work with them.

Nico

-- 
Latest project: cinit-0.2.1 (http://linux.schottelius.org/cinit/)
Open Source nutures open minds and free, creative developers.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: files are disappearing in git

[Petr Baudis]

Dear diary, on Fri, Nov 25, 2005 at 10:28:46PM CET, I got a letter
where Nico -telmich- Schottelius <nico-linux-git@schottelius.org> said that...
> > Pasky - do you know of any historical cogito problems like this?
> 
> For information:
> 
> [22:07] srsyg03:packages% ls -l /usr/packages 
> insgesamt 28
> drwxr-xr-x  4 root root 4096 Nov 10 15:52 cogito-/
> drwxr-xr-x  4 root root 4096 Okt 14 15:15 cogito-0.15/
> drwxr-xr-x  4 root root 4096 Nov 23 13:31 cogito-73874dddeec2d0a8e5cd343eec762d98314def63/
> drwxr-xr-x  4 root root 4096 Okt 14 15:15 cvsps-2.1/
> drwxr-xr-x  4 root root 4096 Nov 10 15:52 git-/
> drwxr-xr-x  4 root root 4096 Okt 17 14:09 git-20051016.git/
> drwxr-xr-x  4 root root 4096 Nov 23 13:31 git-c61642185d411e5e3350566a68483e358ca392b9/
> 
> At the time of 2005-11-17 we'll have used a cogito and git version,
> which was from 2005-11-10.

That's really weird - I can't see anything since then that could
influence it. The only possibility is that you were working on those
files before, left them modified but uncommitted, then did a merge which
would touch them, the merge would fail because of local changes, then
you would delete your local instances of the files and try the merge
again.

That's something I fixed only very recently and the fix is not even in
any public release yet; I didn't realize that it could have such
dangerous consequences, but apparently it does. I guess I will release
0.16 final tomorrow.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.