* Error:non-monotonic index after failed recursive "sed" command
@ 2013-01-14 11:40 George Karpenkov
2013-01-14 12:06 ` Matthieu Moy
2013-01-14 12:21 ` Johannes Sixt
0 siblings, 2 replies; 9+ messages in thread
From: George Karpenkov @ 2013-01-14 11:40 UTC (permalink / raw)
To: git
Hi All,
I've managed to corrupt my very valuable repository with a recursive
sed which went wrong.
I wanted to convert all tabs to spaces with the following command:
find ./ -name '*.*' -exec sed -i 's/\t/ /g' {} \;
I think that has changed not only the files in the repo, but the data
files in .git directory itself. As a result, my index became
corrupted, and almost every single command dies:
> git log
error: non-monotonic index
.git/objects/pack/pack-314b1944adebea645526b6724b2044c1313241f5.idx
error: non-monotonic index
.git/objects/pack/pack-75c95b0defe1968b61e4f4e1ab7040d35110bfdc.idx
...
error: non-monotonic index
.git/objects/pack/pack-3da0da48d05140b55f4af1cf87c55a2d7898bdd5.idx
fatal: bad object HEAD
Output for git fsck is even worse:
> git fsck
error: non-monotonic index
.git/objects/pack/pack-434f8445672a92f123accffce651bdb693bd8fcb.idx
...
error: non-monotonic index
.git/objects/pack/pack-0c9d5ae4e2b46dd78dace7533adf6cdfe10326ef.idx
error: non-monotonic index
.git/objects/pack/pack-e8bd5c7f85e96e7e548a62954a8f7c7f223ba9e0.idx
Segmentation fault (core dumped)
Any advice? I've lost about 2 weeks worth of work.
Is there anything better I can try to do other then trying to
reconstruct the data manually from git blobs?
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 11:40 Error:non-monotonic index after failed recursive "sed" command George Karpenkov @ 2013-01-14 12:06 ` Matthieu Moy 2013-01-14 12:12 ` John Keeping 2013-01-14 12:21 ` Johannes Sixt 1 sibling, 1 reply; 9+ messages in thread From: Matthieu Moy @ 2013-01-14 12:06 UTC (permalink / raw) To: George Karpenkov; +Cc: git George Karpenkov <george@metaworld.ru> writes: > Hi All, > > I've managed to corrupt my very valuable repository with a recursive > sed which went wrong. > I wanted to convert all tabs to spaces with the following command: > > find ./ -name '*.*' -exec sed -i 's/\t/ /g' {} \; Clearly, this is a dangerous command as it impacts .git/. However, Git partially protects you from this kind of error, since object files and pack files are read-only by default. My obvious first advice is: make backups of your corrupted repository. Yes, I said backup_s_, better safe than sorry. Then, the errors you get are in *.idx files, which are basically index for pack files, for quicker access. You can try removing these files, and then running "git index-pack" on each pack file, like $ rm .git/objects/pack/pack-*.idx $ git index-pack .git/objects/pack/pack-4745076928ca4df932a3727b8cc25e83574560bb.pack 4745076928ca4df932a3727b8cc25e83574560bb $ git index-pack .git/objects/pack/pack-c74a6514f653d0269cdcdf9c1c102d326706bbda.pack c74a6514f653d0269cdcdf9c1c102d326706bbda -- Matthieu Moy http://www-verimag.imag.fr/~moy/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 12:06 ` Matthieu Moy @ 2013-01-14 12:12 ` John Keeping 0 siblings, 0 replies; 9+ messages in thread From: John Keeping @ 2013-01-14 12:12 UTC (permalink / raw) To: Matthieu Moy; +Cc: George Karpenkov, git On Mon, Jan 14, 2013 at 01:06:16PM +0100, Matthieu Moy wrote: > George Karpenkov <george@metaworld.ru> writes: >> I've managed to corrupt my very valuable repository with a recursive >> sed which went wrong. >> I wanted to convert all tabs to spaces with the following command: >> >> find ./ -name '*.*' -exec sed -i 's/\t/ /g' {} \; > > Clearly, this is a dangerous command as it impacts .git/. However, Git > partially protects you from this kind of error, since object files and > pack files are read-only by default. > > My obvious first advice is: make backups of your corrupted repository. > Yes, I said backup_s_, better safe than sorry. Having backed up the corrupt state, another option is to just try running the reverse command: find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; I had a quick grep over some pack indices here and ' ' doesn't occur in any of mine whereas '\t' is very common so you may find that the only ' ' sequences are the ones you introduced. John ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 11:40 Error:non-monotonic index after failed recursive "sed" command George Karpenkov 2013-01-14 12:06 ` Matthieu Moy @ 2013-01-14 12:21 ` Johannes Sixt 2013-01-14 19:06 ` Junio C Hamano 1 sibling, 1 reply; 9+ messages in thread From: Johannes Sixt @ 2013-01-14 12:21 UTC (permalink / raw) To: George Karpenkov; +Cc: git Am 1/14/2013 12:40, schrieb George Karpenkov: > I've managed to corrupt my very valuable repository with a recursive > sed which went wrong. > I wanted to convert all tabs to spaces with the following command: > > find ./ -name '*.*' -exec sed -i 's/\t/ /g' {} \; > > I think that has changed not only the files in the repo, but the data > files in .git directory itself. As a result, my index became > corrupted, and almost every single command dies: > >> git log > error: non-monotonic index > ..git/objects/pack/pack-314b1944adebea645526b6724b2044c1313241f5.idx > error: non-monotonic index > ..git/objects/pack/pack-75c95b0defe1968b61e4f4e1ab7040d35110bfdc.idx > .... > error: non-monotonic index > ..git/objects/pack/pack-3da0da48d05140b55f4af1cf87c55a2d7898bdd5.idx > fatal: bad object HEAD > > Output for git fsck is even worse: > >> git fsck > error: non-monotonic index > ..git/objects/pack/pack-434f8445672a92f123accffce651bdb693bd8fcb.idx > .... > error: non-monotonic index > ..git/objects/pack/pack-0c9d5ae4e2b46dd78dace7533adf6cdfe10326ef.idx > error: non-monotonic index > ..git/objects/pack/pack-e8bd5c7f85e96e7e548a62954a8f7c7f223ba9e0.idx > Segmentation fault (core dumped) > > Any advice? I've lost about 2 weeks worth of work. > Is there anything better I can try to do other then trying to > reconstruct the data manually from git blobs? First things first: MAKE A COPY OF THE CURRENT STATE of the repository, including the worktree, so that you have something to go back to if things get worse later. (At the very least, we want to debug the segfault that we see above.) So far that's all I can say about your case. Everything that follows is just a shot in the dark, and others may have better ideas. Also, look here: https://github.com/gitster/git/blob/master/Documentation/howto/recover-corrupted-blob-object.txt You can remove the *.idx files, because they do not carry essential information. Now try git fsck; git repack. Try the reverse edit: find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; Remove .git/index; it can be reconstructed (of course, assuming you started all this with a clean index; if not, you lose the staged changes). -- Hannes ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 12:21 ` Johannes Sixt @ 2013-01-14 19:06 ` Junio C Hamano 2013-01-14 19:13 ` Matthieu Moy 0 siblings, 1 reply; 9+ messages in thread From: Junio C Hamano @ 2013-01-14 19:06 UTC (permalink / raw) To: Johannes Sixt; +Cc: George Karpenkov, git Johannes Sixt <j.sixt@viscovery.net> writes: > Am 1/14/2013 12:40, schrieb George Karpenkov: >> I've managed to corrupt my very valuable repository with a recursive >> sed which went wrong. >> I wanted to convert all tabs to spaces with the following command: >> >> find ./ -name '*.*' -exec sed -i 's/\t/ /g' {} \; >> >> I think that has changed not only the files in the repo, but the data >> files in .git directory itself. As a result, my index became >> corrupted, and almost every single command dies: >> >>> git log >> error: non-monotonic index >> ..git/objects/pack/pack-314b1944adebea645526b6724b2044c1313241f5.idx >> error: non-monotonic index >> ..git/objects/pack/pack-75c95b0defe1968b61e4f4e1ab7040d35110bfdc.idx >> .... > ... > Try the reverse edit: > > find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; > > Remove .git/index; it can be reconstructed (of course, assuming you > started all this with a clean index; if not, you lose the staged changes). Everybody seems to be getting an impression that .idx is the only thing that got corrupt. Where does that come from? I do not see anything that prevents the original command line from touching *.pack files. Loose objects may have been left unmolested as their names do not match '*.*', though. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 19:06 ` Junio C Hamano @ 2013-01-14 19:13 ` Matthieu Moy 2013-01-14 22:57 ` George Karpenkov 0 siblings, 1 reply; 9+ messages in thread From: Matthieu Moy @ 2013-01-14 19:13 UTC (permalink / raw) To: Junio C Hamano; +Cc: Johannes Sixt, George Karpenkov, git Junio C Hamano <gitster@pobox.com> writes: > Everybody seems to be getting an impression that .idx is the only > thing that got corrupt. Where does that come from? It's the only thing that appear in the error message. This does not imply that it is the only corrupt thing, but gives a little hope that it may still be the case. Actually, I thought the "read-only" protection should have protected files in object/ directory, but a little testing shows that "sed -i" gladly accepts to modify read-only files (technically, it does not modify it, but creates a temporary file with the new content, and then renames it to the new location). So, the hope that pack files are uncorrupted is rather thin unfortunately. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 19:13 ` Matthieu Moy @ 2013-01-14 22:57 ` George Karpenkov 2013-01-14 23:45 ` Philip Oakley 0 siblings, 1 reply; 9+ messages in thread From: George Karpenkov @ 2013-01-14 22:57 UTC (permalink / raw) To: Matthieu Moy; +Cc: Junio C Hamano, Johannes Sixt, git Thanks everyone! Progress so far: After executing reverse sed command: find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; And trying to switch the branch I get: > git checkout X error: failed to read object 51a980792f26875d00acb79a19f043420f542cfa at offset 41433013 from .git/objects/pack/pack-8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack fatal: packed object 51a980792f26875d00acb79a19f043420f542cfa (stored in .git/objects/pack/pack- 8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack) is corrupt So the actual .pack file is corrupt, unfortunately. On Tue, Jan 15, 2013 at 6:13 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: > Junio C Hamano <gitster@pobox.com> writes: > >> Everybody seems to be getting an impression that .idx is the only >> thing that got corrupt. Where does that come from? > > It's the only thing that appear in the error message. This does not > imply that it is the only corrupt thing, but gives a little hope that it > may still be the case. > > Actually, I thought the "read-only" protection should have protected > files in object/ directory, but a little testing shows that "sed -i" > gladly accepts to modify read-only files (technically, it does not > modify it, but creates a temporary file with the new content, and then > renames it to the new location). So, the hope that pack files are > uncorrupted is rather thin unfortunately. > > -- > Matthieu Moy > http://www-verimag.imag.fr/~moy/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 22:57 ` George Karpenkov @ 2013-01-14 23:45 ` Philip Oakley 2013-01-15 0:54 ` George Karpenkov 0 siblings, 1 reply; 9+ messages in thread From: Philip Oakley @ 2013-01-14 23:45 UTC (permalink / raw) To: George Karpenkov, Matthieu Moy; +Cc: Junio C Hamano, Johannes Sixt, git From: "George Karpenkov" <george@metaworld.ru> Sent: Monday, January 14, 2013 10:57 PM > Thanks everyone! > > Progress so far: > > After executing reverse sed command: > find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; Have you counted how many substitutions there are in the pack file(s). It may be sufficiently small that you can simply try all the possible combinations of fwd and reverse substitutions. Even if it takes a few days the computer won't get bored ;-) > > And trying to switch the branch I get: > >> git checkout X > > error: failed to read object 51a980792f26875d00acb79a19f043420f542cfa > at offset 41433013 from > .git/objects/pack/pack-8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack > fatal: packed object 51a980792f26875d00acb79a19f043420f542cfa (stored > in .git/objects/pack/pack- > 8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack) is corrupt > > So the actual .pack file is corrupt, unfortunately. > > On Tue, Jan 15, 2013 at 6:13 AM, Matthieu Moy > <Matthieu.Moy@grenoble-inp.fr> wrote: >> Junio C Hamano <gitster@pobox.com> writes: >> >>> Everybody seems to be getting an impression that .idx is the only >>> thing that got corrupt. Where does that come from? >> >> It's the only thing that appear in the error message. This does not >> imply that it is the only corrupt thing, but gives a little hope that >> it >> may still be the case. >> >> Actually, I thought the "read-only" protection should have protected >> files in object/ directory, but a little testing shows that "sed -i" >> gladly accepts to modify read-only files (technically, it does not >> modify it, but creates a temporary file with the new content, and >> then >> renames it to the new location). So, the hope that pack files are >> uncorrupted is rather thin unfortunately. >> >> -- >> Matthieu Moy >> http://www-verimag.imag.fr/~moy/ > -- ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Error:non-monotonic index after failed recursive "sed" command 2013-01-14 23:45 ` Philip Oakley @ 2013-01-15 0:54 ` George Karpenkov 0 siblings, 0 replies; 9+ messages in thread From: George Karpenkov @ 2013-01-15 0:54 UTC (permalink / raw) To: Philip Oakley; +Cc: Matthieu Moy, Junio C Hamano, Johannes Sixt, git Happy ending! Turns out i have actually made a backup 3 days ago. My other work was on a branch + in a stash. Commits done on a branch were already present in a backup. I was able to get the stash working by copying corrupted .pack files from the backup, luckily all the new work wasn't packed yet. So i've just verified the log messages to see that no new commits were made, created a patch from the corrupted git repo of the stash, applied it on the backup, and wo-hooo, everything worked. And then I've pushed to origin to avoid such silliness in the future. Thanks and Regards, George On Tue, Jan 15, 2013 at 10:45 AM, Philip Oakley <philipoakley@iee.org> wrote: > From: "George Karpenkov" <george@metaworld.ru> > Sent: Monday, January 14, 2013 10:57 PM > >> Thanks everyone! >> >> Progress so far: >> >> After executing reverse sed command: >> find .git -name '*.*' -exec sed -i 's/ /\t/g' {} \; > > > Have you counted how many substitutions there are in the pack file(s). It > may be sufficiently small that you can simply try all the possible > combinations of fwd and reverse substitutions. Even if it takes a few days > the computer won't get bored ;-) > >> >> And trying to switch the branch I get: >> >>> git checkout X >> >> >> error: failed to read object 51a980792f26875d00acb79a19f043420f542cfa >> at offset 41433013 from >> .git/objects/pack/pack-8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack >> fatal: packed object 51a980792f26875d00acb79a19f043420f542cfa (stored >> in .git/objects/pack/pack- >> 8d629235ee9fec9c6683d42e3edb21a1b0f6e027.pack) is corrupt >> >> So the actual .pack file is corrupt, unfortunately. >> >> On Tue, Jan 15, 2013 at 6:13 AM, Matthieu Moy >> <Matthieu.Moy@grenoble-inp.fr> wrote: >>> >>> Junio C Hamano <gitster@pobox.com> writes: >>> >>>> Everybody seems to be getting an impression that .idx is the only >>>> thing that got corrupt. Where does that come from? >>> >>> >>> It's the only thing that appear in the error message. This does not >>> imply that it is the only corrupt thing, but gives a little hope that it >>> may still be the case. >>> >>> Actually, I thought the "read-only" protection should have protected >>> files in object/ directory, but a little testing shows that "sed -i" >>> gladly accepts to modify read-only files (technically, it does not >>> modify it, but creates a temporary file with the new content, and then >>> renames it to the new location). So, the hope that pack files are >>> uncorrupted is rather thin unfortunately. >>> >>> -- >>> Matthieu Moy >>> http://www-verimag.imag.fr/~moy/ >> >> -- > > ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-01-15 0:55 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-01-14 11:40 Error:non-monotonic index after failed recursive "sed" command George Karpenkov 2013-01-14 12:06 ` Matthieu Moy 2013-01-14 12:12 ` John Keeping 2013-01-14 12:21 ` Johannes Sixt 2013-01-14 19:06 ` Junio C Hamano 2013-01-14 19:13 ` Matthieu Moy 2013-01-14 22:57 ` George Karpenkov 2013-01-14 23:45 ` Philip Oakley 2013-01-15 0:54 ` George Karpenkov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).