Rollback of git commands

Rollback of git commands

[Jon Smirl]

Could a rollback log be implemented in git? It would make things way
easier when you screw something up.You'd only roll back things that
impacted the object store, not things like checkout.

stg is also starting to store significant state in the .git directory.
It may be better to move this state into a git object. That would
allow the state history to be tracked and rollback be implemented.

The same logic can be applied to moving all tracking information in
the .git directory into objects. You then just need to track a single
SHA1 pointing to the .git config info in the object db.

Rollback in the object store is simple, just move the SHA1 in the
config files back to where it was before the action was done. What we
are missing is a way to rollback the config files.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Junio C Hamano]

"Jon Smirl" <jonsmirl@gmail.com> writes:

> Could a rollback log be implemented in git? It would make things way
> easier when you screw something up. You'd only roll back things that
> impacted the object store, not things like checkout.

The object store is append only, and if you disregard SHA-1 collisions,
I do not think there is much you can gain from being able to roll back
only object store.

For example, you would want to be able to roll back where your 'master'
branch was pointing at before you started that botched operation.  That
is not in the object store at all (we have reflogs for that).

Another example, you might have done quite an elaborate interactive add
to stage only some changes to a path, but then accidentally said "git
add" that path to stage the whole thing.  You may say "oops, that state
was only in the index and now it is lost."  The blob that records the
staged content _DOES_ exist in the object store in such a case so it is
not lost --- there is nothing to roll back.  What you lost is a pointer
into the object store (we do not have anything like reflog for
individual index entry --- not that I would suggest adding one).

Creating a blob that records all of .git/config, output from
for-each-ref, output from "symbolic-ref HEAD" and output from ls-files
-s every time you run _any_ git operation, and restore the state when
you want to, would conceptually work, as you suggest, but I am not sure
how practical it would be, performancewise, spacewise, and
semanticswise.

Re: Rollback of git commands

[Jon Smirl]

On 11/27/07, Junio C Hamano <gitster@pobox.com> wrote:
> "Jon Smirl" <jonsmirl@gmail.com> writes:
>
> > Could a rollback log be implemented in git? It would make things way
> > easier when you screw something up. You'd only roll back things that
> > impacted the object store, not things like checkout.
>
> The object store is append only, and if you disregard SHA-1 collisions,
> I do not think there is much you can gain from being able to roll back
> only object store.
>
> For example, you would want to be able to roll back where your 'master'
> branch was pointing at before you started that botched operation.  That
> is not in the object store at all (we have reflogs for that).
>
> Another example, you might have done quite an elaborate interactive add
> to stage only some changes to a path, but then accidentally said "git
> add" that path to stage the whole thing.  You may say "oops, that state
> was only in the index and now it is lost."  The blob that records the
> staged content _DOES_ exist in the object store in such a case so it is
> not lost --- there is nothing to roll back.  What you lost is a pointer
> into the object store (we do not have anything like reflog for
> individual index entry --- not that I would suggest adding one).
>
> Creating a blob that records all of .git/config, output from
> for-each-ref, output from "symbolic-ref HEAD" and output from ls-files
> -s every time you run _any_ git operation, and restore the state when
> you want to, would conceptually work, as you suggest, but I am not sure
> how practical it would be, performancewise, spacewise, and
> semanticswise.

I'm only looking for a command that would rollback the effect of
changes to the object store (you don't have to remove the objects).
Losing complex staging would be ok since it can be recreated.

Let's take my recent problem as an example. I typed 'git rebase
linus/master' instead of 'stg rebase linus/master'. Then I typed 'stg
repair'. The repair failed and left me in a mess. Both of these are
easy to rollback except for the fact that stg has stored a bunch of
state in .git/*.

After doing the commands I located my last commit before the rebase
and edited master back to  it. But my system was still messed up since
moving master got me out of sync with the state stg stored in .git/*.
The 'stg repair' command had changed the stored state.

Instead lets store the contents of .git/* (minus the data itself) as
objects. Then commit a new object after each command. Now rollback can
be accomplished by walking back this chain off commits. Rollback would
wipe out any staging, and effectively reset the working tree to match
the point rolled back to. I don't think we need to capture the entire
state of 'ls-file -s' to achieve this.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Jon Smirl]

Rollback is too strong of name for this. Checkpoints would be better.
The idea is to record the total system state at convenient moments and
then allow moving back to the checkpointed state. The object store
supports this, but the rest of the state in .git/* isn't being
recorded.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[David Symonds]

On Nov 28, 2007 12:49 PM, Jon Smirl <jonsmirl@gmail.com> wrote:
> Rollback is too strong of name for this. Checkpoints would be better.
> The idea is to record the total system state at convenient moments and
> then allow moving back to the checkpointed state. The object store
> supports this, but the rest of the state in .git/* isn't being
> recorded.

rsync -a .git /somewhere/safe

I fear that what you ask becomes a chicken-and-egg scenario: where/how
is this checkpointing information going to be stored? If it's tightly
integrated with Git, what happens when you want to roll-back a
checkpoint-restore?


Dave.

Re: Rollback of git commands

[Ingo Molnar]

* David Symonds <dsymonds@gmail.com> wrote:

> On Nov 28, 2007 12:49 PM, Jon Smirl <jonsmirl@gmail.com> wrote:
> > Rollback is too strong of name for this. Checkpoints would be better.
> > The idea is to record the total system state at convenient moments and
> > then allow moving back to the checkpointed state. The object store
> > supports this, but the rest of the state in .git/* isn't being
> > recorded.
> 
> rsync -a .git /somewhere/safe
> 
> I fear that what you ask becomes a chicken-and-egg scenario: where/how 
> is this checkpointing information going to be stored? If it's tightly 
> integrated with Git, what happens when you want to roll-back a 
> checkpoint-restore?

well, it would/could be the normal undo/redo semantics of editors: you 
can undo-redo in a linear history fashion, in an unlimited way, but the 
moment you modify any past point of history then the redo future is 
overriden. (but the 'past' up to that point is still recorded and 
available)

and this could all be driven via .git/logs - as long as all other 
metadata is imported into the object store and the root of this git tree 
would be represented in a single file. The logs are append-only as well, 
the loss of them means the loss of undo/redo information, nothing else. 
Figuring out the linear history from the logs would be computationally 
expensive, but that's not a big issue as 'undo/redo' would be a rare 
operation anyway.

But i guess i must be missing some obvious complication?

	Ingo

Re: Rollback of git commands

[Sergei Organov]

Ingo Molnar <mingo@elte.hu> writes:

> * David Symonds <dsymonds@gmail.com> wrote:
>
>> On Nov 28, 2007 12:49 PM, Jon Smirl <jonsmirl@gmail.com> wrote:
>> > Rollback is too strong of name for this. Checkpoints would be better.
>> > The idea is to record the total system state at convenient moments and
>> > then allow moving back to the checkpointed state. The object store
>> > supports this, but the rest of the state in .git/* isn't being
>> > recorded.
>> 
>> rsync -a .git /somewhere/safe
>> 
>> I fear that what you ask becomes a chicken-and-egg scenario: where/how 
>> is this checkpointing information going to be stored? If it's tightly 
>> integrated with Git, what happens when you want to roll-back a 
>> checkpoint-restore?
>
> well, it would/could be the normal undo/redo semantics of editors: you 
> can undo-redo in a linear history fashion, in an unlimited way, but the 
> moment you modify any past point of history then the redo future is 
> overriden. (but the 'past' up to that point is still recorded and 
> available)

Or it could be Emacs-like: 'undo' is just another operation that is a
subject for further undo's ;) Then there is no need for 'redo', and no
need to override either the future or the past. Besides this obvious
technical superiority will help to maintain git's reputation of being
hard to grok ;)

-- 
Sergei.

Re: Rollback of git commands

[Nicolas Pitre]

On Wed, 28 Nov 2007, Sergei Organov wrote:

> Ingo Molnar <mingo@elte.hu> writes:
> 
> > well, it would/could be the normal undo/redo semantics of editors: you 
> > can undo-redo in a linear history fashion, in an unlimited way, but the 
> > moment you modify any past point of history then the redo future is 
> > overriden. (but the 'past' up to that point is still recorded and 
> > available)
> 
> Or it could be Emacs-like: 'undo' is just another operation that is a
> subject for further undo's ;) Then there is no need for 'redo', and no
> need to override either the future or the past.

The reflog does just that in fact, when it records your 'git reset' 
operations.

> Besides this obvious technical superiority will help to maintain git's 
> reputation of being hard to grok ;)

Sure!  ;)


Nicolas

Re: Rollback of git commands

[Geert Bosch]

On Nov 27, 2007, at 20:49, Jon Smirl wrote:
> Rollback is too strong of name for this. Checkpoints would be better.
> The idea is to record the total system state at convenient moments and
> then allow moving back to the checkpointed state. The object store
> supports this, but the rest of the state in .git/* isn't being
> recorded.

I have always wondered why refs and tags are not
in the regular object store. In a way, there
should be just one root pointer (SHA1) pointing
to a tree with refs, etc.

As for the problem of "lots of loose objects", each
command could write a single pack file. By directly
writing deltas instead of complete new tree objects,
storage requirements should be modest. Also, just
writing a single new pack file is very efficient
for I/O purposes.

To prevent too many pack files, a merge policy
could allow a new pack to include (and thus obsolete)
a number of similarly sized or smaller packs.
After the new file has been successfully written, the
old files are no longer necessary and can be
moved to a "trashes" area to be expired.

   -Geert

Re: Rollback of git commands

[Jeff King]

On Tue, Nov 27, 2007 at 11:07:29PM -0500, Geert Bosch wrote:

> I have always wondered why refs and tags are not
> in the regular object store. In a way, there
> should be just one root pointer (SHA1) pointing
> to a tree with refs, etc.

Assuming that they also retain the "having an object implies having all
of the objects it points to" property, then it makes it hard to talk
about subsets or single refs. If I fetch from you and you communicate
your repo state as some hash, then I am stuck getting _all_ of your
refs to complete this property.

-Peff

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Jeff King <peff@peff.net> wrote:
> On Tue, Nov 27, 2007 at 11:07:29PM -0500, Geert Bosch wrote:
>
> > I have always wondered why refs and tags are not
> > in the regular object store. In a way, there
> > should be just one root pointer (SHA1) pointing
> > to a tree with refs, etc.
>
> Assuming that they also retain the "having an object implies having all
> of the objects it points to" property, then it makes it hard to talk
> about subsets or single refs. If I fetch from you and you communicate
> your repo state as some hash, then I am stuck getting _all_ of your
> refs to complete this property.

push/pull would still work at the branch level. The local state
tracking objects wouldn't be exchanged.

>
> -Peff
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Jeff King]

On Wed, Nov 28, 2007 at 11:23:30AM -0500, Jon Smirl wrote:

> > Assuming that they also retain the "having an object implies having all
> > of the objects it points to" property, then it makes it hard to talk
> > about subsets or single refs. If I fetch from you and you communicate
> > your repo state as some hash, then I am stuck getting _all_ of your
> > refs to complete this property.
> 
> push/pull would still work at the branch level. The local state
> tracking objects wouldn't be exchanged.

Fair enough.

My spider sense still tingles about this, and I have the feeling that
you might run into weird merge problems when two refs are updated
simultaneously. I guess that shouldn't happen since you will write out
the complete "here are the refs" after every operation, leaving no room
for simultaneous updates, but I haven't given it enough thought to be
sure there aren't funny corner cases.

-Peff

Re: Rollback of git commands

[Sean]

On Tue, November 27, 2007 8:33 pm, Jon Smirl said:

Hi Jon,

> I'm only looking for a command that would rollback the effect of
> changes to the object store (you don't have to remove the objects).
> Losing complex staging would be ok since it can be recreated.
>
> Let's take my recent problem as an example. I typed 'git rebase
> linus/master' instead of 'stg rebase linus/master'. Then I typed 'stg
> repair'. The repair failed and left me in a mess. Both of these are
> easy to rollback except for the fact that stg has stored a bunch of
> state in .git/*.
>
> After doing the commands I located my last commit before the rebase
> and edited master back to  it. But my system was still messed up since
> moving master got me out of sync with the state stg stored in .git/*.
> The 'stg repair' command had changed the stored state.

 From your description is seems that Git proper was able to handle the
situation just fine.   It sounds instead like you're describing a problem
with Stg where it became confused without a way to restore _its_ meta
data.  There's not much Git itself can do to help in this situation
unless Stg stores all of its meta-data as standard Git objects, rather
than just using the .git directory.

Sean

Re: Rollback of git commands

[Jon Smirl]

On 11/27/07, Sean <seanlkml@sympatico.ca> wrote:
> On Tue, November 27, 2007 8:33 pm, Jon Smirl said:
>
> Hi Jon,
>
> > I'm only looking for a command that would rollback the effect of
> > changes to the object store (you don't have to remove the objects).
> > Losing complex staging would be ok since it can be recreated.
> >
> > Let's take my recent problem as an example. I typed 'git rebase
> > linus/master' instead of 'stg rebase linus/master'. Then I typed 'stg
> > repair'. The repair failed and left me in a mess. Both of these are
> > easy to rollback except for the fact that stg has stored a bunch of
> > state in .git/*.
> >
> > After doing the commands I located my last commit before the rebase
> > and edited master back to  it. But my system was still messed up since
> > moving master got me out of sync with the state stg stored in .git/*.
> > The 'stg repair' command had changed the stored state.
>
>  From your description is seems that Git proper was able to handle the
> situation just fine.   It sounds instead like you're describing a problem
> with Stg where it became confused without a way to restore _its_ meta
> data.  There's not much Git itself can do to help in this situation
> unless Stg stores all of its meta-data as standard Git objects, rather
> than just using the .git directory.

Patch management is an important part of the work flow. I would hope
that git implements patch management as a core feature in future
versions. stgit/guilt/quilt are valuable since they blazed the trail
and figured out what commands are useful. As time passes these
features can become more highly integrated into core git.

Of course you've never screwed up a repository using git commands,
right? I've messed up plenty. A good way to mess up a repo is to get
the data in .git/* out of sync with what is in the repo. I'm getting
good enough with git that I can fix most mess up with a few edits, but
it took me two years to get to that point. Rolling back to a check
point is way easier. User error and a command failing are both equally
valid ways to mess up a repo.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Sean]

On Tue, November 27, 2007 11:37 pm, Jon Smirl said:

> Patch management is an important part of the work flow. I would hope
> that git implements patch management as a core feature in future
> versions. stgit/guilt/quilt are valuable since they blazed the trail
> and figured out what commands are useful. As time passes these
> features can become more highly integrated into core git.

Think this is a separate topic from where we started though.

> Of course you've never screwed up a repository using git commands,
> right? I've messed up plenty. A good way to mess up a repo is to get
> the data in .git/* out of sync with what is in the repo. I'm getting
> good enough with git that I can fix most mess up with a few edits, but
> it took me two years to get to that point. Rolling back to a check
> point is way easier. User error and a command failing are both equally
> valid ways to mess up a repo.

What are you looking for that reflogs don't already handle?

Cheers,
Sean

Re: Rollback of git commands

[Jon Smirl]

On 11/27/07, Sean <seanlkml@sympatico.ca> wrote:
> On Tue, November 27, 2007 11:37 pm, Jon Smirl said:
>
> > Patch management is an important part of the work flow. I would hope
> > that git implements patch management as a core feature in future
> > versions. stgit/guilt/quilt are valuable since they blazed the trail
> > and figured out what commands are useful. As time passes these
> > features can become more highly integrated into core git.
>
> Think this is a separate topic from where we started though.
>
> > Of course you've never screwed up a repository using git commands,
> > right? I've messed up plenty. A good way to mess up a repo is to get
> > the data in .git/* out of sync with what is in the repo. I'm getting
> > good enough with git that I can fix most mess up with a few edits, but
> > it took me two years to get to that point. Rolling back to a check
> > point is way easier. User error and a command failing are both equally
> > valid ways to mess up a repo.
>
> What are you looking for that reflogs don't already handle?

A UI that doesn't need a year of using git before you know what to do with it.
Delta tracking of changes made in .git/* that aren't currently being tracked.
reflogs is a piece of the complete solution.

A higher level of integration for stgit would probably make it more
bullet proof.

>
> Cheers,
> Sean
>
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Nicolas Pitre]

On Tue, 27 Nov 2007, Jon Smirl wrote:

> Of course you've never screwed up a repository using git commands,
> right? I've messed up plenty. A good way to mess up a repo is to get
> the data in .git/* out of sync with what is in the repo. I'm getting
> good enough with git that I can fix most mess up with a few edits, but
> it took me two years to get to that point. Rolling back to a check
> point is way easier. User error and a command failing are both equally
> valid ways to mess up a repo.

The reflog contains all your check points, for every modifications you 
make, even the stupid ones.  You should look at it.


Nicolas

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> On Tue, 27 Nov 2007, Jon Smirl wrote:
>
> > Of course you've never screwed up a repository using git commands,
> > right? I've messed up plenty. A good way to mess up a repo is to get
> > the data in .git/* out of sync with what is in the repo. I'm getting
> > good enough with git that I can fix most mess up with a few edits, but
> > it took me two years to get to that point. Rolling back to a check
> > point is way easier. User error and a command failing are both equally
> > valid ways to mess up a repo.
>
> The reflog contains all your check points, for every modifications you
> make, even the stupid ones.  You should look at it.

The state contained in the other config files in .git/* is not getting
check pointed. I can use reflog to move my branch heads around. But
doing that does not undo the changes to the state recorded in .git/*.
After the error I encountered  I moved my branch head back, but the
state stgit had stored in .git/* was out of sync with where the branch
had been moved to.


>
>
> Nicolas
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Nicolas Pitre]

On Wed, 28 Nov 2007, Jon Smirl wrote:

> On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > On Tue, 27 Nov 2007, Jon Smirl wrote:
> >
> > > Of course you've never screwed up a repository using git commands,
> > > right? I've messed up plenty. A good way to mess up a repo is to get
> > > the data in .git/* out of sync with what is in the repo. I'm getting
> > > good enough with git that I can fix most mess up with a few edits, but
> > > it took me two years to get to that point. Rolling back to a check
> > > point is way easier. User error and a command failing are both equally
> > > valid ways to mess up a repo.
> >
> > The reflog contains all your check points, for every modifications you
> > make, even the stupid ones.  You should look at it.
> 
> The state contained in the other config files in .git/* is not getting
> check pointed. I can use reflog to move my branch heads around. But
> doing that does not undo the changes to the state recorded in .git/*.
> After the error I encountered  I moved my branch head back, but the
> state stgit had stored in .git/* was out of sync with where the branch
> had been moved to.

It's up to stgit to version control its state then.  It may even use a 
reflog for it.  All the machinery is there already.


Nicolas

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> On Wed, 28 Nov 2007, Jon Smirl wrote:
>
> > On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > > On Tue, 27 Nov 2007, Jon Smirl wrote:
> > >
> > > > Of course you've never screwed up a repository using git commands,
> > > > right? I've messed up plenty. A good way to mess up a repo is to get
> > > > the data in .git/* out of sync with what is in the repo. I'm getting
> > > > good enough with git that I can fix most mess up with a few edits, but
> > > > it took me two years to get to that point. Rolling back to a check
> > > > point is way easier. User error and a command failing are both equally
> > > > valid ways to mess up a repo.
> > >
> > > The reflog contains all your check points, for every modifications you
> > > make, even the stupid ones.  You should look at it.
> >
> > The state contained in the other config files in .git/* is not getting
> > check pointed. I can use reflog to move my branch heads around. But
> > doing that does not undo the changes to the state recorded in .git/*.
> > After the error I encountered  I moved my branch head back, but the
> > state stgit had stored in .git/* was out of sync with where the branch
> > had been moved to.
>
> It's up to stgit to version control its state then.  It may even use a
> reflog for it.  All the machinery is there already.

Git has state in .git/* too, shouldn't it be version controlling it
too? If git was version controlling the state in .git/* you'd have
checkpoints with the ability to roll back.


>
>
> Nicolas
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Nicolas Pitre]

On Wed, 28 Nov 2007, Jon Smirl wrote:

> On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > On Wed, 28 Nov 2007, Jon Smirl wrote:
> >
> > > On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > > > On Tue, 27 Nov 2007, Jon Smirl wrote:
> > > >
> > > > > Of course you've never screwed up a repository using git commands,
> > > > > right? I've messed up plenty. A good way to mess up a repo is to get
> > > > > the data in .git/* out of sync with what is in the repo. I'm getting
> > > > > good enough with git that I can fix most mess up with a few edits, but
> > > > > it took me two years to get to that point. Rolling back to a check
> > > > > point is way easier. User error and a command failing are both equally
> > > > > valid ways to mess up a repo.
> > > >
> > > > The reflog contains all your check points, for every modifications you
> > > > make, even the stupid ones.  You should look at it.
> > >
> > > The state contained in the other config files in .git/* is not getting
> > > check pointed. I can use reflog to move my branch heads around. But
> > > doing that does not undo the changes to the state recorded in .git/*.
> > > After the error I encountered  I moved my branch head back, but the
> > > state stgit had stored in .git/* was out of sync with where the branch
> > > had been moved to.
> >
> > It's up to stgit to version control its state then.  It may even use a
> > reflog for it.  All the machinery is there already.
> 
> Git has state in .git/* too, shouldn't it be version controlling it
> too? If git was version controlling the state in .git/* you'd have
> checkpoints with the ability to roll back.

Well, the .git directory contains data to identify what is actually 
version controlled.  If you start versioning the data used to implement 
the versioning, don't you get into endless recursion here?


Nicolas




> 
> 
> >
> >
> > Nicolas
> > -
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> 
> -- 
> Jon Smirl
> jonsmirl@gmail.com
> 


Nicolas

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> On Wed, 28 Nov 2007, Jon Smirl wrote:
>
> > On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > > On Wed, 28 Nov 2007, Jon Smirl wrote:
> > >
> > > > On 11/28/07, Nicolas Pitre <nico@cam.org> wrote:
> > > > > On Tue, 27 Nov 2007, Jon Smirl wrote:
> > > > >
> > > > > > Of course you've never screwed up a repository using git commands,
> > > > > > right? I've messed up plenty. A good way to mess up a repo is to get
> > > > > > the data in .git/* out of sync with what is in the repo. I'm getting
> > > > > > good enough with git that I can fix most mess up with a few edits, but
> > > > > > it took me two years to get to that point. Rolling back to a check
> > > > > > point is way easier. User error and a command failing are both equally
> > > > > > valid ways to mess up a repo.
> > > > >
> > > > > The reflog contains all your check points, for every modifications you
> > > > > make, even the stupid ones.  You should look at it.
> > > >
> > > > The state contained in the other config files in .git/* is not getting
> > > > check pointed. I can use reflog to move my branch heads around. But
> > > > doing that does not undo the changes to the state recorded in .git/*.
> > > > After the error I encountered  I moved my branch head back, but the
> > > > state stgit had stored in .git/* was out of sync with where the branch
> > > > had been moved to.
> > >
> > > It's up to stgit to version control its state then.  It may even use a
> > > reflog for it.  All the machinery is there already.
> >
> > Git has state in .git/* too, shouldn't it be version controlling it
> > too? If git was version controlling the state in .git/* you'd have
> > checkpoints with the ability to roll back.
>
> Well, the .git directory contains data to identify what is actually
> version controlled.  If you start versioning the data used to implement
> the versioning, don't you get into endless recursion here?

You would end up with a single non-version controlled sha pointer that
would need to be stored externally.  That pointer would be the
beginning of the rollback log. It's just a normal commit chain.

In a totally version controlled system. .git/* would only contain
objects and the one pointer to the current state of the system.


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Karl Hasselström]

On 2007-11-28 11:26:45 -0500, Nicolas Pitre wrote:

> On Wed, 28 Nov 2007, Jon Smirl wrote:
>
> > The state contained in the other config files in .git/* is not
> > getting check pointed. I can use reflog to move my branch heads
> > around. But doing that does not undo the changes to the state
> > recorded in .git/*. After the error I encountered I moved my
> > branch head back, but the state stgit had stored in .git/* was out
> > of sync with where the branch had been moved to.
>
> It's up to stgit to version control its state then. It may even use
> a reflog for it. All the machinery is there already.

I agree -- this is StGit's job.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Rollback of git commands

[Junio C Hamano]

"Sean" <seanlkml@sympatico.ca> writes:

>> After doing the commands I located my last commit before the rebase
>> and edited master back to  it. But my system was still messed up since
>> moving master got me out of sync with the state stg stored in .git/*.
>> The 'stg repair' command had changed the stored state.
>
>  From your description is seems that Git proper was able to handle the
> situation just fine.   It sounds instead like you're describing a problem
> with Stg where it became confused without a way to restore _its_ meta
> data.  There's not much Git itself can do to help in this situation
> unless Stg stores all of its meta-data as standard Git objects, rather
> than just using the .git directory.

Essentially, he does not want to use "git rebase" and have a way to
disable the command on a branch that stg is actively munging.  And that
is something git proper can help with the user, which is why I earlier
referred him to pre-rebase hook.

As you suggested, however, git proper should not know the internals of
Porcelains, so I'd rather not to have that logic deep in git-rebase
itself.  But the pre-rebase hook shoud be the appropriate place for the
user to actually populate with stg specific logic ("is stg set to
actively munge this branch") and activate it for his repository.

Re: Rollback of git commands

[Karl Hasselström]

On 2007-11-27 20:33:27 -0500, Jon Smirl wrote:

> Let's take my recent problem as an example. I typed 'git rebase
> linus/master' instead of 'stg rebase linus/master'. Then I typed
> 'stg repair'. The repair failed and left me in a mess. Both of these
> are easy to rollback except for the fact that stg has stored a bunch
> of state in .git/*.
>
> After doing the commands I located my last commit before the rebase
> and edited master back to it. But my system was still messed up
> since moving master got me out of sync with the state stg stored in
> .git/*. The 'stg repair' command had changed the stored state.

How exactly did repair mess up? Did it crash, produce a broken result,
an unreasonable but technically valid result, or just not the result
you wanted?

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Karl Hasselström <kha@treskal.com> wrote:
> On 2007-11-27 20:33:27 -0500, Jon Smirl wrote:
>
> > Let's take my recent problem as an example. I typed 'git rebase
> > linus/master' instead of 'stg rebase linus/master'. Then I typed
> > 'stg repair'. The repair failed and left me in a mess. Both of these
> > are easy to rollback except for the fact that stg has stored a bunch
> > of state in .git/*.
> >
> > After doing the commands I located my last commit before the rebase
> > and edited master back to it. But my system was still messed up
> > since moving master got me out of sync with the state stg stored in
> > .git/*. The 'stg repair' command had changed the stored state.
>
> How exactly did repair mess up? Did it crash, produce a broken result,
> an unreasonable but technically valid result, or just not the result
> you wanted?

all my patches applied
git rebase
cursing.... I immediately knew what I had done
update stg and install it
stg repair
four of my 15 patches tried to apply, I received messages that there
were all empty
most stg commands won't work, they complain that the commit references
in the stg .git/* state are not correct.

I then proceed to manually attempt repair.


>
> --
> Karl Hasselström, kha@treskal.com
>       www.treskal.com/kalle
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Daniel Barkalow]

On Wed, 28 Nov 2007, Jon Smirl wrote:

> all my patches applied
> git rebase
> cursing.... I immediately knew what I had done
> update stg and install it
> stg repair
> four of my 15 patches tried to apply, I received messages that there
> were all empty
> most stg commands won't work, they complain that the commit references
> in the stg .git/* state are not correct.
> 
> I then proceed to manually attempt repair.

This sounds like the content of the applied patches got pulled into the 
non-stgit history of the branch it's working on, sort of like a stg commit 
except that stgit didn't know you'd done it. Then cleaning everything up 
from stgit's perspective caused all of those patches to become empty, 
since they were already applied in the base.

I think fundamental issue you're having is that stgit is implementing the 
functionality of quilt using git's engine, not providing a version control 
system for patch series, which is what you really want. I've actually been 
working on a design for a git builtin for the idea that the patch series 
is your work product, and you want to version control that (additionally, 
you want to use git's engine to help with working on the series and 
represent it).

Out of curiousity, are you using stgit as an integrator (with your work 
being keeping a collection of patches produced separately up-to-date) or 
as a patch developer (with your work being producing a state containing a 
single large new feature while maintaining this change as a series of 
self-contained patches)? I've been thinking primarily about the integrator 
task, in part because I've found it easy enough to do the developer task 
without anything other than current git. (That is, "git rebase -i" seems 
to work fine for making changes to a single logical patch series, all of 
whose patches are prepared locally and aren't independantly named in some 
particular fashion; the things that aren't handled are "I need to replace 
the pull of netdev.git with a new pull of netdev.git" or "I need to 
replace '[PATCH] fix-the-frobnozzle-gadget' with 
'[PATCH v2] fix-the-frobnozzle-gadget'.)

The developer assist I'd actually like to see is: "I've got a single 
commit on top of a series of commits on top of an upstream commit; I want 
to distribute the changes made in the final commit to points in the series 
where the code that gets replaced (or context that gets inserted into) in 
the final commit gets introduced, with interactive stuff for sticking 
other hunks into particular commits or into new commits at some point in 
the series." That is, I want to do my revision of a patch series on the 
final commit of the series, and then have these changes distributed to the 
appropriate points, rather than doing work on intermediate states (unless 
what I'm fixing is stub code that gets replaced again in a later patch).

	-Daniel
*This .sig left intentionally blank*

Re: Rollback of git commands

[Steven Grimm]

On Nov 28, 2007, at 1:47 PM, Daniel Barkalow wrote:
> (That is, "git rebase -i" seems
> to work fine for making changes to a single logical patch series,  
> all of
> whose patches are prepared locally and aren't independantly named in  
> some
> particular fashion; the things that aren't handled are "I need to  
> replace
> the pull of netdev.git with a new pull of netdev.git" or "I need to
> replace '[PATCH] fix-the-frobnozzle-gadget' with
> '[PATCH v2] fix-the-frobnozzle-gadget'.)

I use rebase -i for that last case and it works fine -- I mark the  
appropriate commit as "edit" in the patch list and the rebase stops  
there, at which point I can update the patch in any way I see fit:  
tweak it a bit, replace it with a different change entirely, change  
the commit message, etc. What's missing from rebase -i in that  
respect? I guess it's not as easy to script for automated patch  
replacement.

-Steve

Re: Rollback of git commands

[Daniel Barkalow]

On Wed, 28 Nov 2007, Steven Grimm wrote:

> On Nov 28, 2007, at 1:47 PM, Daniel Barkalow wrote:
> >(That is, "git rebase -i" seems
> >to work fine for making changes to a single logical patch series, all of
> >whose patches are prepared locally and aren't independantly named in some
> >particular fashion; the things that aren't handled are "I need to replace
> >the pull of netdev.git with a new pull of netdev.git" or "I need to
> >replace '[PATCH] fix-the-frobnozzle-gadget' with
> >'[PATCH v2] fix-the-frobnozzle-gadget'.)
> 
> I use rebase -i for that last case and it works fine -- I mark the appropriate
> commit as "edit" in the patch list and the rebase stops there, at which point
> I can update the patch in any way I see fit: tweak it a bit, replace it with a
> different change entirely, change the commit message, etc. What's missing from
> rebase -i in that respect? I guess it's not as easy to script for automated
> patch replacement.

Just that you have to find the patch yourself and replace it; if you're 
doing this a lot, you'll want to say "here's a new version of 
fix-the-frobnozzle-gadget, do the right thing". Also, for series items 
that are pulling some remote tree, you want it to remember the info, so 
you can just say "pull git-netdev" and have it fetch the latest from the 
appropriate remote and replace the merge commit with a new merge commit.

I think that people in this role using quilt do it by: "quilt pop -a; 
replace the patch file without looking at the series file; quilt push 
-a"; in the (hopefully) common case, you don't have to worry about any of 
the details, which is why the task remains tractable.

	-Daniel
*This .sig left intentionally blank*

Re: Rollback of git commands

[Jon Smirl]

On 11/28/07, Daniel Barkalow <barkalow@iabervon.org> wrote:
> On Wed, 28 Nov 2007, Jon Smirl wrote:
>
> > all my patches applied
> > git rebase
> > cursing.... I immediately knew what I had done
> > update stg and install it
> > stg repair
> > four of my 15 patches tried to apply, I received messages that there
> > were all empty
> > most stg commands won't work, they complain that the commit references
> > in the stg .git/* state are not correct.
> >
> > I then proceed to manually attempt repair.
>
> This sounds like the content of the applied patches got pulled into the
> non-stgit history of the branch it's working on, sort of like a stg commit
> except that stgit didn't know you'd done it. Then cleaning everything up
> from stgit's perspective caused all of those patches to become empty,
> since they were already applied in the base.
>
> I think fundamental issue you're having is that stgit is implementing the
> functionality of quilt using git's engine, not providing a version control
> system for patch series, which is what you really want. I've actually been
> working on a design for a git builtin for the idea that the patch series
> is your work product, and you want to version control that (additionally,
> you want to use git's engine to help with working on the series and
> represent it).
>
> Out of curiousity, are you using stgit as an integrator (with your work
> being keeping a collection of patches produced separately up-to-date) or
> as a patch developer (with your work being producing a state containing a
> single large new feature while maintaining this change as a series of
> self-contained patches)? I've been thinking primarily about the integrator
> task, in part because I've found it easy enough to do the developer task
> without anything other than current git. (That is, "git rebase -i" seems
> to work fine for making changes to a single logical patch series, all of
> whose patches are prepared locally and aren't independantly named in some
> particular fashion; the things that aren't handled are "I need to replace
> the pull of netdev.git with a new pull of netdev.git" or "I need to
> replace '[PATCH] fix-the-frobnozzle-gadget' with
> '[PATCH v2] fix-the-frobnozzle-gadget'.)

I'm a patch developer. You need to change the patches continuously to
track feedback on the lkml type lists. You also have to rebase in
order to keep tracking head. Other people often work on the same
things and this triggers merges against the pending patches.

Another class of problem is that I can write code a lot faster than I
can get it into the kernel. Currently I have 14 pending PPC patches
that I'm maintaining while I try and get a core change into the i2c
subsystem. All of the other patches depend on the core i2c patch.

Of course the version of the i2c patch that finally gets accepted will
probably cause me to have to rework the whole patch stack again.

stgit is what you need for this work flow. It lets me easily rebase or
edit specific patches. It also lets me easily maintain private debug
patches that I can apply as needed.

I'd just like for stgit to become a core part of git so that is can be
made more bullet proof. I'm losing my patch stack every couple of
weeks. It's normally a "user error" but it is way to easy to make
these user errors.


>
> The developer assist I'd actually like to see is: "I've got a single
> commit on top of a series of commits on top of an upstream commit; I want
> to distribute the changes made in the final commit to points in the series
> where the code that gets replaced (or context that gets inserted into) in
> the final commit gets introduced, with interactive stuff for sticking
> other hunks into particular commits or into new commits at some point in
> the series." That is, I want to do my revision of a patch series on the
> final commit of the series, and then have these changes distributed to the
> appropriate points, rather than doing work on intermediate states (unless
> what I'm fixing is stub code that gets replaced again in a later patch).
>
>         -Daniel
> *This .sig left intentionally blank*
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Rollback of git commands

[Theodore Tso]

On Wed, Nov 28, 2007 at 06:42:56PM -0500, Jon Smirl wrote:
> I'm a patch developer. You need to change the patches continuously to
> track feedback on the lkml type lists. You also have to rebase in
> order to keep tracking head. Other people often work on the same
> things and this triggers merges against the pending patches.

Yeah, I use guilt for projects where I have a patch queue that takes a
while to merge into the mainline, due to quality control or review
guidelines that still need to be met.  Basically the kernel has a
different workflow than git, where each project or developer has a
separate patch queue that is maintained inside git (possibly with
guilt or stgit as helpers) or quilt, instead of a centrally managed pu
branch.  This is I suspect mainly a scaling issue; it's hard to keep a
centrally maintained pu branch because anytime someone wants to make a
change to one of their topic branches, they have to go through the
central maintainer.  The closest analogue to the pu branch is Andrew
Morton's akpm, and he gets grumpy when there are merge conflicts
between the various branches and patch queues that he pulls from in
order to make the -mm tree for integration testing.

> I'd just like for stgit to become a core part of git so that is can be
> made more bullet proof. I'm losing my patch stack every couple of
> weeks. It's normally a "user error" but it is way to easy to make
> these user errors.

I haven't tried using stgit in quite a while mainly because I was
finding it too easy to lose my patch queue.  It was happening to me
*way* to often, and while it sounds like it's gotten better due to
adding more sanity checks --- and when I tried it last, "stg repair"
didn't exist --- it sounds like it is still possible for someone who
typo's a command to get themselves into trouble.  I'm sure it's not an
issue for the stgit developers because they know which commands to
avoid, but if something isn't 100% bulletproof --- I get nervous.

So my solution to this problem is I use guilt.  I *like* the fact that
the primary storage mechanism is actual flat ASCII patch files, since
as a result I've never lost a patch stack --- ASCII patch files are
much more robust.  My way of rebasing a set of patches is very simple;
I just do "guilt pop -a; git merge master; guilt push -a" --- and if
there are any conflicts, then I do a "guilt push -f", fix up the
conflicts, follow it with a "guilt refresh", and then resume pushing
the rest of my patches sing "guilt push -a".  It just works; no muss,
no fuss, no dirty dishes.

Another thing this allows me to do is that it is possible to put the
guilt patch stack itself under git control.  This allows me to go
backwards in time and see what the patch stack looked like n days or n
weeks ago.  A comment in the patch header indicates what version of
the kernel the patch stack was meant against.

For example for ext4 development the guilt patch stack is themselves
stored using git on repo.or.cz, with multiple people having access to
push updates to the patch queue.  Before I start hacking the ext4,
I'll do run the command "(cd .git/patches/ext4dev; git pull") to check
for new updates in the ext4 patch queue, and if it has changed, I'll
do a "guilt pop -a; guilt push -a" to refresh the working directory
and ext4dev branch in my Linux git tree.  When I'm done making
changes, by using commands such as "guilt pop", "guilt refresh",
"guilt pop", et. al., I'll commit changes in the .git/patches/ext4dev
tree, and then push them back to repo.or.cz.  

We also have a system where an automated build/test server checks
every few hours if the patch queue on repo.or.cz has changed, and if
so, it pulls the patch queue, and then builds and tests the patch
queue on a wide variety of hardware architectures (i386, x86_64, ia64,
ppc64, s390, etc.) and sends an e-mail pack to the developers list
telling us which test hardware platforms passed, and which platforms
fail.  Actually, we do two such tests --- one up to the "stable" point
in the patch series, which represents the next set of patches we plan
to push to Linus during the next merge window, and one for the entire
patch queue (stable plus unstable patches that need more work).

It works quite well for us --- and while there are some advanced
feature that stgit has which guilt doesn't, the feeling of safety of
knowing that the ASCII patch files won't get lost, and the ability to
do things such as shared patch queue maintenance and ability to keep
the patch queue under SCM are enough of a win that at least for my
purposes guilt is the better choice.

> stgit is what you need for this work flow. It lets me easily rebase or
> edit specific patches. It also lets me easily maintain private debug
> patches that I can apply as needed.

As mentioned above, I use guilt for major patch queues.  For smaller
sets of patches, such as private debug patches, I just use a separate
git branch for those, and keep them up-to-date by using "git rebase"
and "git rebase -i".  Then when I want to enable some debug patch,
I'll just do a "git merge" in a scratch branch.

							- Ted