My use case

My use case

[Ron Garret]

My recent question about checking out an upstream commit (and ending up 
as a result on a detached head) has generated a lot of discussion and 
useful information.  I thought I'd describe what I'm actually up to 
because someone might have some ideas on how to do it better.

I'm trying to integrate git into a Lisp IDE (Clozure Common Lisp).  
Because it's Lisp, code is often developed incrementally and 
experimentally.  This has two important consequences.  First, the files 
in your working directory can and often do, as part of the normal course 
of events, get into an incoherent state.  Everything in your running 
Lisp image is working just fine, but if you try to recompile your entire 
system something will break, and you can go for a very long time without 
discovering that this has happened.  Second, development can often lead 
to dead-ends where you want to throw everything you've just done away, 
go back to some earlier version of *some but not all* of your code, and 
start over.

In other words, it is not uncommon to want to roll back an individual 
file or set of files to an earlier version and leave the rest of the 
tree alone.  git can do this, but it's not straightforward.  Simply 
rolling back through the history of the current branch doesn't work 
because you might want to roll back file A, but the last dozen revisions 
or so have been changes to file B.  You might also want to roll both A 
and B back to states which never co-existed in the original history.

One approach is to use git rev-list to find those commits where 
particular files changed, but this is sub-optimal for several reasons.  
First, a naive approach calls rev-list for every rollback, and rev-list 
has to traverse the entire history, so it's very inefficient.  Second, 
if you roll back a single file, git doesn't keep track of that file's 
provenance, so you have to manually track which files and have been 
rolled back and which revisions they have been rolled back to.  (There 
was a third problem but I can't think what it was right now.)

I have this intuition that git can be made to really do this right by 
keeping a separate history of every individual file in addition to a 
history of the entire source tree.  Git can't do this directly as far as 
I know.  I'd be writing additional code to generate extra tree and 
commit objects every time a file was saved from the IDE.  But turning 
this intuition into reality is turning out to be quite challenging.  So 
I'm going with the rev-list approach for the first version despite its 
shortcomings.

If anyone has ideas or suggestions, feedback would be much appreciated.  
But this is mostly just FYI.

rg

Re: My use case

[Edward Z. Yang]

Excerpts from Ron Garret's message of Sat Jan 30 03:26:19 -0500 2010:
> In other words, it is not uncommon to want to roll back an individual 
> file or set of files to an earlier version and leave the rest of the 
> tree alone.  git can do this, but it's not straightforward.  Simply 
> rolling back through the history of the current branch doesn't work 
> because you might want to roll back file A, but the last dozen revisions 
> or so have been changes to file B.  You might also want to roll both A 
> and B back to states which never co-existed in the original history.
> 
> One approach is to use git rev-list to find those commits where 
> particular files changed, but this is sub-optimal for several reasons.  
> First, a naive approach calls rev-list for every rollback, and rev-list 
> has to traverse the entire history, so it's very inefficient.  Second, 
> if you roll back a single file, git doesn't keep track of that file's 
> provenance, so you have to manually track which files and have been 
> rolled back and which revisions they have been rolled back to.  (There 
> was a third problem but I can't think what it was right now.)

My approach, in this case, would be to use git log, possibly git log -p,
in order to view the changes in each file you were interested in rolling
back.  If the rollback falls on a commit boundary, great; you can use
`git checkout rev -- path`.  If not, you can perform that, and then
use git checkout -p to selectively revert hunks in the patch (fundamentally,
you can't really get better than that).

> I have this intuition that git can be made to really do this right by 
> keeping a separate history of every individual file in addition to a 
> history of the entire source tree.  Git can't do this directly as far as 
> I know.  I'd be writing additional code to generate extra tree and 
> commit objects every time a file was saved from the IDE.  But turning 
> this intuition into reality is turning out to be quite challenging.  So 
> I'm going with the rev-list approach for the first version despite its 
> shortcomings.

While Git's ability to look at individual file's history is "slower", it's still
quite excellent, and you are encouraged to use it as necessary.

Slightly relatedly, I'd recommend flushing to disk and committing more
often.  It's great that the LISP REPL allows for more daring changes,
but having a history either way is very helpful!

Cheers,
Edward

Re: My use case

[Ron Garret]

In article <1264840729-sup-5264@ezyang>,
 "Edward Z. Yang" <ezyang@MIT.EDU> wrote:

> Excerpts from Ron Garret's message of Sat Jan 30 03:26:19 -0500 2010:
> > In other words, it is not uncommon to want to roll back an individual 
> > file or set of files to an earlier version and leave the rest of the 
> > tree alone.  git can do this, but it's not straightforward.  Simply 
> > rolling back through the history of the current branch doesn't work 
> > because you might want to roll back file A, but the last dozen revisions 
> > or so have been changes to file B.  You might also want to roll both A 
> > and B back to states which never co-existed in the original history.
> > 
> > One approach is to use git rev-list to find those commits where 
> > particular files changed, but this is sub-optimal for several reasons.  
> > First, a naive approach calls rev-list for every rollback, and rev-list 
> > has to traverse the entire history, so it's very inefficient.  Second, 
> > if you roll back a single file, git doesn't keep track of that file's 
> > provenance, so you have to manually track which files and have been 
> > rolled back and which revisions they have been rolled back to.  (There 
> > was a third problem but I can't think what it was right now.)
> 
> My approach, in this case, would be to use git log, possibly git log -p,
> in order to view the changes in each file you were interested in rolling
> back.  If the rollback falls on a commit boundary, great; you can use
> `git checkout rev -- path`.  If not, you can perform that, and then
> use git checkout -p to selectively revert hunks in the patch (fundamentally,
> you can't really get better than that).

Don't forget, I'm integrating this *into* the IDE, not just using it 
*for* the IDE.  So I want to just have a context menu on each code 
window with "SNAPSHOT" and "ROLLBACK" items that Just Work.  The casual 
user won't even know that there's git behind the scenes.

> > I have this intuition that git can be made to really do this right by 
> > keeping a separate history of every individual file in addition to a 
> > history of the entire source tree.  Git can't do this directly as far as 
> > I know.  I'd be writing additional code to generate extra tree and 
> > commit objects every time a file was saved from the IDE.  But turning 
> > this intuition into reality is turning out to be quite challenging.  So 
> > I'm going with the rev-list approach for the first version despite its 
> > shortcomings.
> 
> While Git's ability to look at individual file's history is "slower", it's 
> still
> quite excellent, and you are encouraged to use it as necessary.

Good to know.

> Slightly relatedly, I'd recommend flushing to disk and committing more
> often.  It's great that the LISP REPL allows for more daring changes,
> but having a history either way is very helpful!

Yep!  Save early, save often.  I'm actually considering an auto-save 
option where it takes a snapshot every time you evaluate a form after 
making a change.

rg

Re: My use case

[tytso]

On Sat, Jan 30, 2010 at 12:54:13AM -0800, Ron Garret wrote:
> Don't forget, I'm integrating this *into* the IDE, not just using it 
> *for* the IDE.  So I want to just have a context menu on each code 
> window with "SNAPSHOT" and "ROLLBACK" items that Just Work.  The casual 
> user won't even know that there's git behind the scenes.

This is a workflow question, I suppose, but I find things work much
better if you can get the user to give you explicit commit boundaries
so that (a) bisect works, and (b) they can describe what each commit
does, and (c) so they can more easily move specific bug fixes or
features between different release branches.  The free-form hacking
more may be nice, and very "LISP-like", but there are some real
advantages to having explicitly describable and documented commits.

Best regards,

					- Ted

Re: My use case

[Ron Garret]

In article <20100130174844.GD788@thunk.org>, tytso@mit.edu wrote:

> On Sat, Jan 30, 2010 at 12:54:13AM -0800, Ron Garret wrote:
> > Don't forget, I'm integrating this *into* the IDE, not just using it 
> > *for* the IDE.  So I want to just have a context menu on each code 
> > window with "SNAPSHOT" and "ROLLBACK" items that Just Work.  The casual 
> > user won't even know that there's git behind the scenes.
> 
> This is a workflow question, I suppose, but I find things work much
> better if you can get the user to give you explicit commit boundaries
> so that (a) bisect works, and (b) they can describe what each commit
> does, and (c) so they can more easily move specific bug fixes or
> features between different release branches.  The free-form hacking
> more may be nice, and very "LISP-like", but there are some real
> advantages to having explicitly describable and documented commits.

You are absolutely right.  That is another reason why having the 
individual files tracked separately from the main project would be a 
good thing if I can get it to work.  (It would be kind of like having a 
git-stash on a per-file basis.)

rg

Re: My use case

[Junio C Hamano]

Ron Garret <ron1@flownet.com> writes:

> You are absolutely right.  That is another reason why having the 
> individual files tracked separately from the main project would be a 
> good thing if I can get it to work.  (It would be kind of like having a 
> git-stash on a per-file basis.)

When you have more than one functions defined in a file, and the
interactive Lisp development cycle works primarily on s-exp basis, not
necessarily constrained by file boundaries, don't you want even finer
grained control than "stash per-file"?

I don't think of a good solution myself, but I find your "finer than whole
tree" an interesting topic.

Re: My use case

[Ron Garret]

In article <7vtyu3o1hq.fsf@alter.siamese.dyndns.org>,
 Junio C Hamano <gitster@pobox.com> wrote:

> Ron Garret <ron1@flownet.com> writes:
> 
> > You are absolutely right.  That is another reason why having the 
> > individual files tracked separately from the main project would be a 
> > good thing if I can get it to work.  (It would be kind of like having a 
> > git-stash on a per-file basis.)
> 
> When you have more than one functions defined in a file, and the
> interactive Lisp development cycle works primarily on s-exp basis, not
> necessarily constrained by file boundaries, don't you want even finer
> grained control than "stash per-file"?
> 
> I don't think of a good solution myself, but I find your "finer than whole
> tree" an interesting topic.

You're reading my mind.  My long-term vision for this thing is indeed a 
revision control system based on S-expressions, not files.  (I 
personally think the whole concept of files is fundamentally broken, but 
that's a whole 'nuther kettle of lobster.)  Unfortunately, that opens up 
a Pandora's box of ancillary issues, which would be seriously off-topic 
here.  If you're interested we can talk about it off-line (or we can do 
it here if you think it's not too much of a tangent).

rg