Merging commits together into a super-commit

Merging commits together into a super-commit

[Alex Bennee]

Hi,

I really love the fact I can micro-commit changes when I'm developing.
However at some point the combination of changes I have made can be
considered a single body of work. This is especially true when you start
doing things like re-basing on code that has moved around a lot. You
don't want to be correcting a whole bunch of merge failures for every
commit in your current tree.

So far the only was I can see to do this is a:

git-diff master..HEAD > my.patch

And then re-applying your patch in stages, manually doing the commits.

Am I missing something?

I'm thinking something like git-cherrypick taking multiple commits and
create a new super commit on a new tree. i.e.:

git-cherrypick -m "Valgrind fixes" 12345.. 12678.. 565757..

Merging the existing commit comments would be nice too.

-- 
Alex, homepage: http://www.bennee.com/~alex/
All God's children are not beautiful. Most of God's children are, in
fact, barely presentable. -- Fran Lebowitz, "Metropolitan Life"

RE: Merging commits together into a super-commit

[Raimund Bauer]

> Hi,
> 
> I really love the fact I can micro-commit changes when I'm 
> developing. However at some point the combination of changes 
> I have made can be considered a single body of work. This is 
> especially true when you start doing things like re-basing on 
> code that has moved around a lot. You don't want to be 
> correcting a whole bunch of merge failures for every commit 
> in your current tree.
> 
> So far the only was I can see to do this is a:
> 
> git-diff master..HEAD > my.patch
> 
> And then re-applying your patch in stages, manually doing the commits.
> 
> Am I missing something?

git merge --squash ?

-- 
best regards

  Ray

RE: Merging commits together into a super-commit

[Alex Bennee]

On Thu, 2007-05-10 at 13:19 +0200, Raimund Bauer wrote:
> > Hi,
>  <snip> 
> You don't want to be 
> > correcting a whole bunch of merge failures for every commit 
> > in your current tree.
> > 
> > So far the only was I can see to do this is a:
> > 
> > git-diff master..HEAD > my.patch
> > 
> > And then re-applying your patch in stages, manually doing the commits.
> > 
> > Am I missing something?
> 
> git merge --squash ?

Hmm, that would do although it only works on whole trees, you can't
specify a range of commits. So you would have to cherrypick groups of
changes onto other branches and then merge them together with a couple
of --squashes

However thanks for pointing that one out :-)

-- 
Alex, homepage: http://www.bennee.com/~alex/
A pretty woman can do anything; an ugly woman must do everything.

Re: Merging commits together into a super-commit

[Johannes Sixt]

Alex Bennee wrote:
> I really love the fact I can micro-commit changes when I'm developing.
> However at some point the combination of changes I have made can be
> considered a single body of work. This is especially true when you start
> doing things like re-basing on code that has moved around a lot. You
> don't want to be correcting a whole bunch of merge failures for every
> commit in your current tree.
> 
> So far the only was I can see to do this is a:
> 
> git-diff master..HEAD > my.patch
> 
> And then re-applying your patch in stages, manually doing the commits.
> 
> Am I missing something?
> 
> I'm thinking something like git-cherrypick taking multiple commits and
> create a new super commit on a new tree. i.e.:
> 
> git-cherrypick -m "Valgrind fixes" 12345.. 12678.. 565757..
> 
> Merging the existing commit comments would be nice too.

Here we go:

- cherry-pick them before commit

  $ git cherry-pick -n x
  $ git cherry-pick -n y
  $ git cherry-pick -n z
  $ git commit -m "$(for c in x y z; do git show --stat $c; done)" -e

- merge in a single commit

  $ git merge --squash foo

You didn't really think that git couldn't do that, did you? ;)

-- Hannes

RE: Merging commits together into a super-commit

[Johannes Schindelin]

Hi,

On Thu, 10 May 2007, Alex Bennee wrote:

> On Thu, 2007-05-10 at 13:19 +0200, Raimund Bauer wrote:
> > > Hi,
> >  <snip> 
> > You don't want to be 
> > > correcting a whole bunch of merge failures for every commit 
> > > in your current tree.
> > > 
> > > So far the only was I can see to do this is a:
> > > 
> > > git-diff master..HEAD > my.patch
> > > 
> > > And then re-applying your patch in stages, manually doing the commits.
> > > 
> > > Am I missing something?
> > 
> > git merge --squash ?
> 
> Hmm, that would do although it only works on whole trees, you can't
> specify a range of commits. So you would have to cherrypick groups of
> changes onto other branches and then merge them together with a couple
> of --squashes

Since specifying several commit ranges can always lead to merge conflicts, 
it makes no sense to _not_ accumulate the patches in a separate branch.

Ciao,
Dscho

Re: Merging commits together into a super-commit

[Linus Torvalds]

On Thu, 10 May 2007, Johannes Sixt wrote:
>
> - cherry-pick them before commit
> 
>   $ git cherry-pick -n x
>   $ git cherry-pick -n y
>   $ git cherry-pick -n z

I've done this (actually, mostly with "revert", but cherry-pick and revert 
are literally the same things).

However:

>   $ git commit -m "$(for c in x y z; do git show --stat $c; done)" -e

I'm too lazy to do this part, so I always do it by hand.

> You didn't really think that git couldn't do that, did you? ;)

Clearly git can, but equally clearly it really *would* be pretty nice if 
you could just do

	git cherry-pick x y z

and create one commit and have the message already somewhat done for you 
(and "git revert" doing the same).

So if somebody does that, I'll certainly applaud..

		Linus

Re: Merging commits together into a super-commit

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1932 bytes --]

On Thu, 10 May 2007 09:01:15 -0700 (PDT), Linus Torvalds wrote:
> Clearly git can, but equally clearly it really *would* be pretty nice if
> you could just do
>
> 	git cherry-pick x y z
>
> and create one commit and have the message already somewhat done for you
> (and "git revert" doing the same).

Perhaps with a --squash option. I've already been wanting a
cherry-pick that accepts a range, but that makes separate commits.

Yes, I know there's some git-rebase thing that will do it, but I can
never remember the right syntax for that without studying the
documentation[*]. What I find myself wanting to type is just:

	git cherry-pick A..B

But there is the whole problem of how to deal with any conflict that
appears during the process.

-Carl

[*] I do use one form of rebase without ever needing to consult the
documentation, but it's in the opposite "direction", if you will, from
the cherry-pick-a-range operation. I use it when I want to rebase a
set of commits from my current branch to some other branch, such as:

	git rebase origin

Not surprisingly, what's common about both of the operations above is
that they are really only accepting a single argument, (a range in one
case, and a branch-name in the other). That's what makes for an
easy-to-use command. Things that require multiple branch names in a
particular order, or extra options, (see "git rebase --onto newbase
upstream branch"), need someone smarter than me to drive them.

Another problem is that using the optional final [<branch>] argument
to git-rebase makes it change the current branch, (which no other
git-rebase operation does), and that's another thing I find very
confusing.

I'm sure the most complex form of git-rebase solves some precise
problem that someone has, (and maybe even gets used regularly). But
it's got enough complications that I just ignore it, (and would
instead really prefer being able to just cherry-pick a whole range).


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Merging commits together into a super-commit

[J. Bruce Fields]

On Thu, May 10, 2007 at 09:57:10AM -0700, Carl Worth wrote:
> I'm sure the most complex form of git-rebase solves some precise
> problem that someone has, (and maybe even gets used regularly). But
> it's got enough complications that I just ignore it, (and would
> instead really prefer being able to just cherry-pick a whole range).

I use it all the time to fix up old commits with

	git checkout sha1-of-bad-commit
	...edit, test,...
	git commit -a ---amend
	git rebase --onto HEAD sha1-of-bad-commit original-branch

But though it usually does what I want, I'm in total agreement about the
confusing syntax and branch-switching behavior....

--b.

Re: Merging commits together into a super-commit

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2560 bytes --]

On Thu, 10 May 2007 13:14:57 -0400, "J. Bruce Fields" wrote:
> I use it all the time to fix up old commits with
>
> 	git checkout sha1-of-bad-commit
> 	...edit, test,...
> 	git commit -a ---amend
> 	git rebase --onto HEAD sha1-of-bad-commit original-branch

Excellent. Thanks for the description.

Fixing up some broken (unpublished) commit in the past is a useful
thing to support[*]. Of course, "commit --amend" takes care of this for
the case where it's the most recent commit. For an older commit, here
is exactly what I'd like to do, (showing that my suggestion of
cherry-pick with a range would cover exactly this scenario):

	git tag tainted		# Could be "git branch tainted" just as well

	git reset --hard bad-commit
	...edit, test...
	git commit --amend
	git cherry-pick bad-commit..tainted

	git tag -d tainted

So, compared to the rebase usage, this does add two commands for
bookkeeping the original state and cleaning it up. But the syntax for
the cherry-pick part is quite a bit simpler than the original rebase
at least.

Also, "reset --hard" isn't actually what I want in this case. I'd like
this recipe to use something that would move the current branch to
some other point, but in a safe way, (that is, not destroy any
uncommitted changes that might exist at the beginning). I don't have
any proposal for what that would be.

-Carl

[*] I've also experimented with using other "non-core git" ways of
addressing this same problem. Here are a couple that I haven't found
satisfactory for this particular use case:

stg - This probably works great if you're using it as a primary
      interface. But trying to use it as a quick one-off when
      generally using core git does not work well at all. Instead of
      the two "git tag" commands in my recipe above, an stg recipe
      would involve a lot of additional bookkeeping with stg init, stg
      uncommit [N times for fixing a commit N steps back in the
      history], stg goto, stg push, etc.

cg-admin-rewritehist - This is a powerful tool, but its interface
      isn't geared toward interactively fixing things up, (instead
      requiring a filter to be executed at all steps). So, while it's
      great for changing history, (as its name suggests), such as
      performing a sed operation on the commit messages, or
      eliminating a file, it doesn't help much with the "fix up one
      broken commit in the past".

      It is worth noting that quite unlike rebase,
      cg-admin-rewritehist can deal quite nicely with history that
      involves branching and merging.


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Merging commits together into a super-commit

[Petr Baudis]

On Thu, May 10, 2007 at 08:30:37PM CEST, Carl Worth wrote:
> stg - This probably works great if you're using it as a primary
>       interface. But trying to use it as a quick one-off when
>       generally using core git does not work well at all. Instead of
>       the two "git tag" commands in my recipe above, an stg recipe
>       would involve a lot of additional bookkeeping with stg init, stg
>       uncommit [N times for fixing a commit N steps back in the
>       history], stg goto, stg push, etc.

I think you are underestimating stg here. You can stg init just once per
branch (ever), I think. Then,

	stg uncommit -n N
	stg pop -n N-1
	..hack..
	stg refresh
	stg push -a

It seems to be a bit shorter than the sequence you've presented above,
and overally working with volatile commits using StGIT feels much more
natural to me - and I haven't even ever used quilt seriously! (I have
special antipathy to the git reset UI, too.)

Few days ago Santi Bejar has sent me a bundle with some updates to the
Git homepage (thanks again a lot!). Since I didn't want some of the
patches and wanted to tweak others, what I eventually did was pretty
much this: I fast-forwarded my master to his bundle's head, then
uncommitted the patches, popped them all and repeated the sequence

	stg push
	..review..
		stg refresh
		stg commit
	..or..
		stg delete `stg top`

for each patch.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: Merging commits together into a super-commit

[J. Bruce Fields]

On Thu, May 10, 2007 at 11:30:37AM -0700, Carl Worth wrote:
> So, compared to the rebase usage, this does add two commands for
> bookkeeping the original state and cleaning it up. But the syntax for
> the cherry-pick part is quite a bit simpler than the original rebase
> at least.

Yeah, something like that would be great.  I think I've seen others
suggest similar syntax before, so it's probably just a question of one
of us who want this finding time to write the patches.

> Also, "reset --hard" isn't actually what I want in this case. I'd like
> this recipe to use something that would move the current branch to
> some other point, but in a safe way, (that is, not destroy any
> uncommitted changes that might exist at the beginning). I don't have
> any proposal for what that would be.

The tag creation and cleanup could get to be annoying too.  You could
scrounge through the reflog instead of using a temporary tag, but
depending on the amount of --amend'ing and cherry-picking you do the
reflog entry may end up in a different place each time, so it's probably
hard to make this automatic.

> stg - This probably works great if you're using it as a primary
>       interface. But trying to use it as a quick one-off when
>       generally using core git does not work well at all. Instead of
>       the two "git tag" commands in my recipe above, an stg recipe
>       would involve a lot of additional bookkeeping with stg init, stg
>       uncommit [N times for fixing a commit N steps back in the
>       history], stg goto, stg push, etc.

I also didn't like having to come up with another name for each
patch--I'd rather just run git-log or gitk and cut-n-paste the sha1.

For kernel work I started out working with multiple (sym- or
hard-linked) trees, then used akpm's patch scripts, then stgit.  I think
I'm happiest just using plain git.

The one thing I've never been good at is keeping the history of the
patch series itself.

--b.

Re: Merging commits together into a super-commit

[Petr Baudis]

On Thu, May 10, 2007 at 09:22:12PM CEST, J. Bruce Fields wrote:
> On Thu, May 10, 2007 at 11:30:37AM -0700, Carl Worth wrote:
> > stg - This probably works great if you're using it as a primary
> >       interface. But trying to use it as a quick one-off when
> >       generally using core git does not work well at all. Instead of
> >       the two "git tag" commands in my recipe above, an stg recipe
> >       would involve a lot of additional bookkeeping with stg init, stg
> >       uncommit [N times for fixing a commit N steps back in the
> >       history], stg goto, stg push, etc.
> 
> I also didn't like having to come up with another name for each
> patch--I'd rather just run git-log or gitk and cut-n-paste the sha1.

Actually, you don't have to - if you don't specify the patch names,
stgit will make them up itself using the subject of the commit message
as a base.

And by the way, I absolutely love that - when viewing the stack, it's
very useful to see what commits you still have to go etc. - stg series
is concise yet fully descriptive. I'm pondering about whether something
like this couldn't be incorporated into other git UIs somehow as well.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: Merging commits together into a super-commit

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2310 bytes --]

On Thu, 10 May 2007 21:21:06 +0200, Petr Baudis wrote:
> I think you are underestimating stg here.

Yes, maybe I didn't learn to use it well enough.

> You can stg init just once per branch (ever), I think.

I don't have details now, but I know I ran into some difficulty when
leaving the extra stg state around. It seems that it added stuff that
resulted in some reference of mine becoming ambiguous, ("refspec <foo>
matches more than one" perhaps?). What I do remember is that I couldn't
get one of my standard git push commands to work until I deleted all
of .git/refs/bases and .git/refs/patches and then things started to
work again.

> 	stg uncommit -n N
> 	stg pop -n N-1
> 	..hack..
> 	stg refresh
> 	stg push -a
>
> It seems to be a bit shorter than the sequence you've presented above,
> and overally working with volatile commits using StGIT feels much more
> natural to me - and I haven't even ever used quilt seriously! (I have
> special antipathy to the git reset UI, too.)

The -n option is something I hadn't noticed, and that helps, (except
that what I've got to start with is a git revision name, not a
number).

But there are still some places where an experienced git user runs
into some awkward situations trying to use stg. For example, "stg
refresh" is basically always doing the equivalent of "commit -a" so
there's annoyingly no way to refresh only some of the modified state
into the commit.

Also, if I want to edit a commit message while under the influence of
stg, how do I do that? If I do "git commit --amend" will I seriously
confuse stg, (I'm guessing I would, but I don't know).

It's that kind of uncertainty that makes me uncomfortable to mix git
and stg. And personally, I couldn't get excited about using it alone,
(for example, in addition to the commit message with headline, stg
makes me invent yet _another_ name for every commit---yuck). Not to
mention I'm already quite comfortable with git alone, and all the
flexibility it provides.

Plus, all the stuff that stg provides to allow it to be used
standalone ends up just being noise to the git user that just wants to
do some stack-based manipulation of an unpublished branch, for
example.

So, I'd really like to see something more integrated into git itself
that provides some of the missing functionality.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Merging commits together into a super-commit

[J. Bruce Fields]

On Thu, May 10, 2007 at 09:47:42PM +0200, Petr Baudis wrote:
> Actually, you don't have to - if you don't specify the patch names,
> stgit will make them up itself using the subject of the commit message
> as a base.

You still have to on new patches, right?

> And by the way, I absolutely love that - when viewing the stack, it's
> very useful to see what commits you still have to go etc. - stg series
> is concise yet fully descriptive.

Sure.

I mainly find myself using gitk origin.. for that now.  My main problem
there is just that it's slow.  (And File->Update) seems possibly even
slower than just killing and restarting it.--b.

Using StGIT for tweaking already-committed stuff

[Petr Baudis]

On Thu, May 10, 2007 at 09:48:05PM CEST, Carl Worth wrote:
> On Thu, 10 May 2007 21:21:06 +0200, Petr Baudis wrote:
> > 	stg uncommit -n N
> > 	stg pop -n N-1
> > 	..hack..
> > 	stg refresh
> > 	stg push -a
> >
> > It seems to be a bit shorter than the sequence you've presented above,
> > and overally working with volatile commits using StGIT feels much more
> > natural to me - and I haven't even ever used quilt seriously! (I have
> > special antipathy to the git reset UI, too.)
> 
> The -n option is something I hadn't noticed, and that helps, (except
> that what I've got to start with is a git revision name, not a
> number).

Hmm, yes, I've been thinking myself that it would be quite nice if I
could just tell uncommit git revname right away.

> But there are still some places where an experienced git user runs
> into some awkward situations trying to use stg. For example, "stg
> refresh" is basically always doing the equivalent of "commit -a" so
> there's annoyingly no way to refresh only some of the modified state
> into the commit.

Yes, I fear that StGIT hides the index in a similar way that Cogito
does. It seems like user index usage is undergoing kind of renaissance
these days in Git community (at least it seems to me this way, maybe
it's always been this way), it would probably make sense to allow making
use of index in StGIT as well.

> Also, if I want to edit a commit message while under the influence of
> stg, how do I do that? If I do "git commit --amend" will I seriously
> confuse stg, (I'm guessing I would, but I don't know).

I have no idea, but there's stg refresh -e.

> It's that kind of uncertainty that makes me uncomfortable to mix git
> and stg. And personally, I couldn't get excited about using it alone,
> (for example, in addition to the commit message with headline, stg
> makes me invent yet _another_ name for every commit---yuck). Not to
> mention I'm already quite comfortable with git alone, and all the
> flexibility it provides.

I wouldn't normally use it for projects I have commit access to myself,
but for maintaining own patches for an "external" project, I just find
it much more comfortable than using git. But then again, if this part of
git UI improved as much as some of the other parts in the last half a
year...

And yes, it would be cool if stg new could guess patch name from the
subject line in a similar manner that stg uncommit does.

> Plus, all the stuff that stg provides to allow it to be used
> standalone ends up just being noise to the git user that just wants to
> do some stack-based manipulation of an unpublished branch, for
> example.

I'm sorry, I couldn't parse this. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: Merging commits together into a super-commit

[Robin Rosenberg]

torsdag 10 maj 2007 skrev Carl Worth:
> But there are still some places where an experienced git user runs
> into some awkward situations trying to use stg. For example, "stg
> refresh" is basically always doing the equivalent of "commit -a" so
> there's annoyingly no way to refresh only some of the modified state
> into the commit.

List the files to refresh and you get what you want.

	stg refresh file1 file2...

> Also, if I want to edit a commit message while under the influence of
> stg, how do I do that? If I do "git commit --amend" will I seriously
> confuse stg, (I'm guessing I would, but I don't know).

	stg refresh -e

[...]
> Plus, all the stuff that stg provides to allow it to be used
> standalone ends up just being noise to the git user that just wants to
> do some stack-based manipulation of an unpublished branch, for
> example.
>
> So, I'd really like to see something more integrated into git itself
> that provides some of the missing functionality.

I agree mixing stgit and git is not really comfy until you learn it and still
I mess things up somtimes. Also rebase seems quite a bit faster than stgit, but I
have not intuitive understanding for rebase so I get scared everytime. Having
a gui that lets me mark commits and "drag" them to the new location would
be nice.

-- robin

Re: Using StGIT for tweaking already-committed stuff

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1684 bytes --]

On Thu, 10 May 2007 22:02:53 +0200, Petr Baudis wrote:
>
> I'm sorry, I couldn't parse this. :-)
>

I'll try again.

I like the git user interface. I like it a lot. (It's got a couple of
tiny things that I would do differently if I could start over, but
more importantly it has a lot of big things that I wouldn't have even
thought of if I had started from scratch.)

But with respect to the current topic, there are a couple of features
that the git interface is missing compared to something like stg:

1. Amend a commit that's somewhere besides the tip of a branch,
   (rebuilding every commit that follows)

2. Re-ordering commits that exist on a branch, (again, rebuilding
   every commit that follows).

And what I was trying to say in my confusing paragraph, is that if I
look to stg to add one or both pieces of this functionality, then it
comes with a lot of baggage. For example, "stg --help" lists about 38
sub-commands. And some of those are wholly unnecessary if already
using git, (4 repository commands 6 working-copy commands, for
example). While others exist only to allow a notion of "git commits"
vs. "stg commits" and translating back and forth between them,
(assimilate and uncommit for example).

Now, that's not a critique of stg itself. As you say, it can work
really well if you use it in a standalone fashion to track some
project.

I'd just love to see something more minimal, and incorporated into git
itself, to address the missing functionality. Right now, "cherry-pick
A..B" is all I have to suggest. But maybe later there could be some
sort of push/pop addition as well, (except that obviously the name
"push" isn't available as a sub-command).

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Using StGIT for tweaking already-committed stuff

[Karl Hasselström]

On 2007-05-10 22:02:53 +0200, Petr Baudis wrote:

> On Thu, May 10, 2007 at 09:48:05PM CEST, Carl Worth wrote:
>
> > The -n option is something I hadn't noticed, and that helps,
> > (except that what I've got to start with is a git revision name,
> > not a number).
>
> Hmm, yes, I've been thinking myself that it would be quite nice if I
> could just tell uncommit git revname right away.

It shouldn't be hard to do. Instead of uncommitting a fixed number of
commits, loop until you reach a specified commit.

> > But there are still some places where an experienced git user runs
> > into some awkward situations trying to use stg. For example, "stg
> > refresh" is basically always doing the equivalent of "commit -a"
> > so there's annoyingly no way to refresh only some of the modified
> > state into the commit.
>
> Yes, I fear that StGIT hides the index in a similar way that Cogito
> does. It seems like user index usage is undergoing kind of
> renaissance these days in Git community (at least it seems to me
> this way, maybe it's always been this way), it would probably make
> sense to allow making use of index in StGIT as well.

I agree. It's bad UI for StGIT to behave different from git, given
that easy interoperation is a goal.

> > Also, if I want to edit a commit message while under the influence
> > of stg, how do I do that? If I do "git commit --amend" will I
> > seriously confuse stg, (I'm guessing I would, but I don't know).
>
> I have no idea, but there's stg refresh -e.

Yes, you would confuse it; the patch ref would still point to the old
commit object, but that would no longer be an ancestor of HEAD. StGIT
doesn't know how to recover from this, so you'd have to do it by hand,
which is annoying.

I'm working on making StGIT more robust against this kind of damage,
but I'm not quite done yet.

> And yes, it would be cool if stg new could guess patch name from the
> subject line in a similar manner that stg uncommit does.

Good idea. This would be embarrassingly easy to do.

But you can kind of do it today. Just commit with git (my favorite
here is the emacs modes) and "stg assimilate"!

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Integrate StGIT into Git? (Was: Re: Using StGIT for tweaking already-committed stuff)

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

Hello Folks,

On Thu, May 10, 2007 at 14:16:43 -0700, Carl Worth wrote:
> I'll try again.
> 
> I like the git user interface. I like it a lot. (It's got a couple of
> tiny things that I would do differently if I could start over, but
> more importantly it has a lot of big things that I wouldn't have even
> thought of if I had started from scratch.)
> 
> But with respect to the current topic, there are a couple of features
> that the git interface is missing compared to something like stg:
> 
> 1. Amend a commit that's somewhere besides the tip of a branch,
>    (rebuilding every commit that follows)
> 
> 2. Re-ordering commits that exist on a branch, (again, rebuilding
>    every commit that follows).

I would actually propose to (gradually) add stg functionality into git. If it
was done in stgit-compatible fashion, it would allow using stgit for the bits
still not ported to git and switching back and forth according to user's
taste.

Many commands from stgit either already have git equivalent or do just
a little work beyond what the git command already does, so they could be
easily integrated.

> [...]
> I'd just love to see something more minimal, and incorporated into git
> itself, to address the missing functionality. Right now, "cherry-pick
> A..B" is all I have to suggest. But maybe later there could be some
> sort of push/pop addition as well, (except that obviously the name
> "push" isn't available as a sub-command).

I think that many of them would be actually pretty simple.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Using StGIT for tweaking already-committed stuff

[Yann Dirson]

On Fri, May 11, 2007 at 12:23:47AM +0200, Karl Hasselström wrote:
> On 2007-05-10 22:02:53 +0200, Petr Baudis wrote:
> > Yes, I fear that StGIT hides the index in a similar way that Cogito
> > does. It seems like user index usage is undergoing kind of
> > renaissance these days in Git community (at least it seems to me
> > this way, maybe it's always been this way), it would probably make
> > sense to allow making use of index in StGIT as well.
> 
> I agree. It's bad UI for StGIT to behave different from git, given
> that easy interoperation is a goal.

Well, that's an idea that already appeared in some discussions - I
can't speak for Catalin, but I too think it could be a good thing.

Eg, if we're going to use the patchlogs a bit more (and I wish so), it
will be much less cluttered by using the index to select what to
commit with several git-add's, than when using several stg-refresh's.

As noted elsewhere, there are some commands that are a bit superfluous
(add, rm, and the branch-switching feature directly come to mind).  It
is especially annoying, eg when "stg rm" behaves differently than "git
rm", by not removing the real file.  I have tried recently to avoid
using "git add" and "stg rm", and I am quite pleased with that :)


> > And yes, it would be cool if stg new could guess patch name from the
> > subject line in a similar manner that stg uncommit does.
> 
> Good idea. This would be embarrassingly easy to do.
> 
> But you can kind of do it today. Just commit with git (my favorite
> here is the emacs modes) and "stg assimilate"!

Well, that's arguably a non-orthodox way of doing things, I like the
idea your "stg new" patch much better :)

Best regards,
-- 
Yann.

Re: Using StGIT for tweaking already-committed stuff

[Karl Hasselström]

On 2007-05-11 22:40:17 +0200, Yann Dirson wrote:

> On Fri, May 11, 2007 at 12:23:47AM +0200, Karl Hasselström wrote:
>
> > But you can kind of do it today. Just commit with git (my favorite
> > here is the emacs modes) and "stg assimilate"!
>
> Well, that's arguably a non-orthodox way of doing things, I like the
> idea your "stg new" patch much better :)

It's only unothodox if you expect git and stgit to not always mix so
well. But if we have the ambition that they should interoperate as
near to seamlessly as we can make them, this kind of workflow becomes
very natural.

It shouldn't be necessary with a manual "assimilate" step. If stgit
finds that there are unadorned git commits on top of the patch stack,
it should do the assimilation automatically. With that in place, "stg
new" and "stg refresh" would be nearly superfluous, since git-commit
with and without --amend does the same thing -- the only thing they
won't do is give the user the option of manually choosing the patch
name.

I believe this sort of integration is the way to go. It'll be
beneficial for git users who want to occasionally use some stgit to
rebase their patch series, since they'll not have to learn more than
two or three new commands in addition to the git they already know.
Heavy stgit users will benefit from having the much larger git
community maintaining a large subset of the porcelain they use,
instead of having to duplicate the effort and always lag behind.

This is no binary choice, of course. One could certainly imagine a
compromise where stgit becomes much easier to mix with git than today,
but still retains the current command set.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Using StGIT for tweaking already-committed stuff

[Yann Dirson]

On Sat, May 12, 2007 at 12:43:25AM +0200, Karl Hasselström wrote:
> It's only unothodox if you expect git and stgit to not always mix so
> well. But if we have the ambition that they should interoperate as
> near to seamlessly as we can make them, this kind of workflow becomes
> very natural.

It is great that this is possible, but I'm not sure I'll ever see it
as "very natural" :)

> It shouldn't be necessary with a manual "assimilate" step. If stgit
> finds that there are unadorned git commits on top of the patch stack,
> it should do the assimilation automatically. With that in place, "stg
> new" and "stg refresh" would be nearly superfluous, since git-commit
> with and without --amend does the same thing -- the only thing they
> won't do is give the user the option of manually choosing the patch
> name.

Hm.  I'm not that convinced :)

Eg, imagine a merge commit somewhere in the stack.  What would stgit
do with that ?

> I believe this sort of integration is the way to go. It'll be
> beneficial for git users who want to occasionally use some stgit to
> rebase their patch series, since they'll not have to learn more than
> two or three new commands in addition to the git they already know.
> Heavy stgit users will benefit from having the much larger git
> community maintaining a large subset of the porcelain they use,
> instead of having to duplicate the effort and always lag behind.
> 
> This is no binary choice, of course. One could certainly imagine a
> compromise where stgit becomes much easier to mix with git than today,
> but still retains the current command set.

I quite like the idea of makeing it easier to mix them, and removing
the real duplicates from stgit, but I think that we should be careful
not to remove power from stgit while doing this.

Best regards,
-- 
Yann.

Transactions for git (and stgit) ?

[Yann Dirson]

On Thu, May 10, 2007 at 09:57:10AM -0700, Carl Worth wrote:
> What I find myself wanting to type is just:
> 
> 	git cherry-pick A..B
> 
> But there is the whole problem of how to deal with any conflict that
> appears during the process.

Indeed this is a problem we also have in StGIT, when pushing multiple
patches after rebasing.  Currently we have to deal with the conflict
and forge a new command-line to finish the job, which is quite awkward.

In this respect, the --continue/--skip/--abort set of flags that
git-rebase has are really useful.

In fact, I have plans to deal with such behaviours with stgit
transactions: in this case, we have the need of user interaction in
the middle of a transaction, and the rebase flags mentionned above are
just a way for the user of continuing or aborting the transaction.

However, currently I'm not sure that git-rebase would be very robust
if the user would mess with HEAD before issuing one of these commands.
Maybe git would also benefit from a generic transaction mechanism of
some sort, so "cherry-pick A..B" and possibly others can behave in a
consistent way with rebase ?

It could even be more sensible to implement transactions at the git
level rather than at the stgit one...

Best regards,
-- 
Yann.

Re: Transactions for git (and stgit) ?

[Karl Hasselström]

On 2007-05-12 11:53:12 +0200, Yann Dirson wrote:

> It could even be more sensible to implement transactions at the git
> level rather than at the stgit one...

Yes, please. (Unless a convincing technical argument pops up against
it, of course.) Any stgit invariant that isn't based on a git
invariant is one more thing that can break when git and stgit commands
are mixed.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Using StGIT for tweaking already-committed stuff

[Karl Hasselström]

On 2007-05-12 09:10:23 +0200, Yann Dirson wrote:

> On Sat, May 12, 2007 at 12:43:25AM +0200, Karl Hasselström wrote:
>
> > It shouldn't be necessary with a manual "assimilate" step. If
> > stgit finds that there are unadorned git commits on top of the
> > patch stack, it should do the assimilation automatically. With
> > that in place, "stg new" and "stg refresh" would be nearly
> > superfluous, since git-commit with and without --amend does the
> > same thing -- the only thing they won't do is give the user the
> > option of manually choosing the patch name.
>
> Hm. I'm not that convinced :)
>
> Eg, imagine a merge commit somewhere in the stack. What would stgit
> do with that ?

There are two cases:

  1. The merge commit is below the bottommost patch. This is perfectly
     OK, and nothing special has to be done. The only restriction is
     that we can't uncommit past the merge.

  2. The merge commit is above the topmost patch. (There may or may
     not also be other not-yet-stgitified commits above the topmost
     patch, below or above the merge commit.) In this case, stgit
     should not auto-assimilate the commits on top of the stack (since
     it can't be done for the merge commit), and a number of stgit
     commands (push, pop, new, ...) should refuse to work until the
     user has either reset the branch so that the merge disappears, or
     done "stg commit" on all the patches below the merge.

Note that these are the only cases: stgit should (and does) enforce
the invariant that the applied patches form a consecutive series of
commits, without "holes". This is why "stg new" would be forbidden in
case (2).

The point is not that you should commit merges on top of your patches,
of course. The point is that if you do, stgit should handle it
gracefully. Right now you can commit all your patches and do a merge,
but if you try to do it the other way around, stgit will break down on
you -- but there's no real reason why it should.

> I quite like the idea of makeing it easier to mix them, and removing
> the real duplicates from stgit, but I think that we should be
> careful not to remove power from stgit while doing this.

I agree.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Merging commits together into a super-commit

[Yann Dirson]

On Thu, May 10, 2007 at 12:48:05PM -0700, Carl Worth wrote:
> On Thu, 10 May 2007 21:21:06 +0200, Petr Baudis wrote:
> > I think you are underestimating stg here.
> 
> Yes, maybe I didn't learn to use it well enough.
> 
> > You can stg init just once per branch (ever), I think.
> 
> I don't have details now, but I know I ran into some difficulty when
> leaving the extra stg state around.

I really think we should have a "stg uninit" command.  Note that
currently "stg branch --delete" on master will just do that instead of
really deleting the branch, but that is a known bug (#8732 on gna).

> It seems that it added stuff that
> resulted in some reference of mine becoming ambiguous, ("refspec <foo>
> matches more than one" perhaps?). What I do remember is that I couldn't
> get one of my standard git push commands to work until I deleted all
> of .git/refs/bases and .git/refs/patches and then things started to
> work again.

I remember quite some time ago that cg-push exhibited this behaviour.
However, nowadays I frequently push stgit stacks with git-push without
a problem.

> > 	stg uncommit -n N
> > 	stg pop -n N-1
> > 	..hack..
> > 	stg refresh
> > 	stg push -a
> >
> > It seems to be a bit shorter than the sequence you've presented above,
> > and overally working with volatile commits using StGIT feels much more
> > natural to me - and I haven't even ever used quilt seriously! (I have
> > special antipathy to the git reset UI, too.)
> 
> The -n option is something I hadn't noticed, and that helps, (except
> that what I've got to start with is a git revision name, not a
> number).

While "uncommit to named commit" that Karl implemented helps here, and
that "stg goto <patchname>" may be a viable alternative to "pop -n",
you may also want to try:

	stg uncommit -t <commit>
	..hack..
	stg refresh -p <patchname>

There are still some rough edges with "refresh -p", though[*], but
when it works I love this comfort :)

Best regards,
-- 
Yann.

[*] most notably, it does not work yet if any patch above the one you
want to modify changes the same file)

Re: Merging commits together into a super-commit

[Jakub Narebski]

Yann Dirson wrote:

> On Thu, May 10, 2007 at 12:48:05PM -0700, Carl Worth wrote:
>> On Thu, 10 May 2007 21:21:06 +0200, Petr Baudis wrote:
>> > I think you are underestimating stg here.
>> 
>> Yes, maybe I didn't learn to use it well enough.
>> 
>> > You can stg init just once per branch (ever), I think.
>> 
>> I don't have details now, but I know I ran into some difficulty when
>> leaving the extra stg state around.
> 
> I really think we should have a "stg uninit" command.  Note that
> currently "stg branch --delete" on master will just do that instead of
> really deleting the branch, but that is a known bug (#8732 on gna).

It would be also nice to have command to remove applied patches.
Sometimes I'd muck up StGIT stack by rebasing in git. Applied patches
are in repository, but I'm interested in preserving unapplied ones.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Merging commits together into a super-commit

[Karl Hasselström]

On 2007-05-12 13:34:30 +0200, Yann Dirson wrote:

> I really think we should have a "stg uninit" command. Note that
> currently "stg branch --delete" on master will just do that instead
> of really deleting the branch, but that is a known bug (#8732 on
> gna).

What we should do is delete all stgit metadata when the last patch
goes away.

And we shouldn't have "stg init", either. Initing should be done
automatically when needed.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Merging commits together into a super-commit

[Yann Dirson]

On Sat, May 12, 2007 at 04:02:28PM +0200, Karl Hasselström wrote:
> On 2007-05-12 13:34:30 +0200, Yann Dirson wrote:
> 
> > I really think we should have a "stg uninit" command. Note that
> > currently "stg branch --delete" on master will just do that instead
> > of really deleting the branch, but that is a known bug (#8732 on
> > gna).
> 
> What we should do is delete all stgit metadata when the last patch
> goes away.

This supposes there is no valuable branch-level metadata.  Currently
we have the description - something which could arguably be moved to
the git level as well.  Otherwise that sounds reasonable to me.

> And we shouldn't have "stg init", either. Initing should be done
> automatically when needed.

Good idea as well, that would make stg more accessible to the average
plain-git user.

Best regards,
-- 
Yann.

Re: Merging commits together into a super-commit

[Karl Hasselström]

On 2007-05-12 16:41:45 +0200, Yann Dirson wrote:

> On Sat, May 12, 2007 at 04:02:28PM +0200, Karl Hasselström wrote:
>
> > What we should do is delete all stgit metadata when the last patch
> > goes away.
>
> This supposes there is no valuable branch-level metadata. Currently
> we have the description - something which could arguably be moved to
> the git level as well. Otherwise that sounds reasonable to me.

I left the branch description out of the discussion on purpose, since
it's not that interesting -- if there is a description, we can simply
delete everything except that. And I wholeheartedly agree that the
branch description doesn't belong in stgit; it's orthogonal to the
business of managing a patch stack.

> > And we shouldn't have "stg init", either. Initing should be done
> > automatically when needed.
>
> Good idea as well, that would make stg more accessible to the
> average plain-git user.

Yes, that's my secret evil master plan.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Transactions for git (and stgit) ?

[Yann Dirson]

On Sat, May 12, 2007 at 12:49:19PM +0200, Karl Hasselström wrote:
> On 2007-05-12 11:53:12 +0200, Yann Dirson wrote:
> 
> > It could even be more sensible to implement transactions at the git
> > level rather than at the stgit one...
> 
> Yes, please. (Unless a convincing technical argument pops up against
> it, of course.) Any stgit invariant that isn't based on a git
> invariant is one more thing that can break when git and stgit commands
> are mixed.

For reference, I have written down some design ideas in january[1].
They were written with StGIT in mind, we'll have to see if it
transposes easily to plain git.

I fear it will not be that easy, at least with this design :)

OTOH, implementing transactions in StGIT could provide a first
experience on this particular field, that may later be transposed to
git core - not unlike cogito did for other features.  There is the
risk, however, of seeing a different (hopefully better) design for the
feature in git afterwards, and this in turn is likely to make life
harder for StGIT...

[1] http://marc.info/?t=116803935800001&r=1&w=2

Best regards,
-- 
Yann.

Re: Merging commits together into a super-commit

[Junio C Hamano]

Yann Dirson <ydirson@altern.org> writes:

> On Sat, May 12, 2007 at 04:02:28PM +0200, Karl Hasselström wrote:
>> ...
>> What we should do is delete all stgit metadata when the last patch
>> goes away.
>
> This supposes there is no valuable branch-level metadata.  Currently
> we have the description - something which could arguably be moved to
> the git level as well.  Otherwise that sounds reasonable to me.

Will it be something like

	[branch "master"]
        	description = "My primary development line"

if so I think that is a reasonable thing to do, from git-core's
point of view.  Obviously, gitk, tig, gitweb and friends can use
this, too.

Are there other per-branch information StGIT wants to keep on an
active branch that might benefit the core as well?

>> And we shouldn't have "stg init", either. Initing should be done
>> automatically when needed.
>
> Good idea as well, that would make stg more accessible to the average
> plain-git user.

Yes, I wished for this often myself.

Re: Merging commits together into a super-commit

[Karl Hasselström]

On 2007-05-12 12:27:49 -0700, Junio C Hamano wrote:

> Yann Dirson <ydirson@altern.org> writes:
>
> > This supposes there is no valuable branch-level metadata.
> > Currently we have the description - something which could arguably
> > be moved to the git level as well. Otherwise that sounds
> > reasonable to me.
>
> Will it be something like
>
>       [branch "master"]
>               description = "My primary development line"

Yes, exactly. It's just a simple per-branch description string, just
like in your suggestion.

> if so I think that is a reasonable thing to do, from git-core's
> point of view. Obviously, gitk, tig, gitweb and friends can use
> this, too.

Absolutely.

> Are there other per-branch information StGIT wants to keep on an
> active branch that might benefit the core as well?

No, I'm pretty sure there isn't.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Merging commits together into a super-commit

[Yann Dirson]

On Sat, May 12, 2007 at 12:27:49PM -0700, Junio C Hamano wrote:
> Are there other per-branch information StGIT wants to keep on an
> active branch that might benefit the core as well?

Maybe the "protected" attribute, to forbid commands to touch to the
stack ?  Not sure, however, since the semantics would probably be a
bit different in git (eg. just forbid update-ref) and in StGIT (also
protects unapplied patches).

Best regards,
-- 
Yann

[StGIT PATCH] Store branch description in the config file

[Karl Hasselström]

Instead of storing the branch description in an StGIT-specific file,
store it in the git config file, where tools other than StGIT can read
and write it.

Signed-off-by: Karl Hasselström <kha@treskal.com>
---

On 2007-05-12 12:27:49 -0700, Junio C Hamano wrote:

> Will it be something like
>
>       [branch "master"]
>               description = "My primary development line"

This was easier to do than I'd thought. I don't get quotes around the
description, though; do I have to insert them manually? And what
purpose do they serve?

 stgit/stack.py |   19 ++++++++++++++-----
 1 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/stgit/stack.py b/stgit/stack.py
index c105b21..7048af7 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -451,7 +451,6 @@ class Series(StgitObject):
                                        self.__name)
 
         self.__hidden_file = os.path.join(self._dir(), 'hidden')
-        self.__descr_file = os.path.join(self._dir(), 'description')
 
         # where this series keeps its patches
         self.__patch_dir = os.path.join(self._dir(), 'patches')
@@ -550,11 +549,23 @@ class Series(StgitObject):
         if os.path.isfile(protect_file):
             os.remove(protect_file)
 
+    def __branch_descr(self):
+        return 'branch.%s.description' % self.get_branch()
+
     def get_description(self):
-        return self._get_field('description') or ''
+        # Fall back to the .git/patches/<branch>/description file if
+        # the config variable is unset.
+        return (config.get(self.__branch_descr())
+                or self._get_field('description') or '')
 
     def set_description(self, line):
-        self._set_field('description', line)
+        if line:
+            config.set(self.__branch_descr(), line)
+        else:
+            config.unset(self.__branch_descr())
+        # Delete the old .git/patches/<branch>/description file if it
+        # exists.
+        self._set_field('description', None)
 
     def get_parent_remote(self):
         value = config.get('branch.%s.remote' % self.__name)
@@ -787,8 +798,6 @@ class Series(StgitObject):
             # (move functionality to StgitObject ?)
             if os.path.exists(self.__hidden_file):
                 os.remove(self.__hidden_file)
-            if os.path.exists(self.__descr_file):
-                os.remove(self.__descr_file)
             if os.path.exists(self._dir()+'/orig-base'):
                 os.remove(self._dir()+'/orig-base')