Why can't git-rebase back up?

Why can't git-rebase back up?

[linux]

Newbie question...  In what I assume is a usual technique, I maintain a
"build" branch off of the linux-2.6 history which is what I check out
to build a kernel.  I usually keep it at an official tagged releas,
such as v2.6.16-rc2.

[[ This is because core git won't allow the checked-out HEAD to point
to anything but a branch, and checking out something without having
HEAD point to it is fragile and delicate.  Cogito lets you do this with
cg-seek. ]]

Now, if I want to migrate to a newer base version, I can always use
git-reset --hard v2.6.16-rc3, but that's a bit dangerous.
Preferable is to use git-rebase v2.6.16-rc3, which will preserve
any local edits.

(I could also do it as a merge, but that seems like unnecessary history
clutter.  It's not like local edits are common, anyway.)

But suppose discover a nasty bug in -rc3 and want to move my build branch
back to -rc2.  "git-rebase v2.6.16-rc2" does nothing.  After a bit
of thought, I realize why, but sometime I do want to back up.

What's the best way to do that?  Should git-rebase take an optional
third argument which is the branch head we are moving away from?
E.g. what I want to do would be

	git-rebase v2.6.16-rc2 build v2.6.16-rc3

Or is there some other tool already suited to the job?

(Yes, I'm aware the operations cannot be exact inverses, because if
I applied a local patch that was also included in -rc3, git-rebase
will delete the redundant copy from the build branch, and the backing
up will have no way to know to put it back.  If I wanted to do that,
I would use a merge.)


On a completely unrelated note, how do I find out what git thinks of a
particular file, such as linux-2.6/.config?  It isn't listed in the
output of git-status or git-ls-files, but git-ls-files -i produces an error.
Very confusingly,
git-ls-files -i --exclude-per-directory=.gitignore -X .git/info/exclude .config
also produces no output, even though .* is listed in .gitignore.

Re: Why can't git-rebase back up?

[Andreas Ericsson]

linux@horizon.com wrote:
> Newbie question...  In what I assume is a usual technique, I maintain a
> "build" branch off of the linux-2.6 history which is what I check out
> to build a kernel.  I usually keep it at an official tagged releas,
> such as v2.6.16-rc2.
> 
> [[ This is because core git won't allow the checked-out HEAD to point
> to anything but a branch,


Yes it will. That's how "git reset" works (i.e. pointing HEAD to some 
random object). Think of branches and tags as short and humanly 
understandable names for certain points in the history. You can visit 
any point in history without having a special short name for it.

> and checking out something without having
> HEAD point to it is fragile and delicate.  Cogito lets you do this with
> cg-seek. ]]
> 

It's more than delicate. It's impossible (even for Cogito). You can take 
snapshots of how the tree looked at a certain state and export that 
using git-tar-tree if you wish, but other than that it's impossible to 
visit a certain point in history without pointing HEAD to it (that's how 
visiting that point is done, actually).

> Now, if I want to migrate to a newer base version, I can always use
> git-reset --hard v2.6.16-rc3, but that's a bit dangerous.
> Preferable is to use git-rebase v2.6.16-rc3, which will preserve
> any local edits.
> 
> (I could also do it as a merge, but that seems like unnecessary history
> clutter.  It's not like local edits are common, anyway.)
> 
> But suppose discover a nasty bug in -rc3 and want to move my build branch
> back to -rc2.  "git-rebase v2.6.16-rc2" does nothing.  After a bit
> of thought, I realize why, but sometime I do want to back up.
> 

I'd suggest doing something like this when changing base version (of any 
project, really). Let's say you start, for the first time, with 
v2.6.16-rc2. The workflow would look something like this:

$ git checkout -b mod-2.6.16-rc2 v2.6.16-rc2; # create a branch at the 
desired point in history
$ apply your changes

2.6.16-rc3 is out, and you want to use that instead.
$ git checkout -b mod-2.6.16-rc3 v2.6.16-rc3; # new branch for new trial
$ git rebase mod-2.6.16-rc2; # apply patches you've already made

Now a couple of different things can happen:
1. The rebase works fine. Oh, joy of joy!
2. The rebase doesn't work, but a merge does:
$ git pull . mod-2.6.16-rc2
3. Neither merge or rebase works, but you can manually apply your 
patches, drop the offending ones or resolve the conflicts.
4. There's a bug so you want to regress. Just do:
$ git checkout mod-2.6.16-rc2

In this scenario, only option 3 means trouble and that's inevitable no 
matter how you work since your patches no longer apply.


> What's the best way to do that?  Should git-rebase take an optional
> third argument which is the branch head we are moving away from?
> E.g. what I want to do would be
> 
> 	git-rebase v2.6.16-rc2 build v2.6.16-rc3
> 
> Or is there some other tool already suited to the job?
> 
> (Yes, I'm aware the operations cannot be exact inverses, because if
> I applied a local patch that was also included in -rc3, git-rebase
> will delete the redundant copy from the build branch, and the backing
> up will have no way to know to put it back.  If I wanted to do that,
> I would use a merge.)
> 

If you merge a branch based on top of a later head you'd end up with the 
same commits in your history anyways, since it would have to merge all 
commits up to the merge-base. In essence, you'd do a complicated 
fast-forward, but with reverse history to what I think you think (i.e. 
you'd be pulling 2.6.16-rc3 in on top of your patches).

One way to accomplish this would be to do:
$ git checkout mod-2.6.16-rc3; # get to that point, one way or another
$ git format-patch -k --stdout v2.6.16-rc3 > 2.6.16-rc3.patches.mbox
$ git reset --hard v2.6.16-rc2; # get to that point, one way or another
$ git am -k 2.6.16-rc3.patches.mbox

However, branches and tags are cheap to create, efficient to use, easy 
to remove once you're done with them. They make life easier. Use them. 
You'll be glad you did.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Why can't git-rebase back up?

[linux]

>> [[ This is because core git won't allow the checked-out HEAD to point
>> to anything but a branch,

> Yes it will. That's how "git reset" works (i.e. pointing HEAD to some 
> random object). Think of branches and tags as short and humanly 
> understandable names for certain points in the history. You can visit 
> any point in history without having a special short name for it.

Er... say again?  git reset doesn't touch the .git/HEAD symlink;
it overwrites the file that .git/HEAD points to.

If you know a way to make the core git tools produce a .git/HEAD that
is either not a symlink or does NOT begin with "refs/heads/", I'm quite
curious.

>> and checking out something without having
>> HEAD point to it is fragile and delicate.  Cogito lets you do this with
>> cg-seek. ]]

> It's more than delicate. It's impossible (even for Cogito). You can take 
> snapshots of how the tree looked at a certain state and export that 
> using git-tar-tree if you wish, but other than that it's impossible to 
> visit a certain point in history without pointing HEAD to it (that's how 
> visiting that point is done, actually).

Actually, you're right... Cogito uses refs/heads/cg-seek-point.
But you can do it by hand with git-read-tree and git-checkout-index.
(Very little in git is impossible; some things are just a Really Bad
Idea.)

> Now, if I want to migrate to a newer base version, I can always use
> git-reset --hard v2.6.16-rc3, but that's a bit dangerous.
> Preferable is to use git-rebase v2.6.16-rc3, which will preserve
> any local edits.
> 
> (I could also do it as a merge, but that seems like unnecessary history
> clutter.  It's not like local edits are common, anyway.)
> 
> But suppose discover a nasty bug in -rc3 and want to move my build branch
> back to -rc2.  "git-rebase v2.6.16-rc2" does nothing.  After a bit
> of thought, I realize why, but sometime I do want to back up.
> 

> I'd suggest doing something like this when changing base version (of any 
> project, really).

[Use separate branch names for the two build versions.]

That's certainly a possibility, but kind of annoying to remember what
the "current" version is.  Typically, there are no local changes,
and I'm just using "build" as a conventional name to let me check
out the version.  Just like cg-seek-point.

> However, branches and tags are cheap to create, efficient to use, easy 
> to remove once you're done with them. They make life easier. Use them. 
> You'll be glad you did.

I don't really *want* a branch at all.  I just want to export the
right source tree and compile it.  Local changes are more likely a
mistake than anything else, but core git doesn't have Cogito's "blocked"
flag.  I'm just trying to avoid getting
into the habit of typing "git reset --hard" a lot because that's
dangerous.

It's sort of along the lines of "any workflow that makes you type
'rm -rf *' on a regular basis is a recipe for disaster".

The usual solution, of course, is to write a shell script wrapper with
some safety checking.  For example, I could see if git-name-rev --tags
can come up with a name for the branch head before unleashing git-reset
on it.

But I thought I'd check if the tool already existed.

Re: Why can't git-rebase back up?

[Andreas Ericsson]

linux@horizon.com wrote:
>>>[[ This is because core git won't allow the checked-out HEAD to point
>>>to anything but a branch,
> 
> 
>>Yes it will. That's how "git reset" works (i.e. pointing HEAD to some 
>>random object). Think of branches and tags as short and humanly 
>>understandable names for certain points in the history. You can visit 
>>any point in history without having a special short name for it.
> 
> 
> Er... say again?  git reset doesn't touch the .git/HEAD symlink;
> it overwrites the file that .git/HEAD points to.
> 

My bad. You're right.

> If you know a way to make the core git tools produce a .git/HEAD that
> is either not a symlink or does NOT begin with "refs/heads/", I'm quite
> curious.
> 

The order of precedence works as such:
.git/<anchor-name>
.git/refs/<anchor-name>
.git/refs/tags/<tag-name>
.git/refs/heads/<branch-name>
sha1, with git magic to allow abbreviations

This is why you can't sanely have a branch named HEAD.

> 
>>>and checking out something without having
>>>HEAD point to it is fragile and delicate.  Cogito lets you do this with
>>>cg-seek. ]]
> 
> 
>>It's more than delicate. It's impossible (even for Cogito). You can take 
>>snapshots of how the tree looked at a certain state and export that 
>>using git-tar-tree if you wish, but other than that it's impossible to 
>>visit a certain point in history without pointing HEAD to it (that's how 
>>visiting that point is done, actually).
> 
> 
> Actually, you're right... Cogito uses refs/heads/cg-seek-point.
> But you can do it by hand with git-read-tree and git-checkout-index.
> (Very little in git is impossible; some things are just a Really Bad
> Idea.)
> 
> 
>>Now, if I want to migrate to a newer base version, I can always use
>>git-reset --hard v2.6.16-rc3, but that's a bit dangerous.
>>Preferable is to use git-rebase v2.6.16-rc3, which will preserve
>>any local edits.
>>
>>(I could also do it as a merge, but that seems like unnecessary history
>>clutter.  It's not like local edits are common, anyway.)
>>
>>But suppose discover a nasty bug in -rc3 and want to move my build branch
>>back to -rc2.  "git-rebase v2.6.16-rc2" does nothing.  After a bit
>>of thought, I realize why, but sometime I do want to back up.
>>
> 
> 
>>I'd suggest doing something like this when changing base version (of any 
>>project, really).
> 
> 
> [Use separate branch names for the two build versions.]
> 
> That's certainly a possibility, but kind of annoying to remember what
> the "current" version is.  Typically, there are no local changes,
> and I'm just using "build" as a conventional name to let me check
> out the version.  Just like cg-seek-point.
> 

Then make the current production be 'build', the "soon-to-become 
production" to be 'devel' and tag the ones you're finished with so you 
can revert to them easily if it becomes necessary. That's more or less 
how all projects work, AFAIK.

> 
>>However, branches and tags are cheap to create, efficient to use, easy 
>>to remove once you're done with them. They make life easier. Use them. 
>>You'll be glad you did.
> 
> 
> I don't really *want* a branch at all.  I just want to export the
> right source tree and compile it.


$ git tar-tree v2.6.16-rc3 linux-2.6.16-rc3

OTOH, you could just download the released sources and patches from ftp.

>  Local changes are more likely a
> mistake than anything else, but core git doesn't have Cogito's "blocked"
> flag.  I'm just trying to avoid getting
> into the habit of typing "git reset --hard" a lot because that's
> dangerous.
> 

I don't see why you should have to, even if you don't use a topic-branch 
for fiddling with things. You can still do

	$ git tar-tree <tag> linux-<tag>

to get the snapshot no matter how much you reset and muck about with the 
working tree copy of the current HEAD.

If you're unsure about this, try

$ git checkout -b foo 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2
$ gitk; # behold the dawn of time
$ git tar-tree v2.6.16-rc3 foo | gzip -9 > foo.tar.gz

Note that the first operation takes quite a while, as it has to update 
all the files in the working tree.

> It's sort of along the lines of "any workflow that makes you type
> 'rm -rf *' on a regular basis is a recipe for disaster".
> 
> The usual solution, of course, is to write a shell script wrapper with
> some safety checking.  For example, I could see if git-name-rev --tags
> can come up with a name for the branch head before unleashing git-reset
> on it.
> 
> But I thought I'd check if the tool already existed.
> 

It does, and it doesn't. git checkout -b <name> <commit-ish> does what 
you want, although it creates a branch. "git seek" isn't needed, since 
creating branches is so cheap and far better than having to protect 
certain branch- and tag-names for the sake of creating tools that offer 
less power and flexibility than those already there.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Why can't git-rebase back up?

[Junio C Hamano]

Andreas Ericsson <ae@op5.se> writes:

> The order of precedence works as such:
> .git/<anchor-name>
> .git/refs/<anchor-name>
> .git/refs/tags/<tag-name>
> .git/refs/heads/<branch-name>
> sha1, with git magic to allow abbreviations
>
> This is why you can't sanely have a branch named HEAD.

Actually, you should be able to sanely have a branch named HEAD.

When git barebone Porcelain tools _expect_ branch name from the
user (not just a random committish), they _ought_ to prefix
"refs/heads" in front of it, exactly in order to help
disambiguate things.

Otherwise you have spotted a bug, so please send in a fix.

It is however a different story if you can keep your sanity if
you created a branch called HEAD.  You have to remember to
explicitly say refs/heads/HEAD, where a command wants any
committish or treeish.

Re: Why can't git-rebase back up?

[Junio C Hamano]

linux@horizon.com writes:

> But suppose discover a nasty bug in -rc3 and want to move my build branch
> back to -rc2.  "git-rebase v2.6.16-rc2" does nothing.  After a bit
> of thought, I realize why, but sometime I do want to back up.
>
> What's the best way to do that?  Should git-rebase take an optional
> third argument which is the branch head we are moving away from?
> E.g. what I want to do would be
>
> 	git-rebase v2.6.16-rc2 build v2.6.16-rc3

The one in "next" has a topic branch change which lets you say:

	$ git rebase --onto v2.6.16-rc2 v2.6.16-rc3 build

which is a shorthand for

	$ git checkout build
	$ git rebase --onto v2.6.16-rc2 v2.6.16-rc3

That is, tear out my changes that forked from the development
trail that led to v2.6.16-rc3, and graft them on top of
v2.6.16-rc2.

Re: Why can't git-rebase back up?

[linux]

> The one in "next" has a topic branch change which lets you say:
> 
>	$ git rebase --onto v2.6.16-rc2 v2.6.16-rc3 build
>
> which is a shorthand for
>
>	$ git checkout build
>	$ git rebase --onto v2.6.16-rc2 v2.6.16-rc3
>
> That is, tear out my changes that forked from the development
> trail that led to v2.6.16-rc3, and graft them on top of
> v2.6.16-rc2.

Ah, exactly what I was looking for!
However, why did you decide to make the *destination* the option?

Just giving a naive user's perspective, the argument to git-rebase
is the *destination*, and it "magically" figures out the source.

It would seem more natural to me to have an option to explicltly
specify the "from" and leave the positional argument as the "to".

That also lets you do something about the rebasing-a-merged-branch
problem.  If you have:
      g-->h-->i-->j topic
     /       /
a-->b-->c-->d-->e-->f master

Then I'd expect "git-rebase master topic" to fail with
an error message like:

	Error: "topic" and "master" do not have a unique common
		ancestor.  Possible common ancestors:
		b (master~4)
		d (master~2)
	Rebasing a branch with merges is generally not recommended,
	but if you want to do it, you need to specify where to cut
	off the branch to move.  You probably want one of the above.

And you have to use "git-rebase --from b master" or --from d.
H'm... is that unique?  What if you have a bizarre merge pattern
line


     k--->l---->m
    /            \
    | g-->h-->i-->j topic
    |/        
a-->b-->c-->d-->e-->f master

or 

      g-->h-->i-->j topic
     /       /
a-->b---------->c-->d master

is "--from b" well-defined?  I'll have to play with the existing
code to see what it does.

Re: Why can't git-rebase back up?

[Junio C Hamano]

linux@horizon.com writes:

> That also lets you do something about the rebasing-a-merged-branch
> problem.  If you have:
>
>       g-->h-->i-->j topic
>      /       /
> a-->b-->c-->d-->e-->f master

Ideally, you should be able to come up with:

                        g'->h'->j' topic
                       /
  a-->b-->c-->d-->e-->f master

But it should not matter.  You should not be merging master (or
any other branches that might have interaction with the topic)
into your topic branches to begin with.

Instead, use a separate test branch and merge both topic and
master into it:

             g-->h-->j topic
            /         \       
           /---o---o---o test
          /       / 
     a-->b-->c-->d-->e-->f master

Once your topic is fully cooked, you would merge it into
master.  If the area topic and master touch overlaps, you might
have to resolve the same conflict you would have already
resolved when you merged (j) commit into test in the above
picture, but that is why we have "git rerere" ;-).

BTW, why are we suddenly having many nameless people on this
list, I wonder...

Re: Why can't git-rebase back up?

[linux]

> BTW, why are we suddenly having many nameless people on this
> list, I wonder...

Oh, you know, possibilities like:

- I'm sick of being googled and quizzed about every off-the-cuff
  remark I've ever made.
- My real name is not pronouncable by earthlings.
- My embittered ex joe-jobs me with porn spam whenever he finds a new
  e-mail address for me.
- My day job has licensed BitKeeper and I don't want hassles from
  His Larryness.
- I'm am illegal immigrant and I don't want to get noticed by la Migra.
- My name is Jamal Al-Gashey and it's *so* not worth it.

And I'm never sure if it's politer to use an obvious pseudonym or a
subtle one.
-- 
	-Michael Baldwin <bruce5@wallamaloo.edu.au>

Re: Why can't git-rebase back up?

[Johannes Schindelin]

Hi,

On Sat, 18 Feb 2006, linux@horizon.com wrote:

> > BTW, why are we suddenly having many nameless people on this
> > list, I wonder...
> 
> Oh, you know, possibilities like:
> 
> - I'm sick of being googled and quizzed about every off-the-cuff
>   remark I've ever made.
> - My real name is not pronouncable by earthlings.
> - My embittered ex joe-jobs me with porn spam whenever he finds a new
>   e-mail address for me.
> - My day job has licensed BitKeeper and I don't want hassles from
>   His Larryness.
> - I'm am illegal immigrant and I don't want to get noticed by la Migra.
> - My name is Jamal Al-Gashey and it's *so* not worth it.

- I'm working for NSA, and my name is classified.
- If people knew who I am, they would fault me for my starting a war, 
  instead of commenting on my patches.
- I could use a pseudonym, but that would be a lie, and since you cannot 
  tell if it is a pseudonym to begin with, it is an even bigger lie, and 
  if I did not use a pseudonym, you could mistake it for one, and I could
  not live with that lie at all. (Recurse ad infinitum.)

Having said all that, I think it is important to provide a real and 
traceable name with patches (at least patches which do not do obvious 
things, like fix typos). If only for the reason that incompetent and 
malign people like to sue not only each other.

Ciao,
Dscho