Files different for me

Files different for me

[John Dlugosz]

I'm working with a group, and using git for source code.  I need to change a couple files temporarily and just for me.  I thought, "that's easy", just don't stage them when I check in changes.  But, what do I do when I pull changes from others?  I think it will complain that I have unsaved changes.
What's the best way to do this?

Re: Files different for me

[Feanil Patel]

You could use 'git stash' to stash the changes away for later use.
Then when you want them you can 'git stash apply' them later.

-Feanil

On Wed, Feb 25, 2009 at 10:11 AM, John Dlugosz
<JDlugosz@tradestation.com> wrote:
>
> I'm working with a group, and using git for source code.  I need to change a couple files temporarily and just for me.  I thought, "that's easy", just don't stage them when I check in changes.  But, what do I do when I pull changes from others?  I think it will complain that I have unsaved changes.
> What's the best way to do this?
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Files different for me

[Brian Gernhardt]

On Feb 25, 2009, at 11:11 AM, John Dlugosz wrote:

> I'm working with a group, and using git for source code.  I need to  
> change a couple files temporarily and just for me.  I thought,  
> "that's easy", just don't stage them when I check in changes.  But,  
> what do I do when I pull changes from others?  I think it will  
> complain that I have unsaved changes.
> What's the best way to do this?

Generally when I keep changes like this, I make a commit called "Local  
Changes" or similar and have branch.master.rebase set to true so that  
my changes get rebased on top of origin when I pull.

~~ Brian

Re: Files different for me

[Junio C Hamano]

"John Dlugosz" <JDlugosz@TradeStation.com> writes:

> I'm working with a group, and using git for source code.  I need to
> change a couple files temporarily and just for me.  I thought, "that's
> easy", just don't stage them when I check in changes.  But, what do I do
> when I pull changes from others?  I think it will complain that I have
> unsaved changes.  What's the best way to do this?

[jc: Overlong lines wrapped]

This typically happens when a configuration file of some sort that *must*
be different in each work tree is tracked.

"git pull" and "git merge" do not care when you have local changes to
paths *and* the merge does not involve them, and errors out without
touching any files in your tree when the merge needs to touch them.  You
can safely deal with your local changes after seeing such an error.  This,
and because such a file tend to be modified much less often than the real
contents, means that:

 (1) you do not usually have to worry about this issue, and can keep your
     small local changes you do not mind losing around, and

 (2) when you have to recover, you can easily stash your changes away,
     redo the pull and unstash them.

If you want an "I do not have to think" solution, an easiest recipe to
follow would be:

    $ git stash
    $ git pull
    ... potentially resolve conflicts and make a merge commit ...
    $ git stash pop

But as described above, stash/stash pop are superfluous for most of the
time.

RE: Files different for me

[John Dlugosz]

> Generally when I keep changes like this, I make a commit called "Local

> Changes" or similar and have branch.master.rebase set to true so that

> my changes get rebased on top of origin when I pull.

That sounds ideal.  However, I don't understand the specific steps you
mention.  Looking in the help for git-config,

	branch.<name>.rebase

		When true, rebase the branch <name> on top of the
fetched branch, 
		instead of merging the default branch from the default
remote when 
		"git pull" is run. NOTE: this is a possibly dangerous
operation; 
		do not use it unless you understand the implications 
		(see git-rebase(1) for details).

So, assuming you are working on the "master" branch, this will rebase
the pulled content on top of the existing "master" rather than merging.
If my local changes are committed to "master" first, then this will take
all the commits from other developers that I don't already have in my
local copy and apply them on top of my existing (including Local
Changes).  But since those will now be different commits, what happens
next time?  Ah, "...which introduce the same textual changes..." so
that's covered in how rebase works.

But this will have Local Changes present, and different commits (with
the same textual changes) in my branch.  So what happens when I "push"?

	Oldstuff--A--B--C  remote
	       \
	        LC--X--Y  mine

LC is "Local Changes", X and Y are changes I made, and A, B, C are
changes from other developers.

After a fetch, I have:

	Oldstuff--LC--X--Y--A'--B'--C'  mine

So what happens when I "push"?

In any case, the whole point is that I don't want to publish LC.

Re: Files different for me

[Linus Torvalds]

On Wed, 25 Feb 2009, John Dlugosz wrote:
>
> I'm working with a group, and using git for source code.  I need to 
> change a couple files temporarily and just for me.  I thought, "that's 
> easy", just don't stage them when I check in changes.  But, what do I do 
> when I pull changes from others?  I think it will complain that I have 
> unsaved changes.

If your changes do not touch any of the files that the "git pull" updates, 
then everything is fine. The pull will just work, and your changes will 
still exists in your tree. This is not an accident - git was very much 
designed to work that way, because it's a common usage case for me.

I often have some trivial small changes in my tree (like a pending change 
to the top-level Makefile for the next version number that I just haven't 
committed yet - just a reminder to myself that I'm soon about to release 
another -rc). And I still want to continue to do "git pull" to fetch 
stuff, or even "git am -s" to apply patches.

HOWEVER. If the pull actually wants to modify a file that you have changed 
(ie that same file was changed in the remote), then "git pull" will fail 
gracefully after having done the fetch, saying something like

	Entry 'file-name' not uptodate. Cannot merge.

and at that point you have to decide whethe you want to commit the change, 
"stash" it, or just undo it. Or whether you don't want to do the merge 
yet because you're still working on your own changes, and don't want the 
distraction.

		Linus

Re: Files different for me

[Brian Gernhardt]

On Feb 25, 2009, at 1:02 PM, John Dlugosz wrote:

>> Generally when I keep changes like this, I make a commit called  
>> "Local
>> Changes" or similar and have branch.master.rebase set to true so that
>> my changes get rebased on top of origin when I pull.
>
> That sounds ideal.  However, I don't understand the specific steps you
> mention.  Looking in the help for git-config,

As Junio and Linus have pointed out, most pulls will ignore local  
changes,
so this is likely overkill unless your changes are in commonly changed  
files.

> So, assuming you are working on the "master" branch, this will rebase
> the pulled content on top of the existing "master" rather than  
> merging.
> If my local changes are committed to "master" first, then this will  
> take
> all the commits from other developers that I don't already have in my
> local copy and apply them on top of my existing (including Local
> Changes).  But since those will now be different commits, what happens
> next time?  Ah, "...which introduce the same textual changes..." so
> that's covered in how rebase works.

You can set branch.master.rebase with

   git config branch.master.rebase true

It actually works the other way.  Your changes will be rebased on top of
the work from other developers.

A--B--E--F  origin/master
     \
      C--D master

will become

A--B--E--F  origin/master
           \
            C'--D' master

I share most of my changes using format-patch and e-mail instead of
pushes, so I just don't generate patches for my unimportant changes.
For pushing, I'd suggest either working on new features in a different
branch ("topic" and "local" for example), or using "rebase -i" to move
your local changes to the top and using "push HEAD^".

~~ Brian

Re: Files different for me

[Matthieu Moy]

"John Dlugosz" <JDlugosz@TradeStation.com> writes:

> 	Oldstuff--A--B--C  remote
> 	       \
> 	        LC--X--Y  mine
>
> LC is "Local Changes", X and Y are changes I made, and A, B, C are
> changes from other developers.
>
> After a fetch, I have:
>
> 	Oldstuff--LC--X--Y--A'--B'--C'  mine

No, your changes get rebased:

 	Oldstuff--A--B--C--LC'--X'--Y'   mine

You should just be carrefull not to push in this state, since you'd
push LC too. But you can throw away LC later with "git rebase -i".

-- 
Matthieu

Re: Files different for me

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> If your changes do not touch any of the files that the "git pull" updates, 
> then everything is fine. The pull will just work, and your changes will 
> still exists in your tree. This is not an accident - git was very much 
> designed to work that way, because it's a common usage case for me.
>
> I often have some trivial small changes in my tree (like a pending change 
> to the top-level Makefile for the next version number that I just haven't 
> committed yet - just a reminder to myself that I'm soon about to release 
> another -rc). And I still want to continue to do "git pull" to fetch 
> stuff, or even "git am -s" to apply patches.
>
> HOWEVER. If the pull actually wants to modify a file that you have changed 
> (ie that same file was changed in the remote), then "git pull" will fail 
> gracefully after having done the fetch, saying something like
>
> 	Entry 'file-name' not uptodate. Cannot merge.
>
> and at that point you have to decide whethe you want to commit the change, 
> "stash" it, or just undo it. Or whether you don't want to do the merge 
> yet because you're still working on your own changes, and don't want the 
> distraction.

I've been repeating the above to new people to save you time, but recently
I noticed one thing.

The handling of a case where a pull decides to go ahead (because it does
not have to touch the Makefile you have your codename updates in) but does
not complete with real conflicts, is not as graceful as the other two
cases (merge refusing to run at all without touching anything, or merge
completes cleanly and makes a commit).

You will be left with:

 - Paths that have local changes (index matches HEAD but work tree does
   not match the index --- like your Makefile);

 - Paths cleanly merged (index and HEAD are different but work tree
   already matches the index);

 - Unmerged paths (index has higher stage entries with <<</===/>>> files
   in the work tree);

You, I and experienced users know what to do.  Deal *only* with the last
kind, mark them with "git add" after you are done with each of them, and
make sure you do not say "-a" when committing the result, to exclude the
first kind from the merge result.

I've been wondering if we can make this safer for others.

RE: Files different for me

[John Dlugosz]

Thanks for the clarification, and thanks all on the thread for
explaining things to me.
It appears that fetching with my local changes is not a problem.  For
going the other way, you suggest:

	For pushing, I'd suggest either working on new features in a
different
	branch ("topic" and "local" for example), or using "rebase -i"
to move
	your local changes to the top and using "push HEAD^".

I get the idea of floating "that" change to the top and using "push
HEAD^", though I've never tried --interactive so I'll have to play
around with that on a backup first.

I think if those changes are not committed, it won't be an issue.  

So, my plan at this point is to "stash" my Local Change, and then apply
it.  Then just keep it out of the index.  If I do get a conflict, I can
re-apply my change from the stashed copy.  


Ordinarily, I'll all about working on a feature on a branch, exposing a
"task based" system to other developers.  But, this particular change is
not isolated, and I'm adding parameter to functions and such (I'm
changing the way configuration works and making more things
configurable) all over the place.  I want to push stable work frequently
so other changes don't get too far away making a merge nightmare.

<rant>
Last time, I was careful to publish stable code frequently to the
official "dev" branch, but when another major feature was finished, they
just declared that branch to be the new main-line official branch.
</rant>


-----Original Message-----
From: Brian Gernhardt [mailto:benji@silverinsanity.com] 

You can set branch.master.rebase with

   git config branch.master.rebase true

It actually works the other way.  Your changes will be rebased on top of
the work from other developers.

A--B--E--F  origin/master
     \
      C--D master

will become

A--B--E--F  origin/master
           \
            C'--D' master

I share most of my changes using format-patch and e-mail instead of
pushes, so I just don't generate patches for my unimportant changes.
For pushing, I'd suggest either working on new features in a different
branch ("topic" and "local" for example), or using "rebase -i" to move
your local changes to the top and using "push HEAD^".

~~ Brian

Re: Files different for me

[Linus Torvalds]

On Wed, 25 Feb 2009, Junio C Hamano wrote:
>
> I've been repeating the above to new people to save you time, but recently
> I noticed one thing.
> 
> The handling of a case where a pull decides to go ahead (because it does
> not have to touch the Makefile you have your codename updates in) but does
> not complete with real conflicts, is not as graceful as the other two
> cases (merge refusing to run at all without touching anything, or merge
> completes cleanly and makes a commit).

I agree (although your phrasing was confusing - by "does not complete with 
real conflicts" you made it sound like there were no real conflicts, but 
you must have meant "does not actually finish the merge commit _due_ to 
real conflicts").

That case is one large part of why I wanted to have that "git reset 
--merge" behavior - because it's a good way to get back to the "pre-merge 
with dirty state" situation. Although I have to admit that I don't think 
I've had that happen since the feature got merged ;)

> You will be left with:
> 
>  - Paths that have local changes (index matches HEAD but work tree does
>    not match the index --- like your Makefile);
> 
>  - Paths cleanly merged (index and HEAD are different but work tree
>    already matches the index);
> 
>  - Unmerged paths (index has higher stage entries with <<</===/>>> files
>    in the work tree);

Yes. The good news is that for people who know what they are doing, this 
is all unambiguous. Clean merges will be up-to-date in the index, unmgered 
paths will be marked as such in the index, and your own _real_ dirty state 
will be unambioguously dirty in the working tree.

But I do agree that if you don't know what's up, you now are an in a 
really good position for screwing up, and (for example) resolving the 
merge conflict and then incorrectly committing your own unrelated changes 
with the merge.

> You, I and experienced users know what to do.  Deal *only* with the last
> kind, mark them with "git add" after you are done with each of them, and
> make sure you do not say "-a" when committing the result, to exclude the
> first kind from the merge result.
> 
> I've been wondering if we can make this safer for others.

You're right. We could decide to have a mode (maybe default to it, so that 
people like me can just use a config option to enable "expert" mode) that
simply refuses to do the merge if it doesn't succeed cleanly if there were 
dirty files in the tree.

			Linus

Re: Files different for me

[Jay Soffian]

On Wed, Feb 25, 2009 at 1:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
> The handling of a case where a pull decides to go ahead (because it does
> not have to touch the Makefile you have your codename updates in) but does
> not complete with real conflicts, is not as graceful as the other two
> cases (merge refusing to run at all without touching anything, or merge
> completes cleanly and makes a commit).
>
> You will be left with:
>
>  - Paths that have local changes (index matches HEAD but work tree does
>   not match the index --- like your Makefile);
>
>  - Paths cleanly merged (index and HEAD are different but work tree
>   already matches the index);
>
>  - Unmerged paths (index has higher stage entries with <<</===/>>> files
>   in the work tree);
>
> You, I and experienced users know what to do.  Deal *only* with the last
> kind, mark them with "git add" after you are done with each of them, and
> make sure you do not say "-a" when committing the result, to exclude the
> first kind from the merge result.
>
> I've been wondering if we can make this safer for others.

Have pull detect this case and stash if so, with a message to the user
to pop the stash after they have committed the merge results? Or would
it make more sense to do it in merge? Maybe a pre-merge hook?

j.

RE: Files different for me

[John Dlugosz]

=== re: ===

The handling of a case where a pull decides to go ahead (because it does
not have to touch the Makefile you have your codename updates in) but
does
not complete with real conflicts, is not as graceful as the other two
cases (merge refusing to run at all without touching anything, or merge
completes cleanly and makes a commit).

You will be left with:

 - Paths that have local changes (index matches HEAD but work tree does
   not match the index --- like your Makefile);

 - Paths cleanly merged (index and HEAD are different but work tree
   already matches the index);

 - Unmerged paths (index has higher stage entries with <<</===/>>> files
   in the work tree);

You, I and experienced users know what to do.  Deal *only* with the last
kind, mark them with "git add" after you are done with each of them, and
make sure you do not say "-a" when committing the result, to exclude the
first kind from the merge result.

I've been wondering if we can make this safer for others.

===end===

I've gone over that carefully and I understand (I think) what you are
saying.  The first two are things that were not committed, and should
stay that way (added or not) if they did not conflict.  But they can get
in the way if a merge (on other files) is needed.

In an effort to "wonder" out loud, can you explain how to handle that
with "mergetool"?  For a dumb user like me, it just fixes some files
itself (I guess kdiff is smarter than the normal merge logic) and
presents me with a GUI for things I need to specify.  This should
naturally only go through files with conflicts because of those
"<<</===/>>>" files present.

So, what should I know/do?  "Don't use -a"?  If the idea is to commit
the merged stuff but preserve the status of what I've added but don't
want to commit yet, I'm at a loss.  Using git GUI, it will be backwards:
my additions show, but the freshly merged files are noticed as changes
that could be staged.  I want to un-stage the original, stage the merged
files, commit, then re-stage the original stuff?!

Looking again and what you wrote, I think you are not doing that at all.
You would add the merged files to the index, carefully preserving the
first kind.  Is it possible/easy to do what I thought you meant at
first:  commit just the merged files, and leave the "unaffected" files
still in the index and not committed?

--John

RE: Files different for me

[John Dlugosz]

=== re: ===
Have pull detect this case and stash if so, with a message to the user
to pop the stash after they have committed the merge results? Or would
it make more sense to do it in merge? Maybe a pre-merge hook?
===end===

I've wondered a couple times how to abort a merge.  I ended up just deleting all the funny files.  Did I understand correctly that "git reset --merge" is a new feature?

Perhaps best practice, if I have stuff I've added but don't want to commit yet (why? add the files as you touch them so you don't forget?  keep them from getting confused with the ones you intend to not add at all?), or changes I've not added yet,
would be to "stash" first, do the pull, then "stash apply".

With the existence of a clean way to abort the merge, I could just pull with the assumption that there will be no conflicts, then abort, stash, pull again if needed.  Without the ability to abort the merge, the stakes are high to risk the assumption that all will go well.  So, always stash first.  

Assuming that is correct, it inspires this behavior:
Automatically stash and apply first, then merge.
If merge is clean, delete the stash.
If intervention is needed, I have the stash state to reset to if necessary, and to show me what was already changed if I need to know that.

But I'm still mixed up.  What is the requirement here?  I can understand the need to pull to keep current but not publish my own changes yet.  But why is it necessary to preserve the fact that _some_ (not all) of the changes are in the index?

--John

Re: Files different for me

[Junio C Hamano]

"John Dlugosz" <JDlugosz@TradeStation.com> writes:

> You will be left with:
>
>  - Paths that have local changes (index matches HEAD but work tree does
>    not match the index --- like your Makefile);
>
>  - Paths cleanly merged (index and HEAD are different but work tree
>    already matches the index);
>
>  - Unmerged paths (index has higher stage entries with <<</===/>>> files
>    in the work tree);
>
> You, I and experienced users know what to do.  Deal *only* with the last
> kind, mark them with "git add" after you are done with each of them, and
> make sure you do not say "-a" when committing the result, to exclude the
> first kind from the merge result.
>
> I've been wondering if we can make this safer for others.
>
> ===end===
>
> I've gone over that carefully and I understand (I think) what you are
> saying.  The first two are things that were not committed, and should
> stay that way (added or not) if they did not conflict.  But they can get
> in the way if a merge (on other files) is needed.

No, the latter two *should* be committed.  The first one *must* be
excluded.

"merge" (and "git am" with or without "-3" for patch application) are
carefully written in such a way that:

 (1) They do not tolerate a dirty index.  They stop without touching
     anything if you have *any* staged changes.

 (2) As long as your index is clean, i.e. matches HEAD, there are two
     cases.

     (2-a) They add cleanly merged paths to the index and write the result
           out in the work tree.

     (2-b) They leave unclean merges as unmerged entries in the index and
           write the conflicted merge result in the work tree.

     (2-c) If all paths cleanly merge, then the index is written out as a
           tree and a merge commit is created.

     However, neither of the above happens when you have local changes to
     the paths they need to do so.

     Your local changes to the paths that "merge" (and "git am") does not
     have to touch are tolerated.

Unlike CVS/SVN, you do not merge when you are in the middle of doing
something, potentially risking a huge merge conflict that you are unable
to resolve and redo, and (1) and "However" in (2) are both safety against
such "Merge conflicts between my HEAD and the other branch are intermixed
with my still uncommitted changes, and I am lost" disaster.  You get a
chance to finish what you started working with your index first before
continuing with the merge.

We were discussing something very different.

In the case (2) where your local changes do not interact with the merge,
the merge as the whole can fail due to conflicts in some paths.  If you
ran "git add" before starting this merge, you wouldn't have come this far
but would have been dealt with by the safety of (1).

Now, in such a case, you have:

 - Files you had modifications before starting this merge.  Because we are
   talking about case (2), by definition, you haven't done "git add" to
   them.  The index entries are at stage 0 and match HEAD for these paths.

 - Files cleanly merged.  In the index, they are at stage 0 and may be
   different from HEAD.  The difference from HEAD comes from the changes
   in the branch being merged (or in the case of "am -3", the change the
   patch introduces).  It can never come from your local changes, because
   we are talking about case (2), and you couldn't have done "git add"
   them before you started this merge.

 - Files with conflicts.  These conflict come from changes committed to
   your HEAD and the branch being merged (or in the case of "am -3", the
   change the patch introduces).  You can never have had local changes to
   these paths (see "However" in (2) above).

That means:

 - The paths at stage 0 in the index can and should be committed without
   any further "git add" when recording this merge.  Doing "git add" for
   paths in the first category among the three will include your unrelated
   local changes in the result which is not what you want.  And doing "git
   add" for the paths in the second category is unnecessary; "merge" (and
   "git am") have already updated the index with the merge result.

 - The paths at higher stages (i.e. unmerged) come from the merge, and you
   must resolve them (either in vi or mergetool) and "git add" them.

So the short rule is "resolve and 'git add' to mark the resolution only
for paths with conflicts.  Never 'git add' anything else before making
your commit.  And do not say 'git commit -a' because that is part of the
previous rule."

> In an effort to "wonder" out loud, can you explain how to handle that
> with "mergetool"?  For a dumb user like me, it just fixes some files
> itself (I guess kdiff is smarter than the normal merge logic) and
> presents me with a GUI for things I need to specify.  This should
> naturally only go through files with conflicts because of those
> "<<</===/>>>" files present.
>
> So, what should I know/do?  "Don't use -a"?  If the idea is to commit
> the merged stuff but preserve the status of what I've added but don't
> want to commit yet,

You do not have to worry about that, because you don't "merge" (or "am")
when you already have changes that are "git add"ed to the index.  These
tools correctly detect this case that you are in the middle of something
and refuse to touch neither your index nor your work tree(see (1) above).

Re: Files different for me

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

>> You, I and experienced users know what to do.  Deal *only* with the last
>> kind, mark them with "git add" after you are done with each of them, and
>> make sure you do not say "-a" when committing the result, to exclude the
>> first kind from the merge result.
>> 
>> I've been wondering if we can make this safer for others.
>
> You're right. We could decide to have a mode (maybe default to it, so that 
> people like me can just use a config option to enable "expert" mode) that
> simply refuses to do the merge if it doesn't succeed cleanly if there were 
> dirty files in the tree.

"git merge" has always had this "stash away local changes before starting,
and unstash once done" safety when we try to run multiple strategies.
A patch to trigger it even for a single strategy case may be trivial.

Re: Files different for me

[Linus Torvalds]

On Wed, 25 Feb 2009, Junio C Hamano wrote:
> 
> "git merge" has always had this "stash away local changes before starting,
> and unstash once done" safety when we try to run multiple strategies.
> A patch to trigger it even for a single strategy case may be trivial.

Well, I'd feel better if it was actually in the low-level merge code, the 
way the current "I refuse to merge if the file is dirty" logic is.

			Linus