[Q] Mark files for later commit?

[Q] Mark files for later commit?

[Daniel Nyström]

Hi folks!

I just hit into an situation where a certain file contains a human
written changelog, which is modified (not prepended) for each single
commit in the project, is being committed along with the actual patch.

This approach makes it very difficult to backport the commits, since
it will always try patching the changelog file as well.

The changelog is only supposed to be committed at new releases, but is
kept maintained continuously so stuff won't be forgotten.

So this makes me wonder, is there a way to mark certain files for
being committed later on? Which does not automatically get added to
the staging area (on "git commit -a" or "git add ." and so on) unless
it's specifically mentioned by "git add"?

We've discussed making it generated automatically, but that's not as
easy as it may sound.

How would you like a git feature like described above, marking files
for later inclusion?

Thanks!
--
Daniel

Re: [Q] Mark files for later commit?

[Junio C Hamano]

Daniel Nyström <daniel.nystrom@timeterminal.se> writes:

> So this makes me wonder, is there a way to mark certain files for
> being committed later on? Which does not automatically get added to
> the staging area (on "git commit -a" or "git add ." and so on) unless
> it's specifically mentioned by "git add"?
>
> We've discussed making it generated automatically, but that's not as
> easy as it may sound.
>
> How would you like a git feature like described above, marking files
> for later inclusion?

That does not sound like a feature but merely a source of confusion.

So far, "commit -a", "add", "add ." etc have _all_ been a way to tell git
to add the current state of the content to the index.  What is the point
of making it more complex by letting the user say "I am telling you to add
everything in the working tree by explicitly saying 'git add .', but I do
not really mean it"?

In the meantime, some misguided souls might suggest assume-unchanged, but
that is not guaranteed to work for this purpose, so ignore them.  This is
because assume-unchanged is a promise you make git that you will _not_
change the working tree files, and that promise implies a permission for
git to use blob object recorded in the index that corresponds to such a
path instead of reading from the working tree files while doing certain
operations (such as "git diff") if it is more convenient.

Re: [Q] Mark files for later commit?

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 29.03.2011 08:49:
> Daniel Nyström <daniel.nystrom@timeterminal.se> writes:
> 
>> So this makes me wonder, is there a way to mark certain files for
>> being committed later on? Which does not automatically get added to
>> the staging area (on "git commit -a" or "git add ." and so on) unless
>> it's specifically mentioned by "git add"?
>>
>> We've discussed making it generated automatically, but that's not as
>> easy as it may sound.
>>
>> How would you like a git feature like described above, marking files
>> for later inclusion?
> 
> That does not sound like a feature but merely a source of confusion.
> 
> So far, "commit -a", "add", "add ." etc have _all_ been a way to tell git
> to add the current state of the content to the index.  What is the point
> of making it more complex by letting the user say "I am telling you to add
> everything in the working tree by explicitly saying 'git add .', but I do
> not really mean it"?
> 
> In the meantime, some misguided souls might suggest assume-unchanged, but
> that is not guaranteed to work for this purpose, so ignore them.  This is
> because assume-unchanged is a promise you make git that you will _not_
> change the working tree files, and that promise implies a permission for
> git to use blob object recorded in the index that corresponds to such a
> path instead of reading from the working tree files while doing certain
> operations (such as "git diff") if it is more convenient.
> 

While I've used assume-unchanged before, it really is a great foot-gun
(to shoot yourself into...) to be used only when circumstances force you.

In your case, if you really can't use shortlog or the like, a "light"
way of generating a changelog might be using notes. Attach changelog
notes to each commit (maybe in refs/notes/changelog rather than the
standard). Then, when you need to create the changelog between vX and
vY, you can do

git log --pretty="%N%n" vX..vY

(and go wild on the pretty format, of course). Note that you can share
notes when you set up refspecs etc in a push/pull based workflow. (I'm
not sure how well we support merging/pulling notes refs yet.) This does
not work well in a patch/e-mail-based workflow.

Michael

Re: [Q] Mark files for later commit?

[Johan Herland]

On Tuesday 29 March 2011, Michael J Gruber wrote:
> Note that you can share
> notes when you set up refspecs etc in a push/pull based workflow. (I'm
> not sure how well we support merging/pulling notes refs yet.)

Since v1.7.4, you can use "git notes merge" for merging notes ref. It's not 
yet integrated with "pull", though, so typically you want to fetch it to a 
different ref name, and then manually "git notes merge" that into the 
original notes ref.

E.g. if you want to sync "refs/notes/foo" with remote "bob", you can "git 
fetch bob refs/notes/foo:refs/notes/bobs_foo", and then
merge them with "git notes --ref=foo merge bobs_foo".

> This does not work well in a patch/e-mail-based workflow.

...although there have been mailing list discussions on getting notes added 
to "git format-patch" output (after the "---"), and then picked up again by 
"git am".


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: [Q] Mark files for later commit?

[Michael J Gruber]

Johan Herland venit, vidit, dixit 29.03.2011 09:38:
> On Tuesday 29 March 2011, Michael J Gruber wrote:
>> Note that you can share
>> notes when you set up refspecs etc in a push/pull based workflow. (I'm
>> not sure how well we support merging/pulling notes refs yet.)
> 
> Since v1.7.4, you can use "git notes merge" for merging notes ref. It's not 
> yet integrated with "pull", though, so typically you want to fetch it to a 
> different ref name, and then manually "git notes merge" that into the 
> original notes ref.

Yes, but what I meant with "well" was: If I merge regular branches, do
the notes get merged automatically? And the answer is "no", and as you
explain above, "git notes merge" copes with that (and with merging notes
for the same commit).

> E.g. if you want to sync "refs/notes/foo" with remote "bob", you can "git 
> fetch bob refs/notes/foo:refs/notes/bobs_foo", and then
> merge them with "git notes --ref=foo merge bobs_foo".

Longing for that refs/ layout overhaul...

>> This does not work well in a patch/e-mail-based workflow.
> 
> ...although there have been mailing list discussions on getting notes added 
> to "git format-patch" output (after the "---"), and then picked up again by 
> "git am".

So, here my "not well" was even overly positive ;)

Michael

Re: [Q] Mark files for later commit?

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Daniel Nyström <daniel.nystrom@timeterminal.se> writes:
>
>> So this makes me wonder, is there a way to mark certain files for
>> being committed later on? Which does not automatically get added to
>> the staging area (on "git commit -a" or "git add ." and so on) unless
>> it's specifically mentioned by "git add"?
>>
>> We've discussed making it generated automatically, but that's not as
>> easy as it may sound.
>>
>> How would you like a git feature like described above, marking files
>> for later inclusion?
>
> That does not sound like a feature but merely a source of confusion.

Let's step back a bit, as I think "mark them not to be added" is a classic
XY problem.

First let's concentrate on this part: "a certain file contains a human
written changelog, which is modified (not prepended) for each single
commit in the project, is being committed along with the actual patch."

Doesn't that sound quite similar to our release notes?

So far, I've been updating the file after topics graduate to the 'master'
branch, but there is no fundamental reason not to make it a responsibility
of the patch contributors to add an entry to the release notes when they
make their changes to the system.  Then when the topics graduate, the
entry that describes the topic will be automatically merged into the
release notes of the 'master' branch.

What will happen if we made that the convention of our project?  For one
thing, I don't have to spend time on coming up with a concise and clear
description of the topics (the contributor that worked on the topic should
be better equipped to describe it than I am).  People may make their own
entries in the single release notes file in various places (by convention,
the entries are sorted by the name of the command the change affects,
except for the ones that have system-wide effects that are described at
the top), and merging them might produce many conflicts, though.

But what choice would I have?  Obviously I shouldn't be punting on the
merges, taking one side of the release notes file as the merge result
using "-s ours" or other silly magic and discarding the valuable input
from contributors.  I _do_ want all the release notes entries from all
topics.  Merging such a conflict-heavy file and keeping it maintained
properly is an important part of the job (iow, "suck it up").

I suspect that using the "union" merge attribute may be a possibility in
this case; even though the release notes is not an "append only" file, it
is an "insert only" file, and it is a very good candidate for the union
merge driver.

Now let's look at this part: "The changelog is only supposed to be
committed at new releases, but is kept maintained continuously so stuff
won't be forgotten."

You are willing to leave your changes uncommitted in the working tree, and
not committing it until the release (perhaps using your Y solution that
lets you mark it not to be committed without --force option), is perfectly
Ok by you.  You are not getting any accident protection benefit from your
SCM until the next release point and the intermediate changes between
releases you make to this file are not version controlled.

If that is the case, then probably you shouldn't be even updating the
"changelog" file constantly to begin with.  Instead, how about creating a
separate "changelog+" file that is not tracked, and keep its contents
maintained continuously so stuff won't be forgotten?  When the time to
release comes, you can "mv changelog changelog", and commit it.

That would still require you to find a relevant entry in changelog+ and
modify the true changelog file if you cherry pick a particular change to
an older maintenance track and make a release out of it.  Keeping the
entries in the changelog, and cherry picking the changes together with the
associated entries will probably reduce that "remembering" mental burden,
while it may require trivial conflict resolutions in the file that can be
helped with the union merge driver.

Just a few cents.

Re: [Q] Mark files for later commit?

[Daniel Nyström]

Den 29 mars 2011 09:51 skrev Junio C Hamano <gitster@pobox.com>:
>> Daniel Nyström <daniel.nystrom@timeterminal.se> writes:
>>> How would you like a git feature like described above, marking files
>>> for later inclusion?
>>
>> That does not sound like a feature but merely a source of confusion.
>
> If that is the case, then probably you shouldn't be even updating the
> "changelog" file constantly to begin with.  Instead, how about creating a
> separate "changelog+" file that is not tracked, and keep its contents
> maintained continuously so stuff won't be forgotten?  When the time to
> release comes, you can "mv changelog changelog", and commit it.

If we ignore this particular case, how would a "git hold <file>..." feature do?
--
$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   package/qt/qt.mk
#
# Changed but on hold:
#   (use "git add --holds <file>..." to update what will be committed)
#
#	modified:   CHANGES
#
no changes added to commit (use "git add" and/or "git commit -a")
--
Would there be other use-cases?

Re: [Q] Mark files for later commit?

[Michael J Gruber]

Daniel Nyström venit, vidit, dixit 29.03.2011 11:18:
> Den 29 mars 2011 09:51 skrev Junio C Hamano <gitster@pobox.com>:
>>> Daniel Nyström <daniel.nystrom@timeterminal.se> writes:
>>>> How would you like a git feature like described above, marking files
>>>> for later inclusion?
>>>
>>> That does not sound like a feature but merely a source of confusion.
>>
>> If that is the case, then probably you shouldn't be even updating the
>> "changelog" file constantly to begin with.  Instead, how about creating a
>> separate "changelog+" file that is not tracked, and keep its contents
>> maintained continuously so stuff won't be forgotten?  When the time to
>> release comes, you can "mv changelog changelog", and commit it.
> 
> If we ignore this particular case, how would a "git hold <file>..." feature do?
> --
> $ git status
> # On branch master
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #   (use "git checkout -- <file>..." to discard changes in working directory)
> #
> #	modified:   package/qt/qt.mk
> #
> # Changed but on hold:
> #   (use "git add --holds <file>..." to update what will be committed)
> #
> #	modified:   CHANGES
> #
> no changes added to commit (use "git add" and/or "git commit -a")
> --
> Would there be other use-cases?

As Junio pointed out, your "hold file" (ChangeLog+) is really not
versioned (tracked) at all, so it has no place in the worktree.
Otherwise you'll have constant nagging during the release cycle one way
or the other, if you want git to remind you of the files on hold. If you
don't need the reminder, git does not need to know about the file.
Simply store it somewhere else (such as in .git/description if you don't
use that, or under .git/info/).

Michael

Re: [Q] Mark files for later commit?

[Daniel Nyström]

Den 29 mars 2011 14:01 skrev Michael J Gruber <git@drmicha.warpmail.net>:
> Daniel Nyström venit, vidit, dixit 29.03.2011 11:18:
>> Den 29 mars 2011 09:51 skrev Junio C Hamano <gitster@pobox.com>:
>>>> Daniel Nyström <daniel.nystrom@timeterminal.se> writes:
>>>>> How would you like a git feature like described above, marking files
>>>>> for later inclusion?
>>>>
>>>> That does not sound like a feature but merely a source of confusion.
>>>
>>> If that is the case, then probably you shouldn't be even updating the
>>> "changelog" file constantly to begin with.  Instead, how about creating a
>>> separate "changelog+" file that is not tracked, and keep its contents
>>> maintained continuously so stuff won't be forgotten?  When the time to
>>> release comes, you can "mv changelog changelog", and commit it.
>>
>> If we ignore this particular case, how would a "git hold <file>..." feature do?
>
> As Junio pointed out, your "hold file" (ChangeLog+) is really not
> versioned (tracked) at all, so it has no place in the worktree.
> Otherwise you'll have constant nagging during the release cycle one way
> or the other, if you want git to remind you of the files on hold. If you
> don't need the reminder, git does not need to know about the file.
> Simply store it somewhere else (such as in .git/description if you don't
> use that, or under .git/info/).

Good points, indeed. But untracked files will be tracked on "git add
." which files on hold won't.

But I agree, the "hold" feature requires a better motivation and more
actual use cases. I thought it may catch some interests on the list.