Advice on strategy for "temporary" commits

Advice on strategy for "temporary" commits

[David Tweed]

Hi,

I've been working with my system taking automatic
hourly git snapshots of (filtered portions of) my home
directory for a couple of months. Being able to look
back to what files looked like mid-afternoon yesterday,
or on 18 Nov, is proving modestly useful. However,
I'm thinking about adding "temporary" commits every
ten minutes which then get discarded after 5 hours-ish
(in addition to the long-term archival hourly commits).
This is motivated by the desire to have finer granularity
for testing/bisecting short-term regressions but not having
ridiculously fine-grained changes clogging up the
archive long-term. (I'm aware that with the commits
being primarily taken on a timed basis I'll have more
non-compiling changes than is usual in a repository, so
that this may not turn out to be useful in practice.)

Looking through the git docs, it looks like the most
natural way of doing this is to make the 10-min commits
(via cron & tagging them under a special tag "temporary
commits only" directory) and then use

git-rebase --onto start-tag end-tag branch

every so often (via cron again) to chop the older
temporary commits between start-tag and end-tag
out of the database.

However, I'm not remotely expert on all the other
things you can do with git, so I'm just checking
there's not a way considered better/safer (eg, a separate
branch or repository).

Many thanks for any insight,
-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
Details are all that matters; God dwells there, and you never get to
see Him if you don't struggle to get them right. -- Stephen Jay Gould

Re: Advice on strategy for "temporary" commits

[Alex Riesen]

On 3/8/07, David Tweed <david.tweed@gmail.com> wrote:
> I've been working with my system taking automatic
> hourly git snapshots of (filtered portions of) my home
> directory for a couple of months. Being able to look
> back to what files looked like mid-afternoon yesterday,
> or on 18 Nov, is proving modestly useful. However,
> I'm thinking about adding "temporary" commits every
> ten minutes which then get discarded after 5 hours-ish
> (in addition to the long-term archival hourly commits).
> This is motivated by the desire to have finer granularity
> for testing/bisecting short-term regressions but not having
> ridiculously fine-grained changes clogging up the
> archive long-term. (I'm aware that with the commits
> being primarily taken on a timed basis I'll have more
> non-compiling changes than is usual in a repository, so
> that this may not turn out to be useful in practice.)

Try using temporary and primary branch. Commit
10-minutes to the temporary branch, reset it to the
head of primary branch after you did a commit to
it and repack. Commits from temporary branch will
be removed.
You even can setup/modify your editor, to do a
temporary commit every time you save a file, for
extra ganularity.

Re: Advice on strategy for "temporary" commits

[J. Bruce Fields]

On Thu, Mar 08, 2007 at 02:39:46PM +0000, David Tweed wrote:
> Hi,
> 
> I've been working with my system taking automatic
> hourly git snapshots of (filtered portions of) my home
> directory for a couple of months. Being able to look
> back to what files looked like mid-afternoon yesterday,
> or on 18 Nov, is proving modestly useful. However,
> I'm thinking about adding "temporary" commits every
> ten minutes which then get discarded after 5 hours-ish
> (in addition to the long-term archival hourly commits).
> This is motivated by the desire to have finer granularity
> for testing/bisecting short-term regressions but not having
> ridiculously fine-grained changes clogging up the
> archive long-term. (I'm aware that with the commits
> being primarily taken on a timed basis I'll have more
> non-compiling changes than is usual in a repository, so
> that this may not turn out to be useful in practice.)
> 
> Looking through the git docs, it looks like the most
> natural way of doing this is to make the 10-min commits
> (via cron & tagging them under a special tag "temporary
> commits only" directory) and then use
> 
> git-rebase --onto start-tag end-tag branch
> 
> every so often (via cron again) to chop the older
> temporary commits between start-tag and end-tag
> out of the database.

You don't want to run git-rebase out of a cron job, because it may
require human interaction.

The simplest thing might be to make the temporary commits onto a
separate branch, and throw that branch away periodically.

--b.

Re: Advice on strategy for "temporary" commits

[David Tweed]

> You don't want to run git-rebase out of a cron job, because it may
> require human interaction.
>
> The simplest thing might be to make the temporary commits onto a
> separate branch, and throw that branch away periodically.

Thanks to you & Alex for suggesting this.

So, at this point I need to ask an embarassingly basic
question: how do I "change branches" from T (say), in order to commit
to a different branch A, without changing the contents of
the working directory back to match what it was at the
time of the last commit to A? (I know this is not the
thing you normally want to do.) Ie, in terms of diagrams
I've got my archival commit branch with its hourly
commits running along the top and the temporary branch
with its temporary commits running along the bottom
and a $ means that, considered just as commited trees
the objects linked are the same:

a-----------a-----------a
 \          $           $
  \-t---t---t---t---t---t---t---t

(In case it's not clear, the "temporary branch" record
extends past the last "archival commit" and throwing
away the temporary commits shouldn't remove any
archival commits.)

So I'm on the temporary branch and have been doing
temporary commits to it and we hit an hour mark.
Cron wants to commit what's _currently_ in my working
directory as a new head to the "archival branch" A
and then swap back the temporary branch to
commit it on that branch and carry on, ie, make
the diagram look like:

a-----------a-----------a-----------a
 \          $           $           $
  \-t---t---t---t---t---t---t---t---t

AIUI neither git-branch nor git-checkout provide
a way to do this. (Clearly the git datastructures
can represent this situation, I'm just not sure how
to ask the tools to do it.)

Again, thanks for all the assistance,
-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
Details are all that matters; God dwells there, and you never get to
see Him if you don't struggle to get them right. -- Stephen Jay Gould

Re: Advice on strategy for "temporary" commits

[Mark Wooding]

David Tweed <david.tweed@gmail.com> wrote:

> a-----------a-----------a-----------a
>  \          $           $           $
>   \-t---t---t---t---t---t---t---t---t
>
> AIUI neither git-branch nor git-checkout provide
> a way to do this. (Clearly the git datastructures
> can represent this situation, I'm just not sure how
> to ask the tools to do it.)

You want the raw git-commit-tree tool.  Suppose your branches are tmp
and hourly (both in refs/heads).  Something like this should make your
hourly commit:

	commit=$(
	  echo hourly-commit |
	  git commit-tree refs/heads/tmp^{tree} -p refs/heads/hourly)
	git update-ref -m "hourly commit" refs/heads/hourly $commit

It might be that you should then start basing your temporary commits on
the most recent archive, so you should also

	git update-ref -m "hourly commit" refs/heads/tmp $commit

Then what you'll end up with is something like

a-----------a-----------a-----------a
 \          $\          $\          $
  \-t---t---t \-t---t---t \-t---t---t

Does that seem sane?

-- [mdw]

Re: Advice on strategy for "temporary" commits

[Jakub Narebski]

[Cc: git@vger.kernel.org]

David Tweed wrote:

> So, at this point I need to ask an embarassingly basic
> question: how do I "change branches" from T (say), in order to commit
> to a different branch A, without changing the contents of
> the working directory back to match what it was at the
> time of the last commit to A?
[...]]
> So I'm on the temporary branch and have been doing
> temporary commits to it and we hit an hour mark.
> Cron wants to commit what's _currently_ in my working
> directory as a new head to the "archival branch" A
> and then swap back the temporary branch to
> commit it on that branch and carry on, ie, make
> the diagram look like:
> 
> a-----------a-----------a-----------a
>  \          $           $           $
>   \-t---t---t---t---t---t---t---t---t
> 
> AIUI neither git-branch nor git-checkout provide
> a way to do this. (Clearly the git datastructures
> can represent this situation, I'm just not sure how
> to ask the tools to do it.)

Try if "git checkout -m" does what you wanted. Or simply
do a merge of "more often" branch into "less often" branch,
perhaps simply not recording it as a merge with 
"git merge --squash" followed by "git commit".

By the way, you know that you can --amend a commit?

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Advice on strategy for "temporary" commits

[David Tweed]

> Try if "git checkout -m" does what you wanted. Or simply
> do a merge of "more often" branch into "less often" branch,
> perhaps simply not recording it as a merge with
> "git merge --squash" followed by "git commit".
>
> By the way, you know that you can --amend a commit?

Jakub suggested primarily being on the temporary branch,
when updating the archival branch is desired to first commit
to the temporary branch, then switch to the archival branch
and do a "git merge --squash temp-branch-name" and commit.
This seems to half-work, in that when it doesn't flag a merge
conflict it does what I want. Unfortunately it often seems to detect
"conflicts" that aren't conflicts for my usage and which make
automatic cron usage impossible, eg,
-------------------------------- 8< -------------------------------
$ git merge --squash temp
 100% (4/4) done
Auto-merged s
CONFLICT (content): Merge conflict in s
Squash commit -- not updating HEAD
Automatic merge failed; fix conflicts and then commit the result.

$ more s
H1
t1
t2
<<<<<<< HEAD:s
=======
t4
t5
>>>>>>> temp:s

--------------------------------- 8< ----------------------------
I _think_ if I could specify an opposite of the "ours" merge
strategy that always takes file contents from the other branches
head commit. An alternative might be to see if I can figure out directly
commiting the relevant file tree to both branches using low-level
git commands avoiding the higher level git processing (since
this isn't really a merge of different development but recording
the same "content state" on two different branches maybe
trying to make a "merge" work is the wrong idea.)

[In case anyone thinks I'm wrong to want to work primarily from cron
jobs, my rationale is that this stuff is personal to me -- ie,
won't be independently changed by anyone else -- and isn't
a focussed product. Years ago I tried using RCS on my home
directory and found I spent lots of time writing contentless
commit messages like "save
at 11.15 on 05/06/02" and that during crunch periods I'd
avoid making check-ins because it was too much extraneous
work; but these were _precisely_ the times I'd be most likely
to rush and do some stupid changes I'd want to back out, so it
didn't really work and so I stopped using RCS. With my "safety net
and historical archive" usage pattern -- which is different from
productised development -- I really want something safe to
run from cron.]

Anyway, thanks for all the help.

-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
Details are all that matters; God dwells there, and you never get to
see Him if you don't struggle to get them right. -- Stephen Jay Gould