Dumb "continuous" commit dumb question

Dumb "continuous" commit dumb question

[Pat LeSmithe]

Hello,

Is it possible to enable git to automatically and continuously "softly"
commit or stage *all* changes to a [subset of] files in a repository,
without my intervention, as they happen?  Perhaps via a daemon which
monitors the disk for explicit file-saving?

Of course, I would still be able to perform explicit commits (with
descriptive comments) and other git commands, in which case there
probably should be smart handling of the recent soft history.  For
example, it could simply be discarded.

I understand that I could simply remember to commit and/or branch early
and often.  But given that changes by an individual on a given branch
are well-ordered by time, and that the "continuous" operation may be
cheap in many situations, a "live" journal could be useful.

Perhaps a better term is branch-aware undo or git with microstructure.

Sorry if this is old stuff or plain crap.

Thanks.

Sincerely,
Pat LeSmith

Re: Dumb "continuous" commit dumb question

[Marcus Griep]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Pat LeSmithe wrote:
> Hello,
> 
> Is it possible to enable git to automatically and continuously "softly"
> commit or stage *all* changes to a [subset of] files in a repository,
> without my intervention, as they happen?  Perhaps via a daemon which
> monitors the disk for explicit file-saving?

Perhaps with a cron job running a script every minute or so.  You'd want
to verify that changes actually took place before attempting a commit.
Also, what log messages would you be providing?  Log messages can be
very important to the context of a change and the ability to later find
them.

If you're using a specific application, such as a Wiki, it may work
nicely, as log messages can be easily gleaned, and many offer post-edit
hooks that you could use to invoke git.

> Of course, I would still be able to perform explicit commits (with
> descriptive comments) and other git commands, in which case there
> probably should be smart handling of the recent soft history.  For
> example, it could simply be discarded.

You may want to use StGit as that may allow you to make such lightweight
commits easily while still allowing reordering, etc., though I am not
familiar with that application porcelain.

> I understand that I could simply remember to commit and/or branch early
> and often.  But given that changes by an individual on a given branch
> are well-ordered by time, and that the "continuous" operation may be
> cheap in many situations, a "live" journal could be useful.
> 
> Perhaps a better term is branch-aware undo or git with microstructure.

Obviously, my final caution is that while this is possible with some
configuration, you may lose valuable "signal" information in the "noise" of
commits that such a continuous operation might incur.  This low SNR may
make it more difficult to find and revert changes you didn't actually want
to happen or to glean a meaningful history of your working tree.

While in the operable borders of Git, it's outside the normal usage patterns.
If you have success, let us know how it worked for you.

-- 
Marcus Griep
GPG Key ID: 0x5E968152
——
http://www.boohaunt.net
את.ψο´


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 793 bytes --]

Re: Dumb "continuous" commit dumb question

[David Tweed]

On Tue, Aug 19, 2008 at 4:47 AM, Pat LeSmithe <qed777@gmail.com> wrote:
>
> Hello,
>
> Is it possible to enable git to automatically and continuously "softly"
> commit or stage *all* changes to a [subset of] files in a repository,
> without my intervention, as they happen?  Perhaps via a daemon which
> monitors the disk for explicit file-saving?
>
> Of course, I would still be able to perform explicit commits (with
> descriptive comments) and other git commands, in which case there
> probably should be smart handling of the recent soft history.  For
> example, it could simply be discarded.
>
> I understand that I could simply remember to commit and/or branch early
> and often.  But given that changes by an individual on a given branch
> are well-ordered by time, and that the "continuous" operation may be
> cheap in many situations, a "live" journal could be useful.

What I do is have a script that runs every 10 minutes that stages
files to the index and then, using the low-level git plumbing, creates
tree and commit objects on a side branch "temp". With this you can
easily commit to the main branch "main" PROVIDING you are commiting a
superset of the changes you're storing to the side-branch. If you
wanted to specify exactly the things in your "main" commit I suspect
you'd have to do some kind of "git-reset --mixed" to rollback the
side-branch state. One thing to be aware of if you do this is that git
expects your index file to be describing what you intend to do on the
current branch you are on (in, eg, git status), but by doing it this
way you'll get output that acts as if you've staged things for your
"main" branch.

This isn't a problem for me as my (idiosyncratic) usage is to have
commits on my main branch every hour via cron anyway.

If you're particularly interested in this approach I can either try
and explain the commands I use in email or you can try and extract
them from my python script chronoversion:

http://www.personal.rdg.ac.uk/~sis05dst/chronoversion.tgz

> Perhaps a better term is branch-aware undo or git with microstructure.

I actually find I don't use the temp branch to actually undo stuff
(partly because I'm not even trying to keep a neat history so I do
modifications primarily via new commits). Instead, I sometimes modify
and extend my research code in a way that I do maybe an hour or so of
new code and refactoring before it's in a state to actually run again.
If when it finally runs something's broken I find it very helpful to
be able to look backwards to see what changes I've made as the problem
either jumps out or I know where to start experimenting.

One problem with chronological backups is that they often don't
compile/run so you can't bisect on them. One thing I have been trying
to figure out is if there's an easy way to modify my build system so
that it makes a commit approximately every 10 minutes but just after a
successful compile. However, that looks to be a bit too complex and
error prone for the moment.

-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"while having code so boring anyone can maintain it, use Python." --
attempted insult seen on slashdot

Re: Dumb "continuous" commit dumb question

[Shawn O. Pearce]

David Tweed <david.tweed@gmail.com> wrote:
> What I do is have a script that runs every 10 minutes that stages
> files to the index and then, using the low-level git plumbing, creates
> tree and commit objects on a side branch "temp". With this you can
> easily commit to the main branch "main" PROVIDING you are commiting a
> superset of the changes you're storing to the side-branch.

Have the script use "export GIT_INDEX_FILE=.git/temp-branch-index"
so that it stages changes into an index which isn't the main one,
and thus has no impact on the main branch.  Thus you can still
commit a subset of the temp branch at any time.

Actually you can do something like this:

	export GIT_INDEX_FILE=.git/temp-branch-index &&
	cp .git/index $GIT_INDEX_FILE &&
	git add . &&
	git add -u &&
	git update-ref refs/heads/temp $(date | git commit-tree $(git write-tree) -p temp)

;-)

-- 
Shawn.

Re: Dumb "continuous" commit dumb question

[Avery Pennarun]

On Tue, Aug 19, 2008 at 10:32 AM, David Tweed <david.tweed@gmail.com> wrote:
> One problem with chronological backups is that they often don't
> compile/run so you can't bisect on them. One thing I have been trying
> to figure out is if there's an easy way to modify my build system so
> that it makes a commit approximately every 10 minutes but just after a
> successful compile. However, that looks to be a bit too complex and
> error prone for the moment.

You could just have a makefile rule or bash alias that does something
like "make && git commit -a -m temp".  Then remember to always run
that instead of 'make' when you're building.

Of course, committing on a side branch and not screwing with the index
would be much less intrusive.

Avery

Re: Dumb "continuous" commit dumb question

[David Tweed]

On Tue, Aug 19, 2008 at 3:54 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
> You could just have a makefile rule or bash alias that does something
> like "make && git commit -a -m temp".  Then remember to always run
> that instead of 'make' when you're building.

As ever, I wanna do something more deviant than that :-) . The idea is
to take a snapshot (if any tracked file has changed) roughly every ten
minutes. If there happens to have been a successful compile around
that time (+/- 1 minute say), grab the snapshot (including detecting
potential newly created files) then. But if there hasn't, I still want
a snapshot roughly on that 10 minute interval. I could try doing
something like "git reset --soft HEAD~1 && git commit -a" if a make
succeeds within 1 minute, on a strictly chronological snapshot but
scripted resets make me a bit nervous.

It's not hyper-important, just something I'm thinking about.

-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"while having code so boring anyone can maintain it, use Python." --
attempted insult seen on slashdot

Re: Dumb "continuous" commit dumb question

[Avery Pennarun]

On Tue, Aug 19, 2008 at 11:02 AM, David Tweed <david.tweed@gmail.com> wrote:
> On Tue, Aug 19, 2008 at 3:54 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> You could just have a makefile rule or bash alias that does something
>> like "make && git commit -a -m temp".  Then remember to always run
>> that instead of 'make' when you're building.
>
> As ever, I wanna do something more deviant than that :-) . The idea is
> to take a snapshot (if any tracked file has changed) roughly every ten
> minutes. If there happens to have been a successful compile around
> that time (+/- 1 minute say), grab the snapshot (including detecting
> potential newly created files) then. But if there hasn't, I still want
> a snapshot roughly on that 10 minute interval. I could try doing
> something like "git reset --soft HEAD~1 && git commit -a" if a make
> succeeds within 1 minute, on a strictly chronological snapshot but
> scripted resets make me a bit nervous.
>
> It's not hyper-important, just something I'm thinking about.

Doing the 10-minute snapshot doesn't preclude the on-make snapshot.
Just commit at *both* times.  But commiting "around the time there was
a successful build" is kind of pointless since you might change a file
two seconds later.  (Or maybe only I'm that idiosyncratic. :))

Shawn's GIT_INDEX_FILE script seems like a good place to start.  If it
were me, I'd use *two* branches here: one for every time I build, and
one for the periodic commits.  Then the build branch would always be
bisectable, and the periodic branch would always have up-to-date data.
 Commits on the periodic branch would use *both* branch heads as
parents, so you'd be able to easily see and diff the full history.

Have fun,

Avery

Re: Dumb "continuous" commit dumb question

[David Tweed]

On Tue, Aug 19, 2008 at 4:08 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
> Doing the 10-minute snapshot doesn't preclude the on-make snapshot.
> Just commit at *both* times.  But commiting "around the time there was
> a successful build" is kind of pointless since you might change a file
> two seconds later.  (Or maybe only I'm that idiosyncratic. :))

I think I was unclear: the only point about the "roughly" was to
emphasize that being dead-on 10 minutes line doesn't matter from the
chronological aspect. Clearly you want to start the commit processes
as soon as a successful compile you decide is worth keeping completes.
(Actually, looking at the commit times on my cron-initiated commits
the time is mostly 1 or 2 seconds past the 10 minute boundary already,
so there's an already up to 2 seconds for my "detect new files" script
to scan the directories.)

> Shawn's GIT_INDEX_FILE script seems like a good place to start.  If it
> were me, I'd use *two* branches here: one for every time I build, and
> one for the periodic commits.  Then the build branch would always be
> bisectable, and the periodic branch would always have up-to-date data.
>  Commits on the periodic branch would use *both* branch heads as
> parents, so you'd be able to easily see and diff the full history.

I think you mean 3 branches: successful compile, short term history
(every 10 minutes) and long-term archival history (every hour). I
already have the last two, although not in as optimal a manner as
Shawn suggested, and wipe the short term history every week or so.

Interesting idea. I'll ponder it.

-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"while having code so boring anyone can maintain it, use Python." --
attempted insult seen on slashdot

Re: Dumb "continuous" commit dumb question

[Jeff King]

On Tue, Aug 19, 2008 at 07:48:53AM -0700, Shawn O. Pearce wrote:

> Actually you can do something like this:
> 
> 	export GIT_INDEX_FILE=.git/temp-branch-index &&
> 	cp .git/index $GIT_INDEX_FILE &&
> 	git add . &&
> 	git add -u &&
> 	git update-ref refs/heads/temp $(date | git commit-tree $(git write-tree) -p temp)

Get with the times, Shawn. It's "git add -A" now. ;)

-Peff