Re: Python extension commands in git - request for policy change

[Joshua Jensen <jjensen@workspacewhiz.com>]

----- Original Message -----
From: Eric S. Raymond
Date: 12/11/2012 8:30 PM
> It might be a good fit for extending git; I wouldn't be very surprised if
> that worked. However, I do have concerns about the "Oh, we'll just
> lash together a binding to C" attitude common among lua programmers; I
> foresee maintainability problems and the possibility of slow death by
> low-level details as that strategy tries to scale up.
I don't understand this statement: "Oh, we'll just lash together a 
binding to C" attitude.

??
> My sense is that git's use cases are better served by a glue language
> in the Python/Perl/Ruby class rather than an extension langage. But
> my mind is open on this issue.
I spend nearly 100% of my Git time on Windows.

Spawning new processes in Windows is dog slow.  Using 'git rebase', 
arguably my favorite Git command, is time-waiting torture.  I'm also on 
about as fast of a Windows machine as money can buy these days.

I have a Git add-on similar to git-media that uses the smudge and clean 
filters to read/write large binary files into a separate storage 
location.  When checking out a workspace, Git shells out to run a filter 
for each file it needs to write to the workspace.

I can get a maximum of 100 processes per second with this technique, 
resulting in just 100 files being written to disk.  However, I tend to 
see closer to 60 files written to disk.

So, I patched Git to allow the smudge/clean filters to load up a DLL 
that executes a Lua script.  The Lua script properly retrieves+caches a 
file locally, or it puts the file on a network share.

The in-process DLL checkout ends up being every bit as fast as when we 
use Perforce to sync files to our local workspace.  Git, then, can be a 
Perforce replacement for our needs.

(For those who don't know, Perforce handles large workspaces with 
massive binary files very efficiently.)

Anyway, my preference is to allow scripts to run in-process within Git, 
because it is far, far faster on Windows.  I imagine it is faster than 
forking processes on non-Windows machines, too, but I have no statistics 
to back that up.

Python, Perl, or Ruby can be embedded, too, but Lua probably embeds the 
easiest and smallest out of those other 3 languages.

And shell scripts tend to be the slowest on Windows due to the excessive 
numbers of process invocations needed to get anything reasonable done.

-Josh