Re: inotify to minimize stat() calls

[Robert Zeh <robert.allan.zeh@gmail.com>]

The delay for commands like git status is much worse on Windows than Linux; for my workflow I would be happy with a Windows only implementation. 

>From the description so far, I have some question: how does the daemon get started and stopped?  Is there one per repository --- this seems to be implied by putting the unix domain socket in $GIT_DIR. Could we automatically reject connections from anything other than localhost when using TCP?

Robert Zeh

On Feb 8, 2013, at 8:56 PM, Junio C Hamano <gitster@pobox.com> wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
>> I checked read-cache.c and preload-index.c code.  To get the
>> discussion rolling, I think something like the outline below may be
>> a good starting point and a feasible weekend hack for somebody
>> competent:
>> 
>> * At the beginning of preload_index(), instead of spawning the
>>   worker thread and doing the lstat() check ourselves, we open a
>>   socket to our daemon (see below) that watches this repository and
>>   make a request for lstat update.  The request will contain:
>> 
>>    - The SHA1 checksum of the index file we just read (to ensure
>>      that we and our daemon share the same baseline to
>>      communicate); and
>> 
>>    - the pathspec data.
>> 
>>   Our daemon, if it already has a fresh data available, will give
>>   us a list of <path, lstat result>.  Our main process runs a loop
>>   that is equivalent to what preload_thread() runs but uses the
>>   lstat() data we obtained from the daemon.  If our daemon says it
>>   does not have a fresh data (or somehow our daemon is dead), we do
>>   the work ourselves.
>> 
>> * Our daemon watches the index file and the working tree, and
>>   waits for the above consumer.  First it reads the index (and
>>   remembers what it read), and whenever an inotify event comes,
>>   does the lstat() and remembers the result.  It never writes
>>   to the index, and does not hold the index lock.  Whenever the
>>   index file changes, it needs to reload the index, and discard
>>   lstat() data it already has for paths that are lost from the
>>   updated index.
> 
> I left the details unsaid in thee above because I thought it was
> fairly obvious from the nature of the "outline", but let me spend a
> few more lines to avoid confusion.
> 
> - The way the daemon "watches" the changes to the working tree and
>   the index may well be very platform dependent.  I said "inotify"
>   above, but the mechanism does not have to be inotify.
> 
> - The channel the daemon and the client communicates would also be
>   system dependent.  UNIX domain socket in $GIT_DIR/ with a
>   well-known name would be one possibility but it does not have to
>   be the only option.
> 
> - The data given from the daemon to the client does not have to
>   include full lstat() information.  They start from the same index
>   info, and the only thing preload_index() wants to know is for
>   which paths it should call ce_mark_uptodate(ce), so the answer
>   given by our daemon can be a list of paths.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html