Re: inotify to minimize stat() calls

[Drew Northup <n1xim.email@gmail.com>]

On Sun, Feb 10, 2013 at 12:24 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Sun, Feb 10, 2013 at 12:10 AM, Ramkumar Ramachandra
> <artagnon@gmail.com> wrote:
>> Finn notes in the commit message that it offers no speedup, because
>> .gitignore files in every directory still have to be read.  I think
>> this is silly: we really should be caching .gitignore, and touching it
>> only when lstat() reports that the file has changed.
>> ...
>> Really, the elephant in the room right now seems to be .gitignore.
>> Until that is fixed, there is really no use of writing this inotify
>> daemon, no?  Can someone enlighten me on how exactly .gitignore files
>> are processed?
>
> .gitignore is a different issue. I think it's mainly used with
> read_directory/fill_directory to collect ignored files (or not-ignored
> files). And it's not always used (well, status and add does, but diff
> should not). I think wee need to measure how much mass lstat
> elimination gains us (especially on big repos) and how much
> .gitignore/.gitattributes caching does. I don't think .gitignore has
> such a big impact though. strace on git.git tells me "git status"
> issues about 2500 lstat calls, and just 740 open+getdents calls (on
> total 3800 syscalls). I will think if we can do something about
> .gitignore/.gitattributes.
> --
> Duy

Duy,
Did your testing turn up anything about the amount of time spent
parsing the .gitignore/.gitattributes files? Not the syscall count,
but the actual time spent running the parser (which I presume is
largely CPU-bound). The other notable bit of information to know would
be how much time is spent applying what has been parsed out of those
files to the content of the tree. Both will give a clear signal of the
prominence of those segments of code versus others elsewhere in the
"git stat" flow path. That information will tell us more clearly what,
if anything, it is worth keeping a cache of and what form that cache
should take.

-- 
-Drew Northup
--------------------------------------------------------------
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59