Re: inotify to minimize stat() calls

[Junio C Hamano <gitster@pobox.com>]

Duy Nguyen <pclouds@gmail.com> writes:

> On Sun, Feb 10, 2013 at 06:17:32PM +0700, Duy Nguyen wrote:
>> The following patch eliminates untracked search code. As we can see,
>> open+getdents also disappears with this patch:
>> 
>> 0.462909 40950 lstat   0.462909 40950 lstat
>> 0.003417 129 brk       0.003417 129 brk
>> 0.000762 53 read       0.000762 53 read
>> 0.000720 36 open       0.000720 36 open
>> 0.000544 12 munmap     0.000454 33 close
>
> .. and the patch is missing:
>
> -- 8< --
> diff --git a/dir.c b/dir.c
> index 57394e4..1963c6f 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -1439,8 +1439,10 @@ int read_directory(struct dir_struct *dir, const char *path, int len, const char
>  		return dir->nr;
>  
>  	simplify = create_simplify(pathspec);
> +#if 0
>  	if (!len || treat_leading_path(dir, path, len, simplify))
>  		read_directory_recursive(dir, path, len, 0, simplify);
> +#endif

The other "lstat()" experiment was a very interesting one, but this
is not yet an interesting experiment to see where in the "ignore"
codepath we are spending times.

We know that we can tell wt_status_collect_untracked() not to bother
with the untracked or ignored files with !s->show_untracked_files
already, but I think the more interesting question is if we can show
the untracked files with less overhead.

If we want to show untrackedd files, it is a given that we need to
read directories to see what paths there are on the filesystem. Is
the opendir/readdir cost dominating in the process? Are we spending
a lot of time sifting the result of opendir/readdir via the ignore
mechanism? Is reading the "ignore" files costing us much to prime
the ignore mechanism?

If readdir cost is dominant, then that makes "cache gitignore" a
nonsense proposition, I think.  If you really want to "cache"
something, you need to have somebody (i.e. a daemon) who constantly
keeps an eye on the filesystem changes and can respond with the up
to date result directly to fill_directory().  I somehow doubt that
it is a direction we would want to go in, though.