Re: [PATCH] Teach "git add" and friends to be paranoid

[Junio C Hamano <gitster@pobox.com>]

Dmitry Potapov <dpotapov@gmail.com> writes:

> ... I think it is possible
> to avoid the overhead of being on the safe side in a few common cases.
> Here is a patch. I have not had time to test it, but changes appear to
> trivial.

Yeah, these are obvious "paranoia not needed" cases.

How much "speed-up" are we talking about, though?  Can we quantify?  I
personally think it is not even worth to quantify it but instead simply
say "avoid unnecessary computation" without saying "speed up", though ;-)

Thanks.

> -- >8 --
> From 3e53610a41c4aad458dff13135a73bb4944f456b Mon Sep 17 00:00:00 2001
> From: Dmitry Potapov <dpotapov@gmail.com>
> Date: Fri, 19 Feb 2010 11:00:51 +0300
> Subject: [PATCH] speed up "git add" by avoiding the paranoid mode
>
> While the paranoid mode preserve the git repository from corruption in the
> case when the added file is changed simultaneously with running "git add",
> it has some overhead. However, in a few common cases, it is possible to
> avoid this mode and still be on the safe side:
>
> 1. If mmap() is implemented as reading the whole file in memory.
>
> 2. If the whole file was read in memory as result of applying some filter.
>
> 3. If the added file is small, it is faster to use read() than mmap().
>
> Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
> ---
>  sha1_file.c |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/sha1_file.c b/sha1_file.c
> index d8a7722..4efeb21 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -2469,6 +2469,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,
>  		                   write_object ? safe_crlf : 0)) {
>  			buf = strbuf_detach(&nbuf, &size);
>  			re_allocated = 1;
> +			paranoid = 0;
>  		}
>  	}
>  
> @@ -2490,7 +2491,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
>  	size_t size = xsize_t(st->st_size);
>  
>  	flag = write_object ? INDEX_MEM_WRITE_OBJECT : 0;
> -	if (!S_ISREG(st->st_mode)) {
> +	if (!S_ISREG(st->st_mode) || size < 262144) {
>  		struct strbuf sbuf = STRBUF_INIT;
>  		if (strbuf_read(&sbuf, fd, 4096) >= 0)
>  			ret = index_mem(sha1, sbuf.buf, sbuf.len,
> @@ -2500,7 +2501,9 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
>  		strbuf_release(&sbuf);
>  	} else if (size) {
>  		void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
> +#ifndef NO_MMAP
>  		flag |= INDEX_MEM_PARANOID;
> +#endif
>  		ret = index_mem(sha1, buf, size, type, path, flag);
>  		munmap(buf, size);
>  	} else
> -- 
> 1.7.0
>
> -- >8 --