From: Thomas Rast <trast@student.ethz.ch>
To: Junio C Hamano <gitster@pobox.com>
Cc: <git@vger.kernel.org>
Subject: git_checkattr() is inefficient when repeated [Re: [PATCH 00/11] Micro-optimizing lookup_object()]
Date: Fri, 12 Aug 2011 17:59:24 +0200 [thread overview]
Message-ID: <201108121759.24884.trast@student.ethz.ch> (raw)
In-Reply-To: <1313085196-13249-1-git-send-email-gitster@pobox.com>
Junio C Hamano wrote:
> 4-way cuckoo
Cool stuff!
While looking at the performance of it, I noticed something odd about
packing: stracing the command you gave for your timings
strace -o pack.trace \
./git-pack-objects --count-only --keep-true-parents --honor-pack-keep \
--non-empty --all --reflog --no-reuse-delta --delta-base-offset \
--stdout </dev/null >/dev/null
yields the fairly crazy
$ grep -c 'open.*attrib' pack.trace
4398
including runs such as (with a line of context for clarity)
munmap(0x7f9cd39f7000, 4096) = 0
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("t/.gitattributes", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
So calling git_checkattr in a loop is quite inefficient. Indeed
there's a good optimization opportunity: compiled for 4-way hashing I
have (best of 3)
6.76user 0.23system 0:07.02elapsed 99%CPU (0avgtext+0avgdata 479792maxresident)
but making no_try_delta() in pack-objects.c a dummy 'return 0' gives
6.45user 0.13system 0:06.61elapsed 99%CPU (0avgtext+0avgdata 478256maxresident)
Which would be a 4.5% speedup. Obviously that won't quite be
attainable since we want the attributes mechanism to work, but we
still shouldn't have to open 4398 .gitattributes files when there are
only 8 .gitattributes plus one .git/info/attributes.
--
Thomas Rast
trast@{inf,student}.ethz.ch
next prev parent reply other threads:[~2011-08-12 15:59 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-08-11 17:53 [PATCH 00/11] Micro-optimizing lookup_object() Junio C Hamano
2011-08-11 17:53 ` [PATCH 01/11] object.c: code movement for readability Junio C Hamano
2011-08-11 17:53 ` [PATCH 02/11] object.c: remove duplicated code for object hashing Junio C Hamano
2011-08-11 17:53 ` [PATCH 03/11] pack-objects --count-only Junio C Hamano
2011-08-11 17:53 ` [PATCH 04/11] object: next_size() helper for readability Junio C Hamano
2011-08-11 17:53 ` [PATCH 05/11] object hash: we know the table size is a power of two Junio C Hamano
2011-08-11 17:53 ` [PATCH 06/11] object: growing the hash-table more aggressively does not help much Junio C Hamano
2011-08-11 17:53 ` [PATCH 07/11] object: try naive cuckoo hashing Junio C Hamano
2011-08-11 17:53 ` [PATCH 08/11] object: try 5-way cuckoo -- use all 20-bytes of SHA-1 Junio C Hamano
2011-08-11 17:53 ` [PATCH 09/11] object: try 4-way cuckoo Junio C Hamano
2011-08-11 17:53 ` [PATCH 10/11] object: try 3-way cuckoo Junio C Hamano
2011-08-11 17:53 ` [PATCH 11/11] object: try 2-way cuckoo again Junio C Hamano
2011-08-11 23:33 ` [FFT/PATCH 12/11] object.c: make object hash implementation more opaque Junio C Hamano
2011-08-12 15:59 ` Thomas Rast [this message]
2011-08-15 23:19 ` git_checkattr() is inefficient when repeated [Re: [PATCH 00/11] Micro-optimizing lookup_object()] Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201108121759.24884.trast@student.ethz.ch \
--to=trast@student.ethz.ch \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).