From: Eric Sunshine <sunshine@sunshineco.com>
To: Jeff King <peff@peff.net>
Cc: "Git List" <git@vger.kernel.org>, "René Scharfe" <l.s.r@web.de>,
"Rasmus Villemoes" <rv@rasmusvillemoes.dk>
Subject: Re: [PATCH 7/9] strbuf_getwholeline: use getdelim if it is available
Date: Fri, 17 Apr 2015 06:16:48 -0400 [thread overview]
Message-ID: <CAPig+cSKtMPQGxp1Y2GinVRh2y--QyJh_nxhDez2CGFPP6B=xg@mail.gmail.com> (raw)
In-Reply-To: <20150416090138.GG17938@peff.net>
On Thu, Apr 16, 2015 at 5:01 AM, Jeff King <peff@peff.net> wrote:
> We spend a lot of time in strbuf_getwholeline in a tight
> loop reading characters from a stdio handle into a buffer.
> The libc getdelim() function can do this for us with less
> overhead. It's in POSIX.1-2008, and was a GNU extension
> before that. Therefore we can't rely on it, but can fall
> back to the existing getc loop when it is not available.
>
> The HAVE_GETDELIM knob is turned on automatically for Linux,
> where we have glibc. We don't need to set any new
> feature-test macros, because we already define _GNU_SOURCE.
> Other systems that implement getdelim may need to other
> macros (probably _POSIX_C_SOURCE >= 200809L), but we can
> address that along with setting the Makefile knob after
> testing the feature on those systems.
> [...]
>
> Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> If somebody has a FreeBSD or OS X system to test on, I'd
> love to see what is needed to compile with HAVE_GETDELIM
> there.
Modern Mac OS X, 10.10.x Yosemite, has getdelim() and git builds fine
with HAVE_GETDELIM. I also tested on old Snow Leopard 10.5.8 from
2009. It does not have getdelim(). Unfortunately, I haven't been able
to determine when getdelim() was introduced on the Mac OS X, thus have
been unable to craft a simple rule for config.mak.uname.
> And to confirm that the performance is much better.
> Sharing my 1.6GB packed-refs file would be hard, but you
> should be able to generate something large and ridiculous.
> I'll leave that as an exercise to the reader.
>
> diff --git a/Makefile b/Makefile
> index 5f3987f..36655d5 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -359,6 +359,8 @@ all::
> # compiler is detected to support it.
> #
> # Define HAVE_BSD_SYSCTL if your platform has a BSD-compatible sysctl function.
> +#
> +# Define HAVE_GETDELIM if your system has the getdelim() function.
>
> GIT-VERSION-FILE: FORCE
> @$(SHELL_PATH) ./GIT-VERSION-GEN
> @@ -1437,6 +1439,10 @@ ifdef HAVE_BSD_SYSCTL
> BASIC_CFLAGS += -DHAVE_BSD_SYSCTL
> endif
>
> +ifdef HAVE_GETDELIM
> + BASIC_CFLAGS += -DHAVE_GETDELIM
> +endif
> +
> ifeq ($(TCLTK_PATH),)
> NO_TCLTK = NoThanks
> endif
> diff --git a/config.mak.uname b/config.mak.uname
> index f4e77cb..d26665f 100644
> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -36,6 +36,7 @@ ifeq ($(uname_S),Linux)
> HAVE_DEV_TTY = YesPlease
> HAVE_CLOCK_GETTIME = YesPlease
> HAVE_CLOCK_MONOTONIC = YesPlease
> + HAVE_GETDELIM = YesPlease
> endif
> ifeq ($(uname_S),GNU/kFreeBSD)
> HAVE_ALLOCA_H = YesPlease
next prev parent reply other threads:[~2015-04-17 10:16 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-05 1:06 [PATCH 0/6] address packed-refs speed regressions Jeff King
2015-04-05 1:07 ` [PATCH 1/6] strbuf_getwholeline: use getc macro Jeff King
2015-04-05 1:08 ` [PATCH 2/6] git-compat-util: add fallbacks for unlocked stdio Jeff King
2015-04-05 1:11 ` [PATCH 3/6] strbuf_getwholeline: use getc_unlocked Jeff King
2015-04-05 4:56 ` Jeff King
2015-04-05 5:27 ` Jeff King
2015-04-05 5:35 ` Jeff King
2015-04-05 20:49 ` Junio C Hamano
2015-04-05 14:36 ` Duy Nguyen
2015-04-05 18:24 ` Jeff King
2015-04-05 20:09 ` Junio C Hamano
2015-04-07 13:48 ` Rasmus Villemoes
2015-04-07 19:04 ` Jeff King
2015-04-07 22:43 ` Rasmus Villemoes
2015-04-08 0:17 ` Jeff King
2015-04-05 1:11 ` [PATCH 4/6] strbuf: add an optimized 1-character strbuf_grow Jeff King
2015-04-06 2:13 ` Eric Sunshine
2015-04-06 5:05 ` Jeff King
2015-04-05 1:11 ` [PATCH 5/6] t1430: add another refs-escape test Jeff King
2015-04-05 1:15 ` [PATCH 6/6] refname_is_safe: avoid expensive normalize_path_copy call Jeff King
2015-04-05 13:41 ` [PATCH 0/6] address packed-refs speed regressions René Scharfe
2015-04-05 18:52 ` Jeff King
2015-04-05 18:59 ` Jeff King
2015-04-05 23:04 ` René Scharfe
2015-04-05 22:39 ` René Scharfe
2015-04-06 4:49 ` Jeff King
2015-04-16 8:47 ` [PATCH v2 0/9] " Jeff King
2015-04-16 8:48 ` [PATCH 1/9] strbuf_getwholeline: use getc macro Jeff King
2015-04-16 8:48 ` [PATCH 2/9] git-compat-util: add fallbacks for unlocked stdio Jeff King
2015-04-16 8:49 ` [PATCH 3/9] strbuf_getwholeline: use getc_unlocked Jeff King
2015-04-16 8:51 ` [PATCH 4/9] config: use getc_unlocked when reading from file Jeff King
2015-04-16 8:53 ` [PATCH 5/9] strbuf_addch: avoid calling strbuf_grow Jeff King
2015-04-16 8:58 ` [PATCH 6/9] strbuf_getwholeline: " Jeff King
2015-04-16 9:01 ` [PATCH 7/9] strbuf_getwholeline: use getdelim if it is available Jeff King
2015-04-17 10:16 ` Eric Sunshine [this message]
2015-04-21 23:09 ` Jeff King
2015-05-08 23:56 ` Eric Sunshine
2015-05-09 1:09 ` Jeff King
2015-06-02 18:22 ` Eric Sunshine
2015-04-22 18:00 ` Johannes Schindelin
2015-04-22 18:06 ` Jeff King
2015-04-16 9:03 ` [PATCH 8/9] read_packed_refs: avoid double-checking sane refs Jeff King
2015-04-16 9:04 ` [PATCH 9/9] t1430: add another refs-escape test Jeff King
2015-04-16 9:25 ` [PATCH v2 0/9] address packed-refs speed regressions Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAPig+cSKtMPQGxp1Y2GinVRh2y--QyJh_nxhDez2CGFPP6B=xg@mail.gmail.com' \
--to=sunshine@sunshineco.com \
--cc=git@vger.kernel.org \
--cc=l.s.r@web.de \
--cc=peff@peff.net \
--cc=rv@rasmusvillemoes.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).