git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Sunshine <sunshine@sunshineco.com>
To: Jeff King <peff@peff.net>
Cc: "Git List" <git@vger.kernel.org>, "René Scharfe" <l.s.r@web.de>,
	"Rasmus Villemoes" <rv@rasmusvillemoes.dk>
Subject: Re: [PATCH 7/9] strbuf_getwholeline: use getdelim if it is available
Date: Fri, 17 Apr 2015 06:16:48 -0400	[thread overview]
Message-ID: <CAPig+cSKtMPQGxp1Y2GinVRh2y--QyJh_nxhDez2CGFPP6B=xg@mail.gmail.com> (raw)
In-Reply-To: <20150416090138.GG17938@peff.net>

On Thu, Apr 16, 2015 at 5:01 AM, Jeff King <peff@peff.net> wrote:
> We spend a lot of time in strbuf_getwholeline in a tight
> loop reading characters from a stdio handle into a buffer.
> The libc getdelim() function can do this for us with less
> overhead. It's in POSIX.1-2008, and was a GNU extension
> before that. Therefore we can't rely on it, but can fall
> back to the existing getc loop when it is not available.
>
> The HAVE_GETDELIM knob is turned on automatically for Linux,
> where we have glibc. We don't need to set any new
> feature-test macros, because we already define _GNU_SOURCE.
> Other systems that implement getdelim may need to other
> macros (probably _POSIX_C_SOURCE >= 200809L), but we can
> address that along with setting the Makefile knob after
> testing the feature on those systems.
> [...]
>
> Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> If somebody has a FreeBSD or OS X system to test on, I'd
> love to see what is needed to compile with HAVE_GETDELIM
> there.

Modern Mac OS X, 10.10.x Yosemite, has getdelim() and git builds fine
with HAVE_GETDELIM. I also tested on old Snow Leopard 10.5.8 from
2009. It does not have getdelim(). Unfortunately, I haven't been able
to determine when getdelim() was introduced on the Mac OS X, thus have
been unable to craft a simple rule for config.mak.uname.

> And to confirm that the performance is much better.
> Sharing my 1.6GB packed-refs file would be hard, but you
> should be able to generate something large and ridiculous.
> I'll leave that as an exercise to the reader.
>
> diff --git a/Makefile b/Makefile
> index 5f3987f..36655d5 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -359,6 +359,8 @@ all::
>  # compiler is detected to support it.
>  #
>  # Define HAVE_BSD_SYSCTL if your platform has a BSD-compatible sysctl function.
> +#
> +# Define HAVE_GETDELIM if your system has the getdelim() function.
>
>  GIT-VERSION-FILE: FORCE
>         @$(SHELL_PATH) ./GIT-VERSION-GEN
> @@ -1437,6 +1439,10 @@ ifdef HAVE_BSD_SYSCTL
>         BASIC_CFLAGS += -DHAVE_BSD_SYSCTL
>  endif
>
> +ifdef HAVE_GETDELIM
> +       BASIC_CFLAGS += -DHAVE_GETDELIM
> +endif
> +
>  ifeq ($(TCLTK_PATH),)
>  NO_TCLTK = NoThanks
>  endif
> diff --git a/config.mak.uname b/config.mak.uname
> index f4e77cb..d26665f 100644
> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -36,6 +36,7 @@ ifeq ($(uname_S),Linux)
>         HAVE_DEV_TTY = YesPlease
>         HAVE_CLOCK_GETTIME = YesPlease
>         HAVE_CLOCK_MONOTONIC = YesPlease
> +       HAVE_GETDELIM = YesPlease
>  endif
>  ifeq ($(uname_S),GNU/kFreeBSD)
>         HAVE_ALLOCA_H = YesPlease

  reply	other threads:[~2015-04-17 10:16 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-05  1:06 [PATCH 0/6] address packed-refs speed regressions Jeff King
2015-04-05  1:07 ` [PATCH 1/6] strbuf_getwholeline: use getc macro Jeff King
2015-04-05  1:08 ` [PATCH 2/6] git-compat-util: add fallbacks for unlocked stdio Jeff King
2015-04-05  1:11 ` [PATCH 3/6] strbuf_getwholeline: use getc_unlocked Jeff King
2015-04-05  4:56   ` Jeff King
2015-04-05  5:27     ` Jeff King
2015-04-05  5:35       ` Jeff King
2015-04-05 20:49         ` Junio C Hamano
2015-04-05 14:36     ` Duy Nguyen
2015-04-05 18:24       ` Jeff King
2015-04-05 20:09     ` Junio C Hamano
2015-04-07 13:48     ` Rasmus Villemoes
2015-04-07 19:04       ` Jeff King
2015-04-07 22:43         ` Rasmus Villemoes
2015-04-08  0:17           ` Jeff King
2015-04-05  1:11 ` [PATCH 4/6] strbuf: add an optimized 1-character strbuf_grow Jeff King
2015-04-06  2:13   ` Eric Sunshine
2015-04-06  5:05     ` Jeff King
2015-04-05  1:11 ` [PATCH 5/6] t1430: add another refs-escape test Jeff King
2015-04-05  1:15 ` [PATCH 6/6] refname_is_safe: avoid expensive normalize_path_copy call Jeff King
2015-04-05 13:41 ` [PATCH 0/6] address packed-refs speed regressions René Scharfe
2015-04-05 18:52   ` Jeff King
2015-04-05 18:59     ` Jeff King
2015-04-05 23:04       ` René Scharfe
2015-04-05 22:39     ` René Scharfe
2015-04-06  4:49       ` Jeff King
2015-04-16  8:47 ` [PATCH v2 0/9] " Jeff King
2015-04-16  8:48   ` [PATCH 1/9] strbuf_getwholeline: use getc macro Jeff King
2015-04-16  8:48   ` [PATCH 2/9] git-compat-util: add fallbacks for unlocked stdio Jeff King
2015-04-16  8:49   ` [PATCH 3/9] strbuf_getwholeline: use getc_unlocked Jeff King
2015-04-16  8:51   ` [PATCH 4/9] config: use getc_unlocked when reading from file Jeff King
2015-04-16  8:53   ` [PATCH 5/9] strbuf_addch: avoid calling strbuf_grow Jeff King
2015-04-16  8:58   ` [PATCH 6/9] strbuf_getwholeline: " Jeff King
2015-04-16  9:01   ` [PATCH 7/9] strbuf_getwholeline: use getdelim if it is available Jeff King
2015-04-17 10:16     ` Eric Sunshine [this message]
2015-04-21 23:09       ` Jeff King
2015-05-08 23:56         ` Eric Sunshine
2015-05-09  1:09           ` Jeff King
2015-06-02 18:22             ` Eric Sunshine
2015-04-22 18:00       ` Johannes Schindelin
2015-04-22 18:06         ` Jeff King
2015-04-16  9:03   ` [PATCH 8/9] read_packed_refs: avoid double-checking sane refs Jeff King
2015-04-16  9:04   ` [PATCH 9/9] t1430: add another refs-escape test Jeff King
2015-04-16  9:25   ` [PATCH v2 0/9] address packed-refs speed regressions Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAPig+cSKtMPQGxp1Y2GinVRh2y--QyJh_nxhDez2CGFPP6B=xg@mail.gmail.com' \
    --to=sunshine@sunshineco.com \
    --cc=git@vger.kernel.org \
    --cc=l.s.r@web.de \
    --cc=peff@peff.net \
    --cc=rv@rasmusvillemoes.dk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).