From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
"Johannes Schindelin" <johannes.schindelin@gmx.de>,
"Carlo Marcelo Arenas Belón" <carenas@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 09/10] grep/pcre2: move back to thread-only PCREv2 structures
Date: Thu, 18 Feb 2021 01:07:27 +0100 [thread overview]
Message-ID: <20210218000728.13995-10-avarab@gmail.com> (raw)
In-Reply-To: <20210204210556.25242-1-avarab@gmail.com>
Change the setup of the "pcre2_general_context" to happen per-thread
in compile_pcre2_pattern() instead of in grep_init().
This change brings it in line with how the rest of the pcre2_* members
in the grep_pat structure are set up.
As noted in the preceding commit the approach 513f2b0bbd4 (grep: make
PCRE2 aware of custom allocator, 2019-10-16) took to allocate the
pcre2_general_context seems to have been initially based on a
misunderstanding of how PCREv2 memory allocation works.
The approach of creating a global context in grep_init() is just added
complexity for almost zero gain. On my system it's 24 bytes saved
per-thread. For comparison PCREv2 will then go on to allocate at least
a kilobyte for its own thread-local state.
As noted in 6d423dd542f (grep: don't redundantly compile throwaway
patterns under threading, 2017-05-25) the grep code is intentionally
not trying to micro-optimize allocations by e.g. sharing some PCREv2
structures globally, while making others thread-local.
So let's remove this special case and make all of them thread-local
again for simplicity. With this change we could move the
pcre2_{malloc,free} functions around to live closer to their current
use. I'm not doing that here to keep this change small, that cleanup
will be done in a follow-up commit.
See also the discussion in 94da9193a6 (grep: add support for PCRE v2,
2017-06-01) about thread safety, and Johannes's comments[1] to the
effect that we should be doing what this patch is doing.
1. https://lore.kernel.org/git/nycvar.QRO.7.76.6.1908052120302.46@tvgsbejvaqbjf.bet/
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
builtin/grep.c | 1 -
grep.c | 41 +++++++++++++++--------------------------
grep.h | 3 ++-
3 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 55d06c95130..c69fe993406 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1175,6 +1175,5 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
run_pager(&opt, prefix);
clear_pathspec(&pathspec);
free_grep_patterns(&opt);
- grep_destroy();
return !hit;
}
diff --git a/grep.c b/grep.c
index 0116ff5f09f..2599f329cd6 100644
--- a/grep.c
+++ b/grep.c
@@ -41,7 +41,6 @@ static struct grep_opt grep_defaults = {
};
#ifdef USE_LIBPCRE2
-static pcre2_general_context *pcre2_global_context;
#define GREP_PCRE2_DEBUG_MALLOC 0
static void *pcre2_malloc(PCRE2_SIZE size, MAYBE_UNUSED void *memory_data)
@@ -163,20 +162,9 @@ int grep_config(const char *var, const char *value, void *cb)
* Initialize one instance of grep_opt and copy the
* default values from the template we read the configuration
* information in an earlier call to git_config(grep_config).
- *
- * If using PCRE, make sure that the library is configured
- * to use the same allocator as Git (e.g. nedmalloc on Windows).
- *
- * Any allocated memory needs to be released in grep_destroy().
*/
void grep_init(struct grep_opt *opt, struct repository *repo, const char *prefix)
{
-#if defined(USE_LIBPCRE2)
- if (!pcre2_global_context)
- pcre2_global_context = pcre2_general_context_create(
- pcre2_malloc, pcre2_free, NULL);
-#endif
-
*opt = grep_defaults;
opt->repo = repo;
@@ -186,13 +174,6 @@ void grep_init(struct grep_opt *opt, struct repository *repo, const char *prefix
opt->header_tail = &opt->header_list;
}
-void grep_destroy(void)
-{
-#ifdef USE_LIBPCRE2
- pcre2_general_context_free(pcre2_global_context);
-#endif
-}
-
static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct grep_opt *opt)
{
/*
@@ -384,13 +365,20 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
int patinforet;
size_t jitsizearg;
- /* pcre2_global_context is initialized in grep_init */
+ /*
+ * Call pcre2_general_context_create() before calling any
+ * other pcre2_*(). It sets up our malloc()/free() functions
+ * with which everything else is allocated.
+ */
+ p->pcre2_general_context = pcre2_general_context_create(
+ pcre2_malloc, pcre2_free, NULL);
+ if (!p->pcre2_general_context)
+ die("Couldn't allocate PCRE2 general context");
+
if (opt->ignore_case) {
if (!opt->ignore_locale && has_non_ascii(p->pattern)) {
- if (!pcre2_global_context)
- BUG("pcre2_global_context uninitialized");
- p->pcre2_tables = pcre2_maketables(pcre2_global_context);
- p->pcre2_compile_context = pcre2_compile_context_create(pcre2_global_context);
+ p->pcre2_tables = pcre2_maketables(p->pcre2_general_context);
+ p->pcre2_compile_context = pcre2_compile_context_create(p->pcre2_general_context);
pcre2_set_character_tables(p->pcre2_compile_context,
p->pcre2_tables);
}
@@ -411,7 +399,7 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
p->pcre2_compile_context);
if (p->pcre2_pattern) {
- p->pcre2_match_data = pcre2_match_data_create_from_pattern(p->pcre2_pattern, pcre2_global_context);
+ p->pcre2_match_data = pcre2_match_data_create_from_pattern(p->pcre2_pattern, p->pcre2_general_context);
if (!p->pcre2_match_data)
die("Couldn't allocate PCRE2 match data");
} else {
@@ -491,10 +479,11 @@ static void free_pcre2_pattern(struct grep_pat *p)
pcre2_code_free(p->pcre2_pattern);
pcre2_match_data_free(p->pcre2_match_data);
#ifdef GIT_PCRE2_VERSION_10_34_OR_HIGHER
- pcre2_maketables_free(pcre2_global_context, p->pcre2_tables);
+ pcre2_maketables_free(p->pcre2_general_context, p->pcre2_tables);
#else
free((void *)p->pcre2_tables);
#endif
+ pcre2_general_context_free(p->pcre2_general_context);
}
#else /* !USE_LIBPCRE2 */
static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt)
diff --git a/grep.h b/grep.h
index 64666e92049..72f82b1e302 100644
--- a/grep.h
+++ b/grep.h
@@ -14,6 +14,7 @@
typedef int pcre2_code;
typedef int pcre2_match_data;
typedef int pcre2_compile_context;
+typedef int pcre2_general_context;
#endif
#ifndef PCRE2_MATCH_INVALID_UTF
/* PCRE2_MATCH_* dummy also with !USE_LIBPCRE2, for test-pcre2-config.c */
@@ -75,6 +76,7 @@ struct grep_pat {
pcre2_code *pcre2_pattern;
pcre2_match_data *pcre2_match_data;
pcre2_compile_context *pcre2_compile_context;
+ pcre2_general_context *pcre2_general_context;
const uint8_t *pcre2_tables;
uint32_t pcre2_jit_on;
unsigned fixed:1;
@@ -167,7 +169,6 @@ struct grep_opt {
int grep_config(const char *var, const char *value, void *);
void grep_init(struct grep_opt *, struct repository *repo, const char *prefix);
-void grep_destroy(void);
void grep_commit_pattern_type(enum grep_pattern_type, struct grep_opt *opt);
void append_grep_pat(struct grep_opt *opt, const char *pat, size_t patlen, const char *origin, int no, enum grep_pat_token t);
--
2.30.0.284.gd98b1dd5eaa7
next prev parent reply other threads:[~2021-02-18 0:09 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-16 12:06 [PATCH 0/3] Revive 'pcre2-chartables-leakfix' Johannes Schindelin via GitGitGadget
2019-10-16 12:06 ` [PATCH 1/3] grep: make PCRE1 aware of custom allocator Carlo Marcelo Arenas Belón via GitGitGadget
2019-10-16 12:06 ` [PATCH 2/3] grep: make PCRE2 " Carlo Marcelo Arenas Belón via GitGitGadget
2019-10-16 12:06 ` [PATCH 3/3] grep: avoid leak of chartables in PCRE2 Johannes Schindelin via GitGitGadget
2019-10-16 12:10 ` [PATCH v2 0/3] Revive 'pcre2-chartables-leakfix' Johannes Schindelin via GitGitGadget
2019-10-16 12:10 ` [PATCH v2 1/3] grep: make PCRE1 aware of custom allocator Carlo Marcelo Arenas Belón via GitGitGadget
2019-10-16 12:10 ` [PATCH v2 2/3] grep: make PCRE2 " Carlo Marcelo Arenas Belón via GitGitGadget
2019-10-18 1:38 ` Junio C Hamano
2021-02-04 21:05 ` [PATCH 00/10] grep/pcre2: memory allocation fixes Ævar Arnfjörð Bjarmason
2021-02-10 21:34 ` Junio C Hamano
2021-02-18 0:07 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2021-03-04 0:34 ` Junio C Hamano
2021-02-18 0:07 ` [PATCH v2 01/10] grep/pcre2: drop needless assignment + assert() on opt->pcre2 Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 02/10] grep/pcre2: drop needless assignment to NULL Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 03/10] grep/pcre2: correct reference to grep_init() in comment Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 04/10] grep/pcre2: prepare to add debugging to pcre2_malloc() Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 05/10] grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug mode Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 06/10] grep/pcre2: use compile-time PCREv2 version test Ævar Arnfjörð Bjarmason
2021-03-04 0:14 ` Junio C Hamano
2021-02-18 0:07 ` [PATCH v2 07/10] grep/pcre2: use pcre2_maketables_free() function Ævar Arnfjörð Bjarmason
2021-02-18 0:07 ` [PATCH v2 08/10] grep/pcre2: actually make pcre2 use custom allocator Ævar Arnfjörð Bjarmason
2021-03-04 0:24 ` Junio C Hamano
2021-02-18 0:07 ` Ævar Arnfjörð Bjarmason [this message]
2021-03-04 0:27 ` [PATCH v2 09/10] grep/pcre2: move back to thread-only PCREv2 structures Junio C Hamano
2021-02-18 0:07 ` [PATCH v2 10/10] grep/pcre2: move definitions of pcre2_{malloc,free} Ævar Arnfjörð Bjarmason
2021-03-04 0:28 ` Junio C Hamano
2021-02-04 21:05 ` [PATCH 01/10] grep/pcre2: drop needless assignment + assert() on opt->pcre2 Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 02/10] grep/pcre2: drop needless assignment to NULL Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 03/10] grep/pcre2: correct reference to grep_init() in comment Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 04/10] grep/pcre2: prepare to add debugging to pcre2_malloc() Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 05/10] grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug mode Ævar Arnfjörð Bjarmason
2021-02-10 10:38 ` Johannes Schindelin
2021-02-04 21:05 ` [PATCH 06/10] grep/pcre2: use compile-time PCREv2 version test Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 07/10] grep/pcre2: use pcre2_maketables_free() function Ævar Arnfjörð Bjarmason
2021-02-10 10:43 ` Johannes Schindelin
2021-03-04 0:16 ` Junio C Hamano
2021-02-04 21:05 ` [PATCH 08/10] grep/pcre2: actually make pcre2 use custom allocator Ævar Arnfjörð Bjarmason
2021-02-10 12:38 ` Johannes Schindelin
2021-02-04 21:05 ` [PATCH 09/10] grep/pcre2: move back to thread-only PCREv2 structures Ævar Arnfjörð Bjarmason
2021-02-04 21:05 ` [PATCH 10/10] grep/pcre2: move definitions of pcre2_{malloc,free} Ævar Arnfjörð Bjarmason
2021-02-10 12:40 ` Johannes Schindelin
2019-10-16 12:10 ` [PATCH v2 3/3] grep: avoid leak of chartables in PCRE2 Carlo Marcelo Arenas Belón via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210218000728.13995-10-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=carenas@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.