* [PATCH 1/9] sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one()
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
@ 2014-06-30 16:55 ` Jeff King
2014-06-30 16:57 ` [PATCH 2/9] add strip_suffix function Jeff King
` (8 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:55 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
From: René Scharfe <l.s.r@web.de>
Instead of using strbuf to create a message string in case a path is
too long for our fixed-size buffer, replace that buffer with a strbuf
and thus get rid of the limitation.
Helped-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
---
sha1_file.c | 42 ++++++++++++++++--------------------------
1 file changed, 16 insertions(+), 26 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 34d527f..394fa45 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1177,48 +1177,37 @@ static void report_pack_garbage(struct string_list *list)
static void prepare_packed_git_one(char *objdir, int local)
{
- /* Ensure that this buffer is large enough so that we can
- append "/pack/" without clobbering the stack even if
- strlen(objdir) were PATH_MAX. */
- char path[PATH_MAX + 1 + 4 + 1 + 1];
- int len;
+ struct strbuf path = STRBUF_INIT;
+ size_t dirnamelen;
DIR *dir;
struct dirent *de;
struct string_list garbage = STRING_LIST_INIT_DUP;
- sprintf(path, "%s/pack", objdir);
- len = strlen(path);
- dir = opendir(path);
+ strbuf_addstr(&path, objdir);
+ strbuf_addstr(&path, "/pack");
+ dir = opendir(path.buf);
if (!dir) {
if (errno != ENOENT)
error("unable to open object pack directory: %s: %s",
- path, strerror(errno));
+ path.buf, strerror(errno));
+ strbuf_release(&path);
return;
}
- path[len++] = '/';
+ strbuf_addch(&path, '/');
+ dirnamelen = path.len;
while ((de = readdir(dir)) != NULL) {
- int namelen = strlen(de->d_name);
struct packed_git *p;
- if (len + namelen + 1 > sizeof(path)) {
- if (report_garbage) {
- struct strbuf sb = STRBUF_INIT;
- strbuf_addf(&sb, "%.*s/%s", len - 1, path, de->d_name);
- report_garbage("path too long", sb.buf);
- strbuf_release(&sb);
- }
- continue;
- }
-
if (is_dot_or_dotdot(de->d_name))
continue;
- strcpy(path + len, de->d_name);
+ strbuf_setlen(&path, dirnamelen);
+ strbuf_addstr(&path, de->d_name);
if (has_extension(de->d_name, ".idx")) {
/* Don't reopen a pack we already have. */
for (p = packed_git; p; p = p->next) {
- if (!memcmp(path, p->pack_name, len + namelen - 4))
+ if (!memcmp(path.buf, p->pack_name, path.len - 4))
break;
}
if (p == NULL &&
@@ -1226,7 +1215,7 @@ static void prepare_packed_git_one(char *objdir, int local)
* See if it really is a valid .idx file with
* corresponding .pack file that we can map.
*/
- (p = add_packed_git(path, len + namelen, local)) != NULL)
+ (p = add_packed_git(path.buf, path.len, local)) != NULL)
install_packed_git(p);
}
@@ -1237,13 +1226,14 @@ static void prepare_packed_git_one(char *objdir, int local)
has_extension(de->d_name, ".pack") ||
has_extension(de->d_name, ".bitmap") ||
has_extension(de->d_name, ".keep"))
- string_list_append(&garbage, path);
+ string_list_append(&garbage, path.buf);
else
- report_garbage("garbage found", path);
+ report_garbage("garbage found", path.buf);
}
closedir(dir);
report_pack_garbage(&garbage);
string_list_clear(&garbage, 0);
+ strbuf_release(&path);
}
static int sort_pack(const void *a_, const void *b_)
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 2/9] add strip_suffix function
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
2014-06-30 16:55 ` [PATCH 1/9] sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one() Jeff King
@ 2014-06-30 16:57 ` Jeff King
2014-07-02 15:54 ` Junio C Hamano
2014-06-30 16:58 ` [PATCH 3/9] implement ends_with via strip_suffix Jeff King
` (7 subsequent siblings)
9 siblings, 1 reply; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:57 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
Many callers of ends_with want to not only find out whether
a string has a suffix, but want to also strip it off. Doing
that separately has two minor problems:
1. We often run over the string twice (once to find
the suffix, and then once more to find its length to
subtract the suffix length).
2. We have to specify the suffix length again, which means
either a magic number, or repeating ourselves with
strlen("suffix").
Just as we have skip_prefix to avoid these cases with
starts_with, we can add a strip_suffix to avoid them with
ends_with.
Note that we add two forms of strip_suffix here: one that
takes a string, with the resulting length as an
out-parameter; and one that takes a pointer/length pair, and
reuses the length as an out-parameter. The latter is more
efficient when the caller already has the length (e.g., when
using strbufs), but it can be easy to confuse the two, as
they take the same number and types of parameters.
For that reason, the "mem" form puts its length parameter
next to the buffer (since they are a pair), and the string
form puts it at the end (since it is an out-parameter). The
compiler can notice when you get the order wrong, which
should help prevent writing one when you meant the other.
Signed-off-by: Jeff King <peff@peff.net>
---
I hope the word "strip" is OK, as it does not actually NUL-terminate
(doing so would make it unusable for many cases). Between the comment
below and the "const" in the parameter, I think it should be pretty
clear that it does not touch the string. And I could not think of a
better word.
git-compat-util.h | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/git-compat-util.h b/git-compat-util.h
index b6f03b3..d044c42 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -358,6 +358,33 @@ static inline const char *skip_prefix(const char *str, const char *prefix)
return NULL;
}
+/*
+ * If buf ends with suffix, return 1 and subtract the length of the suffix
+ * from *len. Otherwise, return 0 and leave *len untouched.
+ */
+static inline int strip_suffix_mem(const char *buf, size_t *len,
+ const char *suffix)
+{
+ size_t suflen = strlen(suffix);
+ if (*len < suflen || memcmp(buf + (*len - suflen), suffix, suflen))
+ return 0;
+ *len -= suflen;
+ return 1;
+}
+
+/*
+ * If str ends with suffix, return 1 and set *len to the size of the string
+ * without the suffix. Otherwise, return 0 and set *len to the size of the
+ * string.
+ *
+ * Note that we do _not_ NUL-terminate str to the new length.
+ */
+static inline int strip_suffix(const char *str, const char *suffix, size_t *len)
+{
+ *len = strlen(str);
+ return strip_suffix_mem(str, len, suffix);
+}
+
#if defined(NO_MMAP) || defined(USE_WIN32_MMAP)
#ifndef PROT_READ
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 2/9] add strip_suffix function
2014-06-30 16:57 ` [PATCH 2/9] add strip_suffix function Jeff King
@ 2014-07-02 15:54 ` Junio C Hamano
2014-07-02 16:38 ` Jeff King
0 siblings, 1 reply; 27+ messages in thread
From: Junio C Hamano @ 2014-07-02 15:54 UTC (permalink / raw)
To: Jeff King; +Cc: René Scharfe, Duy Nguyen, Git Mailing List
Jeff King <peff@peff.net> writes:
> For that reason, the "mem" form puts its length parameter
> next to the buffer (since they are a pair), and the string
> form puts it at the end (since it is an out-parameter). The
> compiler can notice when you get the order wrong, which
> should help prevent writing one when you meant the other.
Very sensible consideration. I like commits that careful thinking
behind them shows through them.
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> I hope the word "strip" is OK, as it does not actually NUL-terminate
> (doing so would make it unusable for many cases). Between the comment
> below and the "const" in the parameter, I think it should be pretty
> clear that it does not touch the string. And I could not think of a
> better word.
All other words I can think of offhand, trim, chomp, etc., hint
shortening of the input string, and by definition shortening of a
string implies NUL-termination.
The "mem" variant deals with a counted string, however, so its
shortening implies NUL-termination a lot less [*1*] and should be
fine.
If we want to avoid implying NUL-termination, the only way to do so
would be to use wording that does not hint shortening. At least for
the C-string variant, which is measuring the length of the basename
part (i.e. `basename $str $suffix`) without touching anything else,
e.g. basename_length("hello.c", ".c", &len), but at the same time
you want to make it a boolean to signal if the string ends with the
suffix, so perhaps has_suffix("hello.c", ".c", &len)?
[Footnote]
*1* ... but not entirely, because we often NUL-terminate even
counted strings (the buffer returned from read_sha1_file() and
the payload of strbuf are two examples).
> git-compat-util.h | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
>
> diff --git a/git-compat-util.h b/git-compat-util.h
> index b6f03b3..d044c42 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -358,6 +358,33 @@ static inline const char *skip_prefix(const char *str, const char *prefix)
> return NULL;
> }
>
> +/*
> + * If buf ends with suffix, return 1 and subtract the length of the suffix
> + * from *len. Otherwise, return 0 and leave *len untouched.
> + */
> +static inline int strip_suffix_mem(const char *buf, size_t *len,
> + const char *suffix)
> +{
> + size_t suflen = strlen(suffix);
> + if (*len < suflen || memcmp(buf + (*len - suflen), suffix, suflen))
> + return 0;
> + *len -= suflen;
> + return 1;
> +}
> +
> +/*
> + * If str ends with suffix, return 1 and set *len to the size of the string
> + * without the suffix. Otherwise, return 0 and set *len to the size of the
> + * string.
> + *
> + * Note that we do _not_ NUL-terminate str to the new length.
> + */
> +static inline int strip_suffix(const char *str, const char *suffix, size_t *len)
> +{
> + *len = strlen(str);
> + return strip_suffix_mem(str, len, suffix);
> +}
> +
> #if defined(NO_MMAP) || defined(USE_WIN32_MMAP)
>
> #ifndef PROT_READ
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 2/9] add strip_suffix function
2014-07-02 15:54 ` Junio C Hamano
@ 2014-07-02 16:38 ` Jeff King
2014-07-02 17:41 ` Junio C Hamano
0 siblings, 1 reply; 27+ messages in thread
From: Jeff King @ 2014-07-02 16:38 UTC (permalink / raw)
To: Junio C Hamano; +Cc: René Scharfe, Duy Nguyen, Git Mailing List
On Wed, Jul 02, 2014 at 08:54:44AM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > For that reason, the "mem" form puts its length parameter
> > next to the buffer (since they are a pair), and the string
> > form puts it at the end (since it is an out-parameter). The
> > compiler can notice when you get the order wrong, which
> > should help prevent writing one when you meant the other.
>
> Very sensible consideration. I like commits that careful thinking
> behind them shows through them.
I would like to take credit for advanced thinking, but I actually did
what felt natural, and only noticed the "compiler will tell you when you
are wrong" effect when I got it wrong while writing a later patch in the
series. :)
> If we want to avoid implying NUL-termination, the only way to do so
> would be to use wording that does not hint shortening. At least for
> the C-string variant, which is measuring the length of the basename
> part (i.e. `basename $str $suffix`) without touching anything else,
> e.g. basename_length("hello.c", ".c", &len), but at the same time
> you want to make it a boolean to signal if the string ends with the
> suffix, so perhaps has_suffix("hello.c", ".c", &len)?
I think that invites some confusion with "ends_with", which is the same
thing (but just does not take the "len" parameter). We could just add
this feature to ends_with, and ask callers who do not care to pass NULL,
but that makes those call sites uglier.
Having had a day to mull it over, and having read your email, I think I
still prefer strip_suffix.
-Peff
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 3/9] implement ends_with via strip_suffix
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
2014-06-30 16:55 ` [PATCH 1/9] sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one() Jeff King
2014-06-30 16:57 ` [PATCH 2/9] add strip_suffix function Jeff King
@ 2014-06-30 16:58 ` Jeff King
2014-06-30 16:58 ` [PATCH 4/9] replace has_extension with ends_with Jeff King
` (6 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:58 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
The ends_with function is essentially a simplified version
of strip_suffix, in which we throw away the stripped length.
Implementing it as an inline on top of strip_suffix has two
advantages:
1. We save a bit of duplicated code.
2. The suffix is typically a string literal, and we call
strlen on it. By making the function inline, many
compilers can replace the strlen call with a constant.
Signed-off-by: Jeff King <peff@peff.net>
---
git-compat-util.h | 7 ++++++-
strbuf.c | 9 ---------
2 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/git-compat-util.h b/git-compat-util.h
index d044c42..4cfde49 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -347,7 +347,6 @@ extern void set_error_routine(void (*routine)(const char *err, va_list params));
extern void set_die_is_recursing_routine(int (*routine)(void));
extern int starts_with(const char *str, const char *prefix);
-extern int ends_with(const char *str, const char *suffix);
static inline const char *skip_prefix(const char *str, const char *prefix)
{
@@ -385,6 +384,12 @@ static inline int strip_suffix(const char *str, const char *suffix, size_t *len)
return strip_suffix_mem(str, len, suffix);
}
+static inline int ends_with(const char *str, const char *suffix)
+{
+ size_t len;
+ return strip_suffix(str, suffix, &len);
+}
+
#if defined(NO_MMAP) || defined(USE_WIN32_MMAP)
#ifndef PROT_READ
diff --git a/strbuf.c b/strbuf.c
index ac62982..99dbeba 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -11,15 +11,6 @@ int starts_with(const char *str, const char *prefix)
return 0;
}
-int ends_with(const char *str, const char *suffix)
-{
- int len = strlen(str), suflen = strlen(suffix);
- if (len < suflen)
- return 0;
- else
- return !strcmp(str + len - suflen, suffix);
-}
-
/*
* Used as the default ->buf value, so that people can always assume
* buf is non NULL and ->buf is NUL terminated even for a freshly
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 4/9] replace has_extension with ends_with
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (2 preceding siblings ...)
2014-06-30 16:58 ` [PATCH 3/9] implement ends_with via strip_suffix Jeff King
@ 2014-06-30 16:58 ` Jeff King
2014-06-30 16:58 ` [PATCH 5/9] use strip_suffix instead of ends_with in simple cases Jeff King
` (5 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:58 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
These two are almost the same function, with the exception
that has_extension only matches if there is content before
the suffix. So ends_with(".exe", ".exe") is true, but
has_extension would not be.
This distinction does not matter to any of the callers,
though, and we can just replace uses of has_extension with
ends_with. We prefer the "ends_with" name because it is more
generic, and there is nothing about the function that
requires it to be used for file extensions.
Signed-off-by: Jeff King <peff@peff.net>
---
builtin/index-pack.c | 4 ++--
builtin/verify-pack.c | 4 ++--
git-compat-util.h | 7 -------
help.c | 2 +-
refs.c | 4 ++--
sha1_file.c | 10 +++++-----
6 files changed, 12 insertions(+), 19 deletions(-)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 18f57de..46376b6 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1603,7 +1603,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
die(_("--fix-thin cannot be used without --stdin"));
if (!index_name && pack_name) {
int len = strlen(pack_name);
- if (!has_extension(pack_name, ".pack"))
+ if (!ends_with(pack_name, ".pack"))
die(_("packfile name '%s' does not end with '.pack'"),
pack_name);
index_name_buf = xmalloc(len);
@@ -1613,7 +1613,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
}
if (keep_msg && !keep_name && pack_name) {
int len = strlen(pack_name);
- if (!has_extension(pack_name, ".pack"))
+ if (!ends_with(pack_name, ".pack"))
die(_("packfile name '%s' does not end with '.pack'"),
pack_name);
keep_name_buf = xmalloc(len);
diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index 66cd2df..2fd29ce 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -27,9 +27,9 @@ static int verify_one_pack(const char *path, unsigned int flags)
* normalize these forms to "foo.pack" for "index-pack --verify".
*/
strbuf_addstr(&arg, path);
- if (has_extension(arg.buf, ".idx"))
+ if (ends_with(arg.buf, ".idx"))
strbuf_splice(&arg, arg.len - 3, 3, "pack", 4);
- else if (!has_extension(arg.buf, ".pack"))
+ else if (!ends_with(arg.buf, ".pack"))
strbuf_add(&arg, ".pack", 5);
argv[2] = arg.buf;
diff --git a/git-compat-util.h b/git-compat-util.h
index 4cfde49..8cf79ae 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -594,13 +594,6 @@ static inline size_t xsize_t(off_t len)
return (size_t)len;
}
-static inline int has_extension(const char *filename, const char *ext)
-{
- size_t len = strlen(filename);
- size_t extlen = strlen(ext);
- return len > extlen && !memcmp(filename + len - extlen, ext, extlen);
-}
-
/* in ctype.c, for kwset users */
extern const char tolower_trans_tbl[256];
diff --git a/help.c b/help.c
index b266b09..372728f 100644
--- a/help.c
+++ b/help.c
@@ -156,7 +156,7 @@ static void list_commands_in_dir(struct cmdnames *cmds,
continue;
entlen = strlen(de->d_name) - prefix_len;
- if (has_extension(de->d_name, ".exe"))
+ if (ends_with(de->d_name, ".exe"))
entlen -= 4;
add_cmdname(cmds, de->d_name + prefix_len, entlen);
diff --git a/refs.c b/refs.c
index dc45774..e31adfd 100644
--- a/refs.c
+++ b/refs.c
@@ -1162,7 +1162,7 @@ static void read_loose_refs(const char *dirname, struct ref_dir *dir)
if (de->d_name[0] == '.')
continue;
- if (has_extension(de->d_name, ".lock"))
+ if (ends_with(de->d_name, ".lock"))
continue;
strbuf_addstr(&refname, de->d_name);
refdir = *refs->name
@@ -3233,7 +3233,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
if (de->d_name[0] == '.')
continue;
- if (has_extension(de->d_name, ".lock"))
+ if (ends_with(de->d_name, ".lock"))
continue;
strbuf_addstr(name, de->d_name);
if (stat(git_path("logs/%s", name->buf), &st) < 0) {
diff --git a/sha1_file.c b/sha1_file.c
index 394fa45..93b794f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1204,7 +1204,7 @@ static void prepare_packed_git_one(char *objdir, int local)
strbuf_setlen(&path, dirnamelen);
strbuf_addstr(&path, de->d_name);
- if (has_extension(de->d_name, ".idx")) {
+ if (ends_with(de->d_name, ".idx")) {
/* Don't reopen a pack we already have. */
for (p = packed_git; p; p = p->next) {
if (!memcmp(path.buf, p->pack_name, path.len - 4))
@@ -1222,10 +1222,10 @@ static void prepare_packed_git_one(char *objdir, int local)
if (!report_garbage)
continue;
- if (has_extension(de->d_name, ".idx") ||
- has_extension(de->d_name, ".pack") ||
- has_extension(de->d_name, ".bitmap") ||
- has_extension(de->d_name, ".keep"))
+ if (ends_with(de->d_name, ".idx") ||
+ ends_with(de->d_name, ".pack") ||
+ ends_with(de->d_name, ".bitmap") ||
+ ends_with(de->d_name, ".keep"))
string_list_append(&garbage, path.buf);
else
report_garbage("garbage found", path.buf);
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 5/9] use strip_suffix instead of ends_with in simple cases
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (3 preceding siblings ...)
2014-06-30 16:58 ` [PATCH 4/9] replace has_extension with ends_with Jeff King
@ 2014-06-30 16:58 ` Jeff King
2014-06-30 16:59 ` [PATCH 6/9] index-pack: use strip_suffix to avoid magic numbers Jeff King
` (4 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:58 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
When stripping a suffix like:
if (ends_with(str, "foo"))
buf = xmemdupz(str, strlen(str) - 3);
we can instead use strip_suffix to avoid the constant 3,
which must match the literal "foo" (we sometimes use
strlen("foo") instead, but that means we are repeating
ourselves). The example above becomes:
if (strip_suffix(str, "foo", &len))
buf = xmemdupz(str, len);
This also saves a strlen(), since we calculate the string
length when detecting the suffix.
Note that in some cases we also switch from xstrndup to
xmemdupz, which saves a further strlen call.
Signed-off-by: Jeff King <peff@peff.net>
---
builtin/remote.c | 13 +++++++------
builtin/repack.c | 5 ++---
connected.c | 6 +++---
help.c | 5 ++---
4 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index c9102e8..7c2999e 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -265,16 +265,17 @@ static int config_read_branches(const char *key, const char *value, void *cb)
struct string_list_item *item;
struct branch_info *info;
enum { REMOTE, MERGE, REBASE } type;
+ size_t key_len;
key += 7;
- if (ends_with(key, ".remote")) {
- name = xstrndup(key, strlen(key) - 7);
+ if (strip_suffix(key, ".remote", &key_len)) {
+ name = xmemdupz(key, key_len);
type = REMOTE;
- } else if (ends_with(key, ".merge")) {
- name = xstrndup(key, strlen(key) - 6);
+ } else if (strip_suffix(key, ".merge", &key_len)) {
+ name = xmemdupz(key, key_len);
type = MERGE;
- } else if (ends_with(key, ".rebase")) {
- name = xstrndup(key, strlen(key) - 7);
+ } else if (strip_suffix(key, ".rebase", &key_len)) {
+ name = xmemdupz(key, key_len);
type = REBASE;
} else
return 0;
diff --git a/builtin/repack.c b/builtin/repack.c
index ff2216a..a77e743 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -83,16 +83,15 @@ static void get_non_kept_pack_filenames(struct string_list *fname_list)
DIR *dir;
struct dirent *e;
char *fname;
- size_t len;
if (!(dir = opendir(packdir)))
return;
while ((e = readdir(dir)) != NULL) {
- if (!ends_with(e->d_name, ".pack"))
+ size_t len;
+ if (!strip_suffix(e->d_name, ".pack", &len))
continue;
- len = strlen(e->d_name) - strlen(".pack");
fname = xmemdupz(e->d_name, len);
if (!file_exists(mkpath("%s/%s.keep", packdir, fname)))
diff --git a/connected.c b/connected.c
index be0253e..dae9c99 100644
--- a/connected.c
+++ b/connected.c
@@ -31,6 +31,7 @@ static int check_everything_connected_real(sha1_iterate_fn fn,
unsigned char sha1[20];
int err = 0, ac = 0;
struct packed_git *new_pack = NULL;
+ size_t base_len;
if (fn(cb_data, sha1))
return err;
@@ -38,10 +39,9 @@ static int check_everything_connected_real(sha1_iterate_fn fn,
if (transport && transport->smart_options &&
transport->smart_options->self_contained_and_connected &&
transport->pack_lockfile &&
- ends_with(transport->pack_lockfile, ".keep")) {
+ strip_suffix(transport->pack_lockfile, ".keep", &base_len)) {
struct strbuf idx_file = STRBUF_INIT;
- strbuf_addstr(&idx_file, transport->pack_lockfile);
- strbuf_setlen(&idx_file, idx_file.len - 5); /* ".keep" */
+ strbuf_add(&idx_file, transport->pack_lockfile, base_len);
strbuf_addstr(&idx_file, ".idx");
new_pack = add_packed_git(idx_file.buf, idx_file.len, 1);
strbuf_release(&idx_file);
diff --git a/help.c b/help.c
index 372728f..97567c4 100644
--- a/help.c
+++ b/help.c
@@ -145,7 +145,7 @@ static void list_commands_in_dir(struct cmdnames *cmds,
len = buf.len;
while ((de = readdir(dir)) != NULL) {
- int entlen;
+ size_t entlen;
if (!starts_with(de->d_name, prefix))
continue;
@@ -156,8 +156,7 @@ static void list_commands_in_dir(struct cmdnames *cmds,
continue;
entlen = strlen(de->d_name) - prefix_len;
- if (ends_with(de->d_name, ".exe"))
- entlen -= 4;
+ strip_suffix(de->d_name, ".exe", &entlen);
add_cmdname(cmds, de->d_name + prefix_len, entlen);
}
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 6/9] index-pack: use strip_suffix to avoid magic numbers
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (4 preceding siblings ...)
2014-06-30 16:58 ` [PATCH 5/9] use strip_suffix instead of ends_with in simple cases Jeff King
@ 2014-06-30 16:59 ` Jeff King
2014-06-30 17:01 ` [PATCH 7/9] strbuf: implement strbuf_strip_suffix Jeff King
` (3 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 16:59 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
We also switch to using strbufs, which lets us avoid the
potentially dangerous combination of a manual malloc
followed by a strcpy.
Signed-off-by: Jeff King <peff@peff.net>
---
builtin/index-pack.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 46376b6..d4b77fd 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1505,7 +1505,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
const char *curr_index;
const char *index_name = NULL, *pack_name = NULL;
const char *keep_name = NULL, *keep_msg = NULL;
- char *index_name_buf = NULL, *keep_name_buf = NULL;
+ struct strbuf index_name_buf = STRBUF_INIT,
+ keep_name_buf = STRBUF_INIT;
struct pack_idx_entry **idx_objects;
struct pack_idx_option opts;
unsigned char pack_sha1[20];
@@ -1602,24 +1603,22 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
if (fix_thin_pack && !from_stdin)
die(_("--fix-thin cannot be used without --stdin"));
if (!index_name && pack_name) {
- int len = strlen(pack_name);
- if (!ends_with(pack_name, ".pack"))
+ size_t len;
+ if (!strip_suffix(pack_name, ".pack", &len))
die(_("packfile name '%s' does not end with '.pack'"),
pack_name);
- index_name_buf = xmalloc(len);
- memcpy(index_name_buf, pack_name, len - 5);
- strcpy(index_name_buf + len - 5, ".idx");
- index_name = index_name_buf;
+ strbuf_add(&index_name_buf, pack_name, len);
+ strbuf_addstr(&index_name_buf, ".idx");
+ index_name = index_name_buf.buf;
}
if (keep_msg && !keep_name && pack_name) {
- int len = strlen(pack_name);
- if (!ends_with(pack_name, ".pack"))
+ size_t len;
+ if (!strip_suffix(pack_name, ".pack", &len))
die(_("packfile name '%s' does not end with '.pack'"),
pack_name);
- keep_name_buf = xmalloc(len);
- memcpy(keep_name_buf, pack_name, len - 5);
- strcpy(keep_name_buf + len - 5, ".keep");
- keep_name = keep_name_buf;
+ strbuf_add(&keep_name_buf, pack_name, len);
+ strbuf_addstr(&keep_name_buf, ".idx");
+ keep_name = keep_name_buf.buf;
}
if (verify) {
if (!index_name)
@@ -1667,8 +1666,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
else
close(input_fd);
free(objects);
- free(index_name_buf);
- free(keep_name_buf);
+ strbuf_release(&index_name_buf);
+ strbuf_release(&keep_name_buf);
if (pack_name == NULL)
free((void *) curr_pack);
if (index_name == NULL)
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 7/9] strbuf: implement strbuf_strip_suffix
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (5 preceding siblings ...)
2014-06-30 16:59 ` [PATCH 6/9] index-pack: use strip_suffix to avoid magic numbers Jeff King
@ 2014-06-30 17:01 ` Jeff King
2014-06-30 17:02 ` [PATCH 8/9] verify-pack: use strbuf_strip_suffix Jeff King
` (2 subsequent siblings)
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 17:01 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
You can almost get away with just calling "strip_suffix_mem"
on a strbuf's buf and len fields. But we also need to move
the NUL-terminator to satisfy strbuf's invariants. Let's
provide a convenience wrapper that handles this.
Signed-off-by: Jeff King <peff@peff.net>
---
I called strbuf_setlen here because it seemed sensible to use that as an
opaque building block, but we are violating the invariant here for a
moment by setting sb->len for a moment. I think that is fine, as this is
a strbuf function, and can violate the invariant for a moment if it
wants to.
But if we care, we can keep a separate "size_t len" variable, and
strbuf_setlen to that.
Or we can make it less opaque, and replace the setlen with
"sb->buf[sb->len] = 0".
strbuf.h | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/strbuf.h b/strbuf.h
index e9ad03e..ec11742 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -49,6 +49,15 @@ extern int strbuf_reencode(struct strbuf *sb, const char *from, const char *to);
extern void strbuf_tolower(struct strbuf *sb);
extern int strbuf_cmp(const struct strbuf *, const struct strbuf *);
+static inline int strbuf_strip_suffix(struct strbuf *sb, const char *suffix)
+{
+ if (strip_suffix_mem(sb->buf, &sb->len, suffix)) {
+ strbuf_setlen(sb, sb->len);
+ return 1;
+ } else
+ return 0;
+}
+
/*
* Split str (of length slen) at the specified terminator character.
* Return a null-terminated array of pointers to strbuf objects
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 8/9] verify-pack: use strbuf_strip_suffix
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (6 preceding siblings ...)
2014-06-30 17:01 ` [PATCH 7/9] strbuf: implement strbuf_strip_suffix Jeff King
@ 2014-06-30 17:02 ` Jeff King
2014-06-30 17:04 ` [PATCH 9/9] prepare_packed_git_one: refactor duplicate-pack check Jeff King
2014-07-01 1:00 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Junio C Hamano
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 17:02 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
In this code, we try to convert both "foo.idx" and "foo"
into "foo.pack". By stripping the suffix, we can avoid a
confusing use of strbuf_splice, and make it clear that both
cases are adding ".pack" to the end.
Signed-off-by: Jeff King <peff@peff.net>
---
builtin/verify-pack.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index 2fd29ce..972579f 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -27,10 +27,9 @@ static int verify_one_pack(const char *path, unsigned int flags)
* normalize these forms to "foo.pack" for "index-pack --verify".
*/
strbuf_addstr(&arg, path);
- if (ends_with(arg.buf, ".idx"))
- strbuf_splice(&arg, arg.len - 3, 3, "pack", 4);
- else if (!ends_with(arg.buf, ".pack"))
- strbuf_add(&arg, ".pack", 5);
+ if (strbuf_strip_suffix(&arg, ".idx") ||
+ !ends_with(arg.buf, ".pack"))
+ strbuf_addstr(&arg, ".pack");
argv[2] = arg.buf;
memset(&index_pack, 0, sizeof(index_pack));
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 9/9] prepare_packed_git_one: refactor duplicate-pack check
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (7 preceding siblings ...)
2014-06-30 17:02 ` [PATCH 8/9] verify-pack: use strbuf_strip_suffix Jeff King
@ 2014-06-30 17:04 ` Jeff King
2014-07-01 1:00 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Junio C Hamano
9 siblings, 0 replies; 27+ messages in thread
From: Jeff King @ 2014-06-30 17:04 UTC (permalink / raw)
To: René Scharfe; +Cc: Duy Nguyen, Git Mailing List, Junio C Hamano
When we are reloading the list of packs, we check whether a
particular pack has been loaded. This is slightly tricky,
because we load packs based on the presence of their ".idx"
files, but record the name of the matching ".pack" file.
Therefore we want to compare their bases.
The existing code stripped off ".idx" from a file we found,
then compared that whole base length to strings containing
the ".pack" version. This meant we could end up comparing
bytes past what the ".pack" string contained, if the ".idx"
file name was much longer.
In practice, it worked OK because memcmp would end up seeing
a difference in the two strings and would return early
before hitting the full length. However, memcmp may
sometimes read extra bytes past a difference (e.g., because
it is comparing 64-bit words), or is even free to compare in
reverse order.
Furthermore, our memcmp made no guarantees that we matched
the whole pack name, up to ".pack". So "foo.idx" would match
"foo-bar.pack", which is wrong (but does not typically
happen, because our pack names have a fixed size).
We can fix both issues, avoid magic numbers, and document
that we expect to compare against a string with ".pack" by
using strip_suffix.
Signed-off-by: Jeff King <peff@peff.net>
---
This is a teeny bit less efficient than it could be, because we are
verifying our assumption at run-time that each pack name ends in
".pack". I'd venture to say if we cared about efficiency here, the low
hanging fruit would be to avoid the O(n^2) loop to find duplicate pack
names in the first place.
sha1_file.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 93b794f..129a4c5 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1197,6 +1197,7 @@ static void prepare_packed_git_one(char *objdir, int local)
dirnamelen = path.len;
while ((de = readdir(dir)) != NULL) {
struct packed_git *p;
+ size_t base_len;
if (is_dot_or_dotdot(de->d_name))
continue;
@@ -1204,10 +1205,14 @@ static void prepare_packed_git_one(char *objdir, int local)
strbuf_setlen(&path, dirnamelen);
strbuf_addstr(&path, de->d_name);
- if (ends_with(de->d_name, ".idx")) {
+ base_len = path.len;
+ if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
/* Don't reopen a pack we already have. */
for (p = packed_git; p; p = p->next) {
- if (!memcmp(path.buf, p->pack_name, path.len - 4))
+ size_t len;
+ if (strip_suffix(p->pack_name, ".pack", &len) &&
+ len == base_len &&
+ !memcmp(p->pack_name, path.buf, len))
break;
}
if (p == NULL &&
--
2.0.0.566.gfe3e6b2
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 0/9] add strip_suffix as an alternative to ends_with
2014-06-30 16:55 ` [PATCH 0/9] add strip_suffix as an alternative to ends_with Jeff King
` (8 preceding siblings ...)
2014-06-30 17:04 ` [PATCH 9/9] prepare_packed_git_one: refactor duplicate-pack check Jeff King
@ 2014-07-01 1:00 ` Junio C Hamano
9 siblings, 0 replies; 27+ messages in thread
From: Junio C Hamano @ 2014-07-01 1:00 UTC (permalink / raw)
To: Jeff King; +Cc: René Scharfe, Duy Nguyen, Git Mailing List
On Mon, Jun 30, 2014 at 9:55 AM, Jeff King <peff@peff.net> wrote:
> Here's a series to do for ends_with what the recent skip_prefix series
> did for starts_with. Namely: drop some magic numbers and repeated string
> literals, and hopefully make things more readable.
>
> The first patch is René's patch 1/2, with the leak fix from Duy and typo
> fixes in the commit message from me. The rest are new, and replace the
> changes to prepare_packed_git_one done elsewhere in the thread.
>
> [1/9]: sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one()
> [2/9]: add strip_suffix function
> [3/9]: implement ends_with via strip_suffix
> [4/9]: replace has_extension with ends_with
> [5/9]: use strip_suffix instead of ends_with in simple cases
> [6/9]: index-pack: use strip_suffix to avoid magic numbers
> [7/9]: strbuf: implement strbuf_strip_suffix
> [8/9]: verify-pack: use strbuf_strip_suffix
> [9/9]: prepare_packed_git_one: refactor duplicate-pack check
Looked very sensible. Thanks all ;-)
^ permalink raw reply [flat|nested] 27+ messages in thread