* [PATCH v6 01/32] refs: move head_ref{,_submodule} to the common code
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 02/32] refs: move for_each_*ref* functions into " David Turner
` (32 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
These don't use any backend-specific functions. These were previously
defined in terms of the do_head_ref helper function, but since they
are otherwise identical, we don't need that function.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 23 +++++++++++++++++++++++
refs/files-backend.c | 28 ----------------------------
2 files changed, 23 insertions(+), 28 deletions(-)
diff --git a/refs.c b/refs.c
index e2d34b2..4367c14 100644
--- a/refs.c
+++ b/refs.c
@@ -1082,3 +1082,26 @@ int rename_ref_available(const char *oldname, const char *newname)
strbuf_release(&err);
return ret;
}
+
+int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+ struct object_id oid;
+ int flag;
+
+ if (submodule) {
+ if (resolve_gitlink_ref(submodule, "HEAD", oid.hash) == 0)
+ return fn("HEAD", &oid, 0, cb_data);
+
+ return 0;
+ }
+
+ if (!read_ref_full("HEAD", RESOLVE_REF_READING, oid.hash, &flag))
+ return fn("HEAD", &oid, flag, cb_data);
+
+ return 0;
+}
+
+int head_ref(each_ref_fn fn, void *cb_data)
+{
+ return head_ref_submodule(NULL, fn, cb_data);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index b569762..2e2a6d9 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1753,34 +1753,6 @@ static int do_for_each_ref(struct ref_cache *refs, const char *base,
return do_for_each_entry(refs, base, do_one_ref, &data);
}
-static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
-{
- struct object_id oid;
- int flag;
-
- if (submodule) {
- if (resolve_gitlink_ref(submodule, "HEAD", oid.hash) == 0)
- return fn("HEAD", &oid, 0, cb_data);
-
- return 0;
- }
-
- if (!read_ref_full("HEAD", RESOLVE_REF_READING, oid.hash, &flag))
- return fn("HEAD", &oid, flag, cb_data);
-
- return 0;
-}
-
-int head_ref(each_ref_fn fn, void *cb_data)
-{
- return do_head_ref(NULL, fn, cb_data);
-}
-
-int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
-{
- return do_head_ref(submodule, fn, cb_data);
-}
-
int for_each_ref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(&ref_cache, "", fn, 0, 0, cb_data);
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 02/32] refs: move for_each_*ref* functions into common code
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
2016-02-24 22:58 ` [PATCH v6 01/32] refs: move head_ref{,_submodule} to the common code David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 03/32] files-backend: break out ref reading David Turner
` (31 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Make do_for_each_ref take a submodule as an argument instead of a
ref_cache. Since all for_each_*ref* functions are defined in terms of
do_for_each_ref, we can then move them into the common code.
Later, we can simply make do_for_each_ref into a backend function.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 52 +++++++++++++++++++++++++++++++++++++++++++
refs/files-backend.c | 62 +++++-----------------------------------------------
refs/refs-internal.h | 9 ++++++++
3 files changed, 66 insertions(+), 57 deletions(-)
diff --git a/refs.c b/refs.c
index 4367c14..c38311b 100644
--- a/refs.c
+++ b/refs.c
@@ -1105,3 +1105,55 @@ int head_ref(each_ref_fn fn, void *cb_data)
{
return head_ref_submodule(NULL, fn, cb_data);
}
+
+int for_each_ref(each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(NULL, "", fn, 0, 0, cb_data);
+}
+
+int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(submodule, "", fn, 0, 0, cb_data);
+}
+
+int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(NULL, prefix, fn, strlen(prefix), 0, cb_data);
+}
+
+int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data, unsigned int broken)
+{
+ unsigned int flag = 0;
+
+ if (broken)
+ flag = DO_FOR_EACH_INCLUDE_BROKEN;
+ return do_for_each_ref(NULL, prefix, fn, 0, flag, cb_data);
+}
+
+int for_each_ref_in_submodule(const char *submodule, const char *prefix,
+ each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(submodule, prefix, fn, strlen(prefix), 0, cb_data);
+}
+
+int for_each_replace_ref(each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(NULL, git_replace_ref_base, fn,
+ strlen(git_replace_ref_base), 0, cb_data);
+}
+
+int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
+{
+ struct strbuf buf = STRBUF_INIT;
+ int ret;
+ strbuf_addf(&buf, "%srefs/", get_git_namespace());
+ ret = do_for_each_ref(NULL, buf.buf, fn, 0, 0, cb_data);
+ strbuf_release(&buf);
+ return ret;
+}
+
+int for_each_rawref(each_ref_fn fn, void *cb_data)
+{
+ return do_for_each_ref(NULL, "", fn, 0,
+ DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 2e2a6d9..fd664d6 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -518,9 +518,6 @@ static void sort_ref_dir(struct ref_dir *dir)
dir->sorted = dir->nr = i;
}
-/* Include broken references in a do_for_each_ref*() iteration: */
-#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
-
/*
* Return true iff the reference described by entry can be resolved to
* an object in the database. Emit a warning if the referred-to
@@ -1735,10 +1732,13 @@ static int do_for_each_entry(struct ref_cache *refs, const char *base,
* value, stop the iteration and return that value; otherwise, return
* 0.
*/
-static int do_for_each_ref(struct ref_cache *refs, const char *base,
- each_ref_fn fn, int trim, int flags, void *cb_data)
+int do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags, void *cb_data)
{
struct ref_entry_cb data;
+ struct ref_cache *refs;
+
+ refs = get_ref_cache(submodule);
data.base = base;
data.trim = trim;
data.flags = flags;
@@ -1753,58 +1753,6 @@ static int do_for_each_ref(struct ref_cache *refs, const char *base,
return do_for_each_entry(refs, base, do_one_ref, &data);
}
-int for_each_ref(each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(&ref_cache, "", fn, 0, 0, cb_data);
-}
-
-int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(get_ref_cache(submodule), "", fn, 0, 0, cb_data);
-}
-
-int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(&ref_cache, prefix, fn, strlen(prefix), 0, cb_data);
-}
-
-int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data, unsigned int broken)
-{
- unsigned int flag = 0;
-
- if (broken)
- flag = DO_FOR_EACH_INCLUDE_BROKEN;
- return do_for_each_ref(&ref_cache, prefix, fn, 0, flag, cb_data);
-}
-
-int for_each_ref_in_submodule(const char *submodule, const char *prefix,
- each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(get_ref_cache(submodule), prefix, fn, strlen(prefix), 0, cb_data);
-}
-
-int for_each_replace_ref(each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(&ref_cache, git_replace_ref_base, fn,
- strlen(git_replace_ref_base), 0, cb_data);
-}
-
-int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
-{
- struct strbuf buf = STRBUF_INIT;
- int ret;
- strbuf_addf(&buf, "%srefs/", get_git_namespace());
- ret = do_for_each_ref(&ref_cache, buf.buf, fn, 0, 0, cb_data);
- strbuf_release(&buf);
- return ret;
-}
-
-int for_each_rawref(each_ref_fn fn, void *cb_data)
-{
- return do_for_each_ref(&ref_cache, "", fn, 0,
- DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
-}
-
static void unlock_ref(struct ref_lock *lock)
{
/* Do not free lock->lk -- atexit() still looks at them */
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c7dded3..92aae80 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -197,4 +197,13 @@ const char *find_descendant_ref(const char *dirname,
int rename_ref_available(const char *oldname, const char *newname);
+
+/* Include broken references in a do_for_each_ref*() iteration: */
+#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
+
+/*
+ * The common backend for the for_each_*ref* functions
+ */
+int do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags, void *cb_data);
#endif /* REFS_REFS_INTERNAL_H */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 03/32] files-backend: break out ref reading
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
2016-02-24 22:58 ` [PATCH v6 01/32] refs: move head_ref{,_submodule} to the common code David Turner
2016-02-24 22:58 ` [PATCH v6 02/32] refs: move for_each_*ref* functions into " David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-25 0:21 ` Junio C Hamano
2016-02-24 22:58 ` [PATCH v6 04/32] refs: move resolve_ref_unsafe into common code David Turner
` (30 subsequent siblings)
33 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Refactor resolve_ref_1 in terms of a new function read_raw_ref, which
is responsible for reading ref data from the ref storage.
Later, we will make read_raw_ref a pluggable backend function, and make
resolve_ref_unsafe common.
Testing done: Hacked in code to run both old and new version of
resolve_ref_1 and compare all outputs, failing dramatically if outputs
differed. Ran test suite.
Signed-off-by: David Turner <dturner@twopensource.com>
Helped-by: Duy Nguyen <pclouds@gmail.com>
---
refs/files-backend.c | 265 ++++++++++++++++++++++++++++++---------------------
1 file changed, 159 insertions(+), 106 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index fd664d6..ef5f28d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1377,10 +1377,9 @@ static struct ref_entry *get_packed_ref(const char *refname)
/*
* A loose ref file doesn't exist; check for a packed ref. The
- * options are forwarded from resolve_safe_unsafe().
+ * options are forwarded from resolve_ref_unsafe().
*/
static int resolve_missing_loose_ref(const char *refname,
- int resolve_flags,
unsigned char *sha1,
int *flags)
{
@@ -1398,64 +1397,48 @@ static int resolve_missing_loose_ref(const char *refname,
return 0;
}
/* The reference is not a packed reference, either. */
- if (resolve_flags & RESOLVE_REF_READING) {
- errno = ENOENT;
- return -1;
- } else {
- hashclr(sha1);
- return 0;
- }
+ errno = ENOENT;
+ return -1;
}
-/* This function needs to return a meaningful errno on failure */
-static const char *resolve_ref_1(const char *refname,
- int resolve_flags,
- unsigned char *sha1,
- int *flags,
- struct strbuf *sb_refname,
- struct strbuf *sb_path,
- struct strbuf *sb_contents)
+/*
+ * Read a raw ref from the filesystem or packed refs file.
+ *
+ * If the ref is a sha1, fill in sha1 and return 0.
+ *
+ * If the ref is symbolic, fill in *symref with the referrent
+ * (e.g. "refs/heads/master") and return 0. The caller is responsible
+ * for validating the referrent. Set REF_ISSYMREF in flags.
+ *
+ * If the ref is neither a symbolic ref nor a sha1, it is broken. Set
+ * REF_ISBROKEN in flags, set errno to EINVAL, and return -1.
+ *
+ * If the ref doesn't exist, set errno to ENOENT and return -1.
+ *
+ * If there is another error reading the ref, set errno appropriately and
+ * return -1.
+ *
+ * Backend-specific flags might be set in flags as well, regardless of
+ * outcome.
+ *
+ * sb_path is workspace: the caller should allocate and free it.
+ */
+static int read_raw_ref(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags)
{
- int depth = MAXDEPTH;
- int bad_name = 0;
-
- if (flags)
- *flags = 0;
+ struct strbuf sb_contents = STRBUF_INIT;
+ int ret = -1;
+ const char *path;
+ const char *buf;
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- if (flags)
- *flags |= REF_BAD_NAME;
+ strbuf_reset(sb_path);
+ strbuf_git_path(sb_path, "%s", refname);
+ path = sb_path->buf;
- if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
- !refname_is_safe(refname)) {
- errno = EINVAL;
- return NULL;
- }
- /*
- * dwim_ref() uses REF_ISBROKEN to distinguish between
- * missing refs and refs that were present but invalid,
- * to complain about the latter to stderr.
- *
- * We don't know whether the ref exists, so don't set
- * REF_ISBROKEN yet.
- */
- bad_name = 1;
- }
- for (;;) {
- const char *path;
+ for(;;) {
struct stat st;
- char *buf;
int fd;
-
- if (--depth < 0) {
- errno = ELOOP;
- return NULL;
- }
-
- strbuf_reset(sb_path);
- strbuf_git_path(sb_path, "%s", refname);
- path = sb_path->buf;
-
/*
* We might have to loop back here to avoid a race
* condition: first we lstat() the file, then we try
@@ -1465,49 +1448,45 @@ static const char *resolve_ref_1(const char *refname,
* we don't want to report that as an error but rather
* try again starting with the lstat().
*/
- stat_ref:
+
if (lstat(path, &st) < 0) {
if (errno != ENOENT)
- return NULL;
- if (resolve_missing_loose_ref(refname, resolve_flags,
- sha1, flags))
- return NULL;
- if (bad_name) {
- hashclr(sha1);
- if (flags)
- *flags |= REF_ISBROKEN;
- }
- return refname;
+ break;
+ if (resolve_missing_loose_ref(refname, sha1, flags))
+ break;
+ ret = 0;
+ break;
}
/* Follow "normalized" - ie "refs/.." symlinks by hand */
if (S_ISLNK(st.st_mode)) {
- strbuf_reset(sb_contents);
- if (strbuf_readlink(sb_contents, path, 0) < 0) {
+ strbuf_reset(&sb_contents);
+ if (strbuf_readlink(&sb_contents, path, 0) < 0) {
if (errno == ENOENT || errno == EINVAL)
/* inconsistent with lstat; retry */
- goto stat_ref;
+ continue;
else
- return NULL;
+ break;
}
- if (starts_with(sb_contents->buf, "refs/") &&
- !check_refname_format(sb_contents->buf, 0)) {
- strbuf_swap(sb_refname, sb_contents);
- refname = sb_refname->buf;
+ if (starts_with(sb_contents.buf, "refs/") &&
+ !check_refname_format(sb_contents.buf, 0)) {
+ strbuf_swap(&sb_contents, symref);
if (flags)
*flags |= REF_ISSYMREF;
- if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
- hashclr(sha1);
- return refname;
- }
- continue;
+ ret = 0;
+ break;
+ } else {
+ /* bogus symlink ref */
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ break;
}
}
/* Is it a directory? */
if (S_ISDIR(st.st_mode)) {
errno = EISDIR;
- return NULL;
+ break;
}
/*
@@ -1518,35 +1497,110 @@ static const char *resolve_ref_1(const char *refname,
if (fd < 0) {
if (errno == ENOENT)
/* inconsistent with lstat; retry */
- goto stat_ref;
+ continue;
else
- return NULL;
+ break;
}
- strbuf_reset(sb_contents);
- if (strbuf_read(sb_contents, fd, 256) < 0) {
+ strbuf_reset(&sb_contents);
+ if (strbuf_read(&sb_contents, fd, 256) < 0) {
int save_errno = errno;
close(fd);
errno = save_errno;
- return NULL;
+ break;
}
close(fd);
- strbuf_rtrim(sb_contents);
+ strbuf_rtrim(&sb_contents);
+ buf = sb_contents.buf;
+ if (starts_with(buf, "ref:")) {
+ buf += 4;
+ while (isspace(*buf))
+ buf++;
+
+ strbuf_reset(symref);
+ strbuf_addstr(symref, buf);
+ if (flags)
+ *flags |= REF_ISSYMREF;
+ ret = 0;
+ break;
+ }
/*
- * Is it a symbolic ref?
+ * Please note that FETCH_HEAD has additional
+ * data after the sha.
*/
- if (!starts_with(sb_contents->buf, "ref:")) {
- /*
- * Please note that FETCH_HEAD has a second
- * line containing other data.
- */
- if (get_sha1_hex(sb_contents->buf, sha1) ||
- (sb_contents->buf[40] != '\0' && !isspace(sb_contents->buf[40]))) {
+ if (get_sha1_hex(buf, sha1) ||
+ (buf[40] != '\0' && !isspace(buf[40]))) {
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ errno = EINVAL;
+ break;
+ }
+ ret = 0;
+ break;
+ }
+
+ strbuf_release(&sb_contents);
+ return ret;
+}
+
+/* This function needs to return a meaningful errno on failure */
+static const char *resolve_ref_1(const char *refname,
+ int resolve_flags,
+ unsigned char *sha1,
+ int *flags,
+ struct strbuf *sb_refname,
+ struct strbuf *sb_path)
+{
+ int bad_name = 0;
+ int symref_count;
+
+ if (flags)
+ *flags = 0;
+
+ if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+ if (flags)
+ *flags |= REF_BAD_NAME;
+
+ if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+ !refname_is_safe(refname)) {
+ errno = EINVAL;
+ return NULL;
+ }
+ /*
+ * dwim_ref() uses REF_ISBROKEN to distinguish between
+ * missing refs and refs that were present but invalid,
+ * to complain about the latter to stderr.
+ *
+ * We don't know whether the ref exists, so don't set
+ * REF_ISBROKEN yet.
+ */
+ bad_name = 1;
+ }
+
+ for (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {
+ int read_flags = 0;
+
+ if (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {
+ int saved_errno = errno;
+ if (flags)
+ *flags |= read_flags;
+ errno = saved_errno;
+ if (bad_name) {
+ hashclr(sha1);
if (flags)
*flags |= REF_ISBROKEN;
- errno = EINVAL;
+ }
+ if (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {
return NULL;
+ } else {
+ hashclr(sha1);
+ return refname;
}
+ }
+ if (flags)
+ *flags |= read_flags;
+
+ if (!(read_flags & REF_ISSYMREF)) {
if (bad_name) {
hashclr(sha1);
if (flags)
@@ -1554,44 +1608,43 @@ static const char *resolve_ref_1(const char *refname,
}
return refname;
}
- if (flags)
- *flags |= REF_ISSYMREF;
- buf = sb_contents->buf + 4;
- while (isspace(*buf))
- buf++;
- strbuf_reset(sb_refname);
- strbuf_addstr(sb_refname, buf);
+
refname = sb_refname->buf;
if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
hashclr(sha1);
+ if (bad_name && flags)
+ *flags |= REF_ISBROKEN;
return refname;
}
- if (check_refname_format(buf, REFNAME_ALLOW_ONELEVEL)) {
+
+ if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
if (flags)
*flags |= REF_ISBROKEN;
-
if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
- !refname_is_safe(buf)) {
+ !refname_is_safe(refname)) {
errno = EINVAL;
return NULL;
}
bad_name = 1;
}
}
+
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ return NULL;
}
const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
unsigned char *sha1, int *flags)
{
static struct strbuf sb_refname = STRBUF_INIT;
- struct strbuf sb_contents = STRBUF_INIT;
struct strbuf sb_path = STRBUF_INIT;
const char *ret;
ret = resolve_ref_1(refname, resolve_flags, sha1, flags,
- &sb_refname, &sb_path, &sb_contents);
+ &sb_refname, &sb_path);
+
strbuf_release(&sb_path);
- strbuf_release(&sb_contents);
return ret;
}
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [PATCH v6 03/32] files-backend: break out ref reading
2016-02-24 22:58 ` [PATCH v6 03/32] files-backend: break out ref reading David Turner
@ 2016-02-25 0:21 ` Junio C Hamano
2016-02-25 0:51 ` Junio C Hamano
0 siblings, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2016-02-25 0:21 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
David Turner <dturner@twopensource.com> writes:
> Refactor resolve_ref_1 in terms of a new function read_raw_ref, which
> is responsible for reading ref data from the ref storage.
>
> Later, we will make read_raw_ref a pluggable backend function, and make
> resolve_ref_unsafe common.
>
> Testing done: Hacked in code to run both old and new version of
> resolve_ref_1 and compare all outputs, failing dramatically if outputs
> differed. Ran test suite.
>
> Signed-off-by: David Turner <dturner@twopensource.com>
> Helped-by: Duy Nguyen <pclouds@gmail.com>
> ---
> refs/files-backend.c | 265 ++++++++++++++++++++++++++++++---------------------
> 1 file changed, 159 insertions(+), 106 deletions(-)
>
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index fd664d6..ef5f28d 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -1377,10 +1377,9 @@ static struct ref_entry *get_packed_ref(const char *refname)
>
> /*
> * A loose ref file doesn't exist; check for a packed ref. The
> - * options are forwarded from resolve_safe_unsafe().
> + * options are forwarded from resolve_ref_unsafe().
> */
> static int resolve_missing_loose_ref(const char *refname,
> - int resolve_flags,
> unsigned char *sha1,
> int *flags)
The last parameter must be made "unsigned int *flags" ...
> +static int read_raw_ref(const char *refname, unsigned char *sha1,
> + struct strbuf *symref, struct strbuf *sb_path,
> + unsigned int *flags)
> {
... because the last parameter here is "unsigned int *flags", and
you later pass the same variable to both.
> - for (;;) {
> - const char *path;
> + for(;;) {
This is an unwelcome change.
> + break;
> + if (resolve_missing_loose_ref(refname, sha1, flags))
This line has a trailing whitespace.
> + for (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {
> + int read_flags = 0;
... and this must be "unsigned read_flags = 0".
You can of course standardize on signed int, but because this is a
collection of flag bits, there is no reason not to choose unsigned.
I _think_ I can fix everything up before pushing out, so please
check what will appear on 'pu' before rerolling.
Thanks.
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 03/32] files-backend: break out ref reading
2016-02-25 0:21 ` Junio C Hamano
@ 2016-02-25 0:51 ` Junio C Hamano
2016-02-25 20:11 ` David Turner
0 siblings, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2016-02-25 0:51 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
Junio C Hamano <gitster@pobox.com> writes:
> ...
> You can of course standardize on signed int, but because this is a
> collection of flag bits, there is no reason not to choose unsigned.
>
> I _think_ I can fix everything up before pushing out, so please
> check what will appear on 'pu' before rerolling.
I managed to whip them into shape enough to pass my compilation test
;-) Other things I had to tweak were small things like whitespace
issues, initializing statics to 0 or NULL, etc.
Thanks.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 03/32] files-backend: break out ref reading
2016-02-25 0:51 ` Junio C Hamano
@ 2016-02-25 20:11 ` David Turner
2016-02-25 20:27 ` Junio C Hamano
0 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-25 20:11 UTC (permalink / raw)
To: Junio C Hamano, git mailing list
On Wed, 2016-02-24 at 16:51 -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > ...
> > You can of course standardize on signed int, but because this is a
> > collection of flag bits, there is no reason not to choose unsigned.
> >
> > I _think_ I can fix everything up before pushing out, so please
> > check what will appear on 'pu' before rerolling.
>
> I managed to whip them into shape enough to pass my compilation test
> ;-) Other things I had to tweak were small things like whitespace
> issues, initializing statics to 0 or NULL, etc.
Got it, thanks.
Is there some standard workflow here? I just rebased your version of
my series (bd412fa) on top of the prior commit in pu (9db66d9), fixing
the few conflicts. (I think there is in general something I'm missing
about how to maintain a patch set under the git.git workflow).
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 03/32] files-backend: break out ref reading
2016-02-25 20:11 ` David Turner
@ 2016-02-25 20:27 ` Junio C Hamano
0 siblings, 0 replies; 56+ messages in thread
From: Junio C Hamano @ 2016-02-25 20:27 UTC (permalink / raw)
To: David Turner; +Cc: git mailing list, Stefan Beller
David Turner <dturner@twopensource.com> writes:
> Is there some standard workflow here? I just rebased your version of
> my series (bd412fa) on top of the prior commit in pu (9db66d9), fixing
> the few conflicts. (I think there is in general something I'm missing
> about how to maintain a patch set under the git.git workflow).
We usually do not see multiple large topics stomping on each other's
toes, so it would be fair to say that we are still learning to find
some standard workflow. The topic overlaps with what Stefan's
submodule-parallel-update topic touches, and both topics are
constantly rerolled, so I've been doing:
* Pick a commit on 'master' as a stable base, and do not change
this unless absolutely necessary.
* Merge the tip of Stefan's series to it.
* Apply your series on top (if you didn't rebase, but Stefan
rerolled, this will be cherry-pick of your latest round).
to rebuild dt/refs-backend-lmdb topic every time either one of you
rerolled.
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PATCH v6 04/32] refs: move resolve_ref_unsafe into common code
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (2 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 03/32] files-backend: break out ref reading David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-25 0:22 ` Junio C Hamano
2016-02-24 22:58 ` [PATCH v6 05/32] refs: add a backend method structure with transaction functions David Turner
` (29 subsequent siblings)
33 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Now that resolve_ref_unsafe's only interaction with the backend is
through read_raw_ref, we can move it into the common code. Later,
we'll replace read_raw_ref with a backend function.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++
refs/files-backend.c | 115 ++-------------------------------------------------
refs/refs-internal.h | 6 +++
3 files changed, 114 insertions(+), 111 deletions(-)
diff --git a/refs.c b/refs.c
index c38311b..f4c0a85 100644
--- a/refs.c
+++ b/refs.c
@@ -1157,3 +1157,107 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
return do_for_each_ref(NULL, "", fn, 0,
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
}
+/* This function needs to return a meaningful errno on failure */
+static const char *resolve_ref_1(const char *refname,
+ int resolve_flags,
+ unsigned char *sha1,
+ int *flags,
+ struct strbuf *sb_refname,
+ struct strbuf *sb_path)
+{
+ int bad_name = 0;
+ int symref_count;
+
+ if (flags)
+ *flags = 0;
+
+ if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+ if (flags)
+ *flags |= REF_BAD_NAME;
+
+ if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+ !refname_is_safe(refname)) {
+ errno = EINVAL;
+ return NULL;
+ }
+ /*
+ * dwim_ref() uses REF_ISBROKEN to distinguish between
+ * missing refs and refs that were present but invalid,
+ * to complain about the latter to stderr.
+ *
+ * We don't know whether the ref exists, so don't set
+ * REF_ISBROKEN yet.
+ */
+ bad_name = 1;
+ }
+
+ for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
+ int read_flags = 0;
+
+ if (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {
+ int saved_errno = errno;
+ if (flags)
+ *flags |= read_flags;
+ errno = saved_errno;
+ if (bad_name) {
+ hashclr(sha1);
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ }
+ if (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {
+ return NULL;
+ } else {
+ hashclr(sha1);
+ return refname;
+ }
+ }
+ if (flags)
+ *flags |= read_flags;
+
+ if (!(read_flags & REF_ISSYMREF)) {
+ if (bad_name) {
+ hashclr(sha1);
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ }
+ return refname;
+ }
+
+ refname = sb_refname->buf;
+ if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
+ hashclr(sha1);
+ if (bad_name && flags)
+ *flags |= REF_ISBROKEN;
+ return refname;
+ }
+
+ if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
+ !refname_is_safe(refname)) {
+ errno = EINVAL;
+ return NULL;
+ }
+ bad_name = 1;
+ }
+ }
+
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ return NULL;
+}
+
+const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
+ unsigned char *sha1, int *flags)
+{
+ static struct strbuf sb_refname = STRBUF_INIT;
+ struct strbuf sb_path = STRBUF_INIT;
+ const char *ret;
+
+ ret = resolve_ref_1(refname, resolve_flags, sha1, flags,
+ &sb_refname, &sb_path);
+
+ strbuf_release(&sb_path);
+ return ret;
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ef5f28d..5749eff 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1277,8 +1277,6 @@ static struct ref_dir *get_loose_refs(struct ref_cache *refs)
return get_ref_dir(refs->loose);
}
-/* We allow "recursive" symbolic refs. Only within reason, though */
-#define MAXDEPTH 5
#define MAXREFLEN (1024)
/*
@@ -1308,7 +1306,7 @@ static int resolve_gitlink_ref_recursive(struct ref_cache *refs,
char buffer[128], *p;
char *path;
- if (recursion > MAXDEPTH || strlen(refname) > MAXREFLEN)
+ if (recursion > SYMREF_MAXDEPTH || strlen(refname) > MAXREFLEN)
return -1;
path = *refs->name
? git_pathdup_submodule(refs->name, "%s", refname)
@@ -1423,9 +1421,9 @@ static int resolve_missing_loose_ref(const char *refname,
*
* sb_path is workspace: the caller should allocate and free it.
*/
-static int read_raw_ref(const char *refname, unsigned char *sha1,
- struct strbuf *symref, struct strbuf *sb_path,
- unsigned int *flags)
+int read_raw_ref(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags)
{
struct strbuf sb_contents = STRBUF_INIT;
int ret = -1;
@@ -1543,111 +1541,6 @@ static int read_raw_ref(const char *refname, unsigned char *sha1,
return ret;
}
-/* This function needs to return a meaningful errno on failure */
-static const char *resolve_ref_1(const char *refname,
- int resolve_flags,
- unsigned char *sha1,
- int *flags,
- struct strbuf *sb_refname,
- struct strbuf *sb_path)
-{
- int bad_name = 0;
- int symref_count;
-
- if (flags)
- *flags = 0;
-
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- if (flags)
- *flags |= REF_BAD_NAME;
-
- if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
- !refname_is_safe(refname)) {
- errno = EINVAL;
- return NULL;
- }
- /*
- * dwim_ref() uses REF_ISBROKEN to distinguish between
- * missing refs and refs that were present but invalid,
- * to complain about the latter to stderr.
- *
- * We don't know whether the ref exists, so don't set
- * REF_ISBROKEN yet.
- */
- bad_name = 1;
- }
-
- for (symref_count = 0; symref_count < MAXDEPTH; symref_count++) {
- int read_flags = 0;
-
- if (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {
- int saved_errno = errno;
- if (flags)
- *flags |= read_flags;
- errno = saved_errno;
- if (bad_name) {
- hashclr(sha1);
- if (flags)
- *flags |= REF_ISBROKEN;
- }
- if (resolve_flags & RESOLVE_REF_READING || errno != ENOENT) {
- return NULL;
- } else {
- hashclr(sha1);
- return refname;
- }
- }
- if (flags)
- *flags |= read_flags;
-
- if (!(read_flags & REF_ISSYMREF)) {
- if (bad_name) {
- hashclr(sha1);
- if (flags)
- *flags |= REF_ISBROKEN;
- }
- return refname;
- }
-
- refname = sb_refname->buf;
- if (resolve_flags & RESOLVE_REF_NO_RECURSE) {
- hashclr(sha1);
- if (bad_name && flags)
- *flags |= REF_ISBROKEN;
- return refname;
- }
-
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- if (flags)
- *flags |= REF_ISBROKEN;
- if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
- !refname_is_safe(refname)) {
- errno = EINVAL;
- return NULL;
- }
- bad_name = 1;
- }
- }
-
- if (flags)
- *flags |= REF_ISBROKEN;
- return NULL;
-}
-
-const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
- unsigned char *sha1, int *flags)
-{
- static struct strbuf sb_refname = STRBUF_INIT;
- struct strbuf sb_path = STRBUF_INIT;
- const char *ret;
-
- ret = resolve_ref_1(refname, resolve_flags, sha1, flags,
- &sb_refname, &sb_path);
-
- strbuf_release(&sb_path);
- return ret;
-}
-
/*
* Peel the entry (if possible) and return its new peel_status. If
* repeel is true, re-peel the entry even if there is an old peeled
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 92aae80..979a136 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -197,6 +197,8 @@ const char *find_descendant_ref(const char *dirname,
int rename_ref_available(const char *oldname, const char *newname);
+/* We allow "recursive" symbolic refs. Only within reason, though */
+#define SYMREF_MAXDEPTH 5
/* Include broken references in a do_for_each_ref*() iteration: */
#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
@@ -206,4 +208,8 @@ int rename_ref_available(const char *oldname, const char *newname);
*/
int do_for_each_ref(const char *submodule, const char *base,
each_ref_fn fn, int trim, int flags, void *cb_data);
+
+int read_raw_ref(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags);
#endif /* REFS_REFS_INTERNAL_H */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 05/32] refs: add a backend method structure with transaction functions
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (3 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 04/32] refs: move resolve_ref_unsafe into common code David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-27 4:06 ` Jeff King
2016-02-24 22:58 ` [PATCH v6 06/32] refs: add methods for misc ref operations David Turner
` (28 subsequent siblings)
33 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: Ronnie Sahlberg, David Turner
From: Ronnie Sahlberg <sahlberg@google.com>
Add a ref structure for storage backend methods. Start by adding a
method pointer for the transaction commit function.
Add a function set_refs_backend to switch between storage
backends. The files based storage backend is the default.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 41 +++++++++++++++++++++++++++++++++++++++++
refs.h | 7 +++++++
refs/files-backend.c | 10 ++++++++--
refs/refs-internal.h | 12 ++++++++++++
4 files changed, 68 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index f4c0a85..6d16058 100644
--- a/refs.c
+++ b/refs.c
@@ -10,6 +10,39 @@
#include "tag.h"
/*
+ * We always have a files backend and it is the default.
+ */
+static struct ref_storage_be *the_refs_backend = &refs_be_files;
+/*
+ * List of all available backends
+ */
+static struct ref_storage_be *refs_backends = &refs_be_files;
+
+int set_ref_storage_backend(const char *name)
+{
+ struct ref_storage_be *be;
+
+ for (be = refs_backends; be; be = be->next)
+ if (!strcmp(be->name, name)) {
+ the_refs_backend = be;
+ return 0;
+ }
+ return 1;
+}
+
+int ref_storage_backend_exists(const char *name)
+{
+ struct ref_storage_be *be;
+
+ for (be = refs_backends; be; be = be->next)
+ if (!strcmp(be->name, name)) {
+ the_refs_backend = be;
+ return 1;
+ }
+ return 0;
+}
+
+/*
* How to handle various characters in refnames:
* 0: An acceptable character for refs
* 1: End-of-component
@@ -1157,6 +1190,7 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
return do_for_each_ref(NULL, "", fn, 0,
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
}
+
/* This function needs to return a meaningful errno on failure */
static const char *resolve_ref_1(const char *refname,
int resolve_flags,
@@ -1261,3 +1295,10 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
strbuf_release(&sb_path);
return ret;
}
+
+/* backend functions */
+int ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return the_refs_backend->transaction_commit(transaction, err);
+}
diff --git a/refs.h b/refs.h
index 3c3da29..5bc3fbc 100644
--- a/refs.h
+++ b/refs.h
@@ -508,4 +508,11 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,
reflog_expiry_cleanup_fn cleanup_fn,
void *policy_cb_data);
+/*
+ * Switch to an alternate ref storage backend.
+ */
+int set_ref_storage_backend(const char *name);
+
+int ref_storage_backend_exists(const char *name);
+
#endif /* REFS_H */
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 5749eff..45b81cd 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3012,8 +3012,8 @@ static int ref_update_reject_duplicates(struct string_list *refnames,
return 0;
}
-int ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+static int files_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
@@ -3399,3 +3399,9 @@ int reflog_expire(const char *refname, const unsigned char *sha1,
unlock_ref(lock);
return -1;
}
+
+struct ref_storage_be refs_be_files = {
+ NULL,
+ "files",
+ files_transaction_commit,
+};
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 979a136..62ba0c0 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -212,4 +212,16 @@ int do_for_each_ref(const char *submodule, const char *base,
int read_raw_ref(const char *refname, unsigned char *sha1,
struct strbuf *symref, struct strbuf *sb_path,
unsigned int *flags);
+
+/* refs backends */
+typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
+ struct strbuf *err);
+
+struct ref_storage_be {
+ struct ref_storage_be *next;
+ const char *name;
+ ref_transaction_commit_fn *transaction_commit;
+};
+
+extern struct ref_storage_be refs_be_files;
#endif /* REFS_REFS_INTERNAL_H */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [PATCH v6 05/32] refs: add a backend method structure with transaction functions
2016-02-24 22:58 ` [PATCH v6 05/32] refs: add a backend method structure with transaction functions David Turner
@ 2016-02-27 4:06 ` Jeff King
2016-02-29 20:43 ` David Turner
0 siblings, 1 reply; 56+ messages in thread
From: Jeff King @ 2016-02-27 4:06 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds, Ronnie Sahlberg
On Wed, Feb 24, 2016 at 05:58:37PM -0500, David Turner wrote:
> +int set_ref_storage_backend(const char *name)
> +{
> + struct ref_storage_be *be;
> +
> + for (be = refs_backends; be; be = be->next)
> + if (!strcmp(be->name, name)) {
> + the_refs_backend = be;
> + return 0;
> + }
> + return 1;
> +}
So we search through the list and assign the_refs_backend if we find
something, returning 0 for success, and 1 if we found nothing. OK
(though our usual convention is that if 0 is success, -1 is the error
case).
> +int ref_storage_backend_exists(const char *name)
> +{
> + struct ref_storage_be *be;
> +
> + for (be = refs_backends; be; be = be->next)
> + if (!strcmp(be->name, name)) {
> + the_refs_backend = be;
> + return 1;
> + }
> + return 0;
> +}
Here we do the same thing, but we get "1" for exists, and "0"
otherwise. That makes sense if this is about querying for existence. But
why does the query function still set the_refs_backend?
I'm guessing the assignment in the second one is just copy-pasta, but
maybe the whole thing would be simpler if they could share the
implementation, like:
struct ref_storage_be *find_ref_storage_backend(const char *name)
{
struct ref_storage *be;
for (be = refs_backends; be; be = be->next)
if (!strcmp(be->name, name))
return be;
return NULL;
}
int set_ref_storage_backend(const char *name)
{
struct ref_storage *be = find_ref_storage_backend(name);
if (!be)
return -1;
the_refs_backend = be;
return 0;
}
You don't really need an "exists", as you can check that "find" doesn't
return NULL, but you could wrap it, of course.
-Peff
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 05/32] refs: add a backend method structure with transaction functions
2016-02-27 4:06 ` Jeff King
@ 2016-02-29 20:43 ` David Turner
0 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-29 20:43 UTC (permalink / raw)
To: Jeff King; +Cc: git, mhagger, Duy Nguyen
On Fri, 2016-02-26 at 23:06 -0500, Jeff King wrote:
> On Wed, Feb 24, 2016 at 05:58:37PM -0500, David Turner wrote:
>
> > +int set_ref_storage_backend(const char *name)
> > +{
> > + struct ref_storage_be *be;
> > +
> > + for (be = refs_backends; be; be = be->next)
> > + if (!strcmp(be->name, name)) {
> > + the_refs_backend = be;
> > + return 0;
> > + }
> > + return 1;
> > +}
>
> So we search through the list and assign the_refs_backend if we find
> something, returning 0 for success, and 1 if we found nothing. OK
> (though our usual convention is that if 0 is success, -1 is the error
> case).
Will fix.
> > +int ref_storage_backend_exists(const char *name)
> > +{
> > + struct ref_storage_be *be;
> > +
> > + for (be = refs_backends; be; be = be->next)
> > + if (!strcmp(be->name, name)) {
> > + the_refs_backend = be;
> > + return 1;
> > + }
> > + return 0;
> > +}
>
> Here we do the same thing, but we get "1" for exists, and "0"
> otherwise. That makes sense if this is about querying for existence.
> But
> why does the query function still set the_refs_backend?
Yeah, that's wrong.
> I'm guessing the assignment in the second one is just copy-pasta, but
> maybe the whole thing would be simpler if they could share the
> implementation, like:
>
> struct ref_storage_be *find_ref_storage_backend(const char *name)
> {
> struct ref_storage *be;
> for (be = refs_backends; be; be = be->next)
> if (!strcmp(be->name, name))
> return be;
> return NULL;
> }
>
> int set_ref_storage_backend(const char *name)
> {
> struct ref_storage *be = find_ref_storage_backend(name);
> if (!be)
> return -1;
> the_refs_backend = be;
> return 0;
> }
>
> You don't really need an "exists", as you can check that "find"
> doesn't
> return NULL, but you could wrap it, of course.
I'd rather wrap it to keep the backends firmly inside the refs-internal
barrier.
Thanks.
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PATCH v6 06/32] refs: add methods for misc ref operations
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (4 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 05/32] refs: add a backend method structure with transaction functions David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 07/32] refs: add method for do_for_each_ref David Turner
` (27 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: Ronnie Sahlberg, David Turner
From: Ronnie Sahlberg <sahlberg@google.com>
Add ref backend methods for:
read_raw_ref, verify_refname_available, pack_refs, peel_ref,
create_symref, resolve_gitlink_ref.
read_raw_ref becomes static because it's not used outside refs.c
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 37 +++++++++++++++++++++++++++++++++++++
refs/files-backend.c | 33 ++++++++++++++++++++++-----------
refs/refs-internal.h | 27 +++++++++++++++++++++++----
3 files changed, 82 insertions(+), 15 deletions(-)
diff --git a/refs.c b/refs.c
index 6d16058..6fd5121 100644
--- a/refs.c
+++ b/refs.c
@@ -1191,6 +1191,14 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
}
+static int read_raw_ref(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags)
+{
+ return the_refs_backend->read_raw_ref(refname, sha1, symref, sb_path,
+ flags);
+}
+
/* This function needs to return a meaningful errno on failure */
static const char *resolve_ref_1(const char *refname,
int resolve_flags,
@@ -1302,3 +1310,32 @@ int ref_transaction_commit(struct ref_transaction *transaction,
{
return the_refs_backend->transaction_commit(transaction, err);
}
+
+int verify_refname_available(const char *refname, struct string_list *extra,
+ struct string_list *skip, struct strbuf *err)
+{
+ return the_refs_backend->verify_refname_available(refname, extra, skip, err);
+}
+
+int pack_refs(unsigned int flags)
+{
+ return the_refs_backend->pack_refs(flags);
+}
+
+int peel_ref(const char *refname, unsigned char *sha1)
+{
+ return the_refs_backend->peel_ref(refname, sha1);
+}
+
+int create_symref(const char *ref_target, const char *refs_heads_master,
+ const char *logmsg)
+{
+ return the_refs_backend->create_symref(ref_target, refs_heads_master,
+ logmsg);
+}
+
+int resolve_gitlink_ref(const char *path, const char *refname,
+ unsigned char *sha1)
+{
+ return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 45b81cd..3a83c0d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1338,7 +1338,8 @@ static int resolve_gitlink_ref_recursive(struct ref_cache *refs,
return resolve_gitlink_ref_recursive(refs, p, sha1, recursion+1);
}
-int resolve_gitlink_ref(const char *path, const char *refname, unsigned char *sha1)
+static int files_resolve_gitlink_ref(const char *path, const char *refname,
+ unsigned char *sha1)
{
int len = strlen(path), retval;
struct strbuf submodule = STRBUF_INIT;
@@ -1421,9 +1422,9 @@ static int resolve_missing_loose_ref(const char *refname,
*
* sb_path is workspace: the caller should allocate and free it.
*/
-int read_raw_ref(const char *refname, unsigned char *sha1,
- struct strbuf *symref, struct strbuf *sb_path,
- unsigned int *flags)
+static int files_read_raw_ref(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags)
{
struct strbuf sb_contents = STRBUF_INIT;
int ret = -1;
@@ -1576,7 +1577,7 @@ static enum peel_status peel_entry(struct ref_entry *entry, int repeel)
return status;
}
-int peel_ref(const char *refname, unsigned char *sha1)
+static int files_peel_ref(const char *refname, unsigned char *sha1)
{
int flag;
unsigned char base[20];
@@ -2136,7 +2137,7 @@ static void prune_refs(struct ref_to_prune *r)
}
}
-int pack_refs(unsigned int flags)
+static int files_pack_refs(unsigned int flags)
{
struct pack_refs_cb_data cbdata;
@@ -2327,10 +2328,10 @@ out:
return ret;
}
-int verify_refname_available(const char *newname,
- struct string_list *extras,
- struct string_list *skip,
- struct strbuf *err)
+static int files_verify_refname_available(const char *newname,
+ struct string_list *extras,
+ struct string_list *skip,
+ struct strbuf *err)
{
struct ref_dir *packed_refs = get_packed_refs(&ref_cache);
struct ref_dir *loose_refs = get_loose_refs(&ref_cache);
@@ -2750,7 +2751,9 @@ static int create_symref_locked(struct ref_lock *lock, const char *refname,
return 0;
}
-int create_symref(const char *refname, const char *target, const char *logmsg)
+static int files_create_symref(const char *refname,
+ const char *target,
+ const char *logmsg)
{
struct strbuf err = STRBUF_INIT;
struct ref_lock *lock;
@@ -3404,4 +3407,12 @@ struct ref_storage_be refs_be_files = {
NULL,
"files",
files_transaction_commit,
+
+ files_pack_refs,
+ files_peel_ref,
+ files_create_symref,
+
+ files_read_raw_ref,
+ files_verify_refname_available,
+ files_resolve_gitlink_ref,
};
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 62ba0c0..c5f5ef7 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -209,18 +209,37 @@ int rename_ref_available(const char *oldname, const char *newname);
int do_for_each_ref(const char *submodule, const char *base,
each_ref_fn fn, int trim, int flags, void *cb_data);
-int read_raw_ref(const char *refname, unsigned char *sha1,
- struct strbuf *symref, struct strbuf *sb_path,
- unsigned int *flags);
-
/* refs backends */
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
+/* misc methods */
+typedef int pack_refs_fn(unsigned int flags);
+typedef int peel_ref_fn(const char *refname, unsigned char *sha1);
+typedef int create_symref_fn(const char *ref_target,
+ const char *refs_heads_master,
+ const char *logmsg);
+
+/* resolution methods */
+typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,
+ struct strbuf *symref, struct strbuf *sb_path,
+ unsigned int *flags);
+typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);
+typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,
+ unsigned char *sha1);
+
struct ref_storage_be {
struct ref_storage_be *next;
const char *name;
ref_transaction_commit_fn *transaction_commit;
+
+ pack_refs_fn *pack_refs;
+ peel_ref_fn *peel_ref;
+ create_symref_fn *create_symref;
+
+ read_raw_ref_fn *read_raw_ref;
+ verify_refname_available_fn *verify_refname_available;
+ resolve_gitlink_ref_fn *resolve_gitlink_ref;
};
extern struct ref_storage_be refs_be_files;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 07/32] refs: add method for do_for_each_ref
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (5 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 06/32] refs: add methods for misc ref operations David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 08/32] refs: reduce the visibility of do_for_each_ref() David Turner
` (26 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Add a ref backend method for do_for_each_ref.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 8 ++++++++
refs/files-backend.c | 7 +++++--
refs/refs-internal.h | 5 +++++
3 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 6fd5121..cd06f4b 100644
--- a/refs.c
+++ b/refs.c
@@ -1339,3 +1339,11 @@ int resolve_gitlink_ref(const char *path, const char *refname,
{
return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
}
+
+int do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data)
+{
+ return the_refs_backend->do_for_each_ref(submodule, base, fn, trim,
+ flags, cb_data);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3a83c0d..db9e948 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1679,8 +1679,9 @@ static int do_for_each_entry(struct ref_cache *refs, const char *base,
* value, stop the iteration and return that value; otherwise, return
* 0.
*/
-int do_for_each_ref(const char *submodule, const char *base,
- each_ref_fn fn, int trim, int flags, void *cb_data)
+static int files_do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data)
{
struct ref_entry_cb data;
struct ref_cache *refs;
@@ -3415,4 +3416,6 @@ struct ref_storage_be refs_be_files = {
files_read_raw_ref,
files_verify_refname_available,
files_resolve_gitlink_ref,
+
+ files_do_for_each_ref,
};
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c5f5ef7..c9b6745 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -227,6 +227,9 @@ typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,
typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);
typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,
unsigned char *sha1);
+typedef int do_for_each_ref_fn(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data);
struct ref_storage_be {
struct ref_storage_be *next;
@@ -240,6 +243,8 @@ struct ref_storage_be {
read_raw_ref_fn *read_raw_ref;
verify_refname_available_fn *verify_refname_available;
resolve_gitlink_ref_fn *resolve_gitlink_ref;
+
+ do_for_each_ref_fn *do_for_each_ref;
};
extern struct ref_storage_be refs_be_files;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 08/32] refs: reduce the visibility of do_for_each_ref()
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (6 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 07/32] refs: add method for do_for_each_ref David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 09/32] refs: add do_for_each_per_worktree_ref David Turner
` (25 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: Ramsay Jones
From: Ramsay Jones <ramsay@ramsayjones.plus.com>
Now that we have moved do_for_each_ref into refs.c, it no longer needs
to be public.
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---
refs.c | 19 +++++++++++--------
refs/refs-internal.h | 6 ------
2 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/refs.c b/refs.c
index cd06f4b..abc5b0d 100644
--- a/refs.c
+++ b/refs.c
@@ -1139,6 +1139,17 @@ int head_ref(each_ref_fn fn, void *cb_data)
return head_ref_submodule(NULL, fn, cb_data);
}
+/*
+ * The common backend for the for_each_*ref* functions
+ */
+static int do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data)
+{
+ return the_refs_backend->do_for_each_ref(submodule, base, fn, trim,
+ flags, cb_data);
+}
+
int for_each_ref(each_ref_fn fn, void *cb_data)
{
return do_for_each_ref(NULL, "", fn, 0, 0, cb_data);
@@ -1339,11 +1350,3 @@ int resolve_gitlink_ref(const char *path, const char *refname,
{
return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
}
-
-int do_for_each_ref(const char *submodule, const char *base,
- each_ref_fn fn, int trim, int flags,
- void *cb_data)
-{
- return the_refs_backend->do_for_each_ref(submodule, base, fn, trim,
- flags, cb_data);
-}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c9b6745..3702737 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -203,12 +203,6 @@ int rename_ref_available(const char *oldname, const char *newname);
/* Include broken references in a do_for_each_ref*() iteration: */
#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
-/*
- * The common backend for the for_each_*ref* functions
- */
-int do_for_each_ref(const char *submodule, const char *base,
- each_ref_fn fn, int trim, int flags, void *cb_data);
-
/* refs backends */
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 09/32] refs: add do_for_each_per_worktree_ref
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (7 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 08/32] refs: reduce the visibility of do_for_each_ref() David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 10/32] refs: add methods for reflog David Turner
` (24 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Alternate refs backends might still use files to store per-worktree
refs. So the files backend's ref-loading infrastructure should be
available to those backends, just for use on per-worktree refs. Add
do_for_each_per_worktree_ref, which iterates over per-worktree refs.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs/files-backend.c | 16 ++++++++++++++++
refs/refs-internal.h | 7 +++++++
2 files changed, 23 insertions(+)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index db9e948..d315f40 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -565,6 +565,10 @@ static int do_one_ref(struct ref_entry *entry, void *cb_data)
struct ref_entry *old_current_ref;
int retval;
+ if (data->flags & DO_FOR_EACH_PER_WORKTREE_ONLY &&
+ ref_type(entry->name) != REF_TYPE_PER_WORKTREE)
+ return 0;
+
if (!starts_with(entry->name, data->base))
return 0;
@@ -1701,6 +1705,18 @@ static int files_do_for_each_ref(const char *submodule, const char *base,
return do_for_each_entry(refs, base, do_one_ref, &data);
}
+int do_for_each_per_worktree_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data)
+{
+ /*
+ * It's important that this one use the files backend, since
+ * that's what controls the per-worktree refs
+ */
+ return files_do_for_each_ref(submodule, base, fn, trim,
+ flags | DO_FOR_EACH_PER_WORKTREE_ONLY, cb_data);
+}
+
static void unlock_ref(struct ref_lock *lock)
{
/* Do not free lock->lk -- atexit() still looks at them */
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3702737..bb0d588 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -203,6 +203,13 @@ int rename_ref_available(const char *oldname, const char *newname);
/* Include broken references in a do_for_each_ref*() iteration: */
#define DO_FOR_EACH_INCLUDE_BROKEN 0x01
+/* Only include per-worktree refs in a do_for_each_ref*() iteration */
+#define DO_FOR_EACH_PER_WORKTREE_ONLY 0x02
+
+int do_for_each_per_worktree_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data);
+
/* refs backends */
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 10/32] refs: add methods for reflog
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (8 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 09/32] refs: add do_for_each_per_worktree_ref David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 11/32] refs: add method for initial ref transaction commit David Turner
` (23 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner, Ronnie Sahlberg
In the file-based backend, the reflog piggybacks on the ref lock.
Since other backends won't have the same sort of ref lock, ref backends
must also handle reflogs.
Signed-off-by: Ronnie Sahlberg <rsahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
refs/files-backend.c | 36 ++++++++++++++++++++++++------------
refs/refs-internal.h | 27 +++++++++++++++++++++++++++
3 files changed, 97 insertions(+), 12 deletions(-)
diff --git a/refs.c b/refs.c
index abc5b0d..9f06686 100644
--- a/refs.c
+++ b/refs.c
@@ -1350,3 +1350,49 @@ int resolve_gitlink_ref(const char *path, const char *refname,
{
return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
}
+
+int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ return the_refs_backend->for_each_reflog_ent_reverse(refname, fn,
+ cb_data);
+}
+
+int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ return the_refs_backend->for_each_reflog_ent(refname, fn, cb_data);
+}
+
+int for_each_reflog(each_ref_fn fn, void *cb_data)
+{
+ return the_refs_backend->for_each_reflog(fn, cb_data);
+}
+
+int reflog_exists(const char *refname)
+{
+ return the_refs_backend->reflog_exists(refname);
+}
+
+int safe_create_reflog(const char *refname, int force_create,
+ struct strbuf *err)
+{
+ return the_refs_backend->create_reflog(refname, force_create, err);
+}
+
+int delete_reflog(const char *refname)
+{
+ return the_refs_backend->delete_reflog(refname);
+}
+
+int reflog_expire(const char *refname, const unsigned char *sha1,
+ unsigned int flags,
+ reflog_expiry_prepare_fn prepare_fn,
+ reflog_expiry_should_prune_fn should_prune_fn,
+ reflog_expiry_cleanup_fn cleanup_fn,
+ void *policy_cb_data)
+{
+ return the_refs_backend->reflog_expire(refname, sha1, flags,
+ prepare_fn, should_prune_fn,
+ cleanup_fn, policy_cb_data);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d315f40..5e44b10 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2533,7 +2533,8 @@ static int log_ref_setup(const char *refname, struct strbuf *logfile, struct str
}
-int safe_create_reflog(const char *refname, int force_create, struct strbuf *err)
+static int files_create_reflog(const char *refname, int force_create,
+ struct strbuf *err)
{
int ret;
struct strbuf sb = STRBUF_INIT;
@@ -2789,7 +2790,7 @@ static int files_create_symref(const char *refname,
return ret;
}
-int reflog_exists(const char *refname)
+static int files_reflog_exists(const char *refname)
{
struct stat st;
@@ -2797,7 +2798,7 @@ int reflog_exists(const char *refname)
S_ISREG(st.st_mode);
}
-int delete_reflog(const char *refname)
+static int files_delete_reflog(const char *refname)
{
return remove_path(git_path("logs/%s", refname));
}
@@ -2841,7 +2842,9 @@ static char *find_beginning_of_line(char *bob, char *scan)
return scan;
}
-int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void *cb_data)
+static int files_for_each_reflog_ent_reverse(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data)
{
struct strbuf sb = STRBUF_INIT;
FILE *logfp;
@@ -2943,7 +2946,8 @@ int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void
return ret;
}
-int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn, void *cb_data)
+static int files_for_each_reflog_ent(const char *refname,
+ each_reflog_ent_fn fn, void *cb_data)
{
FILE *logfp;
struct strbuf sb = STRBUF_INIT;
@@ -3005,7 +3009,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
return retval;
}
-int for_each_reflog(each_ref_fn fn, void *cb_data)
+static int files_for_each_reflog(each_ref_fn fn, void *cb_data)
{
int retval;
struct strbuf name;
@@ -3315,12 +3319,12 @@ static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
return 0;
}
-int reflog_expire(const char *refname, const unsigned char *sha1,
- unsigned int flags,
- reflog_expiry_prepare_fn prepare_fn,
- reflog_expiry_should_prune_fn should_prune_fn,
- reflog_expiry_cleanup_fn cleanup_fn,
- void *policy_cb_data)
+static int files_reflog_expire(const char *refname, const unsigned char *sha1,
+ unsigned int flags,
+ reflog_expiry_prepare_fn prepare_fn,
+ reflog_expiry_should_prune_fn should_prune_fn,
+ reflog_expiry_cleanup_fn cleanup_fn,
+ void *policy_cb_data)
{
static struct lock_file reflog_lock;
struct expire_reflog_cb cb;
@@ -3425,6 +3429,14 @@ struct ref_storage_be refs_be_files = {
"files",
files_transaction_commit,
+ files_for_each_reflog_ent,
+ files_for_each_reflog_ent_reverse,
+ files_for_each_reflog,
+ files_reflog_exists,
+ files_create_reflog,
+ files_delete_reflog,
+ files_reflog_expire,
+
files_pack_refs,
files_peel_ref,
files_create_symref,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index bb0d588..c92e0c3 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -214,6 +214,25 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
+/* reflog functions */
+typedef int for_each_reflog_ent_fn(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data);
+typedef int for_each_reflog_ent_reverse_fn(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data);
+typedef int for_each_reflog_fn(each_ref_fn fn, void *cb_data);
+typedef int reflog_exists_fn(const char *refname);
+typedef int create_reflog_fn(const char *refname, int force_create,
+ struct strbuf *err);
+typedef int delete_reflog_fn(const char *refname);
+typedef int reflog_expire_fn(const char *refname, const unsigned char *sha1,
+ unsigned int flags,
+ reflog_expiry_prepare_fn prepare_fn,
+ reflog_expiry_should_prune_fn should_prune_fn,
+ reflog_expiry_cleanup_fn cleanup_fn,
+ void *policy_cb_data);
+
/* misc methods */
typedef int pack_refs_fn(unsigned int flags);
typedef int peel_ref_fn(const char *refname, unsigned char *sha1);
@@ -237,6 +256,14 @@ struct ref_storage_be {
const char *name;
ref_transaction_commit_fn *transaction_commit;
+ for_each_reflog_ent_fn *for_each_reflog_ent;
+ for_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;
+ for_each_reflog_fn *for_each_reflog;
+ reflog_exists_fn *reflog_exists;
+ create_reflog_fn *create_reflog;
+ delete_reflog_fn *delete_reflog;
+ reflog_expire_fn *reflog_expire;
+
pack_refs_fn *pack_refs;
peel_ref_fn *peel_ref;
create_symref_fn *create_symref;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 11/32] refs: add method for initial ref transaction commit
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (9 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 10/32] refs: add methods for reflog David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 12/32] refs: add method for delete_refs David Turner
` (22 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner, Ronnie Sahlberg
Signed-off-by: Ronnie Sahlberg <rsahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 6 ++++++
refs/files-backend.c | 5 +++--
refs/refs-internal.h | 1 +
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 9f06686..5ae1903 100644
--- a/refs.c
+++ b/refs.c
@@ -1396,3 +1396,9 @@ int reflog_expire(const char *refname, const unsigned char *sha1,
prepare_fn, should_prune_fn,
cleanup_fn, policy_cb_data);
}
+
+int initial_ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return the_refs_backend->initial_transaction_commit(transaction, err);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 5e44b10..7621a41 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3203,8 +3203,8 @@ static int ref_present(const char *refname,
return string_list_has_string(affected_refnames, refname);
}
-int initial_ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+static int files_initial_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
@@ -3428,6 +3428,7 @@ struct ref_storage_be refs_be_files = {
NULL,
"files",
files_transaction_commit,
+ files_initial_transaction_commit,
files_for_each_reflog_ent,
files_for_each_reflog_ent_reverse,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c92e0c3..5aa3fa0 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -255,6 +255,7 @@ struct ref_storage_be {
struct ref_storage_be *next;
const char *name;
ref_transaction_commit_fn *transaction_commit;
+ ref_transaction_commit_fn *initial_transaction_commit;
for_each_reflog_ent_fn *for_each_reflog_ent;
for_each_reflog_ent_reverse_fn *for_each_reflog_ent_reverse;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 12/32] refs: add method for delete_refs
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (10 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 11/32] refs: add method for initial ref transaction commit David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 13/32] refs: add methods to init refs db David Turner
` (21 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
In the file-based backend, delete_refs has some special optimization
to deal with packed refs. In other backends, we might be able to make
ref deletion faster by putting all deletions into a single
transaction. So we need a special backend function for this.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 5 +++++
refs/files-backend.c | 3 ++-
refs/refs-internal.h | 2 ++
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index 5ae1903..21b103c 100644
--- a/refs.c
+++ b/refs.c
@@ -1402,3 +1402,8 @@ int initial_ref_transaction_commit(struct ref_transaction *transaction,
{
return the_refs_backend->initial_transaction_commit(transaction, err);
}
+
+int delete_refs(struct string_list *refnames)
+{
+ return the_refs_backend->delete_refs(refnames);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7621a41..6b15a2e 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2246,7 +2246,7 @@ static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
return 0;
}
-int delete_refs(struct string_list *refnames)
+static int files_delete_refs(struct string_list *refnames)
{
struct strbuf err = STRBUF_INIT;
int i, result = 0;
@@ -3441,6 +3441,7 @@ struct ref_storage_be refs_be_files = {
files_pack_refs,
files_peel_ref,
files_create_symref,
+ files_delete_refs,
files_read_raw_ref,
files_verify_refname_available,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 5aa3fa0..beef457 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -239,6 +239,7 @@ typedef int peel_ref_fn(const char *refname, unsigned char *sha1);
typedef int create_symref_fn(const char *ref_target,
const char *refs_heads_master,
const char *logmsg);
+typedef int delete_refs_fn(struct string_list *refnames);
/* resolution methods */
typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,
@@ -268,6 +269,7 @@ struct ref_storage_be {
pack_refs_fn *pack_refs;
peel_ref_fn *peel_ref;
create_symref_fn *create_symref;
+ delete_refs_fn *delete_refs;
read_raw_ref_fn *read_raw_ref;
verify_refname_available_fn *verify_refname_available;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 13/32] refs: add methods to init refs db
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (11 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 12/32] refs: add method for delete_refs David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 14/32] refs: add method to rename refs David Turner
` (20 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Alternate refs backends might not need the refs/heads directory and so
on, so we make ref db initialization part of the backend.
Signed-off-by: David Turner <dturner@twopensource.com>
---
builtin/init-db.c | 20 ++++++++++----------
refs.c | 5 +++++
refs.h | 2 ++
refs/files-backend.c | 16 ++++++++++++++++
refs/refs-internal.h | 2 ++
5 files changed, 35 insertions(+), 10 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 07229d6..0c8f4ac 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -177,13 +177,7 @@ static int create_default_files(const char *template_path)
char junk[2];
int reinit;
int filemode;
-
- /*
- * Create .git/refs/{heads,tags}
- */
- safe_create_dir(git_path_buf(&buf, "refs"), 1);
- safe_create_dir(git_path_buf(&buf, "refs/heads"), 1);
- safe_create_dir(git_path_buf(&buf, "refs/tags"), 1);
+ struct strbuf err = STRBUF_INIT;
/* Just look for `init.templatedir` */
git_config(git_init_db_config, NULL);
@@ -207,12 +201,18 @@ static int create_default_files(const char *template_path)
*/
if (shared_repository) {
adjust_shared_perm(get_git_dir());
- adjust_shared_perm(git_path_buf(&buf, "refs"));
- adjust_shared_perm(git_path_buf(&buf, "refs/heads"));
- adjust_shared_perm(git_path_buf(&buf, "refs/tags"));
}
/*
+ * We need to create a "refs" dir in any case so that older
+ * versions of git can tell that this is a repository.
+ */
+ safe_create_dir(git_path("refs"), 1);
+
+ if (refs_init_db(shared_repository, &err))
+ die("failed to set up refs db: %s", err.buf);
+
+ /*
* Create the default symlink from ".git/HEAD" to the "master"
* branch, if it does not exist yet.
*/
diff --git a/refs.c b/refs.c
index 21b103c..baf83ac 100644
--- a/refs.c
+++ b/refs.c
@@ -1316,6 +1316,11 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
}
/* backend functions */
+int refs_init_db(int shared, struct strbuf *err)
+{
+ return the_refs_backend->init_db(shared, err);
+}
+
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
diff --git a/refs.h b/refs.h
index 5bc3fbc..feff82e 100644
--- a/refs.h
+++ b/refs.h
@@ -66,6 +66,8 @@ extern int ref_exists(const char *refname);
extern int is_branch(const char *refname);
+extern int refs_init_db(int shared, struct strbuf *err);
+
/*
* If refname is a non-symbolic reference that refers to a tag object,
* and the tag can be (recursively) dereferenced to a non-tag object,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6b15a2e..67ba4c2 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3424,9 +3424,25 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,
return -1;
}
+static int files_init_db(int shared, struct strbuf *err)
+{
+ /*
+ * Create .git/refs/{heads,tags}
+ */
+ safe_create_dir(git_path("refs/heads"), 1);
+ safe_create_dir(git_path("refs/tags"), 1);
+ if (shared) {
+ adjust_shared_perm(git_path("refs"));
+ adjust_shared_perm(git_path("refs/heads"));
+ adjust_shared_perm(git_path("refs/tags"));
+ }
+ return 0;
+}
+
struct ref_storage_be refs_be_files = {
NULL,
"files",
+ files_init_db,
files_transaction_commit,
files_initial_transaction_commit,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index beef457..dfd0326 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -211,6 +211,7 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,
void *cb_data);
/* refs backends */
+typedef int ref_init_db_fn(int shared, struct strbuf *err);
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
struct strbuf *err);
@@ -255,6 +256,7 @@ typedef int do_for_each_ref_fn(const char *submodule, const char *base,
struct ref_storage_be {
struct ref_storage_be *next;
const char *name;
+ ref_init_db_fn *init_db;
ref_transaction_commit_fn *transaction_commit;
ref_transaction_commit_fn *initial_transaction_commit;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 14/32] refs: add method to rename refs
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (12 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 13/32] refs: add methods to init refs db David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 15/32] refs: handle non-normal ref renames David Turner
` (19 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 5 +++++
refs/files-backend.c | 4 +++-
refs/refs-internal.h | 8 ++++++++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index baf83ac..3a2ad5e 100644
--- a/refs.c
+++ b/refs.c
@@ -1412,3 +1412,8 @@ int delete_refs(struct string_list *refnames)
{
return the_refs_backend->delete_refs(refnames);
}
+
+int rename_ref(const char *oldref, const char *newref, const char *logmsg)
+{
+ return the_refs_backend->rename_ref(oldref, newref, logmsg);
+}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 67ba4c2..4e4b852 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2368,7 +2368,8 @@ static int commit_ref_update(struct ref_lock *lock,
const unsigned char *sha1, const char *logmsg,
int flags, struct strbuf *err);
-int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
+static int files_rename_ref(const char *oldrefname, const char *newrefname,
+ const char *logmsg)
{
unsigned char sha1[20], orig_sha1[20];
int flag = 0, logmoved = 0;
@@ -3458,6 +3459,7 @@ struct ref_storage_be refs_be_files = {
files_peel_ref,
files_create_symref,
files_delete_refs,
+ files_rename_ref,
files_read_raw_ref,
files_verify_refname_available,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index dfd0326..d6fe199 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -195,6 +195,11 @@ const char *find_descendant_ref(const char *dirname,
const struct string_list *extras,
const struct string_list *skip);
+/*
+ * Check if the new name does not conflict with any existing refs
+ * (other than possibly the old ref). Return 0 if the ref can be
+ * renamed to the new name.
+ */
int rename_ref_available(const char *oldname, const char *newname);
/* We allow "recursive" symbolic refs. Only within reason, though */
@@ -241,6 +246,8 @@ typedef int create_symref_fn(const char *ref_target,
const char *refs_heads_master,
const char *logmsg);
typedef int delete_refs_fn(struct string_list *refnames);
+typedef int rename_ref_fn(const char *oldref, const char *newref,
+ const char *logmsg);
/* resolution methods */
typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,
@@ -272,6 +279,7 @@ struct ref_storage_be {
peel_ref_fn *peel_ref;
create_symref_fn *create_symref;
delete_refs_fn *delete_refs;
+ rename_ref_fn *rename_ref;
read_raw_ref_fn *read_raw_ref;
verify_refname_available_fn *verify_refname_available;
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 15/32] refs: handle non-normal ref renames
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (13 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 14/32] refs: add method to rename refs David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 16/32] refs: make lock generic David Turner
` (18 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Forbid cross-backend ref renames. This would be pretty weird, but
since it will break, we should prevent it.
Also make the files backend deal with all non-normal ref renames.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index 3a2ad5e..05b1b2a 100644
--- a/refs.c
+++ b/refs.c
@@ -1415,5 +1415,15 @@ int delete_refs(struct string_list *refnames)
int rename_ref(const char *oldref, const char *newref, const char *logmsg)
{
- return the_refs_backend->rename_ref(oldref, newref, logmsg);
+ if ((ref_type(oldref) == REF_TYPE_NORMAL) !=
+ (ref_type(newref) == REF_TYPE_NORMAL)) {
+ return error(
+ _("Both ref arguments to rename_ref must be normal, "
+ "or both must be per-worktree/pseudorefs"));
+ }
+ if (ref_type(oldref) == REF_TYPE_NORMAL)
+ /* The files backend always deals with non-normal refs */
+ return the_refs_backend->rename_ref(oldref, newref, logmsg);
+ else
+ return refs_be_files.rename_ref(oldref, newref, logmsg);
}
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 16/32] refs: make lock generic
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (14 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 15/32] refs: handle non-normal ref renames David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 17/32] refs: move duplicate check to common code David Turner
` (17 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Instead of using a files-backend-specific struct ref_lock, the generic
ref_transaction struct should provide a void pointer that backends can use
for their own lock data.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs/files-backend.c | 29 ++++++++++++++++-------------
refs/refs-internal.h | 2 +-
2 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4e4b852..9be3158 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3074,11 +3074,12 @@ static int files_transaction_commit(struct ref_transaction *transaction,
*/
for (i = 0; i < n; i++) {
struct ref_update *update = updates[i];
+ struct ref_lock *lock;
if ((update->flags & REF_HAVE_NEW) &&
is_null_sha1(update->new_sha1))
update->flags |= REF_DELETING;
- update->lock = lock_ref_sha1_basic(
+ lock = lock_ref_sha1_basic(
update->refname,
((update->flags & REF_HAVE_OLD) ?
update->old_sha1 : NULL),
@@ -3086,7 +3087,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,
update->flags,
&update->type,
err);
- if (!update->lock) {
+ update->backend_data = lock;
+ if (!lock) {
char *reason;
ret = (errno == ENOTDIR)
@@ -3104,12 +3106,12 @@ static int files_transaction_commit(struct ref_transaction *transaction,
(update->flags & REF_NODEREF));
if (!overwriting_symref &&
- !hashcmp(update->lock->old_oid.hash, update->new_sha1)) {
+ !hashcmp(lock->old_oid.hash, update->new_sha1)) {
/*
* The reference already has the desired
* value, so we don't need to write it.
*/
- } else if (write_ref_to_lockfile(update->lock,
+ } else if (write_ref_to_lockfile(lock,
update->new_sha1,
err)) {
char *write_err = strbuf_detach(err, NULL);
@@ -3118,7 +3120,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,
* The lock was freed upon failure of
* write_ref_to_lockfile():
*/
- update->lock = NULL;
+ update->backend_data = NULL;
strbuf_addf(err,
"cannot update the ref '%s': %s",
update->refname, write_err);
@@ -3134,7 +3136,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,
* We didn't have to write anything to the lockfile.
* Close it to free up the file descriptor:
*/
- if (close_ref(update->lock)) {
+ if (close_ref(lock)) {
strbuf_addf(err, "Couldn't close %s.lock",
update->refname);
goto cleanup;
@@ -3147,16 +3149,16 @@ static int files_transaction_commit(struct ref_transaction *transaction,
struct ref_update *update = updates[i];
if (update->flags & REF_NEEDS_COMMIT) {
- if (commit_ref_update(update->lock,
+ if (commit_ref_update(update->backend_data,
update->new_sha1, update->msg,
update->flags, err)) {
/* freed by commit_ref_update(): */
- update->lock = NULL;
+ update->backend_data = NULL;
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
} else {
/* freed by commit_ref_update(): */
- update->lock = NULL;
+ update->backend_data = NULL;
}
}
}
@@ -3164,16 +3166,17 @@ static int files_transaction_commit(struct ref_transaction *transaction,
/* Perform deletes now that updates are safely completed */
for (i = 0; i < n; i++) {
struct ref_update *update = updates[i];
+ struct ref_lock *lock = update->backend_data;
if (update->flags & REF_DELETING) {
- if (delete_ref_loose(update->lock, update->type, err)) {
+ if (delete_ref_loose(lock, update->type, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
}
if (!(update->flags & REF_ISPRUNING))
string_list_append(&refs_to_delete,
- update->lock->ref_name);
+ lock->ref_name);
}
}
@@ -3189,8 +3192,8 @@ cleanup:
transaction->state = REF_TRANSACTION_CLOSED;
for (i = 0; i < n; i++)
- if (updates[i]->lock)
- unlock_ref(updates[i]->lock);
+ if (updates[i]->backend_data)
+ unlock_ref(updates[i]->backend_data);
string_list_clear(&refs_to_delete, 0);
string_list_clear(&affected_refnames, 0);
return ret;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index d6fe199..8d091cb 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -144,7 +144,7 @@ struct ref_update {
* REF_DELETING, and REF_ISPRUNING:
*/
unsigned int flags;
- struct ref_lock *lock;
+ void *backend_data;
int type;
char *msg;
const char refname[FLEX_ARRAY];
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 17/32] refs: move duplicate check to common code
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (15 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 16/32] refs: make lock generic David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 18/32] refs: allow log-only updates David Turner
` (16 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
The check for duplicate refnames in a transaction is needed for
all backends, so move it to the common code.
ref_transaction_commit_fn gains a new argument, the sorted
string_list of affected refnames.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++--
refs/files-backend.c | 57 ++++---------------------------------------
refs/refs-internal.h | 1 +
3 files changed, 73 insertions(+), 54 deletions(-)
diff --git a/refs.c b/refs.c
index 05b1b2a..daf92ce 100644
--- a/refs.c
+++ b/refs.c
@@ -1140,6 +1140,36 @@ int head_ref(each_ref_fn fn, void *cb_data)
}
/*
+ * Return 1 if there are any duplicate refnames in the updates in
+ * `transaction`, and fill in err with an appropriate error message.
+ * Fill in `refnames` with the refnames from the transaction.
+ */
+static int get_affected_refnames(struct ref_transaction *transaction,
+ struct string_list *refnames,
+ struct strbuf *err)
+{
+ int i, n = transaction->nr;
+ struct ref_update **updates;
+
+ assert(err);
+
+ updates = transaction->updates;
+ /* Fail if a refname appears more than once in the transaction: */
+ for (i = 0; i < n; i++)
+ string_list_append(refnames, updates[i]->refname);
+ string_list_sort(refnames);
+
+ for (i = 1; i < n; i++)
+ if (!strcmp(refnames->items[i - 1].string, refnames->items[i].string)) {
+ strbuf_addf(err,
+ "Multiple updates for ref '%s' not allowed.",
+ refnames->items[i].string);
+ return 1;
+ }
+ return 0;
+}
+
+/*
* The common backend for the for_each_*ref* functions
*/
static int do_for_each_ref(const char *submodule, const char *base,
@@ -1324,7 +1354,29 @@ int refs_init_db(int shared, struct strbuf *err)
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
- return the_refs_backend->transaction_commit(transaction, err);
+ int ret = -1;
+ struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+
+ assert(err);
+
+ if (transaction->state != REF_TRANSACTION_OPEN)
+ die("BUG: commit called for transaction that is not open");
+
+ if (!transaction->nr) {
+ transaction->state = REF_TRANSACTION_CLOSED;
+ return 0;
+ }
+
+ if (get_affected_refnames(transaction, &affected_refnames, err)) {
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto done;
+ }
+
+ ret = the_refs_backend->transaction_commit(transaction,
+ &affected_refnames, err);
+done:
+ string_list_clear(&affected_refnames, 0);
+ return ret;
}
int verify_refname_available(const char *refname, struct string_list *extra,
@@ -1405,7 +1457,20 @@ int reflog_expire(const char *refname, const unsigned char *sha1,
int initial_ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
- return the_refs_backend->initial_transaction_commit(transaction, err);
+ struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+ int ret;
+
+ if (get_affected_refnames(transaction,
+ &affected_refnames, err)) {
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto done;
+ }
+ ret = the_refs_backend->initial_transaction_commit(transaction,
+ &affected_refnames,
+ err);
+done:
+ string_list_clear(&affected_refnames, 0);
+ return ret;
}
int delete_refs(struct string_list *refnames)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 9be3158..413db22 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3020,24 +3020,8 @@ static int files_for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
}
-static int ref_update_reject_duplicates(struct string_list *refnames,
- struct strbuf *err)
-{
- int i, n = refnames->nr;
-
- assert(err);
-
- for (i = 1; i < n; i++)
- if (!strcmp(refnames->items[i - 1].string, refnames->items[i].string)) {
- strbuf_addf(err,
- "Multiple updates for ref '%s' not allowed.",
- refnames->items[i].string);
- return 1;
- }
- return 0;
-}
-
static int files_transaction_commit(struct ref_transaction *transaction,
+ struct string_list *affected_refnames,
struct strbuf *err)
{
int ret = 0, i;
@@ -3045,26 +3029,6 @@ static int files_transaction_commit(struct ref_transaction *transaction,
struct ref_update **updates = transaction->updates;
struct string_list refs_to_delete = STRING_LIST_INIT_NODUP;
struct string_list_item *ref_to_delete;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
-
- assert(err);
-
- if (transaction->state != REF_TRANSACTION_OPEN)
- die("BUG: commit called for transaction that is not open");
-
- if (!n) {
- transaction->state = REF_TRANSACTION_CLOSED;
- return 0;
- }
-
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < n; i++)
- string_list_append(&affected_refnames, updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
/*
* Acquire all locks, verify old values if provided, check
@@ -3083,7 +3047,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,
update->refname,
((update->flags & REF_HAVE_OLD) ?
update->old_sha1 : NULL),
- &affected_refnames, NULL,
+ affected_refnames, NULL,
update->flags,
&update->type,
err);
@@ -3195,7 +3159,6 @@ cleanup:
if (updates[i]->backend_data)
unlock_ref(updates[i]->backend_data);
string_list_clear(&refs_to_delete, 0);
- string_list_clear(&affected_refnames, 0);
return ret;
}
@@ -3208,27 +3171,18 @@ static int ref_present(const char *refname,
}
static int files_initial_transaction_commit(struct ref_transaction *transaction,
+ struct string_list *affected_refnames,
struct strbuf *err)
{
int ret = 0, i;
int n = transaction->nr;
struct ref_update **updates = transaction->updates;
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
assert(err);
if (transaction->state != REF_TRANSACTION_OPEN)
die("BUG: commit called for transaction that is not open");
- /* Fail if a refname appears more than once in the transaction: */
- for (i = 0; i < n; i++)
- string_list_append(&affected_refnames, updates[i]->refname);
- string_list_sort(&affected_refnames);
- if (ref_update_reject_duplicates(&affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto cleanup;
- }
-
/*
* It's really undefined to call this function in an active
* repository or when there are existing references: we are
@@ -3241,7 +3195,7 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,
* so here we really only check that none of the references
* that we are creating already exists.
*/
- if (for_each_rawref(ref_present, &affected_refnames))
+ if (for_each_rawref(ref_present, affected_refnames))
die("BUG: initial ref transaction called with existing refs");
for (i = 0; i < n; i++) {
@@ -3251,7 +3205,7 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,
!is_null_sha1(update->old_sha1))
die("BUG: initial ref transaction with old_sha1 set");
if (verify_refname_available(update->refname,
- &affected_refnames, NULL,
+ affected_refnames, NULL,
err)) {
ret = TRANSACTION_NAME_CONFLICT;
goto cleanup;
@@ -3282,7 +3236,6 @@ static int files_initial_transaction_commit(struct ref_transaction *transaction,
cleanup:
transaction->state = REF_TRANSACTION_CLOSED;
- string_list_clear(&affected_refnames, 0);
return ret;
}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 8d091cb..94d162e 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -218,6 +218,7 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,
/* refs backends */
typedef int ref_init_db_fn(int shared, struct strbuf *err);
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
+ struct string_list *affected_refnames,
struct strbuf *err);
/* reflog functions */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 18/32] refs: allow log-only updates
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (16 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 17/32] refs: move duplicate check to common code David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 19/32] refs: don't dereference on rename David Turner
` (15 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
The refs infrastructure learns about log-only ref updates, which only
update the reflog. Later, we will use this to separate symbolic
reference resolution from ref updating.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs/files-backend.c | 15 ++++++++++-----
refs/refs-internal.h | 7 +++++++
2 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 413db22..8dc80e2 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2711,7 +2711,7 @@ static int commit_ref_update(struct ref_lock *lock,
}
}
}
- if (commit_ref(lock)) {
+ if (!(flags & REF_LOG_ONLY) && commit_ref(lock)) {
error("Couldn't set %s", lock->ref_name);
unlock_ref(lock);
return -1;
@@ -3065,7 +3065,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,
goto cleanup;
}
if ((update->flags & REF_HAVE_NEW) &&
- !(update->flags & REF_DELETING)) {
+ !(update->flags & REF_DELETING) &&
+ !(update->flags & REF_LOG_ONLY)) {
int overwriting_symref = ((update->type & REF_ISSYMREF) &&
(update->flags & REF_NODEREF));
@@ -3095,7 +3096,9 @@ static int files_transaction_commit(struct ref_transaction *transaction,
update->flags |= REF_NEEDS_COMMIT;
}
}
- if (!(update->flags & REF_NEEDS_COMMIT)) {
+
+ if (!(update->flags & REF_LOG_ONLY) &&
+ !(update->flags & REF_NEEDS_COMMIT)) {
/*
* We didn't have to write anything to the lockfile.
* Close it to free up the file descriptor:
@@ -3112,7 +3115,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,
for (i = 0; i < n; i++) {
struct ref_update *update = updates[i];
- if (update->flags & REF_NEEDS_COMMIT) {
+ if (update->flags & REF_NEEDS_COMMIT ||
+ update->flags & REF_LOG_ONLY) {
if (commit_ref_update(update->backend_data,
update->new_sha1, update->msg,
update->flags, err)) {
@@ -3132,7 +3136,8 @@ static int files_transaction_commit(struct ref_transaction *transaction,
struct ref_update *update = updates[i];
struct ref_lock *lock = update->backend_data;
- if (update->flags & REF_DELETING) {
+ if (update->flags & REF_DELETING &&
+ !(update->flags & REF_LOG_ONLY)) {
if (delete_ref_loose(lock, update->type, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto cleanup;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 94d162e..dd76246 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -43,6 +43,13 @@
*/
/*
+ * Used as a flag in ref_update::flags when we want to log a ref
+ * update but not actually perform it. This is used when a symbolic
+ * ref update is split up.
+ */
+#define REF_LOG_ONLY 0x80
+
+/*
* Return true iff refname is minimally safe. "Safe" here means that
* deleting a loose reference by this name will not do any damage, for
* example by causing a file that is not a reference to be deleted.
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 19/32] refs: don't dereference on rename
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (17 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 18/32] refs: allow log-only updates David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 20/32] refs: on symref reflog expire, lock symref not referrent David Turner
` (14 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
When renaming refs, don't dereference either the origin or the destination
before renaming.
The origin does not need to be dereferenced because it is presently
forbidden to rename symbolic refs.
Not dereferencing the destination fixes a bug where renaming on top of
a broken symref would use the pointed-to ref name for the moved
reflog.
Add a test for the reflog bug.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs/files-backend.c | 15 +++++++++------
t/t3200-branch.sh | 9 +++++++++
2 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8dc80e2..7a3c9df 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2372,7 +2372,7 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
const char *logmsg)
{
unsigned char sha1[20], orig_sha1[20];
- int flag = 0, logmoved = 0;
+ int flag = 0, logmoved = 0, resolve_flags;
struct ref_lock *lock;
struct stat loginfo;
int log = !lstat(git_path("logs/%s", oldrefname), &loginfo);
@@ -2382,7 +2382,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
if (log && S_ISLNK(loginfo.st_mode))
return error("reflog for %s is a symlink", oldrefname);
- symref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,
+ resolve_flags = RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE;
+ symref = resolve_ref_unsafe(oldrefname, resolve_flags,
orig_sha1, &flag);
if (flag & REF_ISSYMREF)
return error("refname %s is a symbolic ref, renaming it is not supported",
@@ -2402,8 +2403,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
goto rollback;
}
- if (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL) &&
- delete_ref(newrefname, sha1, REF_NODEREF)) {
+ if (!read_ref_full(newrefname, resolve_flags, sha1, NULL) &&
+ delete_ref(newrefname, NULL, REF_NODEREF)) {
if (errno==EISDIR) {
struct strbuf path = STRBUF_INIT;
int result;
@@ -2427,7 +2428,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
logmoved = log;
- lock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, 0, NULL, &err);
+ lock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, REF_NODEREF,
+ NULL, &err);
if (!lock) {
error("unable to rename '%s' to '%s': %s", oldrefname, newrefname, err.buf);
strbuf_release(&err);
@@ -2445,7 +2447,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
return 0;
rollback:
- lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, 0, NULL, &err);
+ lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, REF_NODEREF,
+ NULL, &err);
if (!lock) {
error("unable to lock %s for rollback: %s", oldrefname, err.buf);
strbuf_release(&err);
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index cdaf6f6..07e9749 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -79,6 +79,15 @@ test_expect_success 'git branch -m dumps usage' '
test_i18ngrep "branch name required" err
'
+test_expect_success 'git branch -m m broken_symref should work' '
+ test_when_finished "git branch -D broken_symref" &&
+ git branch -l m &&
+ git symbolic-ref refs/heads/broken_symref refs/heads/i_am_broken &&
+ git branch -m m broken_symref &&
+ git reflog exists refs/heads/broken_symref &&
+ test_must_fail git reflog exists refs/heads/i_am_broken
+'
+
test_expect_success 'git branch -m m m/m should work' '
git branch -l m &&
git branch -m m m/m &&
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 20/32] refs: on symref reflog expire, lock symref not referrent
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (18 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 19/32] refs: don't dereference on rename David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 21/32] refs: resolve symbolic refs first David Turner
` (13 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
When locking a symbolic ref to expire a reflog, lock the symbolic
ref (using REF_NODEREF) instead of its referent.
Add a test for this.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs/files-backend.c | 3 ++-
t/t1410-reflog.sh | 10 ++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7a3c9df..f1045b4 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3309,7 +3309,8 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,
* reference itself, plus we might need to update the
* reference if --updateref was specified:
*/
- lock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, 0, &type, &err);
+ lock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, REF_NODEREF,
+ &type, &err);
if (!lock) {
error("cannot lock ref '%s': %s", refname, err.buf);
strbuf_release(&err);
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index c623824..9cf91dc 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -338,4 +338,14 @@ test_expect_failure 'reflog with non-commit entries displays all entries' '
test_line_count = 3 actual
'
+test_expect_success 'reflog expire operates on symref not referrent' '
+ git branch -l the_symref &&
+ git branch -l referrent &&
+ git update-ref referrent HEAD &&
+ git symbolic-ref refs/heads/the_symref refs/heads/referrent &&
+ test_when_finished "rm -f .git/refs/heads/referrent.lock" &&
+ touch .git/refs/heads/referrent.lock &&
+ git reflog expire --expire=all the_symref
+'
+
test_done
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 21/32] refs: resolve symbolic refs first
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (19 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 20/32] refs: on symref reflog expire, lock symref not referrent David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 22/32] refs: always handle non-normal refs in files backend David Turner
` (12 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Before committing ref updates, split symbolic ref updates into two
parts: an update to the underlying ref, and a log-only update to the
symbolic ref. This ensures that both references are locked correctly
while their reflogs are updated.
It is still possible to confuse git by concurrent updates, since the
splitting of symbolic refs does not happen under lock. So a symbolic ref
could be replaced by a plain ref in the middle of this operation, which
would lead to reflog discontinuities and missed old-ref checks.
All callers to lock_ref_sha1_basic outside of ref_transaction_commit
now set REF_NODEREF. And ref_transaction_commit only happens on
already-derefernced refs. So we can assume REF_NODEREF when resolving
inside this function.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 69 ++++++++++++++++++++++++++
refs/files-backend.c | 137 ++++++++++++++++++++++++++-------------------------
refs/refs-internal.h | 8 +++
3 files changed, 148 insertions(+), 66 deletions(-)
diff --git a/refs.c b/refs.c
index daf92ce..bdde808 100644
--- a/refs.c
+++ b/refs.c
@@ -1351,6 +1351,71 @@ int refs_init_db(int shared, struct strbuf *err)
return the_refs_backend->init_db(shared, err);
}
+/*
+ * Special case for symbolic refs when REF_NODEREF is not turned on.
+ * Dereference them here, mark them REF_LOG_ONLY, and add an update
+ * for the underlying ref.
+ */
+static int dereference_symrefs(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ int i;
+ int nr = transaction->nr;
+
+ for (i = 0; i < nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+ const char *resolved;
+ unsigned char sha1[20];
+ int resolve_flags = 0;
+ int mustexist = update->flags & REF_HAVE_OLD &&
+ !is_null_sha1(update->old_sha1);
+ int deleting = (update->flags & REF_HAVE_NEW) &&
+ is_null_sha1(update->new_sha1);
+
+ if (mustexist)
+ resolve_flags |= RESOLVE_REF_READING;
+ if (deleting)
+ resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME |
+ RESOLVE_REF_NO_RECURSE;
+
+ if (strcmp(update->refname, "HEAD"))
+ update->flags |= REF_IS_NOT_HEAD;
+
+ resolved = resolve_ref_unsafe(update->refname, resolve_flags,
+ sha1, &update->type);
+ if (!resolved) {
+ /*
+ * We may notice this breakage later and die
+ * with a sensible error message
+ */
+ update->type |= REF_ISBROKEN;
+ continue;
+ }
+
+ hashcpy(update->read_sha1, sha1);
+
+ if (update->flags & REF_NODEREF ||
+ !(update->type & REF_ISSYMREF))
+ continue;
+
+ /* Create a new transaction for the underlying ref */
+ if (ref_transaction_update(transaction,
+ resolved,
+ update->new_sha1,
+ (update->flags & REF_HAVE_OLD) ?
+ update->old_sha1 : NULL,
+ update->flags & ~REF_IS_NOT_HEAD,
+ update->msg, err))
+ return -1;
+
+ /* Make the symbolic ref update non-recursive */
+ update->flags |= REF_LOG_ONLY | REF_NODEREF;
+ update->flags &= ~REF_HAVE_OLD;
+ }
+
+ return 0;
+}
+
int ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
@@ -1367,6 +1432,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
return 0;
}
+ ret = dereference_symrefs(transaction, err);
+ if (ret)
+ goto done;
+
if (get_affected_refnames(transaction, &affected_refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto done;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f1045b4..f378f06 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -7,7 +7,6 @@
struct ref_lock {
char *ref_name;
- char *orig_ref_name;
struct lock_file *lk;
struct object_id old_oid;
};
@@ -1723,7 +1722,6 @@ static void unlock_ref(struct ref_lock *lock)
if (lock->lk)
rollback_lock_file(lock->lk);
free(lock->ref_name);
- free(lock->orig_ref_name);
free(lock);
}
@@ -1779,6 +1777,7 @@ static int remove_empty_directories(struct strbuf *path)
*/
static struct ref_lock *lock_ref_sha1_basic(const char *refname,
const unsigned char *old_sha1,
+ const unsigned char *read_sha1,
const struct string_list *extras,
const struct string_list *skip,
unsigned int flags, int *type_p,
@@ -1786,14 +1785,14 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
{
struct strbuf ref_file = STRBUF_INIT;
struct strbuf orig_ref_file = STRBUF_INIT;
- const char *orig_refname = refname;
struct ref_lock *lock;
int last_errno = 0;
int type;
int lflags = 0;
int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
- int resolve_flags = 0;
+ int resolve_flags = RESOLVE_REF_NO_RECURSE;
int attempts_remaining = 3;
+ int resolved;
assert(err);
@@ -1803,65 +1802,65 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
resolve_flags |= RESOLVE_REF_READING;
if (flags & REF_DELETING)
resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
- if (flags & REF_NODEREF) {
- resolve_flags |= RESOLVE_REF_NO_RECURSE;
+ if (flags & REF_NODEREF)
lflags |= LOCK_NO_DEREF;
- }
- refname = resolve_ref_unsafe(refname, resolve_flags,
- lock->old_oid.hash, &type);
- if (!refname && errno == EISDIR) {
- /*
- * we are trying to lock foo but we used to
- * have foo/bar which now does not exist;
- * it is normal for the empty directory 'foo'
- * to remain.
- */
- strbuf_git_path(&orig_ref_file, "%s", orig_refname);
- if (remove_empty_directories(&orig_ref_file)) {
+ if (type_p && *type_p & REF_ISSYMREF) {
+ hashcpy(lock->old_oid.hash, read_sha1);
+ } else {
+ resolved = !!resolve_ref_unsafe(refname, resolve_flags,
+ lock->old_oid.hash, &type);
+ if (!resolved && errno == EISDIR) {
+ /*
+ * we are trying to lock foo but we used to
+ * have foo/bar which now does not exist;
+ * it is normal for the empty directory 'foo'
+ * to remain.
+ */
+ strbuf_git_path(&orig_ref_file, "%s", refname);
+ if (remove_empty_directories(&orig_ref_file)) {
+ struct ref_dir *loose_refs;
+ loose_refs = get_loose_refs(&ref_cache);
+ last_errno = errno;
+ if (!verify_refname_available_dir(refname, extras, skip,
+ loose_refs, err))
+ strbuf_addf(err, "there are still refs under '%s'",
+ refname);
+ goto error_return;
+ }
+ resolved = !!resolve_ref_unsafe(refname, resolve_flags,
+ lock->old_oid.hash, &type);
+ }
+
+ if (type_p)
+ *type_p = type;
+ if (!resolved) {
last_errno = errno;
- if (!verify_refname_available_dir(orig_refname, extras, skip,
+ if (last_errno != ENOTDIR ||
+ !verify_refname_available_dir(refname, extras, skip,
get_loose_refs(&ref_cache), err))
- strbuf_addf(err, "there are still refs under '%s'",
- orig_refname);
+ strbuf_addf(err,
+ "unable to resolve reference %s: %s",
+ refname, strerror(last_errno));
+
+ goto error_return;
+ }
+ /*
+ * If the ref did not exist and we are creating it, make sure
+ * there is no existing packed ref whose name begins with our
+ * refname, nor a packed ref whose name is a proper prefix of
+ * our refname.
+ */
+ if (is_null_oid(&lock->old_oid) &&
+ verify_refname_available_dir(refname, extras, skip,
+ get_packed_refs(&ref_cache), err)) {
+ last_errno = ENOTDIR;
goto error_return;
}
- refname = resolve_ref_unsafe(orig_refname, resolve_flags,
- lock->old_oid.hash, &type);
- }
- if (type_p)
- *type_p = type;
- if (!refname) {
- last_errno = errno;
- if (last_errno != ENOTDIR ||
- !verify_refname_available_dir(orig_refname, extras, skip,
- get_loose_refs(&ref_cache), err))
- strbuf_addf(err, "unable to resolve reference %s: %s",
- orig_refname, strerror(last_errno));
-
- goto error_return;
- }
-
- if (flags & REF_NODEREF)
- refname = orig_refname;
-
- /*
- * If the ref did not exist and we are creating it, make sure
- * there is no existing packed ref whose name begins with our
- * refname, nor a packed ref whose name is a proper prefix of
- * our refname.
- */
- if (is_null_oid(&lock->old_oid) &&
- verify_refname_available_dir(refname, extras, skip,
- get_packed_refs(&ref_cache), err)) {
- last_errno = ENOTDIR;
- goto error_return;
}
-
lock->lk = xcalloc(1, sizeof(struct lock_file));
lock->ref_name = xstrdup(refname);
- lock->orig_ref_name = xstrdup(orig_refname);
strbuf_git_path(&ref_file, "%s", refname);
retry:
@@ -1893,7 +1892,13 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
goto error_return;
}
}
- if (verify_lock(lock, old_sha1, mustexist, err)) {
+
+ if (type_p && *type_p & REF_ISSYMREF && !(*type_p & REF_ISBROKEN)) {
+ /*
+ * Old hash verification for symrefs happens on their
+ * base ref.
+ */
+ } else if (verify_lock(lock, old_sha1, mustexist, err)) {
last_errno = errno;
goto error_return;
}
@@ -2428,8 +2433,8 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
logmoved = log;
- lock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, REF_NODEREF,
- NULL, &err);
+ lock = lock_ref_sha1_basic(newrefname, NULL, NULL, NULL, NULL,
+ REF_NODEREF, NULL, &err);
if (!lock) {
error("unable to rename '%s' to '%s': %s", oldrefname, newrefname, err.buf);
strbuf_release(&err);
@@ -2447,8 +2452,9 @@ static int files_rename_ref(const char *oldrefname, const char *newrefname,
return 0;
rollback:
- lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, REF_NODEREF,
- NULL, &err);
+ lock = lock_ref_sha1_basic(oldrefname, NULL, NULL, NULL, NULL,
+ REF_NODEREF, NULL, &err);
+
if (!lock) {
error("unable to lock %s for rollback: %s", oldrefname, err.buf);
strbuf_release(&err);
@@ -2676,9 +2682,7 @@ static int commit_ref_update(struct ref_lock *lock,
int flags, struct strbuf *err)
{
clear_loose_ref_cache(&ref_cache);
- if (log_ref_write(lock->ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0 ||
- (strcmp(lock->ref_name, lock->orig_ref_name) &&
- log_ref_write(lock->orig_ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0)) {
+ if (log_ref_write(lock->ref_name, lock->old_oid.hash, sha1, logmsg, flags, err) < 0) {
char *old_msg = strbuf_detach(err, NULL);
strbuf_addf(err, "Cannot update the ref '%s': %s",
lock->ref_name, old_msg);
@@ -2686,7 +2690,7 @@ static int commit_ref_update(struct ref_lock *lock,
unlock_ref(lock);
return -1;
}
- if (strcmp(lock->orig_ref_name, "HEAD") != 0) {
+ if (flags & REF_IS_NOT_HEAD) {
/*
* Special hack: If a branch is updated directly and HEAD
* points to it (may happen on the remote side of a push
@@ -2781,8 +2785,8 @@ static int files_create_symref(const char *refname,
struct ref_lock *lock;
int ret;
- lock = lock_ref_sha1_basic(refname, NULL, NULL, NULL, REF_NODEREF, NULL,
- &err);
+ lock = lock_ref_sha1_basic(refname, NULL, NULL, NULL, NULL, REF_NODEREF,
+ NULL, &err);
if (!lock) {
error("%s", err.buf);
strbuf_release(&err);
@@ -3050,6 +3054,7 @@ static int files_transaction_commit(struct ref_transaction *transaction,
update->refname,
((update->flags & REF_HAVE_OLD) ?
update->old_sha1 : NULL),
+ update->read_sha1,
affected_refnames, NULL,
update->flags,
&update->type,
@@ -3296,7 +3301,7 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,
struct ref_lock *lock;
char *log_file;
int status = 0;
- int type;
+ int type = 0;
struct strbuf err = STRBUF_INIT;
memset(&cb, 0, sizeof(cb));
@@ -3309,7 +3314,7 @@ static int files_reflog_expire(const char *refname, const unsigned char *sha1,
* reference itself, plus we might need to update the
* reference if --updateref was specified:
*/
- lock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, REF_NODEREF,
+ lock = lock_ref_sha1_basic(refname, sha1, NULL, NULL, NULL, REF_NODEREF,
&type, &err);
if (!lock) {
error("cannot lock ref '%s': %s", refname, err.buf);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index dd76246..0d3f9e7 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -49,6 +49,8 @@
*/
#define REF_LOG_ONLY 0x80
+#define REF_IS_NOT_HEAD 0x100
+
/*
* Return true iff refname is minimally safe. "Safe" here means that
* deleting a loose reference by this name will not do any damage, for
@@ -147,6 +149,12 @@ struct ref_update {
*/
unsigned char old_sha1[20];
/*
+ * During the symbolic ref split stage, we resolve refs.
+ * We'll re-resolve non-symbolic refs once they are locked,
+ * but we store this to avoid re-resolving symbolic refs.
+ */
+ unsigned char read_sha1[20];
+ /*
* One or more of REF_HAVE_NEW, REF_HAVE_OLD, REF_NODEREF,
* REF_DELETING, and REF_ISPRUNING:
*/
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 22/32] refs: always handle non-normal refs in files backend
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (20 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 21/32] refs: resolve symbolic refs first David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 23/32] init: allow alternate ref strorage to be set for new repos David Turner
` (11 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Always handle non-normal (per-worktree or pseudo) refs in the files
backend instead of alternate backends.
Sometimes a ref transaction will update both a per-worktree ref and a
normal ref. For instance, an ordinary commit might update
refs/heads/master and HEAD (or at least HEAD's reflog).
Updates to normal refs continue to go through the chosen backend.
Updates to non-normal refs are moved to a separate files backend
transaction.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 90 insertions(+), 20 deletions(-)
diff --git a/refs.c b/refs.c
index bdde808..b2acf0e 100644
--- a/refs.c
+++ b/refs.c
@@ -9,6 +9,11 @@
#include "object.h"
#include "tag.h"
+static const char split_transaction_fail_warning[] = N_(
+ "A ref transaction was split across two refs backends. Part of the "
+ "transaction succeeded, but then the update to the per-worktree refs "
+ "failed. Your repository may be in an inconsistent state.");
+
/*
* We always have a files backend and it is the default.
*/
@@ -791,6 +796,13 @@ void ref_transaction_free(struct ref_transaction *transaction)
free(transaction);
}
+static void add_update_obj(struct ref_transaction *transaction,
+ struct ref_update *update)
+{
+ ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
+ transaction->updates[transaction->nr++] = update;
+}
+
static struct ref_update *add_update(struct ref_transaction *transaction,
const char *refname)
{
@@ -798,8 +810,7 @@ static struct ref_update *add_update(struct ref_transaction *transaction,
struct ref_update *update = xcalloc(1, sizeof(*update) + len);
memcpy((char *)update->refname, refname, len); /* includes NUL */
- ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
- transaction->updates[transaction->nr++] = update;
+ add_update_obj(transaction, update);
return update;
}
@@ -1416,11 +1427,37 @@ static int dereference_symrefs(struct ref_transaction *transaction,
return 0;
}
-int ref_transaction_commit(struct ref_transaction *transaction,
- struct strbuf *err)
+/*
+ * Move all non-normal ref updates into a specially-created
+ * files-backend transaction
+ */
+static int move_abnormal_ref_updates(struct ref_transaction *transaction,
+ struct ref_transaction *files_transaction,
+ struct strbuf *err)
+{
+ int i, normal_nr = 0;
+
+ for (i = 0; i < transaction->nr; i++) {
+ struct ref_update *update = transaction->updates[i];
+
+ if (ref_type(update->refname) == REF_TYPE_NORMAL)
+ transaction->updates[normal_nr++] = update;
+ else
+ add_update_obj(files_transaction, update);
+ }
+
+ transaction->nr = normal_nr;
+ return 0;
+}
+
+static int do_ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err,
+ ref_transaction_commit_fn *commit_fn)
{
int ret = -1;
struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
+ struct string_list files_affected_refnames = STRING_LIST_INIT_NODUP;
+ struct ref_transaction *files_transaction = NULL;
assert(err);
@@ -1436,18 +1473,60 @@ int ref_transaction_commit(struct ref_transaction *transaction,
if (ret)
goto done;
+ if (the_refs_backend != &refs_be_files) {
+ files_transaction = ref_transaction_begin(err);
+ if (!files_transaction)
+ goto done;
+
+ ret = move_abnormal_ref_updates(transaction, files_transaction,
+ err);
+ if (ret)
+ goto done;
+
+ /* files backend commit */
+ if (get_affected_refnames(files_transaction,
+ &files_affected_refnames,
+ err)) {
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto done;
+ }
+ }
+
+ /* main backend commit */
if (get_affected_refnames(transaction, &affected_refnames, err)) {
ret = TRANSACTION_GENERIC_ERROR;
goto done;
}
- ret = the_refs_backend->transaction_commit(transaction,
- &affected_refnames, err);
+ ret = commit_fn(transaction, &affected_refnames, err);
+ if (ret)
+ goto done;
+
+ if (files_transaction) {
+ ret = refs_be_files.transaction_commit(files_transaction,
+ &files_affected_refnames,
+ err);
+ if (ret) {
+ warning(split_transaction_fail_warning);
+ goto done;
+ }
+ }
+
done:
string_list_clear(&affected_refnames, 0);
+ string_list_clear(&files_affected_refnames, 0);
+ if (files_transaction)
+ ref_transaction_free(files_transaction);
return ret;
}
+int ref_transaction_commit(struct ref_transaction *transaction,
+ struct strbuf *err)
+{
+ return do_ref_transaction_commit(transaction, err,
+ the_refs_backend->transaction_commit);
+}
+
int verify_refname_available(const char *refname, struct string_list *extra,
struct string_list *skip, struct strbuf *err)
{
@@ -1467,6 +1546,9 @@ int peel_ref(const char *refname, unsigned char *sha1)
int create_symref(const char *ref_target, const char *refs_heads_master,
const char *logmsg)
{
+ if (ref_type(ref_target) != REF_TYPE_NORMAL)
+ return refs_be_files.create_symref(ref_target, refs_heads_master,
+ logmsg);
return the_refs_backend->create_symref(ref_target, refs_heads_master,
logmsg);
}
@@ -1526,20 +1608,8 @@ int reflog_expire(const char *refname, const unsigned char *sha1,
int initial_ref_transaction_commit(struct ref_transaction *transaction,
struct strbuf *err)
{
- struct string_list affected_refnames = STRING_LIST_INIT_NODUP;
- int ret;
-
- if (get_affected_refnames(transaction,
- &affected_refnames, err)) {
- ret = TRANSACTION_GENERIC_ERROR;
- goto done;
- }
- ret = the_refs_backend->initial_transaction_commit(transaction,
- &affected_refnames,
- err);
-done:
- string_list_clear(&affected_refnames, 0);
- return ret;
+ return do_ref_transaction_commit(transaction, err,
+ the_refs_backend->initial_transaction_commit);
}
int delete_refs(struct string_list *refnames)
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 23/32] init: allow alternate ref strorage to be set for new repos
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (21 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 22/32] refs: always handle non-normal refs in files backend David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 24/32] refs: check submodules' ref storage config David Turner
` (10 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner, SZEDER Gábor
git init learns a new argument --ref-storage. Presently, only
"files" is supported, but later we will add other storage backends.
When this argument is used, the repository's extensions.refStorage
configuration value is set (as well as core.repositoryformatversion),
and the ref storage backend's initdb function is used to set up the
ref database.
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
---
Documentation/git-init-db.txt | 2 +-
Documentation/git-init.txt | 7 +++++-
builtin/init-db.c | 44 +++++++++++++++++++++++++++-------
cache.h | 2 ++
contrib/completion/git-completion.bash | 3 ++-
path.c | 29 ++++++++++++++++++++--
refs.c | 2 ++
refs.h | 5 ++++
setup.c | 12 ++++++++++
t/t0001-init.sh | 25 +++++++++++++++++++
10 files changed, 118 insertions(+), 13 deletions(-)
diff --git a/Documentation/git-init-db.txt b/Documentation/git-init-db.txt
index 648a6cd..d03fb69 100644
--- a/Documentation/git-init-db.txt
+++ b/Documentation/git-init-db.txt
@@ -9,7 +9,7 @@ git-init-db - Creates an empty Git repository
SYNOPSIS
--------
[verse]
-'git init-db' [-q | --quiet] [--bare] [--template=<template_directory>] [--separate-git-dir <git dir>] [--shared[=<permissions>]]
+'git init-db' [-q | --quiet] [--bare] [--template=<template_directory>] [--separate-git-dir <git dir>] [--shared[=<permissions>]] [--ref-storage=<name>]
DESCRIPTION
diff --git a/Documentation/git-init.txt b/Documentation/git-init.txt
index 8174d27..93f8d0c 100644
--- a/Documentation/git-init.txt
+++ b/Documentation/git-init.txt
@@ -12,7 +12,7 @@ SYNOPSIS
'git init' [-q | --quiet] [--bare] [--template=<template_directory>]
[--separate-git-dir <git dir>]
[--shared[=<permissions>]] [directory]
-
+ [--ref-storage=<name>]
DESCRIPTION
-----------
@@ -113,6 +113,11 @@ does not exist, it will be created.
--
+--ref-storage=<name>::
+Type of refs storage backend. Default is to use the original "files"
+storage, which stores ref data in files in `$GIT_DIR/refs` and
+`$GIT_DIR/packed-refs`.
+
TEMPLATE DIRECTORY
------------------
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 0c8f4ac..753cb1c 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -23,6 +23,7 @@ static int init_is_bare_repository = 0;
static int init_shared_repository = -1;
static const char *init_db_template_dir;
static const char *git_link;
+static char *requested_ref_storage_backend;
static void copy_templates_1(struct strbuf *path, struct strbuf *template,
DIR *dir)
@@ -178,6 +179,7 @@ static int create_default_files(const char *template_path)
int reinit;
int filemode;
struct strbuf err = STRBUF_INIT;
+ int repo_version = 0;
/* Just look for `init.templatedir` */
git_config(git_init_db_config, NULL);
@@ -204,6 +206,32 @@ static int create_default_files(const char *template_path)
}
/*
+ * Create the default symref from ".git/HEAD" to the "master"
+ * branch, if it does not exist yet.
+ */
+ path = git_path_buf(&buf, "HEAD");
+ reinit = (!access(path, R_OK)
+ || readlink(path, junk, sizeof(junk)-1) != -1);
+ if (reinit) {
+ if (requested_ref_storage_backend &&
+ strcmp(ref_storage_backend, requested_ref_storage_backend))
+ die(_("You can't change the refs storage type (was %s; you requested %s)"),
+ ref_storage_backend,
+ requested_ref_storage_backend);
+ }
+
+ if (requested_ref_storage_backend)
+ ref_storage_backend = requested_ref_storage_backend;
+ if (strcmp(ref_storage_backend, "files")) {
+ git_config_set("extensions.refStorage", ref_storage_backend);
+ git_config_set("core.repositoryformatversion", ref_storage_backend);
+ if (set_ref_storage_backend(ref_storage_backend))
+ die(_("Unknown ref storage backend %s"),
+ ref_storage_backend);
+ repo_version = 1;
+ }
+
+ /*
* We need to create a "refs" dir in any case so that older
* versions of git can tell that this is a repository.
*/
@@ -212,13 +240,6 @@ static int create_default_files(const char *template_path)
if (refs_init_db(shared_repository, &err))
die("failed to set up refs db: %s", err.buf);
- /*
- * Create the default symlink from ".git/HEAD" to the "master"
- * branch, if it does not exist yet.
- */
- path = git_path_buf(&buf, "HEAD");
- reinit = (!access(path, R_OK)
- || readlink(path, junk, sizeof(junk)-1) != -1);
if (!reinit) {
if (create_symref("HEAD", "refs/heads/master", NULL) < 0)
exit(1);
@@ -226,7 +247,7 @@ static int create_default_files(const char *template_path)
/* This forces creation of new config file */
xsnprintf(repo_version_string, sizeof(repo_version_string),
- "%d", GIT_REPO_VERSION);
+ "%d", repo_version);
git_config_set("core.repositoryformatversion", repo_version_string);
/* Check filemode trustability */
@@ -474,11 +495,18 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)
OPT_BIT('q', "quiet", &flags, N_("be quiet"), INIT_DB_QUIET),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
N_("separate git dir from working tree")),
+ OPT_STRING(0, "ref-storage", &requested_ref_storage_backend,
+ N_("name"), N_("name of backend type to use")),
OPT_END()
};
argc = parse_options(argc, argv, prefix, init_db_options, init_db_usage, 0);
+ if (requested_ref_storage_backend &&
+ !ref_storage_backend_exists(requested_ref_storage_backend))
+ die(_("Unknown ref storage backend %s"),
+ requested_ref_storage_backend);
+
if (real_git_dir && !is_absolute_path(real_git_dir))
real_git_dir = xstrdup(real_path(real_git_dir));
diff --git a/cache.h b/cache.h
index 4307174..a54c97b 100644
--- a/cache.h
+++ b/cache.h
@@ -734,6 +734,8 @@ extern enum object_creation_mode object_creation_mode;
extern char *notes_ref_name;
+extern const char *ref_storage_backend;
+
extern int grafts_replace_parents;
/*
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 45ec47f..0138d03 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1362,7 +1362,8 @@ _git_init ()
return
;;
--*)
- __gitcomp "--quiet --bare --template= --shared --shared="
+ __gitcomp "--quiet --bare --template= --shared --shared=
+ --ref-storage="
return
;;
esac
diff --git a/path.c b/path.c
index 8b7e168..2e67a2b 100644
--- a/path.c
+++ b/path.c
@@ -2,6 +2,7 @@
* Utilities for paths and pathnames
*/
#include "cache.h"
+#include "refs.h"
#include "strbuf.h"
#include "string-list.h"
#include "dir.h"
@@ -511,8 +512,32 @@ int validate_headref(const char *path)
int fd;
ssize_t len;
- if (lstat(path, &st) < 0)
- return -1;
+ if (lstat(path, &st) < 0) {
+ int backend_type_set;
+ struct strbuf config_path = STRBUF_INIT;
+ char *pathdup = xstrdup(path);
+ char *git_dir = dirname(pathdup);
+ char *storage = NULL;
+
+ strbuf_addf(&config_path, "%s/%s", git_dir, "config");
+ free(pathdup);
+
+ if (git_config_from_file(ref_storage_backend_config,
+ config_path.buf, &storage)) {
+ strbuf_release(&config_path);
+ return -1;
+ }
+
+ backend_type_set = !!storage;
+ free((void *)storage);
+ strbuf_release(&config_path);
+ /*
+ * Alternate backends are assumed to keep HEAD
+ * in a valid state, so there's no need to do
+ * further validation.
+ */
+ return backend_type_set ? 0 : -1;
+ }
/* Make sure it is a "refs/.." symlink */
if (S_ISLNK(st.st_mode)) {
diff --git a/refs.c b/refs.c
index b2acf0e..04d8f0c 100644
--- a/refs.c
+++ b/refs.c
@@ -23,6 +23,8 @@ static struct ref_storage_be *the_refs_backend = &refs_be_files;
*/
static struct ref_storage_be *refs_backends = &refs_be_files;
+const char *ref_storage_backend = "files";
+
int set_ref_storage_backend(const char *name)
{
struct ref_storage_be *be;
diff --git a/refs.h b/refs.h
index feff82e..f08e886 100644
--- a/refs.h
+++ b/refs.h
@@ -511,6 +511,11 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,
void *policy_cb_data);
/*
+ * Read the refdb storage backend name out of the config file
+ */
+int ref_storage_backend_config(const char *var, const char *value, void *ptr);
+
+/*
* Switch to an alternate ref storage backend.
*/
int set_ref_storage_backend(const char *name);
diff --git a/setup.c b/setup.c
index 0deb022..1a62277 100644
--- a/setup.c
+++ b/setup.c
@@ -1,5 +1,6 @@
#include "cache.h"
#include "dir.h"
+#include "refs.h"
#include "string-list.h"
static int inside_git_dir = -1;
@@ -263,6 +264,15 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)
return ret;
}
+int ref_storage_backend_config(const char *var, const char *value, void *ptr)
+{
+ char **cdata = ptr;
+
+ if (!strcmp(var, "extensions.refstorage"))
+ *cdata = xstrdup(value);
+ return 0;
+}
+
/*
* Test if it looks like we're at a git directory.
* We want to see:
@@ -390,6 +400,8 @@ static int check_repo_format(const char *var, const char *value, void *cb)
;
else if (!strcmp(ext, "preciousobjects"))
repository_format_precious_objects = git_config_bool(var, value);
+ else if (!strcmp(ext, "refstorage"))
+ ref_storage_backend = xstrdup(value);
else
string_list_append(&unknown_extensions, ext);
}
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 295aa59..31c8279 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -174,6 +174,31 @@ test_expect_success 'reinit' '
test_i18ncmp again/empty again/err2
'
+test_expect_success 'init with bogus storage backend fails' '
+
+ (
+ mkdir again2 &&
+ cd again2 &&
+ test_must_fail git init --ref-storage=test >out2 2>err2 &&
+ test_i18ngrep "Unknown ref storage backend test" err2
+ )
+'
+
+test_expect_failure 'reinit with changed storage backend fails' '
+
+ (
+ mkdir again3 &&
+ cd again3 &&
+ git init >out1 2>err1 &&
+ git init --ref-storage=test >out2 2>err2
+ ) &&
+ test_i18ngrep "Initialized empty" again3/out1 &&
+ test_i18ngrep "Reinitialized existing" again3/out2 &&
+ >again3/empty &&
+ test_i18ncmp again3/empty again3/err1 &&
+ test_i18ncmp again3/empty again3/err2
+'
+
test_expect_success 'init with --template' '
mkdir template-source &&
echo content >template-source/file &&
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 24/32] refs: check submodules' ref storage config
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (22 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 23/32] init: allow alternate ref strorage to be set for new repos David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 25/32] clone: allow ref storage backend to be set for clone David Turner
` (9 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
All submodules must have the same ref storage (for now). Confirm that
this is so before attempting to do anything with submodule refs.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++
refs.h | 2 +-
refs/files-backend.c | 8 ++------
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/refs.c b/refs.c
index 04d8f0c..27c82ec 100644
--- a/refs.c
+++ b/refs.c
@@ -302,6 +302,54 @@ int for_each_tag_ref(each_ref_fn fn, void *cb_data)
return for_each_ref_in("refs/tags/", fn, cb_data);
}
+static int submodule_backend(const char *key, const char *value, void *data)
+{
+ const char **path = data;
+ char **old_path = data;
+ if (!strcmp(key, "extensions.refstorage") &&
+ !git_config_string(path, key, "extensions.refstorage"))
+ free(*old_path);
+
+ return 0;
+}
+
+/*
+ * Check that a submodule exists. If its ref storage backend differs
+ * from the current backend, die. If the submodule exists, return
+ * 0. Else return -1.
+ */
+static int check_submodule_backend(const char *submodule)
+{
+ struct strbuf sb = STRBUF_INIT;
+ char *submodule_storage_backend;
+ int result = -1;
+
+ if (!submodule)
+ return 0;
+
+ submodule_storage_backend = xstrdup("files");
+
+ strbuf_addstr(&sb, submodule);
+ if (!is_nonbare_repository_dir(&sb))
+ goto done;
+
+ strbuf_reset(&sb);
+ strbuf_git_path_submodule(&sb, submodule, "config");
+
+ git_config_from_file(submodule_backend, sb.buf,
+ &submodule_storage_backend);
+ if (strcmp(submodule_storage_backend, ref_storage_backend))
+ die(_("Ref storage '%s' for submodule '%s' does not match our storage, '%s'"),
+ submodule_storage_backend, submodule, ref_storage_backend);
+
+ result = 0;
+done:
+ free(submodule_storage_backend);
+ strbuf_release(&sb);
+
+ return result;
+}
+
int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
{
return for_each_ref_in_submodule(submodule, "refs/tags/", fn, cb_data);
@@ -314,6 +362,7 @@ int for_each_branch_ref(each_ref_fn fn, void *cb_data)
int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
{
+ check_submodule_backend(submodule);
return for_each_ref_in_submodule(submodule, "refs/heads/", fn, cb_data);
}
@@ -324,6 +373,7 @@ int for_each_remote_ref(each_ref_fn fn, void *cb_data)
int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
{
+ check_submodule_backend(submodule);
return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, cb_data);
}
@@ -1135,6 +1185,7 @@ int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
int flag;
if (submodule) {
+ check_submodule_backend(submodule);
if (resolve_gitlink_ref(submodule, "HEAD", oid.hash) == 0)
return fn("HEAD", &oid, 0, cb_data);
@@ -1558,6 +1609,9 @@ int create_symref(const char *ref_target, const char *refs_heads_master,
int resolve_gitlink_ref(const char *path, const char *refname,
unsigned char *sha1)
{
+ if (check_submodule_backend(path))
+ return -1;
+
return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
}
diff --git a/refs.h b/refs.h
index f08e886..456e729 100644
--- a/refs.h
+++ b/refs.h
@@ -511,7 +511,7 @@ extern int reflog_expire(const char *refname, const unsigned char *sha1,
void *policy_cb_data);
/*
- * Read the refdb storage backend name out of the config file
+ * Read the ref storage backend name out of the config file
*/
int ref_storage_backend_config(const char *var, const char *value, void *ptr);
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f378f06..6f13b70 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1355,13 +1355,9 @@ static int files_resolve_gitlink_ref(const char *path, const char *refname,
strbuf_add(&submodule, path, len);
refs = lookup_ref_cache(submodule.buf);
- if (!refs) {
- if (!is_nonbare_repository_dir(&submodule)) {
- strbuf_release(&submodule);
- return -1;
- }
+ if (!refs)
refs = create_ref_cache(submodule.buf);
- }
+
strbuf_release(&submodule);
retval = resolve_gitlink_ref_recursive(refs, refname, sha1, 0);
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 25/32] clone: allow ref storage backend to be set for clone
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (23 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 24/32] refs: check submodules' ref storage config David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 26/32] svn: learn ref-storage argument David Turner
` (8 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner, SZEDER Gábor
Add a new option, --ref-storage, to allow the ref storage backend to
be set on new clones.
Submodules must use the same ref storage as the parent repository, so
we also pass the --ref-storage option option when cloning
submodules.
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
---
Documentation/git-clone.txt | 5 +++++
builtin/clone.c | 5 +++++
builtin/submodule--helper.c | 2 +-
contrib/completion/git-completion.bash | 1 +
git-submodule.sh | 13 +++++++++++++
5 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 45d74be..68f56a7 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,6 +14,7 @@ SYNOPSIS
[-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
[--dissociate] [--separate-git-dir <git dir>]
[--depth <depth>] [--[no-]single-branch]
+ [--ref-storage=<name>]
[--recursive | --recurse-submodules] [--jobs <n>] [--] <repository>
[<directory>]
@@ -224,6 +225,10 @@ objects from the source repository into a pack in the cloned repository.
The number of submodules fetched at the same time.
Defaults to the `submodule.fetchJobs` option.
+--ref-storage=<name>::
+ Type of ref storage backend. Default is to use the original files
+ based ref storage.
+
<repository>::
The (possibly remote) repository to clone from. See the
<<URLS,URLS>> section below for more information on specifying
diff --git a/builtin/clone.c b/builtin/clone.c
index 191f522..5ba0514 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -95,6 +95,8 @@ static struct option builtin_clone_options[] = {
N_("separate git dir from working tree")),
OPT_STRING_LIST('c', "config", &option_config, N_("key=value"),
N_("set config inside the new repository")),
+ OPT_STRING(0, "ref-storage", &ref_storage_backend,
+ N_("name"), N_("name of backend type to use")),
OPT_END()
};
@@ -733,6 +735,9 @@ static int checkout(void)
if (max_jobs != -1)
argv_array_pushf(&args, "--jobs=%d", max_jobs);
+ argv_array_pushl(&args, "--ref-storage",
+ ref_storage_backend, NULL);
+
err = run_command_v_opt(args.argv, RUN_GIT_CMD);
argv_array_clear(&args);
}
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 689c354..879d6fb 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -457,7 +457,7 @@ static int clone_submodule(const char *path, const char *gitdir, const char *url
argv_array_pushl(&cp.args, "--reference", reference, NULL);
if (gitdir && *gitdir)
argv_array_pushl(&cp.args, "--separate-git-dir", gitdir, NULL);
-
+ argv_array_pushl(&cp.args, "--ref-storage", ref_storage_backend, NULL);
argv_array_push(&cp.args, url);
argv_array_push(&cp.args, path);
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 0138d03..cb9c473 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1092,6 +1092,7 @@ _git_clone ()
--depth
--single-branch
--branch
+ --ref-storage=
"
return
;;
diff --git a/git-submodule.sh b/git-submodule.sh
index 6fce0dc..e7ac98b 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -527,6 +527,14 @@ cmd_update()
--checkout)
update="checkout"
;;
+ --ref-storage=*)
+ ref_storage="$1"
+ ;;
+ --ref-storage)
+ case "$2" in '') usage ;; esac
+ ref_storage="$2"
+ shift
+ ;;
--depth)
case "$2" in '') usage ;; esac
depth="--depth=$2"
@@ -560,6 +568,11 @@ cmd_update()
if test -n "$init"
then
cmd_init "--" "$@" || return
+ else
+ if test -n "$ref_storage"
+ then
+ usage
+ fi
fi
git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 26/32] svn: learn ref-storage argument
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (24 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 25/32] clone: allow ref storage backend to be set for clone David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:58 ` [PATCH v6 27/32] refs: register ref storage backends David Turner
` (7 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner, SZEDER Gábor, Eric Wong
git svn learns to pass the ref-storage command-line argument (to init
and clone) through to git init.
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
contrib/completion/git-completion.bash | 2 +-
git-svn.perl | 6 +++++-
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index cb9c473..ba4137d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2477,7 +2477,7 @@ _git_svn ()
--branches= --stdlayout --minimize-url
--no-metadata --use-svm-props --use-svnsync-props
--rewrite-root= --prefix= --use-log-author
- --add-author-from $remote_opts
+ --add-author-from --ref-storage= $remote_opts
"
local cmt_opts="
--edit --rmdir --find-copies-harder --copy-similarity=
diff --git a/git-svn.perl b/git-svn.perl
index fa5f253..62464d1 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -141,7 +141,7 @@ my %fc_opts = ( 'follow-parent|follow!' => \$Git::SVN::_follow_parent,
'localtime' => \$Git::SVN::_localtime,
%remote_opts );
-my ($_trunk, @_tags, @_branches, $_stdlayout);
+my ($_trunk, @_tags, @_branches, $_stdlayout, $_ref_storage);
my %icv;
my %init_opts = ( 'template=s' => \$_template, 'shared:s' => \$_shared,
'trunk|T=s' => \$_trunk, 'tags|t=s@' => \@_tags,
@@ -153,6 +153,7 @@ my %init_opts = ( 'template=s' => \$_template, 'shared:s' => \$_shared,
'use-svnsync-props' => sub { $icv{useSvnsyncProps} = 1 },
'rewrite-root=s' => sub { $icv{rewriteRoot} = $_[1] },
'rewrite-uuid=s' => sub { $icv{rewriteUUID} = $_[1] },
+ 'ref-storage=s' => \$_ref_storage,
%remote_opts );
my %cmt_opts = ( 'edit|e' => \$_edit,
'rmdir' => \$Git::SVN::Editor::_rmdir,
@@ -469,6 +470,9 @@ sub do_git_init_db {
push @init_db, "--shared";
}
}
+ if (defined $_ref_storage) {
+ push @init_db, "--ref-storage=" . $_ref_storage;
+ }
command_noisy(@init_db);
$_repository = Git->repository(Repository => ".git");
}
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 27/32] refs: register ref storage backends
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (25 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 26/32] svn: learn ref-storage argument David Turner
@ 2016-02-24 22:58 ` David Turner
2016-02-24 22:59 ` [PATCH v6 28/32] config: read ref storage config on startup David Turner
` (6 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:58 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Add new function register_ref_storage_backends(). This new function
registers all known ref storage backends... once there are any other
than the default. For now, it just registers the files backend.
Call the function before calling set_ref_storage_backend.
Signed-off-by: David Turner <dturner@twopensource.com>
---
builtin/init-db.c | 3 +++
refs.c | 22 ++++++++++++++++++----
refs.h | 2 ++
3 files changed, 23 insertions(+), 4 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 753cb1c..ddb4c6c 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -220,6 +220,7 @@ static int create_default_files(const char *template_path)
requested_ref_storage_backend);
}
+ register_ref_storage_backends();
if (requested_ref_storage_backend)
ref_storage_backend = requested_ref_storage_backend;
if (strcmp(ref_storage_backend, "files")) {
@@ -502,6 +503,8 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix, init_db_options, init_db_usage, 0);
+ register_ref_storage_backends();
+
if (requested_ref_storage_backend &&
!ref_storage_backend_exists(requested_ref_storage_backend))
die(_("Unknown ref storage backend %s"),
diff --git a/refs.c b/refs.c
index 27c82ec..90de086 100644
--- a/refs.c
+++ b/refs.c
@@ -14,14 +14,11 @@ static const char split_transaction_fail_warning[] = N_(
"transaction succeeded, but then the update to the per-worktree refs "
"failed. Your repository may be in an inconsistent state.");
-/*
- * We always have a files backend and it is the default.
- */
static struct ref_storage_be *the_refs_backend = &refs_be_files;
/*
* List of all available backends
*/
-static struct ref_storage_be *refs_backends = &refs_be_files;
+static struct ref_storage_be *refs_backends = NULL;
const char *ref_storage_backend = "files";
@@ -49,6 +46,23 @@ int ref_storage_backend_exists(const char *name)
return 0;
}
+void register_ref_storage_backend(struct ref_storage_be *be)
+{
+ be->next = refs_backends;
+ refs_backends = be;
+}
+
+void register_ref_storage_backends(void)
+{
+ if (refs_backends)
+ return;
+ /*
+ * Add register_ref_storage_backend(ptr-to-backend)
+ * entries below when you add a new backend.
+ */
+ register_ref_storage_backend(&refs_be_files);
+}
+
/*
* How to handle various characters in refnames:
* 0: An acceptable character for refs
diff --git a/refs.h b/refs.h
index 456e729..9cc3a37 100644
--- a/refs.h
+++ b/refs.h
@@ -522,4 +522,6 @@ int set_ref_storage_backend(const char *name);
int ref_storage_backend_exists(const char *name);
+void register_ref_storage_backends(void);
+
#endif /* REFS_H */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 28/32] config: read ref storage config on startup
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (26 preceding siblings ...)
2016-02-24 22:58 ` [PATCH v6 27/32] refs: register ref storage backends David Turner
@ 2016-02-24 22:59 ` David Turner
2016-02-27 3:56 ` Jeff King
2016-02-24 22:59 ` [PATCH v6 29/32] refs: break out resolve_ref_unsafe_submodule David Turner
` (5 subsequent siblings)
33 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-24 22:59 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
This sets up the existing backend early, so that other code which
reads refs is reading from the right place.
Signed-off-by: David Turner <dturner@twopensource.com>
---
config.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/config.c b/config.c
index b95ac3a..6cea884 100644
--- a/config.c
+++ b/config.c
@@ -11,6 +11,7 @@
#include "strbuf.h"
#include "quote.h"
#include "hashmap.h"
+#include "refs.h"
#include "string-list.h"
#include "utf8.h"
@@ -1207,6 +1208,29 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
}
if (repo_config && !access_or_die(repo_config, R_OK, 0)) {
+ char *storage = NULL;
+
+ /*
+ * make sure we always read the ref storage config
+ * from the extensions section on startup
+ */
+ ret += git_config_from_file(ref_storage_backend_config,
+ repo_config, &storage);
+
+ register_ref_storage_backends();
+ if (!storage)
+ storage = xstrdup("");
+
+ if (!*storage ||
+ !strcmp(storage, "files")) {
+ /* default backend, nothing to do */
+ free(storage);
+ } else {
+ if (set_ref_storage_backend(ref_storage_backend))
+ die(_("Unknown ref storage backend %s"),
+ ref_storage_backend);
+ }
+
ret += git_config_from_file(fn, repo_config, data);
found += 1;
}
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-24 22:59 ` [PATCH v6 28/32] config: read ref storage config on startup David Turner
@ 2016-02-27 3:56 ` Jeff King
2016-02-27 4:14 ` Jeff King
2016-02-29 21:57 ` David Turner
0 siblings, 2 replies; 56+ messages in thread
From: Jeff King @ 2016-02-27 3:56 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
On Wed, Feb 24, 2016 at 05:59:00PM -0500, David Turner wrote:
> @@ -1207,6 +1208,29 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
> }
>
> if (repo_config && !access_or_die(repo_config, R_OK, 0)) {
> + char *storage = NULL;
> +
> + /*
> + * make sure we always read the ref storage config
> + * from the extensions section on startup
> + */
> + ret += git_config_from_file(ref_storage_backend_config,
> + repo_config, &storage);
> +
> + register_ref_storage_backends();
> + if (!storage)
> + storage = xstrdup("");
> +
> + if (!*storage ||
> + !strcmp(storage, "files")) {
> + /* default backend, nothing to do */
> + free(storage);
> + } else {
> + if (set_ref_storage_backend(ref_storage_backend))
> + die(_("Unknown ref storage backend %s"),
> + ref_storage_backend);
> + }
> +
Coverity complains that "storage" leaks here, and I think it does in the
case that we non-default storage, and we successfully set up the
backend. That's a pretty minor point.
However, after looking at this code, I'm rather confused about a few
things.
One is that we read the config and use ref_storage_backend_config[1] to
store the string value in "storage", and then we check whether that is
non-default, and if it is, I'd expect us to feed it to
set_ref_storage_backend. But we don't; we feed ref_storage_backend
instead!
What is that value? It looks like it is the string we set in
check_repo_format when we load the extensions list there. So why
are we re-reading the config here at all? Couldn't we just use
ref_storage_backend in the first place?
My second confusion is why this is happening in git_config_early(). That
function is called during the setup of check_repository_format_gently(),
which is why I think you wanted to put the code here. But it's _also_
called as part of a regular git_config(). Which means we're parsing the
repo config and setting the ref backend all over again, every time we
look at config for other reasons.
So I think this setup probably should be in
check_repository_format_gently(), and should be able to trigger off of
the existing ref_storage_backend string we've already saved (and we
should bail immediately there if we don't know about the backend, as it
means we _don't_ match the repo's extensions and cannot proceed).
-Peff
[1] The ref_storage_backend_config function uses xstrdup(), which I
think will segfault for a value-less boolean config, like:
[extensions]
# notice no "=" sign!
refstorage
The same bug is in check_repo_format, where we assigned
ref_storage_backend. The normal way to do this is to use:
return git_config_string(var, value, &ref_backend_storage);
which checks for the boolean case and complains.
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-27 3:56 ` Jeff King
@ 2016-02-27 4:14 ` Jeff King
2016-02-29 21:57 ` David Turner
2016-02-29 21:57 ` David Turner
1 sibling, 1 reply; 56+ messages in thread
From: Jeff King @ 2016-02-27 4:14 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
On Fri, Feb 26, 2016 at 10:56:34PM -0500, Jeff King wrote:
> My second confusion is why this is happening in git_config_early(). That
> function is called during the setup of check_repository_format_gently(),
> which is why I think you wanted to put the code here. But it's _also_
> called as part of a regular git_config(). Which means we're parsing the
> repo config and setting the ref backend all over again, every time we
> look at config for other reasons.
>
> So I think this setup probably should be in
> check_repository_format_gently(), and should be able to trigger off of
> the existing ref_storage_backend string we've already saved (and we
> should bail immediately there if we don't know about the backend, as it
> means we _don't_ match the repo's extensions and cannot proceed).
By the way, I notice that the default value for the_refs_backend is
"&refs_be_files". It might be safer to make this NULL (or some
&refs_be_null that fills the vtable, but just returns an error for each
call).
Then in check_repository_format_gently(), do:
if (!ref_storage_backend || !*ref_storage_backend)
refs_backend_storage = "files";
set_ref_storage_backend(ref_storage_backend);
That way you could be sure that the backend has been initialized from
the config, and never blindly falls back to the "files" backend. If
there is any code path where we try to access refs but the_refs_backend
is null, that's a bug that needs fixing. But it's a lot easier to find
such bugs if we die loudly in that case rather than silently using
"files".
-Peff
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-27 4:14 ` Jeff King
@ 2016-02-29 21:57 ` David Turner
2016-02-29 22:13 ` Jeff King
0 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-29 21:57 UTC (permalink / raw)
To: Jeff King; +Cc: git, mhagger, pclouds
On Fri, 2016-02-26 at 23:14 -0500, Jeff King wrote:
> On Fri, Feb 26, 2016 at 10:56:34PM -0500, Jeff King wrote:
>
> > My second confusion is why this is happening in git_config_early().
> > That
> > function is called during the setup of
> > check_repository_format_gently(),
> > which is why I think you wanted to put the code here. But it's
> > _also_
> > called as part of a regular git_config(). Which means we're parsing
> > the
> > repo config and setting the ref backend all over again, every time
> > we
> > look at config for other reasons.
> >
> > So I think this setup probably should be in
> > check_repository_format_gently(), and should be able to trigger off
> > of
> > the existing ref_storage_backend string we've already saved (and we
> > should bail immediately there if we don't know about the backend,
> > as it
> > means we _don't_ match the repo's extensions and cannot proceed).
>
> By the way, I notice that the default value for the_refs_backend is
> "&refs_be_files". It might be safer to make this NULL (or some
> &refs_be_null that fills the vtable, but just returns an error for
> each
> call).
In 'refs: register ref storage backends', I do make this change. I
don't do it earlier because we don't have this config stuff in place
yet.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-29 21:57 ` David Turner
@ 2016-02-29 22:13 ` Jeff King
0 siblings, 0 replies; 56+ messages in thread
From: Jeff King @ 2016-02-29 22:13 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
On Mon, Feb 29, 2016 at 04:57:49PM -0500, David Turner wrote:
> > By the way, I notice that the default value for the_refs_backend is
> > "&refs_be_files". It might be safer to make this NULL (or some
> > &refs_be_null that fills the vtable, but just returns an error for
> > each
> > call).
>
> In 'refs: register ref storage backends', I do make this change. I
> don't do it earlier because we don't have this config stuff in place
> yet.
Ah, OK. I was that we did register the files backend as a normal one,
but I missed the assignment to NULL (I do wonder if we want a
refs_be_null to smoke out problem cases with a die("BUG") rather than
segfaulting, but in theory nobody should be doing that).
-Peff
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-27 3:56 ` Jeff King
2016-02-27 4:14 ` Jeff King
@ 2016-02-29 21:57 ` David Turner
2016-02-29 22:10 ` Jeff King
1 sibling, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-29 21:57 UTC (permalink / raw)
To: Jeff King; +Cc: git, mhagger, pclouds
On Fri, 2016-02-26 at 22:56 -0500, Jeff King wrote:
> On Wed, Feb 24, 2016 at 05:59:00PM -0500, David Turner wrote:
>
> > @@ -1207,6 +1208,29 @@ int git_config_early(config_fn_t fn, void
> > *data, const char *repo_config)
> > }
> >
> > if (repo_config && !access_or_die(repo_config, R_OK, 0)) {
> > + char *storage = NULL;
> > +
> > + /*
> > + * make sure we always read the ref storage config
> > + * from the extensions section on startup
> > + */
> > + ret +=
> > git_config_from_file(ref_storage_backend_config,
> > + repo_config,
> > &storage);
> > +
> > + register_ref_storage_backends();
> > + if (!storage)
> > + storage = xstrdup("");
> > +
> > + if (!*storage ||
> > + !strcmp(storage, "files")) {
> > + /* default backend, nothing to do */
> > + free(storage);
> > + } else {
> > + if
> > (set_ref_storage_backend(ref_storage_backend))
> > + die(_("Unknown ref storage backend
> > %s"),
> > + ref_storage_backend);
> > + }
> > +
>
> Coverity complains that "storage" leaks here, and I think it does in
> the
> case that we non-default storage, and we successfully set up the
> backend. That's a pretty minor point.
>
> However, after looking at this code, I'm rather confused about a few
> things.
>
> One is that we read the config and use ref_storage_backend_config[1]
> to
> store the string value in "storage", and then we check whether that
> is
> non-default, and if it is, I'd expect us to feed it to
> set_ref_storage_backend. But we don't; we feed ref_storage_backend
> instead!
>
> What is that value? It looks like it is the string we set in
> check_repo_format when we load the extensions list there. So why
> are we re-reading the config here at all? Couldn't we just use
> ref_storage_backend in the first place?
>
> My second confusion is why this is happening in git_config_early().
> That
> function is called during the setup of
> check_repository_format_gently(),
> which is why I think you wanted to put the code here. But it's _also_
> called as part of a regular git_config(). Which means we're parsing
> the
> repo config and setting the ref backend all over again, every time we
> look at config for other reasons.
>
> So I think this setup probably should be in
> check_repository_format_gently(), and should be able to trigger off
> of
> the existing ref_storage_backend string we've already saved (and we
> should bail immediately there if we don't know about the backend, as
> it
> means we _don't_ match the repo's extensions and cannot proceed).
We apparently don't always call check_repo_format before calling
git_config_early -- or, more to the point, before doing ref operations.
So I think we need this in git_config_early.
I'll fix up the memory safety/leak issues tho.
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-29 21:57 ` David Turner
@ 2016-02-29 22:10 ` Jeff King
2016-02-29 22:21 ` Jeff King
0 siblings, 1 reply; 56+ messages in thread
From: Jeff King @ 2016-02-29 22:10 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
On Mon, Feb 29, 2016 at 04:57:42PM -0500, David Turner wrote:
> > So I think this setup probably should be in
> > check_repository_format_gently(), and should be able to trigger off
> > of
> > the existing ref_storage_backend string we've already saved (and we
> > should bail immediately there if we don't know about the backend, as
> > it
> > means we _don't_ match the repo's extensions and cannot proceed).
>
> We apparently don't always call check_repo_format before calling
> git_config_early -- or, more to the point, before doing ref operations.
> So I think we need this in git_config_early.
That seems horribly broken, though. If a program is accessing refs
without calling check_repository_format, isn't it bypassing all of our
regular version and extension checks?
I think we should be smoking out such cases (e.g., by setting
a null refs-backend, as I mentioned earlier) and fixing them, rather
than working around them by putting the backend setup in the wrong
place.
-Peff
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-29 22:10 ` Jeff King
@ 2016-02-29 22:21 ` Jeff King
2016-02-29 22:59 ` David Turner
0 siblings, 1 reply; 56+ messages in thread
From: Jeff King @ 2016-02-29 22:21 UTC (permalink / raw)
To: David Turner; +Cc: git, mhagger, pclouds
On Mon, Feb 29, 2016 at 05:10:24PM -0500, Jeff King wrote:
> > We apparently don't always call check_repo_format before calling
> > git_config_early -- or, more to the point, before doing ref operations.
> > So I think we need this in git_config_early.
>
> That seems horribly broken, though. If a program is accessing refs
> without calling check_repository_format, isn't it bypassing all of our
> regular version and extension checks?
>
> I think we should be smoking out such cases (e.g., by setting
> a null refs-backend, as I mentioned earlier) and fixing them, rather
> than working around them by putting the backend setup in the wrong
> place.
So even without your series, this isn't too bad to instrument, like the
patch below.
It does reveal a handful of failures in the test suite. I haven't dug
yet, but I strongly suspect those are all bugs that should be fixed.
diff --git a/cache.h b/cache.h
index af2aeb8..a286f99 100644
--- a/cache.h
+++ b/cache.h
@@ -639,6 +639,7 @@ extern int hold_locked_index(struct lock_file *, int);
extern void set_alternate_index_output(const char *);
/* Environment bits from configuration mechanism */
+extern int repo_initialized;
extern int trust_executable_bit;
extern int trust_ctime;
extern int check_stat;
diff --git a/environment.c b/environment.c
index 10451ee..6ee9812 100644
--- a/environment.c
+++ b/environment.c
@@ -12,6 +12,7 @@
#include "fmt-merge-msg.h"
#include "commit.h"
+int repo_initialized;
int trust_executable_bit = 1;
int trust_ctime = 1;
int check_stat = 1;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 339f5c7..a4e9df6 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -947,6 +947,9 @@ static struct ref_cache *lookup_ref_cache(const char *submodule)
{
struct ref_cache *refs;
+ if (!repo_initialized)
+ die("BUG: lookup_ref_cache called without initializing repo");
+
if (!submodule || !*submodule)
return &ref_cache;
@@ -1414,6 +1417,9 @@ static const char *resolve_ref_1(const char *refname,
int depth = MAXDEPTH;
int bad_name = 0;
+ if (!repo_initialized)
+ die("BUG: resolve_ref called without initializing repo");
+
if (flags)
*flags = 0;
diff --git a/setup.c b/setup.c
index 76609fa..ed86094 100644
--- a/setup.c
+++ b/setup.c
@@ -446,6 +446,9 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)
ret = -1;
}
+ if (ret == 0)
+ repo_initialized = 1;
+
strbuf_release(&sb);
return ret;
}
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [PATCH v6 28/32] config: read ref storage config on startup
2016-02-29 22:21 ` Jeff King
@ 2016-02-29 22:59 ` David Turner
0 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-29 22:59 UTC (permalink / raw)
To: Jeff King; +Cc: git, mhagger, pclouds
On Mon, 2016-02-29 at 17:21 -0500, Jeff King wrote:
> On Mon, Feb 29, 2016 at 05:10:24PM -0500, Jeff King wrote:
>
> > > We apparently don't always call check_repo_format before calling
> > > git_config_early -- or, more to the point, before doing ref
> > > operations.
> > > So I think we need this in git_config_early.
> >
> > That seems horribly broken, though. If a program is accessing refs
> > without calling check_repository_format, isn't it bypassing all of
> > our
> > regular version and extension checks?
> >
> > I think we should be smoking out such cases (e.g., by setting
> > a null refs-backend, as I mentioned earlier) and fixing them,
> > rather
> > than working around them by putting the backend setup in the wrong
> > place.
>
> So even without your series, this isn't too bad to instrument, like
> the
> patch below.
>
> It does reveal a handful of failures in the test suite. I haven't dug
> yet, but I strongly suspect those are all bugs that should be fixed.
I'll try to add a patch for these cases.
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PATCH v6 29/32] refs: break out resolve_ref_unsafe_submodule
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (27 preceding siblings ...)
2016-02-24 22:59 ` [PATCH v6 28/32] config: read ref storage config on startup David Turner
@ 2016-02-24 22:59 ` David Turner
2016-02-24 22:59 ` [PATCH v6 30/32] refs: add LMDB refs storage backend David Turner
` (4 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:59 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
It will soon be useful for resolve_ref_unsafe to support submodules.
But since it is called from so many places, changing it would have
been painful. Fortunately, it's just a thin wrapper around (the
former) resolve_ref_1. So now resolve_ref_1 becomes
resolve_ref_unsafe_submodule, and it passes its submodule argument
through to read_raw_ref.
The files backend doesn't need this functionality, but it doesn't
hurt.
Signed-off-by: David Turner <dturner@twopensource.com>
---
refs.c | 41 +++++++++++++++++++++++++----------------
refs/files-backend.c | 8 ++++++--
refs/refs-internal.h | 19 ++++++++++++++++---
3 files changed, 47 insertions(+), 21 deletions(-)
diff --git a/refs.c b/refs.c
index 90de086..9faeae9 100644
--- a/refs.c
+++ b/refs.c
@@ -61,6 +61,9 @@ void register_ref_storage_backends(void)
* entries below when you add a new backend.
*/
register_ref_storage_backend(&refs_be_files);
+#ifdef USE_LIBLMDB
+ register_ref_storage_backend(&refs_be_lmdb);
+#endif
}
/*
@@ -321,8 +324,10 @@ static int submodule_backend(const char *key, const char *value, void *data)
const char **path = data;
char **old_path = data;
if (!strcmp(key, "extensions.refstorage") &&
- !git_config_string(path, key, "extensions.refstorage"))
- free(*old_path);
+ !git_config_string(path, key, "extensions.refstorage")) {
+ free(*old_path);
+ *path = xstrdup(value);
+ }
return 0;
}
@@ -1310,21 +1315,22 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
}
-static int read_raw_ref(const char *refname, unsigned char *sha1,
- struct strbuf *symref, struct strbuf *sb_path,
- unsigned int *flags)
+static int read_raw_ref(const char *submodule, const char *refname,
+ unsigned char *sha1, struct strbuf *symref,
+ struct strbuf *sb_path, unsigned int *flags)
{
- return the_refs_backend->read_raw_ref(refname, sha1, symref, sb_path,
- flags);
+ return the_refs_backend->read_raw_ref(submodule, refname, sha1,
+ symref, sb_path, flags);
}
/* This function needs to return a meaningful errno on failure */
-static const char *resolve_ref_1(const char *refname,
- int resolve_flags,
- unsigned char *sha1,
- int *flags,
- struct strbuf *sb_refname,
- struct strbuf *sb_path)
+const char *resolve_ref_unsafe_submodule(const char *submodule,
+ const char *refname,
+ int resolve_flags,
+ unsigned char *sha1,
+ int *flags,
+ struct strbuf *sb_refname,
+ struct strbuf *sb_path)
{
int bad_name = 0;
int symref_count;
@@ -1355,7 +1361,8 @@ static const char *resolve_ref_1(const char *refname,
for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
int read_flags = 0;
- if (read_raw_ref(refname, sha1, sb_refname, sb_path, &read_flags)) {
+ if (read_raw_ref(submodule, refname, sha1, sb_refname, sb_path,
+ &read_flags)) {
int saved_errno = errno;
if (flags)
*flags |= read_flags;
@@ -1416,8 +1423,8 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
struct strbuf sb_path = STRBUF_INIT;
const char *ret;
- ret = resolve_ref_1(refname, resolve_flags, sha1, flags,
- &sb_refname, &sb_path);
+ ret = resolve_ref_unsafe_submodule(NULL, refname, resolve_flags, sha1,
+ flags, &sb_refname, &sb_path);
strbuf_release(&sb_path);
return ret;
@@ -1660,6 +1667,8 @@ int safe_create_reflog(const char *refname, int force_create,
int delete_reflog(const char *refname)
{
+ if (ref_type(refname) != REF_TYPE_NORMAL)
+ return refs_be_files.delete_reflog(refname);
return the_refs_backend->delete_reflog(refname);
}
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6f13b70..c95cb91 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1421,7 +1421,8 @@ static int resolve_missing_loose_ref(const char *refname,
*
* sb_path is workspace: the caller should allocate and free it.
*/
-static int files_read_raw_ref(const char *refname, unsigned char *sha1,
+static int files_read_raw_ref(const char *submodule,
+ const char *refname, unsigned char *sha1,
struct strbuf *symref, struct strbuf *sb_path,
unsigned int *flags)
{
@@ -1431,7 +1432,10 @@ static int files_read_raw_ref(const char *refname, unsigned char *sha1,
const char *buf;
strbuf_reset(sb_path);
- strbuf_git_path(sb_path, "%s", refname);
+ if (submodule)
+ strbuf_git_path_submodule(sb_path, submodule, "%s", refname);
+ else
+ strbuf_git_path(sb_path, "%s", refname);
path = sb_path->buf;
for(;;) {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 0d3f9e7..9736959 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -230,6 +230,15 @@ int do_for_each_per_worktree_ref(const char *submodule, const char *base,
each_ref_fn fn, int trim, int flags,
void *cb_data);
+
+const char *resolve_ref_unsafe_submodule(const char *submodule,
+ const char *refname,
+ int resolve_flags,
+ unsigned char *sha1,
+ int *flags,
+ struct strbuf *sb_refname,
+ struct strbuf *sb_path);
+
/* refs backends */
typedef int ref_init_db_fn(int shared, struct strbuf *err);
typedef int ref_transaction_commit_fn(struct ref_transaction *transaction,
@@ -266,9 +275,9 @@ typedef int rename_ref_fn(const char *oldref, const char *newref,
const char *logmsg);
/* resolution methods */
-typedef int read_raw_ref_fn(const char *refname, unsigned char *sha1,
- struct strbuf *symref, struct strbuf *sb_path,
- unsigned int *flags);
+typedef int read_raw_ref_fn(const char *submodule, const char *refname,
+ unsigned char *sha1, struct strbuf *symref,
+ struct strbuf *sb_path, unsigned int *flags);
typedef int verify_refname_available_fn(const char *refname, struct string_list *extra, struct string_list *skip, struct strbuf *err);
typedef int resolve_gitlink_ref_fn(const char *path, const char *refname,
unsigned char *sha1);
@@ -305,4 +314,8 @@ struct ref_storage_be {
};
extern struct ref_storage_be refs_be_files;
+
+#ifdef USE_LIBLMDB
+extern struct ref_storage_be refs_be_lmdb;
+#endif
#endif /* REFS_REFS_INTERNAL_H */
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 30/32] refs: add LMDB refs storage backend
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (28 preceding siblings ...)
2016-02-24 22:59 ` [PATCH v6 29/32] refs: break out resolve_ref_unsafe_submodule David Turner
@ 2016-02-24 22:59 ` David Turner
2016-02-24 22:59 ` [PATCH v6 31/32] refs: tests for lmdb backend David Turner
` (3 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:59 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Add a database backend for refs using LMDB. This backend runs git
for-each-ref about 30% faster than the files backend with fully-packed
refs on a repo with ~120k refs. It's also about 4x faster than using
fully-unpacked refs. In addition, and perhaps more importantly, it
avoids case-conflict issues on OS X.
LMDB has a few features that make it suitable for usage in git:
1. It is licensed under the OpenLDAP Public License, which is
GPL-compatible [1].
[1] http://www.gnu.org/licenses/license-list.en.html#newOpenLDAP
2. It is relatively lightweight; it requires only one header file, and
the library code takes under 64k at runtime.
3. It is well-tested: it's been used in OpenLDAP for years.
4. It's very fast. LMDB's benchmarks show that it is among
the fastest key-value stores.
5. It has a relatively simple concurrency story; readers don't
block writers and writers don't block readers.
Ronnie Sahlberg's original version of this patchset used tdb. The
major disadvantage of tdb is that tdb is hard to build on OS X. It's
also not in homebrew. So lmdb seemed simpler.
To test this backend's correctness, I added support to the test suite
to run under LMDB. Dozens of tests use manual ref/reflog
reading/writing, or create submodules without passing --ref-storage to
git init. If those tests are changed to use the update-ref machinery
or test-refs-lmdb-backend (or, in the case of packed-refs, corrupt
refs, and dumb fetch tests, are skipped), the only remaining failing
tests are the git-new-workdir tests. Later, we'll add this support
to the test suite.
Signed-off-by: David Turner <dturner@twopensource.com>
---
.gitignore | 1 +
Documentation/config.txt | 9 +
Documentation/git-clone.txt | 3 +-
Documentation/git-init.txt | 3 +-
Documentation/technical/refs-lmdb-backend.txt | 61 +
Documentation/technical/repository-version.txt | 7 +
Makefile | 12 +
configure.ac | 33 +
contrib/workdir/git-new-workdir | 3 +
path.c | 1 +
refs/lmdb-backend.c | 1886 ++++++++++++++++++++++++
test-refs-lmdb-backend.c | 65 +
12 files changed, 2082 insertions(+), 2 deletions(-)
create mode 100644 Documentation/technical/refs-lmdb-backend.txt
create mode 100644 refs/lmdb-backend.c
create mode 100644 test-refs-lmdb-backend.c
diff --git a/.gitignore b/.gitignore
index 1c2f832..87d45a2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -199,6 +199,7 @@
/test-path-utils
/test-prio-queue
/test-read-cache
+/test-refs-lmdb-backend
/test-regex
/test-revision-walking
/test-run-command
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 06d3659..65b5947 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1153,6 +1153,15 @@ difftool.<tool>.cmd::
difftool.prompt::
Prompt before each invocation of the diff tool.
+extensions.refStorage::
+ Type of ref storage backend. Default is to use the original
+ files based ref storage. When set to "lmdb", refs are stored
+ in an LMDB database. This setting reflects the refs storage
+ backend that was chosen via the --ref-storage option when the
+ repository was originally created. It is currently not
+ possible to change the refs storage backend of an existing
+ repository.
+
fetch.recurseSubmodules::
This option can be either set to a boolean value or to 'on-demand'.
Setting it to a boolean changes the behavior of fetch and pull to
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 68f56a7..e5873fe 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -227,7 +227,8 @@ objects from the source repository into a pack in the cloned repository.
--ref-storage=<name>::
Type of ref storage backend. Default is to use the original files
- based ref storage.
+ based ref storage. Set to "lmdb" to store refs in the LMDB database
+ backend.
<repository>::
The (possibly remote) repository to clone from. See the
diff --git a/Documentation/git-init.txt b/Documentation/git-init.txt
index 93f8d0c..129702b 100644
--- a/Documentation/git-init.txt
+++ b/Documentation/git-init.txt
@@ -116,7 +116,8 @@ does not exist, it will be created.
--ref-storage=<name>::
Type of refs storage backend. Default is to use the original "files"
storage, which stores ref data in files in `$GIT_DIR/refs` and
-`$GIT_DIR/packed-refs`.
+`$GIT_DIR/packed-refs`. Set to "lmdb" to activate the LMDB storage
+backend.
TEMPLATE DIRECTORY
------------------
diff --git a/Documentation/technical/refs-lmdb-backend.txt b/Documentation/technical/refs-lmdb-backend.txt
new file mode 100644
index 0000000..b8b5a0a
--- /dev/null
+++ b/Documentation/technical/refs-lmdb-backend.txt
@@ -0,0 +1,61 @@
+Notes on the LMDB refs backend
+==============================
+
+Design:
+------
+
+Refs and reflogs are stored in a lmdb database in .git/refs.lmdb. All
+keys and values are \0-terminated.
+
+Keys for refs are the name of the ref (e.g. refs/heads/master).
+Values are the value of the ref, in hex
+(e.g. 61f23eb0f81357c19fa91e2b8c6f3906c3a8f9b0).
+
+All per-worktree refs (refs/bisect/* and HEAD) are stored using
+the traditional files-based backend.
+
+Reflogs are stored as a series of database entries.
+
+For non-empty reflogs, there is one entry per logged ref update. The
+key format is logs/[refname]\0[timestamp]. The timestamp is a 64-bit
+unsigned integer number of nanoseconds since 1/1/1970, stored in
+network byte order. This means that reflog entries are
+chronologically ordered. Because LMDB is a btree database, we can
+efficiently iterate over these keys.
+
+For an empty reflog, there is a "header" entry to show that a reflog
+exists. The header has the same format as an ordinary reflog entry,
+but with a timestamp of all zeros and an empty value.
+
+Reflog values are in almost the same format as the original
+files-based reflog, including the trailing LF. The exception is that
+the SHAs are stored in binary instead of hex. The date in the reflog
+value matches the date in the timestamp field.
+
+Weaknesses:
+-----------
+
+The reflog format is somewhat inefficient: a binary format could store
+reflog date/time information in somewhat less space.
+
+The rsync and file:// transports don't work yet, because they
+don't use the refs API.
+
+git new-workdir is incompatible with the lmdb backend. Fortunately,
+git new-workdir is deprecated, and worktrees work fine.
+
+LMDB locks the entire database for write. Any other writer waits
+until the first writer is done before beginning. Readers do not wait
+for writers, and writers do not wait for readers. The underlying
+scheme is approximately MVCC; each reader's queries see the state of
+the database as-of the time that the reader acquired its read lock.
+This is not too far off from the files backend, which loads all refs
+into memory when one is requested.
+
+LMDB requires write access to the filesystem for locking. In the case
+where there are only readers (e.g. read-only repositories), this
+requirement can be relaxed using the MDB_NOLOCK option. But we don't
+yet have a config to do that.
+
+Don't use this over NFS or other remote filesystems. LMDB doesn't work
+there.
diff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt
index 00ad379..b317ba9 100644
--- a/Documentation/technical/repository-version.txt
+++ b/Documentation/technical/repository-version.txt
@@ -86,3 +86,10 @@ for testing format-1 compatibility.
When the config key `extensions.preciousObjects` is set to `true`,
objects in the repository MUST NOT be deleted (e.g., by `git-prune` or
`git repack -d`).
+
+`refStorage`
+~~~~~~~~~~~~
+This extension allows the use of alternate ref storage backends. The
+only defined values are `lmdb`, `files`, and the empty string (meaning
+`files`). The latter two values should never be used, because the
+files backend is the default and does not need repository version 1.
diff --git a/Makefile b/Makefile
index 10566d6..fd80e94 100644
--- a/Makefile
+++ b/Makefile
@@ -1042,6 +1042,17 @@ ifdef USE_LIBPCRE
EXTLIBS += -lpcre
endif
+ifdef USE_LIBLMDB
+ BASIC_CFLAGS += -DUSE_LIBLMDB
+ ifdef LIBLMDBDIR
+ BASIC_CFLAGS += -I$(LIBLMDBDIR)/include
+ EXTLIBS += -L$(LIBLMDBDIR)/$(lib) $(CC_LD_DYNPATH)$(LIBLMDBDIR)/$(lib)
+ endif
+ EXTLIBS += -llmdb
+ LIB_OBJS += refs/lmdb-backend.o
+ TEST_PROGRAMS_NEED_X += test-refs-lmdb-backend
+endif
+
ifdef HAVE_ALLOCA_H
BASIC_CFLAGS += -DHAVE_ALLOCA_H
endif
@@ -2140,6 +2151,7 @@ GIT-BUILD-OPTIONS: FORCE
@echo NO_CURL=\''$(subst ','\'',$(subst ','\'',$(NO_CURL)))'\' >>$@+
@echo NO_EXPAT=\''$(subst ','\'',$(subst ','\'',$(NO_EXPAT)))'\' >>$@+
@echo USE_LIBPCRE=\''$(subst ','\'',$(subst ','\'',$(USE_LIBPCRE)))'\' >>$@+
+ @echo USE_LIBLMDB=\''$(subst ','\'',$(subst ','\'',$(USE_LIBLMDB)))'\' >>$@+
@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
@echo NO_PYTHON=\''$(subst ','\'',$(subst ','\'',$(NO_PYTHON)))'\' >>$@+
@echo NO_UNIX_SOCKETS=\''$(subst ','\'',$(subst ','\'',$(NO_UNIX_SOCKETS)))'\' >>$@+
diff --git a/configure.ac b/configure.ac
index 89e2590..3853bec 100644
--- a/configure.ac
+++ b/configure.ac
@@ -271,6 +271,24 @@ AS_HELP_STRING([], [ARG can be also prefix for libpcre library and hea
dnl it yet.
GIT_CONF_SUBST([LIBPCREDIR])
fi)
+
+USE_LIBLMDB=YesPlease
+AC_ARG_WITH(liblmdb,
+AS_HELP_STRING([--with-liblmdb],[support lmdb (default is YES])
+AS_HELP_STRING([], [ARG can be also prefix for liblmdb library and headers]),
+ if test "$withval" = "no"; then
+ USE_LIBLMDB=
+ elif test "$withval" = "yes"; then
+ USE_LIBLMDB=YesPlease
+ else
+ USE_LIBLMDB=YesPlease
+ LIBLMDBDIR=$withval
+ AC_MSG_NOTICE([Setting LIBLMDBDIR to $LIBLMDBDIR])
+ dnl USE_LIBLMDB can still be modified below, so don't substitute
+ dnl it yet.
+ GIT_CONF_SUBST([LIBLMDBDIR])
+ fi)
+
#
# Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.
AC_FUNC_ALLOCA
@@ -510,6 +528,21 @@ GIT_CONF_SUBST([USE_LIBPCRE])
fi
+if test -n "$USE_LIBLMDB"; then
+
+GIT_STASH_FLAGS($LIBLMDBDIR)
+
+AC_CHECK_LIB([lmdb], [mdb_env_open],
+[USE_LIBLMDB=YesPlease],
+[USE_LIBLMDB=])
+
+GIT_UNSTASH_FLAGS($LIBLMDBDIR)
+
+GIT_CONF_SUBST([USE_LIBLMDB])
+
+fi
+
+
#
# Define NO_CURL if you do not have libcurl installed. git-http-pull and
# git-http-push are not built, and you cannot use http:// and https://
diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 888c34a..d153ddf 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -28,6 +28,9 @@ git_dir=$(cd "$orig_git" 2>/dev/null &&
git rev-parse --git-dir 2>/dev/null) ||
die "Not a git repository: \"$orig_git\""
+ref_storage=$(git config extensions.refstorage || echo "files")
+test "$ref_storage" != "files" && die "git-new-workdir is incompatible with ref storage other than 'files'"
+
case "$git_dir" in
.git)
git_dir="$orig_git/.git"
diff --git a/path.c b/path.c
index 2e67a2b..347aba4 100644
--- a/path.c
+++ b/path.c
@@ -112,6 +112,7 @@ static struct common_dir common_list[] = {
{ 0, 1, 0, "lost-found" },
{ 0, 1, 0, "objects" },
{ 0, 1, 0, "refs" },
+ { 0, 1, 0, "refs.lmdb" },
{ 0, 1, 1, "refs/bisect" },
{ 0, 1, 0, "remotes" },
{ 0, 1, 0, "worktrees" },
diff --git a/refs/lmdb-backend.c b/refs/lmdb-backend.c
new file mode 100644
index 0000000..6b72425
--- /dev/null
+++ b/refs/lmdb-backend.c
@@ -0,0 +1,1886 @@
+/*
+ * This file implements a lmdb backend for refs.
+ *
+ * The design of this backend relies on lmdb's write lock -- that is, any
+ * write transaction blocks all other writers. Thus, as soon as a ref
+ * transaction is opened, we know that any values we read won't
+ * change out from under us, and we have a fully-consistent view of the
+ * database.
+ *
+ * We store the content of refs including the trailing \0 so that
+ * standard C string functions can handle them. Just like struct
+ * strbuf.
+ */
+#include "cache.h"
+#include <lmdb.h>
+#include "object.h"
+#include "refs.h"
+#include "refs-internal.h"
+#include "tag.h"
+#include "lockfile.h"
+
+static MDB_env *env;
+
+static char *db_path;
+
+struct lmdb_transaction {
+ MDB_txn *txn;
+ MDB_dbi dbi;
+ MDB_cursor *cursor;
+ const char *submodule;
+ int flags;
+};
+
+static struct lmdb_transaction transaction;
+
+static int in_write_transaction(void)
+{
+ return transaction.txn && !(transaction.flags & MDB_RDONLY);
+}
+
+static int cleanup_registered;
+
+static void cleanup_txn(void)
+{
+ if (transaction.txn) {
+ mdb_txn_abort(transaction.txn);
+ }
+}
+
+static void init_env(MDB_env **env, const char *path)
+{
+ int ret;
+ if (*env)
+ return;
+
+ assert(path);
+
+ ret = mdb_env_create(env);
+ if (ret)
+ die("BUG: mdb_env_create failed: %s", mdb_strerror(ret));
+ ret = mdb_env_set_maxreaders(*env, 1000);
+ if (ret)
+ die("BUG: mdb_env_set_maxreaders failed: %s", mdb_strerror(ret));
+ ret = mdb_env_set_mapsize(*env, (1<<30));
+ if (ret)
+ die("BUG: mdb_set_mapsize failed: %s", mdb_strerror(ret));
+ for (;;) {
+ /*
+ * The LMDB docs say, "Opening a database can fail if
+ * another process is opening or closing it at exactly
+ * the same time." They don't specify what will
+ * happen in this case. From reading LDMB's code, it
+ * appears that init_env will return an error code
+ * from fcntl: EAGAIN or EACCES. So we'll retry in
+ * those cases.
+ */
+ ret = mdb_env_open(*env, path, MDB_NOSUBDIR, 0664);
+ if (ret == 0)
+ break;
+ if (ret != EAGAIN && ret != EACCES)
+ die("BUG: mdb_env_open (%s) failed: %s", path,
+ mdb_strerror(ret));
+ }
+
+ if (!cleanup_registered) {
+ cleanup_registered = 1;
+ atexit(cleanup_txn);
+ }
+}
+
+static int lmdb_init_db(int shared, struct strbuf *err)
+{
+ /*
+ * To create a db, all we need to do is set up the db path
+ * variable.
+ */
+
+ if (!db_path)
+ db_path = xstrdup(real_path(git_path("refs.lmdb")));
+ return 0;
+}
+
+static void mdb_cursor_open_or_die(struct lmdb_transaction *transaction,
+ MDB_cursor **cursor)
+{
+ int ret = mdb_cursor_open(transaction->txn, transaction->dbi, cursor);
+ if (ret)
+ die("BUG: mdb_cursor_open failed: %s", mdb_strerror(ret));
+}
+
+static void submodule_path(struct strbuf *sb, const char *submodule,
+ const char *refname)
+{
+ if (submodule)
+ strbuf_git_path_submodule(sb, submodule, "%s", refname);
+ else
+ strbuf_git_path(sb, "%s", refname);
+}
+
+static int read_per_worktree_ref(const char *submodule, const char *refname,
+ struct MDB_val *val, int *needs_free)
+{
+ struct strbuf sb = STRBUF_INIT;
+ struct strbuf path = STRBUF_INIT;
+ struct stat st;
+ int ret = -1;
+ int symlink_head;
+
+ submodule_path(&path, submodule, refname);
+
+#ifdef NO_SYMLINK_HEAD
+ symlink_head = 0;
+#else
+ symlink_head = 1;
+#endif
+
+ if (symlink_head) {
+ if (lstat(path.buf, &st)) {
+ if (errno == ENOENT)
+ ret = MDB_NOTFOUND;
+ goto done;
+ }
+ if (S_ISLNK(st.st_mode)) {
+ strbuf_readlink(&sb, path.buf, 0);
+ if (starts_with(sb.buf, "refs/") &&
+ !check_refname_format(sb.buf, 0)) {
+ val->mv_data = xstrfmt("ref: %s", sb.buf);
+ val->mv_size = strlen(val->mv_data) + 1;
+ ret = 0;
+ } else {
+ ret = MDB_NOTFOUND;
+ }
+ strbuf_release(&sb);
+ goto done;
+ }
+ }
+
+ if (strbuf_read_file(&sb, path.buf, 200) < 0) {
+ strbuf_release(&sb);
+ if (errno == ENOENT)
+ ret = MDB_NOTFOUND;
+ goto done;
+ }
+ strbuf_rtrim(&sb);
+
+ val->mv_data = strbuf_detach(&sb, &val->mv_size);
+ val->mv_size++;
+
+ ret = 0;
+done:
+ strbuf_release(&path);
+ *needs_free = !ret;
+ return ret;
+}
+
+static void write_per_worktree_ref(const char *submodule, const char *refname,
+ MDB_val *val)
+{
+ static struct lock_file lock;
+ int fd;
+ int len = val->mv_size - 1;
+ struct strbuf path = STRBUF_INIT;
+
+ submodule_path(&path, submodule, refname);
+ safe_create_leading_directories(path.buf);
+
+ fd = hold_lock_file_for_update(&lock, path.buf, LOCK_DIE_ON_ERROR);
+ strbuf_release(&path);
+
+ if (write_in_full(fd, val->mv_data, len) != len ||
+ write_in_full(fd, "\n", 1) != 1)
+ die_errno(_("failed to write new HEAD"));
+
+ if (commit_lock_file(&lock))
+ die_errno(_("failed to write new HEAD"));
+}
+
+static int del_per_worktree_ref(const char *submodule, const char *refname,
+ MDB_val *val)
+{
+ struct strbuf path = STRBUF_INIT;
+ int result;
+
+ /*
+ * Returning deleted ref data is not yet implemented, but no
+ * callers need it.
+ */
+ assert(val == NULL);
+
+ submodule_path(&path, submodule, refname);
+
+ result = unlink(path.buf);
+ strbuf_release(&path);
+ if (result && errno != ENOENT)
+ return 1;
+
+ return 0;
+}
+
+/*
+ * Read a ref. If the ref is a per-worktree ref, read it from disk.
+ * Otherwise, read it from LMDB. LMDB manages its own memory, so the
+ * data returned in *val will ordinarily not need to be freed. But
+ * when a per-worktree ref is (successfully) read, non-LMDB memory is
+ * allocated. In this case, *needs_free is set so that the caller can
+ * free the memory when it is done with it.
+ */
+static int mdb_get_or_die(struct lmdb_transaction *transaction, MDB_val *key,
+ MDB_val *val, int *needs_free)
+{
+ int ret;
+
+ if (ref_type(key->mv_data) != REF_TYPE_NORMAL)
+ return read_per_worktree_ref(transaction->submodule,
+ key->mv_data, val, needs_free);
+
+ *needs_free = 0;
+ ret = mdb_get(transaction->txn, transaction->dbi, key, val);
+ if (ret) {
+ if (ret != MDB_NOTFOUND)
+ die("BUG: mdb_get failed: %s", mdb_strerror(ret));
+ return ret;
+ }
+ return 0;
+}
+
+static int mdb_del_or_die(struct lmdb_transaction *transaction, MDB_val *key,
+ MDB_val *val)
+{
+ int ret;
+
+ if (ref_type(key->mv_data) != REF_TYPE_NORMAL)
+ die("BUG: this backend should only try to delete normal refs");
+
+ ret = mdb_del(transaction->txn, transaction->dbi, key, val);
+ if (ret) {
+ if (ret != MDB_NOTFOUND)
+ die("BUG: mdb_del failed: %s", mdb_strerror(ret));
+ return ret;
+ }
+ return 0;
+}
+
+static void mdb_put_or_die(struct lmdb_transaction *transaction, MDB_val *key,
+ MDB_val *val, int mode)
+{
+ int ret;
+
+ if (ref_type(key->mv_data) != REF_TYPE_NORMAL)
+ die("BUG: this backend should only try to write normal refs");
+
+ ret = mdb_put(transaction->txn, transaction->dbi, key, val, mode);
+ if (ret) {
+ if (ret == MDB_BAD_VALSIZE)
+ die(_("Ref name %s too long (max size is %d)"),
+ (const char *)key->mv_data,
+ mdb_env_get_maxkeysize(env));
+ else
+ die("BUG: mdb_put failed: (%s -> %s) %s",
+ (const char *)key->mv_data,
+ (const char *)val->mv_data, mdb_strerror(ret));
+ }
+}
+
+static int mdb_cursor_get_or_die(MDB_cursor *cursor, MDB_val *key, MDB_val *val, int mode)
+{
+ int ret;
+
+ ret = mdb_cursor_get(cursor, key, val, mode);
+ if (ret) {
+ if (ret != MDB_NOTFOUND)
+ die("BUG: mdb_cursor_get failed: %s", mdb_strerror(ret));
+ return ret;
+ }
+ assert(((char *)val->mv_data)[val->mv_size - 1] == 0);
+ return 0;
+}
+
+static int mdb_cursor_del_or_die(MDB_cursor *cursor, int flags)
+{
+ int ret = mdb_cursor_del(cursor, flags);
+ if (ret) {
+ if (ret != MDB_NOTFOUND)
+ die("BUG: mdb_cursor_del failed: %s", mdb_strerror(ret));
+ return ret;
+ }
+ return 0;
+}
+
+/*
+ * Begin a transaction. Because only one transaction per thread is
+ * permitted, we use a global transaction object. If a read-write
+ * transaction is presently already in-progress, and a read-only
+ * transaction is requested, the read-write transaction will be
+ * returned instead. If a read-write transaction is requested and a
+ * read-only transaction is open, the read-only transaction will be
+ * closed.
+ *
+ * It is a bug to request a read-write transaction during another
+ * read-write transaction.
+ *
+ * As a result, it is unsafe to retain read-only transactions past the
+ * point where a read-write transaction might be needed. For
+ * instance, any call that has callbacks outside this module must
+ * conclude all of its reads from the database before calling those
+ * callbacks, or must reacquire the transaction after its callbacks
+ * are completed.
+ */
+static int lmdb_transaction_begin_flags(struct strbuf *err, unsigned int flags)
+{
+ int ret;
+ MDB_txn *txn;
+ static size_t last_txnid = 0;
+ int force_restart = 0;
+ MDB_envinfo stat;
+
+ if (!db_path)
+ db_path = xstrdup(real_path(git_path("refs.lmdb")));
+
+ init_env(&env, db_path);
+
+ /*
+ * Since each transaction sees a consistent view of the db,
+ * downstream processes that write the db won't be seen in
+ * this transaction. We can check if the last transaction id
+ * has changed since this read transaction was started, and if
+ * so, we want to reopen the transaction.
+ */
+
+ mdb_env_info(env, &stat);
+ if (stat.me_last_txnid != last_txnid) {
+ force_restart = 1;
+ last_txnid = stat.me_last_txnid;
+ }
+
+ if (!transaction.txn) {
+ ret = mdb_txn_begin(env, NULL, flags, &txn);
+ if (ret) {
+ strbuf_addf(err, "BUG: mdb_txn_begin failed: %s",
+ mdb_strerror(ret));
+ return -1;
+ }
+ ret = mdb_dbi_open(txn, NULL, 0, &transaction.dbi);
+ if (ret) {
+ strbuf_addf(err, "BUG: mdb_txn_open failed: %s",
+ mdb_strerror(ret));
+ return -1;
+ }
+ transaction.txn = txn;
+ transaction.flags = flags;
+ return 0;
+ }
+
+ if (transaction.flags == flags && !(flags & MDB_RDONLY))
+ die("BUG: rw transaction started during another rw txn");
+
+ if (force_restart || (transaction.flags != flags && transaction.flags & MDB_RDONLY)) {
+ /*
+ * RO -> RW, or forced restart due to possible changes
+ * from downstream processes.
+ */
+ mdb_txn_abort(transaction.txn);
+ transaction.txn = NULL;
+ ret = mdb_txn_begin(env, NULL, flags, &txn);
+ if (ret) {
+ strbuf_addf(err, "BUG: restarting txn: mdb_txn_begin failed: %s",
+ mdb_strerror(ret));
+ return -1;
+ }
+ ret = mdb_dbi_open(txn, NULL, 0, &transaction.dbi);
+ if (ret) {
+ strbuf_addf(err, "BUG: mdb_txn_open failed: %s",
+ mdb_strerror(ret));
+ return -1;
+ }
+ transaction.txn = txn;
+ transaction.flags = flags;
+ }
+ /* RW -> RO just keeps the RW txn */
+ return 0;
+}
+
+static struct lmdb_transaction *lmdb_transaction_begin_flags_or_die(int flags)
+{
+ struct strbuf err = STRBUF_INIT;
+ if (lmdb_transaction_begin_flags(&err, flags))
+ die("%s", err.buf);
+ return &transaction;
+}
+
+static int verify_refname_available_txn(struct lmdb_transaction *transaction,
+ const char *refname,
+ struct string_list *extras,
+ struct string_list *skip,
+ struct strbuf *err)
+{
+ MDB_cursor *cursor;
+ MDB_val key;
+ MDB_val val;
+ int mdb_ret;
+ size_t refname_len;
+ char *search_key;
+ const char *extra_refname;
+ int ret = 1;
+ size_t i;
+
+ mdb_cursor_open_or_die(transaction, &cursor);
+
+ refname_len = strlen(refname) + 2;
+ key.mv_size = refname_len;
+ search_key = xmalloc(refname_len);
+ memcpy(search_key, refname, refname_len - 2);
+ search_key[refname_len - 2] = '/';
+ search_key[refname_len - 1] = 0;
+ key.mv_data = search_key;
+
+ /* Check for subdirs of refname: we start at refname/ */
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);
+
+ while (!mdb_ret) {
+ if (starts_with(key.mv_data, refname) &&
+ ((char *)key.mv_data)[refname_len - 2] == '/') {
+ if (skip && string_list_has_string(skip, key.mv_data))
+ goto next;
+
+ strbuf_addf(err, _("'%s' exists; cannot create '%s'"), (char *)key.mv_data, refname);
+ goto done;
+ }
+ break;
+ next:
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);
+ }
+
+ /* Check for parent dirs of refname. */
+ for (i = 0; i < refname_len - 2; i++) {
+ if (search_key[i] == '/') {
+ search_key[i] = 0;
+ if (skip && string_list_has_string(skip, search_key)) {
+ search_key[i] = '/';
+ continue;
+ }
+
+ if (extras && string_list_has_string(extras, search_key)) {
+ strbuf_addf(err, _("cannot process '%s' and '%s' at the same time"),
+ refname, search_key);
+ goto done;
+ }
+
+ key.mv_data = search_key;
+ key.mv_size = i + 1;
+ if (!mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET)) {
+ strbuf_addf(err, _("'%s' exists; cannot create '%s'"), (char *)key.mv_data, refname);
+ goto done;
+ }
+ search_key[i] = '/';
+ }
+ }
+
+ extra_refname = find_descendant_ref(refname, extras, skip);
+ if (extra_refname) {
+ strbuf_addf(err,
+ _("cannot process '%s' and '%s' at the same time"),
+ refname, extra_refname);
+ ret = 1;
+ } else {
+ ret = 0;
+ }
+done:
+ mdb_cursor_close(cursor);
+ free(search_key);
+ return ret;
+}
+
+static MDB_env *submodule_txn_begin(struct lmdb_transaction *transaction)
+{
+ int ret;
+ MDB_env *submodule_env = NULL;
+ struct strbuf path = STRBUF_INIT;
+
+ strbuf_git_path_submodule(&path, transaction->submodule, "refs.lmdb");
+
+ init_env(&submodule_env, path.buf);
+
+ ret = mdb_txn_begin(submodule_env, NULL, MDB_RDONLY, &transaction->txn);
+ if (ret)
+ die("BUG: mdb_txn_begin failed: %s", mdb_strerror(ret));
+
+ ret = mdb_dbi_open(transaction->txn, NULL, 0, &transaction->dbi);
+ if (ret)
+ die("BUG: mdb_txn_open failed: %s", mdb_strerror(ret));
+
+ strbuf_release(&path);
+ return submodule_env;
+}
+
+static struct lmdb_transaction *get_submodule_transaction(const char *submodule)
+{
+ static char *last_submodule = NULL;
+ static struct lmdb_transaction transaction = {NULL};
+ static MDB_env *env = NULL;
+
+ if (submodule == NULL) {
+ if (last_submodule != NULL) {
+ if (env)
+ mdb_env_close(env);
+ env = NULL;
+ free((char *)transaction.submodule);
+ transaction.submodule = NULL;
+ free(last_submodule);
+ last_submodule = NULL;
+ }
+ return lmdb_transaction_begin_flags_or_die(MDB_RDONLY);
+ }
+
+ if (submodule != NULL && last_submodule != NULL &&
+ !strcmp(submodule, last_submodule) && transaction.txn != NULL)
+ return &transaction;
+
+ free((char *)transaction.submodule);
+ transaction.submodule = xstrdup(submodule);
+ transaction.txn = NULL;
+
+ last_submodule = xstrdup(submodule);
+ env = submodule_txn_begin(&transaction);
+
+ return &transaction;
+}
+
+static int lmdb_read_raw_ref(const char *submodule, const char *refname,
+ unsigned char *sha1, struct strbuf *symref,
+ struct strbuf *sb_path, unsigned int *flags)
+{
+ char *buf;
+ struct MDB_val key, val;
+ struct strbuf err = STRBUF_INIT;
+ int needs_free = 0;
+ int ret = -1;
+ struct lmdb_transaction *transaction;
+
+ transaction = get_submodule_transaction(submodule);
+
+ val.mv_size = 0;
+ val.mv_data = NULL;
+
+ if (flags)
+ *flags = 0;
+
+ key.mv_data = (void *)refname;
+ key.mv_size = strlen(refname) + 1;
+ if (mdb_get_or_die(transaction, &key, &val, &needs_free)) {
+ errno = ENOENT;
+ return -1;
+ }
+
+ buf = val.mv_data;
+ assert(buf[val.mv_size - 1] == 0);
+
+ if (starts_with(buf, "ref:")) {
+ buf += 4;
+ while (isspace(*buf))
+ buf++;
+
+ strbuf_reset(symref);
+ strbuf_addstr(symref, buf);
+ if (flags)
+ *flags |= REF_ISSYMREF;
+ ret = 0;
+ goto done;
+ }
+
+ /*
+ * Please note that FETCH_HEAD has additional
+ * data after the sha.
+ */
+ if (get_sha1_hex(buf, sha1) ||
+ (buf[40] != '\0' && !isspace(buf[40]))) {
+ if (flags)
+ *flags |= REF_ISBROKEN;
+ errno = EINVAL;
+ goto done;
+ }
+ ret = 0;
+ goto done;
+
+done:
+ if (needs_free)
+ free(val.mv_data);
+ return ret;
+
+}
+
+static void write_u64(char *buf, uint64_t number)
+{
+ int i;
+
+ for (i = 0; i < 8; i++)
+ buf[i] = (number >> (i * 8)) & 0xff;
+}
+
+static int show_one_reflog_ent(struct strbuf *sb, each_reflog_ent_fn fn, void *cb_data)
+{
+ unsigned char osha1[20], nsha1[20];
+ char *email_end, *message;
+ unsigned long timestamp;
+ int tz;
+
+ /* old (raw sha) new (raw sha) name <email> SP time TAB msg LF */
+ if (sb->len < 41 || sb->buf[sb->len - 1] != '\n' ||
+ !(email_end = strchr(sb->buf + 40, '>')) ||
+ email_end[1] != ' ')
+ return 0; /* corrupt? */
+
+ timestamp = strtoul(email_end + 2, &message, 10);
+
+ if (!timestamp ||
+ !message || message[0] != ' ' ||
+ (message[1] != '+' && message[1] != '-') ||
+ !isdigit(message[2]) || !isdigit(message[3]) ||
+ !isdigit(message[4]) || !isdigit(message[5]))
+ return 0; /* corrupt? */
+
+ hashcpy(osha1, (const unsigned char *)sb->buf);
+ hashcpy(nsha1, (const unsigned char *)sb->buf + 20);
+
+ email_end[1] = '\0';
+ tz = strtol(message + 1, NULL, 10);
+ if (message[6] != '\t')
+ message += 6;
+ else
+ message += 7;
+ return fn(osha1, nsha1, sb->buf + 40, timestamp, tz, message, cb_data);
+}
+
+static void format_reflog_entry(struct strbuf *buf,
+ const unsigned char *old_sha1,
+ const unsigned char *new_sha1,
+ const char *committer, const char *msg)
+{
+ int len;
+ int msglen;
+
+ assert(buf->len == 0);
+ strbuf_add(buf, old_sha1, 20);
+ strbuf_add(buf, new_sha1, 20);
+ strbuf_addstr(buf, committer);
+ strbuf_addch(buf, '\n');
+
+ len = buf->len;
+ msglen = msg ? strlen(msg) : 0;
+ if (msglen) {
+ int copied;
+ strbuf_grow(buf, msglen + 1);
+ copied = copy_reflog_msg(buf->buf + 40 + strlen(committer), msg) - 1;
+ buf->len = len + copied;
+ buf->buf[buf->len] = 0;
+ }
+}
+
+static int log_ref_write(const char *refname,
+ const unsigned char *old_sha1,
+ const unsigned char *new_sha1,
+ const char *msg,
+ int flags,
+ struct strbuf *err)
+{
+ MDB_val key, val;
+ uint64_t now = getnanotime();
+ int result;
+ struct strbuf log_key = STRBUF_INIT;
+ int refname_len;
+ MDB_cursor *cursor;
+ struct strbuf buf = STRBUF_INIT;
+ const char *timestamp;
+ uint64_t zero = 0;
+
+ if (log_all_ref_updates < 0)
+ log_all_ref_updates = !is_bare_repository();
+
+ /* it is assumed that we are in a ref transaction here */
+ assert(transaction.txn);
+
+ result = safe_create_reflog(refname, flags & REF_FORCE_CREATE_REFLOG, err);
+ if (result)
+ return result;
+
+ /* "logs/" + refname + \0 + 8-byte timestamp for sorting and expiry. */
+ refname_len = strlen(refname);
+
+ strbuf_addf(&log_key, "logs/%s", refname);
+ strbuf_add(&log_key, &zero, 8);
+ key.mv_data = log_key.buf;
+ key.mv_size = log_key.len + 1;
+
+ mdb_cursor_open_or_die(&transaction, &cursor);
+
+ /* if no reflog exists, we're done */
+ if (mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE) ||
+ strcmp(key.mv_data, log_key.buf))
+ goto done;
+
+ /* Is this a header? We only need the header for empty reflogs */
+ timestamp = (const char *)key.mv_data + refname_len + 6;
+ if (ntohll(*(uint64_t *)timestamp) == 0)
+ mdb_cursor_del_or_die(cursor, 0);
+
+ key.mv_data = log_key.buf;
+
+ write_u64((char *)key.mv_data + refname_len + 6, htonll(now));
+
+ format_reflog_entry(&buf, old_sha1, new_sha1,
+ git_committer_info(0), msg);
+ assert(buf.len >= 42);
+ val.mv_data = buf.buf;
+ val.mv_size = buf.len + 1;
+
+ mdb_put_or_die(&transaction, &key, &val, 0);
+ strbuf_release(&buf);
+
+done:
+ strbuf_release(&log_key);
+ mdb_cursor_close(cursor);
+ return 0;
+}
+
+static int lmdb_verify_refname_available(const char *refname,
+ struct string_list *extras,
+ struct string_list *skip,
+ struct strbuf *err)
+{
+ lmdb_transaction_begin_flags_or_die(MDB_RDONLY);
+ return verify_refname_available_txn(&transaction, refname, extras, skip, err);
+}
+
+/*
+ * Attempt to resolve `refname` to `old_sha1` (if old_sha1 is
+ * non-null). The return value is a pointer to a newly-allocated
+ * string containing the next ref name that this resolves to. So if
+ * HEAD is a symbolic ref to refs/heads/example, which is itself a
+ * symbolic ref to refs/heads/foo, return refs/heads/example,
+ * and fill in resolved_sha1 with the sha of refs/heads/foo.
+ */
+static char *check_ref(MDB_txn *txn, const char *refname,
+ const unsigned char *old_sha1,
+ unsigned char *resolved_sha1, int flags,
+ int *type_p)
+{
+ int mustexist = (old_sha1 && !is_null_sha1(old_sha1));
+ int resolve_flags = 0;
+ int type;
+ char *resolved_refname;
+
+ if (mustexist)
+ resolve_flags |= RESOLVE_REF_READING;
+ if (flags & REF_DELETING) {
+ resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
+ if (flags & REF_NODEREF)
+ resolve_flags |= RESOLVE_REF_NO_RECURSE;
+ }
+
+ /*
+ * The first time we resolve the refname, we're just trying to
+ * see if there is any ref at all by this name, even if it is
+ * a broken symref.
+ */
+ refname = resolve_ref_unsafe(refname, resolve_flags,
+ resolved_sha1, &type);
+ if (type_p)
+ *type_p = type;
+
+ if (!refname)
+ return NULL;
+
+ /*
+ * Need to copy refname here because the resolve_ref_unsafe
+ * returns a pointer to a static buffer that could get mangled
+ * by the second call.
+ */
+ resolved_refname = xstrdup(refname);
+
+ if (old_sha1) {
+ if (flags & REF_NODEREF) {
+ resolve_flags &= ~RESOLVE_REF_NO_RECURSE;
+
+ resolve_ref_unsafe(refname, resolve_flags,
+ resolved_sha1, &type);
+ }
+ if (hashcmp(old_sha1, resolved_sha1)) {
+ error(_("ref %s is at %s but expected %s"), refname,
+ sha1_to_hex(resolved_sha1), sha1_to_hex(old_sha1));
+
+ return NULL;
+ }
+ }
+ return resolved_refname;
+}
+
+static int mdb_transaction_commit(struct lmdb_transaction *transaction,
+ struct strbuf *err)
+{
+ int result;
+
+ result = mdb_txn_commit(transaction->txn);
+ if (result && err)
+ strbuf_addstr(err, mdb_strerror(result));
+
+ transaction->txn = NULL;
+ return result;
+}
+
+static void mdb_transaction_abort(struct lmdb_transaction *transaction)
+{
+ mdb_txn_abort(transaction->txn);
+ transaction->txn = NULL;
+}
+
+static int lmdb_delete_reflog(const char *refname)
+{
+ MDB_val key, val;
+ char *log_path;
+ int len;
+ MDB_cursor *cursor;
+ int ret = 0;
+ int mdb_ret;
+ struct strbuf err = STRBUF_INIT;
+ int in_transaction;
+
+ in_transaction = in_write_transaction();
+
+ log_path = xstrfmt("logs/%s", refname);
+ len = strlen(log_path) + 1;
+
+ key.mv_data = log_path;
+ key.mv_size = len;
+
+ if (!in_transaction)
+ lmdb_transaction_begin_flags_or_die(0);
+
+ mdb_cursor_open_or_die(&transaction, &cursor);
+
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);
+
+ while (!mdb_ret) {
+ if (key.mv_size < len)
+ break;
+
+ if (!starts_with(key.mv_data, log_path) ||
+ ((char *)key.mv_data)[len - 1] != 0)
+ break;
+
+ mdb_cursor_del_or_die(cursor, 0);
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);
+ }
+
+ free(log_path);
+ mdb_cursor_close(cursor);
+ transaction.cursor = NULL;
+
+ if (!in_transaction && mdb_transaction_commit(&transaction, &err)) {
+ warning("%s", err.buf);
+ ret = 01;
+ }
+ strbuf_release(&err);
+ return ret;
+}
+
+#define REF_NO_REFLOG 0x8000
+
+static int lmdb_transaction_update(const char *refname,
+ const unsigned char *new_sha1,
+ const unsigned char *old_sha1,
+ unsigned int flags, const char *msg,
+ struct strbuf *err)
+{
+ const char *orig_refname = refname;
+ MDB_val key, val;
+ unsigned char resolved_sha1[20];
+ int type;
+ int ret = -1;
+
+ if ((flags & REF_HAVE_NEW) && is_null_sha1(new_sha1))
+ flags |= REF_DELETING;
+
+ if (new_sha1 && !is_null_sha1(new_sha1) &&
+ check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
+ strbuf_addf(err, _("refusing to update ref with bad name %s"),
+ refname);
+ return TRANSACTION_GENERIC_ERROR;
+ }
+
+ refname = check_ref(transaction.txn, orig_refname, old_sha1,
+ resolved_sha1, flags, &type);
+ if (refname == NULL) {
+ strbuf_addf(err, _("cannot lock the ref '%s'"), orig_refname);
+ return TRANSACTION_GENERIC_ERROR;
+ }
+
+ if (!(flags & REF_DELETING) && is_null_sha1(resolved_sha1) &&
+ verify_refname_available_txn(&transaction, refname, NULL, NULL, err))
+ return TRANSACTION_NAME_CONFLICT;
+
+ if (flags & REF_NODEREF) {
+ free((void *)refname);
+ refname = orig_refname;
+ }
+
+ key.mv_size = strlen(refname) + 1;
+ key.mv_data = (void *)refname;
+
+ if ((flags & REF_HAVE_NEW) && !is_null_sha1(new_sha1)) {
+ int overwriting_symref = ((type & REF_ISSYMREF) &&
+ (flags & REF_NODEREF));
+
+ struct object *o = parse_object(new_sha1);
+ if (!o) {
+ strbuf_addf(err,
+ _("Trying to write ref %s with nonexistent object %s"),
+ refname, sha1_to_hex(new_sha1));
+ goto done;
+ }
+ if (o->type != OBJ_COMMIT && is_branch(refname)) {
+ strbuf_addf(err,
+ _("Trying to write non-commit object %s to branch %s"),
+ sha1_to_hex(new_sha1), refname);
+ goto done;
+ }
+
+ if (!overwriting_symref
+ && !hashcmp(resolved_sha1, new_sha1)) {
+ /*
+ * The reference already has the desired
+ * value, so we don't need to write it.
+ */
+ flags |= REF_NO_REFLOG;
+ } else {
+ val.mv_size = 41;
+ if (new_sha1)
+ val.mv_data = sha1_to_hex(new_sha1);
+ else
+ val.mv_data = sha1_to_hex(null_sha1);
+ mdb_put_or_die(&transaction, &key, &val, 0);
+ }
+ }
+
+ if (flags & REF_DELETING) {
+ if (mdb_del_or_die(&transaction, &key, NULL)) {
+ if (old_sha1 && !is_null_sha1(old_sha1)) {
+ strbuf_addf(err, _("No such ref %s"), refname);
+ ret = TRANSACTION_GENERIC_ERROR;
+ goto done;
+ }
+ }
+ lmdb_delete_reflog(orig_refname);
+ } else if (!(flags & REF_NO_REFLOG)) {
+ if (!new_sha1)
+ new_sha1 = null_sha1;
+ if (log_ref_write(orig_refname, resolved_sha1,
+ new_sha1, msg, flags, err) < 0)
+ goto done;
+ if (strcmp(refname, orig_refname) &&
+ log_ref_write(refname, resolved_sha1,
+ new_sha1, msg, flags, err) < 0)
+ goto done;
+ }
+
+ ret = 0;
+done:
+ if (refname != orig_refname)
+ free((void *) refname);
+ return ret;
+}
+
+static int lmdb_transaction_commit(struct ref_transaction *ref_transaction,
+ struct string_list *affected_refnames,
+ struct strbuf *err)
+{
+ int ret = 0, i;
+ int n = ref_transaction->nr;
+ struct ref_update **updates = ref_transaction->updates;
+
+ /*
+ * We might already be in a write transaction, because some
+ * lmdb backend functionality is implemented in terms of
+ * (other stuff) + ref_transaction_commit
+ */
+ if (!in_write_transaction())
+ lmdb_transaction_begin_flags_or_die(0);
+
+ for (i = 0; i < n; i++) {
+ struct ref_update *update = updates[i];
+
+ if (lmdb_transaction_update(update->refname,
+ update->new_sha1,
+ (update->flags & REF_HAVE_OLD) ?
+ update->old_sha1 : NULL,
+ update->flags,
+ update->msg,
+ err)) {
+ mdb_transaction_abort(&transaction);
+ ret = -1;
+ goto cleanup;
+ }
+
+ }
+ ret = mdb_transaction_commit(&transaction, err);
+
+cleanup:
+ ref_transaction->state = REF_TRANSACTION_CLOSED;
+ return ret;
+}
+
+static int rename_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
+ const char *email, unsigned long timestamp, int tz,
+ const char *message, void *cb_data)
+{
+
+ const char *newrefname = cb_data;
+ MDB_val key, new_key, val;
+ struct strbuf new_key_buf = STRBUF_INIT;
+
+ assert(transaction.cursor);
+
+ if (mdb_cursor_get_or_die(transaction.cursor, &key, &val, MDB_GET_CURRENT))
+ die("BUG: renaming ref: mdb_cursor_get failed to get current");
+
+ /* This must really be a reflog entry */
+ assert(val.mv_size > 42);
+
+ strbuf_addf(&new_key_buf, "logs/%s", newrefname);
+ strbuf_add(&new_key_buf, (char *)key.mv_data + key.mv_size - 8, 8);
+
+ new_key.mv_size = new_key_buf.len + 1;
+ new_key.mv_data = new_key_buf.buf;
+ mdb_put_or_die(&transaction, &new_key, &val, 0);
+ mdb_cursor_del_or_die(transaction.cursor, 0);
+ strbuf_release(&new_key_buf);
+ return 0;
+}
+
+static int lmdb_rename_ref(const char *oldref, const char *newref, const char *logmsg)
+{
+ unsigned char orig_sha1[20];
+ int flag = 0, resolve_flags;
+ int log = reflog_exists(oldref);
+ const char *symref = NULL;
+ struct strbuf err = STRBUF_INIT;
+ struct ref_transaction *ref_transaction;
+
+ if (!strcmp(oldref, newref))
+ return 0;
+
+ lmdb_transaction_begin_flags_or_die(0);
+
+ ref_transaction = ref_transaction_begin(&err);
+ if (!ref_transaction)
+ die("%s", err.buf);
+
+ resolve_flags = RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE;
+ symref = resolve_ref_unsafe(oldref, resolve_flags,
+ orig_sha1, &flag);
+ if (flag & REF_ISSYMREF) {
+ error(_("refname %s is a symbolic ref, renaming it is not supported"),
+ oldref);
+ goto fail;
+ }
+ if (!symref) {
+ mdb_transaction_abort(&transaction);
+ error(_("refname %s not found"), oldref);
+ goto fail;
+ }
+ if (!rename_ref_available(oldref, newref))
+ goto fail;
+
+ /* Copy the reflog from the old to the new */
+ if (log) {
+ struct strbuf old_log_sentinel = STRBUF_INIT;
+ MDB_val key;
+ int log_all;
+
+ log_all = log_all_ref_updates;
+ log_all_ref_updates = 1;
+ if (safe_create_reflog(newref, 0, &err)) {
+ error(_("can't create reflog for %s: %s"), newref, err.buf);
+ strbuf_release(&err);
+ goto fail;
+ }
+ log_all_ref_updates = log_all;
+
+ for_each_reflog_ent(oldref, rename_reflog_ent, (void *)newref);
+ strbuf_addf(&old_log_sentinel, "logs/%sxxxxxxxx", oldref);
+ memset(old_log_sentinel.buf + old_log_sentinel.len - 8, 0, 8);
+
+ key.mv_size = old_log_sentinel.len;
+ key.mv_data = old_log_sentinel.buf;
+
+ /* It's OK if the old reflog is missing */
+ mdb_del_or_die(&transaction, &key, NULL);
+ strbuf_release(&old_log_sentinel);
+ }
+
+ if (ref_transaction_delete(ref_transaction, oldref,
+ orig_sha1, REF_NODEREF, NULL, &err)) {
+ error(_("unable to delete old %s"), oldref);
+ goto fail;
+ }
+
+ if (ref_transaction_update(ref_transaction, newref, orig_sha1, NULL,
+ REF_NODEREF, logmsg, &err)) {
+ error("%s", err.buf);
+ goto fail;
+ }
+
+ if (ref_transaction_commit(ref_transaction, &err)) {
+ error("%s", err.buf);
+ goto fail;
+ }
+
+ return 0;
+
+fail:
+ ref_transaction_free(ref_transaction);
+ strbuf_release(&err);
+ mdb_transaction_abort(&transaction);
+ transaction.txn = NULL;
+ return 1;
+}
+
+static int lmdb_delete_refs(struct string_list *refnames)
+{
+ int i;
+ struct strbuf err = STRBUF_INIT;
+ int result = 0;
+
+ if (!refnames->nr)
+ return 0;
+
+ lmdb_transaction_begin_flags_or_die(0);
+
+ for (i = 0; i < refnames->nr; i++) {
+ const char *refname = refnames->items[i].string;
+
+ if (lmdb_transaction_update(refname, null_sha1, NULL,
+ REF_DELETING, NULL, &err))
+ result |= error(_("could not remove reference %s: %s"),
+ refname, err.buf);
+ }
+
+ result |= mdb_transaction_commit(&transaction, &err);
+ strbuf_release(&err);
+ return 0;
+}
+
+static int lmdb_for_each_reflog_ent_order(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data, int reverse)
+{
+ MDB_val key, val;
+ char *search_key;
+ char *log_path;
+ int len;
+ MDB_cursor *cursor;
+ int ret = 0;
+ struct strbuf sb = STRBUF_INIT;
+ enum MDB_cursor_op direction = reverse ? MDB_PREV : MDB_NEXT;
+ uint64_t zero = 0ULL;
+
+ log_path = xstrfmt("logs/%s", refname);
+ len = strlen(log_path) + 1;
+
+ if (reverse) {
+ /*
+ * For a reverse search, start at the key
+ * lexicographically after the searched-for key.
+ * That's the one with \1 appended to the key.
+ */
+ search_key = xstrfmt("%s\1", log_path);
+ key.mv_size = len + 1;
+ } else {
+ search_key = xstrdup(log_path);
+ key.mv_size = len;
+ }
+
+ key.mv_data = search_key;
+
+ lmdb_transaction_begin_flags_or_die(MDB_RDONLY);
+
+ mdb_cursor_open_or_die(&transaction, &cursor);
+
+ transaction.cursor = cursor;
+
+ /*
+ * MDB's cursor API requires that the first mdb_cursor_get be
+ * called with MDB_SET_RANGE. For reverse searches, this will
+ * give us the entry one-past the entry we're looking for, so
+ * we should jump back using MDB_PREV.
+ */
+ mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);
+ if (direction == MDB_PREV)
+ mdb_cursor_get_or_die(cursor, &key, &val, direction);
+
+ do {
+ if (key.mv_size < len)
+ break;
+
+ if (!starts_with(key.mv_data, log_path) || ((char *)key.mv_data)[len - 1] != 0)
+ break;
+
+ if (!memcmp(&zero, ((char *)key.mv_data) + key.mv_size - 8, 8))
+ continue;
+
+ assert(val.mv_size != 0);
+
+ strbuf_add(&sb, val.mv_data, val.mv_size - 1);
+ ret = show_one_reflog_ent(&sb, fn, cb_data);
+ if (ret)
+ break;
+
+ strbuf_reset(&sb);
+ } while (!mdb_cursor_get_or_die(cursor, &key, &val, direction));
+
+ strbuf_release(&sb);
+ free(log_path);
+ free(search_key);
+ mdb_cursor_close(cursor);
+ return ret;
+}
+
+static int lmdb_for_each_reflog_ent(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ if (ref_type(refname) != REF_TYPE_NORMAL)
+ return refs_be_files.for_each_reflog_ent(refname, fn, cb_data);
+ return lmdb_for_each_reflog_ent_order(refname, fn, cb_data, 0);
+}
+
+static int lmdb_for_each_reflog_ent_reverse(const char *refname,
+ each_reflog_ent_fn fn,
+ void *cb_data)
+{
+ if (ref_type(refname) != REF_TYPE_NORMAL)
+ return refs_be_files.for_each_reflog_ent_reverse(refname, fn, cb_data);
+ return lmdb_for_each_reflog_ent_order(refname, fn, cb_data, 1);
+}
+
+static int lmdb_reflog_exists(const char *refname)
+{
+ MDB_val key, val;
+ char *log_path;
+ int len;
+ MDB_cursor *cursor;
+ int ret = 1;
+
+ if (ref_type(refname) != REF_TYPE_NORMAL)
+ return refs_be_files.reflog_exists(refname);
+
+ log_path = xstrfmt("logs/%s", refname);
+ len = strlen(log_path) + 1;
+
+ key.mv_data = log_path;
+ key.mv_size = len;
+
+ lmdb_transaction_begin_flags_or_die(MDB_RDONLY);
+ mdb_cursor_open_or_die(&transaction, &cursor);
+
+ if (mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE) ||
+ !starts_with(key.mv_data, log_path))
+ ret = 0;
+
+ free(log_path);
+ mdb_cursor_close(cursor);
+
+ return ret;
+}
+
+struct wrapped_each_ref_fn {
+ each_ref_fn *fn;
+ void *cb_data;
+};
+
+static int check_reflog(const char *refname,
+ const struct object_id *oid, int flags, void *cb_data)
+{
+ struct wrapped_each_ref_fn *wrapped = cb_data;
+
+ if (reflog_exists(refname))
+ return wrapped->fn(refname, oid, 0, wrapped->cb_data);
+
+ return 0;
+}
+
+static int lmdb_for_each_reflog(each_ref_fn fn, void *cb_data)
+{
+ struct wrapped_each_ref_fn wrapped = {fn, cb_data};
+ int result = head_ref(fn, cb_data);
+ if (result)
+ return result;
+ return for_each_ref(check_reflog, &wrapped);
+}
+
+static void strbuf_reflog_header(struct strbuf *sb, const char *refname)
+{
+ uint64_t zero = 0;
+
+ strbuf_addf(sb, "logs/%s", refname);
+ strbuf_add(sb, &zero, 8);
+}
+
+static int lmdb_create_reflog(const char *refname, int force_create, struct strbuf *err)
+{
+ /*
+ * We mark that there is a reflog by creating a key of the
+ * form logs/$refname followed by nine \0 (one for
+ * string-termination, 8 in lieu of a timestamp), with an empty
+ * value.
+ */
+
+ int in_transaction = in_write_transaction();
+ MDB_val key, val;
+ struct strbuf key_buf = STRBUF_INIT;
+
+ if (!force_create && !should_autocreate_reflog(refname))
+ return 0;
+
+ if (!in_transaction)
+ lmdb_transaction_begin_flags_or_die(0);
+
+ strbuf_reflog_header(&key_buf, refname);
+ key.mv_size = key_buf.len + 1;
+ key.mv_data = key_buf.buf;
+
+ val.mv_size = 0;
+ val.mv_data = NULL;
+ mdb_put_or_die(&transaction, &key, &val, 0);
+
+ strbuf_release(&key_buf);
+ if (!in_transaction)
+ return mdb_transaction_commit(&transaction, err);
+ return 0;
+}
+
+struct expire_reflog_cb {
+ unsigned int flags;
+ reflog_expiry_should_prune_fn *should_prune_fn;
+ void *policy_cb;
+ unsigned char last_kept_sha1[20];
+};
+
+static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
+ const char *email, unsigned long timestamp, int tz,
+ const char *message, void *cb_data)
+{
+ struct expire_reflog_cb *cb = cb_data;
+ struct expire_reflog_policy_cb *policy_cb = cb->policy_cb;
+
+ if (cb->flags & EXPIRE_REFLOGS_REWRITE)
+ osha1 = cb->last_kept_sha1;
+
+ if ((*cb->should_prune_fn)(osha1, nsha1, email, timestamp, tz,
+ message, policy_cb)) {
+ if (cb->flags & EXPIRE_REFLOGS_DRY_RUN)
+ printf("would prune %s", message);
+ else {
+ if (cb->flags & EXPIRE_REFLOGS_VERBOSE)
+ printf("prune %s", message);
+
+ mdb_cursor_del_or_die(transaction.cursor, 0);
+ }
+ } else {
+ hashcpy(cb->last_kept_sha1, nsha1);
+ if (cb->flags & EXPIRE_REFLOGS_VERBOSE)
+ printf("keep %s", message);
+ }
+ return 0;
+}
+
+static int write_ref(const char *refname, const unsigned char *sha1)
+{
+ struct strbuf err = STRBUF_INIT;
+ struct ref_transaction *transaction;
+
+ transaction = ref_transaction_begin(&err);
+ if (!transaction) {
+ error("%s", err.buf);
+ strbuf_release(&err);
+ return -1;
+ }
+
+ if (ref_transaction_update(transaction, refname, sha1, NULL,
+ REF_NO_REFLOG, NULL, &err)) {
+ error("%s", err.buf);
+ strbuf_release(&err);
+ return -1;
+ }
+
+ if (ref_transaction_commit(transaction, &err)) {
+ error("%s", err.buf);
+ strbuf_release(&err);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int lmdb_reflog_expire(const char *refname, const unsigned char *sha1,
+ unsigned int flags,
+ reflog_expiry_prepare_fn prepare_fn,
+ reflog_expiry_should_prune_fn should_prune_fn,
+ reflog_expiry_cleanup_fn cleanup_fn,
+ void *policy_cb_data)
+{
+ struct expire_reflog_cb cb;
+ int dry_run = flags & EXPIRE_REFLOGS_DRY_RUN;
+ int status = 0;
+ struct strbuf err = STRBUF_INIT;
+ unsigned char resolved_sha1[20];
+ int type;
+ char *resolved;
+
+ if (ref_type(refname) != REF_TYPE_NORMAL)
+ return refs_be_files.reflog_expire(refname, sha1, flags, prepare_fn,
+ should_prune_fn, cleanup_fn,
+ policy_cb_data);
+
+ memset(&cb, 0, sizeof(cb));
+ cb.flags = flags;
+ cb.policy_cb = policy_cb_data;
+ cb.should_prune_fn = should_prune_fn;
+
+ lmdb_transaction_begin_flags_or_die(dry_run ? MDB_RDONLY : 0);
+
+ resolved = check_ref(transaction.txn, refname, sha1,
+ resolved_sha1, 0, &type);
+ if (!resolved)
+ die(_("Failed to resolve %s"), refname);
+ free(resolved);
+
+ (*prepare_fn)(refname, sha1, cb.policy_cb);
+ lmdb_for_each_reflog_ent(refname, expire_reflog_ent, &cb);
+ (*cleanup_fn)(cb.policy_cb);
+
+ if (!dry_run) {
+ /*
+ * It doesn't make sense to adjust a reference pointed
+ * to by a symbolic ref based on expiring entries in
+ * the symbolic reference's reflog. Nor can we update
+ * a reference if there are no remaining reflog
+ * entries.
+ */
+ int update = (flags & EXPIRE_REFLOGS_UPDATE_REF) &&
+ !(type & REF_ISSYMREF) &&
+ !is_null_sha1(cb.last_kept_sha1);
+
+ if (mdb_transaction_commit(&transaction, &err)) {
+ status |= error(_("couldn't write logs/%s: %s"),
+ refname, err.buf);
+ strbuf_release(&err);
+ } else if (update && write_ref(refname, cb.last_kept_sha1)) {
+ status |= error(_("couldn't set %s"),
+ refname);
+ }
+ }
+ return status;
+}
+
+static int lmdb_pack_refs(unsigned int flags)
+{
+ /* This concept does not exist in this backend. */
+ return 0;
+}
+
+static int lmdb_peel_ref(const char *refname, unsigned char *sha1)
+{
+ int flag;
+ unsigned char base[20];
+
+ if (read_ref_full(refname, RESOLVE_REF_READING, base, &flag))
+ return -1;
+
+ return peel_object(base, sha1);
+}
+
+static int lmdb_create_symref(const char *ref_target,
+ const char *refs_heads_master,
+ const char *logmsg)
+{
+
+ struct strbuf err = STRBUF_INIT;
+ unsigned char old_sha1[20], new_sha1[20];
+ MDB_val key, val;
+ char *valdata;
+ int ret = 0;
+ int in_transaction;
+
+ in_transaction = in_write_transaction();
+
+ if (logmsg && read_ref(ref_target, old_sha1))
+ hashclr(old_sha1);
+
+ key.mv_size = strlen(ref_target) + 1;
+ key.mv_data = xstrdup(ref_target);
+
+ valdata = xstrfmt("ref: %s", refs_heads_master);
+ val.mv_data = valdata;
+ val.mv_size = strlen(valdata) + 1;
+
+ if (!in_transaction)
+ lmdb_transaction_begin_flags_or_die(0);
+
+ mdb_put_or_die(&transaction, &key, &val, 0);
+
+ /* TODO: Don't create ref d/f conflicts */
+
+ if (logmsg && !read_ref(refs_heads_master, new_sha1) &&
+ log_ref_write(ref_target, old_sha1, new_sha1, logmsg, 0, &err)) {
+ error(_("create_symref: log_ref_write failed: %s"), err.buf);
+ ret = -1;
+ goto done;
+ }
+
+ if (!in_transaction && mdb_transaction_commit(&transaction, &err)) {
+ error(_("create_symref: commit failed: %s"), err.buf);
+ ret = -1;
+ }
+
+done:
+ strbuf_release(&err);
+ free(key.mv_data);
+ free(valdata);
+
+ return ret;
+}
+
+static int lmdb_resolve_gitlink_ref(const char *submodule, const char *refname,
+ unsigned char *sha1)
+{
+ static struct strbuf sb_refname = STRBUF_INIT;
+ struct strbuf sb_path = STRBUF_INIT;
+ const char *ret;
+
+ ret = resolve_ref_unsafe_submodule(submodule, refname,
+ RESOLVE_REF_READING, sha1,
+ NULL, &sb_refname, &sb_path);
+ strbuf_release(&sb_path);
+ strbuf_release(&sb_refname);
+ return ret ? 0 : -1;
+}
+
+/*
+ * Call fn for each reference for which the refname begins with base.
+ * If trim is non-zero, then trim that many characters off the
+ * beginning of each refname before passing the refname to fn. flags
+ * can be DO_FOR_EACH_INCLUDE_BROKEN to include broken references in
+ * the iteration. If fn ever returns a non-zero value, stop the
+ * iteration and return that value; otherwise, return 0.
+ */
+static int lmdb_do_for_each_ref(const char *submodule, const char *base,
+ each_ref_fn fn, int trim, int flags,
+ void *cb_data)
+{
+
+ MDB_val key, val;
+ MDB_cursor *cursor;
+ int baselen;
+ char *search_key;
+ int retval;
+ int mdb_ret;
+ struct lmdb_transaction *transaction;
+
+ retval = do_for_each_per_worktree_ref(submodule, base, fn,
+ trim, flags, cb_data);
+ if (retval)
+ return retval;
+
+ if (ref_paranoia < 0)
+ ref_paranoia = git_env_bool("GIT_REF_PARANOIA", 0);
+ if (ref_paranoia)
+ flags |= DO_FOR_EACH_INCLUDE_BROKEN;
+
+ if (!base || !*base) {
+ base = "refs/";
+ trim = 0;
+ }
+
+ transaction = get_submodule_transaction(submodule);
+
+ search_key = xstrdup(base);
+ baselen = strlen(base);
+ key.mv_size = baselen + 1;
+ key.mv_data = search_key;
+
+ mdb_cursor_open_or_die(transaction, &cursor);
+
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);
+
+ while (!mdb_ret) {
+ struct object_id oid;
+ int parsed_flags = 0;
+ const char *refname;
+ struct strbuf sb_refname = STRBUF_INIT;
+ struct strbuf sb_path = STRBUF_INIT;
+
+ if (memcmp(key.mv_data, base, baselen))
+ break;
+
+ refname = (const char *)key.mv_data;
+ const char *c = resolve_ref_unsafe_submodule(submodule, refname, 0, oid.hash,
+ &parsed_flags, &sb_refname,
+ &sb_path);
+ if (!(parsed_flags & REF_ISBROKEN) && is_null_oid(&oid))
+ parsed_flags |= REF_ISBROKEN;
+
+ strbuf_release(&sb_path);
+ strbuf_release(&sb_refname);
+
+ if (flags & DO_FOR_EACH_INCLUDE_BROKEN ||
+ (!(parsed_flags & REF_ISBROKEN) &&
+ has_sha1_file(oid.hash))) {
+ retval = fn(refname + (trim ? baselen : 0), &oid, parsed_flags, cb_data);
+ if (retval)
+ break;
+ }
+
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);
+ }
+
+ mdb_cursor_close(cursor);
+ free(search_key);
+
+ if (submodule) {
+ MDB_env *env = mdb_txn_env(transaction->txn);
+ mdb_transaction_abort(transaction);
+ mdb_env_close(env);
+ }
+ return retval;
+}
+
+/* For testing only! */
+int test_refdb_raw_read(const char *key)
+{
+ MDB_val key_val, val;
+ char *keydup;
+ int ret;
+ int needs_free = 0;
+
+ lmdb_transaction_begin_flags_or_die(MDB_RDONLY);
+ keydup = xstrdup(key);
+ key_val.mv_data = keydup;
+ key_val.mv_size = strlen(key) + 1;
+
+ ret = mdb_get_or_die(&transaction, &key_val, &val, &needs_free);
+ free(keydup);
+ switch (ret) {
+ case 0:
+ printf("%s\n", (char *)val.mv_data);
+ return 0;
+ case MDB_NOTFOUND:
+ fprintf(stderr, "%s not found\n", key);
+ return 1;
+ default:
+ return 2;
+ }
+ if (needs_free)
+ free(val.mv_data);
+}
+
+/* For testing only! */
+void test_refdb_raw_write(const char *key, const char *value)
+{
+ MDB_val key_val, val;
+ char *keydup, *valdup;
+
+ if (ref_type(key) != REF_TYPE_NORMAL) {
+ val.mv_data = (void *)value;
+ val.mv_size = strlen(value) + 1;
+ write_per_worktree_ref(NULL, key, &val);
+ return;
+ }
+
+ lmdb_transaction_begin_flags_or_die(0);
+
+ keydup = xstrdup(key);
+ key_val.mv_data = keydup;
+ key_val.mv_size = strlen(key) + 1;
+
+ valdup = xstrdup(value);
+ val.mv_data = valdup;
+ val.mv_size = strlen(value) + 1;
+
+ mdb_put_or_die(&transaction, &key_val, &val, 0);
+ assert(mdb_transaction_commit(&transaction, NULL) == 0);
+
+ free(keydup);
+ free(valdup);
+}
+
+/* For testing only! */
+int test_refdb_raw_delete(const char *key)
+{
+ MDB_val key_val;
+ char *keydup;
+ int ret;
+
+ if (ref_type(key) != REF_TYPE_NORMAL)
+ return del_per_worktree_ref(NULL, key, NULL);
+
+ lmdb_transaction_begin_flags_or_die(0);
+ keydup = xstrdup(key);
+ key_val.mv_data = keydup;
+ key_val.mv_size = strlen(key) + 1;
+
+ ret = mdb_del_or_die(&transaction, &key_val, NULL);
+
+ assert(mdb_transaction_commit(&transaction, NULL) == 0);
+
+ free(keydup);
+ return ret;
+}
+
+static int print_raw_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
+ const char *email, unsigned long timestamp,
+ int tz, const char *message, void *cb_data)
+{
+ int *any = cb_data;
+ *any = 1;
+
+ if (*message != '\n')
+ printf("%s %s %s %lu %+05d\t%s", sha1_to_hex(osha1),
+ sha1_to_hex(nsha1),
+ email, timestamp, tz, message);
+ else
+ printf("%s %s %s %lu %+05d\n", sha1_to_hex(osha1),
+ sha1_to_hex(nsha1),
+ email, timestamp, tz);
+ return 0;
+}
+
+/* For testing only! */
+int test_refdb_raw_reflog(const char *refname)
+{
+ int any = 0;
+
+ for_each_reflog_ent(refname, print_raw_reflog_ent, &any);
+
+ return !any;
+}
+
+/* For testing only! */
+void test_refdb_raw_delete_reflog(char *refname)
+{
+ MDB_val key, val;
+ int mdb_ret;
+ char *search_key;
+ MDB_cursor *cursor;
+ int len;
+
+ search_key = xstrfmt("logs/%s", refname ? refname : "");
+ len = strlen(search_key) + 1;
+
+ key.mv_data = search_key;
+ key.mv_size = len;
+
+ lmdb_transaction_begin_flags_or_die(0);
+
+ mdb_cursor_open_or_die(&transaction, &cursor);
+
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_SET_RANGE);
+ while (!mdb_ret) {
+ if (!starts_with(key.mv_data, search_key))
+ break;
+ if (refname && ((char *)val.mv_data)[len - 1] == 0)
+ break;
+
+ mdb_cursor_del_or_die(cursor, 0);
+ mdb_ret = mdb_cursor_get_or_die(cursor, &key, &val, MDB_NEXT);
+ }
+
+ free(search_key);
+ mdb_cursor_close(cursor);
+
+ assert(mdb_transaction_commit(&transaction, NULL) == 0);
+ return;
+}
+
+static void format_lmdb_reflog_ent(struct strbuf *dst, struct strbuf *src)
+{
+ unsigned char osha1[20], nsha1[20];
+ const char *msg;
+
+ get_sha1_hex(src->buf, osha1);
+ get_sha1_hex(src->buf + 41, nsha1);
+
+ msg = strchr(src->buf + 82, '\t');
+ if (msg)
+ msg += 1;
+
+ format_reflog_entry(dst, osha1, nsha1, src->buf + 82, msg);
+}
+
+/* For testing only! */
+void test_refdb_raw_append_reflog(const char *refname)
+{
+ struct strbuf input = STRBUF_INIT;
+ struct strbuf sb = STRBUF_INIT;
+ uint64_t now = getnanotime();
+ MDB_val key, val;
+ struct strbuf key_buf = STRBUF_INIT;
+
+ strbuf_reflog_header(&key_buf, refname);
+ key.mv_size = key_buf.len + 1;
+ key.mv_data = key_buf.buf;
+
+ lmdb_transaction_begin_flags_or_die(0);
+
+ /* We do not remove the header entry here, because this is
+ * just for tests, so it's OK to be a bit inefficient */
+
+ while (strbuf_getline(&input, stdin) != EOF) {
+ /* "logs/" + \0 + 8-byte timestamp for sorting and expiry */
+ write_u64((char *)key.mv_data + key.mv_size - 8, htonll(now++));
+
+ /*
+ * Convert the input from files-reflog format to
+ * lmdb-reflog-format
+ */
+
+ format_lmdb_reflog_ent(&sb, &input);
+ val.mv_data = sb.buf;
+ val.mv_size = sb.len + 1;
+ mdb_put_or_die(&transaction, &key, &val, 0);
+ strbuf_reset(&sb);
+ input.len = 0;
+ }
+
+ strbuf_release(&input);
+ strbuf_release(&sb);
+ assert(mdb_transaction_commit(&transaction, NULL) == 0);
+ strbuf_release(&key_buf);
+}
+
+struct ref_storage_be refs_be_lmdb = {
+ NULL,
+ "lmdb",
+ lmdb_init_db,
+ lmdb_transaction_commit,
+ lmdb_transaction_commit, /* initial commit */
+
+ lmdb_for_each_reflog_ent,
+ lmdb_for_each_reflog_ent_reverse,
+ lmdb_for_each_reflog,
+ lmdb_reflog_exists,
+ lmdb_create_reflog,
+ lmdb_delete_reflog,
+ lmdb_reflog_expire,
+
+ lmdb_pack_refs,
+ lmdb_peel_ref,
+ lmdb_create_symref,
+ lmdb_delete_refs,
+ lmdb_rename_ref,
+
+ lmdb_read_raw_ref,
+ lmdb_verify_refname_available,
+ lmdb_resolve_gitlink_ref,
+
+ lmdb_do_for_each_ref,
+};
diff --git a/test-refs-lmdb-backend.c b/test-refs-lmdb-backend.c
new file mode 100644
index 0000000..5347178
--- /dev/null
+++ b/test-refs-lmdb-backend.c
@@ -0,0 +1,65 @@
+#include "cache.h"
+#include "string-list.h"
+#include "parse-options.h"
+#include "refs.h"
+#include "refs/refs-internal.h"
+
+static const char * const test_refs_be_lmdb_usage[] = {
+ "git test-refs-lmdb-backend <key>",
+ "git test-refs-lmdb-backend <key> <value>",
+ NULL,
+};
+
+int test_refdb_raw_read(const char *key);
+void test_refdb_raw_write(const char *key, const char *value);
+int test_refdb_raw_reflog(const char *refname);
+int test_refdb_raw_delete(const char *key);
+void test_refdb_raw_delete_reflog(const char *refname);
+void test_refdb_raw_append_reflog(const char *refname);
+
+int main(int argc, const char **argv)
+{
+ const char *delete = NULL;
+ const char *reflog = NULL;
+ const char *append_reflog = NULL;
+ int delete_missing_error = 0;
+ int clear_reflog = 0;
+
+ struct option options[] = {
+ OPT_STRING('d', NULL, &delete, "branch", "delete refdb entry"),
+ OPT_STRING('l', NULL, &reflog, "branch", "show reflog"),
+ OPT_STRING('a', NULL, &append_reflog, "branch", "append to reflog"),
+ OPT_BOOL('c', NULL, &clear_reflog, "delete reflog. If a branch is provided, the reflog for that branch will be deleted; else all reflogs will be deleted."),
+ OPT_BOOL('x', NULL, &delete_missing_error,
+ "deleting a missing key is an error"),
+ OPT_END(),
+ };
+
+ argc = parse_options(argc, argv, "", options, test_refs_be_lmdb_usage,
+ 0);
+
+ if (!append_reflog && !clear_reflog && !delete && !reflog && argc != 1 && argc != 2)
+ usage_with_options(test_refs_be_lmdb_usage,
+ options);
+
+ git_config(git_default_config, NULL);
+
+ if (set_ref_storage_backend("lmdb"))
+ die("could not set lmdb reference backend");
+
+ if (clear_reflog) {
+ test_refdb_raw_delete_reflog(argv[0]);
+ } else if (append_reflog) {
+ test_refdb_raw_append_reflog(append_reflog);
+ } else if (reflog) {
+ return test_refdb_raw_reflog(reflog);
+ } else if (delete) {
+ if (test_refdb_raw_delete(delete) && delete_missing_error)
+ return 1;
+ } else if (argc == 1) {
+ return test_refdb_raw_read(argv[0]);
+ } else {
+ test_refdb_raw_write(argv[0], argv[1]);
+ }
+ return 0;
+}
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 31/32] refs: tests for lmdb backend
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (29 preceding siblings ...)
2016-02-24 22:59 ` [PATCH v6 30/32] refs: add LMDB refs storage backend David Turner
@ 2016-02-24 22:59 ` David Turner
2016-02-24 22:59 ` [PATCH v6 32/32] tests: add ref-storage argument David Turner
` (2 subsequent siblings)
33 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-24 22:59 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Add tests for the database backend.
Signed-off-by: David Turner <dturner@twopensource.com>
Helped-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
---
t/t1460-refs-lmdb-backend.sh | 1109 +++++++++++++++++++++++++++++++++++
t/t1470-refs-lmdb-backend-reflog.sh | 359 ++++++++++++
t/t1480-refs-lmdb-submodule.sh | 85 +++
t/test-lib.sh | 1 +
4 files changed, 1554 insertions(+)
create mode 100755 t/t1460-refs-lmdb-backend.sh
create mode 100755 t/t1470-refs-lmdb-backend-reflog.sh
create mode 100755 t/t1480-refs-lmdb-submodule.sh
diff --git a/t/t1460-refs-lmdb-backend.sh b/t/t1460-refs-lmdb-backend.sh
new file mode 100755
index 0000000..31442e7
--- /dev/null
+++ b/t/t1460-refs-lmdb-backend.sh
@@ -0,0 +1,1109 @@
+#!/bin/sh
+#
+# Copyright (c) 2015 Twitter, Inc
+# Copyright (c) 2006 Shawn Pearce
+# This test is based on t1400-update-ref.sh
+#
+
+test_description='Test lmdb refs backend'
+TEST_NO_CREATE_REPO=1
+. ./test-lib.sh
+
+if ! test_have_prereq LMDB
+then
+ skip_all="Skipping lmdb refs backend tests, lmdb backend not built"
+ test_done
+fi
+
+raw_ref() {
+ test-refs-lmdb-backend "$1"
+}
+
+delete_ref() {
+ test-refs-lmdb-backend -d "$1"
+}
+
+write_ref() {
+ test-refs-lmdb-backend "$1" "$2"
+}
+
+raw_reflog() {
+ test-refs-lmdb-backend -l "$1"
+}
+
+delete_all_reflogs() {
+ test-refs-lmdb-backend -c
+}
+
+append_reflog() {
+ test-refs-lmdb-backend -a "$1"
+}
+
+Z=$_z40
+
+test_expect_success setup '
+ git init --ref-storage=lmdb &&
+ for name in A B C D E F
+ do
+ test_tick &&
+ T=$(git write-tree) &&
+ sha1=$(echo $name | git commit-tree $T) &&
+ eval $name=$sha1
+ done
+'
+
+m=refs/heads/master
+n_dir=refs/heads/gu
+n=$n_dir/fixes
+
+test_expect_success \
+ "create $m" \
+ "git update-ref $m $A &&
+ test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "create $m" \
+ "git update-ref $m $B $A &&
+ test $B"' = $(raw_ref '"$m"')'
+test_expect_success "fail to delete $m with stale ref" '
+ test_must_fail git update-ref -d $m $A &&
+ test $B = "$(raw_ref $m)"
+'
+test_expect_success "delete $m" '
+ git update-ref -d $m $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+test_expect_success "delete $m without oldvalue verification" "
+ git update-ref $m $A &&
+ test $A = \$(raw_ref $m) &&
+ git update-ref -d $m &&
+ ! raw_ref $m
+"
+delete_ref $m
+
+test_expect_success \
+ "fail to create $n" \
+ "git update-ref $n_dir $A &&
+ test_must_fail git update-ref $n $A >out 2>err"
+
+delete_ref $n_dir
+rm -f out err
+
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $A &&
+ test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $B $A &&
+ test $B"' = $(raw_ref '"$m"')'
+test_expect_success "fail to delete $m (by HEAD) with stale ref" '
+ test_must_fail git update-ref -d HEAD $A &&
+ test $B = $(raw_ref '"$m"')
+'
+test_expect_success "delete $m (by HEAD)" '
+ git update-ref -d HEAD $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+test_expect_success \
+ "create $m (by HEAD)" \
+ "git update-ref HEAD $A &&
+ test $A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "pack refs" \
+ "git pack-refs --all"
+test_expect_success \
+ "move $m (by HEAD)" \
+ "git update-ref HEAD $B $A &&
+ test $B"' = $(raw_ref '"$m"')'
+test_expect_success "delete $m (by HEAD) should remove both packed and loose $m" '
+ git update-ref -d HEAD $B &&
+ ! raw_ref $m
+'
+delete_ref $m
+
+OLD_HEAD=$(raw_ref HEAD)
+test_expect_success "delete symref without dereference" '
+ git update-ref --no-deref -d HEAD &&
+ ! raw_ref HEAD
+'
+write_ref HEAD "$OLD_HEAD"
+
+test_expect_success "delete symref without dereference when the referred ref is packed" '
+ echo foo >foo.c &&
+ git add foo.c &&
+ git commit -m foo &&
+ git pack-refs --all &&
+ git update-ref --no-deref -d HEAD &&
+ ! raw_ref HEAD
+'
+write_ref HEAD "$OLD_HEAD"
+delete_ref $m
+
+test_expect_success 'update-ref -d is not confused by self-reference' '
+ git symbolic-ref refs/heads/self refs/heads/self &&
+ test_when_finished "delete_ref refs/heads/self" &&
+ test_must_fail git update-ref -d refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+ git symbolic-ref refs/heads/self refs/heads/self &&
+ test_when_finished "delete_ref refs/heads/self" &&
+ git update-ref --no-deref -d refs/heads/self
+'
+
+test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+ test-refs-lmdb-backend refs/heads/bad "" &&
+ test_when_finished "delete_ref refs/heads/bad" &&
+ git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
+ test_when_finished "delete_ref refs/heads/ref-to-bad" &&
+ raw_ref refs/heads/ref-to-bad &&
+ git update-ref --no-deref -d refs/heads/ref-to-bad &&
+ ! raw_ref refs/heads/ref-to-bad
+'
+
+test_expect_success '(not) create HEAD with old sha1' "
+ test_must_fail git update-ref HEAD $A $B
+"
+test_expect_success "(not) prior created .git/$m" "
+ ! raw_ref $m
+"
+delete_ref $m
+
+test_expect_success \
+ "create HEAD" \
+ "git update-ref HEAD $A"
+test_expect_success '(not) change HEAD with wrong SHA1' "
+ test_must_fail git update-ref HEAD $B $Z
+"
+test_expect_success "(not) changed .git/$m" "
+ ! test $B"' = $(raw_ref '"$m"')
+'
+
+: a repository with working tree always has reflog these days...
+delete_all_reflogs
+: | append_reflog $m
+delete_ref $m
+
+test_expect_success \
+ "create $m (logged by touch)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:30" \
+ git update-ref HEAD '"$A"' -m "Initial Creation" &&
+ test '"$A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "update $m (logged by touch)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:31" \
+ git update-ref HEAD'" $B $A "'-m "Switch" &&
+ test '"$B"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "set $m (logged by touch)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:41" \
+ git update-ref HEAD'" $A &&
+ test $A"' = $(raw_ref '"$m"')'
+
+cat >expect <<EOF
+$Z $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000 Initial Creation
+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150260 +0000 Switch
+$B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000
+EOF
+test_expect_success \
+ "verifying $m's log" \
+ "raw_reflog $m >actual &&
+ test_cmp expect actual"
+delete_ref $m
+delete_all_reflogs
+: | append_reflog $m
+rm -f actual expect
+
+test_expect_success \
+ 'enable core.logAllRefUpdates' \
+ 'git config core.logAllRefUpdates true &&
+ test true = $(git config --bool --get core.logAllRefUpdates)'
+
+test_expect_success \
+ "create $m (logged by config)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:32" \
+ git update-ref HEAD'" $A "'-m "Initial Creation" &&
+ test '"$A"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "update $m (logged by config)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:33" \
+ git update-ref HEAD'" $B $A "'-m "Switch" &&
+ test '"$B"' = $(raw_ref '"$m"')'
+test_expect_success \
+ "set $m (logged by config)" \
+ 'GIT_COMMITTER_DATE="2005-05-26 23:43" \
+ git update-ref HEAD '"$A &&
+ test $A"' = $(raw_ref '"$m"')'
+
+cat >expect <<EOF
+$Z $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150320 +0000 Initial Creation
+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 +0000 Switch
+$B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 +0000
+EOF
+test_expect_success \
+ "verifying $m's log" \
+ 'raw_reflog $m >actual &&
+ test_cmp expect actual'
+delete_ref $m
+rm -f expect
+
+git update-ref $m $D
+git reflog expire --expire=all $m
+
+append_reflog $m <<EOF
+0000000000000000000000000000000000000000 $C $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150320 -0500
+$C $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150350 -0500
+$A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 -0500
+$F $Z $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150680 -0500
+$Z $E $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 -0500
+EOF
+
+ed="Thu, 26 May 2005 18:32:00 -0500"
+gd="Thu, 26 May 2005 18:33:00 -0500"
+ld="Thu, 26 May 2005 18:43:00 -0500"
+test_expect_success \
+ 'Query "master@{May 25 2005}" (before history)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{May 25 2005}" >o 2>e &&
+ test '"$C"' = $(cat o) &&
+ test "warning: Log for '\'master\'' only goes back to $ed." = "$(cat e)"'
+test_expect_success \
+ "Query master@{2005-05-25} (before history)" \
+ 'rm -f o e &&
+ git rev-parse --verify master@{2005-05-25} >o 2>e &&
+ test '"$C"' = $(cat o) &&
+ echo test "warning: Log for '\'master\'' only goes back to $ed." = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{May 26 2005 23:31:59}" (1 second before history)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{May 26 2005 23:31:59}" >o 2>e &&
+ test '"$C"' = $(cat o) &&
+ test "warning: Log for '\''master'\'' only goes back to $ed." = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{May 26 2005 23:32:00}" (exactly history start)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{May 26 2005 23:32:00}" >o 2>e &&
+ test '"$C"' = $(cat o) &&
+ test "" = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{May 26 2005 23:32:30}" (first non-creation change)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{May 26 2005 23:32:30}" >o 2>e &&
+ test '"$A"' = $(cat o) &&
+ test "" = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{2005-05-26 23:33:01}" (middle of history with gap)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{2005-05-26 23:33:01}" >o 2>e &&
+ test '"$B"' = $(cat o) &&
+ test "warning: Log for ref '"$m has gap after $gd"'." = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{2005-05-26 23:38:00}" (middle of history)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{2005-05-26 23:38:00}" >o 2>e &&
+ test '"$Z"' = $(cat o) &&
+ test "" = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{2005-05-26 23:43:00}" (exact end of history)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{2005-05-26 23:43:00}" >o 2>e &&
+ test '"$E"' = $(cat o) &&
+ test "" = "$(cat e)"'
+test_expect_success \
+ 'Query "master@{2005-05-28}" (past end of history)' \
+ 'rm -f o e &&
+ git rev-parse --verify "master@{2005-05-28}" >o 2>e &&
+ test '"$D"' = $(cat o) &&
+ test "warning: Log for ref '"$m unexpectedly ended on $ld"'." = "$(cat e)"'
+
+
+git reflog expire --expire=all $m
+delete_ref $m
+
+test_expect_success \
+ 'creating initial files' \
+ 'echo TEST >F &&
+ git add F &&
+ GIT_AUTHOR_DATE="2005-05-26 23:30" \
+ GIT_COMMITTER_DATE="2005-05-26 23:30" git commit -m add -a &&
+ h_TEST=$(git rev-parse --verify HEAD) &&
+ echo The other day this did not work. >M &&
+ echo And then Bob told me how to fix it. >>M &&
+ echo OTHER >F &&
+ GIT_AUTHOR_DATE="2005-05-26 23:41" \
+ GIT_COMMITTER_DATE="2005-05-26 23:41" git commit -F M -a &&
+ h_OTHER=$(git rev-parse --verify HEAD) &&
+ GIT_AUTHOR_DATE="2005-05-26 23:44" \
+ GIT_COMMITTER_DATE="2005-05-26 23:44" git commit --amend &&
+ h_FIXED=$(git rev-parse --verify HEAD) &&
+ echo Merged initial commit and a later commit. >M &&
+ echo $h_TEST >.git/MERGE_HEAD &&
+ GIT_AUTHOR_DATE="2005-05-26 23:45" \
+ GIT_COMMITTER_DATE="2005-05-26 23:45" git commit -F M &&
+ h_MERGED=$(git rev-parse --verify HEAD) &&
+ rm -f M'
+
+cat >expect <<EOF
+$Z $h_TEST $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000 commit (initial): add
+$h_TEST $h_OTHER $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000 commit: The other day this did not work.
+$h_OTHER $h_FIXED $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117151040 +0000 commit (amend): The other day this did not work.
+$h_FIXED $h_MERGED $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117151100 +0000 commit (merge): Merged initial commit and a later commit.
+EOF
+test_expect_success \
+ 'git commit logged updates' \
+ "raw_reflog $m >actual &&
+ test_cmp expect actual"
+unset h_TEST h_OTHER h_FIXED h_MERGED
+
+test_expect_success \
+ 'git cat-file blob master:F (expect OTHER)' \
+ 'test OTHER = $(git cat-file blob master:F)'
+test_expect_success \
+ 'git cat-file blob master@{2005-05-26 23:30}:F (expect TEST)' \
+ 'test TEST = $(git cat-file blob "master@{2005-05-26 23:30}:F")'
+test_expect_success \
+ 'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
+ 'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
+
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+E='""'
+F='%s\0'
+pws='path with space'
+
+test_expect_success 'stdin test setup' '
+ echo "$pws" >"$pws" &&
+ git add -- "$pws" &&
+ git commit -m "$pws"
+'
+
+test_expect_success '-z fails without --stdin' '
+ test_must_fail git update-ref -z $m $m $m 2>err &&
+ grep "usage: git update-ref" err
+'
+
+test_expect_success 'stdin works with no input' '
+ >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on empty line' '
+ echo "" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin fails on only whitespace' '
+ echo " " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin fails on leading whitespace' '
+ echo " create $a $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: whitespace before command: create $a $m" err
+'
+
+test_expect_success 'stdin fails on unknown command' '
+ echo "unknown $a" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin fails on unbalanced quotes' '
+ echo "create $a \"master" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on invalid escape' '
+ echo "create $a \"ma\zter\"" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: badly quoted argument: \\\"ma\\\\zter\\\"" err
+'
+
+test_expect_success 'stdin fails on junk after quoted argument' '
+ echo "create \"$a\"master" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: unexpected character after quoted argument: \\\"$a\\\"master" err
+'
+
+test_expect_success 'stdin fails create with no ref' '
+ echo "create " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: create: missing <ref>" err
+'
+
+test_expect_success 'stdin fails create with no new value' '
+ echo "create $a" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: create $a: missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails create with too many arguments' '
+ echo "create $a $m $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: create $a: extra input: $m" err
+'
+
+test_expect_success 'stdin fails update with no ref' '
+ echo "update " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: update: missing <ref>" err
+'
+
+test_expect_success 'stdin fails update with no new value' '
+ echo "update $a" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: update $a: missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails update with too many arguments' '
+ echo "update $a $m $m $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: update $a: extra input: $m" err
+'
+
+test_expect_success 'stdin fails delete with no ref' '
+ echo "delete " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: delete: missing <ref>" err
+'
+
+test_expect_success 'stdin fails delete with too many arguments' '
+ echo "delete $a $m $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: delete $a: extra input: $m" err
+'
+
+test_expect_success 'stdin fails verify with too many arguments' '
+ echo "verify $a $m $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: verify $a: extra input: $m" err
+'
+
+test_expect_success 'stdin fails option with unknown name' '
+ echo "option unknown" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+ cat >stdin <<-EOF &&
+ create $a $m
+ create $b $m
+ create $a $m
+ EOF
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works' '
+ echo "create $a $m" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin succeeds with quoted argument' '
+ git update-ref -d $a &&
+ echo "create $a \"$m\"" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin succeeds with escaped character' '
+ git update-ref -d $a &&
+ echo "create $a \"ma\\163ter\"" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref creates with zero old value' '
+ echo "update $b $m $Z" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ git update-ref -d $b
+'
+
+test_expect_success 'stdin update ref creates with empty old value' '
+ echo "update $b $m $E" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+ echo "create refs/blobs/pws \"$m:$pws\"" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse "$m:$pws" >expect &&
+ git rev-parse refs/blobs/pws >actual &&
+ test_cmp expect actual &&
+ git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+ echo "update $c $m $m~1" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$c'"'"'" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref fails with bad old value' '
+ echo "update $c $m does-not-exist" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: update $c: invalid <oldvalue>: does-not-exist" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+ echo "create $c does-not-exist" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: create $c: invalid <newvalue>: does-not-exist" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with zero new value' '
+ echo "create $c " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: create $c: zero <newvalue>" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+ echo "update $b $m~1 $m" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+ echo "delete $a $m~1" >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$a'"'"'" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with zero old value' '
+ echo "delete $a " >stdin &&
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: delete $a: zero <oldvalue>" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works option no-deref' '
+ git symbolic-ref refs/TESTSYMREF $b &&
+ cat >stdin <<-EOF &&
+ option no-deref
+ update refs/TESTSYMREF $a $b
+ EOF
+ git update-ref --stdin <stdin &&
+ git rev-parse refs/TESTSYMREF >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works option no-deref' '
+ git symbolic-ref refs/TESTSYMREF $b &&
+ cat >stdin <<-EOF &&
+ option no-deref
+ delete refs/TESTSYMREF $b
+ EOF
+ git update-ref --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/TESTSYMREF &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+ echo "delete $b $m~1" >stdin &&
+ git update-ref --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin update/create/verify combination works' '
+ cat >stdin <<-EOF &&
+ update $a $m
+ create $b $m
+ verify $c
+ EOF
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin verify succeeds for correct value' '
+ git rev-parse $m >expect &&
+ echo "verify $m $m" >stdin &&
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin verify succeeds for missing reference' '
+ echo "verify refs/heads/missing $Z" >stdin &&
+ git update-ref --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/heads/missing
+'
+
+test_expect_success 'stdin verify treats no value as missing' '
+ echo "verify refs/heads/missing" >stdin &&
+ git update-ref --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/heads/missing
+'
+
+test_expect_success 'stdin verify fails for wrong value' '
+ git rev-parse $m >expect &&
+ echo "verify $m $m~1" >stdin &&
+ test_must_fail git update-ref --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin verify fails for mistaken null value' '
+ git rev-parse $m >expect &&
+ echo "verify $m $Z" >stdin &&
+ test_must_fail git update-ref --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin verify fails for mistaken empty value' '
+ M=$(git rev-parse $m) &&
+ test_when_finished "git update-ref $m $M" &&
+ git rev-parse $m >expect &&
+ echo "verify $m" >stdin &&
+ test_must_fail git update-ref --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+ cat >stdin <<-EOF &&
+ update $a $m $m
+ update $b $m $m
+ update $c $Z $E
+ EOF
+ git update-ref --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+ git update-ref $c $m &&
+ cat >stdin <<-EOF &&
+ update $a $m $m
+ update $b $m $m
+ update $c ''
+ EOF
+ test_must_fail git update-ref --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$c'"'"'" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ git rev-parse $c >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+ git pack-refs --all &&
+ git update-ref $c $m~1 &&
+ cat >stdin <<-EOF &&
+ delete $a $m
+ update $b $Z $m
+ update $c $E $m~1
+ EOF
+ git update-ref --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q $a &&
+ test_must_fail git rev-parse --verify -q $b &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z works on empty input' '
+ >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z fails on empty line' '
+ echo "" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin -z fails on empty command' '
+ printf $F "" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin -z fails on only whitespace' '
+ printf $F " " >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin -z fails on leading whitespace' '
+ printf $F " create $a" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: whitespace before command: create $a" err
+'
+
+test_expect_success 'stdin -z fails on unknown command' '
+ printf $F "unknown $a" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin -z fails create with no ref' '
+ printf $F "create " >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: create: missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails create with no new value' '
+ printf $F "create $a" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: create $a: unexpected end of input when reading <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails create with too many arguments' '
+ printf $F "create $a" "$m" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails update with no ref' '
+ printf $F "update " >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: update: missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails update with too few args' '
+ printf $F "update $a" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: update $a: unexpected end of input when reading <oldvalue>" err
+'
+
+test_expect_success 'stdin -z emits warning with empty new value' '
+ git update-ref $a $m &&
+ printf $F "update $a" "" "" >stdin &&
+ git update-ref -z --stdin <stdin 2>err &&
+ grep "warning: update $a: missing <newvalue>, treating as zero" err &&
+ test_must_fail git rev-parse --verify -q $a
+'
+
+test_expect_success 'stdin -z fails update with no new value' '
+ printf $F "update $a" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: update $a: unexpected end of input when reading <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails update with no old value' '
+ printf $F "update $a" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: update $a: unexpected end of input when reading <oldvalue>" err
+'
+
+test_expect_success 'stdin -z fails update with too many arguments' '
+ printf $F "update $m" "$m" "$m" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails delete with no ref' '
+ printf $F "delete " >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: delete: missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails delete with no old value' '
+ printf $F "delete $a" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: delete $a: unexpected end of input when reading <oldvalue>" err
+'
+
+test_expect_success 'stdin -z fails delete with too many arguments' '
+ printf $F "delete $m" "$m" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with too many arguments' '
+ printf $F "verify $m" "$m" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with no old value' '
+ printf $F "verify $a" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: verify $a: unexpected end of input when reading <oldvalue>" err
+'
+
+test_expect_success 'stdin -z fails option with unknown name' '
+ printf $F "option unknown" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin -z fails with duplicate refs' '
+ printf $F "create $a" "$m" "create $b" "$m" "create $a" "$m" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin -z create ref works' '
+ printf $F "create $a" "$m" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update ref creates with zero old value' '
+ printf $F "update $b" "$m" "$Z" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ git update-ref -d $b
+'
+
+test_expect_success 'stdin -z update ref creates with empty old value' '
+ printf $F "update $b" "$m" "" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with path with space to blob' '
+ printf $F "create refs/blobs/pws" "$m:$pws" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse "$m:$pws" >expect &&
+ git rev-parse refs/blobs/pws >actual &&
+ test_cmp expect actual &&
+ git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin -z update ref fails with wrong old value' '
+ printf $F "update $c" "$m" "$m~1" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$c'"'"'" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref fails with bad old value' '
+ printf $F "update $c" "$m" "does-not-exist" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: update $c: invalid <oldvalue>: does-not-exist" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails when ref exists' '
+ git update-ref $c $m &&
+ git rev-parse "$c" >expect &&
+ printf $F "create $c" "$m~1" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$c'"'"'" err &&
+ git rev-parse "$c" >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref fails with bad new value' '
+ git update-ref -d "$c" &&
+ printf $F "create $c" "does-not-exist" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: create $c: invalid <newvalue>: does-not-exist" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with empty new value' '
+ printf $F "create $c" "" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: create $c: missing <newvalue>" err &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref works with right old value' '
+ printf $F "update $b" "$m~1" "$m" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with wrong old value' '
+ printf $F "delete $a" "$m~1" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$a'"'"'" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with zero old value' '
+ printf $F "delete $a" "$Z" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: delete $a: zero <oldvalue>" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update symref works option no-deref' '
+ git symbolic-ref refs/TESTSYMREF $b &&
+ printf $F "option no-deref" "update refs/TESTSYMREF" "$a" "$b" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse refs/TESTSYMREF >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete symref works option no-deref' '
+ git symbolic-ref refs/TESTSYMREF $b &&
+ printf $F "option no-deref" "delete refs/TESTSYMREF" "$b" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/TESTSYMREF &&
+ git rev-parse $m~1 >expect &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref works with right old value' '
+ printf $F "delete $b" "$m~1" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin -z update/create/verify combination works' '
+ printf $F "update $a" "$m" "" "create $b" "$m" "verify $c" "" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z verify succeeds for correct value' '
+ git rev-parse $m >expect &&
+ printf $F "verify $m" "$m" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z verify succeeds for missing reference' '
+ printf $F "verify refs/heads/missing" "$Z" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/heads/missing
+'
+
+test_expect_success 'stdin -z verify treats no value as missing' '
+ printf $F "verify refs/heads/missing" "" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q refs/heads/missing
+'
+
+test_expect_success 'stdin -z verify fails for wrong value' '
+ git rev-parse $m >expect &&
+ printf $F "verify $m" "$m~1" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z verify fails for mistaken null value' '
+ git rev-parse $m >expect &&
+ printf $F "verify $m" "$Z" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z verify fails for mistaken empty value' '
+ M=$(git rev-parse $m) &&
+ test_when_finished "git update-ref $m $M" &&
+ git rev-parse $m >expect &&
+ printf $F "verify $m" "" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin &&
+ git rev-parse $m >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update refs works with identity updates' '
+ printf $F "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "$Z" "" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs fails with wrong old value' '
+ git update-ref $c $m &&
+ printf $F "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "$m" "$Z" >stdin &&
+ test_must_fail git update-ref -z --stdin <stdin 2>err &&
+ grep "fatal: cannot lock the ref '"'"'$c'"'"'" err &&
+ git rev-parse $m >expect &&
+ git rev-parse $a >actual &&
+ test_cmp expect actual &&
+ git rev-parse $b >actual &&
+ test_cmp expect actual &&
+ git rev-parse $c >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete refs works with packed and loose refs' '
+ git pack-refs --all &&
+ git update-ref $c $m~1 &&
+ printf $F "delete $a" "$m" "update $b" "$Z" "$m" "update $c" "" "$m~1" >stdin &&
+ git update-ref -z --stdin <stdin &&
+ test_must_fail git rev-parse --verify -q $a &&
+ test_must_fail git rev-parse --verify -q $b &&
+ test_must_fail git rev-parse --verify -q $c
+'
+
+test_done
diff --git a/t/t1470-refs-lmdb-backend-reflog.sh b/t/t1470-refs-lmdb-backend-reflog.sh
new file mode 100755
index 0000000..83bfba0
--- /dev/null
+++ b/t/t1470-refs-lmdb-backend-reflog.sh
@@ -0,0 +1,359 @@
+#!/bin/sh
+#
+# Copyright (c) 2015 Twitter, Inc
+# Copyright (c) 2007 Junio C Hamano
+#
+
+test_description='Test prune and reflog expiration'
+TEST_NO_CREATE_REPO=1
+. ./test-lib.sh
+
+if ! test_have_prereq LMDB
+then
+ skip_all="Skipping lmdb refs backend tests, lmdb backend not built"
+ test_done
+fi
+
+raw_reflog() {
+ cat .git/logs/$1 2>/dev/null || test-refs-lmdb-backend -l "$1"
+}
+
+append_reflog() {
+ test-refs-lmdb-backend -a "$1"
+}
+
+check_have () {
+ gaah= &&
+ for N in "$@"
+ do
+ eval "o=\$$N" && git cat-file -t $o || {
+ echo Gaah $N
+ gaah=$N
+ break
+ }
+ done &&
+ test -z "$gaah"
+}
+
+check_fsck () {
+ output=$(git fsck --full)
+ case "$1" in
+ '')
+ test -z "$output" ;;
+ *)
+ echo "$output" | grep "$1" ;;
+ esac
+}
+
+corrupt () {
+ aa=${1%??????????????????????????????????????} zz=${1#??}
+ mv .git/objects/$aa/$zz .git/$aa$zz
+}
+
+recover () {
+ aa=${1%??????????????????????????????????????} zz=${1#??}
+ mkdir -p .git/objects/$aa
+ mv .git/$aa$zz .git/objects/$aa/$zz
+}
+
+check_dont_have () {
+ gaah= &&
+ for N in "$@"
+ do
+ eval "o=\$$N"
+ git cat-file -t $o && {
+ echo Gaah $N
+ gaah=$N
+ break
+ }
+ done
+ test -z "$gaah"
+}
+
+test_expect_success setup '
+ git init --ref-storage=lmdb &&
+ mkdir -p A/B &&
+ echo rat >C &&
+ echo ox >A/D &&
+ echo tiger >A/B/E &&
+ git add . &&
+
+ test_tick && git commit -m rabbit &&
+ H=`git rev-parse --verify HEAD` &&
+ A=`git rev-parse --verify HEAD:A` &&
+ B=`git rev-parse --verify HEAD:A/B` &&
+ C=`git rev-parse --verify HEAD:C` &&
+ D=`git rev-parse --verify HEAD:A/D` &&
+ E=`git rev-parse --verify HEAD:A/B/E` &&
+ check_fsck &&
+
+ test_chmod +x C &&
+ git add C &&
+ test_tick && git commit -m dragon &&
+ L=`git rev-parse --verify HEAD` &&
+ check_fsck &&
+
+ rm -f C A/B/E &&
+ echo snake >F &&
+ echo horse >A/G &&
+ git add F A/G &&
+ test_tick && git commit -a -m sheep &&
+ F=`git rev-parse --verify HEAD:F` &&
+ G=`git rev-parse --verify HEAD:A/G` &&
+ I=`git rev-parse --verify HEAD:A` &&
+ J=`git rev-parse --verify HEAD` &&
+ check_fsck &&
+
+ rm -f A/G &&
+ test_tick && git commit -a -m monkey &&
+ K=`git rev-parse --verify HEAD` &&
+ check_fsck &&
+
+ check_have A B C D E F G H I J K L &&
+
+ git prune &&
+
+ check_have A B C D E F G H I J K L &&
+
+ check_fsck &&
+
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 4 reflog
+'
+
+test_expect_success rewind '
+ test_tick && git reset --hard HEAD~2 &&
+ test -f C &&
+ test -f A/B/E &&
+ ! test -f F &&
+ ! test -f A/G &&
+
+ check_have A B C D E F G H I J K L &&
+
+ git prune &&
+
+ check_have A B C D E F G H I J K L &&
+
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 5 reflog
+'
+
+test_expect_success 'corrupt and check' '
+
+ corrupt $F &&
+ check_fsck "missing blob $F"
+
+'
+
+test_expect_success 'reflog expire --dry-run should not touch reflog' '
+
+ git reflog expire --dry-run \
+ --expire=$(($test_tick - 10000)) \
+ --expire-unreachable=$(($test_tick - 10000)) \
+ --stale-fix \
+ --all &&
+
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 5 reflog &&
+
+ check_fsck "missing blob $F"
+'
+
+test_expect_success 'reflog expire' '
+
+ git reflog expire --verbose \
+ --expire=$(($test_tick - 10000)) \
+ --expire-unreachable=$(($test_tick - 10000)) \
+ --stale-fix \
+ --all &&
+
+ echo git reflog expire --verbose \
+ --expire=$(($test_tick - 10000)) \
+ --expire-unreachable=$(($test_tick - 10000)) \
+ --stale-fix \
+ --all &&
+
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 2 reflog &&
+
+ check_fsck "dangling commit $K"
+'
+
+test_expect_success 'prune and fsck' '
+
+ git prune &&
+ check_fsck &&
+
+ check_have A B C D E H L &&
+ check_dont_have F G I J K
+
+'
+
+test_expect_success 'recover and check' '
+
+ recover $F &&
+ check_fsck "dangling blob $F"
+
+'
+
+test_expect_success 'delete' '
+ echo 1 > C &&
+ test_tick &&
+ git commit -m rat C &&
+
+ echo 2 > C &&
+ test_tick &&
+ git commit -m ox C &&
+
+ echo 3 > C &&
+ test_tick &&
+ git commit -m tiger C &&
+
+ HEAD_entry_count=$(git reflog | wc -l) &&
+ master_entry_count=$(git reflog show master | wc -l) &&
+
+ test $HEAD_entry_count = 5 &&
+ test $master_entry_count = 5 &&
+
+
+ git reflog delete master@{1} &&
+ git reflog show master > output &&
+ test $(($master_entry_count - 1)) = $(wc -l < output) &&
+ test $HEAD_entry_count = $(git reflog | wc -l) &&
+ ! grep ox < output &&
+
+ master_entry_count=$(wc -l < output) &&
+
+ git reflog delete HEAD@{1} &&
+ test $(($HEAD_entry_count -1)) = $(git reflog | wc -l) &&
+ test $master_entry_count = $(git reflog show master | wc -l) &&
+
+ HEAD_entry_count=$(git reflog | wc -l) &&
+
+ git reflog delete master@{07.04.2005.15:15:00.-0700} &&
+ git reflog show master > output &&
+ test $(($master_entry_count - 1)) = $(wc -l < output) &&
+ ! grep dragon < output
+
+'
+
+test_expect_success 'rewind2' '
+
+ test_tick && git reset --hard HEAD~2 &&
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 4 reflog
+'
+
+test_expect_success '--expire=never' '
+
+ git reflog expire --verbose \
+ --expire=never \
+ --expire-unreachable=never \
+ --all &&
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 4 reflog
+'
+
+test_expect_success 'gc.reflogexpire=never' '
+
+ git config gc.reflogexpire never &&
+ git config gc.reflogexpireunreachable never &&
+ git reflog expire --verbose --all &&
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 4 reflog
+'
+
+test_expect_success 'gc.reflogexpire=false' '
+
+ git config gc.reflogexpire false &&
+ git config gc.reflogexpireunreachable false &&
+ git reflog expire --verbose --all &&
+ raw_reflog refs/heads/master >reflog &&
+ test_when_finished rm -f reflog &&
+ test_line_count = 4 reflog &&
+
+ git config --unset gc.reflogexpire &&
+ git config --unset gc.reflogexpireunreachable
+
+'
+
+test_expect_success 'checkout should not delete log for packed ref' '
+ test $(git reflog master | wc -l) = 4 &&
+ git branch foo &&
+ git pack-refs --all &&
+ git checkout foo &&
+ test $(git reflog master | wc -l) = 4
+'
+
+test_expect_success 'stale dirs do not cause d/f conflicts (reflogs on)' '
+ test_when_finished "git branch -d one || git branch -d one/two" &&
+
+ git branch one/two master &&
+ echo "one/two@{0} branch: Created from master" >expect &&
+ git log -g --format="%gd %gs" one/two >actual &&
+ test_cmp expect actual &&
+ git branch -d one/two &&
+
+ # now logs/refs/heads/one is a stale directory, but
+ # we should move it out of the way to create "one" reflog
+ git branch one master &&
+ echo "one@{0} branch: Created from master" >expect &&
+ git log -g --format="%gd %gs" one >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'stale dirs do not cause d/f conflicts (reflogs off)' '
+ test_when_finished "git branch -d one || git branch -d one/two" &&
+
+ git branch one/two master &&
+ echo "one/two@{0} branch: Created from master" >expect &&
+ git log -g --format="%gd %gs" one/two >actual &&
+ test_cmp expect actual &&
+ git branch -d one/two &&
+
+ # same as before, but we only create a reflog for "one" if
+ # it already exists, which it does not
+ git -c core.logallrefupdates=false branch one master &&
+ : >expect &&
+ git log -g --format="%gd %gs" one >actual &&
+ test_cmp expect actual
+'
+
+# Triggering the bug detected by this test requires a newline to fall
+# exactly BUFSIZ-1 bytes from the end of the file. We don't know
+# what that value is, since it's platform dependent. However, if
+# we choose some value N, we also catch any D which divides N evenly
+# (since we will read backwards in chunks of D). So we choose 8K,
+# which catches glibc (with an 8K BUFSIZ) and *BSD (1K).
+#
+# Each line is 114 characters, so we need 75 to still have a few before the
+# last 8K. The 89-character padding on the final entry lines up our
+# newline exactly.
+test_expect_success 'parsing reverse reflogs at BUFSIZ boundaries' '
+ git checkout -b reflogskip &&
+ z38=00000000000000000000000000000000000000 &&
+ ident="abc <xyz> 0000000001 +0000" &&
+ for i in $(test_seq 1 75); do
+ printf "$z38%02d $z38%02d %s\t" $i $(($i+1)) "$ident" &&
+ if test $i = 75; then
+ for j in $(test_seq 1 89); do
+ printf X
+ done
+ else
+ printf X
+ fi &&
+ printf "\n"
+ done | append_reflog refs/heads/reflogskip &&
+ git rev-parse reflogskip@{73} >actual &&
+ echo ${z38}03 >expect &&
+ test_cmp expect actual
+'
+
+test_done
diff --git a/t/t1480-refs-lmdb-submodule.sh b/t/t1480-refs-lmdb-submodule.sh
new file mode 100755
index 0000000..57a5fd2
--- /dev/null
+++ b/t/t1480-refs-lmdb-submodule.sh
@@ -0,0 +1,85 @@
+#!/bin/sh
+#
+# Copyright (c) 2016 Twitter, Inc
+# Based on t5531-deep-submodule-push.sh
+
+test_description='Test lmdb refs backend'
+TEST_NO_CREATE_REPO=1
+. ./test-lib.sh
+
+if ! test_have_prereq LMDB
+then
+ skip_all="Skipping lmdb refs backend tests, lmdb backend not built"
+ test_done
+fi
+
+test_expect_success setup '
+ mkdir pub.git &&
+ GIT_DIR=pub.git git init --bare &&
+ GIT_DIR=pub.git git config receive.fsckobjects true &&
+ mkdir work &&
+ (
+ cd work &&
+ git init --ref-storage=lmdb &&
+ git config push.default matching &&
+ mkdir -p gar/bage &&
+ (
+ cd gar/bage &&
+ git init --ref-storage=lmdb &&
+ git config push.default matching &&
+ >junk &&
+ git add junk &&
+ git commit -m "Initial junk"
+ ) &&
+ git add gar/bage &&
+ git commit -m "Initial superproject"
+ )
+'
+
+test_expect_success 'submodules have same ref storage' '
+ git init --ref-storage=lmdb test &&
+ (
+ cd test &&
+ git submodule add ../work/gar/bage w
+ ) &&
+ (
+ cd test/w &&
+ git config extensions.refstorage >cfg &&
+ echo lmdb >expect &&
+ test_cmp cfg expect
+ )
+'
+
+test_expect_success 'push with correct backend' '
+ (
+ cd work/gar/bage &&
+ >junk2 &&
+ git add junk2 &&
+ git commit -m "Second junk"
+ ) &&
+ (
+ cd work &&
+ git add gar/bage &&
+ git commit -m "Second commit for gar/bage" &&
+ git push --recurse-submodules=check ../pub.git master
+ )
+'
+
+test_expect_success 'commit with different backend fails' '
+ (
+ cd work/gar/bage &&
+ test_commit junk3 &&
+ # manually convert to files-backend
+ gitdir="$(git rev-parse --git-dir)" &&
+ mkdir -p "$gitdir/refs/heads" &&
+ git rev-parse HEAD >"$gitdir/refs/heads/master" &&
+ git config --local --unset extensions.refStorage &&
+ rm -r "$gitdir/refs.lmdb"
+ ) &&
+ (
+ cd work &&
+ test_must_fail git add gar/bage
+ )
+'
+
+test_done
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 0b47eb6..ce40770 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -960,6 +960,7 @@ test -z "$NO_PERL" && test_set_prereq PERL
test -z "$NO_PYTHON" && test_set_prereq PYTHON
test -n "$USE_LIBPCRE" && test_set_prereq LIBPCRE
test -z "$NO_GETTEXT" && test_set_prereq GETTEXT
+test -n "$USE_LIBLMDB" && test_set_prereq LMDB
# Can we rely on git's output in the C locale?
if test -n "$GETTEXT_POISON"
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* [PATCH v6 32/32] tests: add ref-storage argument
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (30 preceding siblings ...)
2016-02-24 22:59 ` [PATCH v6 31/32] refs: tests for lmdb backend David Turner
@ 2016-02-24 22:59 ` David Turner
2016-02-29 22:52 ` SZEDER Gábor
2016-02-25 0:08 ` [PATCH v6 00/32] refs backend Ramsay Jones
2016-02-25 12:57 ` Duy Nguyen
33 siblings, 1 reply; 56+ messages in thread
From: David Turner @ 2016-02-24 22:59 UTC (permalink / raw)
To: git, mhagger, pclouds; +Cc: David Turner
Add a --ref-storage argument to make it easy to test alternate ref
storage backends. This means that every test that calls git init or
git clone must use the new ref storage argument.
Modify many tests to work under alternate ref storage backends.
Introduce abstractions for raw ref/reflog reading/writing in tests
instead of directly frobbing the filesystem.
Conditionally skip tests that are not expected to succeed under this
condition. Most of this is straightforward. Of particular note are the
following test changes:
* The rearrangement of commands in t1401 is because without HEAD in
the right place, git doesn't recognized the trash dir as a git repo,
so no git commands work.
* t1430-bad-ref-name specifically blocks lmdb because other alternate
backends might want to keep this test.
Signed-off-by: David Turner <dturner@twopensource.com>
---
t/README | 6 +++
t/lib-submodule-update.sh | 15 +++++--
t/lib-t6000.sh | 7 ++-
t/t0008-ignores.sh | 2 +-
t/t0062-revision-walking.sh | 6 +++
t/t1021-rerere-in-workdir.sh | 6 +++
t/t1200-tutorial.sh | 8 +++-
t/t1302-repo-version.sh | 6 +++
t/t1305-config-include.sh | 17 ++++++--
t/t1400-update-ref.sh | 6 +++
t/t1401-symbolic-ref.sh | 17 +++++---
t/t1404-update-ref-df-conflicts.sh | 8 +++-
t/t1410-reflog.sh | 6 +++
t/t1430-bad-ref-name.sh | 6 +++
t/t1450-fsck.sh | 12 ++---
t/t1506-rev-parse-diagnosis.sh | 4 +-
t/t2013-checkout-submodule.sh | 2 +-
t/t2105-update-index-gitfile.sh | 4 +-
t/t2107-update-index-basic.sh | 6 +--
t/t2201-add-update-typechange.sh | 4 +-
t/t3001-ls-files-others-exclude.sh | 2 +-
t/t3010-ls-files-killed-modified.sh | 4 +-
t/t3040-subprojects-basic.sh | 4 +-
t/t3050-subprojects-fetch.sh | 2 +-
t/t3200-branch.sh | 75 ++++++++++++++++++--------------
t/t3210-pack-refs.sh | 7 +++
t/t3211-peel-ref.sh | 6 +++
t/t3308-notes-merge.sh | 2 +-
t/t3404-rebase-interactive.sh | 2 +-
t/t3600-rm.sh | 2 +-
t/t3800-mktag.sh | 4 +-
t/t3903-stash.sh | 2 +-
t/t4010-diff-pathspec.sh | 2 +-
t/t4020-diff-external.sh | 2 +-
t/t4027-diff-submodule.sh | 2 +-
t/t4035-diff-quiet.sh | 2 +-
t/t4255-am-submodule.sh | 2 +-
t/t5000-tar-tree.sh | 3 +-
t/t5304-prune.sh | 2 +-
t/t5312-prune-corruption.sh | 11 ++++-
t/t5500-fetch-pack.sh | 10 ++---
t/t5510-fetch.sh | 30 ++++++-------
t/t5526-fetch-submodules.sh | 4 +-
t/t5527-fetch-odd-refs.sh | 7 +++
t/t5537-fetch-shallow.sh | 7 +++
t/t5700-clone-reference.sh | 42 +++++++++---------
t/t6001-rev-list-graft.sh | 3 +-
t/t6010-merge-base.sh | 2 +-
t/t6050-replace.sh | 4 +-
t/t6120-describe.sh | 6 ++-
t/t6301-for-each-ref-errors.sh | 12 ++---
t/t7201-co.sh | 2 +-
t/t7300-clean.sh | 25 ++++++-----
t/t7400-submodule-basic.sh | 18 ++++----
t/t7402-submodule-rebase.sh | 2 +-
t/t7405-submodule-merge.sh | 10 ++---
t/t9104-git-svn-follow-parent.sh | 3 +-
t/t9115-git-svn-dcommit-funky-renames.sh | 2 +-
t/t9350-fast-export.sh | 6 +--
t/t9902-completion.sh | 3 +-
t/t9903-bash-prompt.sh | 1 +
t/test-lib-functions.sh | 53 +++++++++++++++++++++-
t/test-lib.sh | 11 +++++
63 files changed, 368 insertions(+), 181 deletions(-)
diff --git a/t/README b/t/README
index 1dc908e..8e047cb 100644
--- a/t/README
+++ b/t/README
@@ -178,6 +178,12 @@ appropriately before running "make".
this feature by setting the GIT_TEST_CHAIN_LINT environment
variable to "1" or "0", respectively.
+--ref-storage=<backend>::
+ If --ref-storage is set, run tests under an alternate ref
+ backend. Tests that rely on the original files backend will
+ be skipped. You may also enable or disable this feature by
+ setting the GIT_TEST_REF_STORAGE environment variable
+
You can also set the GIT_TEST_INSTALLED environment variable to
the bindir of an existing git installation to test that installation.
You still need to have built this git sandbox, from which various
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 79cdd34..b037eb3 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -32,7 +32,7 @@
# invalid_sub1
#
create_lib_submodule_repo () {
- git init submodule_update_repo &&
+ git init $ref_storage_arg submodule_update_repo &&
(
cd submodule_update_repo &&
echo "expect" >>.gitignore &&
@@ -125,7 +125,16 @@ test_git_directory_is_unchanged () {
# "$1/.git/config" lacks it...
git config --unset core.worktree
) &&
- diff -r ".git/modules/$1" "$1/.git" &&
+ if test "$ref_storage" = "lmdb"
+ then
+ diff -x refs.lmdb -r ".git/modules/$1" "$1/.git" &&
+ mdb_dump -p ".git/modules/$1/refs.lmdb" >one &&
+ mdb_dump -p "$1/.git/refs.lmdb" >two &&
+ diff one two &&
+ rm one two
+ else
+ diff -r ".git/modules/$1" "$1/.git"
+ fi
(
# ... and then restore.
cd ".git/modules/$1" &&
@@ -147,7 +156,7 @@ prolog () {
# should be updated to an existing commit.
reset_work_tree_to () {
rm -rf submodule_update &&
- git clone submodule_update_repo submodule_update &&
+ git clone $ref_storage_arg submodule_update_repo submodule_update &&
(
cd submodule_update &&
rm -rf sub1 &&
diff --git a/t/lib-t6000.sh b/t/lib-t6000.sh
index 3f2d873..5fb4345 100644
--- a/t/lib-t6000.sh
+++ b/t/lib-t6000.sh
@@ -4,11 +4,10 @@ mkdir -p .git/refs/tags
>sed.script
-# Answer the sha1 has associated with the tag. The tag must exist in .git/refs/tags
+# Answer the sha1 has associated with the tag. The tag must exist under refs/tags
tag () {
_tag=$1
- test -f ".git/refs/tags/$_tag" || error "tag: \"$_tag\" does not exist"
- cat ".git/refs/tags/$_tag"
+ git rev-parse --verify "refs/tags/$_tag" || error "tag: \"$_tag\" does not exist"
}
# Generate a commit using the text specified to make it unique and the tree
@@ -26,7 +25,7 @@ save_tag () {
_tag=$1
test -n "$_tag" || error "usage: save_tag tag commit-args ..."
shift 1
- "$@" >".git/refs/tags/$_tag"
+ write_ref "refs/tags/$_tag" $("$@")
echo "s/$(tag $_tag)/$_tag/g" >sed.script.tmp
cat sed.script >>sed.script.tmp
diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh
index 89544dd..63fe6d5 100755
--- a/t/t0008-ignores.sh
+++ b/t/t0008-ignores.sh
@@ -184,7 +184,7 @@ test_expect_success 'setup' '
fi &&
(
cd a/submodule &&
- git init &&
+ git init $ref_storage_arg &&
echo a >a &&
git add a &&
git commit -m"commit in submodule"
diff --git a/t/t0062-revision-walking.sh b/t/t0062-revision-walking.sh
index 113c728..0bd3981 100755
--- a/t/t0062-revision-walking.sh
+++ b/t/t0062-revision-walking.sh
@@ -7,6 +7,12 @@ test_description='Test revision walking api'
. ./test-lib.sh
+if test "$ref_storage" != "files"
+then
+ skip_all="Alternate storage doesn't do test-revision-walking"
+ test_done
+fi
+
cat >run_twice_expected <<-EOF
1st
> add b
diff --git a/t/t1021-rerere-in-workdir.sh b/t/t1021-rerere-in-workdir.sh
index 301e071..8fe5de7 100755
--- a/t/t1021-rerere-in-workdir.sh
+++ b/t/t1021-rerere-in-workdir.sh
@@ -3,6 +3,12 @@
test_description='rerere run in a workdir'
. ./test-lib.sh
+if test "$ref_storage" != "files"
+then
+ skip_all="Workdirs don't support alternate ref backends"
+ test_done
+fi
+
test_expect_success SYMLINKS setup '
git config rerere.enabled true &&
>world &&
diff --git a/t/t1200-tutorial.sh b/t/t1200-tutorial.sh
index 397ccb6..c67c7da 100755
--- a/t/t1200-tutorial.sh
+++ b/t/t1200-tutorial.sh
@@ -93,12 +93,16 @@ test_expect_success 'git whatchanged -p --root' '
test_expect_success 'git tag my-first-tag' '
git tag my-first-tag &&
- test_cmp .git/refs/heads/master .git/refs/tags/my-first-tag
+ git rev-parse --verify refs/heads/master >expect &&
+ git rev-parse --verify refs/tags/my-first-tag >actual &&
+ test_cmp expect actual
'
test_expect_success 'git checkout -b mybranch' '
git checkout -b mybranch &&
- test_cmp .git/refs/heads/master .git/refs/heads/mybranch
+ git rev-parse --verify refs/heads/master >expect &&
+ git rev-parse --verify refs/heads/mybranch >actual &&
+ test_cmp expect actual
'
cat > branch.expect <<EOF
diff --git a/t/t1302-repo-version.sh b/t/t1302-repo-version.sh
index 9bcd349..270a8d3 100755
--- a/t/t1302-repo-version.sh
+++ b/t/t1302-repo-version.sh
@@ -7,6 +7,12 @@ test_description='Test repository version check'
. ./test-lib.sh
+if test "$ref_storage" != "files"
+then
+ skip_all="Alternate ref storage sets core.repositoryformatversion=1"
+ test_done
+fi
+
test_expect_success 'setup' '
cat >test.patch <<-\EOF &&
diff --git a/test.txt b/test.txt
diff --git a/t/t1305-config-include.sh b/t/t1305-config-include.sh
index 9ba2ba1..220cd63 100755
--- a/t/t1305-config-include.sh
+++ b/t/t1305-config-include.sh
@@ -45,14 +45,23 @@ test_expect_success 'include options can still be examined' '
test_cmp expect actual
'
-test_expect_success 'listing includes option and expansion' '
- echo "[test]one = 1" >one &&
- echo "[include]path = one" >.gitconfig &&
- cat >expect <<-\EOF &&
+write_expected_config()
+{
+ cat <<-\EOF &&
include.path=one
test.one=1
EOF
+ if test "$ref_storage" != "files"
+ then
+ echo "extensions.refstorage=$ref_storage"
+ fi
+}
+
+test_expect_success 'listing includes option and expansion' '
+ echo "[test]one = 1" >one &&
+ echo "[include]path = one" >.gitconfig &&
git config --list >actual.full &&
+ write_expected_config >expect &&
grep -v ^core actual.full >actual &&
test_cmp expect actual
'
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index af1b20d..878dc53 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -6,6 +6,12 @@
test_description='Test git update-ref and basic ref logging'
. ./test-lib.sh
+if test "$ref_storage" != "files"
+then
+ skip_all="This test is ref storage backend-specific"
+ test_done
+fi
+
Z=$_z40
test_expect_success setup '
diff --git a/t/t1401-symbolic-ref.sh b/t/t1401-symbolic-ref.sh
index 417eecc..5bfe7d2 100755
--- a/t/t1401-symbolic-ref.sh
+++ b/t/t1401-symbolic-ref.sh
@@ -35,15 +35,16 @@ reset_to_sane
test_expect_success 'symbolic-ref deletes HEAD' '
git symbolic-ref -d HEAD &&
- test_path_is_file .git/refs/heads/foo &&
- test_path_is_missing .git/HEAD
+ test_path_is_missing .git/HEAD &&
+ reset_to_sane &&
+ git rev-parse --verify refs/heads/foo
'
reset_to_sane
test_expect_success 'symbolic-ref deletes dangling HEAD' '
git symbolic-ref HEAD refs/heads/missing &&
git symbolic-ref -d HEAD &&
- test_path_is_missing .git/refs/heads/missing &&
+ test_must_fail git rev-parse --verify refs/heads/missing &&
test_path_is_missing .git/HEAD
'
reset_to_sane
@@ -58,7 +59,7 @@ reset_to_sane
test_expect_success 'symbolic-ref fails to delete real ref' '
echo "fatal: Cannot delete refs/heads/foo, not a symbolic ref" >expect &&
test_must_fail git symbolic-ref -d refs/heads/foo >actual 2>&1 &&
- test_path_is_file .git/refs/heads/foo &&
+ git rev-parse --verify refs/heads/foo &&
test_cmp expect actual
'
reset_to_sane
@@ -114,7 +115,13 @@ test_expect_success 'symbolic-ref writes reflog entry' '
test_cmp expect actual
'
-test_expect_success 'symbolic-ref does not create ref d/f conflicts' '
+if test "$ref_storage" = "files"
+then
+ test_cmd=test_expect_success
+else
+ test_cmd=test_expect_failure
+fi
+$test_cmd 'symbolic-ref does not create ref d/f conflicts' '
git checkout -b df &&
test_commit df &&
test_must_fail git symbolic-ref refs/heads/df/conflict refs/heads/df &&
diff --git a/t/t1404-update-ref-df-conflicts.sh b/t/t1404-update-ref-df-conflicts.sh
index 66bafb5..e918abc 100755
--- a/t/t1404-update-ref-df-conflicts.sh
+++ b/t/t1404-update-ref-df-conflicts.sh
@@ -96,7 +96,13 @@ test_expect_success 'new ref is a deeper prefix of existing packed' '
'
-test_expect_success 'one new ref is a simple prefix of another' '
+expect=test_expect_success
+if test "$ref_storage" != "files"
+then
+ expect=test_expect_failure
+fi
+
+$expect 'one new ref is a simple prefix of another' '
prefix=refs/5 &&
test_update_rejected $prefix "a e" false "b c c/x d" \
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index 9cf91dc..41ff1a2 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -6,6 +6,12 @@
test_description='Test prune and reflog expiration'
. ./test-lib.sh
+if test "$ref_storage" != "files"
+then
+ skip_all="This test is ref storage backend-specific"
+ test_done
+fi
+
check_have () {
gaah= &&
for N in "$@"
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index c465abe..bfcec3c 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -3,6 +3,12 @@
test_description='Test handling of ref names that check-ref-format rejects'
. ./test-lib.sh
+if test "$ref_storage" = "lmdb"
+then
+ skip_all="The lmdb backend refuses to save refs with bad names"
+ test_done
+fi
+
test_expect_success setup '
test_commit one &&
test_commit two
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index e66b7cb..5fd3921 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -75,7 +75,7 @@ test_expect_success 'object with bad sha1' '
'
test_expect_success 'branch pointing to non-commit' '
- git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+ write_ref refs/heads/invalid $(git rev-parse HEAD^{tree}) &&
test_when_finished "git update-ref -d refs/heads/invalid" &&
test_must_fail git fsck 2>out &&
cat out &&
@@ -220,7 +220,7 @@ test_expect_success 'tag pointing to nonexistent' '
tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
test_when_finished "remove_object $tag" &&
- echo $tag >.git/refs/tags/invalid &&
+ write_ref refs/tags/invalid $tag &&
test_when_finished "git update-ref -d refs/tags/invalid" &&
test_must_fail git fsck --tags >out &&
cat out &&
@@ -241,7 +241,7 @@ test_expect_success 'tag pointing to something else than its type' '
tag=$(git hash-object -t tag -w --stdin <wrong-tag) &&
test_when_finished "remove_object $tag" &&
- echo $tag >.git/refs/tags/wrong &&
+ write_ref refs/tags/wrong $tag &&
test_when_finished "git update-ref -d refs/tags/wrong" &&
test_must_fail git fsck --tags
'
@@ -258,7 +258,7 @@ test_expect_success 'tag with incorrect tag name & missing tagger' '
tag=$(git hash-object -t tag -w --stdin <wrong-tag) &&
test_when_finished "remove_object $tag" &&
- echo $tag >.git/refs/tags/wrong &&
+ write_ref refs/tags/wrong $tag &&
test_when_finished "git update-ref -d refs/tags/wrong" &&
git fsck --tags 2>out &&
@@ -282,7 +282,7 @@ test_expect_success 'tag with bad tagger' '
tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
test_when_finished "remove_object $tag" &&
- echo $tag >.git/refs/tags/wrong &&
+ write_ref refs/tags/wrong $tag &&
test_when_finished "git update-ref -d refs/tags/wrong" &&
test_must_fail git fsck --tags 2>out &&
grep "error in tag .*: invalid author/committer" out
@@ -301,7 +301,7 @@ test_expect_success 'tag with NUL in header' '
tag=$(git hash-object --literally -t tag -w --stdin <tag-NUL-header) &&
test_when_finished "remove_object $tag" &&
- echo $tag >.git/refs/tags/wrong &&
+ write_ref refs/tags/wrong $tag &&
test_when_finished "git update-ref -d refs/tags/wrong" &&
test_must_fail git fsck --tags 2>out &&
cat out &&
diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index 613d9bf..a9a15ca 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -167,7 +167,7 @@ test_expect_success 'relative path when cwd is outside worktree' '
'
test_expect_success 'relative path when startup_info is NULL' '
- test_must_fail test-match-trees HEAD:./file.txt HEAD:./file.txt 2>error &&
+ test_must_fail test-match-trees $(git rev-parse HEAD):./file.txt $(git rev-parse HEAD):./file.txt 2>error &&
grep "BUG: startup_info struct is not initialized." error
'
@@ -213,7 +213,7 @@ test_expect_success 'arg before dashdash must be a revision (ambiguous)' '
{
# we do not want to use rev-parse here, because
# we are testing it
- cat .git/refs/heads/foobar &&
+ raw_ref refs/heads/foobar &&
printf "%s\n" --
} >expect &&
git rev-parse foobar -- >actual &&
diff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh
index 6847f75..94e8ec8 100755
--- a/t/t2013-checkout-submodule.sh
+++ b/t/t2013-checkout-submodule.sh
@@ -8,7 +8,7 @@ test_description='checkout can handle submodules'
test_expect_success 'setup' '
mkdir submodule &&
(cd submodule &&
- git init &&
+ git init $ref_storage_arg &&
test_commit first) &&
git add submodule &&
test_tick &&
diff --git a/t/t2105-update-index-gitfile.sh b/t/t2105-update-index-gitfile.sh
index a7f3d47..ce151a4 100755
--- a/t/t2105-update-index-gitfile.sh
+++ b/t/t2105-update-index-gitfile.sh
@@ -11,7 +11,7 @@ test_description='git update-index for gitlink to .git file.
test_expect_success 'submodule with absolute .git file' '
mkdir sub1 &&
(cd sub1 &&
- git init &&
+ git init $ref_storage_arg &&
REAL="$(pwd)/.real" &&
mv .git "$REAL" &&
echo "gitdir: $REAL" >.git &&
@@ -25,7 +25,7 @@ test_expect_success 'add gitlink to absolute .git file' '
test_expect_success 'submodule with relative .git file' '
mkdir sub2 &&
(cd sub2 &&
- git init &&
+ git init $ref_storage_arg &&
mv .git .real &&
echo "gitdir: .real" >.git &&
test_commit first)
diff --git a/t/t2107-update-index-basic.sh b/t/t2107-update-index-basic.sh
index dfe02f4..b00adef 100755
--- a/t/t2107-update-index-basic.sh
+++ b/t/t2107-update-index-basic.sh
@@ -22,7 +22,7 @@ test_expect_success 'update-index -h with corrupt index' '
mkdir broken &&
(
cd broken &&
- git init &&
+ git init $ref_storage_arg &&
>.git/index &&
test_expect_code 129 git update-index -h >usage 2>&1
) &&
@@ -43,7 +43,7 @@ test_expect_success '--cacheinfo does not accept blob null sha1' '
'
test_expect_success '--cacheinfo does not accept gitlink null sha1' '
- git init submodule &&
+ git init submodule $ref_storage_arg &&
(cd submodule && test_commit foo) &&
git add submodule &&
git rev-parse :submodule >expect &&
@@ -70,7 +70,7 @@ test_expect_success '.lock files cleaned up' '
(
cd cleanup &&
mkdir worktree &&
- git init repo &&
+ git init $ref_storage_arg repo &&
cd repo &&
git config core.worktree ../../worktree &&
# --refresh triggers late setup_work_tree,
diff --git a/t/t2201-add-update-typechange.sh b/t/t2201-add-update-typechange.sh
index 954fc51..882a5b7 100755
--- a/t/t2201-add-update-typechange.sh
+++ b/t/t2201-add-update-typechange.sh
@@ -46,7 +46,7 @@ test_expect_success modify '
>yomin/yomin &&
(
cd yomin &&
- git init &&
+ git init $ref_storage_arg &&
git add yomin &&
git commit -m "sub initial"
) &&
@@ -60,7 +60,7 @@ test_expect_success modify '
>yonk/yonk &&
(
cd yonk &&
- git init &&
+ git init $ref_storage_arg &&
git add yonk &&
git commit -m "sub initial"
) &&
diff --git a/t/t3001-ls-files-others-exclude.sh b/t/t3001-ls-files-others-exclude.sh
index 3fc484e..b26e68e 100755
--- a/t/t3001-ls-files-others-exclude.sh
+++ b/t/t3001-ls-files-others-exclude.sh
@@ -197,7 +197,7 @@ test_expect_success 'subdirectory ignore (setup)' '
mkdir -p top/l1/l2 &&
(
cd top &&
- git init &&
+ git init $ref_storage_arg &&
echo /.gitignore >.gitignore &&
echo l1 >>.gitignore &&
echo l2 >l1/.gitignore &&
diff --git a/t/t3010-ls-files-killed-modified.sh b/t/t3010-ls-files-killed-modified.sh
index 580e158..3cf2a74 100755
--- a/t/t3010-ls-files-killed-modified.sh
+++ b/t/t3010-ls-files-killed-modified.sh
@@ -55,9 +55,9 @@ test_expect_success 'git update-index --add to add various paths.' '
: >path9 &&
date >path10 &&
git update-index --add -- path0 path?/file? pathx/ju path7 path8 path9 path10 &&
- git init submod1 &&
+ git init $ref_storage_arg submod1 &&
git -C submod1 commit --allow-empty -m "empty 1" &&
- git init submod2 &&
+ git init $ref_storage_arg submod2 &&
git -C submod2 commit --allow-empty -m "empty 2" &&
git update-index --add submod[12] &&
(
diff --git a/t/t3040-subprojects-basic.sh b/t/t3040-subprojects-basic.sh
index 0a4ff6d..368c03e 100755
--- a/t/t3040-subprojects-basic.sh
+++ b/t/t3040-subprojects-basic.sh
@@ -11,10 +11,10 @@ test_expect_success 'setup: create superproject' '
test_expect_success 'setup: create subprojects' '
mkdir sub1 &&
- ( cd sub1 && git init && : >Makefile && git add * &&
+ ( cd sub1 && git init $ref_storage_arg && : >Makefile && git add * &&
git commit -q -m "subproject 1" ) &&
mkdir sub2 &&
- ( cd sub2 && git init && : >Makefile && git add * &&
+ ( cd sub2 && git init $ref_storage_arg && : >Makefile && git add * &&
git commit -q -m "subproject 2" ) &&
git update-index --add sub1 &&
git add sub2 &&
diff --git a/t/t3050-subprojects-fetch.sh b/t/t3050-subprojects-fetch.sh
index 2f5f41a..8d26bb9 100755
--- a/t/t3050-subprojects-fetch.sh
+++ b/t/t3050-subprojects-fetch.sh
@@ -8,7 +8,7 @@ test_expect_success setup '
test_tick &&
mkdir -p sub && (
cd sub &&
- git init &&
+ git init $ref_storage_arg &&
>subfile &&
git add subfile &&
git commit -m "subproject commit #1"
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 07e9749..f7287ef 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -19,26 +19,26 @@ test_expect_success 'prepare a trivial repository' '
test_expect_success 'git branch --help should not have created a bogus branch' '
test_might_fail git branch --man --help </dev/null >/dev/null 2>&1 &&
- test_path_is_missing .git/refs/heads/--help
+ test_must_fail git rev-parse --verify refs/heads/--help
'
test_expect_success 'branch -h in broken repository' '
mkdir broken &&
(
cd broken &&
- git init &&
- >.git/refs/heads/master &&
+ git init $ref_storage_arg &&
+ write_ref refs/heads/master "" &&
test_expect_code 129 git branch -h >usage 2>&1
) &&
test_i18ngrep "[Uu]sage" broken/usage
'
test_expect_success 'git branch abc should create a branch' '
- git branch abc && test_path_is_file .git/refs/heads/abc
+ git branch abc && git rev-parse --verify refs/heads/abc
'
test_expect_success 'git branch a/b/c should create a branch' '
- git branch a/b/c && test_path_is_file .git/refs/heads/a/b/c
+ git branch a/b/c && git rev-parse --verify refs/heads/a/b/c
'
test_expect_success 'git branch HEAD should fail' '
@@ -51,14 +51,14 @@ EOF
test_expect_success 'git branch -l d/e/f should create a branch and a log' '
GIT_COMMITTER_DATE="2005-05-26 23:30" \
git branch -l d/e/f &&
- test_path_is_file .git/refs/heads/d/e/f &&
- test_path_is_file .git/logs/refs/heads/d/e/f &&
- test_cmp expect .git/logs/refs/heads/d/e/f
+ git rev-parse --verify refs/heads/d/e/f &&
+ raw_reflog refs/heads/d/e/f > reflog &&
+ test_cmp expect reflog
'
test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
git branch -d d/e/f &&
- test_path_is_missing .git/refs/heads/d/e/f &&
+ test_must_fail git rev-parse --verify refs/heads/d/e/f &&
test_must_fail git reflog exists refs/heads/d/e/f
'
@@ -156,34 +156,34 @@ test_expect_success 'git branch -M master2 master2 should work when master is ch
test_expect_success 'git branch -v -d t should work' '
git branch t &&
- test_path_is_file .git/refs/heads/t &&
+ git rev-parse --verify refs/heads/t &&
git branch -v -d t &&
- test_path_is_missing .git/refs/heads/t
+ test_must_fail git rev-parse --verify refs/heads/t
'
test_expect_success 'git branch -v -m t s should work' '
git branch t &&
- test_path_is_file .git/refs/heads/t &&
+ git rev-parse --verify refs/heads/t &&
git branch -v -m t s &&
- test_path_is_missing .git/refs/heads/t &&
- test_path_is_file .git/refs/heads/s &&
+ test_must_fail git rev-parse --verify refs/heads/t &&
+ git rev-parse --verify refs/heads/s &&
git branch -d s
'
test_expect_success 'git branch -m -d t s should fail' '
git branch t &&
- test_path_is_file .git/refs/heads/t &&
+ git rev-parse refs/heads/t &&
test_must_fail git branch -m -d t s &&
git branch -d t &&
- test_path_is_missing .git/refs/heads/t
+ test_must_fail git rev-parse refs/heads/t
'
test_expect_success 'git branch --list -d t should fail' '
git branch t &&
- test_path_is_file .git/refs/heads/t &&
+ git rev-parse refs/heads/t &&
test_must_fail git branch --list -d t &&
git branch -d t &&
- test_path_is_missing .git/refs/heads/t
+ test_must_fail git rev-parse refs/heads/t
'
test_expect_success 'git branch --column' '
@@ -263,6 +263,10 @@ EOF
test_cmp expected actual
'
+# All alternate ref storage backends require config, so we just skip
+# the "no-config" test if we're using one.
+if test $ref_storage = "files"
+then
mv .git/config .git/config-saved
test_expect_success 'git branch -m q q2 without config should succeed' '
@@ -271,6 +275,7 @@ test_expect_success 'git branch -m q q2 without config should succeed' '
'
mv .git/config-saved .git/config
+fi
git config branch.s/s.dummy Hello
@@ -294,26 +299,26 @@ test_expect_success 'deleting a symref' '
git symbolic-ref refs/heads/symref refs/heads/target &&
echo "Deleted branch symref (was refs/heads/target)." >expect &&
git branch -d symref >actual &&
- test_path_is_file .git/refs/heads/target &&
- test_path_is_missing .git/refs/heads/symref &&
+ git rev-parse refs/heads/target &&
+ test_must_fail git rev-parse refs/heads/symref &&
test_i18ncmp expect actual
'
test_expect_success 'deleting a dangling symref' '
git symbolic-ref refs/heads/dangling-symref nowhere &&
- test_path_is_file .git/refs/heads/dangling-symref &&
+ raw_ref refs/heads/dangling-symref &&
echo "Deleted branch dangling-symref (was nowhere)." >expect &&
git branch -d dangling-symref >actual &&
- test_path_is_missing .git/refs/heads/dangling-symref &&
+ test_must_fail git rev-parse refs/heads/dangling-symref &&
test_i18ncmp expect actual
'
test_expect_success 'deleting a self-referential symref' '
git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
- test_path_is_file .git/refs/heads/self-reference &&
+ raw_ref refs/heads/self-reference &&
echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
git branch -d self-reference >actual &&
- test_path_is_missing .git/refs/heads/self-reference &&
+ test_must_fail git rev-parse refs/heads/self-reference &&
test_i18ncmp expect actual
'
@@ -321,16 +326,20 @@ test_expect_success 'renaming a symref is not allowed' '
git symbolic-ref refs/heads/master2 refs/heads/master &&
test_must_fail git branch -m master2 master3 &&
git symbolic-ref refs/heads/master2 &&
- test_path_is_file .git/refs/heads/master &&
- test_path_is_missing .git/refs/heads/master3
+ git rev-parse refs/heads/master &&
+ test_must_fail git rev-parse refs/heads/master3
'
+# lmdb doesn't support store reflogs in the filesystem
+if test $ref_storage != "lmdb"
+then
test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
git branch -l u &&
mv .git/logs/refs/heads/u real-u &&
ln -s real-u .git/logs/refs/heads/u &&
test_must_fail git branch -m u v
'
+fi
test_expect_success 'test tracking setup via --track' '
git config remote.local.url . &&
@@ -556,9 +565,9 @@ EOF
test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
GIT_COMMITTER_DATE="2005-05-26 23:30" \
git checkout -b g/h/i -l master &&
- test_path_is_file .git/refs/heads/g/h/i &&
- test_path_is_file .git/logs/refs/heads/g/h/i &&
- test_cmp expect .git/logs/refs/heads/g/h/i
+ git rev-parse refs/heads/g/h/i &&
+ raw_reflog refs/heads/g/h/i > reflog &&
+ test_cmp expect reflog
'
test_expect_success 'checkout -b makes reflog by default' '
@@ -908,17 +917,17 @@ test_expect_success '--merged catches invalid object names' '
test_expect_success 'tracking with unexpected .fetch refspec' '
rm -rf a b c d &&
- git init a &&
+ git init $ref_storage_arg a &&
(
cd a &&
test_commit a
) &&
- git init b &&
+ git init $ref_storage_arg b &&
(
cd b &&
test_commit b
) &&
- git init c &&
+ git init $ref_storage_arg c &&
(
cd c &&
test_commit c &&
@@ -926,7 +935,7 @@ test_expect_success 'tracking with unexpected .fetch refspec' '
git remote add b ../b &&
git fetch --all
) &&
- git init d &&
+ git init $ref_storage_arg d &&
(
cd d &&
git remote add c ../c &&
diff --git a/t/t3210-pack-refs.sh b/t/t3210-pack-refs.sh
index 9b182a0..5d55be3 100755
--- a/t/t3210-pack-refs.sh
+++ b/t/t3210-pack-refs.sh
@@ -11,6 +11,13 @@ semantic is still the same.
'
. ./test-lib.sh
+backend=$ref_storage
+if test "$backend" = "lmdb"
+then
+ skip_all="The lmdb ref storage doesn't pack refs"
+ test_done
+fi
+
test_expect_success 'enable reflogs' '
git config core.logallrefupdates true
'
diff --git a/t/t3211-peel-ref.sh b/t/t3211-peel-ref.sh
index 3b7caca..84d1e8e 100755
--- a/t/t3211-peel-ref.sh
+++ b/t/t3211-peel-ref.sh
@@ -3,6 +3,12 @@
test_description='tests for the peel_ref optimization of packed-refs'
. ./test-lib.sh
+if test "$ref_storage" = "lmdb"
+then
+ skip_all="The lmdb ref storage doesn't pack refs"
+ test_done
+fi
+
test_expect_success 'create annotated tag in refs/tags' '
test_commit base &&
git tag -m annotated foo
diff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh
index 19aed7e..49148a2 100755
--- a/t/t3308-notes-merge.sh
+++ b/t/t3308-notes-merge.sh
@@ -79,7 +79,7 @@ test_expect_success 'fail to merge empty notes ref into empty notes ref (z => y)
test_expect_success 'fail to merge into various non-notes refs' '
test_must_fail git -c "core.notesRef=refs/notes" notes merge x &&
test_must_fail git -c "core.notesRef=refs/notes/" notes merge x &&
- mkdir -p .git/refs/notes/dir &&
+ git update-ref refs/notes/dir/test HEAD &&
test_must_fail git -c "core.notesRef=refs/notes/dir" notes merge x &&
test_must_fail git -c "core.notesRef=refs/notes/dir/" notes merge x &&
test_must_fail git -c "core.notesRef=refs/heads/master" notes merge x &&
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 544f9ad..5508eb0 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -608,7 +608,7 @@ test_expect_success 'submodule rebase setup' '
git checkout A &&
mkdir sub &&
(
- cd sub && git init && >elif &&
+ cd sub && git init $ref_storage_arg && >elif &&
git add elif && git commit -m "submodule initial"
) &&
echo 1 >file1 &&
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index d046d98..01d207e 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -220,7 +220,7 @@ test_expect_success 'Remove nonexistent file returns nonzero exit status' '
test_expect_success 'Call "rm" from outside the work tree' '
mkdir repo &&
(cd repo &&
- git init &&
+ git init $ref_storage_arg &&
echo something > somefile &&
git add somefile &&
git commit -m "add a file" &&
diff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh
index 8eb4794..e7ac52e 100755
--- a/t/t3800-mktag.sh
+++ b/t/t3800-mktag.sh
@@ -224,7 +224,7 @@ EOF
test_expect_success \
'allow empty tag email' \
- 'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+ 'write_ref refs/tags/mytag $(git mktag <tag.sig 2>message)'
############################################################
# 16. disallow spaces in tag email
@@ -352,7 +352,7 @@ EOF
test_expect_success \
'create valid tag' \
- 'git mktag <tag.sig >.git/refs/tags/mytag 2>message'
+ 'write_ref refs/tags/mytag $(git mktag <tag.sig 2>message)'
############################################################
# 25. check mytag
diff --git a/t/t3903-stash.sh b/t/t3903-stash.sh
index 2142c1f..1e3b50e 100755
--- a/t/t3903-stash.sh
+++ b/t/t3903-stash.sh
@@ -671,7 +671,7 @@ test_expect_success 'store updates stash ref and reflog' '
git reset --hard &&
! grep quux bazzy &&
git stash store -m quuxery $STASH_ID &&
- test $(cat .git/refs/stash) = $STASH_ID &&
+ test $(git rev-parse stash) = $STASH_ID &&
git reflog --format=%H stash| grep $STASH_ID &&
git stash pop &&
grep quux bazzy
diff --git a/t/t4010-diff-pathspec.sh b/t/t4010-diff-pathspec.sh
index 43c488b..76b3b64 100755
--- a/t/t4010-diff-pathspec.sh
+++ b/t/t4010-diff-pathspec.sh
@@ -112,7 +112,7 @@ test_expect_success 'diff-tree -r with wildcard' '
test_expect_success 'setup submodules' '
test_tick &&
- git init submod &&
+ git init $ref_storage_arg submod &&
( cd submod && test_commit first; ) &&
git add submod &&
git commit -m first &&
diff --git a/t/t4020-diff-external.sh b/t/t4020-diff-external.sh
index 0446201..b03b4f6 100755
--- a/t/t4020-diff-external.sh
+++ b/t/t4020-diff-external.sh
@@ -247,7 +247,7 @@ test_expect_success 'clean up crlf leftovers' '
'
test_expect_success 'submodule diff' '
- git init sub &&
+ git init $ref_storage_arg sub &&
( cd sub && test_commit sub1 ) &&
git add sub &&
test_tick &&
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index 518bf95..147351d 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -344,7 +344,7 @@ test_expect_success 'combined (empty submodule)' '
test_expect_success 'combined (with submodule)' '
rm -fr sub &&
- git clone --no-checkout . sub &&
+ git clone $ref_storage_arg --no-checkout . sub &&
git diff >actual &&
test_cmp expect.withsub actual
'
diff --git a/t/t4035-diff-quiet.sh b/t/t4035-diff-quiet.sh
index 461f4bb..553cabe 100755
--- a/t/t4035-diff-quiet.sh
+++ b/t/t4035-diff-quiet.sh
@@ -13,7 +13,7 @@ test_expect_success 'setup' '
git commit -a -m second &&
mkdir -p test-outside/repo && (
cd test-outside/repo &&
- git init &&
+ git init $ref_storage_arg &&
echo "1 1" >a &&
git add . &&
git commit -m 1
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 0ba8194..427b601 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -22,7 +22,7 @@ test_expect_success 'setup diff.submodule' '
test_commit one &&
INITIAL=$(git rev-parse HEAD) &&
- git init submodule &&
+ git init $ref_storage_arg submodule &&
(
cd submodule &&
test_commit two &&
diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
index 4b68bba..79249ef 100755
--- a/t/t5000-tar-tree.sh
+++ b/t/t5000-tar-tree.sh
@@ -190,7 +190,8 @@ test_expect_success \
test_expect_success \
'git get-tar-commit-id' \
'git get-tar-commit-id <b.tar >b.commitid &&
- test_cmp .git/$(git symbolic-ref HEAD) b.commitid'
+ raw_ref $(git symbolic-ref HEAD) > expect &&
+ test_cmp expect b.commitid'
test_expect_success 'git archive with --output, override inferred format' '
git archive --format=tar --output=d4.zip HEAD &&
diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh
index ad7ad2f..cfe8038 100755
--- a/t/t5304-prune.sh
+++ b/t/t5304-prune.sh
@@ -93,7 +93,7 @@ test_expect_success 'prune: prune nonsense parameters' '
test_expect_success 'prune: prune unreachable heads' '
git config core.logAllRefUpdates false &&
- mv .git/logs .git/logs.old &&
+ delete_all_reflogs &&
: > file2 &&
git add file2 &&
git commit -m temporary &&
diff --git a/t/t5312-prune-corruption.sh b/t/t5312-prune-corruption.sh
index da9d599..f118735 100755
--- a/t/t5312-prune-corruption.sh
+++ b/t/t5312-prune-corruption.sh
@@ -21,7 +21,7 @@ test_expect_success 'create history reachable only from a bogus-named ref' '
test_tick && git commit --allow-empty -m bogus &&
bogus=$(git rev-parse HEAD) &&
git cat-file commit $bogus >saved &&
- echo $bogus >.git/refs/heads/bogus..name &&
+ write_ref refs/heads/bogus..name $bogus &&
git reset --hard HEAD^
'
@@ -47,7 +47,7 @@ test_expect_success 'destructive repack keeps packed object' '
# subsequent tests will have different corruptions
test_expect_success 'clean up bogus ref' '
- rm .git/refs/heads/bogus..name
+ delete_ref refs/heads/bogus..name
'
# We create two new objects here, "one" and "two". Our
@@ -85,6 +85,13 @@ test_expect_success 'pack-refs does not silently delete broken loose ref' '
test_cmp expect actual
'
+backend=$ref_storage
+if test "$backend" = "lmdb"
+then
+ skip="The lmdb backend doesn't write a packed-refs file"
+ test_done
+fi
+
# we do not want to count on running pack-refs to
# actually pack it, as it is perfectly reasonable to
# skip processing a broken ref
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index e5f83bf..9b7aeac 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -30,7 +30,7 @@ add () {
test_tick &&
commit=$(echo "$text" | git commit-tree $tree $parents) &&
eval "$name=$commit; export $name" &&
- echo $commit > .git/refs/heads/$branch &&
+ git update-ref refs/heads/$branch $commit &&
eval ${branch}TIP=$commit
}
@@ -45,10 +45,10 @@ pull_to_client () {
case "$heads" in
*A*)
- echo $ATIP > .git/refs/heads/A;;
+ git update-ref refs/heads/A $ATIP;;
esac &&
case "$heads" in *B*)
- echo $BTIP > .git/refs/heads/B;;
+ git update-ref refs/heads/B $BTIP;;
esac &&
git symbolic-ref HEAD refs/heads/$(echo $heads \
| sed -e "s/^\(.\).*$/\1/") &&
@@ -92,8 +92,8 @@ test_expect_success 'setup' '
cur=$(($cur+1))
done &&
add B1 $A1 &&
- echo $ATIP > .git/refs/heads/A &&
- echo $BTIP > .git/refs/heads/B &&
+ git update-ref refs/heads/A $ATIP &&
+ git update-ref refs/heads/B $BTIP &&
git symbolic-ref HEAD refs/heads/B
'
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 0c10c85..9591271 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -28,20 +28,20 @@ test_expect_success setup '
git commit -a -m original'
test_expect_success "clone and setup child repos" '
- git clone . one &&
+ git clone $ref_storage_arg . one &&
(
cd one &&
echo >file updated by one &&
git commit -a -m "updated by one"
) &&
- git clone . two &&
+ git clone $ref_storage_arg . two &&
(
cd two &&
git config branch.master.remote one &&
git config remote.one.url ../one/.git/ &&
git config remote.one.fetch refs/heads/master:refs/heads/one
) &&
- git clone . three &&
+ git clone $ref_storage_arg . three &&
(
cd three &&
git config branch.master.remote two &&
@@ -53,8 +53,8 @@ test_expect_success "clone and setup child repos" '
echo "Pull: refs/heads/one:refs/heads/one"
} >.git/remotes/two
) &&
- git clone . bundle &&
- git clone . seven
+ git clone $ref_storage_arg . bundle &&
+ git clone $ref_storage_arg . seven
'
test_expect_success "fetch test" '
@@ -63,7 +63,7 @@ test_expect_success "fetch test" '
git commit -a -m "updated by origin" &&
cd two &&
git fetch &&
- test -f .git/refs/heads/one &&
+ git rev-parse --verify refs/heads/one &&
mine=$(git rev-parse refs/heads/one) &&
his=$(cd ../one && git rev-parse refs/heads/master) &&
test "z$mine" = "z$his"
@@ -73,8 +73,8 @@ test_expect_success "fetch test for-merge" '
cd "$D" &&
cd three &&
git fetch &&
- test -f .git/refs/heads/two &&
- test -f .git/refs/heads/one &&
+ git rev-parse --verify refs/heads/two &&
+ git rev-parse --verify refs/heads/one &&
master_in_two=$(cd ../two && git rev-parse master) &&
one_in_two=$(cd ../two && git rev-parse one) &&
{
@@ -180,7 +180,7 @@ test_expect_success 'fetch tags when there is no tags' '
mkdir notags &&
cd notags &&
- git init &&
+ git init $ref_storage_arg &&
git fetch -t ..
@@ -194,7 +194,7 @@ test_expect_success 'fetch following tags' '
mkdir four &&
cd four &&
- git init &&
+ git init $ref_storage_arg &&
git fetch .. :track &&
git show-ref --verify refs/tags/anno &&
@@ -204,7 +204,7 @@ test_expect_success 'fetch following tags' '
test_expect_success 'fetch uses remote ref names to describe new refs' '
cd "$D" &&
- git init descriptive &&
+ git init $ref_storage_arg descriptive &&
(
cd descriptive &&
git config remote.o.url .. &&
@@ -238,7 +238,7 @@ test_expect_success 'fetch must not resolve short tag name' '
mkdir five &&
cd five &&
- git init &&
+ git init $ref_storage_arg &&
test_must_fail git fetch .. anno:five
@@ -251,7 +251,7 @@ test_expect_success 'fetch can now resolve short remote name' '
mkdir six &&
cd six &&
- git init &&
+ git init $ref_storage_arg &&
git fetch .. six:six
'
@@ -529,7 +529,7 @@ test_expect_success "should be able to fetch with duplicate refspecs" '
mkdir dups &&
(
cd dups &&
- git init &&
+ git init $ref_storage_arg &&
git config branch.master.remote three &&
git config remote.three.url ../three/.git &&
git config remote.three.fetch +refs/heads/*:refs/remotes/origin/* &&
@@ -665,7 +665,7 @@ test_expect_success 'fetching a one-level ref works' '
test_commit extra &&
git reset --hard HEAD^ &&
git update-ref refs/foo extra &&
- git init one-level &&
+ git init $ref_storage_arg one-level &&
(
cd one-level &&
git fetch .. HEAD refs/foo
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 954d0e4..0fef8ac 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -38,7 +38,7 @@ test_expect_success setup '
mkdir deepsubmodule &&
(
cd deepsubmodule &&
- git init &&
+ git init $ref_storage_arg &&
echo deepsubcontent > deepsubfile &&
git add deepsubfile &&
git commit -m new deepsubfile
@@ -46,7 +46,7 @@ test_expect_success setup '
mkdir submodule &&
(
cd submodule &&
- git init &&
+ git init $ref_storage_arg &&
echo subcontent > subfile &&
git add subfile &&
git submodule add "$pwd/deepsubmodule" subdir/deepsubmodule &&
diff --git a/t/t5527-fetch-odd-refs.sh b/t/t5527-fetch-odd-refs.sh
index 207899a..b9ecfa0 100755
--- a/t/t5527-fetch-odd-refs.sh
+++ b/t/t5527-fetch-odd-refs.sh
@@ -26,6 +26,13 @@ test_expect_success 'suffix ref is ignored during fetch' '
test_cmp expect actual
'
+backend=$ref_storage
+if test "$backend" = "lmdb"
+then
+ skip="The lmdb backend doesn't do crazy-long refs"
+ test_done
+fi
+
test_expect_success 'try to create repo with absurdly long refname' '
ref240=$_z40/$_z40/$_z40/$_z40/$_z40/$_z40 &&
ref1440=$ref240/$ref240/$ref240/$ref240/$ref240/$ref240 &&
diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
index df8d2f0..114d25a 100755
--- a/t/t5537-fetch-shallow.sh
+++ b/t/t5537-fetch-shallow.sh
@@ -173,6 +173,13 @@ EOF
)
'
+backend=$ref_storage
+if test "$backend" = "lmdb"
+then
+ skip="The lmdb backend needs write access for its locks"
+ test_done
+fi
+
test_expect_success POSIXPERM,SANITY 'shallow fetch from a read-only repo' '
cp -R .git read-only.git &&
find read-only.git -print | xargs chmod -w &&
diff --git a/t/t5700-clone-reference.sh b/t/t5700-clone-reference.sh
index 4320082..49e7868 100755
--- a/t/t5700-clone-reference.sh
+++ b/t/t5700-clone-reference.sh
@@ -34,14 +34,14 @@ test_expect_success 'preparing first repository' '
'
test_expect_success 'preparing second repository' '
- git clone A B &&
+ git clone $ref_storage_arg A B &&
commit_in B file2 &&
git -C B repack -ad &&
git -C B prune
'
test_expect_success 'cloning with reference (-l -s)' '
- git clone -l -s --reference B A C
+ git clone $ref_storage_arg -l -s --reference B A C
'
test_expect_success 'existence of info/alternates' '
@@ -57,7 +57,7 @@ test_expect_success 'that reference gets used' '
'
test_expect_success 'cloning with reference (no -l -s)' '
- GIT_TRACE_PACKET=$U.D git clone --reference B "file://$(pwd)/A" D
+ GIT_TRACE_PACKET=$U.D git clone $ref_storage_arg --reference B "file://$(pwd)/A" D
'
test_expect_success 'fetched no objects' '
@@ -108,20 +108,20 @@ test_expect_success 'preparing alternate repository #1' '
'
test_expect_success 'cloning alternate repo #2 and adding changes to repo #1' '
- git clone F G &&
+ git clone $ref_storage_arg F G &&
commit_in F file2
'
test_expect_success 'cloning alternate repo #1, using #2 as reference' '
- git clone --reference G F H
+ git clone $ref_storage_arg --reference G F H
'
test_expect_success 'cloning with reference being subset of source (-l -s)' '
- git clone -l -s --reference A B E
+ git clone $ref_storage_arg -l -s --reference A B E
'
test_expect_success 'cloning with multiple references drops duplicates' '
- git clone -s --reference B --reference A --reference B A dups &&
+ git clone $ref_storage_arg -s --reference B --reference A --reference B A dups &&
test_line_count = 2 dups/.git/objects/info/alternates
'
@@ -129,11 +129,11 @@ test_expect_success 'clone with reference from a tagged repository' '
(
cd A && git tag -a -m tagged HEAD
) &&
- git clone --reference=A A I
+ git clone $ref_storage_arg --reference=A A I
'
test_expect_success 'prepare branched repository' '
- git clone A J &&
+ git clone $ref_storage_arg A J &&
(
cd J &&
git checkout -b other master^ &&
@@ -145,7 +145,7 @@ test_expect_success 'prepare branched repository' '
'
test_expect_success 'fetch with incomplete alternates' '
- git init K &&
+ git init $ref_storage_arg K &&
echo "$base_dir/A/.git/objects" >K/.git/objects/info/alternates &&
(
cd K &&
@@ -160,29 +160,29 @@ test_expect_success 'fetch with incomplete alternates' '
'
test_expect_success 'clone using repo with gitfile as a reference' '
- git clone --separate-git-dir=L A M &&
- git clone --reference=M A N &&
+ git clone $ref_storage_arg --separate-git-dir=L A M &&
+ git clone $ref_storage_arg --reference=M A N &&
echo "$base_dir/L/objects" >expected &&
test_cmp expected "$base_dir/N/.git/objects/info/alternates"
'
test_expect_success 'clone using repo pointed at by gitfile as reference' '
- git clone --reference=M/.git A O &&
+ git clone $ref_storage_arg --reference=M/.git A O &&
echo "$base_dir/L/objects" >expected &&
test_cmp expected "$base_dir/O/.git/objects/info/alternates"
'
test_expect_success 'clone and dissociate from reference' '
- git init P &&
+ git init $ref_storage_arg P &&
(
cd P && test_commit one
) &&
- git clone P Q &&
+ git clone $ref_storage_arg P Q &&
(
cd Q && test_commit two
) &&
- git clone --no-local --reference=P Q R &&
- git clone --no-local --reference=P --dissociate Q S &&
+ git clone $ref_storage_arg --no-local --reference=P Q R &&
+ git clone $ref_storage_arg --no-local --reference=P --dissociate Q S &&
# removing the reference P would corrupt R but not S
rm -fr P &&
test_must_fail git -C R fsck &&
@@ -198,14 +198,14 @@ test_expect_success 'clone, dissociate from partial reference and repack' '
test_commit two &&
git repack
) &&
- git clone --bare P Q &&
+ git clone $ref_storage_arg --bare P Q &&
(
cd P &&
git checkout -b second &&
test_commit three &&
git repack
) &&
- git clone --bare --dissociate --reference=P Q R &&
+ git clone $ref_storage_arg --bare --dissociate --reference=P Q R &&
ls R/objects/pack/*.pack >packs.txt &&
test_line_count = 1 packs.txt
'
@@ -214,9 +214,9 @@ test_expect_success 'clone, dissociate from alternates' '
rm -fr A B C &&
test_create_repo A &&
commit_in A file1 &&
- git clone --reference=A A B &&
+ git clone $ref_storage_arg --reference=A A B &&
test_line_count = 1 B/.git/objects/info/alternates &&
- git clone --local --dissociate B C &&
+ git clone $ref_storage_arg --local --dissociate B C &&
! test -f C/.git/objects/info/alternates &&
( cd C && git fsck )
'
diff --git a/t/t6001-rev-list-graft.sh b/t/t6001-rev-list-graft.sh
index 05ddc69..0c009ab 100755
--- a/t/t6001-rev-list-graft.sh
+++ b/t/t6001-rev-list-graft.sh
@@ -20,7 +20,8 @@ test_expect_success setup '
git commit -a -m "Third in one history." &&
A2=$(git rev-parse --verify HEAD) &&
- rm -f .git/refs/heads/master .git/index &&
+ delete_ref refs/heads/master &&
+ rm -f .git/index &&
echo >fileA fileA again &&
echo >subdir/fileB fileB again &&
diff --git a/t/t6010-merge-base.sh b/t/t6010-merge-base.sh
index 39b3238..b2003c5 100755
--- a/t/t6010-merge-base.sh
+++ b/t/t6010-merge-base.sh
@@ -34,7 +34,7 @@ doit () {
commit=$(echo $NAME | git commit-tree $T $PARENTS) &&
- echo $commit >.git/refs/tags/$NAME &&
+ git update-ref refs/tags/$NAME $commit &&
echo $commit
}
diff --git a/t/t6050-replace.sh b/t/t6050-replace.sh
index c630aba..60f0011 100755
--- a/t/t6050-replace.sh
+++ b/t/t6050-replace.sh
@@ -124,13 +124,13 @@ tagger T A Gger <> 0 +0000
EOF
test_expect_success 'tag replaced commit' '
- git mktag <tag.sig >.git/refs/tags/mytag 2>message
+ write_ref refs/tags/mytag $(git mktag <tag.sig 2>message)
'
test_expect_success '"git fsck" works' '
git fsck master >fsck_master.out &&
grep "dangling commit $R" fsck_master.out &&
- grep "dangling tag $(cat .git/refs/tags/mytag)" fsck_master.out &&
+ grep "dangling tag $(raw_ref refs/tags/mytag)" fsck_master.out &&
test -z "$(git fsck)"
'
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 85f2694..ca30c18 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -128,7 +128,8 @@ test_expect_success 'no warning was displayed for A' '
'
test_expect_success 'rename tag A to Q locally' '
- mv .git/refs/tags/A .git/refs/tags/Q
+ write_ref refs/tags/Q $(raw_ref refs/tags/A) &&
+ delete_ref refs/tags/A
'
cat - >err.expect <<EOF
warning: tag 'A' is really 'Q' here
@@ -138,7 +139,8 @@ test_expect_success 'warning was displayed for Q' '
test_i18ncmp err.expect err.actual
'
test_expect_success 'rename tag Q back to A' '
- mv .git/refs/tags/Q .git/refs/tags/A
+ write_ref refs/tags/A $(raw_ref refs/tags/Q) &&
+ delete_ref refs/tags/Q
'
test_expect_success 'pack tag refs' 'git pack-refs'
diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index cdb67a0..f7ba4ca 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -16,8 +16,8 @@ test_expect_success setup '
test_expect_success 'Broken refs are reported correctly' '
r=refs/heads/bogus &&
- : >.git/$r &&
- test_when_finished "rm -f .git/$r" &&
+ write_ref $r '' &&
+ test_when_finished "delete_ref $r" &&
echo "warning: ignoring broken ref $r" >broken-err &&
git for-each-ref >out 2>err &&
test_cmp full-list out &&
@@ -26,8 +26,8 @@ test_expect_success 'Broken refs are reported correctly' '
test_expect_success 'NULL_SHA1 refs are reported correctly' '
r=refs/heads/zeros &&
- echo $ZEROS >.git/$r &&
- test_when_finished "rm -f .git/$r" &&
+ write_ref $r $ZEROS &&
+ test_when_finished "delete_ref $r" &&
echo "warning: ignoring broken ref $r" >zeros-err &&
git for-each-ref >out 2>err &&
test_cmp full-list out &&
@@ -39,8 +39,8 @@ test_expect_success 'NULL_SHA1 refs are reported correctly' '
test_expect_success 'Missing objects are reported correctly' '
r=refs/heads/missing &&
- echo $MISSING >.git/$r &&
- test_when_finished "rm -f .git/$r" &&
+ write_ref $r $MISSING &&
+ test_when_finished "delete_ref $r" &&
echo "fatal: missing object $MISSING for $r" >missing-err &&
test_must_fail git for-each-ref 2>err &&
test_cmp missing-err err &&
diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index 8859236..ab1fb99 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -65,7 +65,7 @@ test_expect_success setup '
test_expect_success "checkout from non-existing branch" '
git checkout -b delete-me master &&
- rm .git/refs/heads/delete-me &&
+ git update-ref -d --no-deref refs/heads/delete-me &&
test refs/heads/delete-me = "$(git symbolic-ref HEAD)" &&
git checkout master &&
test refs/heads/master = "$(git symbolic-ref HEAD)"
diff --git a/t/t7300-clean.sh b/t/t7300-clean.sh
index 86ceb38..36832ec 100755
--- a/t/t7300-clean.sh
+++ b/t/t7300-clean.sh
@@ -431,7 +431,7 @@ test_expect_success 'nested git work tree' '
mkdir -p foo bar baz/boo &&
(
cd foo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit nested hello.world
) &&
(
@@ -440,7 +440,7 @@ test_expect_success 'nested git work tree' '
) &&
(
cd baz/boo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit deeply.nested deeper.world
) &&
git clean -f -d &&
@@ -455,6 +455,7 @@ test_expect_success 'should clean things that almost look like git but are not'
rm -fr almost_git almost_bare_git almost_submodule &&
mkdir -p almost_git/.git/objects &&
mkdir -p almost_git/.git/refs &&
+ mkdir -p almost_git/.git/refs.lmdb &&
cat >almost_git/.git/HEAD <<-\EOF &&
garbage
EOF
@@ -475,7 +476,7 @@ test_expect_success 'should not clean submodules' '
mkdir repo to_clean &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit msg hello.world
) &&
git submodule add ./repo/.git sub1 &&
@@ -511,7 +512,7 @@ test_expect_success POSIXPERM 'should avoid cleaning possible submodules' '
test_expect_success 'nested (empty) git should be kept' '
rm -fr empty_repo to_clean &&
- git init empty_repo &&
+ git init $ref_storage_arg empty_repo &&
mkdir to_clean &&
>to_clean/should_clean.this &&
git clean -f -d &&
@@ -521,7 +522,7 @@ test_expect_success 'nested (empty) git should be kept' '
test_expect_success 'nested bare repositories should be cleaned' '
rm -fr bare1 bare2 subdir &&
- git init --bare bare1 &&
+ git init $ref_storage_arg --bare bare1 &&
git clone --local --bare . bare2 &&
mkdir subdir &&
cp -r bare2 subdir/bare3 &&
@@ -534,7 +535,7 @@ test_expect_success 'nested bare repositories should be cleaned' '
test_expect_failure 'nested (empty) bare repositories should be cleaned even when in .git' '
rm -fr strange_bare &&
mkdir strange_bare &&
- git init --bare strange_bare/.git &&
+ git init $ref_storage_arg --bare strange_bare/.git &&
git clean -f -d &&
test_path_is_missing strange_bare
'
@@ -552,7 +553,7 @@ test_expect_success 'giving path in nested git work tree will remove it' '
mkdir repo &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
mkdir -p bar/baz &&
test_commit msg bar/baz/hello.world
) &&
@@ -567,7 +568,7 @@ test_expect_success 'giving path to nested .git will not remove it' '
mkdir repo untracked &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit msg hello.world
) &&
git clean -f -d repo/.git &&
@@ -582,7 +583,7 @@ test_expect_success 'giving path to nested .git/ will remove contents' '
mkdir repo untracked &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit msg hello.world
) &&
git clean -f -d repo/.git/ &&
@@ -596,7 +597,7 @@ test_expect_success 'force removal of nested git work tree' '
mkdir -p foo bar baz/boo &&
(
cd foo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit nested hello.world
) &&
(
@@ -605,7 +606,7 @@ test_expect_success 'force removal of nested git work tree' '
) &&
(
cd baz/boo &&
- git init &&
+ git init $ref_storage_arg &&
test_commit deeply.nested deeper.world
) &&
git clean -f -f -d &&
@@ -619,7 +620,7 @@ test_expect_success 'git clean -e' '
mkdir repo &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
touch known 1 2 3 &&
git add known &&
git clean -f -e 1 -e 2 &&
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5991e3c..131a23f 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -32,7 +32,7 @@ test_expect_success 'setup - repository in init subdirectory' '
mkdir init &&
(
cd init &&
- git init &&
+ git init $ref_storage_arg &&
echo a >a &&
git add a &&
git commit -m "submodule commit 1" &&
@@ -52,8 +52,8 @@ test_expect_success 'setup - hide init subdirectory' '
'
test_expect_success 'setup - repository to add submodules to' '
- git init addtest &&
- git init addtest-ignore
+ git init $ref_storage_arg addtest &&
+ git init $ref_storage_arg addtest-ignore
'
# The 'submodule add' tests need some repository to add as a submodule.
@@ -540,7 +540,7 @@ test_expect_success 'add submodules without specifying an explicit path' '
mkdir repo &&
(
cd repo &&
- git init &&
+ git init $ref_storage_arg &&
echo r >r &&
git add r &&
git commit -m "repo commit 1"
@@ -585,11 +585,11 @@ test_expect_success 'set up for relative path tests' '
mkdir reltest &&
(
cd reltest &&
- git init &&
+ git init $ref_storage_arg &&
mkdir sub &&
(
cd sub &&
- git init &&
+ git init $ref_storage_arg &&
test_commit foo
) &&
git add sub &&
@@ -754,7 +754,7 @@ test_expect_success '../bar/a/b/c works with relative local path - ../foo/bar.gi
cp pristine-.git-config .git/config &&
cp pristine-.gitmodules .gitmodules &&
mkdir -p a/b/c &&
- (cd a/b/c; git init) &&
+ (cd a/b/c; git init $ref_storage_arg) &&
git config remote.origin.url ../foo/bar.git &&
git submodule add ../bar/a/b/c ./a/b/c &&
git submodule init &&
@@ -975,7 +975,7 @@ test_expect_success 'submodule with UTF-8 name' '
mkdir "$svname" &&
(
cd "$svname" &&
- git init &&
+ git init $ref_storage_arg &&
>sub &&
git add sub &&
git commit -m "init sub"
@@ -990,7 +990,7 @@ test_expect_success 'submodule add clone shallow submodule' '
pwd=$(pwd) &&
(
cd super &&
- git init &&
+ git init $ref_storage_arg &&
git submodule add --depth=1 file://"$pwd"/example2 submodule &&
(
cd submodule &&
diff --git a/t/t7402-submodule-rebase.sh b/t/t7402-submodule-rebase.sh
index 8e32f19..85166da 100755
--- a/t/t7402-submodule-rebase.sh
+++ b/t/t7402-submodule-rebase.sh
@@ -13,7 +13,7 @@ test_expect_success setup '
git add file &&
test_tick &&
git commit -m initial &&
- git clone . submodule &&
+ git clone $ref_storage_arg . submodule &&
git add submodule &&
test_tick &&
git commit -m submodule &&
diff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh
index 0d5b42a..9d147df 100755
--- a/t/t7405-submodule-merge.sh
+++ b/t/t7405-submodule-merge.sh
@@ -18,7 +18,7 @@ test_expect_success setup '
mkdir sub &&
(cd sub &&
- git init &&
+ git init $ref_storage_arg &&
echo original > file &&
git add file &&
test_tick &&
@@ -68,10 +68,10 @@ test_expect_success setup '
test_expect_success 'setup for merge search' '
mkdir merge-search &&
(cd merge-search &&
- git init &&
+ git init $ref_storage_arg &&
mkdir sub &&
(cd sub &&
- git init &&
+ git init $ref_storage_arg &&
echo "file-a" > file-a &&
git add file-a &&
git commit -m "sub-a" &&
@@ -232,10 +232,10 @@ test_expect_success 'merging with a modify/modify conflict between merge bases'
test_expect_success 'setup for recursive merge with submodule' '
mkdir merge-recursive &&
(cd merge-recursive &&
- git init &&
+ git init $ref_storage_arg &&
mkdir sub &&
(cd sub &&
- git init &&
+ git init $ref_storage_arg &&
test_commit a &&
git checkout -b sub-b master &&
test_commit b &&
diff --git a/t/t9104-git-svn-follow-parent.sh b/t/t9104-git-svn-follow-parent.sh
index cd480ed..0c9c8f9 100755
--- a/t/t9104-git-svn-follow-parent.sh
+++ b/t/t9104-git-svn-follow-parent.sh
@@ -213,7 +213,8 @@ test_expect_success "multi-fetch continues to work" "
"
test_expect_success "multi-fetch works off a 'clean' repository" '
- rm -r "$GIT_DIR/svn" "$GIT_DIR/refs/remotes" "$GIT_DIR/logs" &&
+ rm -rf "$GIT_DIR/svn" "$GIT_DIR/refs/remotes" &&
+ git reflog expire --all --expire=all &&
mkdir "$GIT_DIR/svn" &&
git svn multi-fetch
'
diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh
index 6a48e40..83ea46b 100755
--- a/t/t9115-git-svn-dcommit-funky-renames.sh
+++ b/t/t9115-git-svn-dcommit-funky-renames.sh
@@ -60,7 +60,7 @@ test_expect_success 'add a file with plus signs' '
'
test_expect_success 'clone the repository to test rebase' '
- git svn clone "$svnrepo" test-rebase &&
+ git svn clone $ref_storage_arg "$svnrepo" test-rebase &&
(
cd test-rebase &&
echo test-rebase >test-rebase &&
diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index b5149fd..b36008c 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -42,7 +42,7 @@ test_expect_success 'fast-export | fast-import' '
WER=$(git rev-parse --verify wer) &&
MUSS=$(git rev-parse --verify muss) &&
mkdir new &&
- git --git-dir=new/.git init &&
+ git --git-dir=new/.git init $ref_storage_arg &&
git fast-export --all |
(cd new &&
git fast-import &&
@@ -158,7 +158,7 @@ test_expect_success 'setup submodule' '
mkdir sub &&
(
cd sub &&
- git init &&
+ git init $ref_storage_arg &&
echo test file > file &&
git add file &&
git commit -m sub_initial
@@ -183,7 +183,7 @@ test_expect_success 'submodule fast-export | fast-import' '
SUBENT2=$(git ls-tree master sub) &&
rm -rf new &&
mkdir new &&
- git --git-dir=new/.git init &&
+ git --git-dir=new/.git init $ref_storage_arg &&
git fast-export --signed-tags=strip --all |
(cd new &&
git fast-import &&
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 2ba62fb..d5751e2 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -126,7 +126,7 @@ actual="$TRASH_DIRECTORY/actual"
test_expect_success 'setup for __gitdir tests' '
mkdir -p subdir/subsubdir &&
- git init otherrepo
+ git init $ref_storage_arg otherrepo
'
test_expect_success '__gitdir - from command line (through $__git_dir)' '
@@ -177,6 +177,7 @@ test_expect_success '__gitdir - cwd is a .git directory' '
test_expect_success '__gitdir - parent is a .git directory' '
echo "$(pwd -P)/.git" >expected &&
(
+ mkdir -p .git/refs/heads &&
cd .git/refs/heads &&
__gitdir >"$actual"
) &&
diff --git a/t/t9903-bash-prompt.sh b/t/t9903-bash-prompt.sh
index ffbfa0e..caa706c 100755
--- a/t/t9903-bash-prompt.sh
+++ b/t/t9903-bash-prompt.sh
@@ -148,6 +148,7 @@ test_expect_success 'prompt - inside .git directory' '
test_expect_success 'prompt - deep inside .git directory' '
printf " (GIT_DIR!)" >expected &&
(
+ mkdir -p .git/refs/heads &&
cd .git/refs/heads &&
__git_ps1 >"$actual"
) &&
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index c64e5a5..9a8bb72 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -783,8 +783,8 @@ test_create_repo () {
repo="$1"
mkdir -p "$repo"
(
- cd "$repo" || error "Cannot setup test environment"
- "$GIT_EXEC_PATH/git-init" "--template=$GIT_BUILD_DIR/templates/blt/" >&3 2>&4 ||
+ cd "$repo" || error "Cannot setup test environment"
+ "$GIT_EXEC_PATH/git-init" $ref_storage_arg "--template=$GIT_BUILD_DIR/templates/blt/" >&3 2>&4 ||
error "cannot run git init -- have you built things yet?"
mv .git/hooks .git/hooks-disabled
) || exit
@@ -941,3 +941,52 @@ mingw_read_file_strip_cr_ () {
eval "$1=\$$1\$line"
done
}
+
+raw_ref() {
+ if test $ref_storage = "lmdb"
+ then
+ test-refs-lmdb-backend "$1" 2>/dev/null
+ else
+ cat ".git/$1"
+ fi
+}
+
+delete_ref() {
+ if test $ref_storage = "lmdb"
+ then
+ test-refs-lmdb-backend -d "$1" 2>/dev/null
+ else
+ rm ".git/$1"
+ fi
+}
+
+write_ref() {
+ if test $ref_storage = "lmdb"
+ then
+ test-refs-lmdb-backend "$1" "$2" 2>/dev/null
+ else
+ echo "$2" > ".git/$1"
+ fi
+}
+
+raw_reflog() {
+ if test $ref_storage = "lmdb"
+ then
+ test-refs-lmdb-backend -l "$1" 2>/dev/null
+ else
+ cat ".git/logs/$1"
+ fi
+}
+
+delete_all_reflogs() {
+ if test $ref_storage = "lmdb"
+ then
+ test-refs-lmdb-backend -c
+ fi
+ # We have to do this in any case to handle logs for per-worktree refs
+ rm -rf .git/logs
+}
+
+append_reflog() {
+ test-refs-lmdb-backend -a "$1"
+}
diff --git a/t/test-lib.sh b/t/test-lib.sh
index ce40770..e736554 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -186,6 +186,8 @@ test "x$TERM" != "xdumb" && (
) &&
color=t
+ref_storage="files"
+ref_storage_arg=
while test "$#" -ne 0
do
case "$1" in
@@ -201,6 +203,9 @@ do
exit 1;
}
run_list=$1; shift ;;
+ --ref-storage=*)
+ GIT_TEST_REF_STORAGE=$(expr "z$1" : 'z[^=]*=\(.*\)'); shift
+ ;;
--run=*)
run_list=$(expr "z$1" : 'z[^=]*=\(.*\)'); shift ;;
-h|--h|--he|--hel|--help)
@@ -247,6 +252,12 @@ do
esac
done
+if test -n "$GIT_TEST_REF_STORAGE"
+then
+ ref_storage=$GIT_TEST_REF_STORAGE
+ ref_storage_arg="--ref-storage=$ref_storage"
+fi
+
if test -n "$valgrind_only"
then
test -z "$valgrind" && valgrind=memcheck
--
2.4.2.767.g62658d5-twtrsrc
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [PATCH v6 32/32] tests: add ref-storage argument
2016-02-24 22:59 ` [PATCH v6 32/32] tests: add ref-storage argument David Turner
@ 2016-02-29 22:52 ` SZEDER Gábor
2016-02-29 23:50 ` David Turner
0 siblings, 1 reply; 56+ messages in thread
From: SZEDER Gábor @ 2016-02-29 22:52 UTC (permalink / raw)
To: David Turner; +Cc: SZEDER Gábor, git, mhagger, pclouds
Hi,
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 2ba62fb..d5751e2 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -126,7 +126,7 @@ actual="$TRASH_DIRECTORY/actual"
>
> test_expect_success 'setup for __gitdir tests' '
> mkdir -p subdir/subsubdir &&
> - git init otherrepo
> + git init $ref_storage_arg otherrepo
> '
>
> test_expect_success '__gitdir - from command line (through $__git_dir)' '
> @@ -177,6 +177,7 @@ test_expect_success '__gitdir - cwd is a .git directory' '
> test_expect_success '__gitdir - parent is a .git directory' '
> echo "$(pwd -P)/.git" >expected &&
> (
> + mkdir -p .git/refs/heads &&
> cd .git/refs/heads &&
> __gitdir >"$actual"
> ) &&
> diff --git a/t/t9903-bash-prompt.sh b/t/t9903-bash-prompt.sh
> index ffbfa0e..caa706c 100755
> --- a/t/t9903-bash-prompt.sh
> +++ b/t/t9903-bash-prompt.sh
> @@ -148,6 +148,7 @@ test_expect_success 'prompt - inside .git directory' '
> test_expect_success 'prompt - deep inside .git directory' '
> printf " (GIT_DIR!)" >expected &&
> (
> + mkdir -p .git/refs/heads &&
> cd .git/refs/heads &&
> __git_ps1 >"$actual"
> ) &&
As far as these tests are concerned any subdirectory under .git/ would
do, it doesn't have to be .git/refs/heads. How about cd-ing into a
different directory instead of creating the missing ones?
---- >8 ----
Subject: [PATCH] fixup! tests: add ref-storage argument
---
t/t9902-completion.sh | 3 +--
t/t9903-bash-prompt.sh | 3 +--
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index d5751e25eb50..605816a9b996 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -177,8 +177,7 @@ test_expect_success '__gitdir - cwd is a .git directory' '
test_expect_success '__gitdir - parent is a .git directory' '
echo "$(pwd -P)/.git" >expected &&
(
- mkdir -p .git/refs/heads &&
- cd .git/refs/heads &&
+ cd .git/objects &&
__gitdir >"$actual"
) &&
test_cmp expected "$actual"
diff --git a/t/t9903-bash-prompt.sh b/t/t9903-bash-prompt.sh
index 683b4705167f..6068ccb92eb9 100755
--- a/t/t9903-bash-prompt.sh
+++ b/t/t9903-bash-prompt.sh
@@ -148,8 +148,7 @@ test_expect_success 'prompt - inside .git directory' '
test_expect_success 'prompt - deep inside .git directory' '
printf " (GIT_DIR!)" >expected &&
(
- mkdir -p .git/refs/heads &&
- cd .git/refs/heads &&
+ cd .git/objects &&
__git_ps1 >"$actual"
) &&
test_cmp expected "$actual"
--
2.7.2.410.g92cb358
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PATCH v6 00/32] refs backend
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (31 preceding siblings ...)
2016-02-24 22:59 ` [PATCH v6 32/32] tests: add ref-storage argument David Turner
@ 2016-02-25 0:08 ` Ramsay Jones
2016-02-25 19:32 ` David Turner
2016-02-25 12:57 ` Duy Nguyen
33 siblings, 1 reply; 56+ messages in thread
From: Ramsay Jones @ 2016-02-25 0:08 UTC (permalink / raw)
To: David Turner, git, mhagger, pclouds
On 24/02/16 22:58, David Turner wrote:
> This version incorporates fixes for function scope from Ramsay Jones.
>
> It breaks patches down a bit more, to make them easier to review and
> understand. Michael Haggerty had some suggestions here.
>
> As suggested by Duy Nguyen, it replaces the resolve_ref_unsafe backend
> method with read_raw_ref, moving resolve_ref_unsafe to the common
> code.
>
> It adds support for running tests under alternate ref backends.
>
> It fixes two indentation errors, one reported by Eric Wong.
>
> It updates the documentation to include info about a few more
> limitations on the LMDB backend. It incorporates other LMDB-related
> suggestions from Duy Nguyen, e.g. using MDB_NOSUBDIR.
>
> There's a fix for non-normal ref renames, also from Duy Nguyen.
>
> Symbolic ref splitting has a bug fix, suggested by Michael Haggerty.
>
> There are other, more minor changes suggested by Michael Haggerty and
> Duy Nguyen.
>
>
> David Turner (29):
> refs: move head_ref{,_submodule} to the common code
> refs: move for_each_*ref* functions into common code
> files-backend: break out ref reading
> refs: move resolve_ref_unsafe into common code
> refs: add method for do_for_each_ref
> refs: add do_for_each_per_worktree_ref
> refs: add methods for reflog
> refs: add method for initial ref transaction commit
> refs: add method for delete_refs
> refs: add methods to init refs db
> refs: add method to rename refs
> refs: handle non-normal ref renames
> refs: make lock generic
> refs: move duplicate check to common code
> refs: allow log-only updates
> refs: don't dereference on rename
> refs: on symref reflog expire, lock symref not referrent
> refs: resolve symbolic refs first
> refs: always handle non-normal refs in files backend
> init: allow alternate ref strorage to be set for new repos
> refs: check submodules' ref storage config
> clone: allow ref storage backend to be set for clone
> svn: learn ref-storage argument
> refs: register ref storage backends
> config: read ref storage config on startup
> refs: break out resolve_ref_unsafe_submodule
> refs: add LMDB refs storage backend
> refs: tests for lmdb backend
> tests: add ref-storage argument
>
> Ramsay Jones (1):
> refs: reduce the visibility of do_for_each_ref()
Ah, sorry if it wasn't clear, but there is no need to keep this
as a separate patch - it should simply be squashed into the relevant
patch in your series.
ATB,
Ramsay Jones
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 00/32] refs backend
2016-02-25 0:08 ` [PATCH v6 00/32] refs backend Ramsay Jones
@ 2016-02-25 19:32 ` David Turner
0 siblings, 0 replies; 56+ messages in thread
From: David Turner @ 2016-02-25 19:32 UTC (permalink / raw)
To: Ramsay Jones, git, mhagger, pclouds
On Thu, 2016-02-25 at 00:08 +0000, Ramsay Jones wrote:
> > Ramsay Jones (1):
> > refs: reduce the visibility of do_for_each_ref()
>
> Ah, sorry if it wasn't clear, but there is no need to keep this
> as a separate patch - it should simply be squashed into the relevant
> patch in your series.
>
> ATB,
> Ramsay Jones
It actually seemed cleaner to do it this way for some reason.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 00/32] refs backend
2016-02-24 22:58 [PATCH v6 00/32] refs backend David Turner
` (32 preceding siblings ...)
2016-02-25 0:08 ` [PATCH v6 00/32] refs backend Ramsay Jones
@ 2016-02-25 12:57 ` Duy Nguyen
2016-02-25 19:27 ` David Turner
33 siblings, 1 reply; 56+ messages in thread
From: Duy Nguyen @ 2016-02-25 12:57 UTC (permalink / raw)
To: David Turner; +Cc: Git Mailing List, Michael Haggerty
A couple of build warnings I found, haven't really read the code yet.
These two can easily be fixed
refs/lmdb-backend.c: In function 'lmdb_read_raw_ref':
refs/lmdb-backend.c:554:16: warning: unused variable 'err' [-Wunused-variable]
struct strbuf err = STRBUF_INIT;
^
refs/lmdb-backend.c: In function 'lmdb_do_for_each_ref':
refs/lmdb-backend.c:1625:15: warning: unused variable 'c' [-Wunused-variable]
const char *c = resolve_ref_unsafe_submodule(submodule, refname, 0, oid.hash,
^
-Wshadow also gives a bunch of warnings, mostly about "transaction"
and "env". Whether you want to fix them is really up to you.
--
Duy
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [PATCH v6 00/32] refs backend
2016-02-25 12:57 ` Duy Nguyen
@ 2016-02-25 19:27 ` David Turner
2016-02-25 19:31 ` Stefan Beller
2016-02-25 20:31 ` Duy Nguyen
0 siblings, 2 replies; 56+ messages in thread
From: David Turner @ 2016-02-25 19:27 UTC (permalink / raw)
To: Duy Nguyen; +Cc: Git Mailing List, Michael Haggerty
On Thu, 2016-02-25 at 19:57 +0700, Duy Nguyen wrote:
> A couple of build warnings I found, haven't really read the code yet.
> These two can easily be fixed
>
> refs/lmdb-backend.c: In function 'lmdb_read_raw_ref':
> refs/lmdb-backend.c:554:16: warning: unused variable 'err' [-Wunused
> -variable]
> struct strbuf err = STRBUF_INIT;
> ^
> refs/lmdb-backend.c: In function 'lmdb_do_for_each_ref':
> refs/lmdb-backend.c:1625:15: warning: unused variable 'c' [-Wunused
> -variable]
> const char *c = resolve_ref_unsafe_submodule(submodule, refname,
> 0, oid.hash,
> ^
>
> -Wshadow also gives a bunch of warnings, mostly about "transaction"
> and "env". Whether you want to fix them is really up to you.
> --
> Duy
Will fix these, thanks. I've now configured my config.mak to do -Wall
again, but -Wshadow produces a ton of complaints on the rest of git's
code. We should probably fix those.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 00/32] refs backend
2016-02-25 19:27 ` David Turner
@ 2016-02-25 19:31 ` Stefan Beller
2016-02-25 20:31 ` Duy Nguyen
1 sibling, 0 replies; 56+ messages in thread
From: Stefan Beller @ 2016-02-25 19:31 UTC (permalink / raw)
To: David Turner; +Cc: Duy Nguyen, Git Mailing List, Michael Haggerty
On Thu, Feb 25, 2016 at 11:27 AM, David Turner <dturner@twopensource.com> wrote:
> On Thu, 2016-02-25 at 19:57 +0700, Duy Nguyen wrote:
>> A couple of build warnings I found, haven't really read the code yet.
>> These two can easily be fixed
>>
>> refs/lmdb-backend.c: In function 'lmdb_read_raw_ref':
>> refs/lmdb-backend.c:554:16: warning: unused variable 'err' [-Wunused
>> -variable]
>> struct strbuf err = STRBUF_INIT;
>> ^
>> refs/lmdb-backend.c: In function 'lmdb_do_for_each_ref':
>> refs/lmdb-backend.c:1625:15: warning: unused variable 'c' [-Wunused
>> -variable]
>> const char *c = resolve_ref_unsafe_submodule(submodule, refname,
>> 0, oid.hash,
>> ^
>>
>> -Wshadow also gives a bunch of warnings, mostly about "transaction"
>> and "env". Whether you want to fix them is really up to you.
>> --
>> Duy
>
> Will fix these, thanks. I've now configured my config.mak to do -Wall
> again, but -Wshadow produces a ton of complaints on the rest of git's
> code. We should probably fix those.
It is a proposal for the GSoC microprojects to fix all -Wshadow problems
in say one file. Maybe we'll see some activity there :)
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PATCH v6 00/32] refs backend
2016-02-25 19:27 ` David Turner
2016-02-25 19:31 ` Stefan Beller
@ 2016-02-25 20:31 ` Duy Nguyen
1 sibling, 0 replies; 56+ messages in thread
From: Duy Nguyen @ 2016-02-25 20:31 UTC (permalink / raw)
To: David Turner; +Cc: Git Mailing List, Michael Haggerty
On Fri, Feb 26, 2016 at 2:27 AM, David Turner <dturner@twopensource.com> wrote:
> but -Wshadow produces a ton of complaints on the rest of git's
> code. We should probably fix those.
I know, but what I wrote was meant for new code only (e.g. make
refs/lmdb-backend.o CFLAGS=-Wshadow is clean). I think renaming the
global env and transaction to the_env and the_transaction probably
does the trick. Again, this is controversial whether it's a good thing
to do. So your choice.
--
Duy
^ permalink raw reply [flat|nested] 56+ messages in thread