* [PATCHv5 13/14] Allow flexible organization of notes trees, using both commit date and SHA1
From: Johan Herland @ 2009-09-08 2:27 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
This is a major expansion of the notes lookup code to allow for variations
in the notes tree organization. The variations allowed include mixing fanout
schemes based on the commit dates of the annotated commits (aka. date-based
fanout) with fanout schemes based on the SHA1 of the annotated commits (aka.
SHA1-based fanout).
Using date-based fanout in the notes tree structure enables considerable
speedup in the notes lookup process, since notes are almost always looked up
sequentially in the (reverse) chronological order of their associated commits.
Furthermore, organizing notes in a way that allow (near) sequential lookup,
enables us to decrease memory consumption both by lazily loading parts of the
notes tree structure on-demand, and freeing parts of the notes structure that
are unlikely to be used again soon.
The new flexible organization of the notes tree changes the rules for valid
note tree entries. The new rules are as follows:
1. Note objects are named by the SHA1 of the commit they annotate, possibly
split across several SHA1-based fanout levels (this is the same as is
implemented earlier in this series).
2. Note entries are located within zero or more date-based fanout levels.
3. Date-based fanout schemes may use the year, month and day values of the
associated commit's timestamp. The values must be prefixed by 'y', 'm'
and 'd' (respectively) in the notes tree.
4. The date-based components can be combined in one fanout level, or split
across multiple fanout levels. Individual components may not be split
across multiple fanout levels.
5. The year/month/date values must be specified in that order, and month or
date values may not occur without the preceding year or month value.
6. All entries of a tree object in the notes tree structure must follow the
same scheme used at that level.
Thus, the following example note entries are all valid locations for a note
annotating commit 123456789abcdef0123456789abcdef0123456789 at 2009-09-01:
- 123456789abcdef0123456789abcdef0123456789
- 12/3456789abcdef0123456789abcdef0123456789
- 1234/56789abcdef0123456789abcdef0123456789
- 12/34/56789abcdef0123456789abcdef0123456789
- 1234/5678/9abcdef0123456789abcdef0123456789
- 1234/56/78/9abcdef0123456789abcdef0123456789
- y2009/123456789abcdef0123456789abcdef0123456789
- y2009/m09/12/3456789abcdef0123456789abcdef0123456789
- y2009/m09/d01/123456789abcdef0123456789abcdef0123456789
- y2009m09/12/34/56789abcdef0123456789abcdef0123456789
- y2009m09/d01/1234/567/89abcdef0123456789abcdef0123456789
- y2009/m09d01/12/34/56/78/9abcdef0123456789abcdef0123456789
- y2009m09d01/123456789abcdef0123456789abcdef0123456789
Conversely, the following example note entries are all invalid:
- 1/23456789abcdef0123456789abcdef0123456789 (violates #1)
- 123/456789abcdef0123456789abcdef0123456789 (violates #1)
- 12/345/6789abcdef0123456789abcdef0123456789 (violates #1)
- y2009123456789abcdef0123456789abcdef0123456789 (violates #2)
- 2009/09/01/123456789abcdef0123456789abcdef0123456789 (violates #3)
- y20/09/m09/12/3456789abcdef0123456789abcdef0123456789 (violates #4)
- y20/09m09/d01/123456789abcdef0123456789abcdef0123456789 (violates #4)
- y2009m/09/12/34/56789abcdef0123456789abcdef0123456789 (violates #4)
- y2009/d01/1234/5678/9abcdef0123456789abcdef0123456789 (violates #5)
- m09/y2009/d01/12/34/56/78/9abcdef0123456789abcdef0123456789 (violates #5)
>From rule #6, we see that the following example notes tree is valid:
- y2009m09/0123456789abcdef0123456789abcdef012345678
- y2009m09/123456789abcdef0123456789abcdef0123456789
- y2008m01/d31/23/456789abcdef0123456789abcdef0123456789a
- y2008m01/d31/34/56789abcdef0123456789abcdef0123456789ab
- y2008m01/d16/4567/89abcdef0123456789abcdef0123456789abc
- y2008m01/d16/5678/9abcdef0123456789abcdef0123456789abcd
Conversely the following structure is invalid (violates rule #6):
- y2009m09/0123456789abcdef0123456789abcdef012345678
- y2009m09/12/3456789abcdef0123456789abcdef0123456789
- y2008m01/d31/23/456789abcdef0123456789abcdef0123456789a
- y2008m01/34/56789abcdef0123456789abcdef0123456789ab
- y2008m01/d16/45/6789abcdef0123456789abcdef0123456789abc
- y2008/m01d16/5678/9abcdef0123456789abcdef0123456789abcd
The flexibility added by this patch adds considerable complexity to the notes
tree parser, but the runtime and memory usage is not significantly affected
(except for the effects introduced by the chosen notes tree structure).
Internally, the 16-tree data structure introduced in earlier patches is still
used to hold the SHA1-based fanout levels and the note entries themselves.
However, this patch adds a hierarchical date-based linked-list structure
around the 16-tree structure that mirrors the fanout scheme used in the
actual notes tree.
Signed-off-by: Johan Herland <johan@herland.net>
---
notes.c | 403 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
1 files changed, 364 insertions(+), 39 deletions(-)
diff --git a/notes.c b/notes.c
index 6926aa6..a3e3f83 100644
--- a/notes.c
+++ b/notes.c
@@ -7,6 +7,70 @@
#include "tree-walk.h"
/*
+ * Format of entries in the notes tree structure:
+ *
+ * note-entry ::= (period sep)? sha1-spec
+ * period ::= year sep?
+ * (month sep?
+ * (date sep?)?
+ * )?;
+ * year ::= 'y' yearnum;
+ * month ::= 'm' monthnum;
+ * date ::= 'd' datenum;
+ * yearnum ::= # 4-digit decimal year, from annotated commit's timestamp;
+ * monthnum ::= # 2-digit decimal month, from annotated commit's timestamp;
+ * datenum ::= # 2-digit decimal date, from annotated commit's timestamp;
+ * sha1-spec ::= (hex-fragment sep?){20}
+ * sep ::= '/';
+ * hex-fragment ::= # Fragment of hexsha1 (2 bytes);
+ * hexsha1 ::= # SHA1 of annotated commit in hex format (40 bytes);
+ *
+ * Thus, the following example note entries are all valid:
+ * - 0123456789abcdef0123456789abcdef012345678
+ * - 01/23456789abcdef0123456789abcdef012345678
+ * - 0123/456789abcdef0123456789abcdef012345678
+ * - 01/23/456789abcdef0123456789abcdef012345678
+ * - 0123/4567/89abcdef0123456789abcdef012345678
+ * - 0123/45/67/89abcdef0123456789abcdef012345678
+ * - y2009/0123456789abcdef0123456789abcdef012345678
+ * - y2009/m09/01/23456789abcdef0123456789abcdef012345678
+ * - y2009/m09/d01/0123456789abcdef0123456789abcdef012345678
+ * - y2009m09/01/23/456789abcdef0123456789abcdef012345678
+ * - y2009m09/d01/0123/4567/89abcdef0123456789abcdef012345678
+ * - y2009/m09d01/01/23/45/67/89abcdef0123456789abcdef012345678
+ * - y2009m09d01/0123456789abcdef0123456789abcdef012345678
+ *
+ * and the following example note entries are all invalid:
+ * - 0/123456789abcdef0123456789abcdef012345678
+ * - 012/3456789abcdef0123456789abcdef012345678
+ * - 01/234/56789abcdef0123456789abcdef012345678
+ * - y20090123456789abcdef0123456789abcdef012345678
+ * - y20/09/m09/01/23456789abcdef0123456789abcdef012345678
+ * - y20/09m09/d01/0123456789abcdef0123456789abcdef012345678
+ * - y2009m/09/01/23/456789abcdef0123456789abcdef012345678
+ * - y2009/d01/0123/4567/89abcdef0123456789abcdef012345678
+ * - m09/y2009/d01/01/23/45/67/89abcdef0123456789abcdef012345678
+ *
+ * In addition to the above per-entry rules, we require that _all_ entries at
+ * a given level in the notes tree (levels are separated by '/') follow the
+ * exact same format at that level. Thus the following structure is valid:
+ * - y2009m09/0123456789abcdef0123456789abcdef012345678
+ * - y2009m09/123456789abcdef0123456789abcdef0123456789
+ * - y2008m01/d31/23/456789abcdef0123456789abcdef0123456789a
+ * - y2008m01/d31/34/56789abcdef0123456789abcdef0123456789ab
+ * - y2008m01/d16/4567/89abcdef0123456789abcdef0123456789abc
+ * - y2008m01/d16/5678/9abcdef0123456789abcdef0123456789abcd
+ *
+ * but the following structure is invalid:
+ * - y2009m09/0123456789abcdef0123456789abcdef012345678
+ * - y2009m09/12/3456789abcdef0123456789abcdef0123456789
+ * - y2008m01/d31/23/456789abcdef0123456789abcdef0123456789a
+ * - y2008m01/34/56789abcdef0123456789abcdef0123456789ab
+ * - y2008m01/d16/45/6789abcdef0123456789abcdef0123456789abc
+ * - y2008/m01d16/5678/9abcdef0123456789abcdef0123456789abcd
+ */
+
+/*
* Use a non-balancing simple 16-tree structure with struct int_node as
* internal nodes, and struct leaf_node as leaf nodes. Each int_node has a
* 16-array of pointers to its children.
@@ -17,9 +81,45 @@
* - ptr & 3 == 3 - pointer to subtree entry - cast to struct leaf_node *
*
* The root node is a statically allocated struct int_node.
+ *
+ * In order to allow date-based fanout schemes in addition to the original
+ * SHA1-based fanout schemes, we need to overload this structure, as follows:
+ * If the first pointer in the 16-array is ~0 (i.e. 0xffffffff on 32-bit
+ * systems and 0xffffffffffffffff on 64-bit systems), then the int_node is NOT
+ * to be interpreted as a 16-array of child node pointers. Rather, the int_node
+ * now represents a period-based node with the following properties:
+ * - The node has a pointer to a "child" node of type struct int_node, which is
+ * EITHER a "regular" int_node object representing the root node of a 16-tree
+ * structure holding notes associated with commits with timestamps within
+ * that time period, OR another period-based int_node representing some
+ * subdivision of the time period.
+ * - The node also has a pointer to a "previous" period-based int_node, which
+ * represents the previous time period for which there exist note objects.
+ * - The node has a pointer to a "parent" node, which is the period-based
+ * int_node that has this int_node as one of its children. This is needed
+ * when traversing the date-based int_nodes looking for a period matching the
+ * given commit. For top-level objects, this is set to NULL.
+ * - The node stores the SHA1 sum of the tree object that represents its child
+ * (within the notes tree structure). Thus, we keep a reference to the child
+ * structure that without necessarily allocating the child node (and
+ * underlying structure).
+ * - Finally, the node has a period string, which indicates the time period of
+ * the notes contained within, typically of the form "YYYY", "YYYY-MM" or
+ * "YYYY-MM-DD", depending on the granularity of the corresponding
+ * period-based entries in the notes tree structure.
*/
struct int_node {
- void *a[16];
+ union {
+ void *a[16];
+ struct {
+ void *magic; /* ~0 "enables" this part of the union */
+ struct int_node *child;
+ struct int_node *prev;
+ struct int_node *parent;
+ unsigned char tree_sha1[20];
+ char period[11]; /* Enough to hold "YYYY-MM-DD" */
+ };
+ };
};
/*
@@ -51,12 +151,18 @@ struct leaf_node {
#define SUBTREE_SHA1_PREFIXCMP(key_sha1, subtree_sha1) \
(memcmp(key_sha1, subtree_sha1, subtree_sha1[19]))
+#define SUBTREE_DATE_PREFIXCMP(commit_date, subtree_date) \
+ (prefixcmp(commit_date, subtree_date))
+
static struct int_node root_node;
+static struct int_node *cur_node;
+
static int initialized;
-static void load_subtree(struct leaf_node *subtree, struct int_node *node,
- unsigned int n);
+static void load_subtree(const unsigned char *sha1,
+ const unsigned char *prefix, unsigned int prefix_len,
+ struct int_node *node, struct int_node *parent, int n);
/*
* To find a leaf_node:
@@ -94,7 +200,8 @@ static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
if (!SUBTREE_SHA1_PREFIXCMP(key_sha1, l->key_sha1)) {
/* unpack tree and resume search */
tree->a[i] = NULL;
- load_subtree(l, tree, n);
+ load_subtree(l->val_sha1, l->key_sha1, l->key_sha1[19],
+ tree, NULL, (int) n);
free(l);
return note_tree_find(tree, n, key_sha1);
}
@@ -117,7 +224,8 @@ static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
if (!SUBTREE_SHA1_PREFIXCMP(key_sha1, l->key_sha1)) {
/* unpack tree and resume search */
tree->a[0] = NULL;
- load_subtree(l, tree, n);
+ load_subtree(l->val_sha1, l->key_sha1, l->key_sha1[19], tree,
+ NULL, (int) n);
free(l);
return note_tree_find(tree, n, key_sha1);
}
@@ -173,16 +281,28 @@ static int note_tree_insert(struct int_node *tree, unsigned char n,
/* Free the entire notes data contained in the given tree */
static void note_tree_free(struct int_node *tree)
{
- unsigned int i;
- for (i = 0; i < 16; i++) {
- void *p = tree->a[i];
- switch(GET_PTR_TYPE(p)) {
- case PTR_TYPE_INTERNAL:
- note_tree_free(CLR_PTR_TYPE(p));
- /* fall through */
- case PTR_TYPE_NOTE:
- case PTR_TYPE_SUBTREE:
- free(CLR_PTR_TYPE(p));
+ if (tree->magic == (void *) ~0) {
+ if (tree->prev) {
+ note_tree_free(tree->prev);
+ free(tree->prev);
+ }
+ if (tree->child) {
+ note_tree_free(tree->child);
+ free(tree->child);
+ }
+ }
+ else {
+ unsigned int i;
+ for (i = 0; i < 16; i++) {
+ void *p = tree->a[i];
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_INTERNAL:
+ note_tree_free(CLR_PTR_TYPE(p));
+ /* fall through */
+ case PTR_TYPE_NOTE:
+ case PTR_TYPE_SUBTREE:
+ free(CLR_PTR_TYPE(p));
+ }
}
}
}
@@ -215,29 +335,139 @@ static int get_sha1_hex_segment(const char *hex, unsigned int hex_len,
return len;
}
-static void load_subtree(struct leaf_node *subtree, struct int_node *node,
- unsigned int n)
+/*
+ * Parse year/month/date strings, and generate the corresponding period string
+ * for the given path entry:
+ * - prefix must follow one of these forms: "", "YYYY", "YYYY-MM"
+ * - path should follow one of these forms: "yYYYY", "yYYYYmMM", "yYYYYmMMdDD",
+ * "mMMdDD", "mMM" or "dDD"
+ * The resulting string (which follows the form "YYYY", "YYYY-MM" or
+ * "YYYY-MM-DD") is returned as a static string. If path is not valid in the
+ * given (prefix) context, NULL is returned.
+ */
+static const char *parse_period(const char *prefix, unsigned int prefix_len,
+ const char *path, unsigned int path_len)
+{
+ static char result[11];
+ char expect_type; /* y/m/d for year/month/day-based fanout */
+ unsigned int expect_len, value;
+ char *endptr, *target = result;
+
+ switch (prefix_len) {
+ case 0:
+ /* No prefix, expect year-based fanout in path */
+ expect_type = 'y';
+ expect_len = 4;
+ break;
+ case 4:
+ /* Year in prefix, expect month-based fanout in path */
+ expect_type = 'm';
+ expect_len = 2;
+ break;
+ case 7:
+ /* "YYYY-MM" in prefix, expect day-based fanout in path */
+ expect_type = 'd';
+ expect_len = 2;
+ break;
+ default:
+ die("Date-based notes tree loading invoked with invalid "
+ "prefix '%.*s'", prefix_len, prefix);
+ }
+
+ if (path[0] != expect_type) {
+ warning("Unexpected entry path in date-based notes tree: '%s' "
+ "(skipping)", path);
+ return NULL;
+ }
+ value = (unsigned int) strtoul(path + 1, &endptr, 10);
+ switch (expect_type) {
+ case 'y':
+ if (value < 1969 || value >= 3000) {
+ warning("Invalid year value in date-based notes tree:"
+ " '%s' (skipping)", path);
+ return NULL;
+ }
+ break;
+ case 'm':
+ if (value < 1 || value > 12) {
+ warning("Invalid month value in date-based notes tree:"
+ " '%s' (skipping)", path);
+ return NULL;
+ }
+ break;
+ case 'd':
+ if (value < 1 || value > 31) {
+ warning("Invalid day value in date-based notes tree:"
+ " '%s' (skipping)", path);
+ return NULL;
+ }
+ break;
+ }
+
+ if (prefix == result) {
+ target = result + prefix_len;
+ prefix = NULL;
+ prefix_len = 0;
+ }
+ prefix_len = snprintf(target, 11, "%.*s%s%0*u", prefix_len, prefix,
+ expect_len == 2 ? "-" : "", expect_len, value);
+ prefix_len += target - result;
+ assert(prefix_len < 11);
+
+ if (*endptr) /* there are more components in this path */
+ return parse_period(result, prefix_len, endptr,
+ path_len - (endptr - path));
+ return result;
+}
+
+static void load_date_subtree(struct tree_desc *tree_desc,
+ const char *prefix, unsigned int prefix_len,
+ struct int_node *node, struct int_node *parent)
+{
+ struct name_entry entry;
+ struct int_node *cur_node = NULL;
+ struct int_node *new_node;
+
+ while (tree_entry(tree_desc, &entry)) {
+ const char *period = parse_period(
+ prefix, prefix_len, entry.path, strlen(entry.path));
+ if (!period)
+ continue;
+ if (tree_desc->size) /* this is not the last tree entry */
+ new_node = (struct int_node *)
+ xmalloc(sizeof(struct int_node));
+ else /* this is the last entry, store directly into node */
+ new_node = node;
+
+ new_node->magic = (void *) ~0;
+ new_node->child = NULL;
+ new_node->prev = cur_node;
+ new_node->parent = parent;
+ hashcpy(new_node->tree_sha1, entry.sha1);
+ strcpy(new_node->period, period);
+ cur_node = new_node;
+ }
+ assert(!cur_node || cur_node == node);
+}
+
+static void load_sha1_subtree(struct tree_desc *tree_desc,
+ const unsigned char *prefix, unsigned int prefix_len,
+ struct int_node *node, unsigned char n)
{
unsigned char commit_sha1[20];
- unsigned int prefix_len;
int status;
- void *buf;
- struct tree_desc desc;
struct name_entry entry;
- buf = fill_tree_descriptor(&desc, subtree->val_sha1);
- if (!buf)
- die("Could not read %s for notes-index",
- sha1_to_hex(subtree->val_sha1));
-
- prefix_len = subtree->key_sha1[19];
assert(prefix_len * 2 >= n);
- memcpy(commit_sha1, subtree->key_sha1, prefix_len);
- while (tree_entry(&desc, &entry)) {
+ memcpy(commit_sha1, prefix, prefix_len);
+ while (tree_entry(tree_desc, &entry)) {
int len = get_sha1_hex_segment(entry.path, strlen(entry.path),
commit_sha1 + prefix_len, 20 - prefix_len);
- if (len < 0)
+ if (len < 0) {
+ warning("Invalid value in notes tree: '%s' (skipping)",
+ entry.path);
continue; /* entry.path is not a SHA1 sum. Skip */
+ }
len += prefix_len;
/*
@@ -258,6 +488,42 @@ static void load_subtree(struct leaf_node *subtree, struct int_node *node,
assert(!status);
}
}
+}
+
+static void load_subtree(const unsigned char *sha1,
+ const unsigned char *prefix, unsigned int prefix_len,
+ struct int_node *node, struct int_node *parent, int n)
+{
+ void *buf;
+ struct tree_desc desc;
+
+ buf = fill_tree_descriptor(&desc, sha1);
+ if (!buf)
+ die("Could not read notes subtree at %s", sha1_to_hex(sha1));
+ /*
+ * After fill_tree_descriptor(), we can peek at the first tree entry
+ * in desc.entry.
+ */
+ switch (desc.entry.path[0]) {
+ case 'd':
+ if (strlen(desc.entry.path) != 3)
+ break;
+ /* fall-through */
+ case 'm':
+ case 'y':
+ /* path cannot be a SHA1 fragment */
+ load_date_subtree(&desc, (const char *) prefix, prefix_len,
+ node, parent);
+ free(buf);
+ return;
+ }
+ if (n < 0) {
+ /* Arriving from a date-based subtree; reset prefix */
+ n = 0;
+ prefix = NULL;
+ prefix_len = 0;
+ }
+ load_sha1_subtree(&desc, prefix, prefix_len, node, n);
free(buf);
}
@@ -265,23 +531,81 @@ static void initialize_notes(const char *notes_ref_name)
{
unsigned char sha1[20], commit_sha1[20];
unsigned mode;
- struct leaf_node root_tree;
if (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||
get_tree_entry(commit_sha1, "", sha1, &mode))
return;
- hashclr(root_tree.key_sha1);
- hashcpy(root_tree.val_sha1, sha1);
- load_subtree(&root_tree, &root_node, 0);
+ load_subtree(sha1, NULL, 0, &root_node, NULL, 0);
+ cur_node = &root_node;
}
-static unsigned char *lookup_notes(const unsigned char *commit_sha1)
+static unsigned char *lookup_notes(const struct commit *commit)
{
- struct leaf_node *found = note_tree_find(&root_node, 0, commit_sha1);
- if (found)
- return found->val_sha1;
- return NULL;
+ struct int_node *node = cur_node, *seen_node = cur_node;
+ struct leaf_node *found;
+ const char *short_date;
+
+ if (!node)
+ return NULL;
+
+ /* Convert commit->date to YYYY-MM-DD format */
+ short_date = show_date(commit->date, 0, DATE_SHORT);
+
+ while (node->magic == (void *) ~0) { /* date-based node */
+ int cmp = SUBTREE_DATE_PREFIXCMP(short_date, node->period);
+ if (cmp == 0) {
+ /* Search inside child node */
+ if (!node->child) {
+ /* Must unpack child node first */
+ node->child = (struct int_node *)
+ xcalloc(sizeof(struct int_node), 1);
+ load_subtree(node->tree_sha1,
+ (const unsigned char *) node->period,
+ strlen(node->period), node->child,
+ node, -1);
+ }
+ seen_node = node;
+ node = node->child;
+ }
+ else if (cmp > 0) {
+ /* Search in past node */
+ if (node->prev)
+ node = node->prev;
+ else
+ node = node->parent;
+ }
+ else {
+ /* Search in future node */
+ if (!node->parent) {
+ /* Restart from root_node */
+ seen_node = node;
+ node = &root_node;
+ }
+ else
+ node = node->parent;
+ }
+ if (!node || node == seen_node) {
+ /* We've been here before, give up search */
+ return NULL;
+ }
+ }
+ while (cur_node &&
+ SUBTREE_DATE_PREFIXCMP(cur_node->period, seen_node->period) < 0)
+ {
+ /*
+ * We're about to move cur_node backwards in history. We are
+ * unlikely to need this cur_node in the future, so free() it.
+ */
+ note_tree_free(cur_node->child);
+ cur_node->child = NULL;
+ cur_node = cur_node->parent;
+ }
+ cur_node = seen_node;
+
+ /* Drill down further with SHA1-based lookup */
+ found = note_tree_find(node, 0, commit->object.sha1);
+ return found ? found->val_sha1 : NULL;
}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
@@ -303,7 +627,7 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
initialized = 1;
}
- sha1 = lookup_notes(commit->object.sha1);
+ sha1 = lookup_notes(commit);
if (!sha1)
return;
@@ -342,6 +666,7 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
void free_commit_notes()
{
+ cur_node = NULL;
note_tree_free(&root_node);
memset(&root_node, 0, sizeof(struct int_node));
initialized = 0;
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 14/14] Add test cases for date-based fanouts
From: Johan Herland @ 2009-09-08 2:27 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
Signed-off-by: Johan Herland <johan@herland.net>
---
t/t3303-notes-subtrees.sh | 64 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 64 insertions(+), 0 deletions(-)
diff --git a/t/t3303-notes-subtrees.sh b/t/t3303-notes-subtrees.sh
index d24203e..d3cbf6d 100755
--- a/t/t3303-notes-subtrees.sh
+++ b/t/t3303-notes-subtrees.sh
@@ -134,4 +134,68 @@ test_expect_success 'verify notes in 4/36-fanout overriding 2/38-fanout' 'verify
test_expect_success 'test notes in 2/38-fanout overriding 2/2/36-fanout' 'test_preferred "s|^..|&/|" "s|^\(..\)\(..\)|\1/\2/|"'
test_expect_success 'verify notes in 2/38-fanout overriding 2/2/36-fanout' 'verify_notes'
+test_date_based () {
+ (
+ start_note_commit &&
+ nr=$number_of_commits &&
+ git log --format="%H %ct" refs/heads/master |
+ while read sha1 date_t; do
+ date=$(date -u -d "@$date_t" +"$1")
+ note_path="$date/$(echo "$sha1" | sed "$2")"
+ cat <<INPUT_END &&
+M 100644 inline $note_path
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+ nr=$(($nr-1))
+ done
+ ) |
+ git fast-import --quiet
+}
+
+test_expect_success 'test notes in y/40-fanout' 'test_date_based "y%Y" ""'
+test_expect_success 'verify notes in y/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/2/38-fanout' 'test_date_based "y%Y" "s|^..|&/|"'
+test_expect_success 'verify notes in y/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ym/40-fanout' 'test_date_based "y%Ym%m" ""'
+test_expect_success 'verify notes in ym/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ym/2/38-fanout' 'test_date_based "y%Ym%m" "s|^..|&/|"'
+test_expect_success 'verify notes in ym/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ymd/40-fanout' 'test_date_based "y%Ym%md%d" ""'
+test_expect_success 'verify notes in ymd/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ymd/2/38-fanout' 'test_date_based "y%Ym%md%d" "s|^..|&/|"'
+test_expect_success 'verify notes in ymd/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/m/40-fanout' 'test_date_based "y%Y/m%m" ""'
+test_expect_success 'verify notes in y/m/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/m/2/38-fanout' 'test_date_based "y%Y/m%m" "s|^..|&/|"'
+test_expect_success 'verify notes in y/m/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/md/40-fanout' 'test_date_based "y%Y/m%md%d" ""'
+test_expect_success 'verify notes in y/md/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/md/2/38-fanout' 'test_date_based "y%Y/m%md%d" "s|^..|&/|"'
+test_expect_success 'verify notes in y/md/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ym/d/40-fanout' 'test_date_based "y%Ym%m/d%d" ""'
+test_expect_success 'verify notes in ym/d/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in ym/d/2/38-fanout' 'test_date_based "y%Ym%m/d%d" "s|^..|&/|"'
+test_expect_success 'verify notes in ym/d/2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/m/d/40-fanout' 'test_date_based "y%Y/m%m/d%d" ""'
+test_expect_success 'verify notes in y/m/d/40-fanout' 'verify_notes'
+
+test_expect_success 'test notes in y/m/d/2/38-fanout' 'test_date_based "y%Y/m%m/d%d" "s|^..|&/|"'
+test_expect_success 'verify notes in y/m/d/2/38-fanout' 'verify_notes'
+
test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 12/14] Selftests verifying semantics when loading notes trees with various fanouts
From: Johan Herland @ 2009-09-08 2:27 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
Add selftests verifying:
- that we are able to parse notes trees with various fanout schemes
- that notes trees with conflicting fanout schemes are parsed as expected
Signed-off-by: Johan Herland <johan@herland.net>
---
t/t3303-notes-subtrees.sh | 137 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 137 insertions(+), 0 deletions(-)
create mode 100755 t/t3303-notes-subtrees.sh
diff --git a/t/t3303-notes-subtrees.sh b/t/t3303-notes-subtrees.sh
new file mode 100755
index 0000000..d24203e
--- /dev/null
+++ b/t/t3303-notes-subtrees.sh
@@ -0,0 +1,137 @@
+#!/bin/sh
+
+test_description='Test commit notes organized in subtrees'
+
+. ./test-lib.sh
+
+number_of_commits=100
+
+start_note_commit () {
+ test_tick &&
+ cat <<INPUT_END
+commit refs/notes/commits
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes
+COMMIT
+
+from refs/notes/commits^0
+deleteall
+INPUT_END
+
+}
+
+verify_notes () {
+ git log | grep "^ " > output &&
+ i=$number_of_commits &&
+ while [ $i -gt 0 ]; do
+ echo " commit #$i" &&
+ echo " note for commit #$i" &&
+ i=$(($i-1));
+ done > expect &&
+ test_cmp expect output
+}
+
+test_expect_success "setup: create $number_of_commits commits" '
+
+ (
+ nr=0 &&
+ while [ $nr -lt $number_of_commits ]; do
+ nr=$(($nr+1)) &&
+ test_tick &&
+ cat <<INPUT_END
+commit refs/heads/master
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+commit #$nr
+COMMIT
+
+M 644 inline file
+data <<EOF
+file in commit #$nr
+EOF
+
+INPUT_END
+
+ done &&
+ test_tick &&
+ cat <<INPUT_END
+commit refs/notes/commits
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+no notes
+COMMIT
+
+deleteall
+
+INPUT_END
+
+ ) |
+ git fast-import --quiet &&
+ git config core.notesRef refs/notes/commits
+'
+
+test_sha1_based () {
+ (
+ start_note_commit &&
+ nr=$number_of_commits &&
+ git rev-list refs/heads/master |
+ while read sha1; do
+ note_path=$(echo "$sha1" | sed "$1")
+ cat <<INPUT_END &&
+M 100644 inline $note_path
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+ nr=$(($nr-1))
+ done
+ ) |
+ git fast-import --quiet
+}
+
+test_expect_success 'test notes in 2/38-fanout' 'test_sha1_based "s|^..|&/|"'
+test_expect_success 'verify notes in 2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in 4/36-fanout' 'test_sha1_based "s|^....|&/|"'
+test_expect_success 'verify notes in 4/36-fanout' 'verify_notes'
+
+test_expect_success 'test notes in 2/2/36-fanout' 'test_sha1_based "s|^\(..\)\(..\)|\1/\2/|"'
+test_expect_success 'verify notes in 2/2/36-fanout' 'verify_notes'
+
+test_preferred () {
+ (
+ start_note_commit &&
+ nr=$number_of_commits &&
+ git rev-list refs/heads/master |
+ while read sha1; do
+ preferred_note_path=$(echo "$sha1" | sed "$1")
+ ignored_note_path=$(echo "$sha1" | sed "$2")
+ cat <<INPUT_END &&
+M 100644 inline $ignored_note_path
+data <<EOF
+IGNORED note for commit #$nr
+EOF
+
+M 100644 inline $preferred_note_path
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+ nr=$(($nr-1))
+ done
+ ) |
+ git fast-import --quiet
+}
+
+test_expect_success 'test notes in 4/36-fanout overriding 2/38-fanout' 'test_preferred "s|^....|&/|" "s|^..|&/|"'
+test_expect_success 'verify notes in 4/36-fanout overriding 2/38-fanout' 'verify_notes'
+
+test_expect_success 'test notes in 2/38-fanout overriding 2/2/36-fanout' 'test_preferred "s|^..|&/|" "s|^\(..\)\(..\)|\1/\2/|"'
+test_expect_success 'verify notes in 2/38-fanout overriding 2/2/36-fanout' 'verify_notes'
+
+test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 11/14] Teach the notes lookup code to parse notes trees with various fanout schemes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
The semantics used when parsing notes trees (with regards to fanout subtrees)
follow Dscho's proposal fairly closely:
- No concatenation/merging of notes is performed. If there are several notes
objects referencing a given commit, only one of those objects are used.
- If a notes object for a given commit is present in the "root" notes tree,
no subtrees are consulted; the object in the root tree is used directly.
- If there are more than one subtree that prefix-matches the given commit,
only the subtree with the longest matching prefix is consulted. This
means that if the given commit is e.g. "deadbeef", and the notes tree have
subtrees "de" and "dead", then the following paths in the notes tree are
searched: "deadbeef", "dead/beef". Note that "de/adbeef" is NOT searched.
- Fanout directories (subtrees) must references a whole number of bytes
from the SHA1 sum they subdivide. E.g. subtrees "dead" and "de" are
acceptable; "d" and "dea" are not.
- Multiple levels of fanout are allowed. All the above rules apply
recursively. E.g. "de/adbeef" is preferred over "de/adbe/ef", etc.
This patch changes the in-memory datastructure for holding parsed notes:
Instead of holding all note (and subtree) entries in a hash table, a
simple 16-tree structure is used instead. The tree structure consists of
16-arrays as internal nodes, and note/subtree entries as leaf nodes. The
tree is traversed by indexing subsequent nibbles of the search key until
a leaf node is encountered. If a subtree entry is encountered while
searching for a note, the subtree is unpacked into the 16-tree structure,
and the search continues into that subtree.
The new algorithm performs significantly better in the cases where only
a fraction of the notes need to be looked up (this is assumed to be the
common case for notes lookup). The new code even performs marginally
better in the worst case (where _all_ the notes are looked up).
In addition to this, comes the massive performance win associated with
organizing the notes tree according to some fanout scheme. Even a simple
2/38 fanout scheme is dramatically quicker to traverse (going from tens of
seconds to sub-second runtimes).
As for memory usage, the new code is marginally better than the old code in
the worst case, but in the case of looking up only some notes from a notes
tree with proper fanout, the new code uses only a small fraction of the
memory needed to hold the entire notes tree.
However, there is one casualty of this patch. The old notes lookup code was
able to parse notes that were associated with non-SHA1s (e.g. refs). The new
code requires the referenced object to be named by a SHA1 sum. Still, this
is not considered a major setback, since the notes infrastructure was not
originally intended to annotate objects outside the Git object database.
Signed-off-by: Johan Herland <johan@herland.net>
---
notes.c | 317 +++++++++++++++++++++++++++++++++++++++++++++++++--------------
1 files changed, 248 insertions(+), 69 deletions(-)
diff --git a/notes.c b/notes.c
index 008c3d4..6926aa6 100644
--- a/notes.c
+++ b/notes.c
@@ -6,103 +6,282 @@
#include "strbuf.h"
#include "tree-walk.h"
-struct entry {
- unsigned char commit_sha1[20];
- unsigned char notes_sha1[20];
+/*
+ * Use a non-balancing simple 16-tree structure with struct int_node as
+ * internal nodes, and struct leaf_node as leaf nodes. Each int_node has a
+ * 16-array of pointers to its children.
+ * The bottom 2 bits of each pointer is used to identify the pointer type
+ * - ptr & 3 == 0 - NULL pointer, assert(ptr == NULL)
+ * - ptr & 3 == 1 - pointer to next internal node - cast to struct int_node *
+ * - ptr & 3 == 2 - pointer to note entry - cast to struct leaf_node *
+ * - ptr & 3 == 3 - pointer to subtree entry - cast to struct leaf_node *
+ *
+ * The root node is a statically allocated struct int_node.
+ */
+struct int_node {
+ void *a[16];
};
-struct hash_map {
- struct entry *entries;
- off_t count, size;
+/*
+ * Leaf nodes come in two variants, note entries and subtree entries,
+ * distinguished by the LSb of the leaf node pointer (see above).
+ * As a note entry, the key is the SHA1 of the referenced commit, and the
+ * value is the SHA1 of the note object.
+ * As a subtree entry, the key is the prefix SHA1 (w/trailing NULs) of the
+ * referenced commit, using the last byte of the key to store the length of
+ * the prefix. The value is the SHA1 of the tree object containing the notes
+ * subtree.
+ */
+struct leaf_node {
+ unsigned char key_sha1[20];
+ unsigned char val_sha1[20];
};
-static int initialized;
-static struct hash_map hash_map;
+#define PTR_TYPE_NULL 0
+#define PTR_TYPE_INTERNAL 1
+#define PTR_TYPE_NOTE 2
+#define PTR_TYPE_SUBTREE 3
-static int hash_index(struct hash_map *map, const unsigned char *sha1)
-{
- int i = ((*(unsigned int *)sha1) % map->size);
+#define GET_PTR_TYPE(ptr) ((uintptr_t) (ptr) & 3)
+#define CLR_PTR_TYPE(ptr) ((void *) ((uintptr_t) (ptr) & ~3))
+#define SET_PTR_TYPE(ptr, type) ((void *) ((uintptr_t) (ptr) | (type)))
- for (;;) {
- unsigned char *current = map->entries[i].commit_sha1;
+#define GET_NIBBLE(n, sha1) (((sha1[n >> 1]) >> ((n & 0x01) << 2)) & 0x0f)
- if (!hashcmp(sha1, current))
- return i;
+#define SUBTREE_SHA1_PREFIXCMP(key_sha1, subtree_sha1) \
+ (memcmp(key_sha1, subtree_sha1, subtree_sha1[19]))
- if (is_null_sha1(current))
- return -1 - i;
+static struct int_node root_node;
- if (++i == map->size)
- i = 0;
+static int initialized;
+
+static void load_subtree(struct leaf_node *subtree, struct int_node *node,
+ unsigned int n);
+
+/*
+ * To find a leaf_node:
+ * 1. Start at the root node, with n = 0
+ * 2. Use the nth nibble of the key as an index into a:
+ * - If a[n] is an int_node, recurse into that node and increment n
+ * - If a leaf_node with matching key, return leaf_node (assert note entry)
+ * - If a matching subtree entry, unpack that subtree entry (and remove it);
+ * restart search at the current level.
+ * - Otherwise, we end up at a NULL pointer, or a non-matching leaf_node.
+ * Backtrack out of the recursion, one level at a time and check a[0]:
+ * - If a[0] at the current level is a matching subtree entry, unpack that
+ * subtree entry (and remove it); restart search at the current level.
+ */
+static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
+ const unsigned char *key_sha1)
+{
+ struct leaf_node *l;
+ unsigned char i = GET_NIBBLE(n, key_sha1);
+ void *p = tree->a[i];
+
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_INTERNAL:
+ l = note_tree_find(CLR_PTR_TYPE(p), n + 1, key_sha1);
+ if (l)
+ return l;
+ break;
+ case PTR_TYPE_NOTE:
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!hashcmp(key_sha1, l->key_sha1))
+ return l; /* return note object matching given key */
+ break;
+ case PTR_TYPE_SUBTREE:
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!SUBTREE_SHA1_PREFIXCMP(key_sha1, l->key_sha1)) {
+ /* unpack tree and resume search */
+ tree->a[i] = NULL;
+ load_subtree(l, tree, n);
+ free(l);
+ return note_tree_find(tree, n, key_sha1);
+ }
+ break;
+ case PTR_TYPE_NULL:
+ default:
+ assert(!p);
+ break;
}
+
+ /*
+ * Did not find key at this (or any lower) level.
+ * Check if there's a matching subtree entry in tree->a[0].
+ * If so, unpack tree and resume search.
+ */
+ p = tree->a[0];
+ if (GET_PTR_TYPE(p) != PTR_TYPE_SUBTREE)
+ return NULL;
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!SUBTREE_SHA1_PREFIXCMP(key_sha1, l->key_sha1)) {
+ /* unpack tree and resume search */
+ tree->a[0] = NULL;
+ load_subtree(l, tree, n);
+ free(l);
+ return note_tree_find(tree, n, key_sha1);
+ }
+ return NULL;
}
-static void add_entry(const unsigned char *commit_sha1,
- const unsigned char *notes_sha1)
+/*
+ * To insert a leaf_node:
+ * 1. Start at the root node, with n = 0
+ * 2. Use the nth nibble of the key as an index into a:
+ * - If a[n] is NULL, store the tweaked pointer directly into a[n]
+ * - If a[n] is an int_node, recurse into that node and increment n
+ * - If a[n] is a leaf_node:
+ * 1. Check if they're equal, and handle that (abort? overwrite?)
+ * 2. Create a new int_node, and store both leaf_nodes there
+ * 3. Store the new int_node into a[n].
+ */
+static int note_tree_insert(struct int_node *tree, unsigned char n,
+ const struct leaf_node *entry, unsigned char type)
{
- int index;
-
- if (hash_map.count + 1 > hash_map.size >> 1) {
- int i, old_size = hash_map.size;
- struct entry *old = hash_map.entries;
-
- hash_map.size = old_size ? old_size << 1 : 64;
- hash_map.entries = (struct entry *)
- xcalloc(sizeof(struct entry), hash_map.size);
-
- for (i = 0; i < old_size; i++)
- if (!is_null_sha1(old[i].commit_sha1)) {
- index = -1 - hash_index(&hash_map,
- old[i].commit_sha1);
- memcpy(hash_map.entries + index, old + i,
- sizeof(struct entry));
- }
- free(old);
+ struct int_node *new_node;
+ const struct leaf_node *l;
+ int ret;
+ unsigned char i = GET_NIBBLE(n, entry->key_sha1);
+ void *p = tree->a[i];
+ assert(GET_PTR_TYPE(entry) == PTR_TYPE_NULL);
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_NULL:
+ assert(!p);
+ tree->a[i] = SET_PTR_TYPE(entry, type);
+ return 0;
+ case PTR_TYPE_INTERNAL:
+ return note_tree_insert(CLR_PTR_TYPE(p), n + 1, entry, type);
+ default:
+ assert(GET_PTR_TYPE(p) == PTR_TYPE_NOTE ||
+ GET_PTR_TYPE(p) == PTR_TYPE_SUBTREE);
+ l = (const struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!hashcmp(entry->key_sha1, l->key_sha1))
+ return -1; /* abort insert on matching key */
+ new_node = (struct int_node *)
+ xcalloc(sizeof(struct int_node), 1);
+ ret = note_tree_insert(new_node, n + 1,
+ CLR_PTR_TYPE(p), GET_PTR_TYPE(p));
+ if (ret) {
+ free(new_node);
+ return -1;
+ }
+ tree->a[i] = SET_PTR_TYPE(new_node, PTR_TYPE_INTERNAL);
+ return note_tree_insert(new_node, n + 1, entry, type);
}
+}
- index = hash_index(&hash_map, commit_sha1);
- if (index < 0) {
- index = -1 - index;
- hash_map.count++;
+/* Free the entire notes data contained in the given tree */
+static void note_tree_free(struct int_node *tree)
+{
+ unsigned int i;
+ for (i = 0; i < 16; i++) {
+ void *p = tree->a[i];
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_INTERNAL:
+ note_tree_free(CLR_PTR_TYPE(p));
+ /* fall through */
+ case PTR_TYPE_NOTE:
+ case PTR_TYPE_SUBTREE:
+ free(CLR_PTR_TYPE(p));
+ }
}
+}
- hashcpy(hash_map.entries[index].commit_sha1, commit_sha1);
- hashcpy(hash_map.entries[index].notes_sha1, notes_sha1);
+/*
+ * Convert a partial SHA1 hex string to the corresponding partial SHA1 value.
+ * - hex - Partial SHA1 segment in ASCII hex format
+ * - hex_len - Length of above segment. Must be multiple of 2 between 0 and 40
+ * - sha1 - Partial SHA1 value is written here
+ * - sha1_len - Max #bytes to store in sha1, Must be >= hex_len / 2, and < 20
+ * Returns -1 on error (invalid arguments or invalid SHA1 (not in hex format).
+ * Otherwise, returns number of bytes written to sha1 (i.e. hex_len / 2).
+ * Pads sha1 with NULs up to sha1_len (not included in returned length).
+ */
+static int get_sha1_hex_segment(const char *hex, unsigned int hex_len,
+ unsigned char *sha1, unsigned int sha1_len)
+{
+ unsigned int i, len = hex_len >> 1;
+ if (hex_len % 2 != 0 || len > sha1_len)
+ return -1;
+ for (i = 0; i < len; i++) {
+ unsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);
+ if (val & ~0xff)
+ return -1;
+ *sha1++ = val;
+ hex += 2;
+ }
+ for (; i < sha1_len; i++)
+ *sha1++ = 0;
+ return len;
}
-static void initialize_hash_map(const char *notes_ref_name)
+static void load_subtree(struct leaf_node *subtree, struct int_node *node,
+ unsigned int n)
{
- unsigned char sha1[20], commit_sha1[20];
- unsigned mode;
+ unsigned char commit_sha1[20];
+ unsigned int prefix_len;
+ int status;
+ void *buf;
struct tree_desc desc;
struct name_entry entry;
- void *buf;
+
+ buf = fill_tree_descriptor(&desc, subtree->val_sha1);
+ if (!buf)
+ die("Could not read %s for notes-index",
+ sha1_to_hex(subtree->val_sha1));
+
+ prefix_len = subtree->key_sha1[19];
+ assert(prefix_len * 2 >= n);
+ memcpy(commit_sha1, subtree->key_sha1, prefix_len);
+ while (tree_entry(&desc, &entry)) {
+ int len = get_sha1_hex_segment(entry.path, strlen(entry.path),
+ commit_sha1 + prefix_len, 20 - prefix_len);
+ if (len < 0)
+ continue; /* entry.path is not a SHA1 sum. Skip */
+ len += prefix_len;
+
+ /*
+ * If commit SHA1 is complete (len == 20), assume note object
+ * If commit SHA1 is incomplete (len < 20), assume note subtree
+ */
+ if (len <= 20) {
+ unsigned char type = PTR_TYPE_NOTE;
+ struct leaf_node *l = (struct leaf_node *)
+ xcalloc(sizeof(struct leaf_node), 1);
+ hashcpy(l->key_sha1, commit_sha1);
+ hashcpy(l->val_sha1, entry.sha1);
+ if (len < 20) {
+ l->key_sha1[19] = (unsigned char) len;
+ type = PTR_TYPE_SUBTREE;
+ }
+ status = note_tree_insert(node, n, l, type);
+ assert(!status);
+ }
+ }
+ free(buf);
+}
+
+static void initialize_notes(const char *notes_ref_name)
+{
+ unsigned char sha1[20], commit_sha1[20];
+ unsigned mode;
+ struct leaf_node root_tree;
if (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||
get_tree_entry(commit_sha1, "", sha1, &mode))
return;
- buf = fill_tree_descriptor(&desc, sha1);
- if (!buf)
- die("Could not read %s for notes-index", sha1_to_hex(sha1));
-
- while (tree_entry(&desc, &entry))
- if (!get_sha1(entry.path, commit_sha1))
- add_entry(commit_sha1, entry.sha1);
- free(buf);
+ hashclr(root_tree.key_sha1);
+ hashcpy(root_tree.val_sha1, sha1);
+ load_subtree(&root_tree, &root_node, 0);
}
static unsigned char *lookup_notes(const unsigned char *commit_sha1)
{
- int index;
-
- if (!hash_map.size)
- return NULL;
-
- index = hash_index(&hash_map, commit_sha1);
- if (index < 0)
- return NULL;
- return hash_map.entries[index].notes_sha1;
+ struct leaf_node *found = note_tree_find(&root_node, 0, commit_sha1);
+ if (found)
+ return found->val_sha1;
+ return NULL;
}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
@@ -120,7 +299,7 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
else if (!notes_ref_name)
notes_ref_name = GIT_NOTES_DEFAULT_REF;
- initialize_hash_map(notes_ref_name);
+ initialize_notes(notes_ref_name);
initialized = 1;
}
@@ -163,7 +342,7 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
void free_commit_notes()
{
- free(hash_map.entries);
- memset(&hash_map, 0, sizeof(struct hash_map));
+ note_tree_free(&root_node);
+ memset(&root_node, 0, sizeof(struct int_node));
initialized = 0;
}
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 03/14] Speed up git notes lookup
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To avoid looking up each and every commit in the notes ref's tree
object, which is very expensive, speed things up by slurping the tree
object's contents into a hash_map.
The idea for the hashmap singleton is from David Reiss, initial
benchmarking by Jeff King.
Note: the implementation allows for arbitrary entries in the notes
tree object, ignoring those that do not reference a valid object. This
allows you to annotate arbitrary branches, or objects.
This patch has been improved by the following contributions:
- Junio C Hamano: fixed an obvious error in initialize_hash_map()
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
notes.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
1 files changed, 102 insertions(+), 10 deletions(-)
diff --git a/notes.c b/notes.c
index 401966d..9172154 100644
--- a/notes.c
+++ b/notes.c
@@ -4,15 +4,112 @@
#include "refs.h"
#include "utf8.h"
#include "strbuf.h"
+#include "tree-walk.h"
+
+struct entry {
+ unsigned char commit_sha1[20];
+ unsigned char notes_sha1[20];
+};
+
+struct hash_map {
+ struct entry *entries;
+ off_t count, size;
+};
static int initialized;
+static struct hash_map hash_map;
+
+static int hash_index(struct hash_map *map, const unsigned char *sha1)
+{
+ int i = ((*(unsigned int *)sha1) % map->size);
+
+ for (;;) {
+ unsigned char *current = map->entries[i].commit_sha1;
+
+ if (!hashcmp(sha1, current))
+ return i;
+
+ if (is_null_sha1(current))
+ return -1 - i;
+
+ if (++i == map->size)
+ i = 0;
+ }
+}
+
+static void add_entry(const unsigned char *commit_sha1,
+ const unsigned char *notes_sha1)
+{
+ int index;
+
+ if (hash_map.count + 1 > hash_map.size >> 1) {
+ int i, old_size = hash_map.size;
+ struct entry *old = hash_map.entries;
+
+ hash_map.size = old_size ? old_size << 1 : 64;
+ hash_map.entries = (struct entry *)
+ xcalloc(sizeof(struct entry), hash_map.size);
+
+ for (i = 0; i < old_size; i++)
+ if (!is_null_sha1(old[i].commit_sha1)) {
+ index = -1 - hash_index(&hash_map,
+ old[i].commit_sha1);
+ memcpy(hash_map.entries + index, old + i,
+ sizeof(struct entry));
+ }
+ free(old);
+ }
+
+ index = hash_index(&hash_map, commit_sha1);
+ if (index < 0) {
+ index = -1 - index;
+ hash_map.count++;
+ }
+
+ hashcpy(hash_map.entries[index].commit_sha1, commit_sha1);
+ hashcpy(hash_map.entries[index].notes_sha1, notes_sha1);
+}
+
+static void initialize_hash_map(const char *notes_ref_name)
+{
+ unsigned char sha1[20], commit_sha1[20];
+ unsigned mode;
+ struct tree_desc desc;
+ struct name_entry entry;
+ void *buf;
+
+ if (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||
+ get_tree_entry(commit_sha1, "", sha1, &mode))
+ return;
+
+ buf = fill_tree_descriptor(&desc, sha1);
+ if (!buf)
+ die("Could not read %s for notes-index", sha1_to_hex(sha1));
+
+ while (tree_entry(&desc, &entry))
+ if (!get_sha1(entry.path, commit_sha1))
+ add_entry(commit_sha1, entry.sha1);
+ free(buf);
+}
+
+static unsigned char *lookup_notes(const unsigned char *commit_sha1)
+{
+ int index;
+
+ if (!hash_map.size)
+ return NULL;
+
+ index = hash_index(&hash_map, commit_sha1);
+ if (index < 0)
+ return NULL;
+ return hash_map.entries[index].notes_sha1;
+}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
const char *output_encoding)
{
static const char utf8[] = "utf-8";
- struct strbuf name = STRBUF_INIT;
- unsigned char sha1[20];
+ unsigned char *sha1;
char *msg, *msg_p;
unsigned long linelen, msglen;
enum object_type type;
@@ -23,17 +120,12 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
else if (!notes_ref_name)
notes_ref_name = GIT_NOTES_DEFAULT_REF;
- if (notes_ref_name && read_ref(notes_ref_name, sha1))
- notes_ref_name = NULL;
+ initialize_hash_map(notes_ref_name);
initialized = 1;
}
- if (!notes_ref_name)
- return;
-
- strbuf_addf(&name, "%s:%s", notes_ref_name,
- sha1_to_hex(commit->object.sha1));
- if (get_sha1(name.buf, sha1))
+ sha1 = lookup_notes(commit->object.sha1);
+ if (!sha1)
return;
if (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 08/14] Add flags to get_commit_notes() to control the format of the note string
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
This patch adds the following flags to get_commit_notes() for adjusting the
format of the produced note string:
- NOTES_SHOW_HEADER: Print "Notes:" line before the notes contents
- NOTES_INDENT: Indent notes contents by 4 spaces
Suggested-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
---
notes.c | 8 +++++---
notes.h | 5 ++++-
pretty.c | 3 ++-
3 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/notes.c b/notes.c
index 9172154..84c30c1 100644
--- a/notes.c
+++ b/notes.c
@@ -106,7 +106,7 @@ static unsigned char *lookup_notes(const unsigned char *commit_sha1)
}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
- const char *output_encoding)
+ const char *output_encoding, int flags)
{
static const char utf8[] = "utf-8";
unsigned char *sha1;
@@ -146,12 +146,14 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
if (msglen && msg[msglen - 1] == '\n')
msglen--;
- strbuf_addstr(sb, "\nNotes:\n");
+ if (flags & NOTES_SHOW_HEADER)
+ strbuf_addstr(sb, "\nNotes:\n");
for (msg_p = msg; msg_p < msg + msglen; msg_p += linelen + 1) {
linelen = strchrnul(msg_p, '\n') - msg_p;
- strbuf_addstr(sb, " ");
+ if (flags & NOTES_INDENT)
+ strbuf_addstr(sb, " ");
strbuf_add(sb, msg_p, linelen);
strbuf_addch(sb, '\n');
}
diff --git a/notes.h b/notes.h
index 79d21b6..7f3eed4 100644
--- a/notes.h
+++ b/notes.h
@@ -1,7 +1,10 @@
#ifndef NOTES_H
#define NOTES_H
+#define NOTES_SHOW_HEADER 1
+#define NOTES_INDENT 2
+
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
- const char *output_encoding);
+ const char *output_encoding, int flags);
#endif
diff --git a/pretty.c b/pretty.c
index e25db81..01eadd0 100644
--- a/pretty.c
+++ b/pretty.c
@@ -978,7 +978,8 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit,
strbuf_addch(sb, '\n');
if (fmt != CMIT_FMT_ONELINE)
- get_commit_notes(commit, sb, encoding);
+ get_commit_notes(commit, sb, encoding,
+ NOTES_SHOW_HEADER | NOTES_INDENT);
free(reencoded);
}
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 06/14] fast-import: Add support for importing commit notes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
Introduce a 'notemodify' subcommand of the 'commit' command. This subcommand
is similar to 'filemodify', except that no mode is supplied (all notes have
mode 0644), and the path is set to the hex SHA1 of the given "comittish".
This enables fast import of note objects along with their associated commits,
since the notes can now be named using the mark references of their
corresponding commits.
The patch also includes a test case of the added functionality.
Signed-off-by: Johan Herland <johan@herland.net>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-fast-import.txt | 45 +++++++++--
fast-import.c | 88 +++++++++++++++++++-
t/t9300-fast-import.sh | 166 +++++++++++++++++++++++++++++++++++++
3 files changed, 289 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index f1c94b4..bb198c2 100644
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -325,7 +325,7 @@ change to the project.
data
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
- (filemodify | filedelete | filecopy | filerename | filedeleteall)*
+ (filemodify | filedelete | filecopy | filerename | filedeleteall | notemodify)*
LF?
....
@@ -348,14 +348,13 @@ commit message use a 0 length data. Commit messages are free-form
and are not interpreted by Git. Currently they must be encoded in
UTF-8, as fast-import does not permit other encodings to be specified.
-Zero or more `filemodify`, `filedelete`, `filecopy`, `filerename`
-and `filedeleteall` commands
+Zero or more `filemodify`, `filedelete`, `filecopy`, `filerename`,
+`filedeleteall` and `notemodify` commands
may be included to update the contents of the branch prior to
creating the commit. These commands may be supplied in any order.
However it is recommended that a `filedeleteall` command precede
-all `filemodify`, `filecopy` and `filerename` commands in the same
-commit, as `filedeleteall`
-wipes the branch clean (see below).
+all `filemodify`, `filecopy`, `filerename` and `notemodify` commands in
+the same commit, as `filedeleteall` wipes the branch clean (see below).
The `LF` after the command is optional (it used to be required).
@@ -604,6 +603,40 @@ more memory per active branch (less than 1 MiB for even most large
projects); so frontends that can easily obtain only the affected
paths for a commit are encouraged to do so.
+`notemodify`
+^^^^^^^^^^^^
+Included in a `commit` command to add a new note (annotating a given
+commit) or change the content of an existing note. This command has
+two different means of specifying the content of the note.
+
+External data format::
+ The data content for the note was already supplied by a prior
+ `blob` command. The frontend just needs to connect it to the
+ commit that is to be annotated.
++
+....
+ 'N' SP <dataref> SP <committish> LF
+....
++
+Here `<dataref>` can be either a mark reference (`:<idnum>`)
+set by a prior `blob` command, or a full 40-byte SHA-1 of an
+existing Git blob object.
+
+Inline data format::
+ The data content for the note has not been supplied yet.
+ The frontend wants to supply it as part of this modify
+ command.
++
+....
+ 'N' SP 'inline' SP <committish> LF
+ data
+....
++
+See below for a detailed description of the `data` command.
+
+In both formats `<committish>` is any of the commit specification
+expressions also accepted by `from` (see above).
+
`mark`
~~~~~~
Arranges for fast-import to save a reference to the current object, allowing
diff --git a/fast-import.c b/fast-import.c
index dcfb8fa..1e91358 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -22,8 +22,8 @@ Format of STDIN stream:
('author' sp name sp '<' email '>' sp when lf)?
'committer' sp name sp '<' email '>' sp when lf
commit_msg
- ('from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)?
- ('merge' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)*
+ ('from' sp committish lf)?
+ ('merge' sp committish lf)*
file_change*
lf?;
commit_msg ::= data;
@@ -41,15 +41,18 @@ Format of STDIN stream:
file_obm ::= 'M' sp mode sp (hexsha1 | idnum) sp path_str lf;
file_inm ::= 'M' sp mode sp 'inline' sp path_str lf
data;
+ note_obm ::= 'N' sp (hexsha1 | idnum) sp committish lf;
+ note_inm ::= 'N' sp 'inline' sp committish lf
+ data;
new_tag ::= 'tag' sp tag_str lf
- 'from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf
+ 'from' sp committish lf
('tagger' sp name sp '<' email '>' sp when lf)?
tag_msg;
tag_msg ::= data;
reset_branch ::= 'reset' sp ref_str lf
- ('from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)?
+ ('from' sp committish lf)?
lf?;
checkpoint ::= 'checkpoint' lf
@@ -88,6 +91,7 @@ Format of STDIN stream:
# stream formatting is: \, " and LF. Otherwise these values
# are UTF8.
#
+ committish ::= (ref_str | hexsha1 | sha1exp_str | idnum);
ref_str ::= ref;
sha1exp_str ::= sha1exp;
tag_str ::= tag;
@@ -2053,6 +2057,80 @@ static void file_change_cr(struct branch *b, int rename)
leaf.tree);
}
+static void note_change_n(struct branch *b)
+{
+ const char *p = command_buf.buf + 2;
+ static struct strbuf uq = STRBUF_INIT;
+ struct object_entry *oe = oe;
+ struct branch *s;
+ unsigned char sha1[20], commit_sha1[20];
+ uint16_t inline_data = 0;
+
+ /* <dataref> or 'inline' */
+ if (*p == ':') {
+ char *x;
+ oe = find_mark(strtoumax(p + 1, &x, 10));
+ hashcpy(sha1, oe->sha1);
+ p = x;
+ } else if (!prefixcmp(p, "inline")) {
+ inline_data = 1;
+ p += 6;
+ } else {
+ if (get_sha1_hex(p, sha1))
+ die("Invalid SHA1: %s", command_buf.buf);
+ oe = find_object(sha1);
+ p += 40;
+ }
+ if (*p++ != ' ')
+ die("Missing space after SHA1: %s", command_buf.buf);
+
+ /* <committish> */
+ s = lookup_branch(p);
+ if (s) {
+ hashcpy(commit_sha1, s->sha1);
+ } else if (*p == ':') {
+ uintmax_t commit_mark = strtoumax(p + 1, NULL, 10);
+ struct object_entry *commit_oe = find_mark(commit_mark);
+ if (commit_oe->type != OBJ_COMMIT)
+ die("Mark :%" PRIuMAX " not a commit", commit_mark);
+ hashcpy(commit_sha1, commit_oe->sha1);
+ } else if (!get_sha1(p, commit_sha1)) {
+ unsigned long size;
+ char *buf = read_object_with_reference(commit_sha1,
+ commit_type, &size, commit_sha1);
+ if (!buf || size < 46)
+ die("Not a valid commit: %s", p);
+ free(buf);
+ } else
+ die("Invalid ref name or SHA1 expression: %s", p);
+
+ if (inline_data) {
+ static struct strbuf buf = STRBUF_INIT;
+
+ if (p != uq.buf) {
+ strbuf_addstr(&uq, p);
+ p = uq.buf;
+ }
+ read_next_command();
+ parse_data(&buf);
+ store_object(OBJ_BLOB, &buf, &last_blob, sha1, 0);
+ } else if (oe) {
+ if (oe->type != OBJ_BLOB)
+ die("Not a blob (actually a %s): %s",
+ typename(oe->type), command_buf.buf);
+ } else {
+ enum object_type type = sha1_object_info(sha1, NULL);
+ if (type < 0)
+ die("Blob not found: %s", command_buf.buf);
+ if (type != OBJ_BLOB)
+ die("Not a blob (actually a %s): %s",
+ typename(type), command_buf.buf);
+ }
+
+ tree_content_set(&b->branch_tree, sha1_to_hex(commit_sha1), sha1,
+ S_IFREG | 0644, NULL);
+}
+
static void file_change_deleteall(struct branch *b)
{
release_tree_content_recursive(b->branch_tree.tree);
@@ -2222,6 +2300,8 @@ static void parse_new_commit(void)
file_change_cr(b, 1);
else if (!prefixcmp(command_buf.buf, "C "))
file_change_cr(b, 0);
+ else if (!prefixcmp(command_buf.buf, "N "))
+ note_change_n(b);
else if (!strcmp("deleteall", command_buf.buf))
file_change_deleteall(b);
else {
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index d33fc55..2f5c323 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -1089,6 +1089,172 @@ test_expect_success 'P: fail on blob mark in gitlink' '
test_must_fail git fast-import <input'
###
+### series Q (notes)
+###
+
+note1_data="Note for the first commit"
+note2_data="Note for the second commit"
+note3_data="Note for the third commit"
+
+test_tick
+cat >input <<INPUT_END
+blob
+mark :2
+data <<EOF
+$file2_data
+EOF
+
+commit refs/heads/notes-test
+mark :3
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+first (:3)
+COMMIT
+
+M 644 :2 file2
+
+blob
+mark :4
+data $file4_len
+$file4_data
+commit refs/heads/notes-test
+mark :5
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+second (:5)
+COMMIT
+
+M 644 :4 file4
+
+commit refs/heads/notes-test
+mark :6
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+third (:6)
+COMMIT
+
+M 644 inline file5
+data <<EOF
+$file5_data
+EOF
+
+M 755 inline file6
+data <<EOF
+$file6_data
+EOF
+
+blob
+mark :7
+data <<EOF
+$note1_data
+EOF
+
+blob
+mark :8
+data <<EOF
+$note2_data
+EOF
+
+commit refs/notes/foobar
+mark :9
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes (:9)
+COMMIT
+
+N :7 :3
+N :8 :5
+N inline :6
+data <<EOF
+$note3_data
+EOF
+
+INPUT_END
+test_expect_success \
+ 'Q: commit notes' \
+ 'git fast-import <input &&
+ git whatchanged notes-test'
+test_expect_success \
+ 'Q: verify pack' \
+ 'for p in .git/objects/pack/*.pack;do git verify-pack $p||exit;done'
+
+commit1=$(git rev-parse notes-test~2)
+commit2=$(git rev-parse notes-test^)
+commit3=$(git rev-parse notes-test)
+
+cat >expect <<EOF
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+first (:3)
+EOF
+test_expect_success \
+ 'Q: verify first commit' \
+ 'git cat-file commit notes-test~2 | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+parent $commit1
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+second (:5)
+EOF
+test_expect_success \
+ 'Q: verify second commit' \
+ 'git cat-file commit notes-test^ | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+parent $commit2
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+third (:6)
+EOF
+test_expect_success \
+ 'Q: verify third commit' \
+ 'git cat-file commit notes-test | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+notes (:9)
+EOF
+test_expect_success \
+ 'Q: verify notes commit' \
+ 'git cat-file commit refs/notes/foobar | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect.unsorted <<EOF
+100644 blob $commit1
+100644 blob $commit2
+100644 blob $commit3
+EOF
+cat expect.unsorted | sort >expect
+test_expect_success \
+ 'Q: verify notes tree' \
+ 'git cat-file -p refs/notes/foobar^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ test_cmp expect actual'
+
+echo "$note1_data" >expect
+test_expect_success \
+ 'Q: verify note for first commit' \
+ 'git cat-file blob refs/notes/foobar:$commit1 >actual && test_cmp expect actual'
+
+echo "$note2_data" >expect
+test_expect_success \
+ 'Q: verify note for second commit' \
+ 'git cat-file blob refs/notes/foobar:$commit2 >actual && test_cmp expect actual'
+
+echo "$note3_data" >expect
+test_expect_success \
+ 'Q: verify note for third commit' \
+ 'git cat-file blob refs/notes/foobar:$commit3 >actual && test_cmp expect actual'
+
+###
### series R (feature and option)
###
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 07/14] t3302-notes-index-expensive: Speed up create_repo()
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
Creating repos with 10/100/1000/10000 commits and notes takes a lot of time.
However, using git-fast-import to do the job is a lot more efficient than
using plumbing commands to do the same.
This patch decreases the overall run-time of this test on my machine from
~3 to ~1 minutes.
Signed-off-by: Johan Herland <johan@herland.net>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/t3302-notes-index-expensive.sh | 74 ++++++++++++++++++++++++--------------
1 files changed, 47 insertions(+), 27 deletions(-)
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 0ef3e95..ee84fc4 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -16,30 +16,50 @@ test -z "$GIT_NOTES_TIMING_TESTS" && {
create_repo () {
number_of_commits=$1
nr=0
- parent=
test -d .git || {
git init &&
- tree=$(git write-tree) &&
- while [ $nr -lt $number_of_commits ]; do
- test_tick &&
- commit=$(echo $nr | git commit-tree $tree $parent) ||
- return
- parent="-p $commit"
- nr=$(($nr+1))
- done &&
- git update-ref refs/heads/master $commit &&
- {
- GIT_INDEX_FILE=.git/temp; export GIT_INDEX_FILE;
- git rev-list HEAD | cat -n | sed "s/^[ ][ ]*/ /g" |
- while read nr sha1; do
- blob=$(echo note $nr | git hash-object -w --stdin) &&
- echo $sha1 | sed "s/^/0644 $blob 0 /"
- done | git update-index --index-info &&
- tree=$(git write-tree) &&
+ (
+ while [ $nr -lt $number_of_commits ]; do
+ nr=$(($nr+1))
+ mark=$(($nr+$nr))
+ notemark=$(($mark+1))
+ test_tick &&
+ cat <<INPUT_END &&
+commit refs/heads/master
+mark :$mark
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+commit #$nr
+COMMIT
+
+M 644 inline file
+data <<EOF
+file in commit #$nr
+EOF
+
+blob
+mark :$notemark
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+ echo "N :$notemark :$mark" >> note_commit
+ done &&
test_tick &&
- commit=$(echo notes | git commit-tree $tree) &&
- git update-ref refs/notes/commits $commit
- } &&
+ cat <<INPUT_END &&
+commit refs/notes/commits
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes
+COMMIT
+
+INPUT_END
+
+ cat note_commit
+ ) |
+ git fast-import --quiet &&
git config core.notesRef refs/notes/commits
}
}
@@ -48,13 +68,13 @@ test_notes () {
count=$1 &&
git config core.notesRef refs/notes/commits &&
git log | grep "^ " > output &&
- i=1 &&
- while [ $i -le $count ]; do
- echo " $(($count-$i))" &&
- echo " note $i" &&
- i=$(($i+1));
+ i=$count &&
+ while [ $i -gt 0 ]; do
+ echo " commit #$i" &&
+ echo " note for commit #$i" &&
+ i=$(($i-1));
done > expect &&
- git diff expect output
+ test_cmp expect output
}
cat > time_notes << \EOF
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 05/14] Teach "-m <msg>" and "-F <file>" to "git notes edit"
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
The "-m" and "-F" options are already the established method
(in both git-commit and git-tag) to specify a commit/tag message
without invoking the editor. This patch teaches "git notes edit"
to respect the same options for specifying a notes message without
invoking the editor.
Multiple "-m" and/or "-F" options are concatenated as separate
paragraphs.
The patch also updates the "git notes" documentation and adds
selftests for the new functionality. Unfortunately, the added
selftests include a couple of lines with trailing whitespace
(without these the test will fail). This may cause git to warn
about "whitespace errors".
This patch has been improved by the following contributions:
- Thomas Rast: fix trailing whitespace in t3301
Signed-off-by: Johan Herland <johan@herland.net>
---
Documentation/git-notes.txt | 16 ++++++++++-
git-notes.sh | 64 +++++++++++++++++++++++++++++++++++++-----
t/t3301-notes.sh | 36 ++++++++++++++++++++++++
3 files changed, 107 insertions(+), 9 deletions(-)
diff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt
index 7136016..94cceb1 100644
--- a/Documentation/git-notes.txt
+++ b/Documentation/git-notes.txt
@@ -8,7 +8,7 @@ git-notes - Add/inspect commit notes
SYNOPSIS
--------
[verse]
-'git-notes' (edit | show) [commit]
+'git-notes' (edit [-F <file> | -m <msg>] | show) [commit]
DESCRIPTION
-----------
@@ -33,6 +33,20 @@ show::
Show the notes for a given commit (defaults to HEAD).
+OPTIONS
+-------
+-m <msg>::
+ Use the given note message (instead of prompting).
+ If multiple `-m` (or `-F`) options are given, their
+ values are concatenated as separate paragraphs.
+
+-F <file>::
+ Take the note message from the given file. Use '-' to
+ read the note message from the standard input.
+ If multiple `-F` (or `-m`) options are given, their
+ values are concatenated as separate paragraphs.
+
+
Author
------
Written by Johannes Schindelin <johannes.schindelin@gmx.de>
diff --git a/git-notes.sh b/git-notes.sh
index f06c254..e642e47 100755
--- a/git-notes.sh
+++ b/git-notes.sh
@@ -1,16 +1,59 @@
#!/bin/sh
-USAGE="(edit | show) [commit]"
+USAGE="(edit [-F <file> | -m <msg>] | show) [commit]"
. git-sh-setup
-test -n "$3" && usage
-
test -z "$1" && usage
ACTION="$1"; shift
test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="$(git config core.notesref)"
test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="refs/notes/commits"
+MESSAGE=
+while test $# != 0
+do
+ case "$1" in
+ -m)
+ test "$ACTION" = "edit" || usage
+ shift
+ if test "$#" = "0"; then
+ die "error: option -m needs an argument"
+ else
+ if [ -z "$MESSAGE" ]; then
+ MESSAGE="$1"
+ else
+ MESSAGE="$MESSAGE
+
+$1"
+ fi
+ shift
+ fi
+ ;;
+ -F)
+ test "$ACTION" = "edit" || usage
+ shift
+ if test "$#" = "0"; then
+ die "error: option -F needs an argument"
+ else
+ if [ -z "$MESSAGE" ]; then
+ MESSAGE="$(cat "$1")"
+ else
+ MESSAGE="$MESSAGE
+
+$(cat "$1")"
+ fi
+ shift
+ fi
+ ;;
+ -*)
+ usage
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
COMMIT=$(git rev-parse --verify --default HEAD "$@") ||
die "Invalid commit: $@"
@@ -29,19 +72,24 @@ edit)
test -f "$GIT_INDEX_FILE" && rm "$GIT_INDEX_FILE"
' 0
- GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
-
CURRENT_HEAD=$(git show-ref "$GIT_NOTES_REF" | cut -f 1 -d ' ')
if [ -z "$CURRENT_HEAD" ]; then
PARENT=
else
PARENT="-p $CURRENT_HEAD"
git read-tree "$GIT_NOTES_REF" || die "Could not read index"
- git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
fi
- core_editor="$(git config core.editor)"
- ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+ if [ -z "$MESSAGE" ]; then
+ GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
+ if [ ! -z "$CURRENT_HEAD" ]; then
+ git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
+ fi
+ core_editor="$(git config core.editor)"
+ ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+ else
+ echo "$MESSAGE" > "$MSG_FILE"
+ fi
grep -v ^# < "$MSG_FILE" | git stripspace > "$MSG_FILE".processed
mv "$MSG_FILE".processed "$MSG_FILE"
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
index 73e53be..1e34f48 100755
--- a/t/t3301-notes.sh
+++ b/t/t3301-notes.sh
@@ -110,5 +110,41 @@ test_expect_success 'show multi-line notes' '
git log -2 > output &&
test_cmp expect-multiline output
'
+test_expect_success 'create -m and -F notes (setup)' '
+ : > a4 &&
+ git add a4 &&
+ test_tick &&
+ git commit -m 4th &&
+ echo "xyzzy" > note5 &&
+ git notes edit -m spam -F note5 -m "foo
+bar
+baz"
+'
+
+whitespace=" "
+cat > expect-m-and-F << EOF
+commit 15023535574ded8b1a89052b32673f84cf9582b8
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:16:13 2005 -0700
+
+ 4th
+
+Notes:
+ spam
+$whitespace
+ xyzzy
+$whitespace
+ foo
+ bar
+ baz
+EOF
+
+printf "\n" >> expect-m-and-F
+cat expect-multiline >> expect-m-and-F
+
+test_expect_success 'show -m and -F notes' '
+ git log -3 > output &&
+ test_cmp expect-m-and-F output
+'
test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 04/14] Add an expensive test for git-notes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
git-notes have the potential of being pretty expensive, so test with
a lot of commits. A lot. So to make things cheaper, you have to
opt-in explicitely, by setting the environment variable
GIT_NOTES_TIMING_TESTS.
This patch has been improved by the following contributions:
- Junio C Hamano: tests: fix "export var=val"
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
t/t3302-notes-index-expensive.sh | 98 ++++++++++++++++++++++++++++++++++++++
1 files changed, 98 insertions(+), 0 deletions(-)
create mode 100755 t/t3302-notes-index-expensive.sh
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
new file mode 100755
index 0000000..0ef3e95
--- /dev/null
+++ b/t/t3302-notes-index-expensive.sh
@@ -0,0 +1,98 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Johannes E. Schindelin
+#
+
+test_description='Test commit notes index (expensive!)'
+
+. ./test-lib.sh
+
+test -z "$GIT_NOTES_TIMING_TESTS" && {
+ say Skipping timing tests
+ test_done
+ exit
+}
+
+create_repo () {
+ number_of_commits=$1
+ nr=0
+ parent=
+ test -d .git || {
+ git init &&
+ tree=$(git write-tree) &&
+ while [ $nr -lt $number_of_commits ]; do
+ test_tick &&
+ commit=$(echo $nr | git commit-tree $tree $parent) ||
+ return
+ parent="-p $commit"
+ nr=$(($nr+1))
+ done &&
+ git update-ref refs/heads/master $commit &&
+ {
+ GIT_INDEX_FILE=.git/temp; export GIT_INDEX_FILE;
+ git rev-list HEAD | cat -n | sed "s/^[ ][ ]*/ /g" |
+ while read nr sha1; do
+ blob=$(echo note $nr | git hash-object -w --stdin) &&
+ echo $sha1 | sed "s/^/0644 $blob 0 /"
+ done | git update-index --index-info &&
+ tree=$(git write-tree) &&
+ test_tick &&
+ commit=$(echo notes | git commit-tree $tree) &&
+ git update-ref refs/notes/commits $commit
+ } &&
+ git config core.notesRef refs/notes/commits
+ }
+}
+
+test_notes () {
+ count=$1 &&
+ git config core.notesRef refs/notes/commits &&
+ git log | grep "^ " > output &&
+ i=1 &&
+ while [ $i -le $count ]; do
+ echo " $(($count-$i))" &&
+ echo " note $i" &&
+ i=$(($i+1));
+ done > expect &&
+ git diff expect output
+}
+
+cat > time_notes << \EOF
+ mode=$1
+ i=1
+ while [ $i -lt $2 ]; do
+ case $1 in
+ no-notes)
+ GIT_NOTES_REF=non-existing; export GIT_NOTES_REF
+ ;;
+ notes)
+ unset GIT_NOTES_REF
+ ;;
+ esac
+ git log >/dev/null
+ i=$(($i+1))
+ done
+EOF
+
+time_notes () {
+ for mode in no-notes notes
+ do
+ echo $mode
+ /usr/bin/time sh ../time_notes $mode $1
+ done
+}
+
+for count in 10 100 1000 10000; do
+
+ mkdir $count
+ (cd $count;
+
+ test_expect_success "setup $count" "create_repo $count"
+
+ test_expect_success 'notes work' "test_notes $count"
+
+ test_expect_success 'notes timing' "time_notes 100"
+ )
+done
+
+test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 02/14] Add a script to edit/inspect notes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
The script 'git notes' allows you to edit and show commit notes, by
calling either
git notes show <commit>
or
git notes edit <commit>
This patch has been improved by the following contributions:
- Tor Arne Vestbø: fix printing of multi-line notes
- Michael J Gruber: test and handle empty notes gracefully
- Thomas Rast:
- only clean up message file when editing
- use GIT_EDITOR and core.editor over VISUAL/EDITOR
- t3301: fix confusing quoting in test for valid notes ref
- t3301: use test_must_fail instead of !
- refuse to edit notes outside refs/notes/
- Junio C Hamano: tests: fix "export var=val"
- Christian Couder: documentation: fix 'linkgit' macro in "git-notes.txt"
- Johan Herland: minor cleanup and bugfixing in git-notes.sh (v2)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Tor Arne Vestbø <tavestbo@trolltech.com>
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
.gitignore | 1 +
Documentation/git-notes.txt | 46 +++++++++++++++++
Makefile | 1 +
command-list.txt | 1 +
git-notes.sh | 73 +++++++++++++++++++++++++++
t/t3301-notes.sh | 114 +++++++++++++++++++++++++++++++++++++++++++
6 files changed, 236 insertions(+), 0 deletions(-)
create mode 100644 Documentation/git-notes.txt
create mode 100755 git-notes.sh
create mode 100755 t/t3301-notes.sh
diff --git a/.gitignore b/.gitignore
index c446290..703241b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -86,6 +86,7 @@ git-mktag
git-mktree
git-name-rev
git-mv
+git-notes
git-pack-redundant
git-pack-objects
git-pack-refs
diff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt
new file mode 100644
index 0000000..7136016
--- /dev/null
+++ b/Documentation/git-notes.txt
@@ -0,0 +1,46 @@
+git-notes(1)
+============
+
+NAME
+----
+git-notes - Add/inspect commit notes
+
+SYNOPSIS
+--------
+[verse]
+'git-notes' (edit | show) [commit]
+
+DESCRIPTION
+-----------
+This command allows you to add notes to commit messages, without
+changing the commit. To discern these notes from the message stored
+in the commit object, the notes are indented like the message, after
+an unindented line saying "Notes:".
+
+To disable commit notes, you have to set the config variable
+core.notesRef to the empty string. Alternatively, you can set it
+to a different ref, something like "refs/notes/bugzilla". This setting
+can be overridden by the environment variable "GIT_NOTES_REF".
+
+
+SUBCOMMANDS
+-----------
+
+edit::
+ Edit the notes for a given commit (defaults to HEAD).
+
+show::
+ Show the notes for a given commit (defaults to HEAD).
+
+
+Author
+------
+Written by Johannes Schindelin <johannes.schindelin@gmx.de>
+
+Documentation
+-------------
+Documentation by Johannes Schindelin
+
+GIT
+---
+Part of the linkgit:git[7] suite
diff --git a/Makefile b/Makefile
index 37b8f85..8e8376c 100644
--- a/Makefile
+++ b/Makefile
@@ -316,6 +316,7 @@ SCRIPT_SH += git-merge-one-file.sh
SCRIPT_SH += git-merge-resolve.sh
SCRIPT_SH += git-mergetool.sh
SCRIPT_SH += git-mergetool--lib.sh
+SCRIPT_SH += git-notes.sh
SCRIPT_SH += git-parse-remote.sh
SCRIPT_SH += git-pull.sh
SCRIPT_SH += git-quiltimport.sh
diff --git a/command-list.txt b/command-list.txt
index fb03a2e..4296941 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -74,6 +74,7 @@ git-mktag plumbingmanipulators
git-mktree plumbingmanipulators
git-mv mainporcelain common
git-name-rev plumbinginterrogators
+git-notes mainporcelain
git-pack-objects plumbingmanipulators
git-pack-redundant plumbinginterrogators
git-pack-refs ancillarymanipulators
diff --git a/git-notes.sh b/git-notes.sh
new file mode 100755
index 0000000..f06c254
--- /dev/null
+++ b/git-notes.sh
@@ -0,0 +1,73 @@
+#!/bin/sh
+
+USAGE="(edit | show) [commit]"
+. git-sh-setup
+
+test -n "$3" && usage
+
+test -z "$1" && usage
+ACTION="$1"; shift
+
+test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="$(git config core.notesref)"
+test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="refs/notes/commits"
+
+COMMIT=$(git rev-parse --verify --default HEAD "$@") ||
+die "Invalid commit: $@"
+
+case "$ACTION" in
+edit)
+ if [ "${GIT_NOTES_REF#refs/notes/}" = "$GIT_NOTES_REF" ]; then
+ die "Refusing to edit notes in $GIT_NOTES_REF (outside of refs/notes/)"
+ fi
+
+ MSG_FILE="$GIT_DIR/new-notes-$COMMIT"
+ GIT_INDEX_FILE="$MSG_FILE.idx"
+ export GIT_INDEX_FILE
+
+ trap '
+ test -f "$MSG_FILE" && rm "$MSG_FILE"
+ test -f "$GIT_INDEX_FILE" && rm "$GIT_INDEX_FILE"
+ ' 0
+
+ GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
+
+ CURRENT_HEAD=$(git show-ref "$GIT_NOTES_REF" | cut -f 1 -d ' ')
+ if [ -z "$CURRENT_HEAD" ]; then
+ PARENT=
+ else
+ PARENT="-p $CURRENT_HEAD"
+ git read-tree "$GIT_NOTES_REF" || die "Could not read index"
+ git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
+ fi
+
+ core_editor="$(git config core.editor)"
+ ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+
+ grep -v ^# < "$MSG_FILE" | git stripspace > "$MSG_FILE".processed
+ mv "$MSG_FILE".processed "$MSG_FILE"
+ if [ -s "$MSG_FILE" ]; then
+ BLOB=$(git hash-object -w "$MSG_FILE") ||
+ die "Could not write into object database"
+ git update-index --add --cacheinfo 0644 $BLOB $COMMIT ||
+ die "Could not write index"
+ else
+ test -z "$CURRENT_HEAD" &&
+ die "Will not initialise with empty tree"
+ git update-index --force-remove $COMMIT ||
+ die "Could not update index"
+ fi
+
+ TREE=$(git write-tree) || die "Could not write tree"
+ NEW_HEAD=$(echo Annotate $COMMIT | git commit-tree $TREE $PARENT) ||
+ die "Could not annotate"
+ git update-ref -m "Annotate $COMMIT" \
+ "$GIT_NOTES_REF" $NEW_HEAD $CURRENT_HEAD
+;;
+show)
+ git rev-parse -q --verify "$GIT_NOTES_REF":$COMMIT > /dev/null ||
+ die "No note for commit $COMMIT."
+ git show "$GIT_NOTES_REF":$COMMIT
+;;
+*)
+ usage
+esac
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
new file mode 100755
index 0000000..73e53be
--- /dev/null
+++ b/t/t3301-notes.sh
@@ -0,0 +1,114 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Johannes E. Schindelin
+#
+
+test_description='Test commit notes'
+
+. ./test-lib.sh
+
+cat > fake_editor.sh << \EOF
+echo "$MSG" > "$1"
+echo "$MSG" >& 2
+EOF
+chmod a+x fake_editor.sh
+VISUAL=./fake_editor.sh
+export VISUAL
+
+test_expect_success 'cannot annotate non-existing HEAD' '
+ (MSG=3 && export MSG && test_must_fail git notes edit)
+'
+
+test_expect_success setup '
+ : > a1 &&
+ git add a1 &&
+ test_tick &&
+ git commit -m 1st &&
+ : > a2 &&
+ git add a2 &&
+ test_tick &&
+ git commit -m 2nd
+'
+
+test_expect_success 'need valid notes ref' '
+ (MSG=1 GIT_NOTES_REF=/ && export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit) &&
+ (MSG=2 GIT_NOTES_REF=/ && export MSG GIT_NOTES_REF &&
+ test_must_fail git notes show)
+'
+
+test_expect_success 'refusing to edit in refs/heads/' '
+ (MSG=1 GIT_NOTES_REF=refs/heads/bogus &&
+ export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit)
+'
+
+test_expect_success 'refusing to edit in refs/remotes/' '
+ (MSG=1 GIT_NOTES_REF=refs/remotes/bogus &&
+ export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit)
+'
+
+# 1 indicates caught gracefully by die, 128 means git-show barked
+test_expect_success 'handle empty notes gracefully' '
+ git notes show ; test 1 = $?
+'
+
+test_expect_success 'create notes' '
+ git config core.notesRef refs/notes/commits &&
+ MSG=b1 git notes edit &&
+ test ! -f .git/new-notes &&
+ test 1 = $(git ls-tree refs/notes/commits | wc -l) &&
+ test b1 = $(git notes show) &&
+ git show HEAD^ &&
+ test_must_fail git notes show HEAD^
+'
+
+cat > expect << EOF
+commit 268048bfb8a1fb38e703baceb8ab235421bf80c5
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:14:13 2005 -0700
+
+ 2nd
+
+Notes:
+ b1
+EOF
+
+test_expect_success 'show notes' '
+ ! (git cat-file commit HEAD | grep b1) &&
+ git log -1 > output &&
+ test_cmp expect output
+'
+test_expect_success 'create multi-line notes (setup)' '
+ : > a3 &&
+ git add a3 &&
+ test_tick &&
+ git commit -m 3rd &&
+ MSG="b3
+c3c3c3c3
+d3d3d3" git notes edit
+'
+
+cat > expect-multiline << EOF
+commit 1584215f1d29c65e99c6c6848626553fdd07fd75
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:15:13 2005 -0700
+
+ 3rd
+
+Notes:
+ b3
+ c3c3c3c3
+ d3d3d3
+EOF
+
+printf "\n" >> expect-multiline
+cat expect >> expect-multiline
+
+test_expect_success 'show multi-line notes' '
+ git log -2 > output &&
+ test_cmp expect-multiline output
+'
+
+test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 01/14] Introduce commit notes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1252376822-6138-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Commit notes are blobs which are shown together with the commit
message. These blobs are taken from the notes ref, which you can
configure by the config variable core.notesRef, which in turn can
be overridden by the environment variable GIT_NOTES_REF.
The notes ref is a branch which contains "files" whose names are
the names of the corresponding commits (i.e. the SHA-1).
The rationale for putting this information into a ref is this: we
want to be able to fetch and possibly union-merge the notes,
maybe even look at the date when a note was introduced, and we
want to store them efficiently together with the other objects.
This patch has been improved by the following contributions:
- Thomas Rast: fix core.notesRef documentation
- Tor Arne Vestbø: fix printing of multi-line notes
- Alex Riesen: Using char array instead of char pointer costs less BSS
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Tor Arne Vestbø <tavestbo@trolltech.com>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Documentation/config.txt | 13 +++++++++
Makefile | 2 +
cache.h | 4 +++
commit.c | 1 +
config.c | 5 +++
environment.c | 1 +
notes.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++
notes.h | 7 +++++
pretty.c | 5 +++
9 files changed, 106 insertions(+), 0 deletions(-)
create mode 100644 notes.c
create mode 100644 notes.h
diff --git a/Documentation/config.txt b/Documentation/config.txt
index d445395..728d787 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -443,6 +443,19 @@ On some file system/operating system combinations, this is unreliable.
Set this config setting to 'rename' there; However, This will remove the
check that makes sure that existing object files will not get overwritten.
+core.notesRef::
+ When showing commit messages, also show notes which are stored in
+ the given ref. This ref is expected to contain files named
+ after the full SHA-1 of the commit they annotate.
++
+If such a file exists in the given ref, the referenced blob is read, and
+appended to the commit message, separated by a "Notes:" line. If the
+given ref itself does not exist, it is not an error, but means that no
+notes should be printed.
++
+This setting defaults to "refs/notes/commits", and can be overridden by
+the `GIT_NOTES_REF` environment variable.
+
add.ignore-errors::
Tells 'git-add' to continue adding files when some files cannot be
added due to indexing errors. Equivalent to the '--ignore-errors'
diff --git a/Makefile b/Makefile
index ce882d0..37b8f85 100644
--- a/Makefile
+++ b/Makefile
@@ -426,6 +426,7 @@ LIB_H += ll-merge.h
LIB_H += log-tree.h
LIB_H += mailmap.h
LIB_H += merge-recursive.h
+LIB_H += notes.h
LIB_H += object.h
LIB_H += pack.h
LIB_H += pack-refs.h
@@ -509,6 +510,7 @@ LIB_OBJS += match-trees.o
LIB_OBJS += merge-file.o
LIB_OBJS += merge-recursive.o
LIB_OBJS += name-hash.o
+LIB_OBJS += notes.o
LIB_OBJS += object.o
LIB_OBJS += pack-check.o
LIB_OBJS += pack-refs.o
diff --git a/cache.h b/cache.h
index ae324c9..917138b 100644
--- a/cache.h
+++ b/cache.h
@@ -371,6 +371,8 @@ static inline enum object_type object_type(unsigned int mode)
#define GITATTRIBUTES_FILE ".gitattributes"
#define INFOATTRIBUTES_FILE "info/attributes"
#define ATTRIBUTE_MACRO_PREFIX "[attr]"
+#define GIT_NOTES_REF_ENVIRONMENT "GIT_NOTES_REF"
+#define GIT_NOTES_DEFAULT_REF "refs/notes/commits"
extern int is_bare_repository_cfg;
extern int is_bare_repository(void);
@@ -565,6 +567,8 @@ enum object_creation_mode {
extern enum object_creation_mode object_creation_mode;
+extern char *notes_ref_name;
+
extern int grafts_replace_parents;
#define GIT_REPO_VERSION 0
diff --git a/commit.c b/commit.c
index a6c6f70..a0a77a6 100644
--- a/commit.c
+++ b/commit.c
@@ -5,6 +5,7 @@
#include "utf8.h"
#include "diff.h"
#include "revision.h"
+#include "notes.h"
int save_commit_buffer = 1;
diff --git a/config.c b/config.c
index e87edea..70a7d34 100644
--- a/config.c
+++ b/config.c
@@ -467,6 +467,11 @@ static int git_default_core_config(const char *var, const char *value)
return 0;
}
+ if (!strcmp(var, "core.notesref")) {
+ notes_ref_name = xstrdup(value);
+ return 0;
+ }
+
if (!strcmp(var, "core.pager"))
return git_config_string(&pager_program, var, value);
diff --git a/environment.c b/environment.c
index 5de6837..571ab56 100644
--- a/environment.c
+++ b/environment.c
@@ -49,6 +49,7 @@ enum push_default_type push_default = PUSH_DEFAULT_MATCHING;
#define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS
#endif
enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
+char *notes_ref_name;
int grafts_replace_parents = 1;
/* Parallel index stat data preload? */
diff --git a/notes.c b/notes.c
new file mode 100644
index 0000000..401966d
--- /dev/null
+++ b/notes.c
@@ -0,0 +1,68 @@
+#include "cache.h"
+#include "commit.h"
+#include "notes.h"
+#include "refs.h"
+#include "utf8.h"
+#include "strbuf.h"
+
+static int initialized;
+
+void get_commit_notes(const struct commit *commit, struct strbuf *sb,
+ const char *output_encoding)
+{
+ static const char utf8[] = "utf-8";
+ struct strbuf name = STRBUF_INIT;
+ unsigned char sha1[20];
+ char *msg, *msg_p;
+ unsigned long linelen, msglen;
+ enum object_type type;
+
+ if (!initialized) {
+ const char *env = getenv(GIT_NOTES_REF_ENVIRONMENT);
+ if (env)
+ notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
+ else if (!notes_ref_name)
+ notes_ref_name = GIT_NOTES_DEFAULT_REF;
+ if (notes_ref_name && read_ref(notes_ref_name, sha1))
+ notes_ref_name = NULL;
+ initialized = 1;
+ }
+
+ if (!notes_ref_name)
+ return;
+
+ strbuf_addf(&name, "%s:%s", notes_ref_name,
+ sha1_to_hex(commit->object.sha1));
+ if (get_sha1(name.buf, sha1))
+ return;
+
+ if (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||
+ type != OBJ_BLOB)
+ return;
+
+ if (output_encoding && *output_encoding &&
+ strcmp(utf8, output_encoding)) {
+ char *reencoded = reencode_string(msg, output_encoding, utf8);
+ if (reencoded) {
+ free(msg);
+ msg = reencoded;
+ msglen = strlen(msg);
+ }
+ }
+
+ /* we will end the annotation by a newline anyway */
+ if (msglen && msg[msglen - 1] == '\n')
+ msglen--;
+
+ strbuf_addstr(sb, "\nNotes:\n");
+
+ for (msg_p = msg; msg_p < msg + msglen; msg_p += linelen + 1) {
+ linelen = strchrnul(msg_p, '\n') - msg_p;
+
+ strbuf_addstr(sb, " ");
+ strbuf_add(sb, msg_p, linelen);
+ strbuf_addch(sb, '\n');
+ }
+
+ free(msg);
+}
diff --git a/notes.h b/notes.h
new file mode 100644
index 0000000..79d21b6
--- /dev/null
+++ b/notes.h
@@ -0,0 +1,7 @@
+#ifndef NOTES_H
+#define NOTES_H
+
+void get_commit_notes(const struct commit *commit, struct strbuf *sb,
+ const char *output_encoding);
+
+#endif
diff --git a/pretty.c b/pretty.c
index f5983f8..e25db81 100644
--- a/pretty.c
+++ b/pretty.c
@@ -6,6 +6,7 @@
#include "string-list.h"
#include "mailmap.h"
#include "log-tree.h"
+#include "notes.h"
#include "color.h"
static char *user_format;
@@ -975,5 +976,9 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit,
*/
if (fmt == CMIT_FMT_EMAIL && sb->len <= beginning_of_body)
strbuf_addch(sb, '\n');
+
+ if (fmt != CMIT_FMT_ONELINE)
+ get_commit_notes(commit, sb, encoding);
+
free(reencoded);
}
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv5 00/14] git notes
From: Johan Herland @ 2009-09-08 2:26 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
Yet another iteration of the 'git notes' feature. Rebased on top of 'next':
- Patches 1-9 are unchanged from (patches 1-7, 11-12 of) the last iteration.
- Patch 10 teaches the notes code to free its data structures on request.
- Patch 11 introduces the 16-tree notes lookup code that handles SHA1-based
fanout schemes. This is pretty much unchanged from patch 8 in the previous
iteration.
- Patch 12 adds selftests that verify correct parsing of notes trees with
various SHA1-based fanouts.
- Patch 13 introduces a flexible parser for a variety of date-based and
SHA1-based fanout schemes. This is the interesting part, as far as this
iteration is concerned.
- Patch 14 adds selftests that verify correct parsing of notes trees with
various date-based fanouts.
Note that the series does not yet include code for _writing_ notes into a
suitably structured notes tree. That will be done in a later iteration.
I have some performance numbers that I will send in a separate email.
Have fun! :)
...Johan
Johan Herland (9):
Teach "-m <msg>" and "-F <file>" to "git notes edit"
fast-import: Add support for importing commit notes
t3302-notes-index-expensive: Speed up create_repo()
Add flags to get_commit_notes() to control the format of the note string
Teach notes code to free its internal data structures on request.
Teach the notes lookup code to parse notes trees with various fanout schemes
Selftests verifying semantics when loading notes trees with various fanouts
Allow flexible organization of notes trees, using both commit date and SHA1
Add test cases for date-based fanouts
Johannes Schindelin (5):
Introduce commit notes
Add a script to edit/inspect notes
Speed up git notes lookup
Add an expensive test for git-notes
Add '%N'-format for pretty-printing commit notes
.gitignore | 1 +
Documentation/config.txt | 13 +
Documentation/git-fast-import.txt | 45 +++-
Documentation/git-notes.txt | 60 ++++
Documentation/pretty-formats.txt | 1 +
Makefile | 3 +
cache.h | 4 +
command-list.txt | 1 +
commit.c | 1 +
config.c | 5 +
environment.c | 1 +
fast-import.c | 88 +++++-
git-notes.sh | 121 +++++++
notes.c | 673 +++++++++++++++++++++++++++++++++++++
notes.h | 12 +
pretty.c | 10 +
t/t3301-notes.sh | 150 ++++++++
t/t3302-notes-index-expensive.sh | 118 +++++++
t/t3303-notes-subtrees.sh | 201 +++++++++++
t/t9300-fast-import.sh | 166 +++++++++
20 files changed, 1664 insertions(+), 10 deletions(-)
create mode 100644 Documentation/git-notes.txt
create mode 100755 git-notes.sh
create mode 100644 notes.c
create mode 100644 notes.h
create mode 100755 t/t3301-notes.sh
create mode 100755 t/t3302-notes-index-expensive.sh
create mode 100755 t/t3303-notes-subtrees.sh
^ permalink raw reply
* What's cooking in git.git (Sep 2009, #02; Mon, 07)
From: Junio C Hamano @ 2009-09-08 0:56 UTC (permalink / raw)
To: git
What's cooking in git.git (Sep 2009, #02; Mon, 07)
--------------------------------------------------
Here are the topics that have been cooking. Commits prefixed with '-' are
only in 'pu' while commits prefixed with '+' are in 'next'. The ones
marked with '.' do not appear in any of the integration branches, but I am
still holding onto them.
In 1.7.0, we plan to correct handful of warts in the interfaces everybody
agrees that they were mistakes. The resulting system may not be strictly
backward compatible. Currently planeed changes are:
* refuse push to update the checked out branch in a non-bare repo by
default
Make "git push" into a repository to update the branch that is checked
out fail by default. You can countermand this default by setting a
configuration variable in the receiving repository.
http://thread.gmane.org/gmane.comp.version-control.git/107758/focus=108007
* refuse push to delete the current branch by default
Make "git push $there :$killed" to delete the branch that is pointed at
by its HEAD fail by default. You can countermand this default by
setting a configuration variable in the receiving repository.
http://thread.gmane.org/gmane.comp.version-control.git/108862/focus=108936
* git-send-email won't make deep threads by default
Many people said that by default when sending more than 2 patches the
threading git-send-email makes by default is hard to read, and they
prefer the default be one cover letter and each patch as a direct
follow-up to the cover letter. You can countermand this by setting a
configuration variable.
http://article.gmane.org/gmane.comp.version-control.git/109790
* git-status won't be "git-commit --dry-run" anymore
http://thread.gmane.org/gmane.comp.version-control.git/125989/focus=125993
* "git-diff -w --exit-code" will exit success if only differences it
found are whitespace changes that are stripped away from the output.
http://thread.gmane.org/gmane.comp.version-control.git/119731/focus=119751
During 1.6.5 cycle, 'next' will hold topics meant for 1.6.5 and 1.7.0.
I tagged and pushed out 1.6.5-rc0. There are still some topics cooking in
'next' I would rather not leave out, but I've been hoping to keep this
cycle short, and I'm also hoping to tag the real 1.6.5 before I vanish for
about a week and half later in the month. so in the worst case there may
have to be a 1.6.6 to include the leftover topics.
--------------------------------------------------
[Graduated to "master"]
* np/maint-1.6.3-deepen (2009-09-03) 2 commits
(merged to 'next' on 2009-09-06 at d597fba)
+ pack-objects: free preferred base memory after usage
+ make shallow repository deepening more network efficient
Further optimization and fixing of the object transfer around the "shallow
repository" codepaths.
* js/stash-dwim (2009-07-27) 1 commit.
(merged to 'next' on 2009-08-16 at 67896c4)
+ Make 'git stash -k' a short form for 'git stash save --keep-index'
(this branch is used by tr/reset-checkout-patch.)
* tr/reset-checkout-patch (2009-08-18) 10 commits.
(merged to 'next' on 2009-09-03 at d4f2ed7)
+ stash: simplify defaulting to "save" and reject unknown options
(merged to 'next' on 2009-08-27 at d314281)
+ Make test case number unique
(merged to 'next' on 2009-08-18 at e465bb3)
+ tests: disable interactive hunk selection tests if perl is not available
(merged to 'next' on 2009-08-16 at 67896c4)
+ DWIM 'git stash save -p' for 'git stash -p'
+ Implement 'git stash save --patch'
+ Implement 'git checkout --patch'
+ Implement 'git reset --patch'
+ builtin-add: refactor the meat of interactive_add()
+ Add a small patch-mode testing library
+ git-apply--interactive: Refactor patch mode code
(this branch uses js/stash-dwim.)
DWIMmery of the two series tightened for safety a few days ago. Otherwise
the same as what have been cooking in 'next' for a few weeks.
* jc/upload-pack-hook (2009-08-28) 2 commits
(merged to 'next' on 2009-08-31 at f9933a5)
+ upload-pack: feed "kind [clone|fetch]" to post-upload-pack hook
+ upload-pack: add a trigger for post-upload-pack hook
* jk/clone-b (2009-08-26) 1 commit
(merged to 'next' on 2009-08-30 at 10a68d1)
+ clone: add --branch option to select a different HEAD
* jc/mailinfo-scissors (2009-08-26) 5 commits
(merged to 'next' on 2009-08-30 at 5fc6248)
+ mailinfo.scissors: new configuration
+ am/mailinfo: Disable scissors processing by default
+ Documentation: describe the scissors mark support of "git am"
+ Teach mailinfo to ignore everything before -- >8 -- mark
+ builtin-mailinfo.c: fix confusing internal API to mailinfo()
I didn't pick up the patch to simplify the definition of scissors. I do
not have strong opinion on it either way, and nobody on the list seemed to
care too much either.
--------------------------------------------------
[New Topics]
* cb/maint-1.6.3-grep-relative-up (2009-09-05) 2 commits.
(merged to 'next' on 2009-09-07 at f9b5b48)
+ grep: accept relative paths outside current working directory
+ grep: fix exit status if external_grep() punts
1.6.5 material.
* jk/1.7.0-status (2009-09-05) 5 commits
- docs: note that status configuration affects only long format
(merged to 'next' on 2009-09-07 at 8a7c563)
+ commit: support alternate status formats
+ status: add --porcelain output format
+ status: refactor format option parsing
+ status: refactor short-mode printing to its own function
(this branch uses jc/1.7.0-status.)
Gives the --short output format to post 1.7.0 "git commit --dry-run" that
is similar to that of post 1.7.0 "git status".
It might be a good idea to make the --short format part of 1.6.6 without
waiting for 1.7.0; it would require some branch shuffling to bring the
short-status patch earlier than the one that makes "status" different from
"commit --dry-run", though.
* jk/unwanted-advices (2009-09-06) 2 commits
- status: make "how to stage" messages optional
- push: make non-fast-forward help message configurable
Post 1.3.0 "newbie friendliness drive" introduced a lot of "helpful"
advice messages that are sometimes found to be irritating once the user
got used to working with git. This series introduces a general mechanism
to allow us to make them less loud using the config mechanism. The config
option hierarchy in these two are "message.*" but it is planned to be
renamed to "advice.*".
* jn/gitweb-show-size (2009-09-07) 1 commit
- gitweb: Add 'show-sizes' feature to show blob sizes in tree view
* jt/pushinsteadof (2009-09-07) 2 commits
- Add url.<base>.pushInsteadOf: URL rewriting for push only
- Wrap rewrite globals in a struct in preparation for adding another set
Probably 1.6.5 material.
* lt/maint-traverse-trees-fix (2009-09-06) 1 commit.
- Prepare 'traverse_trees()' for D/F conflict lookahead
Beginning of the fix to a rather nasty longstanding issue of merging trees
with ("a" "a-b"), ("a/b" "a-b") and just ("a-b").
* ne/rev-cache (2009-09-07) 7 commits
. support for commit grafts, slight change to general mechanism
. support for path name caching in rev-cache
. full integration of rev-cache into git, completed test suite
. administrative functions for rev-cache, start of integration into git
. support for non-commit object caching in rev-cache
. basic revision cache system, no integration or features
. man page and technical discussion for rev-cache
Replaced but I do not think this is ready for 'pu' yet.
--------------------------------------------------
[Stalled]
* jh/notes (2009-08-27) 12 commits.
- Add '%N'-format for pretty-printing commit notes
- Add flags to get_commit_notes() to control the format of the note string
- notes.c: Implement simple memory pooling of leaf nodes
- Selftests verifying semantics when loading notes trees with various fanouts
- Teach the notes lookup code to parse notes trees with various fanout schemes
- t3302-notes-index-expensive: Speed up create_repo()
- fast-import: Add support for importing commit notes
- Teach "-m <msg>" and "-F <file>" to "git notes edit"
- Add an expensive test for git-notes
- Speed up git notes lookup
- Add a script to edit/inspect notes
- Introduce commit notes
I heard the cvs-helper series depends on this one. It seems that the
fan-out strategy is being rethought?
* je/send-email-no-subject (2009-08-05) 1 commit
(merged to 'next' on 2009-08-30 at b6455c2)
+ send-email: confirm on empty mail subjects
The existing tests to covers the positive case (i.e. as long as the user
says "yes" to the "do you really want to send this message that lacks
subject", the message is sent) of this feature, but the feature itself
needs its own test to verify the negative case (i.e. does it correctly
stop if the user says "no"?)
* db/vcs-helper (2009-09-03) 16 commits
- Allow helpers to report in "list" command that the ref is unchanged
- Add support for "import" helper command
- Add a config option for remotes to specify a foreign vcs
- Allow programs to not depend on remotes having urls
- Allow fetch to modify refs
- Use a function to determine whether a remote is valid
- Use a clearer style to issue commands to remote helpers
- Make the "traditionally-supported" URLs a special case
(merged to 'next' on 2009-08-07 at f3533ba)
+ Makefile: install hardlinks for git-remote-<scheme> supported by libcurl if possible
+ Makefile: do not link three copies of git-remote-* programs
+ Makefile: git-http-fetch does not need expat
(merged to 'next' on 2009-08-06 at 15da79d)
+ http-fetch: Fix Makefile dependancies
+ Add transport native helper executables to .gitignore
(merged to 'next' on 2009-08-05 at 33d491e)
+ git-http-fetch: not a builtin
+ Use an external program to implement fetching with curl
+ Add support for external programs for handling native fetches
(this branch is used by jh/cvs-helper.)
I'd really want to have this in 1.6.5 so that we can eject -lcurl from the
main "git" binary. The patches in 'pu' got some review comments, and I
thought Daniel's responses were sensible. Comments?
* jh/cvs-helper (2009-08-18) 8 commits
- More fixes to the git-remote-cvs installation procedure
- Fix the Makefile-generated path to the git_remote_cvs package in git-remote-cvs
- Add simple selftests of git-remote-cvs functionality
- git-remote-cvs: Remote helper program for CVS repositories
- 2/2: Add Python support library for CVS remote helper
- 1/2: Add Python support library for CVS remote helper
- Basic build infrastructure for Python scripts
- Allow helpers to request marks for fast-import
(this branch uses db/vcs-helper.)
Builds on db/vcs-helper. There is a re-roll planned.
* cc/sequencer-rebase-i (2009-08-28) 15 commits
- rebase -i: use "git sequencer--helper --cherry-pick"
- sequencer: add "--cherry-pick" option to "git sequencer--helper"
- sequencer: add "do_commit()" and related functions working on "next_commit"
- pick: libify "pick_help_msg()"
- revert: libify cherry-pick and revert functionnality
- rebase -i: use "git sequencer--helper --fast-forward"
- sequencer: let "git sequencer--helper" callers set "allow_dirty"
- sequencer: add "--fast-forward" option to "git sequencer--helper"
- sequencer: add "do_fast_forward()" to perform a fast forward
- rebase -i: use "git sequencer--helper --reset-hard"
- sequencer: add "--reset-hard" option to "git sequencer--helper"
- sequencer: add "reset_almost_hard()" and related functions
- rebase -i: use "git sequencer--helper --make-patch"
- sequencer: add "make_patch" function to save a patch
- sequencer: add "builtin-sequencer--helper.c"
Migrating "rebase -i" bit by bit to C.
Not a 1.6.5 material yet.
--------------------------------------------------
[Cooking]
* pk/fast-import-tars (2009-09-03) 1 commit
(merged to 'next' on 2009-09-07 at 8fbf027)
+ import-tars: Allow per-tar author and commit message.
* jc/maint-1.6.0-blank-at-eof (2009-09-05) 10 commits.
(merged to 'next' on 2009-09-07 at 165dc3c)
+ core.whitespace: split trailing-space into blank-at-{eol,eof}
+ diff --color: color blank-at-eof
+ diff --whitespace=warn/error: fix blank-at-eof check
+ diff --whitespace=warn/error: obey blank-at-eof
+ diff.c: the builtin_diff() deals with only two-file comparison
+ apply --whitespace: warn blank but not necessarily empty lines at EOF
+ apply --whitespace=warn/error: diagnose blank at EOF
+ apply.c: split check_whitespace() into two
+ apply --whitespace=fix: detect new blank lines at eof correctly
+ apply --whitespace=fix: fix handling of blank lines at the eof
Hopefully in 1.6.5
* pk/fast-import-dirs (2009-09-03) 1 commit
(merged to 'next' on 2009-09-07 at 836cba2)
+ Add script for importing bits-and-pieces to Git.
With an updated key-value quoting rules, which I haven't carefully looked
at. I have to wonder if there are standard libraries to do this sort of
thing without having to hand-roll these logic every time. But perhaps I
shouldn't be too picky on contrib/ material.
* jn/gitweb-blame (2009-09-01) 5 commits
- gitweb: Minify gitweb.js if JSMIN is defined
- gitweb: Create links leading to 'blame_incremental' using JavaScript
(merged to 'next' on 2009-09-07 at 3622199)
+ gitweb: Colorize 'blame_incremental' view during processing
+ gitweb: Incremental blame (using JavaScript)
+ gitweb: Add optional "time to generate page" info in footer
Ajax-y blame.
* sr/gfi-options (2009-09-06) 6 commits
(merged to 'next' on 2009-09-07 at 5f6b0ff)
+ fast-import: test the new option command
+ fast-import: add option command
+ fast-import: test the new feature command
+ fast-import: add feature command
+ fast-import: put marks reading in it's own function
+ fast-import: put option parsing code in separate functions
Perhaps 1.6.5 material but I wasn't sure.
* nd/sparse (2009-08-20) 19 commits
- sparse checkout: inhibit empty worktree
- Add tests for sparse checkout
- read-tree: add --no-sparse-checkout to disable sparse checkout support
- unpack-trees(): ignore worktree check outside checkout area
- unpack_trees(): apply $GIT_DIR/info/sparse-checkout to the final index
- unpack-trees(): "enable" sparse checkout and load $GIT_DIR/info/sparse-checkout
- unpack-trees.c: generalize verify_* functions
- unpack-trees(): add CE_WT_REMOVE to remove on worktree alone
- Introduce "sparse checkout"
- dir.c: export excluded_1() and add_excludes_from_file_1()
- excluded_1(): support exclude files in index
- unpack-trees(): carry skip-worktree bit over in merged_entry()
- Read .gitignore from index if it is skip-worktree
- Avoid writing to buffer in add_excludes_from_file_1()
- Teach Git to respect skip-worktree bit (writing part)
- Teach Git to respect skip-worktree bit (reading part)
- Introduce "skip-worktree" bit in index, teach Git to get/set this bit
- Add test-index-version
- update-index: refactor mark_valid() in preparation for new options
--------------------------------------------------
[For 1.7.0]
* jc/1.7.0-status (2009-09-05) 4 commits
(merged to 'next' on 2009-09-06 at 19d4beb)
+ status: typo fix in usage
(merged to 'next' on 2009-08-22 at b3507bb)
+ git status: not "commit --dry-run" anymore
+ git stat -s: short status output
+ git stat: the beginning of "status that is not a dry-run of commit"
(this branch is used by jk/1.7.0-status.)
With this, "git status" is no longer "git commit --dry-run".
* jc/1.7.0-send-email-no-thread-default (2009-08-22) 1 commit
(merged to 'next' on 2009-08-22 at 5106de8)
+ send-email: make --no-chain-reply-to the default
* jc/1.7.0-diff-whitespace-only-status (2009-08-30) 4 commits.
(merged to 'next' on 2009-08-30 at 0623572)
+ diff.c: fix typoes in comments
(merged to 'next' on 2009-08-27 at 81fb2bd)
+ Make test case number unique
(merged to 'next' on 2009-08-02 at 9c08420)
+ diff: Rename QUIET internal option to QUICK
+ diff: change semantics of "ignore whitespace" options
This changes exit code from "git diff --ignore-whitespace" and friends
when there is no actual output. It is a backward incompatible change, but
we could argue that it is a bugfix.
* jc/1.7.0-push-safety (2009-02-09) 2 commits
(merged to 'next' on 2009-08-02 at 38b82fe)
+ Refuse deleting the current branch via push
+ Refuse updating the current branch in a non-bare repository via push
--------------------------------------------------
[I have been too busy to purge these]
* jc/log-tz (2009-03-03) 1 commit.
- Allow --date=local --date=other-format to work as expected
Maybe some people care about this. I dunno.
* jc/mailinfo-remove-brackets (2009-07-15) 1 commit.
- mailinfo: -b option keeps [bracketed] strings that is not a [PATCH] marker
Maybe some people care about this. I dunno.
* lt/read-directory (2009-05-15) 3 commits.
. Add initial support for pathname conversion to UTF-8
. read_directory(): infrastructure for pathname character set conversion
. Add 'fill_directory()' helper function for directory traversal
^ permalink raw reply
* Re: [PATCH resend] git-pull: fix fetch-options.txt to not document --quiet and --verbose twice in git-pull.txt
From: Junio C Hamano @ 2009-09-07 21:48 UTC (permalink / raw)
To: Emmanuel Trillaud; +Cc: Git Mailing List
In-Reply-To: <9f50533b0909070534q2375a793mf5d676b519eae69@mail.gmail.com>
Emmanuel Trillaud <etrillaud@gmail.com> writes:
> I choose to "suppress" the fetch-options part because IMHO we don't need
> that level of precision.
I tried to see both versions myself and came to the same conclusion.
Thanks.
^ permalink raw reply
* Re: [PATCH 5/6 (v4)] full integration of rev-cache into git, completed test suite
From: Sam Vilain @ 2009-09-07 21:06 UTC (permalink / raw)
To: Nick Edelen
Cc: Junio C Hamano, Nicolas Pitre, Johannes Schindelin,
Michael J Gruber, Jeff King, Shawn O. Pearce, Andreas Ericsson,
Christian Couder, git@vger.kernel.org
In-Reply-To: <op.uzv4covmtdk399@sirnot.private>
On Mon, 2009-09-07 at 16:11 +0200, Nick Edelen wrote:
> This last patch
^^ You don't need to write comments like "This patch"; in the history
such words are meaningless.
> provides a working integration of rev-cache into the revision
> walker, along with some touch-ups:
> - integration into revision walker and list-objects
> - tweak of object generation
"tweak" ?
> - more fluid handling of damaged cache slices
What does this mean?
> - numerous tests for both features from the previous patch, and the
> integration's integrity
> 'Integration' is rather broad -- a more detailed description follows for each
> aspect:
> - rev-cache
> the traversal mechanism is updated to handle many of the non-prune options
> rev-list does (date limiting, slop-handling, etc.), and is adjusted to allow
> for non-fatal cache-traversal failures.
>
> - revision walker
> both limited and unlimited traversal attempt to use the cache when possible,
> smoothly falling back if it's not.
>
> - list-objects
> object listing does not recurse into cached trees, and has been adjusted to
> guarantee commit-tag-tree-blob ordering.
This is quite a long commit message. Is the above detail all useful?
Can it be split into one patch for each of the above integrations?
> Signed-off-by: Nick Edelen <sirnot@gmail.com>
Sam
^ permalink raw reply
* Re: [PATCH resend] git-pull: fix fetch-options.txt to not document --quiet and --verbose twice in git-pull.txt
From: Junio C Hamano @ 2009-09-07 20:08 UTC (permalink / raw)
To: Emmanuel Trillaud; +Cc: Git Mailing List
In-Reply-To: <9f50533b0909070534q2375a793mf5d676b519eae69@mail.gmail.com>
The patch seems to be severely whitespace damaged the same way as in the
previous round.
^ permalink raw reply
* Re: git survey stats
From: Jakub Narebski @ 2009-09-07 20:00 UTC (permalink / raw)
To: Pieter de Bie, git
In-Reply-To: <7da341b10909071155k63ffed19q52073277b3a3bb13@mail.gmail.com>
On Mon, 7 Sep 2009, Pieter de Bie wrote:
>
> I wondered if you could tell me what the total number of users, and
> the percentage of users use GitX, in the current survey?
You can always check it out for yourself in "live" survey summary at
http://www.survs.com/shareResults?survey=2PIMZGU0&rndm=678J66QRA2
http://tinyurl.com/GitSurvey2009Analyze
I have just updated http://git.or.cz/gitwiki/GitSurvey2009 to have
this link.
11) What Git interfaces, implementations, frontends and tools
do you use?
(Choice - Multiple answers)
============================================
Answer | [%] | [N]
--------------------------------------------
git (core) commandline | 97% | 3382
JGit (Java implementation) | 5% | 160
library / language binding | |
(e.g. Grit or Dulwich) | 4% | 133
............................................
Cogito (DEPRECATED) | 0% | 13
pg aka Patchy Git (DEPRECATED) | 0% | 2
Easy Git | 1% | 28
Pyrite | 0% | 6
StGIT | 2% | 59
Guilt | 0% | 13
TopGit | 1% | 38
............................................
gitk | 46% | 1593
git gui | 20% | 699
QGit | 6% | 222
GitView | 1% | 21
Giggle | 4% | 153
GitNub | 4% | 150
**GitX** | 19% | 665 <== here
git-cola | 2% | 56
tig | 6% | 198
TortoiseGit | 5% | 182
Git Extensions | 2% | 69
git-cheetah | 0% | 5
............................................
git-instaweb | 3% | 98
git-sh | 2% | 53
............................................
Gitosis (as admin) | 11% | 380
repo (multiple repositories) | 2% | 60
--------------------------------------------
editor/IDE VC integration | 11% | 393
filemanager integration | |
/ shell extension (any) | 3% | 93
graphical history viewer | |
/ browser (any) | 16% | 553
graphical commit tool (any) | 6% | 217
graphical diff tool | 16% | 564
graphical merge tool | 15% | 514
graphical blame or pickaxe tool| 2% | 82
my own scripts (for daily | |
commandline use, porcelain) | 9% | 309
my own scripts | |
(for special tasks) | 10% | 363
============================================
Other (please specify) | 8% | 293
############################################
Total respondents | 3473
Skipped this question | 195
============================================
Percentage is relative to responders that answered this question.
As you can see GitX is third after (bundled) gitk and git-gui.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: [PATCH] git-rebase-interactive: avoid breaking when GREP_OPTIONS="-H"
From: Junio C Hamano @ 2009-09-07 19:37 UTC (permalink / raw)
To: Carlo Marcelo Arenas Belon; +Cc: git
In-Reply-To: <1252328160-4359-1-git-send-email-carenas@sajinet.com.pe>
Carlo Marcelo Arenas Belon <carenas@sajinet.com.pe> writes:
> if GREP_OPTIONS is set and includes -H, using `grep -c` will fail
> to generate a numeric count and result in the following error :
>
> /usr/libexec/git-core/git-rebase--interactive: line 110: (standard
> input):1+(standard input):0: missing `)' (error token is
> "input):1+(standard input):0")
>
> instead of grep counting use `wc -l` to return the line count.
Thanks.
How does your patch help when the user has GREP_OPTIONS=-C3 in the
environment?
I think a saner workaround for this user environment bug (or GNU grep
misfeature) is to unset GREP_OPTIONS at the beginning of the script, or
even in git-sh-setup.
^ permalink raw reply
* Re: [PATCH] githooks.txt: put hooks into subsections
From: Junio C Hamano @ 2009-09-07 19:30 UTC (permalink / raw)
To: Bert Wesarg; +Cc: Junio C Hamano, Christian Couder, git
In-Reply-To: <6bd645c76041635d77e5ecc626e4386bd97715b4.1252168850.git.bert.wesarg@googlemail.com>
Bert Wesarg <bert.wesarg@googlemail.com> writes:
> All hooks are currently in its own section. Which may confuse users,
> because the section name serves as the hook file name and sections are
> all caps for man pages. Putting them into a new HOOKS section and each
> hook into a subsection keeps the case to lower case.
Sounds like a sane thing to do. Thanks.
^ permalink raw reply
* Re: [PATCH 2/2] git-completion.bash: prevent 'git help' from searching for git repository
From: Junio C Hamano @ 2009-09-07 19:15 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Gerrit Pape, Sverre Rabbelier, Junio C Hamano, git
In-Reply-To: <alpine.DEB.1.00.0909041452080.4605@intel-tinevez-2-302>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Thanks.
>
> Junio, I am serious about this patch (i.e. it is meant for inclusion now).
> Want me to resend?
Thanks.
^ permalink raw reply
* Re: Issue 323 in msysgit: Can't clone over http
From: Junio C Hamano @ 2009-09-07 19:06 UTC (permalink / raw)
To: Tay Ray Chuan; +Cc: git, msysgit, Tom Preston-Werner
In-Reply-To: <20090907172751.6cf38640.rctay89@gmail.com>
Tay Ray Chuan <rctay89@gmail.com> writes:
> Just to clarify: the "check" of CURLE_OK in http-walker.c::fetch_index()
> in v1.6.3 is fundamentally different from the check we have in 48188c2
> (http-walker: verify remote packs, 2009-06-06).
>
> The first "check" is a full-blooded GET request, and we do get back
> actual data (in this case, the pack index file). The second check isn't
> a GET request, just an inconsequential HEAD request; we don't get back
> any real data.
Yeah, I realized after I wrote the message you are responding to and ran
some experiments with github repository myself to see for some of their
URLs HEAD gives 500 when GET gives the contents just fine.
Tom, sorry to have given a confusing list of issues in my earlier message.
Please disregard it. The only funny your HTTP server folks may want to
look into is that GET request to fetch the following URL gives the
contents just fine, but HEAD request to it results in an Internal Server
Error.
http://github.com/tekkub/addontemplate.git/objects/pack/pack-382c25c935b744e909c749532578112d72a4aff9.pack
Sorry about the confusion.
> Agreed. I didn't intend my patch to loosen up error checking, merely to
> be clearer about what we're looking for. If read in another context
> (separate from fixing cloning over github.com), my patch can be seen as
> one that clarifies the verify-remote-pack check:
>
> Case 1: A 404 is received. The pack file is not available, so we stop.
>
> Case 2: Our check failed, due to some reason (request failed,
> unauthorized, etc). Nothing conclusive about availabilty of
> file. Continue anyway.
I am torn about this.
On one hand, if we are going to treat such a half failure as nothing
conclusive, I do not see a point to keep that check to begin with.
On the other hand, if a HEAD request to a URL results in an unauthorized,
what plausible excuse the server operator could give for allowing a GET
request to the same URL? If we are going to keep the check if *.pack that
corresponds to the *.idx will be available, shouldn't we trust whatever
check we perform?
^ permalink raw reply
* Re: [PATCH] grep: fix exit status if external_grep() returns error
From: Junio C Hamano @ 2009-09-07 18:13 UTC (permalink / raw)
To: Clemens Buchacher; +Cc: Jeff King, SZEDER Gábor, git
In-Reply-To: <20090907084801.GB10797@localhost>
Clemens Buchacher <drizzd@aon.at> writes:
> diff --git a/builtin-grep.c b/builtin-grep.c
> index ad0e0a5..b577738 100644
> --- a/builtin-grep.c
> +++ b/builtin-grep.c
> @@ -500,9 +500,9 @@ static int grep_cache(struct grep_opt *opt, const char **paths, int cached,
> * be a lot more optimized
> */
> if (!cached && external_grep_allowed) {
> - hit = external_grep(opt, paths, cached);
> - if (hit >= 0)
> - return hit;
> + int ret = external_grep(opt, paths, cached);
> + if (ret >= 0)
> + return ret;
Well caught, and this deserves to go to maint.
An alternative would be to reset hit to zero if we decide to use the
internal one after this conditional.
Thanks.
^ permalink raw reply
* Re: how to skip branches on git svn clone/fetch when there are errors
From: Daniele Segato @ 2009-09-07 17:55 UTC (permalink / raw)
To: Paolo Ciarrocchi; +Cc: Git Mailing List
In-Reply-To: <4d8e3fd30909070853u6f4bc179qac96edae5c83a864@mail.gmail.com>
Il giorno lun, 07/09/2009 alle 17.53 +0200, Paolo Ciarrocchi ha scritto:
> > I don't know if this is the best solution to solve the issue but it does
> > let me download the repo skipping the problematics paths
>
> Usually these lines are for the commit message, I guess you don't want
> to store these two lines in git.git :-)
no
that was my local commit
(and I initially copied the wrong one....)
If my patch will be accepted (and I really doubt this will be the case)
I'll fix that with a git commit amend / git rebase -i following the git
developers guide-lines :)
^ permalink raw reply
* Re: how to skip branches on git svn clone/fetch when there are errors
From: Paolo Ciarrocchi @ 2009-09-07 15:53 UTC (permalink / raw)
To: Daniele Segato; +Cc: Git Mailing List
In-Reply-To: <9accb4400909070634oee46b78g9270586a2b0eb4b9@mail.gmail.com>
On Mon, Sep 7, 2009 at 3:34 PM, Daniele Segato<daniele.bilug@gmail.com> wrote:
[...]
Ciao Daniele,
> From e8a1a12e83b3f0b18ce842190d8fc8eddaa77f68 Mon Sep 17 00:00:00 2001
> From: Daniele Segato <daniele.bilug@gmail.com>
> Date: Mon, 7 Sep 2009 15:30:14 +0200
> Subject: [PATCH] Ignore error 175007 authorization failed on checkpath
>
> I don't know if this is the best solution to solve the issue but it does
> let me download the repo skipping the problematics paths
Usually these lines are for the commit message, I guess you don't want
to store these two lines in git.git :-)
> ---
So if you want to add comments like you did, write them after the 3
minus and before the content of the patch.
> git-svn.perl | 16 ++++++++++++++++
> 1 files changed, 16 insertions(+), 0 deletions(-)
>
> diff --git a/git-svn.perl b/git-svn.perl
> index a366c89..0ab6453 100755
Ciao,
--
Paolo
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox