[PATCH 3/4] refs.c: add a transaction function to append a reflog entry

[Stefan Beller <sbeller@google.com>]

From: Ronnie Sahlberg <sahlberg@google.com>

Define a new transaction update type, UPDATE_LOG, and a new function
transaction_update_reflog. This function will lock the reflog and append
an entry to it during transaction commit. We can pass a flag to this
function, which can truncate the the reflog file before we write the
update.

When performing a reflog transaction update, only write to the reflog iff
msg is non-NULL. This can then be combined with REFLOG_TRUNCATE to perform
an update that only truncates but does not write. This change only affects
whether or not a reflog entry should be generated and written. If msg == NULL
then no such entry will be written.

Orthogonal to this we have a boolean flag REFLOG_TRUNCATE which is used to
tell the transaction system to "truncate the reflog and thus discard all
previous users".

At the current time the only place where we use msg == NULL is also the
place, where we use REFLOG_TRUNCATE. Even though these two settings are
currently only ever used together it still makes sense to have them through
two separate knobs.

This allows future consumers of this API that may want to do things
differently. For example someone can do:
  msg="Reflog truncated by Bob because ..." + REFLOG_TRUNCATE
and have it truncate the log and have it start fresh with an initial message
that explains the log was truncated. This API allows that.

During one transaction we allow to make multiple reflog updates to the
same ref. This means we only need to lock the reflog once, during the first
update that touches the reflog, and that all further updates can just write the
reflog entry since the reflog is already locked.

This allows us to write code (internally in refs.c) such as:

t = transaction_begin()
transaction_reflog_update(t, "foo", REFLOG_TRUNCATE, NULL);
loop-over-something...
   transaction_reflog_update(t, "foo", 0, <message>);
transaction_commit(t)

where we first truncate the reflog and then build the new content one line
at a time.

While this technically looks like O(n2) behavior it is not that bad.
We only do this loop for transactions that cover a single ref during
reflog expire. This means that the linear search inside
transaction_update_reflog() will find the match on the very first entry
thus making it O(1) and not O(n) or our usecases. Thus the whole expire
becomes O(n) instead of O(n2). If in the future we start doing this for many
refs in one single transaction we might want to optimize this.
But there is no need to complexify the code and optimize for future usecases
that might never materialize at this stage.

Instead of recording the line by line reflog updates in memory, use a
tempfile: .git/tmp_reflog_XXXXXX which we write the entries to as the
transaction is built. Then just rename this file onto the destination
reflog file when the transaction is committed.

todo:
This patch needs to add an atexit() thingy as well to ensure that
any remaining files are unlinked on exit, just like the lock_file() thing does.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---

This is a complete rewrite from previous series. Sorry no diff to previous versions.

 refs.c | 148 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 refs.h |  21 ++++++++++
 2 files changed, 167 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 84e086f..a1af703 100644
--- a/refs.c
+++ b/refs.c
@@ -3517,7 +3517,8 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 }
 
 enum transaction_update_type {
-	UPDATE_SHA1 = 0
+	UPDATE_SHA1 = 0,
+	UPDATE_LOG = 1
 };
 
 /**
@@ -3530,11 +3531,18 @@ struct ref_update {
 	enum transaction_update_type update_type;
 	unsigned char new_sha1[20];
 	unsigned char old_sha1[20];
-	int flags; /* REF_NODEREF? */
+	int flags;  /* The flags to transaction_update_ref[log] are defined
+		     * in refs.h
+		     */
 	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
 	struct ref_lock *lock;
 	int type;
 	char *msg;
+
+	/* used by reflog updates */
+	char *tmp_reflog;
+	int reflog_fd;
+
 	const char refname[FLEX_ARRAY];
 };
 
@@ -3580,6 +3588,7 @@ void transaction_free(struct transaction *transaction)
 		return;
 
 	for (i = 0; i < transaction->nr; i++) {
+		free(transaction->updates[i]->tmp_reflog);
 		free(transaction->updates[i]->msg);
 		free(transaction->updates[i]);
 	}
@@ -3601,6 +3610,116 @@ static struct ref_update *add_update(struct transaction *transaction,
 	return update;
 }
 
+int transaction_update_reflog(struct transaction *transaction,
+			      const char *refname,
+			      const unsigned char *new_sha1,
+			      const unsigned char *old_sha1,
+			      const char *email,
+			      unsigned long timestamp, int tz,
+			      const char *msg, int flags,
+			      struct strbuf *err)
+{
+	struct ref_update *update;
+	struct strbuf buf = STRBUF_INIT;
+	int i;
+
+	if (transaction->state != TRANSACTION_OPEN)
+		die("BUG: update_reflog called for transaction that is not open");
+
+	/* Check if there is another reflog update for this ref already. */
+	for (i = 0; transaction->nr > 0 && i < transaction->nr; i++) {
+		if (transaction->updates[i]->update_type != UPDATE_LOG)
+			continue;
+		if (!strcmp(transaction->updates[i]->refname,
+			    refname)) {
+			break;
+		}
+	}
+	/* When starting the transaction or when we did not find the ref,
+	 * we will need to create a new temporary file. */
+	if (transaction->nr == 0 || i == transaction->nr) {
+		int orig_fd;
+		update = add_update(transaction, refname, UPDATE_LOG);
+
+		orig_fd = open(git_path("logs/%s", refname), O_RDONLY);
+		if (orig_fd < 0) {
+			const char *str = "Cannot open reflog for '%s'. %s";
+
+			strbuf_addf(err, str, refname, strerror(errno));
+			transaction->state = TRANSACTION_CLOSED;
+			return 1;
+		}
+
+		update->tmp_reflog = xstrdup(git_path(".tmp_reflog_XXXXXX"));
+		update->reflog_fd = mkstemp(update->tmp_reflog);
+		if (update->reflog_fd == -1) {
+			const char *str = "Could not create temporary "
+			  "reflog for '%s'. %s";
+
+			close(orig_fd);
+			strbuf_addf(err, str, refname, strerror(errno));
+			transaction->state = TRANSACTION_CLOSED;
+			return 1;
+		}
+		if (adjust_shared_perm(update->tmp_reflog)) {
+			strbuf_addf(err, "Could not fix permission bits for "
+				    "reflog: %s. %s",
+				    update->tmp_reflog, strerror(errno));
+			close(orig_fd);
+			unlink_or_warn(update->tmp_reflog);
+			close(update->reflog_fd);
+			update->reflog_fd = -1;
+			transaction->state = TRANSACTION_CLOSED;
+			return 1;
+		}
+		if (copy_fd(orig_fd, update->reflog_fd)) {
+			strbuf_addf(err, "Could not copy reflog: %s. %s",
+				    refname, strerror(errno));
+			close(orig_fd);
+			unlink_or_warn(update->tmp_reflog);
+			close(update->reflog_fd);
+			update->reflog_fd = -1;
+			transaction->state = TRANSACTION_CLOSED;
+			return 1;
+		}
+		close(orig_fd);
+	} else {
+		update = transaction->updates[i];
+	}
+
+	if (flags & REFLOG_TRUNCATE) {
+		if (lseek(update->reflog_fd, 0, SEEK_SET) < 0 ||
+			ftruncate(update->reflog_fd, 0)) {
+			strbuf_addf(err, "Could not truncate reflog: %s. %s",
+				    refname, strerror(errno));
+			unlink_or_warn(update->tmp_reflog);
+			close(update->reflog_fd);
+			update->reflog_fd = -1;
+			transaction->state = TRANSACTION_CLOSED;
+			return 1;
+		}
+	}
+	if (email)
+		strbuf_addf(&buf, "%s %lu %+05d", email, timestamp, tz);
+
+	if (msg &&
+	    log_ref_write_fd(update->reflog_fd,
+			     old_sha1, new_sha1,
+			     buf.buf, msg)) {
+		strbuf_addf(err, "Could not write to reflog: %s. %s",
+			    refname, strerror(errno));
+		unlink_or_warn(update->tmp_reflog);
+		close(update->reflog_fd);
+		update->reflog_fd = -1;
+		transaction->state = TRANSACTION_CLOSED;
+		strbuf_release(&buf);
+		return 1;
+	}
+	strbuf_release(&buf);
+
+	return 0;
+}
+
 int transaction_update_ref(struct transaction *transaction,
 			   const char *refname,
 			   const unsigned char *new_sha1,
@@ -3815,6 +3934,31 @@ int transaction_commit(struct transaction *transaction,
 	}
 	for (i = 0; i < delnum; i++)
 		unlink_or_warn(git_path("logs/%s", delnames[i]));
+
+	/* Commit all reflog files */
+	for (i = 0; i < n; i++) {
+		struct ref_update *update = updates[i];
+
+		if (update->update_type != UPDATE_LOG)
+			continue;
+		if (update->reflog_fd == -1)
+			continue;
+		if (close(update->reflog_fd) == -1) {
+			error("Could not commit temporary reflog: %s. %s",
+			      update->refname, strerror(errno));
+			update->reflog_fd = -1;
+			continue;
+		}
+		update->reflog_fd = -1;
+		if (rename(update->tmp_reflog,
+			   git_path("logs/%s", update->refname))) {
+			error("Could not commit reflog: %s. %s",
+			      update->refname, strerror(errno));
+			update->reflog_fd = -1;
+			continue;
+		}
+	}
+
 	clear_loose_ref_cache(&ref_cache);
 
 cleanup:
diff --git a/refs.h b/refs.h
index 556adfd..9f70b89 100644
--- a/refs.h
+++ b/refs.h
@@ -328,6 +328,27 @@ int transaction_delete_ref(struct transaction *transaction,
 			   struct strbuf *err);
 
 /*
+ * Flags controlling transaction_update_reflog().
+ * REFLOG_TRUNCATE: Truncate the reflog.
+ *
+ * Flags >= 0x100 are reserved for internal use.
+ */
+#define REFLOG_TRUNCATE 0x01
+/*
+ * Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
+ * this update will first truncate the reflog before writing the entry.
+ * If msg is NULL no update will be written to the log.
+ */
+int transaction_update_reflog(struct transaction *transaction,
+			      const char *refname,
+			      const unsigned char *new_sha1,
+			      const unsigned char *old_sha1,
+			      const char *email,
+			      unsigned long timestamp, int tz,
+			      const char *msg, int flags,
+			      struct strbuf *err);
+
+/*
  * Commit all of the changes that have been queued in transaction, as
  * atomically as possible.
  *
-- 
2.2.0.rc3