* [RFH] zlib gurus out there?
From: Junio C Hamano @ 2006-03-07 23:48 UTC (permalink / raw)
To: git; +Cc: Nicolas Pitre, Linus Torvalds
In-Reply-To: <Pine.LNX.4.64.0602250012230.31162@localhost.localdomain>
I've been staring at reusing existing data while packing, and
this occurred to me...
During packing, suppose that we chose to store an object in
base form, undeltified. And also suppose we have that object
loose in .git/objects/??/ directory. We already have it in
deflated form, but with its own header. I started wondering if
we can somehow reuse this.
A short object format brush-up lesson is in order here.
* An undeltified object in a pack is represented like this:
(1) the header is a dense variable size binary data, that
encodes type and inflated length;
(2) deflated data immediately follows the header.
* On the other hand, a loose object is represented like this:
(1) the header looks like sprintf("%s %lu%c", type, len, 0);
(2) concatenate the data to the header;
(3) SHA1 checksum of the above becomes the object name.
(4) deflate the header and data using the same z_stream, in two
steps, like this (sha1_file.c::write_sha1_file):
/* Compress it */
stream.next_out = compressed;
stream.avail_out = size;
/* First header.. */
stream.next_in = hdr;
stream.avail_in = hdrlen;
while (deflate(&stream, 0) == Z_OK)
/* nothing */;
/* Then the data itself.. */
stream.next_in = buf;
stream.avail_in = len;
while (deflate(&stream, Z_FINISH) == Z_OK)
/* nothing */;
deflateEnd(&stream);
size = stream.total_out;
So I thought... if we cause a full flush after the header part,
I can find the flush boundaries from a loose object file and
copy the rest into a packfile I am generating, after placing the
binary encoded header. If this works, we do not have to inflate
loose object to read it and deflate it to store that in the
pack. We will get a better packing as well, since we deflate
loose objects with Z_BEST_COMPRESSION, while packs are done with
Z_DEFAULT_COMPRESSION. While pack-objects read from a loose
object, if we can detect that there is no full flush after the
header, we would do the traditional inflate-deflate cycle, so
this would be backward compatible.
However, I am stuck with the first step, which is to do a full
flush after the header. An obvious change to the code quoted
above writes out a corrupt object:
/* First header.. */
stream.next_in = hdr;
stream.avail_in = hdrlen;
- while (deflate(&stream, 0) == Z_OK)
+ while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
/* nothing */;
git-fsck-objects complains that sha1 does not match. It appears
that the sha1_file.c::unpack_sha1_rest() somehow barfs upon
seeing the full flush, but I haven't dug into it yet.
Would anybody with more experience with zlib want to help?
^ permalink raw reply
* [PATCH] Update http-push functionality
From: Nick Hengeveld @ 2006-03-08 0:13 UTC (permalink / raw)
To: git
This brings http-push functionality more in line with the ssh/git version,
by borrowing bits from send-pack and rev-list to process refspecs and
revision history in more standard ways. Also, the status of remote objects
is determined using PROPFIND requests for the object directory rather than
HEAD requests for each object - while it may be less efficient for small
numbers of objects, this approach is able to get the status of all remote
loose objects in a maximum of 256 requests.
---
I've tested this by running "git push --all --tags remote" into a newly
initialized repo, and by rewinding one or multiple overlapping heads and
pushing combinations of them.
This also lays a bit of groundwork for updating remote info/refs.
Getting my head around the underlying object/refspec/revision stuff made
it hurt, and I've got to say that you guys rule.
Makefile | 2
http-push.c | 1060 ++++++++++++++++++++++++++++++++++++++++-------------------
2 files changed, 727 insertions(+), 335 deletions(-)
9d2903e7bf846eeefd3f108207d7c212139ff97c
diff --git a/Makefile b/Makefile
index b6d8804..1c555da 100644
--- a/Makefile
+++ b/Makefile
@@ -526,7 +526,7 @@ git-http-fetch$X: fetch.o http.o http-fe
$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL)
-git-http-push$X: http.o http-push.o $(LIB_FILE)
+git-http-push$X: revision.o http.o http-push.o $(LIB_FILE)
$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL) $(EXPAT_LIBEXPAT)
diff --git a/http-push.c b/http-push.c
index fe92560..970cbfe 100644
--- a/http-push.c
+++ b/http-push.c
@@ -5,6 +5,8 @@
#include "tag.h"
#include "blob.h"
#include "http.h"
+#include "refs.h"
+#include "revision.h"
#include <expat.h>
@@ -42,14 +44,24 @@ enum XML_Status {
#define DAV_ACTIVELOCK_OWNER ".prop.lockdiscovery.activelock.owner.href"
#define DAV_ACTIVELOCK_TIMEOUT ".prop.lockdiscovery.activelock.timeout"
#define DAV_ACTIVELOCK_TOKEN ".prop.lockdiscovery.activelock.locktoken.href"
+#define DAV_PROPFIND_RESP ".multistatus.response"
+#define DAV_PROPFIND_NAME ".multistatus.response.href"
+#define DAV_PROPFIND_COLLECTION ".multistatus.response.propstat.prop.resourcetype.collection"
/* DAV request body templates */
-#define PROPFIND_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:prop xmlns:R=\"%s\">\n<D:supportedlock/>\n</D:prop>\n</D:propfind>"
+#define PROPFIND_SUPPORTEDLOCK_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:prop xmlns:R=\"%s\">\n<D:supportedlock/>\n</D:prop>\n</D:propfind>"
+#define PROPFIND_ALL_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:allprop/>\n</D:propfind>"
#define LOCK_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:lockinfo xmlns:D=\"DAV:\">\n<D:lockscope><D:exclusive/></D:lockscope>\n<D:locktype><D:write/></D:locktype>\n<D:owner>\n<D:href>mailto:%s</D:href>\n</D:owner>\n</D:lockinfo>"
#define LOCK_TIME 600
#define LOCK_REFRESH 30
+/* bits #0-4 in revision.h */
+
+#define LOCAL (1u << 5)
+#define REMOTE (1u << 6)
+#define PUSHING (1u << 7)
+
static int pushing = 0;
static int aborted = 0;
static char remote_dir_exists[256];
@@ -61,17 +73,19 @@ static int push_verbosely = 0;
static int push_all = 0;
static int force_all = 0;
+static struct object_list *objects = NULL;
+
struct repo
{
char *url;
+ int path_len;
struct packed_git *packs;
};
static struct repo *remote = NULL;
+static struct remote_lock *remote_locks = NULL;
enum transfer_state {
- NEED_CHECK,
- RUN_HEAD,
NEED_PUSH,
RUN_MKCOL,
RUN_PUT,
@@ -82,10 +96,10 @@ enum transfer_state {
struct transfer_request
{
- unsigned char sha1[20];
+ struct object *obj;
char *url;
char *dest;
- struct active_lock *lock;
+ struct remote_lock *lock;
struct curl_slist *headers;
struct buffer buffer;
char filename[PATH_MAX];
@@ -114,14 +128,23 @@ struct xml_ctx
void *userData;
};
-struct active_lock
+struct remote_lock
{
char *url;
char *owner;
char *token;
time_t start_time;
long timeout;
+ int active;
int refreshing;
+ struct remote_lock *next;
+};
+
+struct remote_dentry
+{
+ char *base;
+ char *name;
+ int is_dir;
};
static void finish_request(struct transfer_request *request);
@@ -134,42 +157,9 @@ static void process_response(void *callb
finish_request(request);
}
-static void start_check(struct transfer_request *request)
-{
- char *hex = sha1_to_hex(request->sha1);
- struct active_request_slot *slot;
- char *posn;
-
- request->url = xmalloc(strlen(remote->url) + 55);
- strcpy(request->url, remote->url);
- posn = request->url + strlen(remote->url);
- strcpy(posn, "objects/");
- posn += 8;
- memcpy(posn, hex, 2);
- posn += 2;
- *(posn++) = '/';
- strcpy(posn, hex + 2);
-
- slot = get_active_slot();
- slot->callback_func = process_response;
- slot->callback_data = request;
- curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, request->errorstr);
- curl_easy_setopt(slot->curl, CURLOPT_URL, request->url);
- curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 1);
-
- if (start_active_slot(slot)) {
- request->slot = slot;
- request->state = RUN_HEAD;
- } else {
- request->state = ABORTED;
- free(request->url);
- request->url = NULL;
- }
-}
-
static void start_mkcol(struct transfer_request *request)
{
- char *hex = sha1_to_hex(request->sha1);
+ char *hex = sha1_to_hex(request->obj->sha1);
struct active_request_slot *slot;
char *posn;
@@ -203,7 +193,7 @@ static void start_mkcol(struct transfer_
static void start_put(struct transfer_request *request)
{
- char *hex = sha1_to_hex(request->sha1);
+ char *hex = sha1_to_hex(request->obj->sha1);
struct active_request_slot *slot;
char *posn;
char type[20];
@@ -214,7 +204,7 @@ static void start_put(struct transfer_re
ssize_t size;
z_stream stream;
- unpacked = read_sha1_file(request->sha1, type, &len);
+ unpacked = read_sha1_file(request->obj->sha1, type, &len);
hdrlen = sprintf(hdr, "%s %lu", type, len) + 1;
/* Set it up */
@@ -309,64 +299,90 @@ static void start_move(struct transfer_r
}
}
-static int refresh_lock(struct active_lock *lock)
+static int refresh_lock(struct remote_lock *check_lock)
{
struct active_request_slot *slot;
char *if_header;
char timeout_header[25];
struct curl_slist *dav_headers = NULL;
- int rc = 0;
+ struct remote_lock *lock;
+ int time_remaining;
+ time_t current_time;
- lock->refreshing = 1;
+ /* Refresh all active locks if they're close to expiring */
+ for (lock = remote_locks; lock; lock = lock->next) {
+ if (!lock->active)
+ continue;
- if_header = xmalloc(strlen(lock->token) + 25);
- sprintf(if_header, "If: (<opaquelocktoken:%s>)", lock->token);
- sprintf(timeout_header, "Timeout: Second-%ld", lock->timeout);
- dav_headers = curl_slist_append(dav_headers, if_header);
- dav_headers = curl_slist_append(dav_headers, timeout_header);
+ current_time = time(NULL);
+ time_remaining = lock->start_time + lock->timeout
+ - current_time;
+ if (time_remaining > LOCK_REFRESH)
+ continue;
- slot = get_active_slot();
- curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
- curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null);
- curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url);
- curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
- curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+ lock->refreshing = 1;
- if (start_active_slot(slot)) {
- run_active_slot(slot);
- if (slot->curl_result != CURLE_OK) {
- fprintf(stderr, "Got HTTP error %ld\n", slot->http_code);
- } else {
- lock->start_time = time(NULL);
- rc = 1;
+ if_header = xmalloc(strlen(lock->token) + 25);
+ sprintf(if_header, "If: (<opaquelocktoken:%s>)", lock->token);
+ sprintf(timeout_header, "Timeout: Second-%ld", lock->timeout);
+ dav_headers = curl_slist_append(dav_headers, if_header);
+ dav_headers = curl_slist_append(dav_headers, timeout_header);
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result != CURLE_OK) {
+ fprintf(stderr, "Got HTTP error %ld\n", slot->http_code);
+ lock->active = 0;
+ } else {
+ lock->active = 1;
+ lock->start_time = time(NULL);
+ }
}
+
+ lock->refreshing = 0;
+ curl_slist_free_all(dav_headers);
+ free(if_header);
}
- lock->refreshing = 0;
- curl_slist_free_all(dav_headers);
- free(if_header);
+ if (check_lock)
+ return check_lock->active;
+ else
+ return 0;
+}
- return rc;
+static void release_request(struct transfer_request *request)
+{
+ struct transfer_request *entry = request_queue_head;
+
+ if (request == request_queue_head) {
+ request_queue_head = request->next;
+ } else {
+ while (entry->next != NULL && entry->next != request)
+ entry = entry->next;
+ if (entry->next == request)
+ entry->next = entry->next->next;
+ }
+
+ if (request->url != NULL)
+ free(request->url);
+ free(request);
}
static void finish_request(struct transfer_request *request)
{
- time_t current_time = time(NULL);
- int time_remaining;
-
request->curl_result = request->slot->curl_result;
request->http_code = request->slot->http_code;
request->slot = NULL;
- /* Refresh the lock if it is close to timing out */
- time_remaining = request->lock->start_time + request->lock->timeout
- - current_time;
- if (time_remaining < LOCK_REFRESH && !request->lock->refreshing) {
- if (!refresh_lock(request->lock)) {
- fprintf(stderr, "Unable to refresh remote lock\n");
- aborted = 1;
- }
- }
+ /* Keep locks active */
+ refresh_lock(request->lock);
if (request->headers != NULL)
curl_slist_free_all(request->headers);
@@ -377,27 +393,14 @@ static void finish_request(struct transf
request->url = NULL;
}
- if (request->state == RUN_HEAD) {
- if (request->http_code == 404) {
- request->state = NEED_PUSH;
- } else if (request->curl_result == CURLE_OK) {
- remote_dir_exists[request->sha1[0]] = 1;
- request->state = COMPLETE;
- } else {
- fprintf(stderr, "HEAD %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
- request->curl_result, request->http_code);
- request->state = ABORTED;
- aborted = 1;
- }
- } else if (request->state == RUN_MKCOL) {
+ if (request->state == RUN_MKCOL) {
if (request->curl_result == CURLE_OK ||
request->http_code == 405) {
- remote_dir_exists[request->sha1[0]] = 1;
+ remote_dir_exists[request->obj->sha1[0]] = 1;
start_put(request);
} else {
fprintf(stderr, "MKCOL %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
@@ -407,21 +410,21 @@ static void finish_request(struct transf
start_move(request);
} else {
fprintf(stderr, "PUT %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
}
} else if (request->state == RUN_MOVE) {
if (request->curl_result == CURLE_OK) {
- if (push_verbosely)
- fprintf(stderr,
- "sent %s\n",
- sha1_to_hex(request->sha1));
+ fprintf(stderr, " sent %s\n",
+ sha1_to_hex(request->obj->sha1));
request->state = COMPLETE;
+ request->obj->flags |= REMOTE;
+ release_request(request);
} else {
fprintf(stderr, "MOVE %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
@@ -429,24 +432,6 @@ static void finish_request(struct transf
}
}
-static void release_request(struct transfer_request *request)
-{
- struct transfer_request *entry = request_queue_head;
-
- if (request == request_queue_head) {
- request_queue_head = request->next;
- } else {
- while (entry->next != NULL && entry->next != request)
- entry = entry->next;
- if (entry->next == request)
- entry->next = entry->next->next;
- }
-
- if (request->url != NULL)
- free(request->url);
- free(request);
-}
-
void fill_active_slots(void)
{
struct transfer_request *request = request_queue_head;
@@ -457,14 +442,12 @@ void fill_active_slots(void)
return;
while (active_requests < max_requests && request != NULL) {
- if (!pushing && request->state == NEED_CHECK) {
- start_check(request);
- curl_multi_perform(curlm, &num_transfers);
- } else if (pushing && request->state == NEED_PUSH) {
- if (remote_dir_exists[request->sha1[0]])
+ if (pushing && request->state == NEED_PUSH) {
+ if (remote_dir_exists[request->obj->sha1[0]] == 1) {
start_put(request);
- else
+ } else {
start_mkcol(request);
+ }
curl_multi_perform(curlm, &num_transfers);
}
request = request->next;
@@ -479,26 +462,34 @@ void fill_active_slots(void)
}
}
-static void add_request(unsigned char *sha1, struct active_lock *lock)
+static void get_remote_object_list(unsigned char parent);
+
+static void add_request(struct object *obj, struct remote_lock *lock)
{
struct transfer_request *request = request_queue_head;
struct packed_git *target;
-
- while (request != NULL && memcmp(request->sha1, sha1, 20))
- request = request->next;
- if (request != NULL)
- return;
- target = find_sha1_pack(sha1, remote->packs);
- if (target)
+ /*
+ * Don't push the object if it's known to exist on the remote
+ * or is already in the request queue
+ */
+ if (remote_dir_exists[obj->sha1[0]] == -1)
+ get_remote_object_list(obj->sha1[0]);
+ if (obj->flags & (REMOTE | PUSHING))
return;
+ target = find_sha1_pack(obj->sha1, remote->packs);
+ if (target) {
+ obj->flags |= REMOTE;
+ return;
+ }
+ obj->flags |= PUSHING;
request = xmalloc(sizeof(*request));
- memcpy(request->sha1, sha1, 20);
+ request->obj = obj;
request->url = NULL;
request->lock = lock;
request->headers = NULL;
- request->state = NEED_CHECK;
+ request->state = NEED_PUSH;
request->next = request_queue_head;
request_queue_head = request;
@@ -698,14 +689,13 @@ static char *quote_ref_url(const char *b
int len, baselen, ch;
baselen = strlen(base);
- len = baselen + 12; /* "refs/heads/" + NUL */
+ len = baselen + 1;
for (cp = ref; (ch = *cp) != 0; cp++, len++)
if (needs_quote(ch))
len += 2; /* extra two hex plus replacement % */
qref = xmalloc(len);
memcpy(qref, base, baselen);
- memcpy(qref + baselen, "refs/heads/", 11);
- for (cp = ref, dp = qref + baselen + 11; (ch = *cp) != 0; cp++) {
+ for (cp = ref, dp = qref + baselen; (ch = *cp) != 0; cp++) {
if (needs_quote(ch)) {
*dp++ = '%';
*dp++ = hex((ch >> 4) & 0xF);
@@ -751,6 +741,27 @@ int fetch_ref(char *ref, unsigned char *
return 0;
}
+static void one_remote_object(const char *hex)
+{
+ unsigned char sha1[20];
+ struct object *obj;
+
+ if (get_sha1_hex(hex, sha1) != 0)
+ return;
+
+ obj = lookup_object(sha1);
+ if (!obj)
+ obj = parse_object(sha1);
+
+ /* Ignore remote objects that don't exist locally */
+ if (!obj)
+ return;
+
+ obj->flags |= REMOTE;
+ if (!object_list_contains(objects, obj))
+ add_object(obj, &objects, NULL, "");
+}
+
static void handle_lockprop_ctx(struct xml_ctx *ctx, int tag_closed)
{
int *lock_flags = (int *)ctx->userData;
@@ -772,7 +783,7 @@ static void handle_lockprop_ctx(struct x
static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed)
{
- struct active_lock *lock = (struct active_lock *)ctx->userData;
+ struct remote_lock *lock = (struct remote_lock *)ctx->userData;
if (tag_closed && ctx->cdata) {
if (!strcmp(ctx->name, DAV_ACTIVELOCK_OWNER)) {
@@ -791,6 +802,57 @@ static void handle_new_lock_ctx(struct x
}
}
+static void one_remote_ref(char *refname);
+static void crawl_remote_refs(char *path);
+
+static void handle_crawl_ref_ctx(struct xml_ctx *ctx, int tag_closed)
+{
+ struct remote_dentry *dentry = (struct remote_dentry *)ctx->userData;
+
+
+ if (tag_closed) {
+ if (!strcmp(ctx->name, DAV_PROPFIND_RESP) && dentry->name) {
+ if (dentry->is_dir) {
+ if (strcmp(dentry->name, dentry->base)) {
+ crawl_remote_refs(dentry->name);
+ }
+ } else {
+ one_remote_ref(dentry->name);
+ }
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_NAME) && ctx->cdata) {
+ dentry->name = xmalloc(strlen(ctx->cdata) -
+ remote->path_len + 1);
+ strcpy(dentry->name,
+ ctx->cdata + remote->path_len);
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_COLLECTION)) {
+ dentry->is_dir = 1;
+ }
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
+ dentry->name = NULL;
+ dentry->is_dir = 0;
+ }
+}
+
+static void handle_remote_object_list_ctx(struct xml_ctx *ctx, int tag_closed)
+{
+ char *path;
+ char *obj_hex;
+
+ if (tag_closed) {
+ if (!strcmp(ctx->name, DAV_PROPFIND_NAME) && ctx->cdata) {
+ path = ctx->cdata + remote->path_len;
+ if (strlen(path) != 50)
+ return;
+ path += 9;
+ obj_hex = xmalloc(strlen(path));
+ strncpy(obj_hex, path, 2);
+ strcpy(obj_hex + 2, path + 3);
+ one_remote_object(obj_hex);
+ free(obj_hex);
+ }
+ }
+}
+
static void
xml_start_tag(void *userData, const char *name, const char **atts)
{
@@ -848,7 +910,7 @@ xml_cdata(void *userData, const XML_Char
strncpy(ctx->cdata, s, len);
}
-static struct active_lock *lock_remote(char *file, long timeout)
+static struct remote_lock *lock_remote(char *path, long timeout)
{
struct active_request_slot *slot;
struct buffer out_buffer;
@@ -858,14 +920,26 @@ static struct active_lock *lock_remote(c
char *url;
char *ep;
char timeout_header[25];
- struct active_lock *new_lock = NULL;
+ struct remote_lock *lock = remote_locks;
XML_Parser parser = XML_ParserCreate(NULL);
enum XML_Status result;
struct curl_slist *dav_headers = NULL;
struct xml_ctx ctx;
- url = xmalloc(strlen(remote->url) + strlen(file) + 1);
- sprintf(url, "%s%s", remote->url, file);
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ /* Make sure the url is not already locked */
+ while (lock && strcmp(lock->url, url)) {
+ lock = lock->next;
+ }
+ if (lock) {
+ free(url);
+ if (refresh_lock(lock))
+ return lock;
+ else
+ return NULL;
+ }
/* Make sure leading directories exist for the remote ref */
ep = strchr(url + strlen(remote->url) + 11, '/');
@@ -921,11 +995,11 @@ static struct active_lock *lock_remote(c
curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
- new_lock = xcalloc(1, sizeof(*new_lock));
- new_lock->owner = NULL;
- new_lock->token = NULL;
- new_lock->timeout = -1;
- new_lock->refreshing = 0;
+ lock = xcalloc(1, sizeof(*lock));
+ lock->owner = NULL;
+ lock->token = NULL;
+ lock->timeout = -1;
+ lock->refreshing = 0;
if (start_active_slot(slot)) {
run_active_slot(slot);
@@ -934,7 +1008,7 @@ static struct active_lock *lock_remote(c
ctx.len = 0;
ctx.cdata = NULL;
ctx.userFunc = handle_new_lock_ctx;
- ctx.userData = new_lock;
+ ctx.userData = lock;
XML_SetUserData(parser, &ctx);
XML_SetElementHandler(parser, xml_start_tag,
xml_end_tag);
@@ -946,7 +1020,7 @@ static struct active_lock *lock_remote(c
fprintf(stderr, "XML error: %s\n",
XML_ErrorString(
XML_GetErrorCode(parser)));
- new_lock->timeout = -1;
+ lock->timeout = -1;
}
}
} else {
@@ -957,23 +1031,26 @@ static struct active_lock *lock_remote(c
free(out_data);
free(in_data);
- if (new_lock->token == NULL || new_lock->timeout <= 0) {
- if (new_lock->token != NULL)
- free(new_lock->token);
- if (new_lock->owner != NULL)
- free(new_lock->owner);
+ if (lock->token == NULL || lock->timeout <= 0) {
+ if (lock->token != NULL)
+ free(lock->token);
+ if (lock->owner != NULL)
+ free(lock->owner);
free(url);
- free(new_lock);
- new_lock = NULL;
+ free(lock);
+ lock = NULL;
} else {
- new_lock->url = url;
- new_lock->start_time = time(NULL);
+ lock->url = url;
+ lock->active = 1;
+ lock->start_time = time(NULL);
+ lock->next = remote_locks;
+ remote_locks = lock;
}
- return new_lock;
+ return lock;
}
-static int unlock_remote(struct active_lock *lock)
+static int unlock_remote(struct remote_lock *lock)
{
struct active_request_slot *slot;
char *lock_token_header;
@@ -1005,15 +1082,171 @@ static int unlock_remote(struct active_l
curl_slist_free_all(dav_headers);
free(lock_token_header);
- if (lock->owner != NULL)
- free(lock->owner);
- free(lock->url);
- free(lock->token);
- free(lock);
+ lock->active = 0;
return rc;
}
+static void crawl_remote_refs(char *path)
+{
+ char *url;
+ struct active_request_slot *slot;
+ struct buffer in_buffer;
+ struct buffer out_buffer;
+ char *in_data;
+ char *out_data;
+ XML_Parser parser = XML_ParserCreate(NULL);
+ enum XML_Status result;
+ struct curl_slist *dav_headers = NULL;
+ struct xml_ctx ctx;
+ struct remote_dentry dentry;
+
+ fprintf(stderr, " %s\n", path);
+
+ dentry.base = path;
+ dentry.name = NULL;
+ dentry.is_dir = 0;
+
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ out_buffer.size = strlen(PROPFIND_ALL_REQUEST);
+ out_data = xmalloc(out_buffer.size + 1);
+ snprintf(out_data, out_buffer.size + 1, PROPFIND_ALL_REQUEST);
+ out_buffer.posn = 0;
+ out_buffer.buffer = out_data;
+
+ in_buffer.size = 4096;
+ in_data = xmalloc(in_buffer.size);
+ in_buffer.posn = 0;
+ in_buffer.buffer = in_data;
+
+ dav_headers = curl_slist_append(dav_headers, "Depth: 1");
+ dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml");
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.size);
+ curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+ curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result == CURLE_OK) {
+ ctx.name = xcalloc(10, 1);
+ ctx.len = 0;
+ ctx.cdata = NULL;
+ ctx.userFunc = handle_crawl_ref_ctx;
+ ctx.userData = &dentry;
+ XML_SetUserData(parser, &ctx);
+ XML_SetElementHandler(parser, xml_start_tag,
+ xml_end_tag);
+ XML_SetCharacterDataHandler(parser, xml_cdata);
+ result = XML_Parse(parser, in_buffer.buffer,
+ in_buffer.posn, 1);
+ free(ctx.name);
+
+ if (result != XML_STATUS_OK) {
+ fprintf(stderr, "XML error: %s\n",
+ XML_ErrorString(
+ XML_GetErrorCode(parser)));
+ }
+ }
+ } else {
+ fprintf(stderr, "Unable to start request\n");
+ }
+
+ free(url);
+ free(out_data);
+ free(in_buffer.buffer);
+ curl_slist_free_all(dav_headers);
+}
+
+static void get_remote_object_list(unsigned char parent)
+{
+ char *url;
+ struct active_request_slot *slot;
+ struct buffer in_buffer;
+ struct buffer out_buffer;
+ char *in_data;
+ char *out_data;
+ XML_Parser parser = XML_ParserCreate(NULL);
+ enum XML_Status result;
+ struct curl_slist *dav_headers = NULL;
+ struct xml_ctx ctx;
+ char path[] = "/objects/XX/";
+ static const char hex[] = "0123456789abcdef";
+ unsigned int val = parent;
+
+ path[9] = hex[val >> 4];
+ path[10] = hex[val & 0xf];
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ out_buffer.size = strlen(PROPFIND_ALL_REQUEST);
+ out_data = xmalloc(out_buffer.size + 1);
+ snprintf(out_data, out_buffer.size + 1, PROPFIND_ALL_REQUEST);
+ out_buffer.posn = 0;
+ out_buffer.buffer = out_data;
+
+ in_buffer.size = 4096;
+ in_data = xmalloc(in_buffer.size);
+ in_buffer.posn = 0;
+ in_buffer.buffer = in_data;
+
+ dav_headers = curl_slist_append(dav_headers, "Depth: 1");
+ dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml");
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.size);
+ curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+ curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result == CURLE_OK) {
+ remote_dir_exists[parent] = 1;
+ ctx.name = xcalloc(10, 1);
+ ctx.len = 0;
+ ctx.cdata = NULL;
+ ctx.userFunc = handle_remote_object_list_ctx;
+ XML_SetUserData(parser, &ctx);
+ XML_SetElementHandler(parser, xml_start_tag,
+ xml_end_tag);
+ XML_SetCharacterDataHandler(parser, xml_cdata);
+ result = XML_Parse(parser, in_buffer.buffer,
+ in_buffer.posn, 1);
+ free(ctx.name);
+
+ if (result != XML_STATUS_OK) {
+ fprintf(stderr, "XML error: %s\n",
+ XML_ErrorString(
+ XML_GetErrorCode(parser)));
+ }
+ } else {
+ remote_dir_exists[parent] = 0;
+ }
+ } else {
+ fprintf(stderr, "Unable to start request\n");
+ }
+
+ free(url);
+ free(out_data);
+ free(in_buffer.buffer);
+ curl_slist_free_all(dav_headers);
+}
+
static int locking_available(void)
{
struct active_request_slot *slot;
@@ -1027,9 +1260,12 @@ static int locking_available(void)
struct xml_ctx ctx;
int lock_flags = 0;
- out_buffer.size = strlen(PROPFIND_REQUEST) + strlen(remote->url) - 2;
+ out_buffer.size =
+ strlen(PROPFIND_SUPPORTEDLOCK_REQUEST) +
+ strlen(remote->url) - 2;
out_data = xmalloc(out_buffer.size + 1);
- snprintf(out_data, out_buffer.size + 1, PROPFIND_REQUEST, remote->url);
+ snprintf(out_data, out_buffer.size + 1,
+ PROPFIND_SUPPORTEDLOCK_REQUEST, remote->url);
out_buffer.posn = 0;
out_buffer.buffer = out_data;
@@ -1085,85 +1321,99 @@ static int locking_available(void)
return lock_flags;
}
-static int is_ancestor(unsigned char *sha1, struct commit *commit)
+static struct object_list **process_blob(struct blob *blob,
+ struct object_list **p,
+ struct name_path *path,
+ const char *name)
{
- struct commit_list *parents;
+ struct object *obj = &blob->object;
- if (parse_commit(commit))
- return 0;
- parents = commit->parents;
- for (; parents; parents = parents->next) {
- if (!memcmp(sha1, parents->item->object.sha1, 20)) {
- return 1;
- } else if (parents->item->object.type == commit_type) {
- if (is_ancestor(
- sha1,
- (struct commit *)&parents->item->object
- ))
- return 1;
- }
+ obj->flags |= LOCAL;
+
+ if (obj->flags & (UNINTERESTING | SEEN))
+ return p;
+
+ obj->flags |= SEEN;
+ return add_object(obj, p, path, name);
+}
+
+static struct object_list **process_tree(struct tree *tree,
+ struct object_list **p,
+ struct name_path *path,
+ const char *name)
+{
+ struct object *obj = &tree->object;
+ struct tree_entry_list *entry;
+ struct name_path me;
+
+ obj->flags |= LOCAL;
+
+ if (obj->flags & (UNINTERESTING | SEEN))
+ return p;
+ if (parse_tree(tree) < 0)
+ die("bad tree object %s", sha1_to_hex(obj->sha1));
+
+ obj->flags |= SEEN;
+ p = add_object(obj, p, NULL, name);
+ me.up = path;
+ me.elem = name;
+ me.elem_len = strlen(name);
+ entry = tree->entries;
+ tree->entries = NULL;
+ while (entry) {
+ struct tree_entry_list *next = entry->next;
+ if (entry->directory)
+ p = process_tree(entry->item.tree, p, &me, entry->name);
+ else
+ p = process_blob(entry->item.blob, p, &me, entry->name);
+ free(entry);
+ entry = next;
}
- return 0;
+ return p;
}
-static void get_delta(unsigned char *sha1, struct object *obj,
- struct active_lock *lock)
+static void get_delta(struct rev_info *revs, struct remote_lock *lock)
{
struct commit *commit;
- struct commit_list *parents;
- struct tree *tree;
- struct tree_entry_list *entry;
+ struct object_list **p = &objects, *pending;
- if (sha1 && !memcmp(sha1, obj->sha1, 20))
- return;
+ while ((commit = get_revision(revs)) != NULL) {
+ p = process_tree(commit->tree, p, NULL, "");
+ commit->object.flags |= LOCAL;
+ if (!(commit->object.flags & UNINTERESTING))
+ add_request(&commit->object, lock);
+ }
- if (aborted)
- return;
+ for (pending = revs->pending_objects; pending; pending = pending->next) {
+ struct object *obj = pending->item;
+ const char *name = pending->name;
- if (obj->type == commit_type) {
- if (push_verbosely)
- fprintf(stderr, "walk %s\n", sha1_to_hex(obj->sha1));
- add_request(obj->sha1, lock);
- commit = (struct commit *)obj;
- if (parse_commit(commit)) {
- fprintf(stderr, "Error parsing commit %s\n",
- sha1_to_hex(obj->sha1));
- aborted = 1;
- return;
+ if (obj->flags & (UNINTERESTING | SEEN))
+ continue;
+ if (obj->type == tag_type) {
+ obj->flags |= SEEN;
+ p = add_object(obj, p, NULL, name);
+ continue;
}
- parents = commit->parents;
- for (; parents; parents = parents->next)
- if (sha1 == NULL ||
- memcmp(sha1, parents->item->object.sha1, 20))
- get_delta(sha1, &parents->item->object,
- lock);
- get_delta(sha1, &commit->tree->object, lock);
- } else if (obj->type == tree_type) {
- if (push_verbosely)
- fprintf(stderr, "walk %s\n", sha1_to_hex(obj->sha1));
- add_request(obj->sha1, lock);
- tree = (struct tree *)obj;
- if (parse_tree(tree)) {
- fprintf(stderr, "Error parsing tree %s\n",
- sha1_to_hex(obj->sha1));
- aborted = 1;
- return;
+ if (obj->type == tree_type) {
+ p = process_tree((struct tree *)obj, p, NULL, name);
+ continue;
}
- entry = tree->entries;
- tree->entries = NULL;
- while (entry) {
- struct tree_entry_list *next = entry->next;
- get_delta(sha1, entry->item.any, lock);
- free(entry->name);
- free(entry);
- entry = next;
+ if (obj->type == blob_type) {
+ p = process_blob((struct blob *)obj, p, NULL, name);
+ continue;
}
- } else if (obj->type == blob_type || obj->type == tag_type) {
- add_request(obj->sha1, lock);
+ die("unknown pending object %s (%s)", sha1_to_hex(obj->sha1), name);
+ }
+
+ while (objects) {
+ if (!(objects->item->flags & UNINTERESTING))
+ add_request(objects->item, lock);
+ objects = objects->next;
}
}
-static int update_remote(unsigned char *sha1, struct active_lock *lock)
+static int update_remote(unsigned char *sha1, struct remote_lock *lock)
{
struct active_request_slot *slot;
char *out_data;
@@ -1218,22 +1468,146 @@ static int update_remote(unsigned char *
return 1;
}
+static struct ref *local_refs, **local_tail;
+static struct ref *remote_refs, **remote_tail;
+
+static int one_local_ref(const char *refname, const unsigned char *sha1)
+{
+ struct ref *ref;
+ int len = strlen(refname) + 1;
+ ref = xcalloc(1, sizeof(*ref) + len);
+ memcpy(ref->new_sha1, sha1, 20);
+ memcpy(ref->name, refname, len);
+ *local_tail = ref;
+ local_tail = &ref->next;
+ return 0;
+}
+
+static void one_remote_ref(char *refname)
+{
+ struct ref *ref;
+ unsigned char remote_sha1[20];
+
+ if (fetch_ref(refname, remote_sha1) != 0) {
+ fprintf(stderr,
+ "Unable to fetch ref %s from %s\n",
+ refname, remote->url);
+ return;
+ }
+
+ int len = strlen(refname) + 1;
+ ref = xcalloc(1, sizeof(*ref) + len);
+ memcpy(ref->old_sha1, remote_sha1, 20);
+ memcpy(ref->name, refname, len);
+ *remote_tail = ref;
+ remote_tail = &ref->next;
+}
+
+static void get_local_heads(void)
+{
+ local_tail = &local_refs;
+ for_each_ref(one_local_ref);
+}
+
+static void get_dav_remote_heads(void)
+{
+ remote_tail = &remote_refs;
+ crawl_remote_refs("refs/");
+}
+
+static int is_zero_sha1(const unsigned char *sha1)
+{
+ int i;
+
+ for (i = 0; i < 20; i++) {
+ if (*sha1++)
+ return 0;
+ }
+ return 1;
+}
+
+static void unmark_and_free(struct commit_list *list, unsigned int mark)
+{
+ while (list) {
+ struct commit_list *temp = list;
+ temp->item->object.flags &= ~mark;
+ list = temp->next;
+ free(temp);
+ }
+}
+
+static int ref_newer(const unsigned char *new_sha1,
+ const unsigned char *old_sha1)
+{
+ struct object *o;
+ struct commit *old, *new;
+ struct commit_list *list, *used;
+ int found = 0;
+
+ /* Both new and old must be commit-ish and new is descendant of
+ * old. Otherwise we require --force.
+ */
+ o = deref_tag(parse_object(old_sha1), NULL, 0);
+ if (!o || o->type != commit_type)
+ return 0;
+ old = (struct commit *) o;
+
+ o = deref_tag(parse_object(new_sha1), NULL, 0);
+ if (!o || o->type != commit_type)
+ return 0;
+ new = (struct commit *) o;
+
+ if (parse_commit(new) < 0)
+ return 0;
+
+ used = list = NULL;
+ commit_list_insert(new, &list);
+ while (list) {
+ new = pop_most_recent_commit(&list, TMP_MARK);
+ commit_list_insert(new, &used);
+ if (new == old) {
+ found = 1;
+ break;
+ }
+ }
+ unmark_and_free(list, TMP_MARK);
+ unmark_and_free(used, TMP_MARK);
+ return found;
+}
+
+static void mark_edge_parents_uninteresting(struct commit *commit)
+{
+ struct commit_list *parents;
+
+ for (parents = commit->parents; parents; parents = parents->next) {
+ struct commit *parent = parents->item;
+ if (!(parent->object.flags & UNINTERESTING))
+ continue;
+ mark_tree_uninteresting(parent->tree);
+ }
+}
+
+static void mark_edges_uninteresting(struct commit_list *list)
+{
+ for ( ; list; list = list->next) {
+ struct commit *commit = list->item;
+
+ if (commit->object.flags & UNINTERESTING) {
+ mark_tree_uninteresting(commit->tree);
+ continue;
+ }
+ mark_edge_parents_uninteresting(commit);
+ }
+}
+
int main(int argc, char **argv)
{
struct transfer_request *request;
struct transfer_request *next_request;
int nr_refspec = 0;
char **refspec = NULL;
- int do_remote_update;
- int new_branch;
- int force_this;
- char *local_ref;
- unsigned char local_sha1[20];
- struct object *local_object = NULL;
- char *remote_ref = NULL;
- unsigned char remote_sha1[20];
- struct active_lock *remote_lock;
- char *remote_path = NULL;
+ struct remote_lock *ref_lock;
+ struct rev_info revs;
int rc = 0;
int i;
@@ -1242,6 +1616,7 @@ int main(int argc, char **argv)
remote = xmalloc(sizeof(*remote));
remote->url = NULL;
+ remote->path_len = 0;
remote->packs = NULL;
argv++;
@@ -1249,7 +1624,7 @@ int main(int argc, char **argv)
char *arg = *argv;
if (*arg == '-') {
- if (!strcmp(arg, "--complete")) {
+ if (!strcmp(arg, "--all")) {
push_all = 1;
continue;
}
@@ -1265,6 +1640,12 @@ int main(int argc, char **argv)
}
if (!remote->url) {
remote->url = arg;
+ char *path = strstr(arg, "//");
+ if (path) {
+ path = index(path+2, '/');
+ if (path)
+ remote->path_len = strlen(path);
+ }
continue;
}
refspec = argv;
@@ -1275,7 +1656,7 @@ int main(int argc, char **argv)
if (!remote->url)
usage(http_push_usage);
- memset(remote_dir_exists, 0, 256);
+ memset(remote_dir_exists, -1, 256);
http_init();
@@ -1293,92 +1674,113 @@ int main(int argc, char **argv)
goto cleanup;
}
- /* Process each refspec */
- for (i = 0; i < nr_refspec; i++) {
- char *ep;
- force_this = 0;
- do_remote_update = 0;
- new_branch = 0;
- local_ref = refspec[i];
- if (*local_ref == '+') {
- force_this = 1;
- local_ref++;
- }
- ep = strchr(local_ref, ':');
- if (ep) {
- remote_ref = ep + 1;
- *ep = 0;
+ /* Get a list of all local and remote heads to validate refspecs */
+ get_local_heads();
+ fprintf(stderr, "Fetching remote heads...\n");
+ get_dav_remote_heads();
+
+ /* match them up */
+ if (!remote_tail)
+ remote_tail = &remote_refs;
+ if (match_refs(local_refs, remote_refs, &remote_tail,
+ nr_refspec, refspec, push_all))
+ return -1;
+ if (!remote_refs) {
+ fprintf(stderr, "No refs in common and none specified; doing nothing.\n");
+ return 0;
+ }
+
+ int ret = 0;
+ int new_refs = 0;
+ struct ref *ref;
+ for (ref = remote_refs; ref; ref = ref->next) {
+ char old_hex[60], *new_hex;
+ if (!ref->peer_ref)
+ continue;
+ if (!memcmp(ref->old_sha1, ref->peer_ref->new_sha1, 20)) {
+ if (push_verbosely || 1)
+ fprintf(stderr, "'%s': up-to-date\n", ref->name);
+ continue;
}
- else
- remote_ref = local_ref;
+
+ if (!force_all &&
+ !is_zero_sha1(ref->old_sha1) &&
+ !ref->force) {
+ if (!has_sha1_file(ref->old_sha1) ||
+ !ref_newer(ref->peer_ref->new_sha1,
+ ref->old_sha1)) {
+ /* We do not have the remote ref, or
+ * we know that the remote ref is not
+ * an ancestor of what we are trying to
+ * push. Either way this can be losing
+ * commits at the remote end and likely
+ * we were not up to date to begin with.
+ */
+ error("remote '%s' is not a strict "
+ "subset of local ref '%s'. "
+ "maybe you are not up-to-date and "
+ "need to pull first?",
+ ref->name,
+ ref->peer_ref->name);
+ ret = -2;
+ continue;
+ }
+ }
+ memcpy(ref->new_sha1, ref->peer_ref->new_sha1, 20);
+ if (is_zero_sha1(ref->new_sha1)) {
+ error("cannot happen anymore");
+ ret = -3;
+ continue;
+ }
+ new_refs++;
+ strcpy(old_hex, sha1_to_hex(ref->old_sha1));
+ new_hex = sha1_to_hex(ref->new_sha1);
+
+ fprintf(stderr, "updating '%s'", ref->name);
+ if (strcmp(ref->name, ref->peer_ref->name))
+ fprintf(stderr, " using '%s'", ref->peer_ref->name);
+ fprintf(stderr, "\n from %s\n to %s\n", old_hex, new_hex);
+
/* Lock remote branch ref */
- if (remote_path)
- free(remote_path);
- remote_path = xmalloc(strlen(remote_ref) + 12);
- sprintf(remote_path, "refs/heads/%s", remote_ref);
- remote_lock = lock_remote(remote_path, LOCK_TIME);
- if (remote_lock == NULL) {
+ ref_lock = lock_remote(ref->name, LOCK_TIME);
+ if (ref_lock == NULL) {
fprintf(stderr, "Unable to lock remote branch %s\n",
- remote_ref);
+ ref->name);
rc = 1;
continue;
}
- /* Resolve local and remote refs */
- if (fetch_ref(remote_ref, remote_sha1) != 0) {
- fprintf(stderr,
- "Remote branch %s does not exist on %s\n",
- remote_ref, remote->url);
- new_branch = 1;
- }
- if (get_sha1(local_ref, local_sha1) != 0) {
- fprintf(stderr, "Error resolving local branch %s\n",
- local_ref);
- rc = 1;
- goto unlock;
- }
-
- /* Find relationship between local and remote */
- local_object = parse_object(local_sha1);
- if (!local_object) {
- fprintf(stderr, "Unable to parse local object %s\n",
- sha1_to_hex(local_sha1));
- rc = 1;
- goto unlock;
- } else if (new_branch) {
- do_remote_update = 1;
- } else {
- if (!memcmp(local_sha1, remote_sha1, 20)) {
- fprintf(stderr,
- "* %s: same as branch '%s' of %s\n",
- local_ref, remote_ref, remote->url);
- } else if (is_ancestor(remote_sha1,
- (struct commit *)local_object)) {
- fprintf(stderr,
- "Remote %s will fast-forward to local %s\n",
- remote_ref, local_ref);
- do_remote_update = 1;
- } else if (force_all || force_this) {
- fprintf(stderr,
- "* %s on %s does not fast forward to local branch '%s', overwriting\n",
- remote_ref, remote->url, local_ref);
- do_remote_update = 1;
- } else {
- fprintf(stderr,
- "* %s on %s does not fast forward to local branch '%s'\n",
- remote_ref, remote->url, local_ref);
- rc = 1;
- goto unlock;
- }
+ /* Set up revision info for this refspec */
+ const char *commit_argv[3];
+ int commit_argc = 2;
+ char *new_sha1_hex = strdup(sha1_to_hex(ref->new_sha1));
+ char *old_sha1_hex = NULL;
+ commit_argv[1] = new_sha1_hex;
+ if (!push_all && !is_zero_sha1(ref->old_sha1)) {
+ old_sha1_hex = xmalloc(42);
+ sprintf(old_sha1_hex, "^%s",
+ sha1_to_hex(ref->old_sha1));
+ commit_argv[2] = old_sha1_hex;
+ commit_argc++;
+ }
+ revs.commits = NULL;
+ setup_revisions(commit_argc, commit_argv, &revs, NULL);
+ revs.tag_objects = 1;
+ revs.tree_objects = 1;
+ revs.blob_objects = 1;
+ free(new_sha1_hex);
+ if (old_sha1_hex) {
+ free(old_sha1_hex);
+ commit_argv[1] = NULL;
}
- /* Generate and check list of required objects */
+ /* Generate a list of objects that need to be pushed */
pushing = 0;
- if (do_remote_update || push_all)
- fetch_indices();
- get_delta(push_all ? NULL : remote_sha1,
- local_object, remote_lock);
+ prepare_revision_walk(&revs);
+ mark_edges_uninteresting(revs.commits);
+ fetch_indices();
+ get_delta(&revs, ref_lock);
finish_all_active_slots();
/* Push missing objects to remote, this would be a
@@ -1388,25 +1790,15 @@ int main(int argc, char **argv)
finish_all_active_slots();
/* Update the remote branch if all went well */
- if (do_remote_update) {
- if (!aborted && update_remote(local_sha1,
- remote_lock)) {
- fprintf(stderr, "%s remote branch %s\n",
- new_branch ? "Created" : "Updated",
- remote_ref);
- } else {
- fprintf(stderr,
- "Unable to %s remote branch %s\n",
- new_branch ? "create" : "update",
- remote_ref);
- rc = 1;
- goto unlock;
- }
+ if (aborted || !update_remote(ref->new_sha1, ref_lock)) {
+ rc = 1;
+ goto unlock;
}
unlock:
- unlock_remote(remote_lock);
- free(remote_path);
+ if (!rc)
+ fprintf(stderr, " done\n");
+ unlock_remote(ref_lock);
}
cleanup:
--
1.2.4.gb477-dirty
^ permalink raw reply related
* Re: [RFH] zlib gurus out there?
From: Linus Torvalds @ 2006-03-08 0:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Nicolas Pitre
In-Reply-To: <7vzmk1izpa.fsf_-_@assigned-by-dhcp.cox.net>
On Tue, 7 Mar 2006, Junio C Hamano wrote:
>
> However, I am stuck with the first step, which is to do a full
> flush after the header. An obvious change to the code quoted
> above writes out a corrupt object:
>
> /* First header.. */
> stream.next_in = hdr;
> stream.avail_in = hdrlen;
> - while (deflate(&stream, 0) == Z_OK)
> + while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
> /* nothing */;
No, I don't think that's good. You're only doing a partial deflate, you
can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
buffer, and you don't.
iirc, wtf, wdik, and ianal.
Linus
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Junio C Hamano @ 2006-03-08 1:22 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0603071658300.32577@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Tue, 7 Mar 2006, Junio C Hamano wrote:
>>
>> However, I am stuck with the first step, which is to do a full
>> flush after the header. An obvious change to the code quoted
>> above writes out a corrupt object:
>>
>> /* First header.. */
>> stream.next_in = hdr;
>> stream.avail_in = hdrlen;
>> - while (deflate(&stream, 0) == Z_OK)
>> + while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
>> /* nothing */;
>
> No, I don't think that's good. You're only doing a partial deflate, you
> can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> buffer, and you don't.
So, in short there is no way to create:
hdr part deflated.
flush.
data part deflated independently.
and have the current sha1_read_file() not to notice that flush,
while I can inspect the deflated stream to find the "flush", and
copy only the defalted data part into a pack? Bummer... I was
really shooting for full backward compatibility.
^ permalink raw reply
* Re: git-svn, tree moves, and --no-stop-on-copy
From: Eric Wong @ 2006-03-08 1:42 UTC (permalink / raw)
To: Yann Dirson; +Cc: GIT list
In-Reply-To: <20060307220837.GB27397@nowhere.earth>
Yann Dirson <ydirson@altern.org> wrote:
> It looks that the --no-stop-on-copy flag has been dropped by error
> during the "options cleanup" commit a couple of days ago. This
> trivial patch appears at first sight to address the problem:
Thanks for the patch, but on second thought...
I'm tempted to drop it as an option... IIRC, the only reason
--no-stop-on-copy exists in git-svn is in case ancient versions of svn
did not support --stop-on-copy. I haven't bothered looking deeply into
SVN history to see if it was always supported or not.
> Before I find out in the doc about --no-stop-on-copy, I did a coupld
> of experimentation. Among them, was using a peg-revision in the URL
> passed to "git-svn init":
>
> $ GIT_SVN_ID=git-oldsvn git-svn init https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165
>
> That succeeds, but then "git-svn fetch" will fail with:
>
> svn: REPORT request failed on '/svnroot/ufoai/!svn/bc/190/trunk/src@165'
> svn: '/svnroot/ufoai/!svn/bc/190/trunk/src@165' path not found
> 256 at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 783
> man::svn_log_raw('https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165', '-r0:HEAD', '--stop-on-copy') called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 219
> main::fetch() called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 81
If you want full repository history for reorganized repositories,
easiest way is to pay the price for full repository and all of its
history.
git-svn init https://svn.sourceforge.net/svnroot/ufoai
git-svn fetch
# this puts all your branches and tags into one single big git tree.
However, the following should always work: (after the following patch,
GIT_SVN_ID=git-oldsvn git-svn init \
https://svn.sourceforge.net/svnroot/ufoai/trunk
GIT_SVN_ID=git-oldsvn git-svn fetch -r1:165
GIT_SVN_ID=git-newsvn git-svn init
https://svn.sourceforge.net/svnroot/ufoai/ufoai/trunk
GIT_SVN_ID=git-newsvn git-svn fetch \
166=`git-rev-parse refs/remotes/git-oldsvn`
Unfortunately, it does not, at least with svn 1.2.3... I have a patch
coming that should fix things for 1.1.1 (and give better 1.1.x support
in general). I'm not sure, but it feels like something is screwed up
with svn 1.2.3dfsg1-3:
This works: svn log -r1 https://svn.sourceforge.net/svnroot/ufoai/trunk
This doesn't: svn co -r1 https://svn.sourceforge.net/svnroot/ufoai/trunk
But this: svn co -r1 https://svn.sourceforge.net/svnroot/ufoai
will create the following structure:
ufoai/{trunk,branches,tags}
I'm quite puzzled about it, as I swear I've seen it work on a different
project recently (of course I cannot remember which :<)
> Maybe git-svn could also be guarded against peg-revisions on init
> command-line, since that appears to confuse it quite a bit :)
Possibly, but having '@' in URLs is valid in some cases outside of
peg-revisions.
> Additionally, it may be worth poiting out in the doc at least one
> valid use of the --no-stop-on-copy flag that is friendly to the user's
> sanity: when the svn repository has undergone a reorg, such that the
> URL passed to "init" indeed moved - at least, when/if it is made to
> work :)
In the face of repository reorgs, git-svn is happiest tracking partial
history. Or tracking the entire repository from the root.
Hopefully I've been reasonably coherent, having insomnia lately.
--
Eric Wong
^ permalink raw reply
* [PATCH] contrib/git-svn: fix UUID reading w/pre-1.2 svn; fetch args
From: Eric Wong @ 2006-03-08 1:57 UTC (permalink / raw)
To: Yann Dirson; +Cc: GIT list
In-Reply-To: <20060308014207.GA31137@localdomain>
Junio: please don't apply this patch to git.git just yet. It seems fine
to me, but I haven't tested it heavily yet (Yann can help me, I hope :)
I hardly slept the past few days and I may have broken something badly
(it pasts all the tests, though).
---
As a side effect, this should also work better for 'init' off
directories that are no longer in the latest revision of the
repository.
Fix 'fetch' args (<rev>=<commit> options) on brand-new heads
Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
contrib/git-svn/git-svn.perl | 26 ++++++++++++++++++--------
1 files changed, 18 insertions(+), 8 deletions(-)
9f59596bde5bdd68d1a0a116f7383df74966de44
diff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl
index c575883..b8d2b3e 100755
--- a/contrib/git-svn/git-svn.perl
+++ b/contrib/git-svn/git-svn.perl
@@ -162,7 +162,8 @@ sub rebuild {
croak "SVN repository location required: $url\n";
}
$SVN_URL ||= $url;
- $SVN_UUID ||= setup_git_svn();
+ $SVN_UUID ||= $uuid;
+ setup_git_svn();
$latest = $rev;
}
assert_revision_eq_or_unknown($rev, $c);
@@ -226,10 +227,12 @@ sub fetch {
push @svn_co,'--ignore-externals' unless $_no_ignore_ext;
sys(@svn_co, $SVN_URL, $SVN_WC);
chdir $SVN_WC or croak $!;
+ read_uuid();
$last_commit = git_commit($base, @parents);
assert_svn_wc_clean($base->{revision}, $last_commit);
} else {
chdir $SVN_WC or croak $!;
+ read_uuid();
$last_commit = file_to_s("$REV_DIR/$base->{revision}");
}
my @svn_up = qw(svn up);
@@ -275,7 +278,9 @@ sub commit {
fetch();
chdir $SVN_WC or croak $!;
- my $svn_current_rev = svn_info('.')->{'Last Changed Rev'};
+ my $info = svn_info('.');
+ read_uuid($info);
+ my $svn_current_rev = $info->{'Last Changed Rev'};
foreach my $c (@revs) {
my $mods = svn_checkout_tree($svn_current_rev, $c);
if (scalar @$mods == 0) {
@@ -314,6 +319,14 @@ sub show_ignore {
########################### utility functions #########################
+sub read_uuid {
+ return if $SVN_UUID;
+ my $info = shift || svn_info('.');
+ $SVN_UUID = $info->{'Repository UUID'} or
+ croak "Repository UUID unreadable\n";
+ s_to_file($SVN_UUID,"$GIT_DIR/$GIT_SVN/info/uuid");
+}
+
sub setup_git_svn {
defined $SVN_URL or croak "SVN repository location required\n";
unless (-d $GIT_DIR) {
@@ -323,14 +336,10 @@ sub setup_git_svn {
mkpath(["$GIT_DIR/$GIT_SVN/info"]);
mkpath([$REV_DIR]);
s_to_file($SVN_URL,"$GIT_DIR/$GIT_SVN/info/url");
- $SVN_UUID = svn_info($SVN_URL)->{'Repository UUID'} or
- croak "Repository UUID unreadable\n";
- s_to_file($SVN_UUID,"$GIT_DIR/$GIT_SVN/info/uuid");
open my $fd, '>>', "$GIT_DIR/$GIT_SVN/info/exclude" or croak $!;
print $fd '.svn',"\n";
close $fd or croak $!;
- return $SVN_UUID;
}
sub assert_svn_wc_clean {
@@ -860,7 +869,6 @@ sub git_commit {
my ($log_msg, @parents) = @_;
assert_revision_unknown($log_msg->{revision});
my $out_fh = IO::File->new_tmpfile or croak $!;
- $SVN_UUID ||= svn_info('.')->{'Repository UUID'};
map_tree_joins() if (@_branch_from && !%tree_map);
@@ -922,7 +930,9 @@ sub git_commit {
}
my @update_ref = ('git-update-ref',"refs/remotes/$GIT_SVN",$commit);
if (my $primary_parent = shift @exec_parents) {
- push @update_ref, $primary_parent;
+ if (!system('git-rev-parse',"refs/remotes/$GIT_SVN")){
+ push @update_ref, $primary_parent;
+ }
}
sys(@update_ref);
sys('git-update-ref',"$GIT_SVN/revs/$log_msg->{revision}",$commit);
--
1.2.4.g198d
^ permalink raw reply related
* Re: [RFH] zlib gurus out there?
From: Linus Torvalds @ 2006-03-08 2:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vslptivbg.fsf@assigned-by-dhcp.cox.net>
On Tue, 7 Mar 2006, Junio C Hamano wrote:
> >
> > No, I don't think that's good. You're only doing a partial deflate, you
> > can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> > buffer, and you don't.
Actually, I misread what you were trying to do, and thought this was the
inflate phase, not the deflate. Now that I understand what you want,
> So, in short there is no way to create:
>
> hdr part deflated.
> flush.
> data part deflated independently.
>
> and have the current sha1_read_file() not to notice that flush,
Actually, try the patch you already tried, except you'll need to add a
deflateEnd(&stream);
deflateInit(&stream, Z_BEST_COMPRESSION);
.. set up output parameters again ..
and you need to change the initial
size = deflateBound(&stream, len+hdrlen);
to
size = deflateBound(&stream, len) + deflateBound(&stream, hdrlen);
and then you might be ok.
That said, I'm not sure I agree with what you're trying to do.
Linus
^ permalink raw reply
* Re: Problems with using git
From: Greg KH @ 2006-03-08 2:16 UTC (permalink / raw)
To: Mark Wooding; +Cc: git
In-Reply-To: <20060304222557.GA24628@kroah.com>
On Sat, Mar 04, 2006 at 02:25:57PM -0800, Greg KH wrote:
> On Sat, Mar 04, 2006 at 10:56:09AM +0000, Mark Wooding wrote:
> > Greg KH <greg@kroah.com> wrote:
> >
> > > The latest development tree, and the latest public betas contain
> > > 1.1.3. If you think this should be newer, I can easily go poke the
> > > proper people...
> >
> > Given that there's a security issue which got fixed in 1.1.5, I think
> > this is really a bit poor.
> >
> > I notice, by contrast, that Debian had managed to repackage and release
> > a new GIT the day after Junio fixed the bug in the first place. That
> > was more than a month ago.
>
> Fair enough, I'll go poke the proper people now...
They jumped and the cogito and git should be updated to the latest
version in the next update.
thanks,
greg k-h
^ permalink raw reply
* Re: Pulling tags from git.git
From: A Large Angry SCM @ 2006-03-08 4:32 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Junio C Hamano, git
In-Reply-To: <440DA82D.3060909@op5.se>
Andreas Ericsson wrote:
> A Large Angry SCM wrote:
>> Andreas Ericsson wrote:
>>
>>> Junio C Hamano wrote:
>>>
>>>> Andreas Ericsson <ae@op5.se> writes:
>>>>
>>>>
>>>>> With the git or git+ssh protocol, tags will be autofollowed
>>>>> when you do a pull (only signed tags, I think). The
>>>>> auto-following is done by detecting tags that are fetched,
>>>>
>>>>
>>>>
>>>> Ah, you are correct. We do not follow lightweight tags; I am
>>>> not sure if we should.
>>>>
>>>
>>> I'm fairly sure we shouldn't. The default update-hook prevents them
>>> (if enabled), and I can't for the life of me think of why anyone
>>> would want to distribute such tags.
>>>
>>> OTOH, preventing unannotated tags from being pushed seems like a
>>> better way than to not have the ability to auto-follow those same
>>> tags. After all, it's better to discourage than to disallow.
>>>
>>
>> Before you do this, please explain why unannotated tags are not
>> useful, and so should not be allowed to be pushed.
>
>
> Imagine Linus, getting his "please pull" emails and doing so only to
> find dozens of temporary tags fetched by the pull. Junio's patch (if I
> read it correctly) unconditionally fetches *ALL* tags reachable from the
> top of the commit-chain, which means there is no longer any way to keep
> temporary tags in a repo from which someone else will pull.
Why is a "pull" bothering with tags? A "fetch" yes, but not a pull.
> I for one riddle my repos with temporary tags whenever I'm trying
> something I'm not so sure of, or find an interesting bug or a design
> decision I'm not 100% sure of. Perhaps I should rather do this with
> branches, but imo branches are for doing work, whereas tags just mark a
> spot in the development so I easily can find them with gitk or some such.
>
> I may be biased by the way we do things at work. In our workflow, all
> tags meant to be distributed have a short note in them which explains
> the rationale of the tag. For example, new versions have a very brief
> changelog that sales-people get on email (a blessing, that, since we
> devs no longer have to update feature-lists and such).
>
> Tags not meant to be distributed are unannotated, and unannotated tags
> are kept out of published repos which are always stored at a central
> server. Everybody synchronize to those central repos, so nobody pulls
> from each other. Perhaps this is how the kernel devs work too, but if it
> ever changes the update hook will no longer be able to safeguard from it
> and the, in my eyes, temporary tags will be distributed in a
> criss-crossing mesh so no-one will ever know where it came from or who
> created it or why. I.e. a Bad Thing.
The distinction here is not annotated tags or temporary tags but _local_
tags. _Your_ workflow conventions treat unannotated tags as local tags
but declaring that unannotated tags can not be pushed is imposing _your_
conventions on other groups. Just as branch names, themselves, can be
meaningful, so can tag names.
^ permalink raw reply
* Some more cvsimport
From: Rajkumar S @ 2006-03-08 8:49 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 677 bytes --]
Hi all,
Thanks every one for replying to my previous mails and to Smurf for his helpful replies
and patch.
The cvs project I am trying to track has 2 branchs. One head and another for releng_1. The
upstream cvs tree gets updated on both branchs. I also make modifications to both. When I
normally do cvsimport, the releng_1 tree gets updated automatically, but if I have checked
out the releng_1 branch in my local git repository when cvsimport happens the files do not
get updated. I can fix this by checking out the master branch just before cvsimport and
then switching back.
I am attaching a test script to simulate the problem and the fix is commented out.
raj
[-- Attachment #2: git_branch_init.sh --]
[-- Type: application/x-shellscript, Size: 2297 bytes --]
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Johannes Schindelin @ 2006-03-08 9:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0603071753370.32577@g5.osdl.org>
Hi,
On Tue, 7 Mar 2006, Linus Torvalds wrote:
> On Tue, 7 Mar 2006, Junio C Hamano wrote:
> > >
> > > No, I don't think that's good. You're only doing a partial deflate, you
> > > can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> > > buffer, and you don't.
>
> Actually, I misread what you were trying to do, and thought this was the
> inflate phase, not the deflate.
I don't think it matters if it is inflate or deflate. ZLib keeps an
internal state depending on the data. That is the whole reason why the
packing is so good: it uses the redundancy in the data already seen to
construct a codebook. (And that's also the reason why you can't start to
deflate in the middle.)
Ciao,
Dscho
^ permalink raw reply
* Re: Pulling tags from git.git
From: Andreas Ericsson @ 2006-03-08 10:13 UTC (permalink / raw)
To: gitzilla; +Cc: Junio C Hamano, git
In-Reply-To: <440E5E40.7090700@gmail.com>
A Large Angry SCM wrote:
>
> Why is a "pull" bothering with tags? A "fetch" yes, but not a pull.
>
A pull is a fetch + merge. I said pull because what little I know of
Linus' workflow is the the emails he gets from susbsystem maintainers
are called "pull requests".
>>
>> Tags not meant to be distributed are unannotated, and unannotated tags
>> are kept out of published repos which are always stored at a central
>> server. Everybody synchronize to those central repos, so nobody pulls
>> from each other. Perhaps this is how the kernel devs work too, but if
>> it ever changes the update hook will no longer be able to safeguard
>> from it and the, in my eyes, temporary tags will be distributed in a
>> criss-crossing mesh so no-one will ever know where it came from or who
>> created it or why. I.e. a Bad Thing.
>
>
> The distinction here is not annotated tags or temporary tags but _local_
> tags. _Your_ workflow conventions treat unannotated tags as local tags
> but declaring that unannotated tags can not be pushed is imposing _your_
> conventions on other groups. Just as branch names, themselves, can be
> meaningful, so can tag names.
>
Yes, that's why I said it's better to discourage than to disallow. The
default update-hook is disabled by default and there are comments
aplenty to make it possible even for the most die-hard point-and-click
monkey to be able to comment out the disallowing of unannotated tags.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* [PATCH] write_sha1_file(): Perform Z_FULL_FLUSH between header and data
From: Sergey Vlasov @ 2006-03-08 10:45 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Nicolas Pitre, Linus Torvalds
In-Reply-To: <7vzmk1izpa.fsf_-_@assigned-by-dhcp.cox.net>
Data after Z_FULL_FLUSH will be compressed independently of the
header, and could therefore be reused without recompressing when
creating a pack.
---
This passes "make test" and unpacking of the whole git repo with
git-fsck-objects afterwards.
However, a straight reuse still will not be possible, because
sha1write_compressed() uses deflateInit(&stream, Z_DEFAULT_COMPRESSION),
which writes zlib headers around the deflate stream, and the zlib footer
contains adler32 checksum. So, as a minimum, you will need to
decompress the object data, calculate its adler32 checksum and write the
zlib header yourself.
sha1_file.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
8b12d9a58e87a4c5b5a2a7b20d06fe29a5afb903
diff --git a/sha1_file.c b/sha1_file.c
index a80d849..34d4da4 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1399,7 +1399,8 @@ int write_sha1_file(void *buf, unsigned
/* Set it up */
memset(&stream, 0, sizeof(stream));
deflateInit(&stream, Z_BEST_COMPRESSION);
- size = deflateBound(&stream, len+hdrlen);
+ /* Additional 6 bytes for the Z_FULL_FLUSH marker */
+ size = deflateBound(&stream, hdrlen) + 6 + deflateBound(&stream, len);
compressed = xmalloc(size);
/* Compress it */
@@ -1412,6 +1413,10 @@ int write_sha1_file(void *buf, unsigned
while (deflate(&stream, 0) == Z_OK)
/* nothing */;
+ /* Flush before data */
+ while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
+ /* nothing */;
+
/* Then the data itself.. */
stream.next_in = buf;
stream.avail_in = len;
--
1.2.GIT
^ permalink raw reply related
* Re: [PATCH] write_sha1_file(): Perform Z_FULL_FLUSH between header and data
From: Junio C Hamano @ 2006-03-08 11:04 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: git
In-Reply-To: <20060308134519.78ea313d.vsu@altlinux.ru>
Sergey Vlasov <vsu@altlinux.ru> writes:
> However, a straight reuse still will not be possible, because
> sha1write_compressed() uses deflateInit(&stream, Z_DEFAULT_COMPRESSION),
> which writes zlib headers around the deflate stream, and the zlib footer
> contains adler32 checksum. So, as a minimum, you will need to
> decompress the object data, calculate its adler32 checksum and write the
> zlib header yourself.
Hmph. Thanks for helping, but it sounds like my original plan
was not useful at all. Probably inflating would be still
cheaper than inflating and then deflating, but it would not be
as cool as a straight copy. Sigh...
^ permalink raw reply
* Update hook in Cygwin
From: Niklas Höglund @ 2006-03-08 12:16 UTC (permalink / raw)
To: git
Hi.
After creating a couple of repositories and pushing and cloning them,
I get the following:
$ git push --all origin
...
hooks/update: line 88: mail: command not found
This is in cygwin. I'm rather glad I don't have the mail command
installed, as I don't want mails going anywhere.
The update hook contains the following comment:
# To enable this hook:
# (1) change the recipient e-mail address
# (2) make this file executable by "chmod +x update".
But my impression after a cursory look at it is that it would always
call "mail" whenever it is run, and since all files are executable in
Windows (AFAIK), it would always be run.
^ permalink raw reply
* Re: Update hook in Cygwin
From: Andreas Ericsson @ 2006-03-08 13:36 UTC (permalink / raw)
To: Niklas Höglund; +Cc: git
In-Reply-To: <ad8ce5c20603080416g5ed6d77el@mail.gmail.com>
Niklas Höglund wrote:
> Hi.
>
> After creating a couple of repositories and pushing and cloning them,
> I get the following:
>
> $ git push --all origin
> ...
> hooks/update: line 88: mail: command not found
>
> This is in cygwin. I'm rather glad I don't have the mail command
> installed, as I don't want mails going anywhere.
>
> The update hook contains the following comment:
>
> # To enable this hook:
> # (1) change the recipient e-mail address
> # (2) make this file executable by "chmod +x update".
>
> But my impression after a cursory look at it is that it would always
> call "mail" whenever it is run, and since all files are executable in
> Windows (AFAIK), it would always be run.
I was under the impression that the cygwin abstraction layer had some
unixy permission thing on top of NTFS. Perhaps that's wrong. If you
remove the hook it won't be called.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [PATCH] write_sha1_file(): Perform Z_FULL_FLUSH between header and data
From: Sergey Vlasov @ 2006-03-08 14:17 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vhd69i4ep.fsf@assigned-by-dhcp.cox.net>
[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]
On Wed, Mar 08, 2006 at 03:04:14AM -0800, Junio C Hamano wrote:
> Sergey Vlasov <vsu@altlinux.ru> writes:
> > However, a straight reuse still will not be possible, because
> > sha1write_compressed() uses deflateInit(&stream, Z_DEFAULT_COMPRESSION),
> > which writes zlib headers around the deflate stream, and the zlib footer
> > contains adler32 checksum. So, as a minimum, you will need to
> > decompress the object data, calculate its adler32 checksum and write the
> > zlib header yourself.
>
> Hmph. Thanks for helping, but it sounds like my original plan
> was not useful at all. Probably inflating would be still
> cheaper than inflating and then deflating, but it would not be
> as cool as a straight copy. Sigh...
Actually you can calculate adler32 checksum of object data from
adler32(header+data) (available at the end of the loose object file),
adler32(header) (which you will need to calculate) and len(data)
(which is available in the header):
#define ADLER32_BASE 65521UL
unsigned int adler32_split(unsigned int adler_full, unsigned int adler_1,
unsigned long len_2)
{
unsigned long s1_1 = adler_1 & 0xffff;
unsigned long s1_2 = (adler_1 >> 16) & 0xffff;
unsigned long rem = len_2 % ADLER32_BASE;
unsigned long s_1_offset = (s1_1 + ADLER32_BASE - 1) % ADLER32_BASE;
unsigned long s_2_offset = (s1_2 + s_1_offset*rem) % ADLER32_BASE;
unsigned long sf_1 = adler_full & 0xffff;
unsigned long sf_2 = (adler_full >> 16) & 0xffff;
unsigned long s2_1 = (sf_1 + ADLER32_BASE - s_1_offset) % ADLER32_BASE;
unsigned long s2_2 = (sf_2 + ADLER32_BASE - s_2_offset) % ADLER32_BASE;
return (s2_2 << 16) | s2_1;
}
However, the resulting code probably won't be pretty...
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] git-blame: Make the output human readable
From: Sergey Vlasov @ 2006-03-08 14:32 UTC (permalink / raw)
To: linux; +Cc: junkio, git
In-Reply-To: <20060306193326.19262.qmail@science.horizon.com>
[-- Attachment #1: Type: text/plain, Size: 973 bytes --]
On 6 Mar 2006 14:33:26 -0500 linux@horizon.com wrote:
> Well, getting 15 characters in UTF-8 is easy (just stop before the 16th
> byte for which ((b & 0xc0) != 0x80)), but what about combining characters?
>
> You've got accents and stuff to worry about. And the annoying fact that
> Unicode defined accents as suffixes, so you have to go past the 15th
> column to include all of the
>
> And then there's that fact that many characters are traditionally
> represented as double-wide forms, even on character terminals.
>
> See http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c for details
> an an example implementation of wcwidth().
>
[skip]
> /* Now find the width of it */
> w = wcwidth(c);
And this won't work, unless you also add that wcwidth() implementation
to git.
The problem is that the wchar_t encoding is not specified anywhere -
glibc uses Unicode for it, but other systems can use whatever they want
(even locale-dependent).
[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]
^ permalink raw reply
* Re: Update hook in Cygwin
From: Shawn Pearce @ 2006-03-08 14:44 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Niklas Höglund, git
In-Reply-To: <440EDDE4.9070405@op5.se>
Andreas Ericsson <ae@op5.se> wrote:
> Niklas Höglund wrote:
> >Hi.
> >
> >After creating a couple of repositories and pushing and cloning them,
> >I get the following:
> >
> >$ git push --all origin
> >...
> >hooks/update: line 88: mail: command not found
> >
> >This is in cygwin. I'm rather glad I don't have the mail command
> >installed, as I don't want mails going anywhere.
> >
> >The update hook contains the following comment:
> >
> ># To enable this hook:
> ># (1) change the recipient e-mail address
> ># (2) make this file executable by "chmod +x update".
> >
> >But my impression after a cursory look at it is that it would always
> >call "mail" whenever it is run, and since all files are executable in
> >Windows (AFAIK), it would always be run.
>
>
> I was under the impression that the cygwin abstraction layer had some
> unixy permission thing on top of NTFS. Perhaps that's wrong. If you
> remove the hook it won't be called.
I've seen the same thing with the hooks on Cywin. I would consider
it to be a bug in either GIT or Cygwin but I haven't decided
which yet.
If you look at the share/git-core/templates directory on a real
UNIX system would see that the hooks are not marked executable by
default when installed. They are copied non-executable into each
new repository by git-init-db. Since they aren't executable they
don't get run.
But on Cygwin the hooks appear to be getting installed and are marked
executable in share/git-core/templates. So when git-init-db copies
them over to the new repository they are by default enabled.
Removing the execute bit from the files in share/git-core/templates
doesn't help; for some reason git-init-db is still copying them with
the execute bit enabled. I haven't spent the time to figure out why
yet; so I just run rm .git/hooks/* on every repository I come across.
BTW: chmod a-x .git/hooks/* also works as the Cygwin unixy permission
layer will remember the change.
--
Shawn.
^ permalink raw reply
* Re: Update hook in Cygwin
From: Christopher Faylor @ 2006-03-08 15:54 UTC (permalink / raw)
To: Andreas Ericsson, Niklas H?glund, Shawn Pearce, git
In-Reply-To: <20060308144413.GA516@spearce.org>
On Wed, Mar 08, 2006 at 09:44:13AM -0500, Shawn Pearce wrote:
>BTW: chmod a-x .git/hooks/* also works as the Cygwin unixy permission
>layer will remember the change.
Using "chmod a-x" should work like linux as long as you're using an NTFS
filesystem and have not specifically turned off Cygwin's handling of this
kind of thing with the CYGWIN=nontsec environment variable.
i.e., it should work in most cases on W2K+
cgf
^ permalink raw reply
* git-rev-list bug?
From: Catalin Marinas @ 2006-03-08 16:19 UTC (permalink / raw)
To: git
Sorry if this was previously discussed. I ran git-rev-list on a linear
graph and tried to filter the results by a file name:
git rev-list since.. path/to/file
but it always shows the child commit of "since" even if it didn't
touch the file. The same behaviour is for git-log (since it uses
git-rev-list) but git-whatchanged seems to be fine.
Is this the intended behaviour? The "stg patches" command based on
git-rev-list used to work fine a few weeks ago but now it is always
reporting the bottom patch in the stack as modifying a given file.
Thanks.
--
Catalin
^ permalink raw reply
* Re: git-svn, tree moves, and --no-stop-on-copy
From: Matthias Urlichs @ 2006-03-08 17:02 UTC (permalink / raw)
To: git
In-Reply-To: <20060307220837.GB27397@nowhere.earth>
Hi, Yann Dirson wrote:
> "svn switch --relocate" does not seem to be of any help. Switching
> manually .git/git-svn/tree/ to the new repository location does not
> help either, since I must obviously update to r166 in that case, and
> then a further "git-svn fetch" fails because it does not find
> .git/git-svn/revs/166 aleady imported.
>
> Any idea as to how to get the work done ?
You can manually edit the .git/corr file. Simply add an entry for #166
that has your reorganized (if necessary) head's SHA1.
--
Matthias Urlichs
^ permalink raw reply
* Re: [PATCH] git-blame: Make the output human readable
From: linux @ 2006-03-08 18:04 UTC (permalink / raw)
To: linux, vsu; +Cc: git, junkio
In-Reply-To: <20060308173249.1faed1d7.vsu@altlinux.ru>
> And this won't work, unless you also add that wcwidth() implementation
> to git.
That was the general idea. It is freely usable.
> The problem is that the wchar_t encoding is not specified anywhere -
> glibc uses Unicode for it, but other systems can use whatever they want
> (even locale-dependent).
Why is that a problem? None of the code mentioned even uses wchar_t.
The code I wrote converts from UTF-8 to straight Unicode, and that's
what Markus Kuhn's wcwidth() expects as an argument.
At no time do we ask the compiler for its opinion on the subject.
^ permalink raw reply
* [PATCH] test-delta needs zlib to compile
From: Nicolas Pitre @ 2006-03-08 18:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Signed-off-by: Nicolas Pitre <nico@cam.org>
---
diff --git a/Makefile b/Makefile
index a5eb0c4..89d67d6 100644
--- a/Makefile
+++ b/Makefile
@@ -565,7 +565,7 @@ test-date$X: test-date.c date.o ctype.o
$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) test-date.c date.o ctype.o
test-delta$X: test-delta.c diff-delta.o patch-delta.o
- $(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $^
+ $(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $^ -lz
check:
for i in *.c; do sparse $(ALL_CFLAGS) $(SPARSE_FLAGS) $$i || exit; done
^ permalink raw reply related
* Re: [PATCH] git-blame: Make the output human readable
From: Sergey Vlasov @ 2006-03-08 18:30 UTC (permalink / raw)
To: linux; +Cc: git, junkio
In-Reply-To: <20060308180422.27978.qmail@science.horizon.com>
[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]
On Wed, Mar 08, 2006 at 01:04:22PM -0500, linux@horizon.com wrote:
> > And this won't work, unless you also add that wcwidth() implementation
> > to git.
>
> That was the general idea. It is freely usable.
>
> > The problem is that the wchar_t encoding is not specified anywhere -
> > glibc uses Unicode for it, but other systems can use whatever they want
> > (even locale-dependent).
>
> Why is that a problem? None of the code mentioned even uses wchar_t.
> The code I wrote converts from UTF-8 to straight Unicode, and that's
> what Markus Kuhn's wcwidth() expects as an argument.
wcwidth() is a standard library function which takes a wchar_t:
http://www.opengroup.org/onlinepubs/009695399/functions/wcwidth.html
It is easy to write a program which assumes that wchar_t is Unicode
without noticing it, because it will work fine with glibc...
So that mk_wcwidth() must be used unconditionally, and not as a
fallback for systems which do not provide wcwidth() in libc.
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox