[PATCH] Don't fetch objects that exist in the local repository

[PATCH] Don't fetch objects that exist in the local repository

[Nick Hengeveld]

Be sure not to fetch objects that already exist in the local repository.
The main process loop no longer performs this check, http-fetch now checks
prior to starting a new request queue entry and when fetch_object() is called,
and local-fetch now checks when fetch_object() is called.

As discussed in this thread: http://marc.theaimsgroup.com/?t=112854890500001

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>


---

 fetch.c       |    2 +-
 http-fetch.c  |   10 +++++++++-
 local-fetch.c |    5 ++++-
 3 files changed, 14 insertions(+), 3 deletions(-)

41b78748fe458224fc1d621f5f0a4df2a3ac3253
diff --git a/fetch.c b/fetch.c
--- a/fetch.c
+++ b/fetch.c
@@ -165,7 +165,7 @@ static int loop(void)
 		 * the queue because we needed to fetch it first.
 		 */
 		if (! (obj->flags & TO_SCAN)) {
-			if (!has_sha1_file(obj->sha1) && fetch(obj->sha1)) {
+			if (fetch(obj->sha1)) {
 				report_missing(obj->type
 					       ? obj->type
 					       : "object", obj->sha1);
diff --git a/http-fetch.c b/http-fetch.c
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -489,7 +489,10 @@ void process_request_queue()
 
 	while (active_requests < max_requests && request != NULL) {
 		if (request->state == WAITING) {
-			start_request(request);
+			if (has_sha1_file(request->sha1))
+				release_request(request);
+			else
+				start_request(request);
 			curl_multi_perform(curlm, &num_transfers);
 		}
 		request = request->next;
@@ -890,6 +893,11 @@ static int fetch_object(struct alt_base 
 	if (request == NULL)
 		return error("Couldn't find request for %s in the queue", hex);
 
+	if (has_sha1_file(request->sha1)) {
+		release_request(request);
+		return 0;
+	}
+
 #ifdef USE_CURL_MULTI
 	int num_transfers;
 	while (request->state == WAITING) {
diff --git a/local-fetch.c b/local-fetch.c
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -166,7 +166,10 @@ static int fetch_file(const unsigned cha
 
 int fetch(unsigned char *sha1)
 {
-	return fetch_file(sha1) && fetch_pack(sha1);
+	if (has_sha1_file(sha1))
+		return 0;
+	else
+		return fetch_file(sha1) && fetch_pack(sha1);
 }
 
 int fetch_ref(char *ref, unsigned char *sha1)

Re: [PATCH] Don't fetch objects that exist in the local repository

[Johannes Schindelin]

Hi,

On Fri, 7 Oct 2005, Nick Hengeveld wrote:

> Be sure not to fetch objects that already exist in the local repository.

Really? I seem to recall a dispute I had with Linus that this is 
unacceptable. He seems worried about incomplete fetches.

Only objects which are descendants of refs are safe, since the refs are 
written only after the objects were fetched successfully.

Ciao,
Dscho

Re: [PATCH] Don't fetch objects that exist in the local repository

[Nick Hengeveld]

On Sat, Oct 08, 2005 at 12:50:53AM +0200, Johannes Schindelin wrote:

> > Be sure not to fetch objects that already exist in the local repository.
> 
> Really? I seem to recall a dispute I had with Linus that this is 
> unacceptable. He seems worried about incomplete fetches.

This patch just moved an existing check from the process queue loop to the
appropriate places in transport-specific code to prevent the transport
from transferring an object that appeared in the local repository after
it was prefetched (eg. via a pack), and to make sure that all objects that
were prefetched are subsequently fetched so the transport can perform the
appropriate cleanup.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Re: [PATCH] Don't fetch objects that exist in the local repository

[Johannes Schindelin]

Hi,

On Fri, 7 Oct 2005, Nick Hengeveld wrote:

> On Sat, Oct 08, 2005 at 12:50:53AM +0200, Johannes Schindelin wrote:
> 
> > > Be sure not to fetch objects that already exist in the local repository.
> > 
> > Really? I seem to recall a dispute I had with Linus that this is 
> > unacceptable. He seems worried about incomplete fetches.
> 
> This patch just moved an existing check from the process queue loop to the
> appropriate places in transport-specific code to prevent the transport
> from transferring an object that appeared in the local repository after
> it was prefetched (eg. via a pack), and to make sure that all objects that
> were prefetched are subsequently fetched so the transport can perform the
> appropriate cleanup.

Ah! Okay, now I understand.

Thanks,
Dscho