[PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

[PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

[Nicolas Pitre]

This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.

That commit made the situation better for repositories with relatively
small number of objects.  However with many objects and a small pack size
limit, the time required to complete the repack tends towards O(n^2),
or even much worse with long delta chains.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
---

Fixing this doesn't appear to be as trivial as I initially thought.
Although I do have some ideas, they're not appropriate so late in 
the -rc period.

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index dcfe62a..e1d3adf 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -445,13 +445,9 @@ static int write_one(struct sha1file *f,
 	if (e->idx.offset || e->preferred_base)
 		return -1;
 
-	/*
-	 * If we are deltified, attempt to write out base object first.
-	 * If that fails due to the pack size limit then the current
-	 * object might still possibly fit undeltified within that limit.
-	 */
-	if (e->delta)
-	       write_one(f, e->delta, offset);
+	/* if we are deltified, write out base object first. */
+	if (e->delta && !write_one(f, e->delta, offset))
+		return 0;
 
 	e->idx.offset = *offset;
 	size = write_object(f, e, *offset);
@@ -505,9 +501,11 @@ static void write_pack_file(void)
 		sha1write(f, &hdr, sizeof(hdr));
 		offset = sizeof(hdr);
 		nr_written = 0;
-		for (i = 0; i < nr_objects; i++)
-			if (write_one(f, objects + i, &offset) == 1)
-				display_progress(progress_state, written);
+		for (; i < nr_objects; i++) {
+			if (!write_one(f, objects + i, &offset))
+				break;
+			display_progress(progress_state, written);
+		}
 
 		/*
 		 * Did we write the wrong # entries in the header?
@@ -582,7 +580,7 @@ static void write_pack_file(void)
 			written_list[j]->offset = (off_t)-1;
 		}
 		nr_remaining -= nr_written;
-	} while (nr_remaining);
+	} while (nr_remaining && i < nr_objects);
 
 	free(written_list);
 	stop_progress(&progress_state);
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 1058d98..7649b81 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -389,7 +389,7 @@ test_expect_success 'verify resulting packs' '
 test_expect_success 'tolerate packsizelimit smaller than biggest object' '
 	git config pack.packSizeLimit 1 &&
 	packname_11=$(git pack-objects test-11 <obj-list) &&
-	test 3 = $(ls test-11-*.pack | wc -l)
+	test 5 = $(ls test-11-*.pack | wc -l)
 '
 
 test_expect_success 'verify resulting packs' '

Re: [PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

[Junio C Hamano]

Nicolas Pitre <nico@fluxnic.net> writes:

> This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.
>
> That commit made the situation better for repositories with relatively
> small number of objects.  However with many objects and a small pack size
> limit, the time required to complete the repack tends towards O(n^2),
> or even much worse with long delta chains.
>
> Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
> ---
>
> Fixing this doesn't appear to be as trivial as I initially thought.
> Although I do have some ideas, they're not appropriate so late in 
> the -rc period.

Ok, so the idea is what a2430dd tried is an issue worth addressing but the
particular execution wasn't good?  We revert it for now, but we will try
again after the release, perhaps doing it differently?

I agree with it if that is your intention, but "Most of" bugs me a bit.

Re: [PATCH] Revert "pack-objects: fix pack generation when using pack_size_limit"

[Nicolas Pitre]

On Mon, 8 Feb 2010, Junio C Hamano wrote:

> Nicolas Pitre <nico@fluxnic.net> writes:
> 
> > This reverts most of commit a2430dde8ceaaaabf05937438249397b883ca77a.
> >
> > That commit made the situation better for repositories with relatively
> > small number of objects.  However with many objects and a small pack size
> > limit, the time required to complete the repack tends towards O(n^2),
> > or even much worse with long delta chains.
> >
> > Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
> > ---
> >
> > Fixing this doesn't appear to be as trivial as I initially thought.
> > Although I do have some ideas, they're not appropriate so late in 
> > the -rc period.
> 
> Ok, so the idea is what a2430dd tried is an issue worth addressing but the
> particular execution wasn't good?  We revert it for now, but we will try
> again after the release, perhaps doing it differently?

Exact.  I did try a few things in the hope that the "fix" would have 
been trivial enough to merge now but none of that worked satisfactorily.

> I agree with it if that is your intention, but "Most of" bugs me a bit.

It's not a straight revert in the sense that I left in some cleanups and 
left out the redundant self consistency check.  Only the significant 
parts were reverted.


Nicolas