[PATCH] remove delta-against-self bit

[PATCH] remove delta-against-self bit

[Nicolas Pitre]

After experimenting with code to add the ability to encode a delta 
against part of the deltified file, it turns out that resulting packs 
are _bigger_ than when this ability is not used.  The raw delta output 
might be smaller, but it doesn't compress as well using gzip with a 
negative net saving on average.

Said bit would in fact be more useful to allow for encoding the copying 
of chunks larger than 64KB providing more savings with large files.  
This will correspond to packs version 3.

While the current code still produces packs version 2, it is made future 
proof so pack versions 2 and 3 are accepted.  Any pack version 2 are 
compatible with version 3 since the redefined bit was never used before.  
When enough time has passed, code to use that bit to produce version 3 
packs could be added.

Signed-off-by: Nicolas Pitre <nico@cam.org>

---

diff --git a/patch-delta.c b/patch-delta.c
index 98c27be..c0e1311 100644
--- a/patch-delta.c
+++ b/patch-delta.c
@@ -44,16 +44,15 @@ void *patch_delta(void *src_buf, unsigne
 		cmd = *data++;
 		if (cmd & 0x80) {
 			unsigned long cp_off = 0, cp_size = 0;
-			const unsigned char *buf;
 			if (cmd & 0x01) cp_off = *data++;
 			if (cmd & 0x02) cp_off |= (*data++ << 8);
 			if (cmd & 0x04) cp_off |= (*data++ << 16);
 			if (cmd & 0x08) cp_off |= (*data++ << 24);
 			if (cmd & 0x10) cp_size = *data++;
 			if (cmd & 0x20) cp_size |= (*data++ << 8);
+			if (cmd & 0x40) cp_size |= (*data++ << 16);
 			if (cp_size == 0) cp_size = 0x10000;
-			buf = (cmd & 0x40) ? dst_buf : src_buf;
-			memcpy(out, buf + cp_off, cp_size);
+			memcpy(out, src_buf + cp_off, cp_size);
 			out += cp_size;
 		} else {
 			memcpy(out, data, cmd);
diff --git a/pack.h b/pack.h
index 657deaa..9dafa2b 100644
--- a/pack.h
+++ b/pack.h
@@ -21,6 +21,7 @@ enum object_type {
  */
 #define PACK_SIGNATURE 0x5041434b	/* "PACK" */
 #define PACK_VERSION 2
+#define pack_version_ok(v) ((v) == htonl(2) || (v) == htonl(3))
 struct pack_header {
 	unsigned int hdr_signature;
 	unsigned int hdr_version;
diff --git a/index-pack.c b/index-pack.c
index 541d7bc..babe34b 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -68,9 +68,9 @@ static void parse_pack_header(void)
 	hdr = (void *)pack_base;
 	if (hdr->hdr_signature != htonl(PACK_SIGNATURE))
 		die("packfile '%s' signature mismatch", pack_name);
-	if (hdr->hdr_version != htonl(PACK_VERSION))
-		die("packfile '%s' version %d different from ours %d",
-		    pack_name, ntohl(hdr->hdr_version), PACK_VERSION);
+	if (!pack_version_ok(hdr->hdr_version))
+		die("packfile '%s' version %d unsupported",
+		    pack_name, ntohl(hdr->hdr_version));
 
 	nr_objects = ntohl(hdr->hdr_entries);
 
diff --git a/pack-check.c b/pack-check.c
index 511f294..67a7ecd 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -16,9 +16,9 @@ static int verify_packfile(struct packed
 	hdr = p->pack_base;
 	if (hdr->hdr_signature != htonl(PACK_SIGNATURE))
 		return error("Packfile %s signature mismatch", p->pack_name);
-	if (hdr->hdr_version != htonl(PACK_VERSION))
-		return error("Packfile version %d different from ours %d",
-			     ntohl(hdr->hdr_version), PACK_VERSION);
+	if (!pack_version_ok(hdr->hdr_version))
+		return error("Packfile version %d unsupported",
+			     ntohl(hdr->hdr_version));
 	nr_objects = ntohl(hdr->hdr_entries);
 	if (num_packed_objects(p) != nr_objects)
 		return error("Packfile claims to have %d objects, "
diff --git a/pack-objects.c b/pack-objects.c
diff --git a/unpack-objects.c b/unpack-objects.c
index 4b5b5cb..815a1b3 100644
--- a/unpack-objects.c
+++ b/unpack-objects.c
@@ -246,13 +246,12 @@ static void unpack_all(void)
 {
 	int i;
 	struct pack_header *hdr = fill(sizeof(struct pack_header));
-	unsigned version = ntohl(hdr->hdr_version);
 	unsigned nr_objects = ntohl(hdr->hdr_entries);
 
 	if (ntohl(hdr->hdr_signature) != PACK_SIGNATURE)
 		die("bad pack file");
-	if (version != PACK_VERSION)
-		die("unable to handle pack file version %d", version);
+	if (!pack_version_ok(hdr->hdr_version))
+		die("unknown pack file version %d", ntohl(hdr->hdr_version));
 	fprintf(stderr, "Unpacking %d objects\n", nr_objects);
 
 	use(sizeof(struct pack_header));

Re: [PATCH] remove delta-against-self bit

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> Said bit would in fact be more useful to allow for encoding the copying 
> of chunks larger than 64KB providing more savings with large files.  
> This will correspond to packs version 3.
>
> While the current code still produces packs version 2, it is made future 
> proof so pack versions 2 and 3 are accepted.  Any pack version 2 are 
> compatible with version 3 since the redefined bit was never used before.  
> When enough time has passed, code to use that bit to produce version 3 
> packs could be added.

I agree with the general direction and this futureproofing is a
good thing to have before 1.2.0, I think.

The bit is however _already_ looked at by the count_delta(),
to assess the extent of damage, IIRC.  Should we be
futureproofing that bit as well?

[PATCH] count-delta.c: Match the delta data semantics change in version 3.

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> I agree with the general direction and this futureproofing is a
> good thing to have before 1.2.0, I think.
>
> The bit is however _already_ looked at by the count_delta(),
> to assess the extent of damage, IIRC.  Should we be
> futureproofing that bit as well?

Something like this?

-- >8 --
This matches the count_delta() logic to the change previous
commit introduces to patch_delta().

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 count-delta.c |    7 ++-----
 1 files changed, 2 insertions(+), 5 deletions(-)

6cb2807fd27b9e970a9b012b925deab99f480d61
diff --git a/count-delta.c b/count-delta.c
index 7559ff6..978a60c 100644
--- a/count-delta.c
+++ b/count-delta.c
@@ -50,13 +50,10 @@ int count_delta(void *delta_buf, unsigne
 			if (cmd & 0x08) cp_off |= (*data++ << 24);
 			if (cmd & 0x10) cp_size = *data++;
 			if (cmd & 0x20) cp_size |= (*data++ << 8);
+			if (cmd & 0x40) cp_size |= (*data++ << 16);
 			if (cp_size == 0) cp_size = 0x10000;
 
-			if (cmd & 0x40)
-				/* copy from dst */
-				;
-			else
-				copied_from_source += cp_size;
+			copied_from_source += cp_size;
 			out += cp_size;
 		} else {
 			/* write literal into dst */
-- 
1.1.6.gce16

Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3.

[Nicolas Pitre]

On Thu, 9 Feb 2006, Junio C Hamano wrote:

> Junio C Hamano <junkio@cox.net> writes:
> 
> > I agree with the general direction and this futureproofing is a
> > good thing to have before 1.2.0, I think.
> >
> > The bit is however _already_ looked at by the count_delta(),
> > to assess the extent of damage, IIRC.  Should we be
> > futureproofing that bit as well?
> 
> Something like this?

Right.


Nicolas

Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3.

[Nicolas Pitre]

On Thu, 9 Feb 2006, Nicolas Pitre wrote:

> On Thu, 9 Feb 2006, Junio C Hamano wrote:
> 
> > Junio C Hamano <junkio@cox.net> writes:
> > 
> > > The bit is however _already_ looked at by the count_delta(),
> > > to assess the extent of damage, IIRC.  Should we be
> > > futureproofing that bit as well?
> > 
> > Something like this?
> 
> Right.

I'm not sure to fully understand what you meant in the comment below. 
but if it was related to the removed code, maybe this patch would make 
sense:

diff --git a/count-delta.c b/count-delta.c
index 978a60c..058a2aa 100644
--- a/count-delta.c
+++ b/count-delta.c
@@ -16,11 +16,7 @@
  *
  * Number of bytes that are _not_ copied from the source is deletion,
  * and number of inserted literal bytes are addition, so sum of them
- * is the extent of damage.  xdelta can express an edit that copies
- * data inside of the destination which originally came from the
- * source.  We do not count that in the following routine, so we are
- * undercounting the source material that remains in the final output
- * that way.
+ * is the extent of damage.
  */
 int count_delta(void *delta_buf, unsigned long delta_size,
 		unsigned long *src_copied, unsigned long *literal_added)

Re: [PATCH] count-delta.c: Match the delta data semantics change in version 3.

[Junio C Hamano]

You are right about that comment.  Thanks.