Re: thin packs ending up fat

[Nicolas Pitre <nico@fluxnic.net>]

On Thu, 12 Jan 2012, Junio C Hamano wrote:

> From: Jeff King <peff@peff.net>
> Subject: [PATCH] thin-pack: try harder to create delta against preferred base
> 
> When creating a pack using objects that reside in existing packs, we try
> to avoid recomputing futile delta between an object (trg) and a candidate
> for its base object (src) if they are stored in the same packfile, and trg
> is not recorded as a delta already. This heuristics makes sense because it
> is likely that we tried to express trg as a delta based on src but it did
> not produce a good delta when we created the existing pack.
> 
> As the pack heuristics prefer producing delta to remove data, and Linus's
> law dictates that the size of a file grows over time, we tend to record
> the newest version of the file as inflated, and older ones as delta
> against it.
> 
> When creating a thin-pack to transfer recent history, it is likely that we
> will try to send an object that is recorded in full, as it is newer.  But
> the heuristics to avoid recomputing futile delta effectively forbids us
> from attempting to express such an object as a delta based on another
> object. Sending an object in full is often more expensive than sending a
> suboptimal delta based on other objects, and it is even more so if we
> could use an object we know the receiving end already has (i.e. referred
> base object) as the delta base.
> 
> Tweak the recomputation avoidance logic, so that we do not punt on
> computing delta against a preferred base object.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Acked-by: Nicolas Pitre <nico@fluxnic.net>

> ---
> 
>  builtin/pack-objects.c |    9 +++++++--
>  1 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index c6e2d87..8bfe3a6 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -1248,11 +1248,16 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
>  		return -1;
>  
>  	/*
> -	 * We do not bother to try a delta that we discarded
> -	 * on an earlier try, but only when reusing delta data.
> +	 * We do not bother to try a delta that we discarded on an
> +	 * earlier try, but only when reusing delta data.  Note that
> +	 * src_entry that is marked as the preferred_base should always
> +	 * be considered, as even if we produce a suboptimal delta against
> +	 * it, we will still save the transfer cost, as we already know
> +	 * the other side has it and we won't send src_entry at all.
>  	 */
>  	if (reuse_delta && trg_entry->in_pack &&
>  	    trg_entry->in_pack == src_entry->in_pack &&
> +	    !src_entry->preferred_base &&
>  	    trg_entry->in_pack_type != OBJ_REF_DELTA &&
>  	    trg_entry->in_pack_type != OBJ_OFS_DELTA)
>  		return 0;
> -- 
> 1.7.9.rc0.53.gc8bc2
>