Disabling Delta Compression on a fetch

Disabling Delta Compression on a fetch

[Todd Rinaldo]

We run an internal gitorious server with many repos on it. Due to legacy issues, some of the repos are rather bloated with large binaries. When several changes happen and are pushed to the gitorious repo and then are later pulled back (fetch and pull) to other clones, we get a broken connection while the gitorious side is generating Delta compression. 

All of the git communication happens on 1 subnet all connected by a single gigabit switch. As I see it, the Delta Compression is actually a performance degradation in our environment.

The solution I've come up with is to set pack.window=0 in /etc/gitconfig on the gitorious server. 

I realize our need is uncommon. I realize by doing this, I am trading increased bandwidth in exchange for starting the send sooner. When I read the documentation for pack.window, it seems more a side effect than the original intent that this variable disables Delta Compression when set to 0.

My question is: Are there are any unintended consequences of this approach anyone can think of? 

Thanks,
Todd Rinaldo

Re: Disabling Delta Compression on a fetch

[Junio C Hamano]

Todd Rinaldo <toddr@cpanel.net> writes:

> The solution I've come up with is to set pack.window=0 in /etc/gitconfig on the gitorious server. 

Wouldn't setting core.bigfilethreshold be a more appropriate solution?

Re: Disabling Delta Compression on a fetch

[Jeff King]

On Thu, Dec 08, 2011 at 11:34:26AM -0600, Todd Rinaldo wrote:

> All of the git communication happens on 1 subnet all connected by a
> single gigabit switch. As I see it, the Delta Compression is actually
> a performance degradation in our environment.
> 
> The solution I've come up with is to set pack.window=0 in
> /etc/gitconfig on the gitorious server. 

An alternative is to mark the binary files as "-delta" with
gitattributes on the server. Then you will get the benefits of delta
compression for other files without bothering to try the binary files.

Note that git won't read the gitattributes file out of the tree in a
bare repository, so these attributes should go either in
$REPO/info/attributes (if they are repo-specific), or in
/etc/gitattributes (if they are in many repos).

I.e., something like:

  echo '*.bin -delta' >/etc/gitattributes

Also, before any of that, make sure that the upstream repos are fully
packed. Git will not try to delta two objects coming from the same pack
(since it will already have tried when they were entering the pack).
That by itself might be enough to solve your problem without any other
configuration.

> My question is: Are there are any unintended consequences of this
> approach anyone can think of?

Other than trading bandwidth, you are also trading space on the client
side, since each client will store the resulting pack.

-Peff