Re: Problems with large compressed binaries when converting from svn

[Johan Herland <johan@herland.net>]

On Tuesday 06 January 2009, Øyvind Harboe wrote:
> I'm converting from svn and I've run into a
> problem with tar.gz and tar.bz2 compressed files.
>
> (This is a separate but only slightly related to previous post).
>
> In subversion we committed large tar.bz2/gz files. These files would
> change relatively rarely, but only very slightly.  The trouble with the
> tar.bz2 format is that if the first byte changes, then the rest of the
> file will also be different. .zip does not have this problem, but .zip
> isn't a very friendly format for our purposes.
>
> Later on the tar.bz2/gz files started to change fairly often, but
> harddrives get bigger much more quickly than the .svn repository grows so
> we just kept doing things the same way rather than reeducate and
> reengineer the procedures.
>
> With .git we need to handle this differently somehow.
>
> Does git have some capability to store diffs of compressed files
> efficiently?
>
> The only other alternative I can think of is to commit uncompressed
> .tar files which is a bit of a bump in the road, but I suppose could be
> made to work.

Git can automate this for you. Take a look at the gitattributes(5) man page, 
specifically the "filter" attribute. You should be able to set up filter 
drivers for .tar.gz files that use "clean=gunzip" and "smudge=gzip" (and a 
similar filter driver for .tar.bz2 files).

If I've understood this right (I haven't used this myself) your checkouts 
should now have .tar.gz and .tar.bz2 files, even though Git only 
stores .tar files internally (thus improving compression across versions 
dramatically).


Have fun! :)

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net