Why does diff --binary include content of files being deleted?

Why does diff --binary include content of files being deleted?

[Allan Wind]

Between tag1 and tag2 I am deleting a number of binary files, and I was 
surprised seeing the binaries being added to the diff when said files
are to be deleted:

git diff --binary tag1 tag2 > /tmp/diff && ls -l /tmp/diff
-rw-r--r-- 1 allan users 555135993 2008-06-21 22:37 /tmp/diff

git diff tag1 tag2 > /tmp/diff2 && ls -l /tmp/diff2
-rw-r--r-- 1 allan users 12862 2008-06-21 23:33 /tmp/diff2



/Allan

Re: Why does diff --binary include content of files being deleted?

[Shawn O. Pearce]

Allan Wind <allan_wind@lifeintegrity.com> wrote:
> Between tag1 and tag2 I am deleting a number of binary files, and I was 
> surprised seeing the binaries being added to the diff when said files
> are to be deleted:
> 
> git diff --binary tag1 tag2 > /tmp/diff && ls -l /tmp/diff
> -rw-r--r-- 1 allan users 555135993 2008-06-21 22:37 /tmp/diff
> 
> git diff tag1 tag2 > /tmp/diff2 && ls -l /tmp/diff2
> -rw-r--r-- 1 allan users 12862 2008-06-21 23:33 /tmp/diff2

In order to apply the diff in reverse we need the old binary data
in the diff output.  So that's why its larger.

In other words lets say you actually wanted to go back to tag1,
after having tag2 for a while:

	$ git checkout tag2
	$ git diff --binary tag1 tag2 | git apply --index -R

without the binary data in the diff, apply cannot run in the
reverse direction to restore back to tag1.  :)

-- 
Shawn.

Re: Why does diff --binary include content of files being deleted?

[Allan Wind]

On 2008-06-21T23:51:18-0400, Shawn O. Pearce wrote:
> In order to apply the diff in reverse we need the old binary data
> in the diff output.  So that's why its larger.

Makes sense, thanks.  In my use case (where I do not need the reverse 
diff) this adds 500+ MB or 43159% overhead.

When I tried to apply the large diff my virtual private server with 512 
MB of memory git apply ran out of memory and died.  Is there any way to 
bound the memory usage of git apply?


/Allan

Re: Why does diff --binary include content of files being deleted?

[Shawn O. Pearce]

Allan Wind <allan_wind@lifeintegrity.com> wrote:
> On 2008-06-21T23:51:18-0400, Shawn O. Pearce wrote:
> > In order to apply the diff in reverse we need the old binary data
> > in the diff output.  So that's why its larger.
> 
> Makes sense, thanks.  In my use case (where I do not need the reverse 
> diff) this adds 500+ MB or 43159% overhead.

Well, then in that case you have to omit --binary I think, or use a
filter script to strip out the binary data from the resulting patch.

> When I tried to apply the large diff my virtual private server with 512 
> MB of memory git apply ran out of memory and died.  Is there any way to 
> bound the memory usage of git apply?

If you submit patches for it.  ;-)

git-apply today is meant for applying small patches, like those that
would typically be written by a developer hacking on open source
software.  It tries to be completely atomic; either the entire
patch applies at once, or the entire patch rejects with no changes
being made to the local directory.  As such it applies everything in
memory, and only writes out when we are sure the patch applies clean.

So no, there isn't a way to make git-apply use less memory.

-- 
Shawn.