Reverting a swath of commits consumes all memory and dies.

Reverting a swath of commits consumes all memory and dies.

[Nick Bowler]

Hi folks,

I just noticed that git-revert when given a large batch of commits will
consume more and more memory as commits are reverted.  Eventually, git's
usage exceeds about half of the available memory and dies because fork
fails with -ENOMEM.  This appears to be gratuitous, because after git
dies you can simply run

  git reset --hard
  git revert --continue

and the revert will pick up right where it left off.  Reverting commits
one at a time also works.  I can reproduce it (for example) with the
following example, using current master git:

  % git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  % cd linux
  % git revert --no-merges --no-edit HEAD~50..

watch git's memory usage increase indefinitely with "top", until...

  error: cannot fork() for commit: Cannot allocate memory
  fatal: revert failed

  % git reset --hard
  % git revert --no-edit --continue

repeat as needed until eventually there's few enough commits left that
the revert completes without running out of memory (or git-revert fails
for some other reason).

Cheers,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Reverting a swath of commits consumes all memory and dies.

[Junio C Hamano]

Nick Bowler <nbowler@elliptictech.com> writes:

> I just noticed that git-revert when given a large batch of commits will
> consume more and more memory as commits are reverted.

[Cc: people who have been involved in builtin/revert.c and sequencer.c]

Unfortunate, but not surprised.

Unlike other sequencing operations like 'rebase' and 'am', multi-commit
cherry-pick/revert were implemented in such a way that it calls into the
infrastructure functions that was meant to be used in "get invoked as a
single command, do that one thing well, and let exit(2) take care of
cleaning up the resources" repeatedly from within a single process.
Probably the series that implemented multi-commit cherry-pick/revert
should have first updated the infrastructure to reduce leaks, but
probably it looked shinier to add features than doing the necessary
legwork ;-)

Perhaps you (or anybody for that matter) can help by running them under
valgrind, which would be a good first step to find and fix the leaks.

Thanks.