Re: Invalid memory access in `git apply`

["René Scharfe" <l.s.r@web.de>]

Am 08.11.2017 um 17:58 schrieb mqudsi@neosmart.net:
> **Resending as it seems that the attachments caused the last email to wind up
> in a black hole**
> 
> There seems to be bug in the `git apply` that leads to out-of-bounds memory
> access when --ignore-space-change is combined with --inaccurate-eof and
> applying a patch.
> 
> On occasion, this can lead to error output like the following:
> 
> 	 mqudsi@ZBook ~> git apply --ignore-space-change --ignore-whitespace
> 	 --allow-overlap --inaccurate-eof without_whitespace.diff
> 	 *** Error in `git': malloc(): memory corruption: 0x0000000002543530 ***
> 	 ======= Backtrace: =========
> 	 /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fdda79c77e5]
> 	 /lib/x86_64-linux-gnu/libc.so.6(+0x8213e)[0x7fdda79d213e]
> 	 /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fdda79d4184]
> 	 /lib/x86_64-linux-gnu/libc.so.6(_IO_file_doallocate+0x55)[0x7fdda79bd1d5]
> 	 /lib/x86_64-linux-gnu/libc.so.6(_IO_doallocbuf+0x34)[0x7fdda79cb594]
> 	 /lib/x86_64-linux-gnu/libc.so.6(_IO_file_overflow+0x1c8)[0x7fdda79ca8f8]
> 	 /lib/x86_64-linux-gnu/libc.so.6(_IO_file_xsputn+0xad)[0x7fdda79c928d]
> 	 /lib/x86_64-linux-gnu/libc.so.6(fputs+0x98)[0x7fdda79be0c8]
> 	 git[0x5386cd]
> 	 git[0x538714]
> 	 git[0x538940]
> 	 git[0x40e220]
> 	 git[0x410a10]
> 	 git[0x41256e]
> 	 git[0x412df7]
> 	 git[0x415935]
> 	 git[0x406436]
> 	 git[0x40555c]
> 
> The original file being patched (clipboard.vim) and the patch file that I had
> attempted to apply (without_whitespace.diff) are attached, along with the
> full, unabridged output of the memory map as a result of the out-of-bounds
> access (memory_map.txt).
> 
> The memory map output was generated under git 2.7.4; repeated attempts to
> reproduce the memory map dump with both 2.7.4 and 2.15 produce the following
> output:
> 
> 	 mqudsi@ZBook ~/.c/nvim> git apply --ignore-space-change  --inaccurate-eof
> 	 --whitespace=fix without_whitespace.diff
> 	 fatal: BUG: caller miscounted postlen: asked 248, orig = 251, used = 249
> 
> Mahmoud Al-Qudsi
> NeoSmart Technologies
> 
> --Attachments--
> 
> * clipboard.vim: http://termbin.com/u25t
> * without_whitespace.diff: http://termbin.com/bu9y
> * memory_map.txt: http://termbin.com/cboz

Thank you for reporting the issue!

There seem to be at least two bugs in git apply and two problems on your
end.  You don't seem need the option --inaccurate-eof and it's causing
trouble for you; I suggest to leave it out.

And the second hunk of your diff doesn't apply because the "<TAB>endif"
context line doesn't match the "endif" line in clipboard.vim which has
no leading whitespace.  --ignore-space-change ignores changes in the
number of whitespace characters, but that number cannot be 0 on only one
side.

If you adjust the diff by removing the tab from that context line or add
one or more spaces in clipboard.vim before the last "endif" then it will
apply without --inaccurate-eof.


One of the bugs is that fuzzy_matchlines() does out-of-bounds reads in
some cases.  You should only notice it with a tool like Valgrind, ASan
or perhaps a hardened malloc(3).  I'll send a separate patch for that.

The second bug is that --inaccurate-eof triggers a sanity check when
used together with --ignore-space-change.  Here's a simpler reproduction
recipe:

  git init repo
  cd repo

  echo 1 >a
  git add a
  git commit -m initial

  echo 2 >a
  git diff >a.diff
	
  git reset --hard
  git apply --ignore-space-change --inaccurate-eof a.diff

Which yields this error message:

  fatal: BUG: caller miscounted postlen: asked 1, orig = 1, used = 2

Perhaps the first thing we'd need would be a couple of tests showing
the expected behavior of git apply --inaccurate-eof with and without
trailing newlines..

René