git segfaults on older Solaris releases

["Tom G. Christensen" <tgc@jupiterrise.com>]

Hello,

While working on an update to the git packages in tgcware(1) I ran into 
segfaults when running the testsuite.

Here's what it looks like on Solaris 7/SPARC:

Core was generated by 
`/export/home/tgc/buildpkg/git/src/git-upstream/git update-index 
should-be-empty'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xfee81ef4 in _doprnt () from /usr/lib/libc.so.1
(gdb) bt
#0  0xfee81ef4 in _doprnt () from /usr/lib/libc.so.1
#1  0xfee83ce4 in vsnprintf () from /usr/lib/libc.so.1
#2  0x00138dbc in strbuf_vaddf (sb=0xffbedd24, fmt=0x1af7b8 "%.*s%s", 
ap=0xffbedde0) at strbuf.c:279
#3  0x00139f78 in xstrvfmt (fmt=0x1af7b8 "%.*s%s", ap=0xffbedde0) at 
strbuf.c:698
#4  0x00139fb4 in xstrfmt (fmt=0x1af7b8 "%.*s%s") at strbuf.c:708
#5  0x0012a0ec in prefix_path_gently (prefix=0x0, len=0, 
remaining_prefix=0x0, path=<optimized out>) at setup.c:103
#6  0x0012a2f0 in prefix_path (prefix=0x0, len=0, path=0xffbee7fc 
"should-be-empty") at setup.c:116
#7  0x00098464 in cmd_update_index (argc=2, argv=<optimized out>, 
prefix=0x0) at builtin/update-index.c:1042
#8  0x00025900 in run_builtin (argv=0xffbee630, argc=2, p=0x1c9adc 
<commands+1260>) at git.c:346
#9  handle_builtin (argc=2, argv=0xffbee630) at git.c:536
#10 0x00025bec in run_argv (argv=0xffbee5c4, argcp=0xffbee60c) at git.c:582
#11 main (argc=2, av=<optimized out>) at git.c:690
(gdb)


The reason for the crash is simple, a null value was passed to the 's' 
format for the *printf family of functions.
To verify I modified git.c:run_builtin() so it would assign "" to prefix 
if NULL just before the status = p->fn(..) call.
This allowed t0000-basic.sh to pass where before it would fail because 
git segfaulted in multiple tests.

Passing a null value to the 's' format is explicitly documented as 
giving undefined results on Solaris, even on Solaris 11(2).
It happens that Solaris 8 and later will tolerate this without crashing, 
though I suspect at least for Solaris 8 and 9 it might require a certain 
patchlevel to do so. Earlier releases will just segfault as shown above.

I bisected it on Solaris 2.6 and found that 75faa45 was the commit that 
caused this problem to appear. The 2.6.x releases build and run fine.

I know of course that Solaris < 8 is not terribly interesting as a 
portability target so I understand if you're unwilling to fix this as it 
seems it might be a somewhat invasive change.

-tgc

1) http://jupiterrise.com/tgcware/tgcware.solaris.html
2) http://docs.oracle.com/cd/E23824_01/html/821-1465/printf-3c.html