Re: [PATCHv19 00/11] Expose submodule parallelism to the user

[Johannes Sixt <j6t@kdbg.org>]

Hi folks,

we have a major breakage in the parallel tasks infrastructure, and I'm
afraid it is already in master.

Instrument the code in sb/submodule-parallel-update like this and enjoy
the fireworks of './t7400-submodule-basic.sh -v -i -x --debug':

diff --git a/git-submodule.sh b/git-submodule.sh
index 0322282..482c7f6 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -690,8 +690,9 @@ cmd_update()
 		cmd_init "--" "$@" || return
 	fi
 
+	set -x
 	{
-	git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+	valgrind git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
 		${wt_prefix:+--prefix "$wt_prefix"} \
 		${prefix:+--recursive-prefix "$prefix"} \
 		${update:+--update "$update"} \
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5572327..717e491 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -337,6 +337,7 @@ test_expect_success 'update should fail when path is used by a file' '
 
 	echo "hello" >init &&
 	test_must_fail git submodule update &&
+	false &&
 
 	test_cmp expect init
 '

The culprit seems to be default_task_finished(), which accesses argv[]
of the struct child_process after finish_command has released it,
provided the child exited with an error, for example:

==3395== Invalid read of size 8
==3395==    at 0x54F991: default_task_finished (run-command.c:932)
==3395==    by 0x49158F: update_clone_task_finished (submodule--helper.c:421)
==3395==    by 0x5504A2: pp_collect_finished (run-command.c:1122)
==3395==    by 0x5507C7: run_processes_parallel (run-command.c:1194)
==3395==    by 0x4918EB: update_clone (submodule--helper.c:483)
==3395==    by 0x4919D8: cmd_submodule__helper (submodule--helper.c:527)
==3395==    by 0x405CBE: run_builtin (git.c:353)
==3395==    by 0x405EAA: handle_builtin (git.c:540)
==3395==    by 0x405FCC: run_argv (git.c:594)
==3395==    by 0x4061BF: main (git.c:701)
==3395==  Address 0x5e49370 is 0 bytes inside a block of size 192 free'd
==3395==    at 0x4C2A37C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==3395==    by 0x4A26EE: argv_array_clear (argv-array.c:73)
==3395==    by 0x54DFC4: child_process_clear (run-command.c:18)
==3395==    by 0x54EFA7: finish_command (run-command.c:539)
==3395==    by 0x550413: pp_collect_finished (run-command.c:1120)
==3395==    by 0x5507C7: run_processes_parallel (run-command.c:1194)
==3395==    by 0x4918EB: update_clone (submodule--helper.c:483)
==3395==    by 0x4919D8: cmd_submodule__helper (submodule--helper.c:527)
==3395==    by 0x405CBE: run_builtin (git.c:353)
==3395==    by 0x405EAA: handle_builtin (git.c:540)
==3395==    by 0x405FCC: run_argv (git.c:594)
==3395==    by 0x4061BF: main (git.c:701)

I haven't thought about a solution, yet. Perhaps you have ideas.

-- Hannes