Re: [PATCH v2 7/8] submodule-config: pass repo upon blob config read

[Matheus Tavares Bernardino <matheus.bernardino@usp.br>]

On Fri, Aug 13, 2021 at 6:05 PM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> When reading the config of a submodule, if reading from a blob, read
> using an explicitly specified repository instead of by adding the
> submodule's ODB as an alternate and then reading an object from
> the_repository.

Great!

At first, I thought this would also allow us to remove another
NEEDSWORK comment in grep_submodule(), together with a lock
protection:

/*
 * NEEDSWORK: repo_read_gitmodules() might call
 * add_to_alternates_memory() via config_from_gitmodules(). This
 * operation causes a race condition with concurrent object readings
 * performed by the worker threads. That's why we need obj_read_lock()
 * here. It should be removed once it's no longer necessary to add the
 * subrepo's odbs to the in-memory alternates list.
 */
obj_read_lock();
repo_read_gitmodules(subrepo, 0);

Back when I wrote this comment, my conclusion was that the alternates
mechanics were the only thread-unsafe object-reading operations in
repo_read_gitmodules()'s call chains. So once the add-to-alternates
mechanics were gone, we could also remove the lock.

But with further inspection now, I see that this is not really the
case. For example, we have a few global variables in packfile.c
collecting some statistics (pack_mmap_calls, pack_open_windows, etc.)
which are updated on obj readings from both the_repository *and*
submodules. So I no longer think its safe to remove the
obj_read_lock() protection here, as the NEEDSWORK comment suggests,
even if we are not using the alternates list anymore.

Do you want to remove this comment in your patchset? I can also send a
follow-up patch explaining this situation and removing the comment
(but not the locking), if you prefer.

> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
[...]
> diff --git a/submodule-config.c b/submodule-config.c
> index 2026120fb3..f95344028b 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -649,9 +649,10 @@ static void config_from_gitmodules(config_fn_t fn, struct repository *repo, void
>                         config_source.file = file;
>                 } else if (repo_get_oid(repo, GITMODULES_INDEX, &oid) >= 0 ||
>                            repo_get_oid(repo, GITMODULES_HEAD, &oid) >= 0) {
> +                       config_source.repo = repo;
>                         config_source.blob = oidstr = xstrdup(oid_to_hex(&oid));
>                         if (repo != the_repository)
> -                               add_to_alternates_memory(repo->objects->odb->path);
> +                               add_submodule_odb_by_path(repo->objects->odb->path);

Ok. Like in grep_submodule(), this should no longer add the submodule
ODB to the alternates list, so this call is now mostly used as a
fallback and also for testing.

To see if we are indeed testing this add-to-alternates case, I
reverted the change that made the code read from the submodule instead
of the_repository:

diff --git a/config.c b/config.c
index a85c12e6cc..cd37a9dcd9 100644
--- a/config.c
+++ b/config.c
@@ -1805,7 +1805,7 @@ int git_config_from_blob_oid(config_fn_t fn,
        unsigned long size;
        int ret;

-       buf = repo_read_object_file(repo, oid, &type, &size);
+       buf = read_object_file(oid, &type, &size);

Then, I ran t7814-grep-recurse-submodules.sh , where you've added the
GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB=1 envvar. This correctly
produced the following error:

BUG: submodule.c:205: register_all_submodule_odb_as_alternates() called
[...]
not ok 23 - grep --recurse-submodules with submodules without
.gitmodules in the working tree

Nice! So the change made by this patch is covered by test 23. I think
it would be nice to mention that in this patch's message.