Bug report: parallel fetch tries to unlock SSH keys over and over again

Bug report: parallel fetch tries to unlock SSH keys over and over again

[Perez-Lopez Áron Ricardo]

What did you do before the bug happened? (Steps to reproduce your issue)
git fetch --all --tags --prune --jobs=10

What did you expect to happen? (Expected behavior)
Being asked for the password of my SSH key, then git updating all of
the remotes.

What happened instead? (Actual behavior)
I get asked for my SSH key many times over, and the fetch does not succeed.

What's different between what you expected and what actually happened?
I should only be asked to unlock my SSH key once. All fetch jobs
should wait for this and then use the unlocked (cached) key.

Anything else you want to add:
If I do just `git fetch` and unlock my key (once), it does get added
to the SSH agent properly, and afterwards I can execute the command
above with no issues.

[System Info]
git version:
git version 2.48.1
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
libcurl: 8.11.1
OpenSSL: OpenSSL 3.4.0 22 Oct 2024
zlib: 1.3.1
uname: Linux 6.12.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 18 Jan 2025
02:26:57 +0000 x86_64
compiler info: gnuc: 14.2
libc info: glibc: 2.40
$SHELL (typically, interactive shell): /bin/bash

[Enabled Hooks]

RE: Bug report: parallel fetch tries to unlock SSH keys over and over again

[rsbecker]

On January 24, 2025 2:47 PM, Perez-Lopez Áron Ricardo wrote:
>What did you do before the bug happened? (Steps to reproduce your issue) git
>fetch --all --tags --prune --jobs=10
>
>What did you expect to happen? (Expected behavior) Being asked for the password
>of my SSH key, then git updating all of the remotes.
>
>What happened instead? (Actual behavior) I get asked for my SSH key many times
>over, and the fetch does not succeed.
>
>What's different between what you expected and what actually happened?
>I should only be asked to unlock my SSH key once. All fetch jobs should wait for this
>and then use the unlocked (cached) key.
>
>Anything else you want to add:
>If I do just `git fetch` and unlock my key (once), it does get added to the SSH agent
>properly, and afterwards I can execute the command above with no issues.
>
>[System Info]
>git version:
>git version 2.48.1
>cpu: x86_64
>no commit associated with this build
>sizeof-long: 8
>sizeof-size_t: 8
>shell-path: /bin/sh
>libcurl: 8.11.1
>OpenSSL: OpenSSL 3.4.0 22 Oct 2024
>zlib: 1.3.1
>uname: Linux 6.12.10-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 18 Jan 2025
>02:26:57 +0000 x86_64
>compiler info: gnuc: 14.2
>libc info: glibc: 2.40
>$SHELL (typically, interactive shell): /bin/bash
>
>[Enabled Hooks]

Add your SSH key to your ssh-agent. The agent will prompt for your passphrase.
Git will not do that. Then perform your git fetch operations. You will not be
Prompted by git. If you are using some other credential manager, you will need
To add your key separately.

--Randall

Re: Bug report: parallel fetch tries to unlock SSH keys over and over again

[Junio C Hamano]

Perez-Lopez Áron Ricardo <perez.aron@gmail.com> writes:

> What did you do before the bug happened? (Steps to reproduce your issue)
> git fetch --all --tags --prune --jobs=10
>
> What did you expect to happen? (Expected behavior)
> Being asked for the password of my SSH key, then git updating all of
> the remotes.
>
> What happened instead? (Actual behavior)
> I get asked for my SSH key many times over, and the fetch does not succeed.

It is hard to tell without seeing how many repositories the above
"--all" expands to (but I'd assume it is way more than 10), if they
share the same credentials or going to different site that requires
different SSH keys but these keys share the same passphrase, etc.

Assuming that they all go to the same host that wants the same SSH
keys, and assuming that you are using ssh agent, if the symptom were
the fetch *does* succeed, but being asked for the passphrase to
unlock the same key number of times, then I can sort of see why it
would be problematic and why it might help to wait for the first
connection to successfully open before starting the other 9 jobs as
you instructed "git fetch" to spawn.

But if the symptom is that the fetch does *not* succeed [*], a
simpler explanation of the symptom may be that the passphrase given
was simply misspelt and/or there is no ssh agent involved that loads
the passphrases and caches them on demand.

FWIW, I push over openssh connection to many sites, but as far as I
know, I need to say "ssh-add" to tell the agent what keys should be
cached and what their passphrases are *well* *before* I initiate
anything that requires ssh connections like "git push".

Sorry that my response is not an answer, but I have never dealt with
a SSH setting where the AddKeysToAgent directive is active.  Others
may have better ideas, but I wonder what good would "wait for the
first one" do, as a general "fix".  Git does not even know what
relationship the underlying SSH connections to these other 9
repositories would have to the connection to the site that hosts the
first repository, and the first one might happen to be hosted at a
site using a completely unrelated SSH key from the rest, in which
case it simply is wasting time to make the other 9 wait.


[footnote]

 * ... and since the command line is using "--all", what does it
   really mean that it does *not* succeed?  Some fetches but some
   others don't?

Re: Bug report: parallel fetch tries to unlock SSH keys over and over again

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

On 2025-01-24 at 19:46:58, Perez-Lopez Áron Ricardo wrote:
> What did you do before the bug happened? (Steps to reproduce your issue)
> git fetch --all --tags --prune --jobs=10
> 
> What did you expect to happen? (Expected behavior)
> Being asked for the password of my SSH key, then git updating all of
> the remotes.
> 
> What happened instead? (Actual behavior)
> I get asked for my SSH key many times over, and the fetch does not succeed.
> 
> What's different between what you expected and what actually happened?
> I should only be asked to unlock my SSH key once. All fetch jobs
> should wait for this and then use the unlocked (cached) key.
> 
> Anything else you want to add:
> If I do just `git fetch` and unlock my key (once), it does get added
> to the SSH agent properly, and afterwards I can execute the command
> above with no issues.

While this would be nice, Git doesn't actually prompt your for your SSH
passphrase, so it has no knowledge as to whether a passphrase is needed
or not.  This is usually done either by your SSH client (such as
OpenSSH), or by some tool which functions as an SSH agent (such as a
system keychain).

Git, in fact, couldn't even just query the agent to see if the key is
there because it's possible in some cases to make the agent require a
password for every usage or after a certain amount of time, so even if
the agent has the key, that doesn't mean that it is or isn't accessible
without a password.

The only way that Git does anything related to the password is to pass
the appropriate file descriptors to the SSH process so that the SSH
client can prompt on the terminal if it so chooses.  So regretfully,
there's no change we can make in Git that will address this in a useful
and general way.  As a workaround, you can explicitly add your SSH key
to the agent, or you can use something like a password manager as your
agent (I use 1Password, but there are others) that will prompt you only
once for all uses, which will avoid the competing prompts.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]