[PATCH v2] credential: clear expired c->credential, unify secret clearing

[PATCH v2] credential: clear expired c->credential, unify secret clearing

[Aaron Plattner]

When a struct credential expires, credential_fill() clears c->password
so that clients don't try to use it later. However, a struct cred that
uses an alternate authtype won't have a password, but might have a
credential stored in c->credential.

This is a problem, for example, when an OAuth2 bearer token is used. In
the system I'm using, the OAuth2 configuration generates and caches a
bearer token that is valid for an hour. After the token expires, git
needs to call back into the credential helper to use a stored refresh
token to get a new bearer token. But if c->credential is still non-NULL,
git will instead try to use the expired token and fail with an error:

 fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'

And on the server:

 [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago

Fix this by clearing both c->password and c->credential for an expired
struct credential. While we're at it, use credential_clear_secrets()
wherever both c->password and c->credential are being cleared, and use
the full credential_clear() in credential_reject() after the credential
has been erased from all of the helpers.

v2: Unify secret clearing into credential_clear_secrets(), use
credential_clear() in credential_reject(), add a comment about why we
can't use credential_clear() in credential_fill().

Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
---
 credential.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/credential.c b/credential.c
index 758528b291..72c6f46b02 100644
--- a/credential.c
+++ b/credential.c
@@ -20,12 +20,11 @@ void credential_init(struct credential *c)
 
 void credential_clear(struct credential *c)
 {
+	credential_clear_secrets(c);
 	free(c->protocol);
 	free(c->host);
 	free(c->path);
 	free(c->username);
-	free(c->password);
-	free(c->credential);
 	free(c->oauth_refresh_token);
 	free(c->authtype);
 	string_list_clear(&c->helpers, 0);
@@ -479,9 +478,14 @@ void credential_fill(struct credential *c, int all_capabilities)
 
 	for (i = 0; i < c->helpers.nr; i++) {
 		credential_do(c, c->helpers.items[i].string, "get");
+
 		if (c->password_expiry_utc < time(NULL)) {
-			/* Discard expired password */
-			FREE_AND_NULL(c->password);
+			/*
+			 * Don't use credential_clear() here: callers such as
+			 * cmd_credential() expect to still be able to call
+			 * credential_write() on a struct credential whose secrets have expired.
+			 */
+			credential_clear_secrets(c);
 			/* Reset expiry to maintain consistency */
 			c->password_expiry_utc = TIME_MAX;
 		}
@@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
 	for (i = 0; i < c->helpers.nr; i++)
 		credential_do(c, c->helpers.items[i].string, "erase");
 
-	FREE_AND_NULL(c->username);
-	FREE_AND_NULL(c->password);
-	FREE_AND_NULL(c->credential);
-	FREE_AND_NULL(c->oauth_refresh_token);
-	c->password_expiry_utc = TIME_MAX;
-	c->approved = 0;
+	credential_clear(c);
 }
 
 static int check_url_component(const char *url, int quiet,
-- 
2.45.2.410.gb47f57dd90.dirty

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Junio C Hamano]

Brian, on top of your topic that was merged last month c5c9acf7
(Merge branch 'bc/credential-scheme-enhancement', 2024-05-08), do
these changes make sense to you as a fix/clean-up?

Thanks.

Aaron Plattner <aplattner@nvidia.com> writes:

> When a struct credential expires, credential_fill() clears c->password
> so that clients don't try to use it later. However, a struct cred that
> uses an alternate authtype won't have a password, but might have a
> credential stored in c->credential.
>
> This is a problem, for example, when an OAuth2 bearer token is used. In
> the system I'm using, the OAuth2 configuration generates and caches a
> bearer token that is valid for an hour. After the token expires, git
> needs to call back into the credential helper to use a stored refresh
> token to get a new bearer token. But if c->credential is still non-NULL,
> git will instead try to use the expired token and fail with an error:
>
>  fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'
>
> And on the server:
>
>  [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago
>
> Fix this by clearing both c->password and c->credential for an expired
> struct credential. While we're at it, use credential_clear_secrets()
> wherever both c->password and c->credential are being cleared, and use
> the full credential_clear() in credential_reject() after the credential
> has been erased from all of the helpers.
>
> v2: Unify secret clearing into credential_clear_secrets(), use
> credential_clear() in credential_reject(), add a comment about why we
> can't use credential_clear() in credential_fill().
>
> Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
> ---
>  credential.c | 19 +++++++++----------
>  1 file changed, 9 insertions(+), 10 deletions(-)
>
> diff --git a/credential.c b/credential.c
> index 758528b291..72c6f46b02 100644
> --- a/credential.c
> +++ b/credential.c
> @@ -20,12 +20,11 @@ void credential_init(struct credential *c)
>  
>  void credential_clear(struct credential *c)
>  {
> +	credential_clear_secrets(c);
>  	free(c->protocol);
>  	free(c->host);
>  	free(c->path);
>  	free(c->username);
> -	free(c->password);
> -	free(c->credential);
>  	free(c->oauth_refresh_token);
>  	free(c->authtype);
>  	string_list_clear(&c->helpers, 0);
> @@ -479,9 +478,14 @@ void credential_fill(struct credential *c, int all_capabilities)
>  
>  	for (i = 0; i < c->helpers.nr; i++) {
>  		credential_do(c, c->helpers.items[i].string, "get");
> +
>  		if (c->password_expiry_utc < time(NULL)) {
> -			/* Discard expired password */
> -			FREE_AND_NULL(c->password);
> +			/*
> +			 * Don't use credential_clear() here: callers such as
> +			 * cmd_credential() expect to still be able to call
> +			 * credential_write() on a struct credential whose secrets have expired.
> +			 */
> +			credential_clear_secrets(c);
>  			/* Reset expiry to maintain consistency */
>  			c->password_expiry_utc = TIME_MAX;
>  		}
> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>  	for (i = 0; i < c->helpers.nr; i++)
>  		credential_do(c, c->helpers.items[i].string, "erase");
>  
> -	FREE_AND_NULL(c->username);
> -	FREE_AND_NULL(c->password);
> -	FREE_AND_NULL(c->credential);
> -	FREE_AND_NULL(c->oauth_refresh_token);
> -	c->password_expiry_utc = TIME_MAX;
> -	c->approved = 0;
> +	credential_clear(c);
>  }
>  
>  static int check_url_component(const char *url, int quiet,

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

On 2024-06-04 at 19:29:28, Aaron Plattner wrote:
> When a struct credential expires, credential_fill() clears c->password
> so that clients don't try to use it later. However, a struct cred that
> uses an alternate authtype won't have a password, but might have a
> credential stored in c->credential.
> 
> This is a problem, for example, when an OAuth2 bearer token is used. In
> the system I'm using, the OAuth2 configuration generates and caches a
> bearer token that is valid for an hour. After the token expires, git
> needs to call back into the credential helper to use a stored refresh
> token to get a new bearer token. But if c->credential is still non-NULL,
> git will instead try to use the expired token and fail with an error:
> 
>  fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'
> 
> And on the server:
> 
>  [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago
> 
> Fix this by clearing both c->password and c->credential for an expired
> struct credential. While we're at it, use credential_clear_secrets()
> wherever both c->password and c->credential are being cleared, and use
> the full credential_clear() in credential_reject() after the credential
> has been erased from all of the helpers.

I think this is fine.  I'm assuming that the credential (and other
appurtenant information, such as the state[] values) are still passed to
the erase call, and if so, I don't see a problem.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Junio C Hamano]

Aaron Plattner <aplattner@nvidia.com> writes:

> When a struct credential expires, credential_fill() clears c->password
> so that clients don't try to use it later. However, a struct cred that
> uses an alternate authtype won't have a password, but might have a
> credential stored in c->credential.
>
> This is a problem, for example, when an OAuth2 bearer token is used. In
> the system I'm using, the OAuth2 configuration generates and caches a
> bearer token that is valid for an hour. After the token expires, git
> needs to call back into the credential helper to use a stored refresh
> token to get a new bearer token. But if c->credential is still non-NULL,
> git will instead try to use the expired token and fail with an error:
>
>  fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'
>
> And on the server:
>
>  [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago
>
> Fix this by clearing both c->password and c->credential for an expired
> struct credential. While we're at it, use credential_clear_secrets()
> wherever both c->password and c->credential are being cleared, and use
> the full credential_clear() in credential_reject() after the credential
> has been erased from all of the helpers.

OK.

>
> v2: Unify secret clearing into credential_clear_secrets(), use
> credential_clear() in credential_reject(), add a comment about why we
> can't use credential_clear() in credential_fill().

This does not belong to the commit log message proper.  Those who
are reading "git log" would not know (and care about) an earlier
attempt.  Writing the changes since the previous round(s) like the
above paragraph is very much appreciated as it helps reviewers who
saw them, but please do so after the three-dash "---" line below (I
can remove the paragraph while queueing, so no need to resend).

Will queue.  Thanks.


> Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
> ---
>  credential.c | 19 +++++++++----------
>  1 file changed, 9 insertions(+), 10 deletions(-)
>
> diff --git a/credential.c b/credential.c
> index 758528b291..72c6f46b02 100644
> --- a/credential.c
> +++ b/credential.c
> @@ -20,12 +20,11 @@ void credential_init(struct credential *c)
>  
>  void credential_clear(struct credential *c)
>  {
> +	credential_clear_secrets(c);
>  	free(c->protocol);
>  	free(c->host);
>  	free(c->path);
>  	free(c->username);
> -	free(c->password);
> -	free(c->credential);
>  	free(c->oauth_refresh_token);
>  	free(c->authtype);
>  	string_list_clear(&c->helpers, 0);
> @@ -479,9 +478,14 @@ void credential_fill(struct credential *c, int all_capabilities)
>  
>  	for (i = 0; i < c->helpers.nr; i++) {
>  		credential_do(c, c->helpers.items[i].string, "get");
> +
>  		if (c->password_expiry_utc < time(NULL)) {
> -			/* Discard expired password */
> -			FREE_AND_NULL(c->password);
> +			/*
> +			 * Don't use credential_clear() here: callers such as
> +			 * cmd_credential() expect to still be able to call
> +			 * credential_write() on a struct credential whose secrets have expired.
> +			 */
> +			credential_clear_secrets(c);
>  			/* Reset expiry to maintain consistency */
>  			c->password_expiry_utc = TIME_MAX;
>  		}
> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>  	for (i = 0; i < c->helpers.nr; i++)
>  		credential_do(c, c->helpers.items[i].string, "erase");
>  
> -	FREE_AND_NULL(c->username);
> -	FREE_AND_NULL(c->password);
> -	FREE_AND_NULL(c->credential);
> -	FREE_AND_NULL(c->oauth_refresh_token);
> -	c->password_expiry_utc = TIME_MAX;
> -	c->approved = 0;
> +	credential_clear(c);
>  }
>  
>  static int check_url_component(const char *url, int quiet,

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Aaron Plattner]

On 6/4/24 3:04 PM, Junio C Hamano wrote:
> Aaron Plattner <aplattner@nvidia.com> writes:
> 
>> When a struct credential expires, credential_fill() clears c->password
>> so that clients don't try to use it later. However, a struct cred that
>> uses an alternate authtype won't have a password, but might have a
>> credential stored in c->credential.
>>
>> This is a problem, for example, when an OAuth2 bearer token is used. In
>> the system I'm using, the OAuth2 configuration generates and caches a
>> bearer token that is valid for an hour. After the token expires, git
>> needs to call back into the credential helper to use a stored refresh
>> token to get a new bearer token. But if c->credential is still non-NULL,
>> git will instead try to use the expired token and fail with an error:
>>
>>   fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'
>>
>> And on the server:
>>
>>   [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago
>>
>> Fix this by clearing both c->password and c->credential for an expired
>> struct credential. While we're at it, use credential_clear_secrets()
>> wherever both c->password and c->credential are being cleared, and use
>> the full credential_clear() in credential_reject() after the credential
>> has been erased from all of the helpers.
> 
> OK.
> 
>>
>> v2: Unify secret clearing into credential_clear_secrets(), use
>> credential_clear() in credential_reject(), add a comment about why we
>> can't use credential_clear() in credential_fill().
> 
> This does not belong to the commit log message proper.  Those who
> are reading "git log" would not know (and care about) an earlier
> attempt.  Writing the changes since the previous round(s) like the
> above paragraph is very much appreciated as it helps reviewers who
> saw them, but please do so after the three-dash "---" line below (I
> can remove the paragraph while queueing, so no need to resend).

Sorry, I wasn't sure what the convention was since this was my first 
patch to the list. Thanks for fixing it. You can feel free to properly 
wrap the last line of that comment that I missed too, if you if you like. :)

> 
> Will queue.  Thanks.

Thanks!

Rahul (CC'd) and I had a series of patches to add something similar to 
the current authtype system but hadn't gotten around to sending them to 
the list before this more flexible mechanism was merged. It's nice that 
this worked out of the box with minimal adjustment.

The credential helper he wrote is specific to the Microsoft "Entra ID" 
identity provider system, but hopefully it'll be generally useful once 
this stuff is in a git release. It really cleans up the authentication 
process over https for sites that support it.

-- Aaron

>> Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
>> ---
>>   credential.c | 19 +++++++++----------
>>   1 file changed, 9 insertions(+), 10 deletions(-)
>>
>> diff --git a/credential.c b/credential.c
>> index 758528b291..72c6f46b02 100644
>> --- a/credential.c
>> +++ b/credential.c
>> @@ -20,12 +20,11 @@ void credential_init(struct credential *c)
>>   
>>   void credential_clear(struct credential *c)
>>   {
>> +	credential_clear_secrets(c);
>>   	free(c->protocol);
>>   	free(c->host);
>>   	free(c->path);
>>   	free(c->username);
>> -	free(c->password);
>> -	free(c->credential);
>>   	free(c->oauth_refresh_token);
>>   	free(c->authtype);
>>   	string_list_clear(&c->helpers, 0);
>> @@ -479,9 +478,14 @@ void credential_fill(struct credential *c, int all_capabilities)
>>   
>>   	for (i = 0; i < c->helpers.nr; i++) {
>>   		credential_do(c, c->helpers.items[i].string, "get");
>> +
>>   		if (c->password_expiry_utc < time(NULL)) {
>> -			/* Discard expired password */
>> -			FREE_AND_NULL(c->password);
>> +			/*
>> +			 * Don't use credential_clear() here: callers such as
>> +			 * cmd_credential() expect to still be able to call
>> +			 * credential_write() on a struct credential whose secrets have expired.
>> +			 */
>> +			credential_clear_secrets(c);
>>   			/* Reset expiry to maintain consistency */
>>   			c->password_expiry_utc = TIME_MAX;
>>   		}
>> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>>   	for (i = 0; i < c->helpers.nr; i++)
>>   		credential_do(c, c->helpers.items[i].string, "erase");
>>   
>> -	FREE_AND_NULL(c->username);
>> -	FREE_AND_NULL(c->password);
>> -	FREE_AND_NULL(c->credential);
>> -	FREE_AND_NULL(c->oauth_refresh_token);
>> -	c->password_expiry_utc = TIME_MAX;
>> -	c->approved = 0;
>> +	credential_clear(c);
>>   }
>>   
>>   static int check_url_component(const char *url, int quiet,

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Rahul Rameshbabu]

On Tue, 04 Jun, 2024 15:19:14 -0700 Aaron Plattner <aplattner@nvidia.com> wrote:
> On 6/4/24 3:04 PM, Junio C Hamano wrote:
>> Aaron Plattner <aplattner@nvidia.com> writes:
>> 
>>> When a struct credential expires, credential_fill() clears c->password
>>> so that clients don't try to use it later. However, a struct cred that
>>> uses an alternate authtype won't have a password, but might have a
>>> credential stored in c->credential.
>>>
>>> This is a problem, for example, when an OAuth2 bearer token is used. In
>>> the system I'm using, the OAuth2 configuration generates and caches a
>>> bearer token that is valid for an hour. After the token expires, git
>>> needs to call back into the credential helper to use a stored refresh
>>> token to get a new bearer token. But if c->credential is still non-NULL,
>>> git will instead try to use the expired token and fail with an error:
>>>
>>>   fatal: Authentication failed for 'https://<oauth2-enabled-server>/repository'
>>>
>>> And on the server:
>>>
>>>   [auth_openidc:error] [client <ip>:34012] oidc_proto_validate_exp: "exp" validation failure (1717522989): JWT expired 224 seconds ago
>>>
>>> Fix this by clearing both c->password and c->credential for an expired
>>> struct credential. While we're at it, use credential_clear_secrets()
>>> wherever both c->password and c->credential are being cleared, and use
>>> the full credential_clear() in credential_reject() after the credential
>>> has been erased from all of the helpers.
>> OK.
>> 
>>>
>>> v2: Unify secret clearing into credential_clear_secrets(), use
>>> credential_clear() in credential_reject(), add a comment about why we
>>> can't use credential_clear() in credential_fill().
>> This does not belong to the commit log message proper.  Those who
>> are reading "git log" would not know (and care about) an earlier
>> attempt.  Writing the changes since the previous round(s) like the
>> above paragraph is very much appreciated as it helps reviewers who
>> saw them, but please do so after the three-dash "---" line below (I
>> can remove the paragraph while queueing, so no need to resend).
>
> Sorry, I wasn't sure what the convention was since this was my first patch to
> the list. Thanks for fixing it. You can feel free to properly wrap the last line
> of that comment that I missed too, if you if you like. :)
>
>> Will queue.  Thanks.
>
> Thanks!
>
> Rahul (CC'd) and I had a series of patches to add something similar to the
> current authtype system but hadn't gotten around to sending them to the list
> before this more flexible mechanism was merged. It's nice that this worked out
> of the box with minimal adjustment.
>
> The credential helper he wrote is specific to the Microsoft "Entra ID" identity
> provider system, but hopefully it'll be generally useful once this stuff is in a
> git release. It really cleans up the authentication process over https for sites
> that support it.

Aaron made a commit to make it work with the authtype/credential
credential-helper infrastructure that landed in git-next.

  https://github.com/Binary-Eater/git-credential-msal/commit/f71ca9c72ca1a2cf73373de76909f6007ac689cb

The support for authtype excites me since a number of large Git
providers like GitHub/GitLab/etc. have utilized Authorization Basic
incorrectly for supporting different authtypes with git previously.
Hoping they will move away from this practice in the future with this
enhancement.

>
> -- Aaron
>
>>> Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
>>> ---
>>>   credential.c | 19 +++++++++----------
>>>   1 file changed, 9 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/credential.c b/credential.c
>>> index 758528b291..72c6f46b02 100644
>>> --- a/credential.c
>>> +++ b/credential.c
>>> @@ -20,12 +20,11 @@ void credential_init(struct credential *c)
>>>     void credential_clear(struct credential *c)
>>>   {
>>> +	credential_clear_secrets(c);
>>>   	free(c->protocol);
>>>   	free(c->host);
>>>   	free(c->path);
>>>   	free(c->username);
>>> -	free(c->password);
>>> -	free(c->credential);
>>>   	free(c->oauth_refresh_token);
>>>   	free(c->authtype);
>>>   	string_list_clear(&c->helpers, 0);
>>> @@ -479,9 +478,14 @@ void credential_fill(struct credential *c, int all_capabilities)
>>>     	for (i = 0; i < c->helpers.nr; i++) {
>>>   		credential_do(c, c->helpers.items[i].string, "get");
>>> +
>>>   		if (c->password_expiry_utc < time(NULL)) {
>>> -			/* Discard expired password */
>>> -			FREE_AND_NULL(c->password);
>>> +			/*
>>> +			 * Don't use credential_clear() here: callers such as
>>> +			 * cmd_credential() expect to still be able to call
>>> +			 * credential_write() on a struct credential whose secrets have expired.
>>> +			 */
>>> +			credential_clear_secrets(c);
>>>   			/* Reset expiry to maintain consistency */
>>>   			c->password_expiry_utc = TIME_MAX;
>>>   		}
>>> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>>>   	for (i = 0; i < c->helpers.nr; i++)
>>>   		credential_do(c, c->helpers.items[i].string, "erase");
>>>   -	FREE_AND_NULL(c->username);
>>> -	FREE_AND_NULL(c->password);
>>> -	FREE_AND_NULL(c->credential);
>>> -	FREE_AND_NULL(c->oauth_refresh_token);
>>> -	c->password_expiry_utc = TIME_MAX;
>>> -	c->approved = 0;
>>> +	credential_clear(c);
>>>   }
>>>     static int check_url_component(const char *url, int quiet,

-- 
Thanks,

Rahul Rameshbabu

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Junio C Hamano]

Rahul Rameshbabu <rrameshbabu@nvidia.com> writes:

>>> Aaron Plattner <aplattner@nvidia.com> writes:
>> ...
>> Rahul (CC'd) and I had a series of patches to add something similar to the
>> current authtype system but hadn't gotten around to sending them to the list
>> before this more flexible mechanism was merged. It's nice that this worked out
>> of the box with minimal adjustment.
>>
>> The credential helper he wrote is specific to the Microsoft "Entra ID" identity
>> provider system, but hopefully it'll be generally useful once this stuff is in a
>> git release. It really cleans up the authentication process over https for sites
>> that support it.
>
> Aaron made a commit to make it work with the authtype/credential
> credential-helper infrastructure that landed in git-next.
>
>   https://github.com/Binary-Eater/git-credential-msal/commit/f71ca9c72ca1a2cf73373de76909f6007ac689cb
>
> The support for authtype excites me since a number of large Git
> providers like GitHub/GitLab/etc. have utilized Authorization Basic
> incorrectly for supporting different authtypes with git previously.
> Hoping they will move away from this practice in the future with this
> enhancement.

Thanks for a huge praise that I do not personally deserve ;-)  Kudos
go to brian and folks who helped reviewing his topic.

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Jeff King]

On Tue, Jun 04, 2024 at 12:29:28PM -0700, Aaron Plattner wrote:

> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>  	for (i = 0; i < c->helpers.nr; i++)
>  		credential_do(c, c->helpers.items[i].string, "erase");
>  
> -	FREE_AND_NULL(c->username);
> -	FREE_AND_NULL(c->password);
> -	FREE_AND_NULL(c->credential);
> -	FREE_AND_NULL(c->oauth_refresh_token);
> -	c->password_expiry_utc = TIME_MAX;
> -	c->approved = 0;
> +	credential_clear(c);
>  }

I'm skeptical of this hunk. The caller will usually have filled in parts
of a credential struct like scheme and host, and then we picked up the
rest from helpers or by prompting the user. Rejecting the credential
should certainly clear the bogus password field and other secrets. But
should it clear the host field?

I think it may be somewhat academic for now because we'll generally exit
the program immediately after rejecting the credential. But occasionally
the topic comes up of retrying auth within a command. So you might have
a loop like this (or knowing our http code, probably some more baroque
equivalent spread across multiple functions):

  credential_from_url(&cred, url);
  for (int attempt = 0; attempt < 5; attempt++) {
	credential_fill(&cred);
	switch (do_something(url, &cred)) {
	case OK: /* it worked */
		return 0;
	case AUTH_ERROR:
		/* try again */
		credential_reject(&cred);
	}
  }
  return -1; /* too many failures */

And in that case you really want to retain the "query" parts of the
credential after the reject. In this toy example you could just move the
url-to-cred parsing into the loop, but in the real world it's often more
complicated.

Arguably even the original code is a bit questionable for this, because
we don't know if the username came from a helper or from the user, or if
it was part of the original URL (e.g., "https://user@example.com/"
should prompt only for the password). But it feels like this hunk is
making it worse.

The rest of the patch made sense to me, though. As would using
credential_clear_secrets() here to replace the equivalent lines.

-Peff

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Aaron Plattner]

On 6/5/24 1:57 AM, Jeff King wrote:
> On Tue, Jun 04, 2024 at 12:29:28PM -0700, Aaron Plattner wrote:
> 
>> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>>   	for (i = 0; i < c->helpers.nr; i++)
>>   		credential_do(c, c->helpers.items[i].string, "erase");
>>   
>> -	FREE_AND_NULL(c->username);
>> -	FREE_AND_NULL(c->password);
>> -	FREE_AND_NULL(c->credential);
>> -	FREE_AND_NULL(c->oauth_refresh_token);
>> -	c->password_expiry_utc = TIME_MAX;
>> -	c->approved = 0;
>> +	credential_clear(c);
>>   }
> 
> I'm skeptical of this hunk. The caller will usually have filled in parts
> of a credential struct like scheme and host, and then we picked up the
> rest from helpers or by prompting the user. Rejecting the credential
> should certainly clear the bogus password field and other secrets. But
> should it clear the host field?
> 
> I think it may be somewhat academic for now because we'll generally exit
> the program immediately after rejecting the credential. But occasionally
> the topic comes up of retrying auth within a command. So you might have
> a loop like this (or knowing our http code, probably some more baroque
> equivalent spread across multiple functions):
> 
>    credential_from_url(&cred, url);
>    for (int attempt = 0; attempt < 5; attempt++) {
> 	credential_fill(&cred);
> 	switch (do_something(url, &cred)) {
> 	case OK: /* it worked */
> 		return 0;
> 	case AUTH_ERROR:
> 		/* try again */
> 		credential_reject(&cred);
> 	}
>    }
>    return -1; /* too many failures */
> 
> And in that case you really want to retain the "query" parts of the
> credential after the reject. In this toy example you could just move the
> url-to-cred parsing into the loop, but in the real world it's often more
> complicated.
> 
> Arguably even the original code is a bit questionable for this, because
> we don't know if the username came from a helper or from the user, or if
> it was part of the original URL (e.g., "https://user@example.com/"
> should prompt only for the password). But it feels like this hunk is
> making it worse.

The comment above credential_reject() mentions that it is "readying the 
credential for another call to `credential_fill`" which does imply that 
you can use it again right away without having to fill in the protocol / 
host / path fields. So you're probably right that this should remain the 
way it was.

> The rest of the patch made sense to me, though. As would using
> credential_clear_secrets() here to replace the equivalent lines.

That's certainly fine with me. Using credential_clear_secrets() to just 
replace those two lines would definitely keep the original behavior of 
this code.

I'll send a v3 patch to do that.

-- Aaron

> 
> -Peff

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Jeff King]

On Wed, Jun 05, 2024 at 09:45:32AM -0700, Aaron Plattner wrote:

> > And in that case you really want to retain the "query" parts of the
> > credential after the reject. In this toy example you could just move the
> > url-to-cred parsing into the loop, but in the real world it's often more
> > complicated.
> > 
> > Arguably even the original code is a bit questionable for this, because
> > we don't know if the username came from a helper or from the user, or if
> > it was part of the original URL (e.g., "https://user@example.com/"
> > should prompt only for the password). But it feels like this hunk is
> > making it worse.
> 
> The comment above credential_reject() mentions that it is "readying the
> credential for another call to `credential_fill`" which does imply that you
> can use it again right away without having to fill in the protocol / host /
> path fields. So you're probably right that this should remain the way it
> was.

Heh, OK. I was the one who wrote that comment originally, which I guess
is why it was in the back of my mind. ;)

As I said, clearing "username" is a little questionable there. But it
also gives the user a chance to update the field, so maybe it's not so
bad. There might be other fields in the same boat, but I think you'd
really have to think about each one. I'm content to leave the code as it
is for now, and if somebody comes up with a case where reject+fill
doesn't behave as they expect, we can think about it further.

> > The rest of the patch made sense to me, though. As would using
> > credential_clear_secrets() here to replace the equivalent lines.
> 
> That's certainly fine with me. Using credential_clear_secrets() to just
> replace those two lines would definitely keep the original behavior of this
> code.
> 
> I'll send a v3 patch to do that.

Great, thanks!

-Peff

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Jun 04, 2024 at 12:29:28PM -0700, Aaron Plattner wrote:
>
>> @@ -528,12 +532,7 @@ void credential_reject(struct credential *c)
>>  	for (i = 0; i < c->helpers.nr; i++)
>>  		credential_do(c, c->helpers.items[i].string, "erase");
>>  
>> -	FREE_AND_NULL(c->username);
>> -	FREE_AND_NULL(c->password);
>> -	FREE_AND_NULL(c->credential);
>> -	FREE_AND_NULL(c->oauth_refresh_token);
>> -	c->password_expiry_utc = TIME_MAX;
>> -	c->approved = 0;
>> +	credential_clear(c);
>>  }
>
> I'm skeptical of this hunk. The caller will usually have filled in parts
> of a credential struct like scheme and host, and then we picked up the
> rest from helpers or by prompting the user. Rejecting the credential
> should certainly clear the bogus password field and other secrets. But
> should it clear the host field?
>
> I think it may be somewhat academic for now because we'll generally exit
> the program immediately after rejecting the credential. But occasionally
> the topic comes up of retrying auth within a command. So you might have
> a loop like this (or knowing our http code, probably some more baroque
> equivalent spread across multiple functions):
>
>   credential_from_url(&cred, url);
>   for (int attempt = 0; attempt < 5; attempt++) {
> 	credential_fill(&cred);
> 	switch (do_something(url, &cred)) {
> 	case OK: /* it worked */
> 		return 0;
> 	case AUTH_ERROR:
> 		/* try again */
> 		credential_reject(&cred);
> 	}
>   }
>   return -1; /* too many failures */
>
> And in that case you really want to retain the "query" parts of the
> credential after the reject. In this toy example you could just move the
> url-to-cred parsing into the loop, but in the real world it's often more
> complicated.
>
> Arguably even the original code is a bit questionable for this, because
> we don't know if the username came from a helper or from the user, or if
> it was part of the original URL (e.g., "https://user@example.com/"
> should prompt only for the password). But it feels like this hunk is
> making it worse.
>
> The rest of the patch made sense to me, though. As would using
> credential_clear_secrets() here to replace the equivalent lines.

So we have clear() that is to "clear everything", clear_secret()
that is to "clear auth material", but we would want another "clear
every members other than used as query keys" level?

That way, anytime we add different kind of "auth material" (like
brian's series did), existing code paths that call clear_secret() do
not have to change, and if we add different kind of "query keys",
the reject code would not have to change?  Or is the reject code
path the only thing that cares about what members are used as query
keys, in which case we do not need the third helper?

Thanks.  

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Jeff King]

On Wed, Jun 05, 2024 at 10:06:41AM -0700, Junio C Hamano wrote:

> So we have clear() that is to "clear everything", clear_secret()
> that is to "clear auth material", but we would want another "clear
> every members other than used as query keys" level?
> 
> That way, anytime we add different kind of "auth material" (like
> brian's series did), existing code paths that call clear_secret() do
> not have to change, and if we add different kind of "query keys",
> the reject code would not have to change?  Or is the reject code
> path the only thing that cares about what members are used as query
> keys, in which case we do not need the third helper?

I can't think of another place besides the reject path where we'd want
that (though I'm certainly open to being corrected if somebody finds
such a spot). But mostly I am not all that confident that the set of
items that reject() is clearing is the best one. So I'd just as soon
leave it as a weird internal detail for now, rather than codifying it in
a function.

I dunno. I guess it is the same lines of code in either spot, but
somehow sticking it in a clear_response() helper seems like an
endorsement that the author knew what they were doing. ;)

-Peff

Re: [PATCH v2] credential: clear expired c->credential, unify secret clearing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> items that reject() is clearing is the best one. So I'd just as soon
> leave it as a weird internal detail for now, rather than codifying it in
> a function.
>
> I dunno. I guess it is the same lines of code in either spot, but
> somehow sticking it in a clear_response() helper seems like an
> endorsement that the author knew what they were doing. ;)

True.  It probably belongs to too premature abstraction.  Thanks.