[PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Jeff King]

After upgrading my Debian unstable box to libcurl 8.18.0~rc2-1, I
noticed a few new test failures. They seem to mostly be caused by
brittle expectations in the tests. These patches fix the tests to handle
both old and new versions (I tested against curl's 8_17_0 and
rc-8_18_0-2 tags).

Daniel: I'm cc-ing you in case you want to double-check that curl's
behavior changes are all OK before the release. I think it's mostly
fine, though the handling of tab versus space in the third patch is
perhaps questionable.

Matthew: I had to make some educated guesses about one of the tests in
patch 2. You might remember the original intent.

  [1/3]: t5551: handle trailing slashes in expected cookies output
  [2/3]: t5563: add missing end-of-line in HTTP header
  [3/3]: t5563: relax whitespace assumptions for unfolded headers

 t/t5551-http-fetch-smart.sh | 15 +++++++++------
 t/t5563-simple-http-auth.sh | 15 +++++++++++----
 2 files changed, 20 insertions(+), 10 deletions(-)

-Peff

[PATCH 1/3] t5551: handle trailing slashes in expected cookies output

[Jeff King]

We check in t5551 that curl updates the expected list of cookies after
making a request. We do this by telling it to read and write cookies
from a particular text file, and then checking that after curl runs, the
file has the expected content.

However, in the upcoming curl 8.18.0, the output file has changed
slightly: curl will canonicalize the paths it writes, due to commit
a093c93994 (cookie: only keep and use the canonical cleaned up path,
2025-12-07). In particular, it strips trailing slashes from the paths we
see in the cookies.txt file.

This doesn't matter to Git, as the cookie handling is all internal to
curl. But our test is overly brittle and breaks as a result.

We can fix it by matching either format. We'll expect the new format
(without trailing slashes) and strip the slashes from curl's output
before comparing. That lets us pass with both old and new versions (I
tested against curl's 8_17_0 and rc-8_18_0-2 tags, which are
respectively before and after the curl change).

In theory it might be nice to try to future-proof this test more by
looking only for the bits we care about, rather than a byte-wise
comparison of the whole file. But after removing comments and blank
lines (which we already do), we care about most of what's there. So it's
not clear to me what a more liberal test would look like. Given that the
format doesn't change all that often, it's probably OK to stop here and
see if it ever breaks again.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t5551-http-fetch-smart.sh | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index b0d4ea7801..73cf531580 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -333,12 +333,12 @@ test_expect_success 'dumb clone via http-backend respects namespace' '
 
 test_expect_success 'cookies stored in http.cookiefile when http.savecookies set' '
 	cat >cookies.txt <<-\EOF &&
-	127.0.0.1	FALSE	/smart_cookies/	FALSE	0	othername	othervalue
+	127.0.0.1	FALSE	/smart_cookies	FALSE	0	othername	othervalue
 	EOF
 	sort >expect_cookies.txt <<-\EOF &&
-	127.0.0.1	FALSE	/smart_cookies/	FALSE	0	othername	othervalue
-	127.0.0.1	FALSE	/smart_cookies/repo.git/	FALSE	0	name	value
-	127.0.0.1	FALSE	/smart_cookies/repo.git/info/	FALSE	0	name	value
+	127.0.0.1	FALSE	/smart_cookies	FALSE	0	othername	othervalue
+	127.0.0.1	FALSE	/smart_cookies/repo.git	FALSE	0	name	value
+	127.0.0.1	FALSE	/smart_cookies/repo.git/info	FALSE	0	name	value
 	EOF
 	git config http.cookiefile cookies.txt &&
 	git config http.savecookies true &&
@@ -351,8 +351,11 @@ test_expect_success 'cookies stored in http.cookiefile when http.savecookies set
 		tag -m "foo" cookie-tag &&
 	git fetch $HTTPD_URL/smart_cookies/repo.git cookie-tag &&
 
-	grep "^[^#]" cookies.txt | sort >cookies_stripped.txt &&
-	test_cmp expect_cookies.txt cookies_stripped.txt
+	# Strip trailing slashes from cookie paths to handle output from both
+	# old curl ("/smart_cookies/") and new ("/smart_cookies").
+	HT="	" &&
+	grep "^[^#]" cookies.txt | sed "s,/$HT,$HT," | sort >cookies_clean.txt &&
+	test_cmp expect_cookies.txt cookies_clean.txt
 '
 
 test_expect_success 'transfer.hiderefs works over smart-http' '
-- 
2.52.0.595.gac9d83db54

[PATCH 2/3] t5563: add missing end-of-line in HTTP header

[Jeff King]

In t5563, we test how various oddly-formatted WWW-Authenticate headers
are passed through curl to git's credential subsystem (and ultimately
out to credential helpers). One test, "access using basic auth with
wwwauth header mixed line-endings" does something odd. It does not mix
line endings at all (which must be CRLF according to the RFC anyway),
but omits the line ending entirely for the final header!

This means that the server produces an incomplete response. We send our
final header, and then the newline which is meant to mark the end of
headers (and the start of the body) becomes the line ending for that
header. And there is no header/body separator in the output at all.

Looking at strace, this is what the client reads:

  recvfrom(9, "WWW-Authenticate: FooBar param1=\"value1\"\r\n \r\n\tparam2=\"value2\"\r\nWWW-Authenticate: Basic realm=\"example.com\"", 16384, 0, NULL, NULL) = 106
  recvfrom(9, "\n", 16384, 0, NULL, NULL) = 1
  recvfrom(9, "", 16384, 0, NULL, NULL) = 0

The headers themselves are produced from the custom-auth.challenge file
we write in the test (which is missing the final CRLF), and then the
header/body separator comes from our lib-httpd/nph-custom-auth.sh CGI.
(Ignore for a moment that it is producing a bare newline, which I think
is a bug; it should be a CRLF but curl is happy with either).

Older versions of curl seemed to be OK with the truncated output, but
the upcoming 8.18.0 release seems to get confused. Specifically, since
67ae101666 (http: unfold response headers earlier, 2025-12-12) our
request to the server fails with insufficient credentials. I traced far
enough to see that curl does relay the header back to us, which we then
pass to a credential helper, which gives us the correct
username/password combination. But on our followup request, curl refuses
to send the Authorization header (and so gets an HTTP 401 again).

The change in curl's behavior is a bit unexpected, but since we are
sending it garbage, it is hard to complain too much. Let's add the
missing CRLF to the header. I _think_ this was just an oversight and not
the intent of the test. And that the "mixed line-endings" really meant
"mixed continuations", since we differ from the previous test in
continuing with both space and tab. So I've likewise updated the test
title to match that assumption.

Signed-off-by: Jeff King <peff@peff.net>
---
I do find it puzzling that we hand curl the credential, but it doesn't
get used in the follow-up request. So I may have mis-analyzed something,
but I really think that's what is happening. I can share the
hacky instrumentation I added if anybody wants to dig further. But since
the original was garbage AFAICT, I didn't think it was worth spending
a lot of time on it.

 t/t5563-simple-http-auth.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh
index 317f33af5a..c1febbae9d 100755
--- a/t/t5563-simple-http-auth.sh
+++ b/t/t5563-simple-http-auth.sh
@@ -469,7 +469,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat
 	EOF
 '
 
-test_expect_success 'access using basic auth with wwwauth header mixed line-endings' '
+test_expect_success 'access using basic auth with wwwauth header mixed continuations' '
 	test_when_finished "per_test_cleanup" &&
 
 	set_credential_reply get <<-EOF &&
@@ -490,7 +490,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi
 	printf "id=default response=WWW-Authenticate: FooBar param1=\"value1\"\r\n" >>"$CHALLENGE" &&
 	printf "id=default response= \r\n" >>"$CHALLENGE" &&
 	printf "id=default response=\tparam2=\"value2\"\r\n" >>"$CHALLENGE" &&
-	printf "id=default response=WWW-Authenticate: Basic realm=\"example.com\"" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: Basic realm=\"example.com\"\r\n" >>"$CHALLENGE" &&
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
-- 
2.52.0.595.gac9d83db54

Re: [PATCH 2/3] t5563: add missing end-of-line in HTTP header

[Matthew John Cheetham]

On 2025-12-18 12:18, Jeff King wrote:

 > In t5563, we test how various oddly-formatted WWW-Authenticate headers
 > are passed through curl to git's credential subsystem (and ultimately
 > out to credential helpers). One test, "access using basic auth with
 > wwwauth header mixed line-endings" does something odd. It does not mix
 > line endings at all (which must be CRLF according to the RFC anyway),
 > but omits the line ending entirely for the final header!

Aha! Yes, the test should be using *all CRLF line endings*, and is
poorly named. I believe the intent here is to test mixed *continuation
line* characters.

E.g, when a continuation line starts with a space, or a tab character,
for the same logical header:

WWW-Authenticate: FooBar param1="value1"\r\n
  \r\n
\tparam2="value2"\r\n


 > This means that the server produces an incomplete response. We send our
 > final header, and then the newline which is meant to mark the end of
 > headers (and the start of the body) becomes the line ending for that
 > header. And there is no header/body separator in the output at all.
 >
 > Looking at strace, this is what the client reads:
 >
 >    recvfrom(9, "WWW-Authenticate: FooBar param1=\"value1\"\r\n 
\r\n\tparam2=\"value2\"\r\nWWW-Authenticate: Basic 
realm=\"example.com\"", 16384, 0, NULL, NULL) = 106
 >    recvfrom(9, "\n", 16384, 0, NULL, NULL) = 1
 >    recvfrom(9, "", 16384, 0, NULL, NULL) = 0
 >
 > The headers themselves are produced from the custom-auth.challenge file
 > we write in the test (which is missing the final CRLF), and then the
 > header/body separator comes from our lib-httpd/nph-custom-auth.sh CGI.
 > (Ignore for a moment that it is producing a bare newline, which I think
 > is a bug; it should be a CRLF but curl is happy with either).
 >
 > Older versions of curl seemed to be OK with the truncated output, but
 > the upcoming 8.18.0 release seems to get confused. Specifically, since
 > 67ae101666 (http: unfold response headers earlier, 2025-12-12) our
 > request to the server fails with insufficient credentials. I traced far
 > enough to see that curl does relay the header back to us, which we then
 > pass to a credential helper, which gives us the correct
 > username/password combination. But on our followup request, curl refuses
 > to send the Authorization header (and so gets an HTTP 401 again).
 >
 > The change in curl's behavior is a bit unexpected, but since we are
 > sending it garbage, it is hard to complain too much. Let's add the
 > missing CRLF to the header. I _think_ this was just an oversight and not
 > the intent of the test. And that the "mixed line-endings" really meant
 > "mixed continuations", since we differ from the previous test in
 > continuing with both space and tab. So I've likewise updated the test
 > title to match that assumption.
 >
 > Signed-off-by: Jeff King <peff@peff.net>
 > ---
 > I do find it puzzling that we hand curl the credential, but it doesn't
 > get used in the follow-up request. So I may have mis-analyzed something,
 > but I really think that's what is happening. I can share the
 > hacky instrumentation I added if anybody wants to dig further. But since
 > the original was garbage AFAICT, I didn't think it was worth spending
 > a lot of time on it.
 >
 >   t/t5563-simple-http-auth.sh | 4 ++--
 >   1 file changed, 2 insertions(+), 2 deletions(-)
 >
 > diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh
 > index 317f33af5a..c1febbae9d 100755
 > --- a/t/t5563-simple-http-auth.sh
 > +++ b/t/t5563-simple-http-auth.sh
 > @@ -469,7 +469,7 @@ test_expect_success 'access using basic auth with 
wwwauth header empty continuat
 >   	EOF
 >   '
 >
 > -test_expect_success 'access using basic auth with wwwauth header 
mixed line-endings' '
 > +test_expect_success 'access using basic auth with wwwauth header 
mixed continuations' '

Perfect! Thanks for fixing my poor naming :)

 >   	test_when_finished "per_test_cleanup" &&
 >
 >   	set_credential_reply get <<-EOF &&
 > @@ -490,7 +490,7 @@ test_expect_success 'access using basic auth with 
wwwauth header mixed line-endi
 >   	printf "id=default response=WWW-Authenticate: FooBar 
param1=\"value1\"\r\n" >>"$CHALLENGE" &&
 >   	printf "id=default response= \r\n" >>"$CHALLENGE" &&
 >   	printf "id=default response=\tparam2=\"value2\"\r\n" >>"$CHALLENGE" &&
 > -	printf "id=default response=WWW-Authenticate: Basic 
realm=\"example.com\"" >>"$CHALLENGE" &&
 > +	printf "id=default response=WWW-Authenticate: Basic 
realm=\"example.com\"\r\n" >>"$CHALLENGE" &&
 >
 >   	test_config_global credential.helper test-helper &&
 >   	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&

Thanks,
Matthew

Re: [PATCH 2/3] t5563: add missing end-of-line in HTTP header

[Jeff King]

On Thu, Dec 18, 2025 at 01:41:54PM +0000, Matthew John Cheetham wrote:

> On 2025-12-18 12:18, Jeff King wrote:
> 
> > In t5563, we test how various oddly-formatted WWW-Authenticate headers
> > are passed through curl to git's credential subsystem (and ultimately
> > out to credential helpers). One test, "access using basic auth with
> > wwwauth header mixed line-endings" does something odd. It does not mix
> > line endings at all (which must be CRLF according to the RFC anyway),
> > but omits the line ending entirely for the final header!
> 
> Aha! Yes, the test should be using *all CRLF line endings*, and is
> poorly named. I believe the intent here is to test mixed *continuation
> line* characters.
> 
> E.g, when a continuation line starts with a space, or a tab character,
> for the same logical header:
> 
> WWW-Authenticate: FooBar param1="value1"\r\n
>  \r\n
> \tparam2="value2"\r\n

Ah, great. I'm happy that my guess was right and there was not something
trickier going on (which would have made coming up with a workaround
more difficult!). Thanks for confirming.

-Peff

[PATCH 3/3] t5563: relax whitespace assumptions for unfolded headers

[Jeff King]

In t5563 we check the handling of WWW-Authenticate headers that have
been folded (i.e., where a continuation line starts with extra
whitespace). Traditionally curl handed each line to us individually, but
in the upcoming v8.18.0, it hands us full lines that have been unfolded.
But it doesn't produce exactly the same unfolding that we did!

In particular, two of the tests send an extra blank continuation line.
Something like this:

  printf 'WWW-Authenticate: foo param1="value1"\r\n'
  printf ' \r\n'
  printf ' param2="value2"\r\n"

We unfold that into:

  WWW-Authenticate: foo param1="value1" param2="value2"

But curl will give us a string with an extra space:

  WWW-Authenticate: foo param1="value1"  param2="value2"

I think curl is actually correct here. RFC 7230 says:

   A user agent that receives an obs-fold in a response message that is
   not within a message/http container MUST replace each received
   obs-fold with one or more SP octets prior to interpreting the field
   value.

So each folded instance turns the initial whitespace into "one or more"
spaces, and the "blank" line becomes a single space. Whereas Git's
unfolding code explicitly avoids this, with the comment "Do not bother
appending the new value if this continuation header is itself empty." in
fwrite_wwwauth().

I think it's mostly academic at this point. These folded continuations
have been deprecated entirely since RFC 7230 came out in 2014, and
there's very little reason for a server to add a blank continuation line
at all. And anybody parsing the unfolded header contents should skip
past the extra whitespace (which is allowed to be present according to
the RFC).

But our tests do a byte-wise comparison, so they care about the
difference between the two outputs. We have two options here:

  1. We can modify Git's unfolding code to behave like modern curl.

  2. We can relax the tests to be happy with either output.

I picked (2) here, just because it seemed less risky to touch only the
tests and not the code (though if any real-world systems _do_ care about
the distinction, they will eventually run into problems when libcurl is
upgraded).

There is one further curiosity here. There's a second test which mixes
tabs and spaces for continuation, like this:

  printf 'WWW-Authenticate: foo param1="value1"\r\n'
  printf '\t\r\n'
  printf ' param2="value2"\r\n"

From the snippet of RFC quoted above, I believe this should produce the
exact same output (the continuation whitespace is replaced with one or
more spaces, even though it is a tab here). But curl retains the tab
instead!

So to implement the "relaxed whitespace" mode in the test, we just
convert any run of multiple whitespace characters to a single space.
This is a bit hacky and over-zealous, but it's easy to do and good
enough for our purposes here. We only enable the relaxed mode for the
two tests which trigger this issue.

Signed-off-by: Jeff King <peff@peff.net>
---
Note that when built against this new version of curl, Git's unfolding
code should never trigger at all. In the long run we should be able to
rip it out, but we probably need to wait a decade or so before we can
bump the minimum libcurl version to 8.18.0.

I guess we could make it a conditional in the code (which would help us
remember to eventually rip it out), but it felt weird to start adding
version conditionals for a version that isn't even released yet. ;)

 t/t5563-simple-http-auth.sh | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh
index c1febbae9d..0967cd501c 100755
--- a/t/t5563-simple-http-auth.sh
+++ b/t/t5563-simple-http-auth.sh
@@ -47,6 +47,13 @@ set_credential_reply () {
 expect_credential_query () {
 	local suffix="$(test -n "$2" && echo "-$2")"
 	cat >"$TRASH_DIRECTORY/$1-expect$suffix.cred" &&
+	if $(test "$3" = "--relax-whitespace")
+	then
+		HT='	' &&
+		sed "s/[ $HT][ $HT]*/ /g" \
+			<"$TRASH_DIRECTORY/$1-query$suffix.cred" >tmp &&
+		mv tmp "$TRASH_DIRECTORY/$1-query$suffix.cred"
+	fi &&
 	test_cmp "$TRASH_DIRECTORY/$1-expect$suffix.cred" \
 		 "$TRASH_DIRECTORY/$1-query$suffix.cred"
 }
@@ -451,7 +458,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
-	expect_credential_query get <<-EOF &&
+	expect_credential_query get "" --relax-whitespace <<-EOF &&
 	capability[]=authtype
 	capability[]=state
 	protocol=http
@@ -495,7 +502,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed continuat
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
-	expect_credential_query get <<-EOF &&
+	expect_credential_query get "" --relax-whitespace <<-EOF &&
 	capability[]=authtype
 	capability[]=state
 	protocol=http
-- 
2.52.0.595.gac9d83db54

Re: [PATCH 3/3] t5563: relax whitespace assumptions for unfolded headers

[Matthew John Cheetham]

On 2025-12-18 12:22, Jeff King wrote:

 > In t5563 we check the handling of WWW-Authenticate headers that have
 > been folded (i.e., where a continuation line starts with extra
 > whitespace). Traditionally curl handed each line to us individually, but
 > in the upcoming v8.18.0, it hands us full lines that have been unfolded.
 > But it doesn't produce exactly the same unfolding that we did!
 >
 > In particular, two of the tests send an extra blank continuation line.
 > Something like this:
 >
 >    printf 'WWW-Authenticate: foo param1="value1"\r\n'
 >    printf ' \r\n'
 >    printf ' param2="value2"\r\n"
 >
 > We unfold that into:
 >
 >    WWW-Authenticate: foo param1="value1" param2="value2"
 >
 > But curl will give us a string with an extra space:
 >
 >    WWW-Authenticate: foo param1="value1"  param2="value2"
 >
 > I think curl is actually correct here. RFC 7230 says:
 >
 >     A user agent that receives an obs-fold in a response message that is
 >     not within a message/http container MUST replace each received
 >     obs-fold with one or more SP octets prior to interpreting the field
 >     value.
 >
 > So each folded instance turns the initial whitespace into "one or more"
 > spaces, and the "blank" line becomes a single space. Whereas Git's
 > unfolding code explicitly avoids this, with the comment "Do not bother
 > appending the new value if this continuation header is itself empty." in
 > fwrite_wwwauth().
 >
 > I think it's mostly academic at this point. These folded continuations
 > have been deprecated entirely since RFC 7230 came out in 2014, and
 > there's very little reason for a server to add a blank continuation line
 > at all. And anybody parsing the unfolded header contents should skip
 > past the extra whitespace (which is allowed to be present according to
 > the RFC).
 >
 > But our tests do a byte-wise comparison, so they care about the
 > difference between the two outputs. We have two options here:
 >
 >    1. We can modify Git's unfolding code to behave like modern curl.
 >
 >    2. We can relax the tests to be happy with either output.
 >
 > I picked (2) here, just because it seemed less risky to touch only the
 > tests and not the code (though if any real-world systems _do_ care about
 > the distinction, they will eventually run into problems when libcurl is
 > upgraded).

I think that is a fair choice.

 > There is one further curiosity here. There's a second test which mixes
 > tabs and spaces for continuation, like this:
 >
 >    printf 'WWW-Authenticate: foo param1="value1"\r\n'
 >    printf '\t\r\n'
 >    printf ' param2="value2"\r\n"
 >
 >  From the snippet of RFC quoted above, I believe this should produce the
 > exact same output (the continuation whitespace is replaced with one or
 > more spaces, even though it is a tab here). But curl retains the tab
 > instead!

Header continuations are just a nightmare :-(

 > So to implement the "relaxed whitespace" mode in the test, we just
 > convert any run of multiple whitespace characters to a single space.
 > This is a bit hacky and over-zealous, but it's easy to do and good
 > enough for our purposes here. We only enable the relaxed mode for the
 > two tests which trigger this issue.
 >
 > Signed-off-by: Jeff King <peff@peff.net>
 > ---
 > Note that when built against this new version of curl, Git's unfolding
 > code should never trigger at all. In the long run we should be able to
 > rip it out, but we probably need to wait a decade or so before we can
 > bump the minimum libcurl version to 8.18.0.
 >
 > I guess we could make it a conditional in the code (which would help us
 > remember to eventually rip it out), but it felt weird to start adding
 > version conditionals for a version that isn't even released yet. ;)
 >
 >   t/t5563-simple-http-auth.sh | 11 +++++++++--
 >   1 file changed, 9 insertions(+), 2 deletions(-)
 >
 > diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh
 > index c1febbae9d..0967cd501c 100755
 > --- a/t/t5563-simple-http-auth.sh
 > +++ b/t/t5563-simple-http-auth.sh
 > @@ -47,6 +47,13 @@ set_credential_reply () {
 >   expect_credential_query () {
 >   	local suffix="$(test -n "$2" && echo "-$2")"
 >   	cat >"$TRASH_DIRECTORY/$1-expect$suffix.cred" &&
 > +	if $(test "$3" = "--relax-whitespace")
 > +	then
 > +		HT='	' &&
 > +		sed "s/[ $HT][ $HT]*/ /g" \
 > +			<"$TRASH_DIRECTORY/$1-query$suffix.cred" >tmp &&
 > +		mv tmp "$TRASH_DIRECTORY/$1-query$suffix.cred"
 > +	fi &&
 >   	test_cmp "$TRASH_DIRECTORY/$1-expect$suffix.cred" \
 >   		 "$TRASH_DIRECTORY/$1-query$suffix.cred"
 >   }
 > @@ -451,7 +458,7 @@ test_expect_success 'access using basic auth with 
wwwauth header empty continuat
 >   	test_config_global credential.helper test-helper &&
 >   	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 >
 > -	expect_credential_query get <<-EOF &&
 > +	expect_credential_query get "" --relax-whitespace <<-EOF &&
 >   	capability[]=authtype
 >   	capability[]=state
 >   	protocol=http
 > @@ -495,7 +502,7 @@ test_expect_success 'access using basic auth with 
wwwauth header mixed continuat
 >   	test_config_global credential.helper test-helper &&
 >   	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 >
 > -	expect_credential_query get <<-EOF &&
 > +	expect_credential_query get "" --relax-whitespace <<-EOF &&
 >   	capability[]=authtype
 >   	capability[]=state
 >   	protocol=http

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Daniel Stenberg]

On Thu, 18 Dec 2025, Jeff King wrote:

> Daniel: I'm cc-ing you in case you want to double-check that curl's behavior 
> changes are all OK before the release. I think it's mostly fine, though the 
> handling of tab versus space in the third patch is perhaps questionable.

Thanks Jeff, allow me to add my comments here on the three patches:

>  [1/3]: t5551: handle trailing slashes in expected cookies output

This is all benign. As you correctly observed, we no longer keep the 
"original" cookie path around and only work with the sanitized version - so 
that's the one stored now. It was already the one used for actual comparisons 
so apart from the change in storage, it *should* not cause any problems.

>  [2/3]: t5563: add missing end-of-line in HTTP header

I believe I made some code checks a little stricter: header lines MUST end 
with at CR or LF (or both) to be treated as a valid one. Your fix for this 
should be good also for older libcurl versions.

>  [3/3]: t5563: relax whitespace assumptions for unfolded headers

This one is material for me to rethink.

I had to completely change our header unfolding logic because we learned that 
we did not apply it early enough, so some header parsing was wrongly done on 
pre-unfolded data. In this process, I also changed the logic that appends the 
following line on the previous line. To avoid having to keep a state, I 
decided to just append the second line onto the first one without trying to 
reduce the whitespace characters to a single one.

I did not fully consider the impact this might have on users such as you. 
Allow me to rework that a little bit further and get the former white-space 
behavior back. Thanks!

-- 

  / daniel.haxx.se

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Daniel Stenberg]

On Thu, 18 Dec 2025, Daniel Stenberg wrote:

>>  [3/3]: t5563: relax whitespace assumptions for unfolded headers

> I did not fully consider the impact this might have on users such as you. 
> Allow me to rework that a little bit further and get the former white-space 
> behavior back. Thanks!

I just merged a fix [1] into curl that should restore the unfolding behavior 
to match previous releases. It would be awesome if you could verify.

[1] = https://github.com/curl/curl/commit/9941e7c95bf26f00fd87888a

-- 

  / daniel.haxx.se

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Jeff King]

On Thu, Dec 18, 2025 at 05:49:27PM +0100, Daniel Stenberg wrote:

> On Thu, 18 Dec 2025, Daniel Stenberg wrote:
> 
> > >  [3/3]: t5563: relax whitespace assumptions for unfolded headers
> 
> > I did not fully consider the impact this might have on users such as
> > you. Allow me to rework that a little bit further and get the former
> > white-space behavior back. Thanks!
> 
> I just merged a fix [1] into curl that should restore the unfolding behavior
> to match previous releases. It would be awesome if you could verify.
> 
> [1] = https://github.com/curl/curl/commit/9941e7c95bf26f00fd87888a

Thanks, I took a look at it. Unfortunately I think it only gets us
halfway there. It drops the extra space when folding this:

  printf 'Foo: bar\r\n'
  printf ' \r\n'
  printf ' baz\r\n'

which will yield:

  Foo: bar baz

and it fixes the first of Git's failing tests. But if we swap out the space for a tab
like this:

  printf 'Foo: bar\r\n'
  printf ' \r\n'
  printf '\tbaz\r\n'

then we get collapsed whitespace, but it's a tab. I.e.:

  Foo: bar\tbaz

(where "\t" is a literal tab). I think that does violate the standard
(which says it should become spaces). I think in most headers the
grammar allows OWS/RWS fields that are spaces or tabs, so in theory it
shouldn't matter. But I wouldn't be surprised if that causes some
surprises in the real world.

Sadly the input buffer to http_parse_headers() is const, so we can't
just write a space over the original tab. ;) But I think rather than
walking back to preserve that final leading whitespace byte, we could
just always add in our own space separately, like this:

diff --git a/lib/http.c b/lib/http.c
index ea62219542..eaa8bf73c2 100644
--- a/lib/http.c
+++ b/lib/http.c
@@ -4388,6 +4388,7 @@ static CURLcode http_parse_headers(struct Curl_easy *data,
     {
       /* preserve the whole original header piece size */
       size_t header_piece = consumed;
+      bool did_unfold = false;
 
       if(data->state.leading_unfold) {
         /* immediately after an unfold, keep only a single whitespace */
@@ -4398,17 +4399,18 @@ static CURLcode http_parse_headers(struct Curl_easy *data,
           blen--;
         }
         if(consumed) {
-          if(iblen > blen) {
-            /* take one step back */
-            consumed++;
-            buf--;
-            blen++;
-          }
           data->state.leading_unfold = FALSE; /* done now */
+          did_unfold = TRUE;
         }
       }
 
       if(consumed) {
+        if (did_unfold) {
+          result = curlx_dyn_addn(&data->state.headerb, " ", 1);
+          if(result)
+            return result;
+        }
+
         result = curlx_dyn_addn(&data->state.headerb, buf, consumed);
         if(result)
           return result;

-Peff

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Daniel Stenberg]

On Fri, 19 Dec 2025, Jeff King wrote:

>> [1] = https://github.com/curl/curl/commit/9941e7c95bf26f00fd87888a
>
> and it fixes the first of Git's failing tests. But if we swap out the space 
> for a tab like this:

Sorry, that was just sloppy of me to not add a test and proper handling for 
that condition. Allow me to fix that in my end. A leading tab in the folding 
part should be replaced by a space.

-- 

  / daniel.haxx.se

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Jeff King]

On Fri, Dec 19, 2025 at 09:47:37AM +0100, Daniel Stenberg wrote:

> On Fri, 19 Dec 2025, Jeff King wrote:
> 
> > > [1] = https://github.com/curl/curl/commit/9941e7c95bf26f00fd87888a
> > 
> > and it fixes the first of Git's failing tests. But if we swap out the
> > space for a tab like this:
> 
> Sorry, that was just sloppy of me to not add a test and proper handling for
> that condition. Allow me to fix that in my end. A leading tab in the folding
> part should be replaced by a space.

Thanks! I ran Git's test suite against a build using your 6c7bc9871f
(http: fix for unfolding line starting with TAB, 2025-12-19) and it
works without the whitespace-relaxing in my third patch.

I also double-checked against the current tip of curl's master, which
includes 3388afd2b6 (http: more unfold fixing, 2025-12-19), and
everything remains fine. Thanks for a prompt fix.


Junio: I think we could just drop the third patch here, if we don't mind
test failures against an unreleased version of curl. It's in debian
unstable now, but presumably they'll move to the released 8.18.0 once
it's out.

-Peff

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Thanks! I ran Git's test suite against a build using your 6c7bc9871f
> (http: fix for unfolding line starting with TAB, 2025-12-19) and it
> works without the whitespace-relaxing in my third patch.
>
> I also double-checked against the current tip of curl's master, which
> includes 3388afd2b6 (http: more unfold fixing, 2025-12-19), and
> everything remains fine. Thanks for a prompt fix.
>
>
> Junio: I think we could just drop the third patch here, if we don't mind
> test failures against an unreleased version of curl. It's in debian
> unstable now, but presumably they'll move to the released 8.18.0 once
> it's out.

I was wondering about the same thing after Daniel started working on
the fix upstream for #3; you caught the behaviour change before it
made to their officially released version, so we do not have to have
a workaround on our end.

Will drop the third patch.  Thanks.

Re: [PATCH 0/3] test-suite fixes for upcoming curl 8.18.0

[Jeff King]

On Thu, Dec 18, 2025 at 01:37:11PM +0100, Daniel Stenberg wrote:

> >  [1/3]: t5551: handle trailing slashes in expected cookies output
> 
> This is all benign. As you correctly observed, we no longer keep the
> "original" cookie path around and only work with the sanitized version - so
> that's the one stored now. It was already the one used for actual
> comparisons so apart from the change in storage, it *should* not cause any
> problems.

OK, good. I did wonder if there might be some subtle behavior change
under the hood, but figured you probably knew what you were doing
(especially since the normalization was the point of that commit, and
not some unexpected side effect).

> >  [2/3]: t5563: add missing end-of-line in HTTP header
> 
> I believe I made some code checks a little stricter: header lines MUST end
> with at CR or LF (or both) to be treated as a valid one. Your fix for this
> should be good also for older libcurl versions.

Makes sense. I think it's accurate to call what our test was doing
garbage that we happened to be lucky was accepted, and the new curl
behavior will not hurt any real world cases.

> >  [3/3]: t5563: relax whitespace assumptions for unfolded headers
> 
> This one is material for me to rethink.
> 
> I had to completely change our header unfolding logic because we learned
> that we did not apply it early enough, so some header parsing was wrongly
> done on pre-unfolded data. In this process, I also changed the logic that
> appends the following line on the previous line. To avoid having to keep a
> state, I decided to just append the second line onto the first one without
> trying to reduce the whitespace characters to a single one.
> 
> I did not fully consider the impact this might have on users such as you.
> Allow me to rework that a little bit further and get the former white-space
> behavior back. Thanks!

I do think you're following the standards in including the extra space,
so that part isn't wrong per se. But it may be kinder to do a bit of
whitespace collapsing. I dunno.

The more fundamental change is that a CURLOPT_HEADERFUNCTION callback is
now fed unfolded headers, rather than getting the lines piecemeal (and
having to do the unfolding itself).

So I'm not sure that we should be worried about a case where old code
preferred the unfolded but whitespace-collapsed headers, and will be
broken if curl does not keep doing that. There was no such code, because
curl was not unfolding at all!

The only code which would confused is a callback that did its own
unfolding and somehow implemented it differently than curl does (which
is what happened here). But every caller should be prepared to take
unfolded data, since after all the server could have avoided folding in
the first place.

So I dunno. While we did see "breakage" here, I am inclined to think it
was mostly about how intimate and brittle the tests were, and that any
real world use would not run into this.

I'd like to think it probably doesn't matter much in the real world
considering the deprecated status of folding in the first place, but
that might be too optimistic. ;)

-Peff