[PATCH v3 0/8] builtin: implement, document and test url-parse

["Matheus Moreira via GitGitGadget" <gitgitgadget@gmail.com>]

This series adds git url-parse, a plumbing builtin for inspecting git URLs.
Git accepts a wider variety of URL forms than any standard parser handles.
The supported forms include RFC URLs, file:// URLs, scp-style
[user@]host:path for SSH, and IPv6 in brackets. Tools wanting to reason
about them have historically had to reimplement git's parsing or shell out
indirectly. With git url-parse, scripts can ask git directly: validate a
URL, extract a component (scheme, user, host, port, path, password), or
both.

The series consists of eight commits.

The first four are preparatory. They rename enum protocol to enum url_scheme
for RFC alignment, move url_is_local_not_ssh and the scheme-detection
routines from connect.c to url.h/url.c, and stop url_get_scheme from dying
on unknown schemes so other parsers can handle unknowns gracefully.

The fifth commit defines the new parser, url_parse, in urlmatch.c. It is
adapted from parse_connect_url and uses the same data structures as
url_normalize. The parser returns NULL on failure with err populated, and
exposes URL components as offset/length pairs into the normalized URL
buffer.

The sixth commit adds the user-facing command, with a helpful error when the
input looks like a local path rather than a URL.

The last two commits are documentation (a manpage) and 53 tests covering URL
form, scp form, IPv6 in URL and scp forms, bracket forms, username
expansion, query/fragment stripping, the local-path error, and
validation-only mode.

Several choices in this series are judgment calls. Happy to amend or follow
up on any of them.

The component name is scheme, not protocol. RFC 1738/3986 calls them
schemes. The series renames enum protocol to enum url_scheme internally, and
the user-facing component name follows the same direction. I considered
accepting both as aliases but decided against the precedent for a new
command. If you would rather see protocol, or both protocol and scheme, that
is easy to change.

Local paths are deliberately not URLs. parse_connect_url accepts bare paths
like /abs/path or ./rel as URL_SCHEME_LOCAL. url_parse rejects them, since
url_normalize requires a scheme://host form, and silent conversion to
file:// has no good answer for relative or tilde forms. The builtin emits a
helpful error suggesting the explicit file:// form. If full git clone parity
is preferred (bare paths accepted via auto-conversion or a new flag), that
could be added.

Absent and empty components are conflated in output. --component user
http://host/ and --component user http://@host/ both produce empty lines.
The underlying struct url_info preserves the distinction: *_off == 0 vs
*_off != 0 with *_len == 0. A future option can expose it without breaking
change. Can amend this patch set if necessary.

Changes since v1:

 * Bug fix: ~user paths with a query string or fragment were leaking the ?
   or # into the path output. The ~user-skip logic in url_parse previously
   ran only for file://. It now runs for git/ssh/scp URLs as well, matching
   what parse_connect_url does and what users expect.

 * Helpful error for local paths instead of the cryptic "invalid URL scheme
   name or missing '://' suffix".

 * -c protocol renamed to -c scheme for consistency with the internal rename
   and the RFC.

 * Documented the deliberate divergence from parse_connect_url (local paths
   and unknown schemes) in the urlmatch commit message.

 * Doc and command-list polish: purehelpers category, asciidoc placeholder
   convention, [synopsis] form.

 * Original micro commit style staged buildup of the builtin collapsed to a
   single self-contained commit. The rest of the series is unchanged in
   shape.

Changes since v2:

 * Fix Windows CI failure: handle DOS drive prefix in the helpful local-path
   error. With this, the message for a drive-letter input like C:/repo (or
   an MSYS-mangled /abs/path that bash rewrites to D:/.../abs/path before
   git sees it) gets the specific file:///<input> suggestion rather than the
   generic fallback. No effect on Linux or macOS, since has_dos_drive_prefix
   is a no-op on non-Windows builds.

 * t9904: relax the grep on the absolute-path test from the literal
   file:///abs/path to the structural file:/// (three slashes). The original
   assertion depended on the input being preserved verbatim, which MSYS does
   not do. The relaxed grep verifies the structurally meaningful property
   (specific URL suggestion was produced, not the generic fallback) and runs
   cross-platform.

Range-diff against v2:

1: 38f797362d = 1: 38f797362d connect: rename enum protocol to url_scheme 2:
a4153e1d24 = 2: a4153e1d24 url: move url_is_local_not_ssh to url.h 3:
e584fb03f3 = 3: e584fb03f3 url: move scheme detection to URL header/source
4: 7381704c38 = 4: 7381704c38 url: return URL_SCHEME_UNKNOWN instead of
dying 5: 89932a70f3 = 5: 89932a70f3 urlmatch: define url_parse function 6:
886a7d659e ! 6: af6c71227b builtin: create url-parse command @@
builtin/url-parse.c (new) + if (*url == '/') + die("'%s' is not a URL; if
you meant a local " + "repository, use 'file://%s'", url, url); ++ if
(has_dos_drive_prefix(url)) ++ die("'%s' is not a URL; if you meant a local
" ++ "repository, use 'file:///%s'", url, url); + die("'%s' is not a URL; if
you meant a local repository, " + "use a 'file://' URL with an absolute
path", url); + } 7: 3c44e0f478 = 7: 2b32cb71a3 doc: describe the url-parse
builtin 8: cf2ae409e6 ! 8: ce41d2ec50 t9904: add tests for the new url-parse
builtin @@ t/t9904-url-parse.sh (new) +test_expect_success 'git url-parse
helpful error for absolute local path' ' + test_must_fail git url-parse
"/abs/path" 2>err && + test_grep "is not a URL" err && -+ test_grep
"file:///abs/path" err ++ test_grep "file:///" err +' + +test_expect_success
'git url-parse helpful error for relative local path' '

Matheus Afonso Martins Moreira (8):
  connect: rename enum protocol to url_scheme
  url: move url_is_local_not_ssh to url.h
  url: move scheme detection to URL header/source
  url: return URL_SCHEME_UNKNOWN instead of dying
  urlmatch: define url_parse function
  builtin: create url-parse command
  doc: describe the url-parse builtin
  t9904: add tests for the new url-parse builtin

 .gitignore                              |   1 +
 Documentation/git-url-parse.adoc        |  80 ++++++
 Documentation/meson.build               |   1 +
 Makefile                                |   1 +
 builtin.h                               |   1 +
 builtin/url-parse.c                     | 135 ++++++++++
 command-list.txt                        |   1 +
 connect.c                               |  78 ++----
 connect.h                               |   1 -
 git.c                                   |   1 +
 meson.build                             |   1 +
 remote.c                                |   1 +
 t/meson.build                           |   1 +
 t/t9904-url-parse.sh                    | 319 ++++++++++++++++++++++++
 t/unit-tests/u-urlmatch-normalization.c |  45 ++++
 url.c                                   |  23 ++
 url.h                                   |  16 ++
 urlmatch.c                              | 127 ++++++++++
 urlmatch.h                              |   1 +
 19 files changed, 780 insertions(+), 54 deletions(-)
 create mode 100644 Documentation/git-url-parse.adoc
 create mode 100644 builtin/url-parse.c
 create mode 100755 t/t9904-url-parse.sh


base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1715%2Fmatheusmoreira%2Furl-parse-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1715/matheusmoreira/url-parse-v3
Pull-Request: https://github.com/git/git/pull/1715

Range-diff vs v2:

 1:  38f797362d = 1:  38f797362d connect: rename enum protocol to url_scheme
 2:  a4153e1d24 = 2:  a4153e1d24 url: move url_is_local_not_ssh to url.h
 3:  e584fb03f3 = 3:  e584fb03f3 url: move scheme detection to URL header/source
 4:  7381704c38 = 4:  7381704c38 url: return URL_SCHEME_UNKNOWN instead of dying
 5:  89932a70f3 = 5:  89932a70f3 urlmatch: define url_parse function
 6:  886a7d659e ! 6:  af6c71227b builtin: create url-parse command
     @@ builtin/url-parse.c (new)
      +		if (*url == '/')
      +			die("'%s' is not a URL; if you meant a local "
      +			    "repository, use 'file://%s'", url, url);
     ++		if (has_dos_drive_prefix(url))
     ++			die("'%s' is not a URL; if you meant a local "
     ++			    "repository, use 'file:///%s'", url, url);
      +		die("'%s' is not a URL; if you meant a local repository, "
      +		    "use a 'file://' URL with an absolute path", url);
      +	}
 7:  3c44e0f478 = 7:  2b32cb71a3 doc: describe the url-parse builtin
 8:  cf2ae409e6 ! 8:  ce41d2ec50 t9904: add tests for the new url-parse builtin
     @@ t/t9904-url-parse.sh (new)
      +test_expect_success 'git url-parse helpful error for absolute local path' '
      +	test_must_fail git url-parse "/abs/path" 2>err &&
      +	test_grep "is not a URL" err &&
     -+	test_grep "file:///abs/path" err
     ++	test_grep "file:///" err
      +'
      +
      +test_expect_success 'git url-parse helpful error for relative local path' '

-- 
gitgitgadget