[PATCH v2 0/8] builtin: implement, document and test url-parse

["Matheus Moreira via GitGitGadget" <gitgitgadget@gmail.com>]

This series adds git url-parse, a plumbing builtin for inspecting git URLs.
Git accepts a wider variety of URL forms than any standard parser handles.
The supported forms include RFC URLs, file:// URLs, scp-style
[user@]host:path for SSH, and IPv6 in brackets. Tools wanting to reason
about them have historically had to reimplement git's parsing or shell out
indirectly. With git url-parse, scripts can ask git directly: validate a
URL, extract a component (scheme, user, host, port, path, password), or
both.

The series consists of eight commits.

The first four are preparatory. They rename enum protocol to enum url_scheme
for RFC alignment, move url_is_local_not_ssh and the scheme-detection
routines from connect.c to url.h/url.c, and stop url_get_scheme from dying
on unknown schemes so other parsers can handle unknowns gracefully.

The fifth commit defines the new parser, url_parse, in urlmatch.c. It is
adapted from parse_connect_url and uses the same data structures as
url_normalize. The parser returns NULL on failure with err populated, and
exposes URL components as offset/length pairs into the normalized URL
buffer.

The sixth commit adds the user-facing command, with a helpful error when the
input looks like a local path rather than a URL.

The last two commits are documentation (a manpage) and 53 tests covering URL
form, scp form, IPv6 in URL and scp forms, bracket forms, username
expansion, query/fragment stripping, the local-path error, and
validation-only mode.

Several choices in this series are judgment calls. Happy to amend or follow
up on any of them.

The component name is scheme, not protocol. RFC 1738/3986 calls them
schemes. The series renames enum protocol to enum url_scheme internally, and
the user-facing component name follows the same direction. I considered
accepting both as aliases but decided against the precedent for a new
command. If you would rather see protocol, or both protocol and scheme, that
is easy to change.

Local paths are deliberately not URLs. parse_connect_url accepts bare paths
like /abs/path or ./rel as URL_SCHEME_LOCAL. url_parse rejects them, since
url_normalize requires a scheme://host form, and silent conversion to
file:// has no good answer for relative or tilde forms. The builtin emits a
helpful error suggesting the explicit file:// form. If full git clone parity
is preferred (bare paths accepted via auto-conversion or a new flag), that
could be added.

Absent and empty components are conflated in output. --component user
http://host/ and --component user http://@host/ both produce empty lines.
The underlying struct url_info preserves the distinction: *_off == 0 vs
*_off != 0 with *_len == 0. A future option can expose it without breaking
change. Can amend this patch set if necessary.

Changes since v1:

 * Bug fix: ~user paths with a query string or fragment were leaking the ?
   or # into the path output. The ~user-skip logic in url_parse previously
   ran only for file://. It now runs for git/ssh/scp URLs as well, matching
   what parse_connect_url does and what users expect.

 * Helpful error for local paths instead of the cryptic "invalid URL scheme
   name or missing '://' suffix".

 * -c protocol renamed to -c scheme for consistency with the internal rename
   and the RFC.

 * Documented the deliberate divergence from parse_connect_url (local paths
   and unknown schemes) in the urlmatch commit message.

 * Doc and command-list polish: purehelpers category, asciidoc placeholder
   convention, [synopsis] form.

 * Original micro commit style staged buildup of the builtin collapsed to a
   single self-contained commit. The rest of the series is unchanged in
   shape.

Matheus Afonso Martins Moreira (8):
  connect: rename enum protocol to url_scheme
  url: move url_is_local_not_ssh to url.h
  url: move scheme detection to URL header/source
  url: return URL_SCHEME_UNKNOWN instead of dying
  urlmatch: define url_parse function
  builtin: create url-parse command
  doc: describe the url-parse builtin
  t9904: add tests for the new url-parse builtin

 .gitignore                              |   1 +
 Documentation/git-url-parse.adoc        |  80 ++++++
 Documentation/meson.build               |   1 +
 Makefile                                |   1 +
 builtin.h                               |   1 +
 builtin/url-parse.c                     | 132 ++++++++++
 command-list.txt                        |   1 +
 connect.c                               |  78 ++----
 connect.h                               |   1 -
 git.c                                   |   1 +
 meson.build                             |   1 +
 remote.c                                |   1 +
 t/meson.build                           |   1 +
 t/t9904-url-parse.sh                    | 319 ++++++++++++++++++++++++
 t/unit-tests/u-urlmatch-normalization.c |  45 ++++
 url.c                                   |  23 ++
 url.h                                   |  16 ++
 urlmatch.c                              | 127 ++++++++++
 urlmatch.h                              |   1 +
 19 files changed, 777 insertions(+), 54 deletions(-)
 create mode 100644 Documentation/git-url-parse.adoc
 create mode 100644 builtin/url-parse.c
 create mode 100755 t/t9904-url-parse.sh


base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1715%2Fmatheusmoreira%2Furl-parse-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1715/matheusmoreira/url-parse-v2
Pull-Request: https://github.com/git/git/pull/1715

Range-diff vs v1:

  -:  ---------- >  1:  38f797362d connect: rename enum protocol to url_scheme
  1:  42eb0cbf68 !  2:  a4153e1d24 url: move helper function to URL header and source
     @@ Metadata
      Author: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
       ## Commit message ##
     -    url: move helper function to URL header and source
     +    url: move url_is_local_not_ssh to url.h
      
     -    It will be used in more places so it should be placed in url.h.
     +    Move url_is_local_not_ssh from connect.c/connect.h
     +    to url.c/url.h so that the new url_parse function
     +    in urlmatch.c, and any future code that needs to
     +    distinguish a local path from an scp style SSH URL,
     +    can reuse the heuristic without depending on connect.c.
     +
     +    No behavior change.
      
          Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
       ## connect.c ##
     -@@ connect.c: enum protocol {
     - 	PROTO_GIT
     +@@ connect.c: enum url_scheme {
     + 	URL_SCHEME_GIT
       };
       
      -int url_is_local_not_ssh(const char *url)
     @@ connect.c: enum protocol {
      -		(has_dos_drive_prefix(url) && is_valid_path(url));
      -}
      -
     - static const char *prot_name(enum protocol protocol)
     + static const char *url_scheme_name(enum url_scheme scheme)
       {
     - 	switch (protocol) {
     + 	switch (scheme) {
      
       ## connect.h ##
      @@ connect.h: int git_connection_is_socket(struct child_process *conn);
     @@ url.h: char *url_decode_parameter_value(const char **query);
       
      +int url_is_local_not_ssh(const char *url);
      +
     - #endif /* URL_H */
     + /*
     +  * The set of unreserved characters as per STD66 (RFC3986) is
     +  * '[A-Za-z0-9-._~]'. These characters are safe to appear in URI
  -:  ---------- >  3:  e584fb03f3 url: move scheme detection to URL header/source
  -:  ---------- >  4:  7381704c38 url: return URL_SCHEME_UNKNOWN instead of dying
  2:  13b81b8aa0 !  5:  89932a70f3 urlmatch: define url_parse function
     @@ Metadata
       ## Commit message ##
          urlmatch: define url_parse function
      
     -    Define general parsing function that supports all Git URLs
     +    Define url_parse, a general parsing function that supports all Git URLs
          including scp style URLs such as hostname:~user/repo.
     -    Has the same interface as the URL normalization function
     -    and uses the same data structures, facilitating its use.
     -    It's adapted from the algorithm used to process URLs in connect.c,
     -    so it should support the same inputs.
     +
     +    It is adapted from the algorithm in connect.c's parse_connect_url
     +    and reuses the shared enum url_scheme and url_get_scheme function
     +    that previous commits made available in url.h. The new parser and
     +    the connect path agree on scheme classification. url_parse has the
     +    same interface as url_normalize and uses the same data structures.
     +
     +    Both functions accept the same URL forms with one deliberate
     +    exception. Bare local paths such as "/abs/path", "./rel"
     +    or "repo" are accepted by parse_connect_url as URL_SCHEME_LOCAL,
     +    but rejected by url_parse because url_normalize requires a URL
     +    with a scheme://host form. A consumer that wants to handle both
     +    URLs and local paths needs to dispatch on url_is_local_not_ssh
     +    before calling url_parse, just as the connect path does internally.
     +
     +    The duplication with parse_connect_url is intentional.
     +    The two functions have different contracts:
     +
     +      - parse_connect_url
     +
     +        Calls die() on an unknown scheme
     +        and returns NUL-terminated host/path
     +        strings for the connect path
     +
     +      - url_parse
     +
     +        Returns NULL on failure while populating
     +        out_info->err, and exposes components
     +        as offset/length pairs into the normalized
     +        URL buffer, matching url_normalize.
     +
     +    Reconciling both is possible, but not in the scope
     +    of the current patch set.
      
          Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
     + ## t/unit-tests/u-urlmatch-normalization.c ##
     +@@ t/unit-tests/u-urlmatch-normalization.c: void test_urlmatch_normalization__equivalents(void)
     + 	compare_normalized_urls("https://@x.y/^/../abc", "httpS://@x.y:0443/abc", 1);
     + 	compare_normalized_urls("https://@x.y/^/..", "httpS://@x.y:0443/", 1);
     + }
     ++
     ++static void check_parsed_path(const char *url, const char *expected_path)
     ++{
     ++	struct url_info info;
     ++	char *parsed = url_parse(url, &info);
     ++	char *path;
     ++
     ++	cl_assert(parsed != NULL);
     ++	path = xstrndup(parsed + info.path_off, info.path_len);
     ++	cl_assert_equal_s(path, expected_path);
     ++	free(path);
     ++	free(parsed);
     ++}
     ++
     ++void test_urlmatch_normalization__parse_scp(void)
     ++{
     ++	check_parsed_path("host:path", "/path");
     ++	check_parsed_path("user@host:path", "/path");
     ++	check_parsed_path("host:~user/repo", "~user/repo");
     ++	check_parsed_path("user@host:~user/repo", "~user/repo");
     ++	check_parsed_path("[host]:src", "/src");
     ++	check_parsed_path("[host:123]:src", "/src");
     ++	check_parsed_path("[::1]:repo", "/repo");
     ++	check_parsed_path("user@[::1]:repo", "/repo");
     ++}
     ++
     ++void test_urlmatch_normalization__parse_url_form(void)
     ++{
     ++	check_parsed_path("ssh://host/repo", "/repo");
     ++	check_parsed_path("ssh://host/~user/repo", "~user/repo");
     ++	check_parsed_path("git://host:9418/repo", "/repo");
     ++	check_parsed_path("git://host/~user/repo", "~user/repo");
     ++	check_parsed_path("ssh://[::1]:1234/repo", "/repo");
     ++	check_parsed_path("http://[2001:db8::1]/repo", "/repo");
     ++}
     ++
     ++void test_urlmatch_normalization__parse_strips_query_and_fragment(void)
     ++{
     ++	check_parsed_path("ssh://host/~user/repo?q", "~user/repo");
     ++	check_parsed_path("ssh://host/~user/repo#frag", "~user/repo");
     ++	check_parsed_path("git://host/~user/repo?q", "~user/repo");
     ++	check_parsed_path("user@host:~user/repo?q", "~user/repo");
     ++	check_parsed_path("https://host/repo?q", "/repo");
     ++	check_parsed_path("https://host/repo#frag", "/repo");
     ++}
     +
       ## urlmatch.c ##
      @@
       #include "hex-ll.h"
     @@ urlmatch.c: char *url_normalize(const char *url, struct url_info *out_info)
       	return url_normalize_1(url, out_info, 0);
       }
       
     -+enum protocol {
     -+	PROTO_UNKNOWN = 0,
     -+	PROTO_LOCAL,
     -+	PROTO_FILE,
     -+	PROTO_SSH,
     -+	PROTO_GIT,
     -+};
     -+
     -+static enum protocol url_get_protocol(const char *name, size_t n)
     -+{
     -+	if (!strncmp(name, "ssh", n))
     -+		return PROTO_SSH;
     -+	if (!strncmp(name, "git", n))
     -+		return PROTO_GIT;
     -+	if (!strncmp(name, "git+ssh", n)) /* deprecated - do not use */
     -+		return PROTO_SSH;
     -+	if (!strncmp(name, "ssh+git", n)) /* deprecated - do not use */
     -+		return PROTO_SSH;
     -+	if (!strncmp(name, "file", n))
     -+		return PROTO_FILE;
     -+	return PROTO_UNKNOWN;
     -+}
     -+
      +char *url_parse(const char *url_orig, struct url_info *out_info)
      +{
      +	struct strbuf url;
      +	char *host, *separator;
      +	char *detached, *normalized;
     -+	enum protocol protocol = PROTO_LOCAL;
     ++	char *url_decoded;
     ++	enum url_scheme scheme = URL_SCHEME_LOCAL;
      +	struct url_info local_info;
     -+	struct url_info *info = out_info? out_info : &local_info;
     ++	struct url_info *info = out_info ? out_info : &local_info;
      +	bool scp_syntax = false;
      +
     -+	if (is_url(url_orig)) {
     -+		url_orig = url_decode(url_orig);
     -+	} else {
     -+		url_orig = xstrdup(url_orig);
     -+	}
     ++	if (is_url(url_orig))
     ++		url_decoded = url_decode(url_orig);
     ++	else
     ++		url_decoded = xstrdup(url_orig);
      +
     -+	strbuf_init(&url, strlen(url_orig) + sizeof("ssh://"));
     -+	strbuf_addstr(&url, url_orig);
     ++	strbuf_init(&url, strlen(url_decoded) + sizeof("ssh://"));
     ++	strbuf_addstr(&url, url_decoded);
     ++	free(url_decoded);
      +
      +	host = strstr(url.buf, "://");
      +	if (host) {
     -+		protocol = url_get_protocol(url.buf, host - url.buf);
     ++		/*
     ++		 * Temporarily NUL-terminate the scheme name
     ++		 * so we can pass it to url_get_scheme(),
     ++		 * then restore the ':' so the buffer
     ++		 * is intact for url_normalize() below.
     ++		 */
     ++		char saved = *host;
     ++		*host = '\0';
     ++		scheme = url_get_scheme(url.buf);
     ++		*host = saved;
      +		host += 3;
      +	} else {
      +		if (!url_is_local_not_ssh(url.buf)) {
      +			scp_syntax = true;
     -+			protocol = PROTO_SSH;
     ++			scheme = URL_SCHEME_SSH;
      +			strbuf_insertstr(&url, 0, "ssh://");
     -+			host = url.buf + 6;
     ++			host = url.buf + strlen("ssh://");
      +		}
      +	}
      +
     -+	/* path starts after ':' in scp style SSH URLs */
     ++	/*
     ++	 * Path starts after ':' in scp style SSH URLs.
     ++	 *
     ++	 * The host portion can begin with an optional "user@",
     ++	 * and the host itself can be wrapped in '[' ']' brackets.
     ++	 * The bracket form is git's legacy way of supporting:
     ++	 *
     ++	 *   - IPv6 literals: [::1]:repo
     ++	 *   - host:port pairs in the short form: [myhost:123]:src
     ++	 *   - Plain hostnames that happen to need bracketing: [host]:path
     ++	 *
     ++	 * Treat '[' followed by 0 or 1 inner colons as the host:port
     ++	 * or plain hostname form and strip the brackets so url_normalize
     ++	 * sees host[:port] natively. Two or more inner colons mark an
     ++	 * IPv6 literal: keep the brackets for url_normalize to recognize.
     ++	 *
     ++	 * The scp path separator is the ':' that follows the host part,
     ++	 * and we must skip over user@ and any '[...]' before searching.
     ++	 */
      +	if (scp_syntax) {
     -+		separator = strchr(host, ':');
     ++		char *user_at;
     ++		char *host_start;
     ++		char *bracket_end;
     ++
     ++		user_at = strchr(host, '@');
     ++		host_start = user_at ? user_at + 1 : host;
     ++
     ++		if (*host_start == '[') {
     ++			char *p;
     ++			int inner_colons;
     ++
     ++			bracket_end = strchr(host_start, ']');
     ++			inner_colons = 0;
     ++			for (p = host_start + 1; bracket_end && p < bracket_end; p++)
     ++				if (*p == ':')
     ++					inner_colons++;
     ++
     ++			if (bracket_end && inner_colons <= 1) {
     ++				size_t close_off = bracket_end - url.buf;
     ++				size_t open_off = host_start - url.buf;
     ++				strbuf_remove(&url, close_off, 1);
     ++				strbuf_remove(&url, open_off, 1);
     ++				separator = url.buf + close_off - 1;
     ++			} else if (bracket_end) {
     ++				separator = strchr(bracket_end + 1, ':');
     ++			} else {
     ++				separator = strchr(host_start, ':');
     ++			}
     ++		} else {
     ++			separator = strchr(host_start, ':');
     ++		}
     ++
      +		if (separator) {
      +			if (separator[1] == '/')
      +				strbuf_remove(&url, separator - url.buf, 1);
     @@ urlmatch.c: char *url_normalize(const char *url, struct url_info *out_info)
      +	normalized = url_normalize(detached, info);
      +	free(detached);
      +
     -+	if (!normalized) {
     ++	if (!normalized)
      +		return NULL;
     -+	}
      +
     -+	/* point path to ~ for URL's like this:
     ++	/*
     ++	 * Point path to ~ for URLs like this:
      +	 *
      +	 *     ssh://host.xz/~user/repo
      +	 *     git://host.xz/~user/repo
      +	 *     host.xz:~user/repo
     -+	 *
      +	 */
     -+	if (protocol == PROTO_GIT || protocol == PROTO_SSH) {
     -+		if (normalized[info->path_off + 1] == '~')
     ++	if (scheme == URL_SCHEME_GIT || scheme == URL_SCHEME_SSH) {
     ++		if (normalized[info->path_off + 1] == '~') {
      +			info->path_off++;
     ++			info->path_len--;
     ++		}
      +	}
      +
      +	return normalized;
  3:  e4781b36d5 !  6:  886a7d659e builtin: create url-parse command
     @@ Commit message
      
          The url-parse builtin command is designed to solve this problem
          by exposing git's native URL parsing facilities as a plumbing command.
     -    Other programs can then call upon git itself to parse the git URLs and
     -    extract their components. This should be quite useful for scripts.
     +    Other programs can then call upon git itself to parse the git URLs
     +    and extract their components. This should be quite useful for scripts.
      
          Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
     @@ Makefile: BUILTIN_OBJS += builtin/update-ref.o
       BUILTIN_OBJS += builtin/verify-pack.o
      
       ## builtin.h ##
     -@@ builtin.h: int cmd_update_server_info(int argc, const char **argv, const char *prefix);
     - int cmd_upload_archive(int argc, const char **argv, const char *prefix);
     - int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix);
     - int cmd_upload_pack(int argc, const char **argv, const char *prefix);
     -+int cmd_url_parse(int argc, const char **argv, const char *prefix);
     - int cmd_var(int argc, const char **argv, const char *prefix);
     - int cmd_verify_commit(int argc, const char **argv, const char *prefix);
     - int cmd_verify_tag(int argc, const char **argv, const char *prefix);
     +@@ builtin.h: int cmd_update_server_info(int argc, const char **argv, const char *prefix, stru
     + int cmd_upload_archive(int argc, const char **argv, const char *prefix, struct repository *repo);
     + int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix, struct repository *repo);
     + int cmd_upload_pack(int argc, const char **argv, const char *prefix, struct repository *repo);
     ++int cmd_url_parse(int argc, const char **argv, const char *prefix, struct repository *repo);
     + int cmd_var(int argc, const char **argv, const char *prefix, struct repository *repo);
     + int cmd_verify_commit(int argc, const char **argv, const char *prefix, struct repository *repo);
     + int cmd_verify_tag(int argc, const char **argv, const char *prefix, struct repository *repo);
      
       ## builtin/url-parse.c (new) ##
      @@
     -+/* SPDX-License-Identifier: GPL-2.0-only
     -+ *
     -+ * url-parse - parses git URLs and extracts their components
     -+ *
     -+ * Copyright © 2024 Matheus Afonso Martins Moreira
     -+ *
     -+ * This program is free software; you can redistribute it and/or modify
     -+ * it under the terms of the GNU General Public License as published by
     -+ * the Free Software Foundation; version 2.
     -+ */
     -+
      +#include "builtin.h"
      +#include "gettext.h"
     ++#include "parse-options.h"
     ++#include "url.h"
     ++#include "urlmatch.h"
     ++
     ++static const char * const builtin_url_parse_usage[] = {
     ++	N_("git url-parse [-c <component>] [--] <url>..."),
     ++	NULL
     ++};
     ++
     ++static char *component_arg;
     ++
     ++static struct option builtin_url_parse_options[] = {
     ++	OPT_STRING('c', "component", &component_arg, N_("component"),
     ++		N_("which URL component to extract")),
     ++	OPT_END(),
     ++};
     ++
     ++enum url_component {
     ++	URL_NONE = 0,
     ++	URL_SCHEME,
     ++	URL_USER,
     ++	URL_PASSWORD,
     ++	URL_HOST,
     ++	URL_PORT,
     ++	URL_PATH,
     ++};
     ++
     ++static void parse_or_die(const char *url, struct url_info *info)
     ++{
     ++	if (url_is_local_not_ssh(url)) {
     ++		if (*url == '/')
     ++			die("'%s' is not a URL; if you meant a local "
     ++			    "repository, use 'file://%s'", url, url);
     ++		die("'%s' is not a URL; if you meant a local repository, "
     ++		    "use a 'file://' URL with an absolute path", url);
     ++	}
     ++	if (!url_parse(url, info))
     ++		die("invalid git URL '%s': %s", url, info->err);
     ++}
     ++
     ++static enum url_component get_component_or_die(const char *arg)
     ++{
     ++	if (!strcmp("path", arg))
     ++		return URL_PATH;
     ++	if (!strcmp("host", arg))
     ++		return URL_HOST;
     ++	if (!strcmp("scheme", arg))
     ++		return URL_SCHEME;
     ++	if (!strcmp("user", arg))
     ++		return URL_USER;
     ++	if (!strcmp("password", arg))
     ++		return URL_PASSWORD;
     ++	if (!strcmp("port", arg))
     ++		return URL_PORT;
     ++	die("invalid git URL component '%s'", arg);
     ++}
     ++
     ++static char *extract_component(enum url_component component,
     ++			       struct url_info *info)
     ++{
     ++	size_t offset, length;
     ++
     ++	switch (component) {
     ++	case URL_SCHEME:
     ++		offset = 0;
     ++		length = info->scheme_len;
     ++		break;
     ++	case URL_USER:
     ++		offset = info->user_off;
     ++		length = info->user_len;
     ++		break;
     ++	case URL_PASSWORD:
     ++		offset = info->passwd_off;
     ++		length = info->passwd_len;
     ++		break;
     ++	case URL_HOST:
     ++		offset = info->host_off;
     ++		length = info->host_len;
     ++		break;
     ++	case URL_PORT:
     ++		offset = info->port_off;
     ++		length = info->port_len;
     ++		break;
     ++	case URL_PATH:
     ++		offset = info->path_off;
     ++		length = info->path_len;
     ++		break;
     ++	case URL_NONE:
     ++		return NULL;
     ++	}
     ++
     ++	return xstrndup(info->url + offset, length);
     ++}
      +
     -+int cmd_url_parse(int argc, const char **argv, const char *prefix)
     ++int cmd_url_parse(int argc,
     ++		  const char **argv,
     ++		  const char *prefix,
     ++		  struct repository *repo UNUSED)
      +{
     ++	struct url_info info;
     ++	enum url_component selected = URL_NONE;
     ++	char *extracted;
     ++	int i;
     ++
     ++	argc = parse_options(argc, argv, prefix, builtin_url_parse_options,
     ++			     builtin_url_parse_usage, 0);
     ++
     ++	if (argc == 0)
     ++		usage_with_options(builtin_url_parse_usage,
     ++				   builtin_url_parse_options);
     ++
     ++	if (component_arg)
     ++		selected = get_component_or_die(component_arg);
     ++
     ++	for (i = 0; i < argc; i++) {
     ++		parse_or_die(argv[i], &info);
     ++
     ++		if (selected != URL_NONE) {
     ++			extracted = extract_component(selected, &info);
     ++			if (extracted) {
     ++				puts(extracted);
     ++				free(extracted);
     ++			}
     ++		}
     ++
     ++		free(info.url);
     ++	}
     ++
      +	return 0;
      +}
      
     @@ command-list.txt: git-update-ref                          plumbingmanipulators
       git-update-server-info                  synchingrepositories
       git-upload-archive                      synchelpers
       git-upload-pack                         synchelpers
     -+git-url-parse                           plumbinginterrogators
     ++git-url-parse                           purehelpers
       git-var                                 plumbinginterrogators
       git-verify-commit                       ancillaryinterrogators
       git-verify-pack                         plumbinginterrogators
     @@ git.c: static struct cmd_struct commands[] = {
       	{ "upload-archive", cmd_upload_archive, NO_PARSEOPT },
       	{ "upload-archive--writer", cmd_upload_archive_writer, NO_PARSEOPT },
       	{ "upload-pack", cmd_upload_pack },
     -+	{ "url-parse", cmd_url_parse, NO_PARSEOPT },
     ++	{ "url-parse", cmd_url_parse },
       	{ "var", cmd_var, RUN_SETUP_GENTLY | NO_PARSEOPT },
       	{ "verify-commit", cmd_verify_commit, RUN_SETUP },
       	{ "verify-pack", cmd_verify_pack },
     +
     + ## meson.build ##
     +@@ meson.build: builtin_sources = [
     +   'builtin/update-server-info.c',
     +   'builtin/upload-archive.c',
     +   'builtin/upload-pack.c',
     ++  'builtin/url-parse.c',
     +   'builtin/var.c',
     +   'builtin/verify-commit.c',
     +   'builtin/verify-pack.c',
  4:  1e0895651c <  -:  ---------- url-parse: add URL parsing helper function
  5:  0bf83ee122 <  -:  ---------- url-parse: enumerate possible URL components
  6:  149c476b1e <  -:  ---------- url-parse: define component extraction helper fn
  7:  eb9ef8a17b <  -:  ---------- url-parse: define string to component converter fn
  8:  a2acfdbc76 <  -:  ---------- url-parse: define usage and options
  9:  5de00324fb <  -:  ---------- url-parse: parse options given on the command line
 10:  15d355a43c <  -:  ---------- url-parse: validate all given git URLs
 11:  4e93509c80 <  -:  ---------- url-parse: output URL components selected by user
 12:  abda074aee !  7:  3c44e0f478 Documentation: describe the url-parse builtin
     @@ Metadata
      Author: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
       ## Commit message ##
     -    Documentation: describe the url-parse builtin
     +    doc: describe the url-parse builtin
      
          The new url-parse builtin validates git URLs
          and optionally extracts their components.
      
     +    Helped-by: Ghanshyam Thakkar <shyamthakkar001@gmail.com>
          Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
     - ## Documentation/git-url-parse.txt (new) ##
     + ## Documentation/git-url-parse.adoc (new) ##
      @@
      +git-url-parse(1)
      +================
     @@ Documentation/git-url-parse.txt (new)
      +
      +SYNOPSIS
      +--------
     -+[verse]
     -+'git url-parse' [<options>] [--] <url>...
     ++[synopsis]
     ++git url-parse [-c <component>] [--] <url>...
      +
      +DESCRIPTION
      +-----------
     @@ Documentation/git-url-parse.txt (new)
      +This command eases interoperability with git URLs by enabling the
      +parsing and extraction of the components of all git URLs.
      +
     ++Any syntactically valid URL is parsed, even if the scheme is not one
     ++git supports for fetching or pushing.
     ++
      +OPTIONS
      +-------
      +
     -+-c <arg>::
     -+--component <arg>::
     -+	Extract the `<arg>` component from the given git URLs.
     -+	`<arg>` can be one of:
     -+	`protocol`, `user`, `password`, `host`, `port`, `path`.
     ++`-c <component>`::
     ++`--component <component>`::
     ++	Extract the _<component>_ component from the given Git URLs.
     ++	_<component>_ can be one of:
     ++	`scheme`, `user`, `password`, `host`, `port`, `path`.
     ++
     ++OUTPUT
     ++------
     ++
     ++When `--component` is given, the requested component of each URL
     ++is printed on its own line, in the order the URLs were given. If
     ++the URL has no such component (for example, a port in a URL that
     ++does not specify one), an empty line is printed in its place.
     ++
     ++When `--component` is not given, no output is produced. The exit
     ++status is zero if every URL parses successfully and non-zero
     ++otherwise, allowing the command to be used purely as a validator.
      +
      +EXAMPLES
      +--------
     @@ Documentation/git-url-parse.txt (new)
      ++
      +------------
      +$ git url-parse --component path https://example.com/user/repo
     -+/usr/repo
     ++/user/repo
      +$ git url-parse --component path example.com:~user/repo
      +~user/repo
      +$ git url-parse --component path example.com:user/repo
     @@ Documentation/git-url-parse.txt (new)
      +$ git url-parse https://example.com/user/repo example.com:~user/repo
      +------------
      +
     ++SEE ALSO
     ++--------
     ++linkgit:git-clone[1],
     ++linkgit:git-fetch[1],
     ++linkgit:git-config[1]
     ++
      +GIT
      +---
      +Part of the linkgit:git[1] suite
     +
     + ## Documentation/meson.build ##
     +@@ Documentation/meson.build: manpages = {
     +   'git-update-server-info.adoc' : 1,
     +   'git-upload-archive.adoc' : 1,
     +   'git-upload-pack.adoc' : 1,
     ++  'git-url-parse.adoc' : 1,
     +   'git-var.adoc' : 1,
     +   'git-verify-commit.adoc' : 1,
     +   'git-verify-pack.adoc' : 1,
 13:  33e128496b !  8:  cf2ae409e6 tests: add tests for the new url-parse builtin
     @@ Metadata
      Author: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
       ## Commit message ##
     -    tests: add tests for the new url-parse builtin
     +    t9904: add tests for the new url-parse builtin
      
          Test git URL parsing, validation and component extraction
          on all documented git URL schemes and syntaxes.
      
     +    Add IPv6 host coverage in URL form:
     +
     +        ssh://[::1]/path
     +        ssh://user@[::1]:1234/path
     +        git://[::1]:9418/path
     +        http://[2001:db8::1]/path
     +        https://[2001:db8::1]/path
     +
     +    In URL form the brackets are kept in the host component (RFC 3986
     +    syntax for IPv6 literals).
     +
     +    Also exercise the bracketed scp short forms that t5601-clone.sh
     +    covers via parse_connect_url:
     +
     +        [host]:path
     +        [host:port]:path
     +        [::1]:repo
     +        user@[::1]:repo
     +        user@[host:port]:path
     +
     +    In scp form, brackets are kept for IPv6 literals (two or more inner
     +    colons) and stripped for plain hostnames or host:port pairs.
     +
     +    Suggested-by: Torsten Bögershausen <tboegi@web.de>
          Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
      
     + ## t/meson.build ##
     +@@ t/meson.build: integration_tests = [
     +   't9901-git-web--browse.sh',
     +   't9902-completion.sh',
     +   't9903-bash-prompt.sh',
     ++  't9904-url-parse.sh',
     + ]
     + 
     + benchmarks = [
     +
       ## t/t9904-url-parse.sh (new) ##
      @@
      +#!/bin/sh
      +#
     -+# Copyright © 2024 Matheus Afonso Martins Moreira
     ++# Copyright (c) 2024 Matheus Afonso Martins Moreira
      +#
      +
      +test_description='git url-parse tests'
     @@ t/t9904-url-parse.sh (new)
      +
      +test_expect_success 'git url-parse -- file urls' '
      +	git url-parse "file:///repository/path" &&
     -+	git url-parse "file:///" &&
      +	git url-parse "file://"
      +'
      +
     -+test_expect_success 'git url-parse -c protocol -- ssh syntax' '
     -+	test ssh = "$(git url-parse -c protocol "ssh://user@example.com:1234/repository/path")" &&
     -+	test ssh = "$(git url-parse -c protocol "ssh://user@example.com/repository/path")" &&
     -+	test ssh = "$(git url-parse -c protocol "ssh://example.com:1234/repository/path")" &&
     -+	test ssh = "$(git url-parse -c protocol "ssh://example.com/repository/path")"
     ++test_expect_success 'git url-parse -c scheme -- ssh syntax' '
     ++	test ssh = "$(git url-parse -c scheme "ssh://user@example.com:1234/repository/path")" &&
     ++	test ssh = "$(git url-parse -c scheme "ssh://user@example.com/repository/path")" &&
     ++	test ssh = "$(git url-parse -c scheme "ssh://example.com:1234/repository/path")" &&
     ++	test ssh = "$(git url-parse -c scheme "ssh://example.com/repository/path")"
      +'
      +
     -+test_expect_success 'git url-parse -c protocol -- git syntax' '
     -+	test git = "$(git url-parse -c protocol "git://example.com:1234/repository/path")" &&
     -+	test git = "$(git url-parse -c protocol "git://example.com/repository/path")"
     ++test_expect_success 'git url-parse -c scheme -- git syntax' '
     ++	test git = "$(git url-parse -c scheme "git://example.com:1234/repository/path")" &&
     ++	test git = "$(git url-parse -c scheme "git://example.com/repository/path")"
      +'
      +
     -+test_expect_success 'git url-parse -c protocol -- http syntax' '
     -+	test https = "$(git url-parse -c protocol "https://example.com:1234/repository/path")" &&
     -+	test https = "$(git url-parse -c protocol "https://example.com/repository/path")" &&
     -+	test http = "$(git url-parse -c protocol "http://example.com:1234/repository/path")" &&
     -+	test http = "$(git url-parse -c protocol "http://example.com/repository/path")"
     ++test_expect_success 'git url-parse -c scheme -- http syntax' '
     ++	test https = "$(git url-parse -c scheme "https://example.com:1234/repository/path")" &&
     ++	test https = "$(git url-parse -c scheme "https://example.com/repository/path")" &&
     ++	test http = "$(git url-parse -c scheme "http://example.com:1234/repository/path")" &&
     ++	test http = "$(git url-parse -c scheme "http://example.com/repository/path")"
      +'
      +
     -+test_expect_success 'git url-parse -c protocol -- scp syntax' '
     -+	test ssh = "$(git url-parse -c protocol "user@example.com:/repository/path")" &&
     -+	test ssh = "$(git url-parse -c protocol "example.com:/repository/path")"
     ++test_expect_success 'git url-parse -c scheme -- scp syntax' '
     ++	test ssh = "$(git url-parse -c scheme "user@example.com:/repository/path")" &&
     ++	test ssh = "$(git url-parse -c scheme "example.com:/repository/path")"
      +'
      +
      +test_expect_success 'git url-parse -c user -- ssh syntax' '
     @@ t/t9904-url-parse.sh (new)
      +	test "" = "$(git url-parse -c user "example.com:/repository/path")"
      +'
      +
     ++test_expect_success 'git url-parse -c password -- http syntax' '
     ++	test secret = "$(git url-parse -c password "https://user:secret@example.com:1234/repository/path")" &&
     ++	test secret = "$(git url-parse -c password "http://user:secret@example.com/repository/path")" &&
     ++	test "" = "$(git url-parse -c password "https://user@example.com/repository/path")" &&
     ++	test "" = "$(git url-parse -c password "https://example.com/repository/path")"
     ++'
     ++
      +test_expect_success 'git url-parse -c host -- ssh syntax' '
      +	test example.com = "$(git url-parse -c host "ssh://user@example.com:1234/repository/path")" &&
      +	test example.com = "$(git url-parse -c host "ssh://user@example.com/repository/path")" &&
     @@ t/t9904-url-parse.sh (new)
      +	test "~user/repository" = "$(git url-parse -c path "example.com:~user/repository")"
      +'
      +
     ++test_expect_success 'git url-parse -c path -- username expansion strips query and fragment' '
     ++	test "~user/repository" = "$(git url-parse -c path "ssh://example.com/~user/repository?query")" &&
     ++	test "~user/repository" = "$(git url-parse -c path "ssh://example.com/~user/repository#fragment")" &&
     ++	test "~user/repository" = "$(git url-parse -c path "git://example.com/~user/repository?query")" &&
     ++	test "~user/repository" = "$(git url-parse -c path "user@example.com:~user/repository?query")"
     ++'
     ++
     ++test_expect_success 'git url-parse -- ssh syntax with IPv6' '
     ++	git url-parse "ssh://user@[::1]:1234/repository/path" &&
     ++	git url-parse "ssh://user@[::1]/repository/path" &&
     ++	git url-parse "ssh://[::1]:1234/repository/path" &&
     ++	git url-parse "ssh://[::1]/repository/path" &&
     ++	git url-parse "ssh://[2001:db8::1]/repository/path"
     ++'
     ++
     ++test_expect_success 'git url-parse -- git syntax with IPv6' '
     ++	git url-parse "git://[::1]:9418/repository/path" &&
     ++	git url-parse "git://[::1]/repository/path"
     ++'
     ++
     ++test_expect_success 'git url-parse -- http syntax with IPv6' '
     ++	git url-parse "https://[::1]:1234/repository/path" &&
     ++	git url-parse "https://[::1]/repository/path" &&
     ++	git url-parse "http://[2001:db8::1]/repository/path"
     ++'
     ++
     ++test_expect_success 'git url-parse -c host -- IPv6 in URL form' '
     ++	test "[::1]" = "$(git url-parse -c host "ssh://user@[::1]:1234/repository/path")" &&
     ++	test "[::1]" = "$(git url-parse -c host "ssh://[::1]/repository/path")" &&
     ++	test "[2001:db8::1]" = "$(git url-parse -c host "ssh://[2001:db8::1]/repository/path")" &&
     ++	test "[::1]" = "$(git url-parse -c host "git://[::1]/repository/path")" &&
     ++	test "[2001:db8::1]" = "$(git url-parse -c host "https://[2001:db8::1]/repository/path")"
     ++'
     ++
     ++test_expect_success 'git url-parse -c port -- IPv6 in URL form' '
     ++	test 1234 = "$(git url-parse -c port "ssh://user@[::1]:1234/repository/path")" &&
     ++	test "" = "$(git url-parse -c port "ssh://[::1]/repository/path")" &&
     ++	test 9418 = "$(git url-parse -c port "git://[::1]:9418/repository/path")"
     ++'
     ++
     ++test_expect_success 'git url-parse -- scp syntax with IPv6' '
     ++	git url-parse "[::1]:repository/path" &&
     ++	git url-parse "user@[::1]:repository/path" &&
     ++	git url-parse "[2001:db8::1]:repo"
     ++'
     ++
     ++test_expect_success 'git url-parse -- scp syntax with bracketed hostname' '
     ++	git url-parse "[myhost]:src" &&
     ++	git url-parse "user@[myhost]:src"
     ++'
     ++
     ++test_expect_success 'git url-parse -- scp syntax with bracketed host:port' '
     ++	git url-parse "[myhost:123]:src" &&
     ++	git url-parse "user@[myhost:123]:src"
     ++'
     ++
     ++test_expect_success 'git url-parse -c host -- scp+IPv6' '
     ++	test "[::1]" = "$(git url-parse -c host "[::1]:repository/path")" &&
     ++	test "[::1]" = "$(git url-parse -c host "user@[::1]:repository/path")" &&
     ++	test "[2001:db8::1]" = "$(git url-parse -c host "[2001:db8::1]:repo")"
     ++'
     ++
     ++test_expect_success 'git url-parse -c path -- scp+IPv6' '
     ++	test "/repository/path" = "$(git url-parse -c path "[::1]:/repository/path")" &&
     ++	test "/repository/path" = "$(git url-parse -c path "[::1]:repository/path")" &&
     ++	test "/repo" = "$(git url-parse -c path "[2001:db8::1]:repo")"
     ++'
     ++
     ++test_expect_success 'git url-parse -c host,port,path -- scp [host:port]:src' '
     ++	test myhost = "$(git url-parse -c host "[myhost:123]:src")" &&
     ++	test 123 = "$(git url-parse -c port "[myhost:123]:src")" &&
     ++	test "/src" = "$(git url-parse -c path "[myhost:123]:src")"
     ++'
     ++
     ++test_expect_success 'git url-parse -c host,path -- scp [host]:src' '
     ++	test myhost = "$(git url-parse -c host "[myhost]:src")" &&
     ++	test "/src" = "$(git url-parse -c path "[myhost]:src")"
     ++'
     ++
     ++test_expect_success 'git url-parse -c user -- scp with user@ and brackets' '
     ++	test user = "$(git url-parse -c user "user@[::1]:repo")" &&
     ++	test user = "$(git url-parse -c user "user@[myhost:123]:src")" &&
     ++	test user = "$(git url-parse -c user "user@[myhost]:src")"
     ++'
     ++
     ++test_expect_success 'git url-parse -- scp+IPv6 with username expansion' '
     ++	test "~user/repo" = "$(git url-parse -c path "[::1]:~user/repo")" &&
     ++	test "~user/repo" = "$(git url-parse -c path "user@[::1]:~user/repo")"
     ++'
     ++
     ++test_expect_success 'git url-parse fails on invalid URL' '
     ++	test_must_fail git url-parse "not a url"
     ++'
     ++
     ++test_expect_success 'git url-parse helpful error for absolute local path' '
     ++	test_must_fail git url-parse "/abs/path" 2>err &&
     ++	test_grep "is not a URL" err &&
     ++	test_grep "file:///abs/path" err
     ++'
     ++
     ++test_expect_success 'git url-parse helpful error for relative local path' '
     ++	test_must_fail git url-parse "./rel" 2>err &&
     ++	test_grep "is not a URL" err &&
     ++	test_grep "absolute path" err
     ++'
     ++
     ++test_expect_success 'git url-parse fails on unknown -c component name' '
     ++	test_must_fail git url-parse -c bogus "https://example.com/repo"
     ++'
     ++
     ++test_expect_success 'git url-parse fails on URL missing host' '
     ++	test_must_fail git url-parse "https://"
     ++'
     ++
     ++test_expect_success 'git url-parse with no URL prints usage' '
     ++	test_must_fail git url-parse 2>err &&
     ++	test_grep "usage:" err
     ++'
     ++
      +test_done

-- 
gitgitgadget