[PATCH 0/7] fast-import: standardize usage string and SYNOPSIS

[PATCH 0/7] fast-import: standardize usage string and SYNOPSIS

[Christian Couder]

The goal of this series is to improve on `git fast-import`'s usage
string as it is obsolete in many ways.

Along the way it modernizes "builtin/fast-import.c" mostly by using
`struct option`, by starting to remove global variables and libify
that command, and by introducing a new `OPT_HIDDEN_GROUP` macro.

`struct option` is used only for the usage string for now and there
are still many global variables left, so it's left to future work to
finish on these directions.

But this is already enough to standardize the usage string and make it
consistent with the SYNOPSIS in the docs, so that the command can be
removed from "t/t0450/adoc-help-mismatches".

Overview of the patches
=======================

  - Patch 1/7: Introduces OPT_HIDDEN_GROUP and improves on the hidden
    option tests.

  - Patches 2/7 and 3/7: Improves on the parse-options API docs.

  - Patch 4/7: Cleans up an 'i' loop counter variable in
    cmd_fast_import().

  - Patches 5/7 and 6/7: Starts libifying "builtin/fast-import.c" by
    introducing a 'struct fast_import_state' and using it to store
    some global variables.

  - Patch 7/7: Improves the usage string and SYNOPSIS by introducing
    `struct option`.

CI tests
========

They all pass, see: https://github.com/chriscool/git/actions/runs/29513714019

Christian Couder (7):
  parse-options: introduce OPT_HIDDEN_GROUP
  api-parse-options.adoc: document per-option flags
  api-parse-options.adoc: document hidden and OPT_*_F option macros
  fast-import: localize 'i' into the 'for' loops using it
  fast-import: introduce 'struct fast_import_state'
  fast-import: move command state globals into 'struct
    fast_import_state'
  fast-import: use struct option for usage string

 Documentation/git-fast-import.adoc            |   2 +-
 .../technical/api-parse-options.adoc          |  79 ++++
 builtin/fast-import.c                         | 372 +++++++++++-------
 parse-options.c                               |   4 +-
 parse-options.h                               |   5 +
 t/helper/test-parse-options.c                 |   4 +
 t/t0040-parse-options.sh                      |  25 +-
 t/t0450/adoc-help-mismatches                  |   1 -
 8 files changed, 344 insertions(+), 148 deletions(-)

-- 
2.55.0.185.g9120d2b5c0

[PATCH 1/7] parse-options: introduce OPT_HIDDEN_GROUP

[Christian Couder]

Hidden options are not shown by `git <cmd> -h`, but are still shown by
`git <cmd> --help-all`. If there are a lot of hidden options or if they
don't belong to the same categories as other options, there is
currently no way to properly group them.

Using `OPT_GROUP("Foo")` means that "Foo" will always be shown which we
don't want if that group contains only hidden options.

To provide a way to have groups shown only when hidden options are
shown, let's implement an OPT_HIDDEN_GROUP macro.

To test this new macro, let's also improve `test-tool parse-options`
and test its output with `--help-all`.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 parse-options.c               |  4 ++--
 parse-options.h               |  5 +++++
 t/helper/test-parse-options.c |  4 ++++
 t/t0040-parse-options.sh      | 25 ++++++++++++++++++++++++-
 4 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/parse-options.c b/parse-options.c
index f4647e0099..640e600de8 100644
--- a/parse-options.c
+++ b/parse-options.c
@@ -1404,6 +1404,8 @@ static enum parse_opt_result usage_with_options_internal(struct parse_opt_ctx_t
 
 		if (opts->type == OPTION_SUBCOMMAND)
 			continue;
+		if (!full && (opts->flags & PARSE_OPT_HIDDEN))
+			continue;
 		if (opts->type == OPTION_GROUP) {
 			fputc('\n', outfile);
 			need_newline = 0;
@@ -1411,8 +1413,6 @@ static enum parse_opt_result usage_with_options_internal(struct parse_opt_ctx_t
 				fprintf(outfile, "%s\n", _(opts->help));
 			continue;
 		}
-		if (!full && (opts->flags & PARSE_OPT_HIDDEN))
-			continue;
 
 		if (need_newline) {
 			fputc('\n', outfile);
diff --git a/parse-options.h b/parse-options.h
index 0d1f738f8d..a28b3cd942 100644
--- a/parse-options.h
+++ b/parse-options.h
@@ -236,6 +236,11 @@ struct option {
 	.type = OPTION_GROUP, \
 	.help = (h), \
 }
+#define OPT_HIDDEN_GROUP(h) { \
+	.type = OPTION_GROUP, \
+	.help = (h), \
+	.flags = PARSE_OPT_HIDDEN, \
+}
 #define OPT_BIT(s, l, v, h, b)      OPT_BIT_F(s, l, v, h, b, 0)
 #define OPT_BITOP(s, l, v, h, set, clear) { \
 	.type = OPTION_BITOP, \
diff --git a/t/helper/test-parse-options.c b/t/helper/test-parse-options.c
index 68579d83f3..f181f0c02d 100644
--- a/t/helper/test-parse-options.c
+++ b/t/helper/test-parse-options.c
@@ -209,6 +209,10 @@ int cmd__parse_options(int argc, const char **argv)
 		OPT_GROUP("Alias"),
 		OPT_STRING('A', "alias-source", &string, "string", "get a string"),
 		OPT_ALIAS('Z', "alias-target", "alias-source"),
+		OPT_HIDDEN_GROUP("Hidden options"),
+		OPT_HIDDEN_BOOL(0, "hidden-bool", &boolean, "get a boolean"),
+		OPT_INTEGER_F('k', "hidden-integer", &integer, "get a integer",
+			      PARSE_OPT_HIDDEN),
 		OPT_END(),
 	};
 	int ret = 0;
diff --git a/t/t0040-parse-options.sh b/t/t0040-parse-options.sh
index ca55ea8228..4040333185 100755
--- a/t/t0040-parse-options.sh
+++ b/t/t0040-parse-options.sh
@@ -7,7 +7,7 @@ test_description='our own option parser'
 
 . ./test-lib.sh
 
-cat >expect <<\EOF
+cat >expect-part1 <<\EOF
 usage: test-tool parse-options <options>
 
     A helper function for the parse-options API.
@@ -41,6 +41,9 @@ String options
     --[no-]string2 <str>  get another string
     --[no-]st <st>        get another string (pervert ordering)
     -o <str>              get another string
+EOF
+
+cat >expect-part2 <<\EOF
     --longhelp            help text of this entry
                           spans multiple lines
     --[no-]list <str>     add str to list
@@ -67,12 +70,32 @@ Alias
 
 EOF
 
+cat >expect-noop <<\EOF
+    --[no-]obsolete       no-op (backward compatibility)
+EOF
+
+cat >expect-hidden <<\EOF
+Hidden options
+    --[no-]hidden-bool    get a boolean
+    -k, --[no-]hidden-integer <n>
+                          get a integer
+
+EOF
+
 test_expect_success 'test help' '
+	cat expect-part1 expect-part2 >expect &&
 	test_must_fail test-tool parse-options -h >output 2>output.err &&
 	test_must_be_empty output.err &&
 	test_cmp expect output
 '
 
+test_expect_success 'test --help-all shows hidden group and options' '
+	cat expect-part1 expect-noop expect-part2 expect-hidden >expect-help-all &&
+	test_must_fail test-tool parse-options --help-all >output 2>output.err &&
+	test_must_be_empty output.err &&
+	test_cmp expect-help-all output
+'
+
 mv expect expect.err
 
 check () {
-- 
2.55.0.185.g9120d2b5c0

[PATCH 2/7] api-parse-options.adoc: document per-option flags

[Christian Couder]

The "Flags" section in "Documentation/technical/api-parse-options.adoc"
documents the flags that can be passed to parse_options() itself. It
does not, however, document the flags that can be set on individual
options through the `flags` member of `struct option` (and through the
`OPT_*_F()` macro variants).

These per-option flags are used throughout the codebase (for example
`PARSE_OPT_HIDDEN` is used to hide an option from `-h` while still
showing it with `--help-all`), but a reader currently has to dig into
"parse-options.h" to find them.

To remediate that, let's add an "Option flags" subsection to the
"Data Structure" section, just before the list of option macros.

Let's also make it explicit that these are distinct from the
parse_options() flags described earlier, and let's describe the `-h`
versus `--help-all` behavior for `PARSE_OPT_HIDDEN`.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 .../technical/api-parse-options.adoc          | 61 +++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/Documentation/technical/api-parse-options.adoc b/Documentation/technical/api-parse-options.adoc
index 880eb94642..fb4580e755 100644
--- a/Documentation/technical/api-parse-options.adoc
+++ b/Documentation/technical/api-parse-options.adoc
@@ -150,6 +150,67 @@ Data Structure
 
 The main data structure is an array of the `option` struct,
 say `static struct option builtin_add_options[]`.
+
+Option flags
+~~~~~~~~~~~~
+
+Each option can carry flags in the `flags` field of its `option`
+struct. These are per-option flags and are distinct from the
+`parse_options()` flags described above; they are usually set through
+the `OPT_*_F()` macro variants (see below) rather than by hand. They
+are the bitwise-or of:
+
+`PARSE_OPT_OPTARG`::
+	The option's argument is optional, i.e. both `--option` and
+	`--option=<value>` are accepted.
+
+`PARSE_OPT_NOARG`::
+	The option takes no argument at all. Using `--option=<value>`
+	is rejected.
+
+`PARSE_OPT_NONEG`::
+	Disable the automatically generated negated `--no-option`
+	form.
+
+`PARSE_OPT_HIDDEN`::
+	Hide the option: it is omitted from the usage shown by
+	`git <cmd> -h`, but is still shown by `git <cmd> --help-all`.
+	The option is parsed as usual either way. This is meant for
+	deprecated, advanced or otherwise uncommon options.
+
+`PARSE_OPT_LASTARG_DEFAULT`::
+	Use the default value (`defval`) when the option is used
+	without an argument, even for an option that normally requires
+	one. Only the last argument on the command line takes effect.
+
+`PARSE_OPT_NODASH`::
+	The option is a single character without a leading dash, such
+	as the `+` used by some commands.
+
+`PARSE_OPT_LITERAL_ARGHELP`::
+	Use the argument help string (`argh`) verbatim in the usage
+	output instead of surrounding it with `<>` or `[]`. Useful when
+	`argh` already contains a hand-formatted description.
+
+`PARSE_OPT_FROM_ALIAS`::
+	Internal flag, set on options that were expanded from a
+	configured alias. It should not be set by callers.
+
+`PARSE_OPT_NOCOMPLETE`::
+	Do not offer this option for completion.
+
+`PARSE_OPT_COMP_ARG`::
+	The option's argument, rather than the option itself, is what
+	should be completed.
+
+`PARSE_OPT_CMDMODE`::
+	The option is one of several mutually exclusive "command mode"
+	options that share the same variable. Using more than one of
+	them at once is rejected.
+
+Macros
+~~~~~~
+
 There are some macros to easily define options:
 
 `OPT__ABBREV(&int_var)`::
-- 
2.55.0.185.g9120d2b5c0

[PATCH 3/7] api-parse-options.adoc: document hidden and OPT_*_F option macros

[Christian Couder]

In "Documentation/technical/api-parse-options.adoc", the list of option
macros does not mention the `OPT_*_F()` macro variants that take a
trailing `flags` argument, nor the `OPT_HIDDEN_GROUP()` and
`OPT_HIDDEN_BOOL()` convenience macros.

Now that a previous commit documents the per-option flags, let's
document these macros too:

  - Add a paragraph explaining the `OPT_*_F` convention and how it
    relates to the per-option flags.

  - Document `OPT_HIDDEN_GROUP()`, introduced in a previous commit,
    right after `OPT_GROUP()`.

  - Document `OPT_HIDDEN_BOOL()` right after `OPT_BOOL()`.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/technical/api-parse-options.adoc | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/Documentation/technical/api-parse-options.adoc b/Documentation/technical/api-parse-options.adoc
index fb4580e755..0e10327d07 100644
--- a/Documentation/technical/api-parse-options.adoc
+++ b/Documentation/technical/api-parse-options.adoc
@@ -213,6 +213,13 @@ Macros
 
 There are some macros to easily define options:
 
+Many of the macros below have an `_F` variant (for example `OPT_BOOL_F`,
+`OPT_STRING_F`, `OPT_INTEGER_F`, `OPT_SET_INT_F`, `OPT_BIT_F` and
+`OPT_CALLBACK_F`) that takes an additional trailing `flags` argument.
+That argument is the bitwise-or of the per-option flags described in the
+"Option flags" section above; the non-`_F` macros are simply defined
+with `flags` set to `0`.
+
 `OPT__ABBREV(&int_var)`::
 	Add `--abbrev[=<n>]`.
 
@@ -236,10 +243,21 @@ There are some macros to easily define options:
 	describes the group or an empty string.
 	Start the description with an upper-case letter.
 
+`OPT_HIDDEN_GROUP(description)`::
+	Like `OPT_GROUP()`, but the group header carries
+	`PARSE_OPT_HIDDEN`, so it is only shown by `--help-all` and not
+	by `-h`. Use it to label a group that contains only hidden
+	options, which would otherwise show an empty header under `-h`.
+
 `OPT_BOOL(short, long, &int_var, description)`::
 	Introduce a boolean option. `int_var` is set to one with
 	`--option` and set to zero with `--no-option`.
 
+`OPT_HIDDEN_BOOL(short, long, &int_var, description)`::
+	Like `OPT_BOOL()`, but the option carries `PARSE_OPT_HIDDEN`,
+	so it is hidden from `-h` while still being shown by
+	`--help-all`.
+
 `OPT_COUNTUP(short, long, &int_var, description)`::
 	Introduce a count-up option.
 	Each use of `--option` increments `int_var`, starting from zero
-- 
2.55.0.185.g9120d2b5c0

[PATCH 4/7] fast-import: localize 'i' into the 'for' loops using it

[Christian Couder]

In cmd_fast_import(), a	local variable 'i' is defined as an
`unsigned int` and then used as a loop counter in four different
`for (i = ...; i < ...; i++) { ... }` loops.

But in three out of the four cases, `unsigned int` isn't the best type
to use.

To give each loop counter the type matching its bound
(int/unsigned/size_t), let's localize 'i' into each loop that uses it.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/fast-import.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index aa656c5195..fd4e13b7ca 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -3936,8 +3936,6 @@ int cmd_fast_import(int argc,
 		    const char *prefix,
 		    struct repository *repo)
 {
-	unsigned int i;
-
 	show_usage_if_asked(argc, argv, fast_import_usage);
 
 	reset_pack_idx_option(&pack_idx_opts);
@@ -3958,7 +3956,7 @@ int cmd_fast_import(int argc,
 	 * line to override stream data). But we must do an early parse of any
 	 * command-line options that impact how we interpret the feature lines.
 	 */
-	for (i = 1; i < argc; i++) {
+	for (int i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (*arg != '-' || !strcmp(arg, "--"))
 			break;
@@ -3971,7 +3969,7 @@ int cmd_fast_import(int argc,
 	global_prefix = prefix;
 
 	rc_free = mem_pool_alloc(&fi_mem_pool, cmd_save * sizeof(*rc_free));
-	for (i = 0; i < (cmd_save - 1); i++)
+	for (unsigned int i = 0; i < (cmd_save - 1); i++)
 		rc_free[i].next = &rc_free[i + 1];
 	rc_free[cmd_save - 1].next = NULL;
 
@@ -4034,9 +4032,9 @@ int cmd_fast_import(int argc,
 
 	if (show_stats) {
 		uintmax_t total_count = 0, duplicate_count = 0;
-		for (i = 0; i < ARRAY_SIZE(object_count_by_type); i++)
+		for (size_t i = 0; i < ARRAY_SIZE(object_count_by_type); i++)
 			total_count += object_count_by_type[i];
-		for (i = 0; i < ARRAY_SIZE(duplicate_count_by_type); i++)
+		for (size_t i = 0; i < ARRAY_SIZE(duplicate_count_by_type); i++)
 			duplicate_count += duplicate_count_by_type[i];
 
 		fprintf(stderr, "%s statistics:\n", argv[0]);
-- 
2.55.0.185.g9120d2b5c0

[PATCH 5/7] fast-import: introduce 'struct fast_import_state'

[Christian Couder]

"builtin/fast-import.c" uses a large number of global variables. This
makes it harder than necessary to reason about and improve. Especially
adding new features requires adding more global variables, while
modernizing and eventually libifying the code becomes more and more
difficult.

To start reverting the sad trend to more and more globals and to start
cleaning things up, let's introduce a 'struct fast_import_state' and
pass an instance of it as the first argument to many functions.

This is similar to what was done for "builtin/apply.c" by introducing a
'struct apply_state', see 07d7e290ff (apply: move 'struct apply_state'
to a header file, 2016-08-11) and related commits.

As a first step only the 'global_argc', 'global_argv' and
'global_prefix' variables are moved into the new struct. More variables
will be moved into it in the following commits.

Some functions receive the new 'state' parameter only to pass it
along or for future use, so they are marked with UNUSED for now to
satisfy '-Werror=unused-parameter'.

This is a mostly mechanical refactoring with no intended behavior
change.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/fast-import.c | 262 ++++++++++++++++++++++--------------------
 1 file changed, 138 insertions(+), 124 deletions(-)

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index fd4e13b7ca..d20353679a 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -184,10 +184,6 @@ static int failure;
 static FILE *pack_edges;
 static unsigned int show_stats = 1;
 static unsigned int quiet;
-static int global_argc;
-static const char **global_argv;
-static const char *global_prefix;
-
 static enum sign_mode signed_tag_mode = SIGN_VERBATIM;
 static enum sign_mode signed_commit_mode = SIGN_VERBATIM;
 static const char *signed_commit_keyid;
@@ -276,10 +272,27 @@ static kh_oid_map_t *sub_oid_map;
 /* Where to write output of cat-blob commands */
 static int cat_blob_fd = STDOUT_FILENO;
 
-static void parse_argv(void);
-static void parse_get_mark(const char *p);
-static void parse_cat_blob(const char *p);
-static void parse_ls(const char *p, struct branch *b);
+/* Command state */
+struct fast_import_state {
+	int argc;
+	const char **argv;
+	const char *prefix;
+};
+
+static void fast_import_state_init(struct fast_import_state *state,
+				   int argc, const char **argv,
+				   const char *prefix)
+{
+	memset(state, 0, sizeof(*state));
+	state->argc = argc;
+	state->argv = argv;
+	state->prefix = prefix;
+}
+
+static void parse_argv(struct fast_import_state *state);
+static void parse_get_mark(struct fast_import_state *state, const char *p);
+static void parse_cat_blob(struct fast_import_state *state, const char *p);
+static void parse_ls(struct fast_import_state *state, const char *p, struct branch *b);
 
 static void for_each_mark(struct mark_set *m, uintmax_t base, each_mark_fn_t callback, void *p)
 {
@@ -1844,7 +1857,7 @@ static void read_marks(void)
 }
 
 
-static int read_next_command(void)
+static int read_next_command(struct fast_import_state *state)
 {
 	static int stdin_eof = 0;
 
@@ -1866,7 +1879,7 @@ static int read_next_command(void)
 			if (!seen_data_command
 				&& !starts_with(command_buf.buf, "feature ")
 				&& !starts_with(command_buf.buf, "option ")) {
-				parse_argv();
+				parse_argv(state);
 			}
 
 			rc = rc_free;
@@ -1898,22 +1911,22 @@ static void skip_optional_lf(void)
 		ungetc(term_char, stdin);
 }
 
-static void parse_mark(void)
+static void parse_mark(struct fast_import_state *state)
 {
 	const char *v;
 	if (skip_prefix(command_buf.buf, "mark :", &v)) {
 		next_mark = strtoumax(v, NULL, 10);
-		read_next_command();
+		read_next_command(state);
 	}
 	else
 		next_mark = 0;
 }
 
-static void parse_original_identifier(void)
+static void parse_original_identifier(struct fast_import_state *state)
 {
 	const char *v;
 	if (skip_prefix(command_buf.buf, "original-oid ", &v))
-		read_next_command();
+		read_next_command(state);
 }
 
 static int parse_data(struct strbuf *sb, uintmax_t limit, uintmax_t *len_res)
@@ -2067,11 +2080,11 @@ static void parse_and_store_blob(
 	}
 }
 
-static void parse_new_blob(void)
+static void parse_new_blob(struct fast_import_state *state)
 {
-	read_next_command();
-	parse_mark();
-	parse_original_identifier();
+	read_next_command(state);
+	parse_mark(state);
+	parse_original_identifier(state);
 	parse_and_store_blob(&last_blob, NULL, next_mark);
 }
 
@@ -2367,7 +2380,7 @@ static void parse_path_space(struct strbuf *sb, const char *p,
 	(*endp)++;
 }
 
-static void file_change_m(const char *p, struct branch *b)
+static void file_change_m(struct fast_import_state *state, const char *p, struct branch *b)
 {
 	static struct strbuf path = STRBUF_INIT;
 	struct object_entry *oe;
@@ -2434,10 +2447,10 @@ static void file_change_m(const char *p, struct branch *b)
 		if (S_ISDIR(mode))
 			die(_("directories cannot be specified 'inline': %s"),
 				command_buf.buf);
-		while (read_next_command() != EOF) {
+		while (read_next_command(state) != EOF) {
 			const char *v;
 			if (skip_prefix(command_buf.buf, "cat-blob ", &v))
-				parse_cat_blob(v);
+				parse_cat_blob(state, v);
 			else {
 				parse_and_store_blob(&last_blob, &oid, 0);
 				break;
@@ -2511,7 +2524,7 @@ static void file_change_cr(const char *p, struct branch *b, int rename)
 		leaf.tree);
 }
 
-static void note_change_n(const char *p, struct branch *b, unsigned char *old_fanout)
+static void note_change_n(struct fast_import_state *state, const char *p, struct branch *b, unsigned char *old_fanout)
 {
 	struct object_entry *oe;
 	struct branch *s;
@@ -2576,7 +2589,7 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa
 		die(_("invalid ref name or SHA1 expression: %s"), p);
 
 	if (inline_data) {
-		read_next_command();
+		read_next_command(state);
 		parse_and_store_blob(&last_blob, &oid, 0);
 	} else if (oe) {
 		if (oe->type != OBJ_BLOB)
@@ -2643,7 +2656,7 @@ static void parse_from_existing(struct branch *b)
 	}
 }
 
-static int parse_objectish(struct branch *b, const char *objectish)
+static int parse_objectish(struct fast_import_state *state, struct branch *b, const char *objectish)
 {
 	struct branch *s;
 	struct object_id oid;
@@ -2686,31 +2699,31 @@ static int parse_objectish(struct branch *b, const char *objectish)
 		b->branch_tree.tree = NULL;
 	}
 
-	read_next_command();
+	read_next_command(state);
 	return 1;
 }
 
-static int parse_from(struct branch *b)
+static int parse_from(struct fast_import_state *state, struct branch *b)
 {
 	const char *from;
 
 	if (!skip_prefix(command_buf.buf, "from ", &from))
 		return 0;
 
-	return parse_objectish(b, from);
+	return parse_objectish(state, b, from);
 }
 
-static int parse_objectish_with_prefix(struct branch *b, const char *prefix)
+static int parse_objectish_with_prefix(struct fast_import_state *state, struct branch *b, const char *prefix)
 {
 	const char *base;
 
 	if (!skip_prefix(command_buf.buf, prefix, &base))
 		return 0;
 
-	return parse_objectish(b, base);
+	return parse_objectish(state, b, base);
 }
 
-static struct hash_list *parse_merge(unsigned int *count)
+static struct hash_list *parse_merge(struct fast_import_state *state, unsigned int *count)
 {
 	struct hash_list *list = NULL, **tail = &list, *n;
 	const char *from;
@@ -2744,7 +2757,7 @@ static struct hash_list *parse_merge(unsigned int *count)
 		tail = &n->next;
 
 		(*count)++;
-		read_next_command();
+		read_next_command(state);
 	}
 	return list;
 }
@@ -2755,7 +2768,7 @@ struct signature_data {
 	struct strbuf data;   /* The actual signature data */
 };
 
-static void parse_one_signature(struct signature_data *sig, const char *v)
+static void parse_one_signature(struct fast_import_state *state, struct signature_data *sig, const char *v)
 {
 	char *args = xstrdup(v); /* Will be freed when sig->hash_algo is freed */
 	char *space = strchr(args, ' ');
@@ -2780,15 +2793,15 @@ static void parse_one_signature(struct signature_data *sig, const char *v)
 		warning(_("'unknown' signature format in gpgsig"));
 
 	/* Read signature data */
-	read_next_command();
+	read_next_command(state);
 	parse_data(&sig->data, 0, NULL);
 }
 
-static void discard_one_signature(void)
+static void discard_one_signature(struct fast_import_state *state)
 {
 	struct strbuf data = STRBUF_INIT;
 
-	read_next_command();
+	read_next_command(state);
 	parse_data(&data, 0, NULL);
 	strbuf_release(&data);
 }
@@ -2826,13 +2839,14 @@ static void store_signature(struct signature_data *stored_sig,
 	}
 }
 
-static void import_one_signature(struct signature_data *sig_sha1,
+static void import_one_signature(struct fast_import_state *state,
+				 struct signature_data *sig_sha1,
 				 struct signature_data *sig_sha256,
 				 const char *v)
 {
 	struct signature_data sig = { NULL, NULL, STRBUF_INIT };
 
-	parse_one_signature(&sig, v);
+	parse_one_signature(state, &sig, v);
 
 	if (!strcmp(sig.hash_algo, "sha1"))
 		store_signature(sig_sha1, &sig, "SHA-1");
@@ -2946,7 +2960,7 @@ static void handle_signature_if_invalid(struct strbuf *new_data,
 	strbuf_release(&tmp_buf);
 }
 
-static void parse_new_commit(const char *arg)
+static void parse_new_commit(struct fast_import_state *state, const char *arg)
 {
 	static struct strbuf msg = STRBUF_INIT;
 	struct signature_data sig_sha1 = { NULL, NULL, STRBUF_INIT };
@@ -2964,16 +2978,16 @@ static void parse_new_commit(const char *arg)
 	if (!b)
 		b = new_branch(arg);
 
-	read_next_command();
-	parse_mark();
-	parse_original_identifier();
+	read_next_command(state);
+	parse_mark(state);
+	parse_original_identifier(state);
 	if (skip_prefix(command_buf.buf, "author ", &v)) {
 		author = parse_ident(v);
-		read_next_command();
+		read_next_command(state);
 	}
 	if (skip_prefix(command_buf.buf, "committer ", &v)) {
 		committer = parse_ident(v);
-		read_next_command();
+		read_next_command(state);
 	}
 	if (!committer)
 		die(_("expected committer but didn't get one"));
@@ -2989,7 +3003,7 @@ static void parse_new_commit(const char *arg)
 			warning(_("stripping a commit signature"));
 			/* fallthru */
 		case SIGN_STRIP:
-			discard_one_signature();
+			discard_one_signature(state);
 			break;
 
 		/* Second, modes that parse the signature */
@@ -3000,24 +3014,24 @@ static void parse_new_commit(const char *arg)
 		case SIGN_STRIP_IF_INVALID:
 		case SIGN_SIGN_IF_INVALID:
 		case SIGN_ABORT_IF_INVALID:
-			import_one_signature(&sig_sha1, &sig_sha256, v);
+			import_one_signature(state, &sig_sha1, &sig_sha256, v);
 			break;
 
 		/* Third, BUG */
 		default:
 			BUG("invalid signed_commit_mode value %d", signed_commit_mode);
 		}
-		read_next_command();
+		read_next_command(state);
 	}
 
 	if (skip_prefix(command_buf.buf, "encoding ", &v)) {
 		encoding = xstrdup(v);
-		read_next_command();
+		read_next_command(state);
 	}
 	parse_data(&msg, 0, NULL);
-	read_next_command();
-	parse_from(b);
-	merge_list = parse_merge(&merge_count);
+	read_next_command(state);
+	parse_from(state, b);
+	merge_list = parse_merge(state, &merge_count);
 
 	/* ensure the branch is active/loaded */
 	if (!b->branch_tree.tree || !max_active_branches) {
@@ -3030,7 +3044,7 @@ static void parse_new_commit(const char *arg)
 	/* file_change* */
 	while (command_buf.len > 0) {
 		if (skip_prefix(command_buf.buf, "M ", &v))
-			file_change_m(v, b);
+			file_change_m(state, v, b);
 		else if (skip_prefix(command_buf.buf, "D ", &v))
 			file_change_d(v, b);
 		else if (skip_prefix(command_buf.buf, "R ", &v))
@@ -3038,18 +3052,18 @@ static void parse_new_commit(const char *arg)
 		else if (skip_prefix(command_buf.buf, "C ", &v))
 			file_change_cr(v, b, 0);
 		else if (skip_prefix(command_buf.buf, "N ", &v))
-			note_change_n(v, b, &prev_fanout);
+			note_change_n(state, v, b, &prev_fanout);
 		else if (!strcmp("deleteall", command_buf.buf))
 			file_change_deleteall(b);
 		else if (skip_prefix(command_buf.buf, "ls ", &v))
-			parse_ls(v, b);
+			parse_ls(state, v, b);
 		else if (skip_prefix(command_buf.buf, "cat-blob ", &v))
-			parse_cat_blob(v);
+			parse_cat_blob(state, v);
 		else {
 			unread_command_buf = 1;
 			break;
 		}
-		if (read_next_command() == EOF)
+		if (read_next_command(state) == EOF)
 			break;
 	}
 
@@ -3187,7 +3201,7 @@ static void handle_tag_signature(struct strbuf *buf, struct strbuf *msg, const c
 	}
 }
 
-static void parse_new_tag(const char *arg)
+static void parse_new_tag(struct fast_import_state *state, const char *arg)
 {
 	static struct strbuf msg = STRBUF_INIT;
 	const char *from;
@@ -3206,8 +3220,8 @@ static void parse_new_tag(const char *arg)
 	else
 		first_tag = t;
 	last_tag = t;
-	read_next_command();
-	parse_mark();
+	read_next_command(state);
+	parse_mark(state);
 
 	/* from ... */
 	if (!skip_prefix(command_buf.buf, "from ", &from))
@@ -3235,15 +3249,15 @@ static void parse_new_tag(const char *arg)
 			type = oe->type;
 	} else
 		die(_("invalid ref name or SHA1 expression: %s"), from);
-	read_next_command();
+	read_next_command(state);
 
 	/* original-oid ... */
-	parse_original_identifier();
+	parse_original_identifier(state);
 
 	/* tagger ... */
 	if (skip_prefix(command_buf.buf, "tagger ", &v)) {
 		tagger = parse_ident(v);
-		read_next_command();
+		read_next_command(state);
 	} else
 		tagger = NULL;
 
@@ -3274,7 +3288,7 @@ static void parse_new_tag(const char *arg)
 		t->pack_id = pack_id;
 }
 
-static void parse_reset_branch(const char *arg)
+static void parse_reset_branch(struct fast_import_state *state, const char *arg)
 {
 	struct branch *b;
 	const char *tag_name;
@@ -3291,8 +3305,8 @@ static void parse_reset_branch(const char *arg)
 	}
 	else
 		b = new_branch(arg);
-	read_next_command();
-	parse_from(b);
+	read_next_command(state);
+	parse_from(state, b);
 	if (b->delete && skip_prefix(b->name, "refs/tags/", &tag_name)) {
 		/*
 		 * Elsewhere, we call dump_branches() before dump_tags(),
@@ -3377,7 +3391,7 @@ static void cat_blob(struct object_entry *oe, struct object_id *oid)
 		free(buf);
 }
 
-static void parse_get_mark(const char *p)
+static void parse_get_mark(struct fast_import_state *state UNUSED, const char *p)
 {
 	struct object_entry *oe;
 	char output[GIT_MAX_HEXSZ + 2];
@@ -3394,7 +3408,7 @@ static void parse_get_mark(const char *p)
 	cat_blob_write(output, the_hash_algo->hexsz + 1);
 }
 
-static void parse_cat_blob(const char *p)
+static void parse_cat_blob(struct fast_import_state *state UNUSED, const char *p)
 {
 	struct object_entry *oe;
 	struct object_id oid;
@@ -3559,7 +3573,7 @@ static void print_ls(int mode, const unsigned char *hash, const char *path)
 	cat_blob_write(line.buf, line.len);
 }
 
-static void parse_ls(const char *p, struct branch *b)
+static void parse_ls(struct fast_import_state *state UNUSED, const char *p, struct branch *b)
 {
 	static struct strbuf path = STRBUF_INIT;
 	struct tree_entry *root = NULL;
@@ -3606,13 +3620,13 @@ static void checkpoint(void)
 	dump_marks();
 }
 
-static void parse_checkpoint(void)
+static void parse_checkpoint(struct fast_import_state *state UNUSED)
 {
 	checkpoint_requested = 1;
 	skip_optional_lf();
 }
 
-static void parse_progress(void)
+static void parse_progress(struct fast_import_state *state UNUSED)
 {
 	fwrite(command_buf.buf, 1, command_buf.len, stdout);
 	fputc('\n', stdout);
@@ -3620,36 +3634,36 @@ static void parse_progress(void)
 	skip_optional_lf();
 }
 
-static void parse_alias(void)
+static void parse_alias(struct fast_import_state *state)
 {
 	struct object_entry *e;
 	struct branch b;
 
 	skip_optional_lf();
-	read_next_command();
+	read_next_command(state);
 
 	/* mark ... */
-	parse_mark();
+	parse_mark(state);
 	if (!next_mark)
 		die(_("expected 'mark' command, got %s"), command_buf.buf);
 
 	/* to ... */
 	memset(&b, 0, sizeof(b));
-	if (!parse_objectish_with_prefix(&b, "to "))
+	if (!parse_objectish_with_prefix(state, &b, "to "))
 		die(_("expected 'to' command, got %s"), command_buf.buf);
 	e = find_object(&b.oid);
 	assert(e);
 	insert_mark(&marks, next_mark, e);
 }
 
-static char* make_fast_import_path(const char *path)
+static char* make_fast_import_path(struct fast_import_state *state, const char *path)
 {
 	if (!relative_marks_paths || is_absolute_path(path))
-		return prefix_filename(global_prefix, path);
+		return prefix_filename(state->prefix, path);
 	return repo_git_path(the_repository, "info/fast-import/%s", path);
 }
 
-static void option_import_marks(const char *marks,
+static void option_import_marks(struct fast_import_state *state, const char *marks,
 					int from_stream, int ignore_missing)
 {
 	if (import_marks_file) {
@@ -3662,7 +3676,7 @@ static void option_import_marks(const char *marks,
 	}
 
 	free(import_marks_file);
-	import_marks_file = make_fast_import_path(marks);
+	import_marks_file = make_fast_import_path(state, marks);
 	import_marks_file_from_stream = from_stream;
 	import_marks_file_ignore_missing = ignore_missing;
 }
@@ -3702,13 +3716,13 @@ static void option_active_branches(const char *branches)
 	max_active_branches = ulong_arg("--active-branches", branches);
 }
 
-static void option_export_marks(const char *marks)
+static void option_export_marks(struct fast_import_state *state, const char *marks)
 {
 	free(export_marks_file);
-	export_marks_file = make_fast_import_path(marks);
+	export_marks_file = make_fast_import_path(state, marks);
 }
 
-static void option_cat_blob_fd(const char *fd)
+static void option_cat_blob_fd(struct fast_import_state *state UNUSED, const char *fd)
 {
 	unsigned long n = ulong_arg("--cat-blob-fd", fd);
 	if (n > (unsigned long) INT_MAX)
@@ -3716,16 +3730,16 @@ static void option_cat_blob_fd(const char *fd)
 	cat_blob_fd = (int) n;
 }
 
-static void option_export_pack_edges(const char *edges)
+static void option_export_pack_edges(struct fast_import_state *state, const char *edges)
 {
-	char *fn = prefix_filename(global_prefix, edges);
+	char *fn = prefix_filename(state->prefix, edges);
 	if (pack_edges)
 		fclose(pack_edges);
 	pack_edges = xfopen(fn, "a");
 	free(fn);
 }
 
-static void option_rewrite_submodules(const char *arg, struct string_list *list)
+static void option_rewrite_submodules(struct fast_import_state *state, const char *arg, struct string_list *list)
 {
 	struct mark_set *ms;
 	FILE *fp;
@@ -3737,7 +3751,7 @@ static void option_rewrite_submodules(const char *arg, struct string_list *list)
 	f++;
 	CALLOC_ARRAY(ms, 1);
 
-	f = prefix_filename(global_prefix, f);
+	f = prefix_filename(state->prefix, f);
 	fp = fopen(f, "r");
 	if (!fp)
 		die_errno(_("cannot read '%s'"), f);
@@ -3750,7 +3764,7 @@ static void option_rewrite_submodules(const char *arg, struct string_list *list)
 	free(s);
 }
 
-static int parse_one_option(const char *option)
+static int parse_one_option(struct fast_import_state *state, const char *option)
 {
 	if (skip_prefix(option, "max-pack-size=", &option)) {
 		unsigned long v;
@@ -3774,7 +3788,7 @@ static int parse_one_option(const char *option)
 	} else if (skip_prefix(option, "active-branches=", &option)) {
 		option_active_branches(option);
 	} else if (skip_prefix(option, "export-pack-edges=", &option)) {
-		option_export_pack_edges(option);
+		option_export_pack_edges(state, option);
 	} else if (skip_prefix(option, "signed-commits=", &option)) {
 		if (parse_sign_mode(option, &signed_commit_mode, &signed_commit_keyid))
 			usagef(_("unknown --signed-commits mode '%s'"), option);
@@ -3795,34 +3809,34 @@ static int parse_one_option(const char *option)
 	return 1;
 }
 
-static void check_unsafe_feature(const char *feature, int from_stream)
+static void check_unsafe_feature(struct fast_import_state *state UNUSED, const char *feature, int from_stream)
 {
 	if (from_stream && !allow_unsafe_features)
 		die(_("feature '%s' forbidden in input without --allow-unsafe-features"),
 		    feature);
 }
 
-static int parse_one_feature(const char *feature, int from_stream)
+static int parse_one_feature(struct fast_import_state *state, const char *feature, int from_stream)
 {
 	const char *arg;
 
 	if (skip_prefix(feature, "date-format=", &arg)) {
 		option_date_format(arg);
 	} else if (skip_prefix(feature, "import-marks=", &arg)) {
-		check_unsafe_feature("import-marks", from_stream);
-		option_import_marks(arg, from_stream, 0);
+		check_unsafe_feature(state, "import-marks", from_stream);
+		option_import_marks(state, arg, from_stream, 0);
 	} else if (skip_prefix(feature, "import-marks-if-exists=", &arg)) {
-		check_unsafe_feature("import-marks-if-exists", from_stream);
-		option_import_marks(arg, from_stream, 1);
+		check_unsafe_feature(state, "import-marks-if-exists", from_stream);
+		option_import_marks(state, arg, from_stream, 1);
 	} else if (skip_prefix(feature, "export-marks=", &arg)) {
-		check_unsafe_feature(feature, from_stream);
-		option_export_marks(arg);
+		check_unsafe_feature(state, feature, from_stream);
+		option_export_marks(state, arg);
 	} else if (!strcmp(feature, "alias")) {
 		; /* Don't die - this feature is supported */
 	} else if (skip_prefix(feature, "rewrite-submodules-to=", &arg)) {
-		option_rewrite_submodules(arg, &sub_marks_to);
+		option_rewrite_submodules(state, arg, &sub_marks_to);
 	} else if (skip_prefix(feature, "rewrite-submodules-from=", &arg)) {
-		option_rewrite_submodules(arg, &sub_marks_from);
+		option_rewrite_submodules(state, arg, &sub_marks_from);
 	} else if (!strcmp(feature, "get-mark")) {
 		; /* Don't die - this feature is supported */
 	} else if (!strcmp(feature, "cat-blob")) {
@@ -3844,23 +3858,23 @@ static int parse_one_feature(const char *feature, int from_stream)
 	return 1;
 }
 
-static void parse_feature(const char *feature)
+static void parse_feature(struct fast_import_state *state, const char *feature)
 {
 	if (seen_data_command)
 		die(_("got feature command '%s' after data command"), feature);
 
-	if (parse_one_feature(feature, 1))
+	if (parse_one_feature(state, feature, 1))
 		return;
 
 	die(_("this version of fast-import does not support feature %s."), feature);
 }
 
-static void parse_option(const char *option)
+static void parse_option(struct fast_import_state *state, const char *option)
 {
 	if (seen_data_command)
 		die(_("got option command '%s' after data command"), option);
 
-	if (parse_one_option(option))
+	if (parse_one_option(state, option))
 		return;
 
 	die(_("this version of fast-import does not support option: %s"), option);
@@ -3896,12 +3910,12 @@ static void git_pack_config(void)
 static const char fast_import_usage[] =
 "git fast-import [--date-format=<f>] [--max-pack-size=<n>] [--big-file-threshold=<n>] [--depth=<n>] [--active-branches=<n>] [--export-marks=<marks.file>]";
 
-static void parse_argv(void)
+static void parse_argv(struct fast_import_state *state)
 {
 	unsigned int i;
 
-	for (i = 1; i < global_argc; i++) {
-		const char *a = global_argv[i];
+	for (i = 1; i < state->argc; i++) {
+		const char *a = state->argv[i];
 
 		if (*a != '-' || !strcmp(a, "--"))
 			break;
@@ -3909,20 +3923,20 @@ static void parse_argv(void)
 		if (!skip_prefix(a, "--", &a))
 			die(_("unknown option %s"), a);
 
-		if (parse_one_option(a))
+		if (parse_one_option(state, a))
 			continue;
 
-		if (parse_one_feature(a, 0))
+		if (parse_one_feature(state, a, 0))
 			continue;
 
 		if (skip_prefix(a, "cat-blob-fd=", &a)) {
-			option_cat_blob_fd(a);
+			option_cat_blob_fd(state, a);
 			continue;
 		}
 
 		die(_("unknown option --%s"), a);
 	}
-	if (i != global_argc)
+	if (i != state->argc)
 		usage(fast_import_usage);
 
 	seen_data_command = 1;
@@ -3936,6 +3950,8 @@ int cmd_fast_import(int argc,
 		    const char *prefix,
 		    struct repository *repo)
 {
+	struct fast_import_state state;
+
 	show_usage_if_asked(argc, argv, fast_import_usage);
 
 	reset_pack_idx_option(&pack_idx_opts);
@@ -3964,9 +3980,7 @@ int cmd_fast_import(int argc,
 			allow_unsafe_features = 1;
 	}
 
-	global_argc = argc;
-	global_argv = argv;
-	global_prefix = prefix;
+	fast_import_state_init(&state, argc, argv, prefix);
 
 	rc_free = mem_pool_alloc(&fi_mem_pool, cmd_save * sizeof(*rc_free));
 	for (unsigned int i = 0; i < (cmd_save - 1); i++)
@@ -3976,34 +3990,34 @@ int cmd_fast_import(int argc,
 	start_packfile();
 	set_die_routine(die_nicely);
 	set_checkpoint_signal();
-	while (read_next_command() != EOF) {
+	while (read_next_command(&state) != EOF) {
 		const char *v;
 		if (!strcmp("blob", command_buf.buf))
-			parse_new_blob();
+			parse_new_blob(&state);
 		else if (skip_prefix(command_buf.buf, "commit ", &v))
-			parse_new_commit(v);
+			parse_new_commit(&state, v);
 		else if (skip_prefix(command_buf.buf, "tag ", &v))
-			parse_new_tag(v);
+			parse_new_tag(&state, v);
 		else if (skip_prefix(command_buf.buf, "reset ", &v))
-			parse_reset_branch(v);
+			parse_reset_branch(&state, v);
 		else if (skip_prefix(command_buf.buf, "ls ", &v))
-			parse_ls(v, NULL);
+			parse_ls(&state, v, NULL);
 		else if (skip_prefix(command_buf.buf, "cat-blob ", &v))
-			parse_cat_blob(v);
+			parse_cat_blob(&state, v);
 		else if (skip_prefix(command_buf.buf, "get-mark ", &v))
-			parse_get_mark(v);
+			parse_get_mark(&state, v);
 		else if (!strcmp("checkpoint", command_buf.buf))
-			parse_checkpoint();
+			parse_checkpoint(&state);
 		else if (!strcmp("done", command_buf.buf))
 			break;
 		else if (!strcmp("alias", command_buf.buf))
-			parse_alias();
+			parse_alias(&state);
 		else if (starts_with(command_buf.buf, "progress "))
-			parse_progress();
+			parse_progress(&state);
 		else if (skip_prefix(command_buf.buf, "feature ", &v))
-			parse_feature(v);
+			parse_feature(&state, v);
 		else if (skip_prefix(command_buf.buf, "option git ", &v))
-			parse_option(v);
+			parse_option(&state, v);
 		else if (starts_with(command_buf.buf, "option "))
 			/* ignore non-git options*/;
 		else
@@ -4015,7 +4029,7 @@ int cmd_fast_import(int argc,
 
 	/* argv hasn't been parsed yet, do so */
 	if (!seen_data_command)
-		parse_argv();
+		parse_argv(&state);
 
 	if (require_explicit_termination && feof(stdin))
 		die(_("stream ends early"));
-- 
2.55.0.185.g9120d2b5c0

[PATCH 6/7] fast-import: move command state globals into 'struct fast_import_state'

[Christian Couder]

A previous commit introduced 'struct fast_import_state' to hold some
command state, and reduce the need for global variables.

Let's continue in the same direction and move two more global variables
that describe the command state into it: 'seen_data_command' and
'allow_unsafe_features'.

All the sites accessing these variables are already in functions that
receive the 'state' parameter (or in cmd_fast_import() which owns the
struct), so no additional threading is needed.

As 'state->allow_unsafe_features' is now dereferenced in
check_unsafe_feature(), its 'state' parameter is no longer unused, so
the UNUSED marker is removed.

The fast_import_state_init() call is moved up before the early
command-line scan for '--allow-unsafe-features', so that this option
can be recorded directly into the struct without being clobbered by
the memset() in fast_import_state_init().

This is a mechanical refactoring with no intended behavior change.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/fast-import.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index d20353679a..53f5d39173 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -257,9 +257,7 @@ static struct recent_command *rc_free;
 static unsigned int cmd_save = 100;
 static uintmax_t next_mark;
 static struct strbuf new_data = STRBUF_INIT;
-static int seen_data_command;
 static int require_explicit_termination;
-static int allow_unsafe_features;
 
 /* Signal handling */
 static volatile sig_atomic_t checkpoint_requested;
@@ -277,6 +275,8 @@ struct fast_import_state {
 	int argc;
 	const char **argv;
 	const char *prefix;
+	int seen_data_command;
+	int allow_unsafe_features;
 };
 
 static void fast_import_state_init(struct fast_import_state *state,
@@ -1876,7 +1876,7 @@ static int read_next_command(struct fast_import_state *state)
 			if (stdin_eof)
 				return EOF;
 
-			if (!seen_data_command
+			if (!state->seen_data_command
 				&& !starts_with(command_buf.buf, "feature ")
 				&& !starts_with(command_buf.buf, "option ")) {
 				parse_argv(state);
@@ -3809,9 +3809,9 @@ static int parse_one_option(struct fast_import_state *state, const char *option)
 	return 1;
 }
 
-static void check_unsafe_feature(struct fast_import_state *state UNUSED, const char *feature, int from_stream)
+static void check_unsafe_feature(struct fast_import_state *state, const char *feature, int from_stream)
 {
-	if (from_stream && !allow_unsafe_features)
+	if (from_stream && !state->allow_unsafe_features)
 		die(_("feature '%s' forbidden in input without --allow-unsafe-features"),
 		    feature);
 }
@@ -3860,7 +3860,7 @@ static int parse_one_feature(struct fast_import_state *state, const char *featur
 
 static void parse_feature(struct fast_import_state *state, const char *feature)
 {
-	if (seen_data_command)
+	if (state->seen_data_command)
 		die(_("got feature command '%s' after data command"), feature);
 
 	if (parse_one_feature(state, feature, 1))
@@ -3871,7 +3871,7 @@ static void parse_feature(struct fast_import_state *state, const char *feature)
 
 static void parse_option(struct fast_import_state *state, const char *option)
 {
-	if (seen_data_command)
+	if (state->seen_data_command)
 		die(_("got option command '%s' after data command"), option);
 
 	if (parse_one_option(state, option))
@@ -3939,7 +3939,7 @@ static void parse_argv(struct fast_import_state *state)
 	if (i != state->argc)
 		usage(fast_import_usage);
 
-	seen_data_command = 1;
+	state->seen_data_command = 1;
 	if (import_marks_file)
 		read_marks();
 	build_mark_map(&sub_marks_from, &sub_marks_to);
@@ -3954,6 +3954,8 @@ int cmd_fast_import(int argc,
 
 	show_usage_if_asked(argc, argv, fast_import_usage);
 
+	fast_import_state_init(&state, argc, argv, prefix);
+
 	reset_pack_idx_option(&pack_idx_opts);
 	git_pack_config();
 
@@ -3977,11 +3979,9 @@ int cmd_fast_import(int argc,
 		if (*arg != '-' || !strcmp(arg, "--"))
 			break;
 		if (!strcmp(arg, "--allow-unsafe-features"))
-			allow_unsafe_features = 1;
+			state.allow_unsafe_features = 1;
 	}
 
-	fast_import_state_init(&state, argc, argv, prefix);
-
 	rc_free = mem_pool_alloc(&fi_mem_pool, cmd_save * sizeof(*rc_free));
 	for (unsigned int i = 0; i < (cmd_save - 1); i++)
 		rc_free[i].next = &rc_free[i + 1];
@@ -4028,7 +4028,7 @@ int cmd_fast_import(int argc,
 	}
 
 	/* argv hasn't been parsed yet, do so */
-	if (!seen_data_command)
+	if (!state.seen_data_command)
 		parse_argv(&state);
 
 	if (require_explicit_termination && feof(stdin))
-- 
2.55.0.185.g9120d2b5c0

[PATCH 7/7] fast-import: use struct option for usage string

[Christian Couder]

Currently `git fast-import -h` shows the following on a single line:

  usage : git fast-import [--date-format=<f>] [--max-pack-size=<n>] \
                          [--big-file-threshold=<n>] [--depth=<n>] \
                          [--active-branches=<n>] \
                          [--export-marks=<marks.file>]

This output has a number of issues like:

  - It's missing a lot of options.
  - It's not consistent with the SYNOPSIS section of the doc.
  - With `--help-all` instead of `-h` additional hidden options should
    be shown, but that's not the case.
  - It's not standard style anymore.
  - Most other Git commands show additional lines for most of the
    options they support.

Also while most commands use the parse-options API to handle their
options, "builtin/fast-import.c" still doesn't use it.

Let's improve on that by using the parse-options API to display the
options when `-h` and `--help-all` are used.

While at it, let's make the SYNOPSIS section of
"Documentation/git-fast-import.adoc" consistent with the new usage
string.

This deliberately leaves it to future work to also use the
parse-options API to actually parse the options.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/git-fast-import.adoc |  2 +-
 builtin/fast-import.c              | 86 +++++++++++++++++++++++++++---
 t/t0450/adoc-help-mismatches       |  1 -
 3 files changed, 81 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-fast-import.adoc b/Documentation/git-fast-import.adoc
index d68bc52b7e..7c5900e048 100644
--- a/Documentation/git-fast-import.adoc
+++ b/Documentation/git-fast-import.adoc
@@ -9,7 +9,7 @@ git-fast-import - Backend for fast Git data importers
 SYNOPSIS
 --------
 [verse]
-frontend | 'git fast-import' [<options>]
+'git fast-import' [<options>]
 
 DESCRIPTION
 -----------
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 53f5d39173..a2952b273f 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -30,6 +30,7 @@
 #include "khash.h"
 #include "date.h"
 #include "gpg-interface.h"
+#include "parse-options.h"
 
 #define PACK_ID_BITS 16
 #define MAX_PACK_ID ((1<<PACK_ID_BITS)-1)
@@ -277,16 +278,18 @@ struct fast_import_state {
 	const char *prefix;
 	int seen_data_command;
 	int allow_unsafe_features;
+	struct option *option;
 };
 
 static void fast_import_state_init(struct fast_import_state *state,
 				   int argc, const char **argv,
-				   const char *prefix)
+				   const char *prefix, struct option *option)
 {
 	memset(state, 0, sizeof(*state));
 	state->argc = argc;
 	state->argv = argv;
 	state->prefix = prefix;
+	state->option = option;
 }
 
 static void parse_argv(struct fast_import_state *state);
@@ -3907,8 +3910,10 @@ static void git_pack_config(void)
 	repo_config(the_repository, git_default_config, NULL);
 }
 
-static const char fast_import_usage[] =
-"git fast-import [--date-format=<f>] [--max-pack-size=<n>] [--big-file-threshold=<n>] [--depth=<n>] [--active-branches=<n>] [--export-marks=<marks.file>]";
+static const char *const fast_import_usage[] = {
+	N_("git fast-import [<options>]"),
+	NULL
+};
 
 static void parse_argv(struct fast_import_state *state)
 {
@@ -3937,7 +3942,7 @@ static void parse_argv(struct fast_import_state *state)
 		die(_("unknown option --%s"), a);
 	}
 	if (i != state->argc)
-		usage(fast_import_usage);
+		usage_with_options(fast_import_usage, state->option);
 
 	state->seen_data_command = 1;
 	if (import_marks_file)
@@ -3952,9 +3957,78 @@ int cmd_fast_import(int argc,
 {
 	struct fast_import_state state;
 
-	show_usage_if_asked(argc, argv, fast_import_usage);
+	unsigned long pack_size_limit, big_file_threshold, depth, active_branches;
+	char *edges, *signed_commits, *signed_tags, *date_format, *import_marks;
+	char *import_marks_if_exists, *export_marks, *submodules_from, *submodules_to;
+	int opt_quiet, opt_show_stats, opt_relative_marks, opt_force, opt_done;
+	int opt_allow_unsafe;
+	int cat_blob;
 
-	fast_import_state_init(&state, argc, argv, prefix);
+	/*
+	 * NEEDSWORK: For now this is used only to render
+	 * `-h`/`--help-all` usage messages. The actual parsing is
+	 * done by parse_one_option()/parse_one_feature().
+	 */
+	struct option fast_import_options[] = {
+		OPT_GROUP(N_("Common")),
+		OPT_STRING_F(0, "date-format", &date_format, N_("fmt"),
+			   N_("format of the commit/tag dates"), PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "stats", &opt_show_stats,
+			   N_("display some basic statistics (objects, packfiles and memory)"),
+			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "quiet", &opt_quiet,
+			   N_("disable the output shown by --stats"), PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "force", &opt_force,
+			   N_("force updating modified existing branches"), PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "done", &opt_done,
+			   N_("require a terminating 'done' command"), PARSE_OPT_NONEG),
+		OPT_UNSIGNED(0, "max-pack-size", &pack_size_limit,
+			     N_("maximum size of each output pack file")),
+		OPT_UNSIGNED(0, "big-file-threshold", &big_file_threshold,
+			     N_("maximum size of a blob that will be deltified")),
+		OPT_UNSIGNED(0, "depth", &depth,
+			     N_("maximum delta depth")),
+		OPT_UNSIGNED(0, "active-branches", &active_branches,
+			     N_("maximum number of branches to maintain active")),
+		OPT_GROUP(N_("Marks")),
+		OPT_STRING_F(0, "import-marks", &import_marks, N_("file"),
+			     N_("import marks from <file>"), PARSE_OPT_NONEG),
+		OPT_STRING_F(0, "import-marks-if-exists", &import_marks_if_exists, N_("file"),
+			     N_("import marks from <file> if it exists"), PARSE_OPT_NONEG),
+		OPT_STRING_F(0, "export-marks", &export_marks, N_("file"),
+			     N_("dump marks to <file>"), PARSE_OPT_NONEG),
+		OPT_BOOL(0, "relative-marks", &opt_relative_marks,
+			 N_("are --(import|export)-marks= paths relative to '.git/info/fast-import'?")),
+		OPT_GROUP(N_("Submodule rewrite")),
+		OPT_STRING_F(0, "rewrite-submodules-from", &submodules_from, N_("name:filename"),
+			     N_("rewrite object IDs for submodule <name> from <filename>"),
+			     PARSE_OPT_NONEG),
+		OPT_STRING_F(0, "rewrite-submodules-to", &submodules_to, N_("name:filename"),
+			     N_("rewrite object IDs for submodule <name> to <filename>"),
+			     PARSE_OPT_NONEG),
+		OPT_GROUP(N_("Signing")),
+		OPT_STRING_F(0, "signed-commits", &signed_commits, N_("mode"),
+			     N_("how to handle signed commits"),
+			     PARSE_OPT_NONEG),
+		OPT_STRING_F(0, "signed-tags", &signed_tags, N_("mode"),
+			     N_("how to handle signed tags"),
+			     PARSE_OPT_NONEG),
+		OPT_HIDDEN_GROUP(N_("Advanced")),
+		OPT_BOOL_F(0, "allow-unsafe-features", &opt_allow_unsafe,
+			   N_("allow unsafe mark commands from the stream"),
+			   PARSE_OPT_HIDDEN | PARSE_OPT_NONEG),
+		OPT_STRING_F(0, "export-pack-edges", &edges, N_("file"),
+			     N_("dump edge commits to <file>"),
+			     PARSE_OPT_HIDDEN | PARSE_OPT_NONEG),
+		OPT_INTEGER_F(0, "cat-blob-fd", &cat_blob,
+			    N_("write some responses to <fd> instead of stdout"),
+			      PARSE_OPT_HIDDEN | PARSE_OPT_NONEG),
+		OPT_END()
+	};
+
+	show_usage_with_options_if_asked(argc, argv, fast_import_usage, fast_import_options);
+
+	fast_import_state_init(&state, argc, argv, prefix, fast_import_options);
 
 	reset_pack_idx_option(&pack_idx_opts);
 	git_pack_config();
diff --git a/t/t0450/adoc-help-mismatches b/t/t0450/adoc-help-mismatches
index e8d6c13ccd..85b039a4be 100644
--- a/t/t0450/adoc-help-mismatches
+++ b/t/t0450/adoc-help-mismatches
@@ -13,7 +13,6 @@ credential
 credential-cache
 credential-store
 fast-export
-fast-import
 fetch-pack
 fmt-merge-msg
 format-patch
-- 
2.55.0.185.g9120d2b5c0

Re: [PATCH 1/7] parse-options: introduce OPT_HIDDEN_GROUP

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> Hidden options are not shown by `git <cmd> -h`, but are still shown by
> `git <cmd> --help-all`. If there are a lot of hidden options or if they
> don't belong to the same categories as other options, there is
> currently no way to properly group them.
>
> Using `OPT_GROUP("Foo")` means that "Foo" will always be shown which we
> don't want if that group contains only hidden options.
>
> To provide a way to have groups shown only when hidden options are
> shown, let's implement an OPT_HIDDEN_GROUP macro.
>
> To test this new macro, let's also improve `test-tool parse-options`
> and test its output with `--help-all`.
>
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---

We have been doing this for quite some time, but I just noticed
that the 'From' address your MUA uses ("Christian Couder
<christian.couder@gmail.com>") does not match your Sign-off.  Could
you add an in-body 'From:' line if you plan to keep sending your
patches from the Gmail address?

I suppose nobody has noticed it so far because .mailmap hides the
discrepancy once the commit lands.

The changes in this step looks alright, though.

Thanks.

Re: [PATCH 7/7] fast-import: use struct option for usage string

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> +	unsigned long pack_size_limit, big_file_threshold, depth, active_branches;
> +	char *edges, *signed_commits, *signed_tags, *date_format, *import_marks;
> +	char *import_marks_if_exists, *export_marks, *submodules_from, *submodules_to;
> +	int opt_quiet, opt_show_stats, opt_relative_marks, opt_force, opt_done;
> +	int opt_allow_unsafe;
> +	int cat_blob;
>  
> -	fast_import_state_init(&state, argc, argv, prefix);
> +	/*
> +	 * NEEDSWORK: For now this is used only to render
> +	 * `-h`/`--help-all` usage messages. The actual parsing is
> +	 * done by parse_one_option()/parse_one_feature().
> +	 */

OK, I am a bit torn on this.  On one hand:

 (1) I do agree that it would be nice to eventually have
     fast_import_state_init() (or some other helper that groks
     argc/argv) use this options array to parse the command line
     arguments.

 (2) I am sympathetic to the position that doing so is a bit
     outside the scope of this series, whose focus is strictly on
     "git fast-import -h" and nothing else.

 (3) I suspect that when fast_import_state_init() does start using
     the options array to initialize the state, the parsed results
     will not be stored in the variables this caller currently holds,
     but will instead live inside the fast_import_state structure.

So in that sense, the huge list of unused function-local variables
above are merely throw-away placeholders.  When the real code is
written, they will disappear, and the references to them in the
fast_import_options[] array will have to be updated to point to
members of the structure (or global variables).

Still, seeing all of those variables left uninitialized leaves a
slightly sour taste.  And because of (3), it would be a clear waste
of time to go through the motions of initializing these throw-away
locals.

Perhaps we would end up in a better position if we bent (2) a bit.
After all, my hesitation likely stems from the feeling that this
series stops short at a slightly awkward spot, having already
completed 90% of the journey.

For example, instead of inventing a local, throw-away
"pack_size_limit" variable, wouldn't it make more sense to refer to
the existing global "max_packsize" variable from the options[]
array below?

> +	struct option fast_import_options[] = {
> +		OPT_GROUP(N_("Common")),
> +		OPT_STRING_F(0, "date-format", &date_format, N_("fmt"),
> +			   N_("format of the commit/tag dates"), PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "stats", &opt_show_stats,
> +			   N_("display some basic statistics (objects, packfiles and memory)"),
> +			   PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "quiet", &opt_quiet,
> +			   N_("disable the output shown by --stats"), PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "force", &opt_force,
> +			   N_("force updating modified existing branches"), PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "done", &opt_done,
> +			   N_("require a terminating 'done' command"), PARSE_OPT_NONEG),
> +		OPT_UNSIGNED(0, "max-pack-size", &pack_size_limit,
> +			     N_("maximum size of each output pack file")),
> +		OPT_UNSIGNED(0, "big-file-threshold", &big_file_threshold,

Thanks.

.mailmap etiquette (was "Re: [PATCH 1/7] parse-options: introduce OPT_HIDDEN_GROUP")

[D. Ben Knoble]

On Thu, Jul 16, 2026 at 5:17 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:
>
> > Hidden options are not shown by `git <cmd> -h`, but are still shown by
> > `git <cmd> --help-all`. If there are a lot of hidden options or if they
> > don't belong to the same categories as other options, there is
> > currently no way to properly group them.
> >
> > Using `OPT_GROUP("Foo")` means that "Foo" will always be shown which we
> > don't want if that group contains only hidden options.
> >
> > To provide a way to have groups shown only when hidden options are
> > shown, let's implement an OPT_HIDDEN_GROUP macro.
> >
> > To test this new macro, let's also improve `test-tool parse-options`
> > and test its output with `--help-all`.
> >
> > Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> > ---
>
> We have been doing this for quite some time, but I just noticed
> that the 'From' address your MUA uses ("Christian Couder
> <christian.couder@gmail.com>") does not match your Sign-off.  Could
> you add an in-body 'From:' line if you plan to keep sending your
> patches from the Gmail address?
>
> I suppose nobody has noticed it so far because .mailmap hides the
> discrepancy once the commit lands.
>
> The changes in this step looks alright, though.
>
> Thanks.

Speaking of .mailmap… if I were going to send future patches under a
new email address, would you prefer

(a) a series with the 1st commit being a .mailmap update (subsequent
commits bearing the new email address, of course, but unrelated to the
.mailmap update)
(b) a one-patch email with a .mailmap update
(c) the same as (b), but only after commits with the new email address
have stabilized in next or master

?

[PS happy to drop anyone from CC who doesn't want this one; list
etiquette is a bit unclear to me on this kind of subject change,
though I would lean towards emptying CC and keeping only Junio…]

-- 
D. Ben Knoble

Re: .mailmap etiquette (was "Re: [PATCH 1/7] parse-options: introduce OPT_HIDDEN_GROUP")

[Junio C Hamano]

"D. Ben Knoble" <ben.knoble@gmail.com> writes:

> Speaking of .mailmap… if I were going to send future patches under a
> new email address, would you prefer
>
> (a) a series with the 1st commit being a .mailmap update (subsequent
> commits bearing the new email address, of course, but unrelated to the
> .mailmap update)
> (b) a one-patch email with a .mailmap update
> (c) the same as (b), but only after commits with the new email address
> have stabilized in next or master

I am not sure what sort of complexity you are anticipating, but
having you send (b) and me applying it directly to 'master'
would be the simplest approach, wouldn't it?  After all, adding
a new entry to .mailmap does not invalidate your old identity;
it merely links the new one to the same person.