From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from flow-b4-smtp.messagingengine.com (flow-b4-smtp.messagingengine.com [202.12.124.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 418B12EA749 for ; Sat, 10 Oct 2026 15:06:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.139 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791644806; cv=none; b=OmXzj8q0EzPLQZuQahTuR5LKWXjOH8FOO5SpXI98hdsp4YiOBbUYDDr3o/2OSZn0pNhJkCUxEwZ2hZ4eIEPUfl+AY6G7ay4NyvLW18z5Fqrqrm/j5Q+fUcZnucllR29Arbl4/e2Fo1Fi/0nUevLH50tDR84NixrsGBfLfY/Nx2M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791644806; c=relaxed/simple; bh=hvB5t9WqWRIRPz//YaUCI6U6nfsXm9szEaJDa72PMkg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=otreKuyrmhSqP+JiNvjnxZOyUTg9htIHRGxJ6XvoFR7RVqCV6quzLXJlVKly5ZiFylHcZ0EOJIS6WpiUMty18hmLdqp597Zg174C4XxiRHP1RMbhZgTxTsEfmJHaANTVOKuJrtt+st7mMkEqZDmhz8oZe7PMrJBkAoP42MI/vPE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=fastmail.com; spf=pass smtp.mailfrom=fastmail.com; dkim=pass (2048-bit key) header.d=fastmail.com header.i=@fastmail.com header.b=W0oLjH6y; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=vJt/Pv1N; arc=none smtp.client-ip=202.12.124.139 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=fastmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fastmail.com header.i=@fastmail.com header.b="W0oLjH6y"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="vJt/Pv1N" Received: from phl-compute-11.internal (phl-compute-11.internal [10.202.2.51]) by mailflow.stl.internal (Postfix) with ESMTP id 896C91300060 for ; Sat, 10 Oct 2026 11:06:43 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-11.internal (MEProxy); Sat, 10 Oct 2026 11:06:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm2; t=1791644803; x= 1791648403; bh=+toli6OGLWC6w74uRZCSL0fZBcoUlVxvDKPWsc7X4U4=; b=W 0oLjH6yV5uW5URyXCcvFdr2qHuYMqTBRd+f+/iauQKnMo6Yk73dgYwyVTWkCi6rB /Jxg9Otk4xVrScDl0xHeyIEhumhbHIlAlqGkYueEP55gVq4eAYLmdj29usEeb5Rm 4HS5FCdZj8BYFKWAnvli7YiQ2qY+oOLjiJtks9i61Pb2oim7z/KvUiCQa9E6S/SK T+aIC8RMICqEMJFLSdNLb2YTaWpgp/bnrTX9H3IYt1Y4z988fAYaI4R22jATVKQ9 GjklCClp4h5noRBA5DcFYf0H2+Bkm3pSswZxjcVky6aEdfFyCBSvAsYLeKSBRWZv 5vpfjvdUho8O+7BNhsdlQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1791644803; x=1791648403; bh=+ toli6OGLWC6w74uRZCSL0fZBcoUlVxvDKPWsc7X4U4=; b=vJt/Pv1NpIYghT9Vx 1y5GzjxdRXYiq8dK/byFHIxq6uUGbH5Dj/+xNSPmUTCbjAnQlxAFiFKMmvBCKD96 zuQfFebCYkhvUJBGnT0D8qKVUGUmHOs7VsyEJvwenueMIbvqK2BF640xcJVRFRth sS8ooHeAB6Ffq0g7FbFm6bd36t8KKVJFxcy2YvtLGuhqhB6jz/9LnTG7mrI/SEw3 Krw3pqF/IRoupc0Iog31j0NjpGUTOVY32u2MTLADU2JKFOq0rrPliWN3N2Uc8WHn 8wxrxkA0LUVUgf+UGDSaXCAzQFW2eELUFvLyd4ZDBIjFNwjZ5vfJ4E5Y5FotdXXG 0atCw== X-DKIM2-Info: draft=ietf-dkim-dkim2-spec-06; repo=github.com/dkim2wg/interop; date=2026-10-04; sw=lmtpprox; action=sign d=fastmail.com a=rsa-sha256; DKIM2-Signature: i=1; m=1; t=1791644803; d=fastmail.com; mf=PG1hcmtjaHVjYXJyb2xsQGZhc3RtYWlsLmNvbT4=; rt=PGdpdEB2Z2VyLmtlcm5lbC5vcmc+; s=fm2:rsa-sha256:a+ZEHaXIxEGU/hn/ca/kEpxbs/i5Tdso+5io1FkeBYljkwh 4L6GXGX8f4Eg8fvWn/xt7UKo3/5iD1wdsnlyfhBN3J1lV3yUerZZJX3Zidd5lTVS e2qCyuUGni+bsuYpc+/J7DvGO1y8/AATgUQOwhrQlIpv3fk5S8Peel3Q13pQiuVv koTJWyFV7gNlez+WkO541YbUNCPyGAlSrTHQE3J5ajo28kxSH4L+dW6YH1cjIUGt OzRo+ofaijkV4faw1kzm/Xf674RpWUeX0z39veyjl9JK2ZPN/33pL5Oo8T5VUd+b BTUgqlcH10cifvOLKl7yQUN3huMiQnr0YVYKtGg==; X-DKIM2-Info: draft=ietf-dkim-dkim2-spec-06; repo=github.com/dkim2wg/interop; date=2026-10-04; sw=lmtpprox; action=mi-m=1; hc=11; hn=cc,content-transfer-encoding,date,feedback-id,from, in-reply-to,message-id,mime-version,references,subject,to; Message-Instance: m=1; h=sha256:PcPmCzRstvBWnKB4KO29LkfQYJ/aybSvLIGLGXXOwo0=:hvB5t9WqWRIRPz//YaUCI6U6nfsXm9szEaJDa72PMkg=; X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTFlsopxBlJyrhE+80+VjJi8Zynjz2f2klZS9pqdfaQ2Qwtv/0BXE37OTEa7xA1xhi MdWWF6c24+rNF0VCzCaZuCEDj1Mk7EVYffEZ9y9vUVBm3Twwn3L4WEu5Pq37ta91ynzIDl 3Qtq8qS+C+SuF0qgQaOfSf1HxVdYnScdCwMslaNwZWxPesQhU4EpOJpsJB1fAx7PgN55EW CauZOY7ajy24CqphL3hjw3IIzg4nYdVvHVu5TNi4v9C1kgfLzXbeelJ6Dq+0cHA+b86SBu 1GUSUt/p12Au7q34R5JnZdoR3pg0FRfPkClvjSCcXafiFYv4HvM+4v8B/tQOlpDU0YY1Nb /x4jyLiLSuLv9HOdRzgFuMjRNWcsg+CcsL3LV8YvhXM740RUXoOfw8XEKOWOmEV6Y2qYzn ZwU99QC9qI5YBcd0kby53bihPBgzi+Um+nXCG2A/z6EkRicjtwlt0B6vDp4KdNnRdqBrPy S2fiiNF7NCZV/QQocDlX/klodQBDELHY9JxxlmROpajIuPD6cQMlHRShYh8S2si/1JS2xP sqdiQByZrb5jyLiMdryySZB11lkMvJFFBpcdwKuW3ZDpjB32adw9Lc2wQfyLHbv62TYWOe gOhENYPqB89L62Bef14kWDVy/8oTXYZHyYerhaZcvGvGpmN/p9Qes1QwTbrA X-ME-Proxy: Feedback-ID: id2564aa6:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 10 Oct 2026 11:06:42 -0400 (EDT) From: "Mark C. Chu-Carroll" To: git@vger.kernel.org Cc: jltobler@gmail.com, ps@pks.im, "Mark C. Chu-Carroll" Subject: [PATCH v4 1/1] repo: add filtering options to "repo structure" Date: Sat, 10 Oct 2026 11:05:41 -0400 Message-ID: <20261010150539.930614-4-markchucarroll@fastmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261010150539.930614-2-markchucarroll@fastmail.com> References: <20260924164503.119506-2-markchucarroll@fastmail.com> <20261010150539.930614-2-markchucarroll@fastmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The current implementation of "git repo stucture" provides summary information about everything in the repository - all of the branches, remotes, tags, stashes, and notes. But sometimes to properly diagnose a problem, it's useful to be able to filter this information to get focused data about the specific part of the repo that's exhibiting a problem. Update "git repo structure" to use rev expressions to filter the set of objects to be processed. This implementation makes an awkward trade-off involving "setup_revisions". In order to accept standard the standard revs parameters, it uses `setup_revisions`, which does a ref walk and populates the pending objects list that will be used by "repo structure". But this ref walk doesn't count refs in the way that "repo structure" requires, and because "setup_revisions" doesn't support a custom callback for its ref-walk, that's not changeable. So in the cases where a user specifies revs, "repo structure" does an initial ref-walk as part of "setup_revisions", and then a second ref-walk in "count_references" in repo.c to count by reference type. Further, if the user doesn't specify a revs parameter, "setup_revisions" is not called, so the "count_references" walk needs to add objects to the pending list. A parameter is added to "count_references" to allow it to distinguish the cases where the pending list is already populated from the case where it needs to populate it. Signed-off-by: Mark C. Chu-Carroll --- Documentation/git-repo.adoc | 32 +++++- builtin/repo.c | 32 ++++-- t/t1901-repo-structure.sh | 211 +++++++++++++++++++++++++++--------- 3 files changed, 210 insertions(+), 65 deletions(-) diff --git a/Documentation/git-repo.adoc b/Documentation/git-repo.adoc index ed7d80c690..67be7efd23 100644 --- a/Documentation/git-repo.adoc +++ b/Documentation/git-repo.adoc @@ -10,7 +10,7 @@ SYNOPSIS [synopsis] git repo info [--format=(lines|nul) | -z] [--all | ...] git repo info --keys [--format=(lines|nul) | -z] -git repo structure [--format=(table|lines|nul) | -z] +git repo structure [--format=(table|lines|nul) | -z] [...] DESCRIPTION ----------- @@ -56,7 +56,7 @@ supported: `nul`::: Similar to `lines`, but using a _NUL_ character after each value. -`structure [--format=(table|lines|nul) | -z]`:: +`structure [--format=(table|lines|nul) | -z] [...]`:: Retrieve statistics about the current repository structure. The following kinds of information are reported: + @@ -66,6 +66,12 @@ supported: * Total disk size of reachable objects by type * Largest reachable objects in the repository by type + +The set of objects counted can be filtered by specifying a collection +of revisions to select which objects will be counted. These parameters +follow the same syntax as the parameters to similar commands like +`git log`. If no includes are specified, then the include set is all +reachable objects. ++ The output format can be chosen through the flag `--format`. Three formats are supported: + @@ -141,6 +147,28 @@ using the `nul` format: git repo info --format=nul layout.bare layout.shallow ------------ +* Generates information about storage usage in the repository: ++ +------------ +git repo structure +------------ ++ + +* Generates information about storage usage in the repository omitting +the branch "foo": ++ +------------ +git repo structure ^foo +------------ ++ +* Generates information about repository objects reachable from +the references "x" and "y", but omitting anything that can +only be reached on a path including "xchild": ++ +------------ +git repo structure x y ^xchild +------------ ++ SEE ALSO -------- linkgit:git-rev-parse[1] diff --git a/builtin/repo.c b/builtin/repo.c index 84e012f83f..081940a85c 100644 --- a/builtin/repo.c +++ b/builtin/repo.c @@ -27,7 +27,7 @@ "git repo info --keys [--format=(lines|nul) | -z]" #define REPO_STRUCTURE_USAGE \ - "git repo structure [--format=(table|lines|nul) | -z]" + "git repo structure [--format=(table|lines|nul) | -z] [...]" static const char *const repo_usage[] = { REPO_INFO_USAGE, @@ -740,6 +740,7 @@ struct count_references_data { struct ref_stats *stats; struct rev_info *revs; struct progress *progress; + bool add_to_pending; }; static int count_references(const struct reference *ref, void *cb_data) @@ -766,10 +767,12 @@ static int count_references(const struct reference *ref, void *cb_data) } /* - * While iterating through references for counting, also add OIDs in + * While iterating through references for counting, if we didn't already + * populate the pending list in setup_revisions, also add OIDs in * preparation for the path walk. */ - add_pending_oid(data->revs, NULL, ref->oid, 0); + if (data->add_to_pending) + add_pending_oid(data->revs, NULL, ref->oid, 0); ref_count = get_total_reference_count(stats); display_progress(data->progress, ref_count); @@ -780,11 +783,13 @@ static int count_references(const struct reference *ref, void *cb_data) static void structure_count_references(struct ref_stats *stats, struct rev_info *revs, struct repository *repo, - int show_progress) + int show_progress, + bool add_to_pending) { struct count_references_data data = { .stats = stats, .revs = revs, + .add_to_pending = add_to_pending }; if (show_progress) @@ -935,6 +940,7 @@ static int cmd_repo_structure(int argc, const char **argv, const char *prefix, struct repo_structure stats = { 0 }; struct rev_info revs; int show_progress = -1; + bool included_revision_args = false; struct option options[] = { OPT_CALLBACK_F(0, "format", &format, N_("format"), N_("output format"), @@ -947,16 +953,20 @@ static int cmd_repo_structure(int argc, const char **argv, const char *prefix, OPT_END() }; - argc = parse_options(argc, argv, prefix, options, repo_structure_usage, 0); - if (argc) - usage(_("too many arguments")); - - repo_init_revisions(repo, &revs, prefix); - + argc = parse_options(argc, argv, prefix, options, repo_structure_usage, + PARSE_OPT_KEEP_ARGV0 | PARSE_OPT_KEEP_UNKNOWN_OPT); if (show_progress < 0) show_progress = isatty(2); - structure_count_references(&stats.refs, &revs, repo, show_progress); + repo_init_revisions(repo, &revs, prefix); + if (argc > 1) { + argc = setup_revisions(argc, argv, &revs, NULL); + included_revision_args = true; + } + if (argc > 1) + usage(_("too many arguments")); + + structure_count_references(&stats.refs, &revs, repo, show_progress, !included_revision_args); structure_count_objects(&stats.objects, &revs, repo, show_progress); switch (format) { diff --git a/t/t1901-repo-structure.sh b/t/t1901-repo-structure.sh index 02cc2b594a..e35b46a277 100755 --- a/t/t1901-repo-structure.sh +++ b/t/t1901-repo-structure.sh @@ -21,58 +21,6 @@ object_type_disk_usage() { fi } -test_expect_success 'empty repository' ' - test_when_finished "rm -rf repo" && - git init repo && - ( - cd repo && - cat >expect <<-\EOF && - | Repository structure | Value | - | ------------------------- | ------ | - | * References | | - | * Count | 0 | - | * Branches | 0 | - | * Tags | 0 | - | * Remotes | 0 | - | * Others | 0 | - | | | - | * Reachable objects | | - | * Count | 0 | - | * Commits | 0 | - | * Trees | 0 | - | * Blobs | 0 | - | * Tags | 0 | - | * Inflated size | 0 B | - | * Commits | 0 B | - | * Trees | 0 B | - | * Blobs | 0 B | - | * Tags | 0 B | - | * Disk size | 0 B | - | * Commits | 0 B | - | * Trees | 0 B | - | * Blobs | 0 B | - | * Tags | 0 B | - | | | - | * Largest objects | | - | * Commits | | - | * Maximum size | 0 B | - | * Maximum parents | 0 | - | * Trees | | - | * Maximum size | 0 B | - | * Maximum entries | 0 | - | * Blobs | | - | * Maximum size | 0 B | - | * Tags | | - | * Maximum size | 0 B | - EOF - - git repo structure >out 2>err && - - test_cmp expect out && - test_line_count = 0 err - ) -' - test_expect_success SHA1 'repository with references and objects' ' test_when_finished "rm -rf repo" && git init repo && @@ -144,6 +92,165 @@ test_expect_success SHA1 'repository with references and objects' ' ) ' +test_expect_success SHA1 'repository with branches' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit_bulk 1005 && + git tag -a foo -m bar && + + oid="$(git rev-parse HEAD)" && + git update-ref refs/remotes/origin/foo "$oid" && + git checkout -b grobble && + test_commit_bulk --ref=refs/heads/grobble 20 && + git checkout master && + test_commit_bulk 20 && + # Also creates a commit, tree, and blob. + git notes add -m foo && + git repo structure --format=lines >actual 2>actual-err && + test_line_count = 0 actual-err && + test_grep "objects.commits.inflated.size=231976" actual && + test_grep "objects.commits.count=1046" actual && + test_grep "objects.blobs.count=1006" actual && + test_grep "objects.trees.count=1006" actual && + test_grep "objects.trees.disk_size=284547" actual && + test_grep "objects.blobs.disk_size=21022" actual && + test_grep "objects.tags.disk_size=126" actual && + test_line_count = 0 actual-err + ) +' + +test_expect_success SHA1 'filter by a positive rev' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit_bulk 1005 && + git tag -a foo -m bar && + + oid="$(git rev-parse HEAD)" && + git update-ref refs/remotes/origin/foo "$oid" && + git checkout -b grobble && + test_commit_bulk --ref=refs/heads/grobble 20 && + git checkout master && + test_commit_bulk 20 && + git notes add -m foo && + + git repo structure --format=lines grobble >actual 2>actual-err && + test_grep "objects.commits.count=1025" actual && + test_grep "objects.commits.disk_size=160455" actual && + test_grep "objects.commits.inflated.size=227371" actual && + test_grep "objects.blobs.count=1005" actual && + test_grep "objects.blobs.disk_size=21003" actual && + test_grep "objects.blobs.inflated_size=11958" actual && + test_grep "objects.trees.count=1005" actual && + test_grep "objects.trees.disk_size=284462" actual && + test_grep "objects.trees.inflated_size=16578363" actual && + test_grep "objects.tags.disk_size=0" actual && + test_line_count = 0 actual-err + ) +' + +test_expect_success SHA1 'filter by both positive and negative revs' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit_bulk 1005 && + git tag -a foo -m bar && + + oid="$(git rev-parse HEAD)" && + git update-ref refs/remotes/origin/foo "$oid" && + git checkout -b grobble && + test_commit_bulk --ref=refs/heads/grobble 20 && + git checkout master && + test_commit_bulk 20 && + # Also creates a commit, tree, and blob. + git notes add -m foo && + git repo structure --format=lines >actual 2>actual-err && + test_line_count = 0 actual-err && + + git repo structure --format=lines grobble ^master >actual 2>actual-err && + test_grep "objects.commits.count=20" actual && + test_grep "objects.commits.disk_size=3122" actual && + test_grep "objects.commits.inflated.size=4411" actual && + test_grep "objects.blobs.count=1005" actual && + test_grep "objects.blobs.disk_size=21003" actual && + test_grep "objects.blobs.inflated_size=11958" actual && + test_grep "objects.trees.count=1" actual && + test_grep "objects.trees.disk_size=53" actual && + test_grep "objects.trees.inflated_size=33063" actual && + test_grep "objects.tags.disk_size=0" actual && + test_line_count = 0 actual-err + ) +' + +test_expect_success 'filter by object type' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit_bulk 1005 && + git tag -a foo -m bar && + + oid="$(git rev-parse HEAD)" && + git update-ref refs/remotes/origin/foo "$oid" && + git checkout -b grobble && + test_commit_bulk --ref=refs/heads/grobble 20 && + git checkout master && + test_commit_bulk 20 && + # Also creates a commit, tree, and blob. + git notes add -m foo && + git repo structure --format=lines --objects --all --filter=object:type=blob >actual 2>actual-err && + cp actual /tmp/test-actual-exclude-blob && + test_grep "objects.commits.disk_size=0" actual && + test_grep "objects.commits.max_size=0" actual && + test_grep "objects.tags.disk_size=0" actual && + test_grep "objects.tags.inflated_size=0" actual && + test_grep "objects.trees.disk_size=0" actual && + test_grep "objects.trees.inflated_size=0" actual && + test_grep "objects.blobs.disk_size=21022" actual && + test_grep "objects.blobs.count=1006" actual && + test_grep "objects.blobs.disk_size=21022" actual && + test_grep "objects.blobs.inflated_size=11962" actual && + test_line_count = 0 actual-err + ) +' + +test_expect_success 'filter using by ref^{tree}' ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit_bulk 1005 && + git tag -a foo -m bar && + + oid="$(git rev-parse HEAD)" && + git update-ref refs/remotes/origin/foo "$oid" && + git checkout -b grobble && + test_commit_bulk --ref=refs/heads/grobble 20 && + git checkout master && + test_commit_bulk 20 && + # Also creates a commit, tree, and blob. + git notes add -m foo && + git repo structure --format=lines HEAD^{tree} >actual 2>err && + test_grep "objects.commits.count=0" actual && + test_grep "objects.trees.count=1" actual && + test_grep "objects.blobs.count=1005" actual && + test_grep "objects.tags.count=0" actual && + test_grep "objects.commits.inflated_size=0" actual && + test_grep "objects.trees.inflated_size=33063" actual && + test_grep "objects.blobs.inflated_size=11958" actual && + test_grep "objects.tags.inflated_size=0" actual && + test_grep "objects.commits.disk_size=0" actual && + test_grep "objects.trees.disk_size=53" actual && + test_grep "objects.blobs.disk_size=21003" actual && + test_grep "objects.tags.disk_size=0" actual && + test_line_count = 0 err + ) +' + test_expect_success SHA1 'lines and nul format' ' test_when_finished "rm -rf repo" && git init repo && -- 2.53.0