From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, me@ttaylorr.com, newren@gmail.com,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Teng Long" <dyroneteng@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Derrick Stolee" <derrickstolee@github.com>,
"Derrick Stolee" <derrickstolee@github.com>
Subject: [PATCH 09/24] bundle-uri: create bundle_list struct and helpers
Date: Fri, 20 May 2022 18:40:27 +0000 [thread overview]
Message-ID: <f87cb15a46d2da367e5abbd5da8bcdcc9e481514.1653072042.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1234.git.1653072042.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
bundle-uri.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++
bundle-uri.h | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 128 insertions(+)
diff --git a/bundle-uri.c b/bundle-uri.c
index 0692a62071a..b9a219d3202 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -5,6 +5,67 @@
#include "object-store.h"
#include "refs.h"
#include "run-command.h"
+#include "hashmap.h"
+#include "pkt-line.h"
+
+static int compare_bundles(const void *hashmap_cmp_fn_data,
+ const struct hashmap_entry *he1,
+ const struct hashmap_entry *he2,
+ const void *id)
+{
+ const struct remote_bundle_info *e1 =
+ container_of(he1, const struct remote_bundle_info, ent);
+ const struct remote_bundle_info *e2 =
+ container_of(he2, const struct remote_bundle_info, ent);
+
+ return strcmp(e1->id, id ? (const char *)id : e2->id);
+}
+
+void init_bundle_list(struct bundle_list *list)
+{
+ memset(list, 0, sizeof(*list));
+
+ /* Implied defaults. */
+ list->mode = BUNDLE_MODE_ALL;
+ list->version = 1;
+
+ hashmap_init(&list->bundles, compare_bundles, NULL, 0);
+}
+
+static int clear_remote_bundle_info(struct remote_bundle_info *bundle,
+ void *data)
+{
+ free(bundle->id);
+ free(bundle->uri);
+ strbuf_release(&bundle->file);
+ return 0;
+}
+
+void clear_bundle_list(struct bundle_list *list)
+{
+ if (!list)
+ return;
+
+ for_all_bundles_in_list(list, clear_remote_bundle_info, NULL);
+ hashmap_clear_and_free(&list->bundles, struct remote_bundle_info, ent);
+}
+
+int for_all_bundles_in_list(struct bundle_list *list,
+ bundle_iterator iter,
+ void *data)
+{
+ struct remote_bundle_info *info;
+ struct hashmap_iter i;
+
+ hashmap_for_each_entry(&list->bundles, &i, info, ent) {
+ int result = iter(info, data);
+
+ if (result)
+ return result;
+ }
+
+ return 0;
+}
static void find_temp_filename(struct strbuf *name)
{
diff --git a/bundle-uri.h b/bundle-uri.h
index 8a152f1ef14..0f95197744c 100644
--- a/bundle-uri.h
+++ b/bundle-uri.h
@@ -1,7 +1,74 @@
#ifndef BUNDLE_URI_H
#define BUNDLE_URI_H
+#include "hashmap.h"
+#include "strbuf.h"
+
struct repository;
+struct strbuf;
+struct string_list;
+
+/**
+ * The remote_bundle_info struct contains information for a single bundle
+ * URI. This may be initialized simply by a given URI or might have
+ * additional metadata associated with it if the bundle was advertised by
+ * a bundle list.
+ */
+struct remote_bundle_info {
+ struct hashmap_entry ent;
+
+ /**
+ * The 'id' is a name given to the bundle for reference
+ * by other bundle infos.
+ */
+ char *id;
+
+ /**
+ * The 'uri' is the location of the remote bundle so
+ * it can be downloaded on-demand. This will be NULL
+ * if there was no table of contents.
+ */
+ char *uri;
+
+ /**
+ * If the bundle has been downloaded, then 'file' is a
+ * filename storing its contents. Otherwise, 'file' is
+ * an empty string.
+ */
+ struct strbuf file;
+};
+
+#define REMOTE_BUNDLE_INFO_INIT { \
+ .file = STRBUF_INIT, \
+}
+
+enum bundle_list_mode {
+ BUNDLE_MODE_NONE = 0,
+ BUNDLE_MODE_ALL,
+ BUNDLE_MODE_ANY
+};
+
+/**
+ * A bundle_list contains an unordered set of remote_bundle_info structs,
+ * as well as information about the bundle listing, such as version and
+ * mode.
+ */
+struct bundle_list {
+ int version;
+ enum bundle_list_mode mode;
+ unsigned forFetch : 1;
+ struct hashmap bundles;
+};
+
+void init_bundle_list(struct bundle_list *list);
+void clear_bundle_list(struct bundle_list *list);
+
+typedef int (*bundle_iterator)(struct remote_bundle_info *bundle,
+ void *data);
+
+int for_all_bundles_in_list(struct bundle_list *list,
+ bundle_iterator iter,
+ void *data);
/**
* Fetch data from the given 'uri' and unbundle the bundle data found
--
gitgitgadget
next prev parent reply other threads:[~2022-05-20 18:42 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-20 18:40 [PATCH 00/24] [RFC] Bundle URIs Combined RFC Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 01/24] docs: document bundle URI standard Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 02/24] remote-curl: add 'get' capability Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 03/24] bundle-uri: create basic file-copy logic Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 04/24] bundle-uri: add support for http(s):// and file:// Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 05/24] fetch: add --bundle-uri option Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 06/24] fetch: add 'refs/bundle/' to log.excludeDecoration Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 07/24] clone: add --bundle-uri option Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 08/24] clone: --bundle-uri cannot be combined with --depth Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` Derrick Stolee via GitGitGadget [this message]
2022-05-20 18:40 ` [PATCH 10/24] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 11/24] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 12/24] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 13/24] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 14/24] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 15/24] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 16/24] protocol v2: add server-side "bundle-uri" skeleton Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 17/24] bundle-uri client: add minimal NOOP client Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 18/24] bundle-uri client: add "git ls-remote-bundle-uri" Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 19/24] bundle-uri: serve URI advertisement from bundle.* config Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 20/24] bundle-uri client: add boolean transfer.bundleURI setting Ævar Arnfjörð Bjarmason via GitGitGadget
2022-05-20 18:40 ` [PATCH 21/24] bundle-uri: allow relative URLs in bundle lists Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 22/24] bundle-uri: download bundles from an advertised list Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 23/24] clone: unbundle the advertised bundles Derrick Stolee via GitGitGadget
2022-05-20 18:40 ` [PATCH 24/24] t5601: basic bundle URI tests Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f87cb15a46d2da367e5abbd5da8bcdcc9e481514.1653072042.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=derrickstolee@github.com \
--cc=dyroneteng@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
--cc=newren@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).