Re: [PATCH 00/17] object-store: carve out the object database subsystem

[Junio C Hamano <gitster@pobox.com>]

Derrick Stolee <stolee@gmail.com> writes:

> Patches 1 and 2 involve renaming some core structures, and I had
> some questions around these names (since we hope to be stuck with
> the new names for a long time). I was thinking out loud on a per-
> patch basis, but now want to collect my thoughts around these:
>
>  * raw_object_store currently describes the abstraction that contains
>    all objects that can be accessed within the repository. This may
>    include multiple alternates. Patch 1 renames this to
>    'object_database'.
>
>  * object_directory currently describes a single directory that
>    has the same structure as $GIT_DIR/objects/ but may be an alternate
>    or a submodule object directory. Patch 2 renames this to
>    'odb_backend'.
>
> My concerns around this are basically around not liking "backend" for
> this purpose. When I think of a backend, I'm thinking about the
> implementation details (like the refs backend being files or reftable)
> and not multiple distinct locations that have their own objects.

Yup, odb_backend_files (aot odb_backend_redis) or something?

>  * 'struct object_directory' could be renamed to 'struct odb_shard' or
>    'struct odb_slice' or similar. I may even recommend 'odb_partition'
>    though that does imply some disjointness that is not guaranteed (an
>    object can exist in multiple parts).
>
>  * In the event that we create multiple implementations for storing
>    objects, then a 'struct odb_shard' could point to a backend to help
>    find the appropriate methods for interacting with its storage.

Hmph, I do not have strong opinions, but I consider it an
implementation detail of one particular backend, namely, the
filesystem based backend, that it can link together multiple
object_directory instances and present them as if they form a single
object database, just like all files within a single object_directory
form an illusion of a single object database (aka key-value store) even
though some objects are stored in individual loose object files while
many others are packed in a single packfile.

I did not expect you would want to go to the world where a single
"shard" consists of an object_directory backed by the filesystem and
some other more database-y backend.  It is an interesting idea, but
we'd need to worry about many things we do not have to worry about
right now.  E.g. what do the precedence rules among different
components within a single "shard" look like?  How do we express "in
this repository, local filesystem-backed piece is consulted first,
and then check this piece backed by low-cost but high-latency
storage backend"?

> I do mention that the rename of the object-store.[c|h] files may be
> unnecessary, or perhaps could be delayed until this series is merged
> and the collateral is calmed.

Right now, merge-fix needed against all other topics in flight look
like this, in order to merge it to 'seen'.

$ git show --oneline -U0 merge-fix/ps/object-store
c8afa0ab8e merge-fix/ps/object-store
diff --git a/fetch-object-info.c b/fetch-object-info.c
index 1b2c6c4d27..6c2abb7805 100644
--- a/fetch-object-info.c
+++ b/fetch-object-info.c
@@ -7 +7 @@
-#include "object-store.h"
+#include "odb.h"
diff --git a/fetch-object-info.h b/fetch-object-info.h
index 7910d7f153..d35284bd6b 100644
--- a/fetch-object-info.h
+++ b/fetch-object-info.h
@@ -6 +6 @@
-#include "object-store.h"
+#include "odb.h"
diff --git a/transport.h b/transport.h
index 5a4f27451a..3ef3acf865 100644
--- a/transport.h
+++ b/transport.h
@@ -8 +8 @@
-#include "object-store.h"
+#include "odb.h"

IOW, not too horrible.