Re: [PATCH 3/7] odb/streaming: support streaming arbitrary object types

[Patrick Steinhardt <ps@pks.im>]

On Tue, Aug 04, 2026 at 11:54:41AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The object database supports the ability to write object streams into
> > it. This functionality is used when we encounter a blob that is larger
> > than "core.bigFileThreshold" so that we don't have to soak large files
> > into memory.
> 
> I am still not sold the benefit of using a single "stream" type both
> for reading and writing yet at this point in my reading (I am not
> yet done 50% of the series yet at step 3/7), but I agree that it
> would be a good thing to be able to stream objects that are not
> blobs.

The reason why I want to unify these two streams is mostly that despite
their name, they basically do the exact same thing: both stream types
allow the user to read data from them in a streaming fashion. The only
thing that's different about the "write" stream is that it doesn't
encode its information as part of the stream itself, whereas the "read"
stream does. So having two types is quite pointless in the first place.

> > diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
> > index 01bb81c63c..4f76db5496 100644
> > --- a/odb/source-inmemory.c
> > +++ b/odb/source-inmemory.c
> > @@ -293,7 +293,7 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source,
> >  	hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid);
> >  
> >  	ret = odb_source_inmemory_write_object(source, data, stream->size,
> > -					       OBJ_BLOB, oid, NULL, NULL, 0);
> > +					       stream->type, oid, NULL, NULL, 0);
> 
> It is a bit annoying that we treat 'inmemory' as if it were a valid
> single word both in the filename and in the function name, but more
> importantly, hash_object_file() (used to compute the object name of
> the object we are writing into the variable 'oid') still hashes
> assuming that the object is a blob.  What is the implication of
> feeding the data to odb_source_in_memory_write_object() as
> stream->type (which is not necessarily OBJ_BLOB) with that 'oid'
> whose object name was computed as OBJ_BLOB?

Oh, that's an oversight on my part. We'd use the wrong object header,
thus arrive at a wrong hash and then ultimately store the object under
the wrong hash in the in-memory source. Which doesn't really matter
after this patch series as we still only write blobs via streams, but
it's a bug waiting to happen. Fixed now.

Patrick