Re: [PATCH RFC 0/3] cat-file: make remote-object-info request in batches

["Pablo Sabater" <pabloosabaterr@gmail.com>]

On Tue Oct 27, 2026 at 7:12 PM WET, Junio C Hamano wrote:
> Pablo Sabater <pabloosabaterr@gmail.com> writes:
>
>> A noticeable difference can be seen from a 10k OID batch to a 50k batch.
>> After that the difference isn't as noticeable (the number of batches to
>> be requested isn't that big).
>>
>> I propose 50k OIDs as the default number of OIDs for the
>> remote-object-info batches. Coincidentally, git backfill's default batch
>> size is 50k as well though it's unrelated.
>
> Interesting.  We cannot share any code with backfill as we only
> retrieve info on objects while backfill does grab the objects
> themselves, but using the same number may make it easy to explain in
> the documentation.
>
>> I have barely done any benchmarking before, so I would appreciate
>> any feedback on the one in patch 2. Is there a better way to do it,
>> or other things I should take into account?
>>
>> Should the server also check for a limit on the number of OIDs it
>> accepts in a single object-info request?
>
> What I am more curious about is how this updated code interacts with
> existing code (i.e., updated "cat-file --batch" clients with deployed
> servers, and vice versa).

The code in this series only changes the client side, the server has
no OID limit. Before this series, what prevented large requests was the
client with its 10k hard limit, so the server is fine with any size
without any change.

The server will see requests of at most 10k OIDs from old clients, and
bigger ones from new clients.

The other change compared to an older client is that now multiple
requests can be received on the same connection (instead of one per
connection), but that should be fine, serve() keeps processing requests
until EOF.

With this new option there's a risk (that was already there if someone
modified their client) of setting a huge amount of OIDs in the same
request, but I'm not sure how bad that is, a 1M OID request vs smaller
ones on the same connection. Could you help me with this?

Anyway, the fix would be to add a cap on the server side. It can't be
lower than 10k, or older clients would get an error they can't do
anything about, and if it's lower than the client's batch size it
should be advertised so newer clients can adapt.

IIRC clients ignore unknown capability keys, so advertising it shouldn't
break older ones. In this case, it would be advertised on the handshake
and the client would die() if the user sets a batch size bigger than
what the server advertises. Ideally the default on the client (the
proposed 50k) would be the min cap on the server, so someone that
doesn't manually set the batch size and uses the default value doesn't
get an error.

If we don't even want to die() we could set the batch size as:
	min(user-size, advertised-size)

Sorry for the brainstorm but I just realised that any client could
modify the size and that a cap on the server side could be important.

Regards,
Pablo