[PATCH GSoC RFC v13 00/12] cat-file: add remote-object-info to batch-command

[Pablo Sabater <pabloosabaterr@gmail.com>]

This path series is a continuation of Eric Ju's (eric.peijian@gmail.com) and
Calvin Wan's (calvinwan@google.com) patch series [1] and [2] respectively.

Sometimes it is beneficial to retrieve information about an object without
having to download it completely. The server logic for retrieving size has
already been implemented and merged in "a2ba162cda (object-info: support for
retrieving object info, 2021-04-20)"[3]. This patch series implement the client
option for it.

Eric's series adds the `remote-object-info` command to
`cat-file --batch-command`. This command allows the client to make an
object-info command request to a server that supports protocol v2.

If the server uses protocol v2 but does not support the object-info capability,
`cat-file --batch-command` will die.

If a user attempts to use `remote-object-info` with protocol v1,
`cat-file --batch-command` will die.

Currently, only the size (%(objectsize)) is supported end to end in this
implementation. The type (%(objecttype)) is known by the client's allow-list
and request path but is not supported on the server side nor the response
parsing. A follow up series will add full end-to-end support for %(objecttype).

The default format for remote-object-info is set to %(objectname) %(objectsize).
Once %(objecttype) is supported, the default format will be unified accordingly.

If the batch command format includes unsupported fields such as %(objecttype),
%(objectsize:disk), or %(deltabase), the command will return empty strings for
each unsupported field.

This series completes Eric's work mainly with the refactor of the validation
of the placeholder with an allow-list that filters what the client asks with
what the server is capable of provide following Jeff King's idea [4].

I have a question for the design:

1. If the format includes unsupported fields such as %(objecttype) or
   %(deltabase) it currently returns an empty string for each unsupported
   field, this follows what for-each-ref does with known but inapplicable
   atoms. However future placeholders that will be implemented: %(rest),
   %(objectmode) can return empty strings. How should we differentiate
   "unsupported" vs "no data".
   Eric proposed to use a placeholder like "???" [5].
   Should a placeholder be used?

2. _tangent/not related with this series_
   'a2ba162cda' is designed to only work with full OIDs, which is
   inconsistent with local `info` that does support short OIDs and in
   case of being ambiguous returns a list of what possibly the user meant.

   Because V2 protocol is thought to be stateless supporting short OIDs
   could become more inconsistent with other remote commands that do not
   support short OIDs. Maybe a --pick-first option? That does accept
   short oids and picks the first match.

   Alternatively, would sending a list of possible OIDs to the client so
   it can re-request with the correct one be ok?

[1]: https://lore.kernel.org/git/20250221190451.12536-1-eric.peijian@gmail.com/
[2]: https://lore.kernel.org/git/20220728230210.2952731-1-calvinwan@google.com/#t
[3]: https://git.kernel.org/pub/scm/git/git.git/commit/?id=a2ba162cda2acc171c3e36acbbc854792b093cb7
[4]: https://lore.kernel.org/git/20250313060250.GH94015@coredump.intra.peff.net/
[5]: https://lore.kernel.org/git/CAN2LT1D3d=yMYVhBjpj5PvyjfTVjwqcFPNViuCJ=f49YbCZuJg@mail.gmail.com/

Changes since v12:

- Remote-object-info no longer dies when the server doesn't recognize
  the object, printing "<oid> missing" like `info` does.
- On 12th commit explicitly cast to int and add a comment explaining why
  the backward iteration of the list.
- Renamed 3rd commit and in the commit, change the signature of
  dispatch_calls() as it is only called with size_t instead of ints.
- Because remote-object-info does not support short oids add a check to
  improve the error report if the oid passed is valid but not long
  enough or if it is an invalid oid.
- Fixed overly long lines.
- Reworded 4th commit.
- Avoid unnecessary request to the server when no placeholder is supported.

Signed-off-by: Pablo Sabater <pabloosabaterr@gmail.com>
---
Calvin Wan (3):
      fetch-pack: move fetch initialization
      serve: advertise object-info feature
      transport: add client support for object-info

Eric Ju (4):
      git-compat-util: add strtoul_ul() with error handling
      cat-file: declare loop counter inside for()
      t1006: split test utility functions into new "lib-cat-file.sh"
      cat-file: add remote-object-info to batch-command

Pablo Sabater (5):
      transport-helper: fix memory leak of helper on disconnect
      fetch-pack: move function to connect.c
      connect: refactor packet writing
      cat-file: validate remote atoms with allow_list
      cat-file: make remote-object-info allow-list dynamic

 Documentation/git-cat-file.adoc        |  25 +-
 Makefile                               |   1 +
 builtin/cat-file.c                     | 221 ++++++++++-
 connect.c                              |  34 ++
 connect.h                              |   8 +
 fetch-object-info.c                    | 106 +++++
 fetch-object-info.h                    |  22 ++
 fetch-pack.c                           |  51 +--
 fetch-pack.h                           |   2 +
 git-compat-util.h                      |  20 +
 meson.build                            |   1 +
 object-file.c                          |  10 +
 odb.h                                  |   3 +
 serve.c                                |   5 +-
 t/lib-cat-file.sh                      |  16 +
 t/meson.build                          |   1 +
 t/t1006-cat-file.sh                    |  13 +-
 t/t1017-cat-file-remote-object-info.sh | 699 +++++++++++++++++++++++++++++++++
 transport-helper.c                     |  13 +-
 transport.c                            |  28 +-
 transport.h                            |  11 +
 21 files changed, 1215 insertions(+), 75 deletions(-)
---
base-commit: 4621f8ce5e9b97aa2e8d0d9ffe9d25df2471074d
change-id: 20260608-ps-eric-work-rebase-b73ae84ba671

Best regards,
--  
Pablo Sabater <pabloosabaterr@gmail.com>