[PATCH v4 0/7] odb: add write operation to ODB transaction interface

[Justin Tobler <jltobler@gmail.com>]

Greetings,

This series lays the groundwork for introducing write operations to the
ODB transaction interface. The eventual goal is for all object writes
performed within a transaction to go through this interface explicitly,
rather than implicitly relying on the transaction to reconfigure ODB
sources so that writes are redirected to a temporary location.

For now, only `odb_transaction_write_object_stream()` is implemented and
wires up the existing logic for streaming "large" blobs directly into a
packfile as part of the transaction.

Most of the patches are structural refactorings to enable this, but
patch 4 introduces a behavioral change in how packfiles that would
exceed "pack.packSizeLimit" are handled.

Changes since V3:
- Fixed leak due to an fd not being closed when exiting prior to
  close_loose_object() being invoked.

Changes since V2:
- Renamed some variables to improve clarity
- Make `odb_write_stream_from_fd()` fully initialize the underlying
  `struct odb_write_stream`
- Move `struct odb_write_stream` to "odb/streaming.h"
- Make the `hash_blob_stream()` helper more generic by operating on a
  `struct odb_write_stream` instead of reading from an fd directly.
- Introduce an `odb_write_stream_release()` helper to free the
  underlying stream data.

Changes since V1:
- Fixed some typos
- Improved error handling
- Removed unnecessary guard statement
- Documented in comments why inflated object size is used to approximate
  if object write will exceed "pack.packSizeLimit".
- Updated `struct odb_write_stream` read() callback to support returning
  errors and using caller provided buffer
- Updated the `hash_blob_stream()` function signature to operate on a
  `struct odb_write_stream` instead of an fd directly
- Renamed some variables/functions for better clarity

Thanks,
-Justin

Justin Tobler (7):
  odb: split `struct odb_transaction` into separate header
  odb/transaction: use pluggable `begin_transaction()`
  odb: update `struct odb_write_stream` read() callback
  object-file: remove flags from transaction packfile writes
  object-file: avoid fd seekback by checking object size upfront
  object-file: generalize packfile writes to use odb_write_stream
  odb/transaction: make `write_object_stream()` pluggable

 Makefile                 |   1 +
 builtin/add.c            |   1 +
 builtin/unpack-objects.c |  21 ++--
 builtin/update-index.c   |   1 +
 cache-tree.c             |   1 +
 meson.build              |   1 +
 object-file.c            | 238 ++++++++++++++++++++-------------------
 odb.c                    |  25 ----
 odb.h                    |  37 +-----
 odb/streaming.c          |  51 +++++++++
 odb/streaming.h          |  30 +++++
 odb/transaction.c        |  35 ++++++
 odb/transaction.h        |  57 ++++++++++
 read-cache.c             |   1 +
 14 files changed, 312 insertions(+), 188 deletions(-)
 create mode 100644 odb/transaction.c
 create mode 100644 odb/transaction.h

Range-diff against v3:
1:  eee372b426 = 1:  eee372b426 odb: split `struct odb_transaction` into separate header
2:  57ac075560 = 2:  57ac075560 odb/transaction: use pluggable `begin_transaction()`
3:  11321ad607 ! 3:  d53ad95712 odb: update `struct odb_write_stream` read() callback
    @@ object-file.c: int odb_source_loose_write_stream(struct odb_source *source,
     +			ssize_t read_len = odb_write_stream_read(in_stream, buf,
     +								 sizeof(buf));
     +			if (read_len < 0) {
    ++				close(fd);
     +				err = -1;
     +				goto cleanup;
     +			}
4:  72d4656eee = 4:  fa7a3ad5dc object-file: remove flags from transaction packfile writes
5:  e4896101ff = 5:  1ca08e0590 object-file: avoid fd seekback by checking object size upfront
6:  b3cb0a707c = 6:  a548401057 object-file: generalize packfile writes to use odb_write_stream
7:  e1d292a7ed = 7:  4765b1024a odb/transaction: make `write_object_stream()` pluggable

base-commit: 5361983c075154725be47b65cca9a2421789e410
-- 
2.54.0.105.g59ff4886a5