[PATCH 0/1] pack-write, pack-bitmap-write: register tmp pack files for cleanup

[Royce Remer <royceremer@gmail.com>]

This patch registers the temporary pack files created by git-gc(1) and
git-maintenance(1) with the tempfile subsystem so they are removed when
the process exits gracefully.

Background
----------

The motivation came from diagnosing disk space in Kubernetes pods
running Gitea as git mirrors.  If a pod was killed mid-gc, the
pack-writing code left orphaned temp files in objects/pack/.
On restart, git gc would start fresh and write new temp files
alongside the existing ones.  Over many restarts these,
accumulated until the underlying volume was exhausted:

  Two repositories examined on a single pod:
    objects/pack/tmp_pack_* -- 27 GiB, 21 GiB, 11 GiB, ... (17 files, ~60 GiB total)
    objects/pack/.tmp-*-pack-*.{pack,rev} -- ~118 GiB across 6 killed repacks

  A second pod had a different repository where two killed repacks left:
    objects/pack/.tmp-*-pack-*.{pack,rev} -- ~39 GiB across 2 killed repacks

  Deleting those files and running git-prune-packed(1) to remove loose
  objects already represented in pack files recovered ~224 GiB on that
  second pod alone.

Obviously, this is dependent on repository sizes and number of
failures and such, but I thought I'd share my extreme example.

Reviewing the gc and maintenance code, I don't see any attempts to
resume or reuse temp files left by a previous invocation; each run
calls odb_mkstemp() unconditionally to create a fresh file.  Any
surviving temp file should be safe to remove.

The tmp_idx, tmp_pack and tmp_bitmap sites predate the tempfile
subsystem (1a9d15db25, 2015-08-10) and so had no mechanism to
register when introduced.  The tmp_rev and tmp_mtimes sites
were added afterward but did not use it either.

Note that git-repack(1) already handles this correctly: it calls
register_tempfile() for the .tmp-<pid>-pack-<sha>.* files it creates
via collect_pack_filenames(), so those are cleaned up on graceful exit.
The lower-level paths invoked by git-gc(1) and git-maintenance(1)
(pack-write.c and pack-bitmap-write.c) go through odb_mkstemp() which
wraps mkstemp(2) directly without registering with the tempfile
subsystem, and so do not benefit from this cleanup.

I have some unit tests covering this, but they required instrumenting
the code to add a wait driven by an environment variable so I could
catch/kill a repack on a tiny mock repo. I decided not to commit
those as I think the fix is self-evident and we're just delegating
to the same tempfile cleanup logic and relying on that coverage.

Royce Remer (1):
  pack-write, pack-bitmap-write: register tmp pack files for cleanup

 pack-bitmap-write.c | 3 +++
 pack-write.c        | 5 +++++
 2 files changed, 8 insertions(+)

-- 
2.55.0.1.ga30d533ec0