Re: [PATCH 1/2] http: use unique tempfiles for packfile URI downloads

[Ted Nyman <tnyman@openai.com>]

> While that does sound like a safe and correct approach, stepping
> back briefly, would it not be wasteful for the second process to
> download the same packfile that the first has already started
> downloading?

Yes. If two fetches overlap, the second download is redundant.

> Are there better ways for these processes to coordinate with each
> other? Instead of appending to the file, what if the second process
> uses a predictable temporary name (which we already use) to open a
> new file with O_CREAT | O_EXCL to avoid this redundant work?

Using the existing pack-<hash>.pack.temp name with O_CREAT | O_EXCL
would prevent concurrent writes, but EEXIST alone would not
distinguish an in-progress download from one left by an earlier
failed or interrupted invocation. The existing .pack.temp name is not
covered by the tmp_* pruning path, so simply waiting for it to
disappear could leave a fetch stuck after a crash.

The waiting case would also need a complete handoff. If the first
process finishes, the second would need to notice the installed pack
and account for the expected index-pack result and keep state. If the
first process fails and removes its temporary file, the second would
need to retry as the downloader. That is possible, but introduces
cross-process coordination and a timeout policy in http-fetch.

The unique tempfile preserves the existing "download, index, then
install" behavior for each invocation and fixes both the
concurrent-append and EOF-resume failures. Avoiding the duplicate
transfer would be useful for large packs, but I would prefer to keep
that as a follow-up unless you think it is necessary for this
correctness fix.

Thanks,
Ted