Re: [PATCH v4 0/6] pack-objects: freshen objects with multi-cruft packs

[Junio C Hamano <gitster@pobox.com>]

Taylor Blau <me@ttaylorr.com> writes:

> Maybe we should call the option something else that makes the cruft pack
> use-case clearer. But in the other thread I came around to the idea that
> this case is too niche to address completely, so I think we can discard
> this round as a terrible idea.

Yeah, I do not think "an earlier attempt wouldn't fill the bucket
fully, so a later logic may try to fill the bucket with the same
material in vain, failing the same way" is a problem that is limited
to cruft pack use case.  If you are repacking real data into 200MB
chunks every week, the 198MB pack created last week that is lying in
the object store that houses the oldest part of the history is
unlikely to be improved by repacking it (since the attempt failed to
fill the remaining 2MB last week, and the oldest part of the history
hasn't changed) but it is entirely possible that with the pieces of
information left by the current set of tools, the machinery may not
realize that fact.

Allowing the resulting pack to deliberately exceed 200MB ceiling is
one possible way (I still do not think it is the best way among
possible ways) to mark the pack "frozen-do not further touch".

Now, if the essense of the problem is when to mark a pack that we
stopped appending to due to max-size limit as "frozen and do not
touch" to avoid this problem, how such a marking is done, and who
uses it to avoid repeated work that is known (to us) to be futile,
there should be better ways to do such a marking.  We have .keep
mechanism which is ugly but proven to work and we know how to deal
with, for example.

I wonder if our pack header or trailer is extensible enough so that
we can record "this pack was closed due to max-size limit of 200MB
and appending even one more object in the queue would have busted
the limit".

No matter how such a marking is done, later repack that sees a pack
marked as such, when it is limited to 200MB, should be able to tell
that and act a bit more intelligently than what it currently does.