Re: A naive proposal for preventing loose object explosions

[Junio C Hamano <gitster@pobox.com>]

mfick@codeaurora.org writes:

> Object lookups should likely not get any slower than if
> repack were not run, and the extra new pack might actually help
> find some objects quicker.

In general, having an extra pack, only to keep objects that you know
are available in other packs, will make _all_ object accesses, not
just the ones that are contained in that extra pack, slower.

Instead of mmapping all the .idx files for all the available
packfiles, we could build a table that records, for each packed
object, from which packfile at what offset the data is available to
optimize the access, but obviously building that in-core table will
take time, so it may not be a good trade-off to do so at runtime (a
precomputed super-.idx that we can mmap at runtime might be a good
way forward if that turns out to be the case).

> Does this sound like it would work?

Sorry, but it is unclear what problem you are trying to solve.

Is it that you do not like that "repack -A" ejects unreferenced
objects and makes it loose, which you may have many?

The loosen_unused_packed_objects() function used by "repack -A"
calls the force_object_loose() function (actually, it is the sole
caller of the function).  If you tweak the latter to stream to a
single new "graveyard" packfile and mark it as "kept until expiry",
would it solve the issue the same way but with much smaller impact?

There already is an infrastructure available to open a single output
packfile and send multiple objects to it in bulk-checkin.c, and I am
wondering if you can take advantage of the framework.  The existing
interface to it assumes that the object data is coming from a file
descriptor (the interface was built to support bulk-checkin of many
objects in an empty repository), and it needs refactoring to allow
stream_to_pack() to take different kind of data sources in the form
of stateful callback function, though.