Re: [PATCH 6/7] odb: introduce object filters to `odb_for_each_object()`

[Patrick Steinhardt <ps@pks.im>]

On Thu, Jul 09, 2026 at 04:43:58PM -0500, Justin Tobler wrote:
> On 26/07/09 10:35AM, Patrick Steinhardt wrote:
> > The function `for_each_bitmapped_object()` can be used to iterate
> > through all objects covered by a bitmap. The benefit of this function is
> > that it allows the caller to efficiently handle some object filters. For
> > example, this can be used to filter out objects of a specific type with
> > some simple bitmap operations. But callers are currently required to
> > manually wire up the use of bitmaps though, and to do so they have to
> > reach into internals of a given object database source.
> > 
> > Introduce a new `struct odb_for_each_object_options::filter` field so
> > that the interface becomes generic. When set, then a backend may
> > optionally use the filter to skip some objects that it would have
> > otherwise yielded.
> > 
> > Note that the respective backends are free to ignore this field if they
> > cannot meaningfully optimize for a given filter, and consequently
> > callers need to verify whether they actually want the returned objects.
> > While annoying, we cannot easily lift this restriction anyway as the
> > object filter infrastructure supports some filters that cannot be
> > answered by the object database alone.
> 
> Huh, this feels rather awkward. So callers will always still have to
> ensure correctness by filtering the result a second time? IIUC, the idea
> is that the backend may be able to more efficiently process object
> filtering so we would want it to attempt the first pass.
> 
> Is there a subset of object filters that we should expect any backend to
> be able to answer? If so, maybe we should define a separate list of
> object filter options specific to this interface? Any filtering not
> supported would have to be deligated to the caller then.

It's a bit awkward, but it's also similar to how we handle this for
example in the reference backends with the exclude patterns. I don't
really think it makes sense to enforce that backends may only handle a
subset of object filters that we know the current backends support, as
that would artificially limit us.

For example, the "loose" backend already cannot efficiently handle many
of the filters that the "packed" backend can handle, like for example
filtering by type. So ultimately, the subset of filters that can be
handled efficiently by both backends is empty. And as the "files"
backend always combines both of these backends we wouldn't be able to
ever use the object filter at all there.

The same could be true for any future backend: we cannot assume how they
store their objects, so they might be able to efficiently handle filters
that the current backends cannot.

An alternative going forward could be to perform filtering of yielded
objects inside `odb_for_each_object()` itself so that it will filter out
any objects that the backends themselves couldn't filter efficiently.
But I'm not sure I want to go there as part of this series -- we only
have a single caller anyway that iterates with a filter, and that caller
already knows to manually filter references.

I'll add a bit of an explanation to the commit message.

Patrick