Re: [PATCH v3 00/42] xfs: per-ag centric allocation alogrithms

Dave Chinner
2023-02-10T05:00:38Z

On Thu, Feb 09, 2023 at 07:09:56PM -0800, Darrick J. Wong wrote:
> On Fri, Feb 10, 2023 at 09:17:43AM +1100, Dave Chinner wrote:
> > This series continues the work towards making shrinking a filesystem possible.
> > We need to be able to stop operations from taking place on AGs that need to be
> > removed by a shrink, so before shrink can be implemented we need to have the
> > infrastructure in place to prevent incursion into AGs that are going to be, or
> > are in the process, of being removed from active duty.
>
> From a quick glance, it looks like all the random things I had comments
> about were fixed, so for patches 14, 20-23, 28, and 42:
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>
> I'm leaving off patch #7 until tomorrow so that I can think about it
> with a non-tired brain. I didn't see anything obviously wrong in the
> diff itself -- but I still need to adjust my mental model per what Dave
> said in his previously reply (active perag refs are for user-facing
> online operations, passive refs are for internal operations) and
> (re)examine how that relates to scrub and repair.
>
> Mostly I tripped over "but repair needs to use passive references once
> the AG has had it's state changed to "offline" -- currently, repair uses
> the same perag reference that scrub _gets. If scrub now gets an
> "active" reference and something needs repair, do we mark the AG offline
> and keep the active reference? Or downgrade it to a passive reference?

So one of the things I was trying to explain and didn't do a very
good job of is that active references are references that prevent AG
operational state changes.

That is, if we want to take an AG offline, or just prevent new
allocations in an AG, we have to wait for all the active references
to drain before we can change the operational state.

This does not prevent an active reference from being taken when an
AG is offline - all an active reference in an offline state does is
prevent the AG from being put back online whilst that active
reference to the offline AG persists.

e.g. repair can drain all the active references on an online AG,
then mark it offline, then take a new active reference to pin the AG
in the offline state while it does the repair work on that AG.

Similarly, shrink can pin an AG in a "being shrunk" operational
state that allows inodes and extents to be freed, but no new
allocations to be made by draining the active references, changing
the state and then pinning by taking a new reference. Then the
shrink process can move all the user data and metadata out of the AG
without needing special tricks to avoid allocating in that AG. If it
fails to move everything or is aborted, it can drop it's active
reference and put the AG back online...

This means that the new allocation code that now takes active
references will be morphing further to be "grab active reference,
check AG opstate allows allocation, if not drop active reference and
skip AG".

Once an AG is ready to be removed, grabbing an active reference
will fail so at that point the AG is skipped without even getting to
state checks. Once all the passive references then drain, the perag
can be RCU freed.

> I've (tried to) design scrub & repair as if they were just another pile
> of higher level code that uses libxfs to manipulate metadata, just like
> fallocate and reflink and all those types of things. But then, I was
> designing for a world where there's only one type of AG reference. :)

For the moment, everything in scrub/repair will work just fine with
passive references. It's only once we start using AG opstate to
control allocator, caching and scanning behaviour that they might
want to start using active references...

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com