Re: [PATCH 5/6] [XFS] Replace per-ag array with a radix tree

Dave Chinner
2009-12-11T00:43:23Z

On Thu, Dec 10, 2009 at 06:45:47PM -0500, Christoph Hellwig wrote:
> On Wed, Dec 02, 2009 at 05:11:38PM +1100, Dave Chinner wrote:
> > - down_read(&mp->m_peraglock);
> > + pag = xfs_perag_get(mp, ag);
> > while (blen < ap->alen) {
> > - pag = xfs_perag_get(mp, ag);
> > if (!pag->pagf_init &&
> > (error = xfs_alloc_pagf_init(mp, args.tp,
> > ag, XFS_ALLOC_FLAG_TRYLOCK))) {
> > xfs_perag_put(pag);
> > - up_read(&mp->m_peraglock);
> > return error;
> > }
> > /*
> > @@ -2801,7 +2799,6 @@ xfs_bmap_btalloc(
> > } else
> > notinit = 1;
> >
> > - xfs_perag_put(pag);
>
> There's a lot of those xfs_perag_get/put moved around here. Having
> those merged into the patch that adds them would be a lot cleaner.

OK, I'll clean all those up into the initial couple of patches.

> > + /* allocate the new per-ag structures */
> > if (nagcount > oagcount) {
> > + /* XXX: (dgc) We don't need the filestream flush anymore? */
> > xfs_filestream_flush(mp);
>
> What was the reason to have it in the first time?

Filestreams keeps reference counts and state on the per-ag
structure. If it was to be re-allocated, then all the references had
to be dropped before reallocation, hence the flush. Now there is
no reallocation, I don't think we need the flush anymore. Removing
it also means updating comments in the filestream code, so I was
going to do all that in a subsequent patch....

> > index 3727104..d6de63d 100644
> > --- a/fs/xfs/xfs_mount.c
> > +++ b/fs/xfs/xfs_mount.c
> > @@ -207,13 +207,17 @@ STATIC void
> > xfs_free_perag(
> > xfs_mount_t *mp)
> > {
> > + xfs_agnumber_t agno;
> > + struct xfs_perag *pag;
> > +
> > + for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) {
> > + spin_lock(&mp->m_perag_lock);
> > + pag = radix_tree_delete(&mp->m_perag_tree, agno);
> > + spin_unlock(&mp->m_perag_lock);
> > + if (!pag)
> > + continue;
>
> Shouldn't this be a BUG_ON/ASSERT?

Probably should be. Will change.

> > + /*
> > + * Walk the current per-ag tree so we don't try to initialise AGs
> > + * that already exist (growfs case). Allocate and insert all the
> > + * AGs we don't find ready for initialisation.
> > + */
> > + for (index = 0; index < agcount; index++) {
> > + pag = xfs_perag_get(mp, index);
> > + if (pag) {
> > + xfs_perag_put(pag);
> > + continue;
> > + }
> > + pag = kmem_zalloc(sizeof(*pag), KM_MAYFAIL);
> > + if (!pag)
> > + return -ENOMEM;
> > + if (radix_tree_preload(GFP_NOFS))
> > + return -ENOMEM;
>
> Leaks the pag object on failure.

Good catch.

>
> > mp->m_maxagi = xfs_initialize_perag(mp, sbp->sb_agcount);
> > + if ((int)mp->m_maxagi < 0) {
> > + cmn_err(CE_WARN, "XFS: Failed per-ag initialisation: %d",
> > + (int)mp->m_maxagi);
> > + error = mp->m_maxagi;
> >
> Just assign it to error first and then later to mp->m_maxagi to avoid
> the cast?

Actually, to avoid all such sign issues, I think that mp->m_maxagi
sho┘ld be assigned in xfs_initialize_perag() and it only returns
error or success. Does that make sense?

> > static inline xfs_perag_t *
> > xfs_perag_get(struct xfs_mount *mp, xfs_agnumber_t agno)
> > {
> > - return &mp->m_perag[agno];
> > + struct xfs_perag *pag;
> > +
> > + spin_lock(&mp->m_perag_lock);
> > + pag = radix_tree_lookup(&mp->m_perag_tree, agno);
> > + spin_unlock(&mp->m_perag_lock);
> > + return pag;
>
> Can't we do this as a lock-less (at least for lookups) radix tree?

I think it can be (RCU-based?) , but I think that makes sense as a
followup optimisation once we have confidence the code is working
as it should.

> And btw, I think we should still have a global sleeping lock to
> serialize the whole growfs operation against other potentional growfs
> callers.

Agreed - mp->m_growlock already does this and this patch series did
not touch it at all so it should still work ;)

Cheers,

Dave.
--
Dave Chinner
david@fromorbit.com