Re: [PATCH] NFS: Fix nfs_netfs_issue_read() xarray locking for writeback interrupt

[Jeff Layton <jlayton@kernel.org>]

On Tue, 2024-01-30 at 09:56 -0500, David Wysochanski wrote:
> On Mon, Jan 29, 2024 at 12:44 PM Jeff Layton <jlayton@kernel.org> wrote:
> > 
> > On Mon, 2024-01-29 at 12:34 -0500, David Wysochanski wrote:
> > > On Mon, Jan 29, 2024 at 12:15 PM David Howells <dhowells@redhat.com> wrote:
> > > > 
> > > > Dave Wysochanski <dwysocha@redhat.com> wrote:
> > > > 
> > > > > -     xas_lock(&xas);
> > > > > +     xas_lock_irqsave(&xas, flags);
> > > > >       xas_for_each(&xas, page, last) {
> > > > 
> > > > You probably want to use RCU, not xas_lock().  The pages are locked and so
> > > > cannot be evicted from the xarray.
> > > > 
> > > 
> > > I tried RCU originally and ran into a problem because NFS can schedule
> > > (see comment on line 328 below)
> > > 
> > > 326         xas_lock_irqsave(&xas, flags);
> > > 327         xas_for_each(&xas, page, last) {
> > > 328                 /* nfs_read_add_folio() may schedule() due to pNFS
> > > layout and other RPCs  */
> > > 329                 xas_pause(&xas);
> > > 330                 xas_unlock_irqrestore(&xas, flags);
> > > 331                 err = nfs_read_add_folio(&pgio, ctx, page_folio(page));
> > > 332                 if (err < 0) {
> > > 333                         netfs->error = err;
> > > 334                         goto out;
> > > 335                 }
> > > 336                 xas_lock_irqsave(&xas, flags);
> > > 337         }
> > > 338         xas_unlock_irqrestore(&xas, flags);
> > > 
> > 
> > Looking at it more closely, I think you might want to just use
> > xa_for_each_start(). That will do the traversal using the rcu_read_lock
> > under the hood, and you should be able to block on every iteration.
> > 
> Thanks Jeff.  Yes, I agree after looking at this further, this is a
> good approach, and much cleaner.  I'll work on a v2 patch (actually
> with xa_for_each_range as you suggested off list) and send after
> a bit of testing -- so far, so good.
> 
> FWIW, my original usage of RCU was outside the whole loop.
> I ran into problems due to nfs_read_add_folio().
> 

Makes sense. In principle you could do this by just dropping and
acquiring the rcu_read_lock in the same places you do the spinlock in
the original patch, but using xa_for_each_range is much simpler.

-- 
Jeff Layton <jlayton@kernel.org>