Re: still nfs problems [Was: Linux 2.6.37-rc8]

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Thu, 2011-01-06 at 12:47 -0500, Trond Myklebust wrote:
> On Thu, 2011-01-06 at 11:40 -0600, James Bottomley wrote: 
> > On Wed, 2011-01-05 at 23:28 +0000, James Bottomley wrote:
> > > Can you explain how the code works? it looks to me like you read the xdr
> > > stuff through the vmap region then write it out directly to the pages? 
> > 
> > OK, I think I see how this is supposed to work:  It's a sequential loop
> > of reading in via the pages (i.e. through the kernel mapping) and then
> > updating those pages via the vmap.  In which case, I think this patch is
> > what you need.
> > 
> > The theory of operation is that the readdir on pages actually uses the
> > network DMA operations to perform, so when it's finished, the underlying
> > page is up to date.  After this you invalidate the vmap range, so we
> > have no cache lines above it (so it picks up the values from the
> > uptodate page).  Finally, after the operation on the vmap region has
> > finished, you flush it so that any updated contents go back to the pages
> > themselves before the next iteration begins.
> > 
> > Does this look right to people?  I've verified it fixes the issues on
> > parisc.
> > 
> > James
> > 
> > ---
> > 
> > diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
> > index 996dd89..bde1911 100644
> > --- a/fs/nfs/dir.c
> > +++ b/fs/nfs/dir.c
> > @@ -587,12 +587,16 @@ int nfs_readdir_xdr_to_array(nfs_readdir_descriptor_t *desc, struct page *page,
> >  		if (status < 0)
> >  			break;
> >  		pglen = status;
> > +
> > +		invalidate_kernel_vmap_range(pages_ptr, pglen);
> > +
> >  		status = nfs_readdir_page_filler(desc, &entry, pages_ptr, page, pglen);
> >  		if (status < 0) {
> >  			if (status == -ENOSPC)
> >  				status = 0;
> >  			break;
> >  		}
> > +		flush_kernel_vmap_range(pages_ptr, pglen);
> 
> Why is this line needed? We're not writing through the virtual mapping.

If you're not altering it, it isn't ... the problem on parisc is that
invalidate is a nop for us because flush does it all, but I can fix
that.

James

> We checked using just the invalidate_kernel_vmap_range(), and that
> appeared to suffice to fix the problem on ARM.
> 
> Cheers
>   Trond