Re: still nfs problems [Was: Linux 2.6.37-rc8]

[Trond Myklebust <Trond.Myklebust@netapp.com>]

On Thu, 2011-01-06 at 11:40 -0600, James Bottomley wrote: 
> On Wed, 2011-01-05 at 23:28 +0000, James Bottomley wrote:
> > Can you explain how the code works? it looks to me like you read the xdr
> > stuff through the vmap region then write it out directly to the pages? 
> 
> OK, I think I see how this is supposed to work:  It's a sequential loop
> of reading in via the pages (i.e. through the kernel mapping) and then
> updating those pages via the vmap.  In which case, I think this patch is
> what you need.
> 
> The theory of operation is that the readdir on pages actually uses the
> network DMA operations to perform, so when it's finished, the underlying
> page is up to date.  After this you invalidate the vmap range, so we
> have no cache lines above it (so it picks up the values from the
> uptodate page).  Finally, after the operation on the vmap region has
> finished, you flush it so that any updated contents go back to the pages
> themselves before the next iteration begins.
> 
> Does this look right to people?  I've verified it fixes the issues on
> parisc.
> 
> James
> 
> ---
> 
> diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
> index 996dd89..bde1911 100644
> --- a/fs/nfs/dir.c
> +++ b/fs/nfs/dir.c
> @@ -587,12 +587,16 @@ int nfs_readdir_xdr_to_array(nfs_readdir_descriptor_t *desc, struct page *page,
>  		if (status < 0)
>  			break;
>  		pglen = status;
> +
> +		invalidate_kernel_vmap_range(pages_ptr, pglen);
> +
>  		status = nfs_readdir_page_filler(desc, &entry, pages_ptr, page, pglen);
>  		if (status < 0) {
>  			if (status == -ENOSPC)
>  				status = 0;
>  			break;
>  		}
> +		flush_kernel_vmap_range(pages_ptr, pglen);

Why is this line needed? We're not writing through the virtual mapping.

We checked using just the invalidate_kernel_vmap_range(), and that
appeared to suffice to fix the problem on ARM.

Cheers
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com