Re: Read/Write NFS I/O performance degraded by FLUSH_STABLE page flushing

Brian R Cowan
2009-05-29T22:20:41Z

I am listening.

Commit is sync. I get that.

The NFS client does Async writes in RHEL 4. They *eventually* get
committed. (Doesn't really matter who causes the commit, does it.)
Read system calls may trigger cache flushing, but since not all of them
are sync writes, the reads don't *always* stall when cache flushes occur.
Builds are fast.

We do sync writes in RHEL 5, so they MUST stop and wait for the NFS server
to come back.
READ system calls stall whan the read triggers a flush of one or more
cache pages.
Builds are slow. Links are at least 4x slower.

I am perfectly willing to send you network traces showing the issue. I can
even DEMONSTRATE it for you using the remote meeting software of your
choice. I can even demonstrate the impact of removing that behavior.

=================================================================
Brian Cowan
Advisory Software Engineer
ClearCase Customer Advocacy Group (CAG)
Rational Software
IBM Software Group
81 Hartwell Ave
Lexington, MA

Phone: 1.781.372.3580
Web: http://www.ibm.com/software/rational/support/

Please be sure to update your PMR using ESR at
http://www-306.ibm.com/software/support/probsub.html or cc all
correspondence to sw_support@us.ibm.com to be sure your PMR is updated in
case I am not available.

From:
Trond Myklebust <trond.myklebust@fys.uio.no>
To:
Brian R Cowan/Cupertino/IBM@IBMUS
Cc:
Chuck Lever <chuck.lever@oracle.com>, linux-nfs@vger.kernel.org,
linux-nfs-owner@vger.kernel.org, Peter Staubach <staubach@redhat.com>
Date:
05/29/2009 06:06 PM
Subject:
Re: Read/Write NFS I/O performance degraded by FLUSH_STABLE page flushing

On Fri, 2009-05-29 at 17:55 -0400, Brian R Cowan wrote:
> So, it is possible that either pdflush is sending the commits or us, or
> that the commits are happening when the file closes, giving us one/tens
of
> commits instead of hundreds or thousands. That's a big difference. The
> write RPCs still happen in RHEL 4, they just don't block the linker, or
at
> least nowhere near as often. Since there is only one application/thread
> (the gcc linker) writing this file, the odds of another task getting
> stalled here are minimal at best.

No, you're not listening! That COMMIT is _synchronous_ and happens
before you can proceed with the READ request. There is no economy of
scale as you seem to assume.

Trond