strange caching behavior

strange caching behavior

[Mark Steele]

I've got a quick question for you NFS gurus. I'll try to keep
this short but it's a bit complicated to explain.

I've got an NFS server which exports to about 40 other
clients. The strangeness has occurred on both 2.4.29 and
2.6.10.

Clients are all running 2.6.11.8 (nfs utils 1.0.6)
and are serving http requests using Apache 1.3.=20

The majority of requests are for php scripts, and the=20
client machines use an opcode cache which compiles the=20
scripts into shared memory and caches them using the=20
filesystem timestamp as a reference to when to update=20
the cached copy.

During normal operations, I'm using about 6 mbits/sec
of bandwidth for my nfs server (pushing out 60 mbits/sec
to the internet). However, if I update the files on the NFS
server, it looks like all client caching goes out the
window and suddenly start pulling 60-80 mbits/sec from
the nfs server (still pushing out the same quantity of
traffic to the internet).

If I shutdown my web servers, and remount them all, traffic
drops back down to 6 mbits/sec (until the next update on
my php scripts).

Anyone have an idea on what could be causing this? (or if
this is expected behavior)

Regards,

Mark Steele
Sr. Systems Engineer
Zango Canada Inc.
mark@zango.com


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: strange caching behavior

[Peter Staubach]

Mark Steele wrote:

>I've got a quick question for you NFS gurus. I'll try to keep
>this short but it's a bit complicated to explain.
>
>I've got an NFS server which exports to about 40 other
>clients. The strangeness has occurred on both 2.4.29 and
>2.6.10.
>
>Clients are all running 2.6.11.8 (nfs utils 1.0.6)
>and are serving http requests using Apache 1.3. 
>
>The majority of requests are for php scripts, and the 
>client machines use an opcode cache which compiles the 
>scripts into shared memory and caches them using the 
>filesystem timestamp as a reference to when to update 
>the cached copy.
>
>During normal operations, I'm using about 6 mbits/sec
>of bandwidth for my nfs server (pushing out 60 mbits/sec
>to the internet). However, if I update the files on the NFS
>server, it looks like all client caching goes out the
>window and suddenly start pulling 60-80 mbits/sec from
>the nfs server (still pushing out the same quantity of
>traffic to the internet).
>
>If I shutdown my web servers, and remount them all, traffic
>drops back down to 6 mbits/sec (until the next update on
>my php scripts).
>
>Anyone have an idea on what could be causing this? (or if
>this is expected behavior)
>

It seems to me that it will depend upon how this opcode cache on the client
really works and maybe whether or not you delete the old versions of the php
scripts when you install the new ones.

When the php scripts change on the server, do the clients flush this opcode
cache and then refill the opcode cache from the current versions of the
scripts again?

    Thanx...

       ps


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: strange caching behavior

[Mark Steele]

Hi Peter,

The opcode cache is working as expected. It detects the changes
in the php scripts and recompiles the new files into memory.

Therefore it sees the change in the modification time of the files.

I am using rsync to update the files on the server from a code=20
repository.

I don't see how either of things things affect the behavior of the
NFS.

Using more caching/less bandwidth when 'freshly' mounted=20
without any file updates=20

VS.=20

Using lots of bandwidth when files are changed on the NFS server
(which is done infrequently)

Mark

-----Original Message-----
From: Peter Staubach [mailto:staubach@redhat.com]=20
Sent: July 8, 2005 2:18 PM
To: Mark Steele
Cc: nfs@lists.sourceforge.net
Subject: Re: [NFS] strange caching behavior

Mark Steele wrote:

>I've got a quick question for you NFS gurus. I'll try to keep
>this short but it's a bit complicated to explain.
>
>I've got an NFS server which exports to about 40 other
>clients. The strangeness has occurred on both 2.4.29 and
>2.6.10.
>
>Clients are all running 2.6.11.8 (nfs utils 1.0.6)
>and are serving http requests using Apache 1.3.=20
>
>The majority of requests are for php scripts, and the=20
>client machines use an opcode cache which compiles the=20
>scripts into shared memory and caches them using the=20
>filesystem timestamp as a reference to when to update=20
>the cached copy.
>
>During normal operations, I'm using about 6 mbits/sec
>of bandwidth for my nfs server (pushing out 60 mbits/sec
>to the internet). However, if I update the files on the NFS
>server, it looks like all client caching goes out the
>window and suddenly start pulling 60-80 mbits/sec from
>the nfs server (still pushing out the same quantity of
>traffic to the internet).
>
>If I shutdown my web servers, and remount them all, traffic
>drops back down to 6 mbits/sec (until the next update on
>my php scripts).
>
>Anyone have an idea on what could be causing this? (or if
>this is expected behavior)
>

It seems to me that it will depend upon how this opcode cache on the
client
really works and maybe whether or not you delete the old versions of the
php
scripts when you install the new ones.

When the php scripts change on the server, do the clients flush this
opcode
cache and then refill the opcode cache from the current versions of the
scripts again?

    Thanx...

       ps


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: strange caching behavior

[Peter Staubach]

Mark Steele wrote:

>Hi Peter,
>
>The opcode cache is working as expected. It detects the changes
>in the php scripts and recompiles the new files into memory.
>
>Therefore it sees the change in the modification time of the files.
>
>I am using rsync to update the files on the server from a code 
>repository.
>
>I don't see how either of things things affect the behavior of the
>NFS.
>
>Using more caching/less bandwidth when 'freshly' mounted 
>without any file updates 
>
>VS. 
>
>Using lots of bandwidth when files are changed on the NFS server
>(which is done infrequently)
>

My thinking that was the NFS client typically does not do things on its
own, but does so based on application activity.  For a large enough
working set, the more applications request from the file system, the
more traffic that there will be over the wire.

What does the increased traffic look like?  Is it metadata style operations
such as GETATTR or LOOKUP or is it data movement like READ?  What is the
typical working set size, in bytes and numbers of files, of these scripts?

    Thanx...

       ps


-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP, 
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs