RedHat 7.3 crashed NetApp?

RedHat 7.3 crashed NetApp?

[H . J . Lu]

Has anyone seen

http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=65069


H.J.

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[seth vidal]

On Thu, 2002-05-16 at 18:39, H . J . Lu wrote:
> Has anyone seen
> 
> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=65069
> 

Is there anyone who can assuage the concerns of sct in this bug:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=59245

-sv



_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Error: RPC request reserved 244 but used 248

[Abhas Abhinav]

Hi...

I have been running nhfsstone on a tcp-mount NFS partition. Both the
client and the server are running 2.4.8. The server uses ReiserFS for
its disk partition. I've applied the respective nfs-tcp patches to the
client and server... and was able to mount the nfs export using mount
-otcp on the client...

However, during the test, the server kept giving the error:

RPC request reserved 244 but used 248

Any idea what this error could mean and how to stop it?

thanks,
Abhas.

-- 
-------------------------------------------------------------------
Abhas Abhinav		   | Free Software at its product-ive best.
CEO, DeepRoot Linux
http://www.deeproot.co.in           ---- Server Appliances ----
Ph: +91 (80) 856 5624          ---- Linux Support and Services ----
-------------------------------------------------------------------


_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Error: RPC request reserved 244 but used 248

[Neil Brown]

On Friday May 17, abhas@deeproot.co.in wrote:
> Hi...
> 
> I have been running nhfsstone on a tcp-mount NFS partition. Both the
> client and the server are running 2.4.8. The server uses ReiserFS for
> its disk partition. I've applied the respective nfs-tcp patches to the
> client and server... and was able to mount the nfs export using mount
> -otcp on the client...
> 
> However, during the test, the server kept giving the error:
> 
> RPC request reserved 244 but used 248
> 
> Any idea what this error could mean and how to stop it?

It means that I can't count, and that the patch you have is a bit
out-of-date.
The latext -ac patch should work fine.

I probably wont be putting out another complete NFS patch until 2.4.19
comes out.

NeilBrown

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Error: RPC request reserved 244 but used 248

[Abhas Abhinav]

On Fri, 17 May 2002, Neil Brown wrote:

>--|> client and the server are running 2.4.8. The server uses ReiserFS for
                                     ^^^^^^^^^^
							This was "small" typo - it should've been
							2.4.18!

>--|It means that I can't count, and that the patch you have is a bit
>--|out-of-date.
>--|The latext -ac patch should work fine.
>--|
>--|I probably wont be putting out another complete NFS patch until 2.4.19
>--|comes out.

I patched the server kernel using the patch-Bd-NfsdAll for 2.4.18 that I
downloaded from cse.unsw.edu.au. So I guess I am using your latest
patches. The client was patched using 2.4.18 + Trond's NFS client
patch linux-2.4.18-NFS_ALL.dif. Other than that both the server and the
client were patched for reiserfs quota support (Chris Mason's patches).

All these patches were downloaded during the last 14 hours...

I am basically looking at test NFS-TCP support for a file server design
for a mail server. We're using reiserfs throughout.

Would the -ac tree be stable enough to experiment with this? And are
there any major advantages in using -ac tree or can I get away with
using 2.4.18-stable.

thanks for your help,
abhas.

-- 
-------------------------------------------------------------------
Abhas Abhinav		   | Free Software at its product-ive best.
CEO, DeepRoot Linux
http://www.deeproot.co.in           ---- Server Appliances ----
Ph: +91 (80) 856 5624          ---- Linux Support and Services ----
-------------------------------------------------------------------


_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Error: RPC request reserved 244 but used 248

[Neil Brown]

On Friday May 17, abhas@deeproot.co.in wrote:
> On Fri, 17 May 2002, Neil Brown wrote:
> 
> >--|> client and the server are running 2.4.8. The server uses ReiserFS for
>                                      ^^^^^^^^^^
> 							This was "small" typo - it should've been
> 							2.4.18!
> 
> >--|It means that I can't count, and that the patch you have is a bit
> >--|out-of-date.
> >--|The latext -ac patch should work fine.
> >--|
> >--|I probably wont be putting out another complete NFS patch until 2.4.19
> >--|comes out.
> 
> I patched the server kernel using the patch-Bd-NfsdAll for 2.4.18 that I
> downloaded from cse.unsw.edu.au. So I guess I am using your latest
> patches. The client was patched using 2.4.18 + Trond's NFS client
> patch linux-2.4.18-NFS_ALL.dif. Other than that both the server and the
> client were patched for reiserfs quota support (Chris Mason's patches).
> 
> All these patches were downloaded during the last 14 hours...

I think you don't understand how my patch repository works, which is
understandable, because it isn't really documented anywhere.

How it works is that every patch I create goes there somewhere.  From
time to time I might recommend particular patches on some mailing
list, or send particular patches to Linus/Alan/Marcelo for inclusion
into trees.
There is no attempt to keep a current 'best-of-breed' patch or to
guarantee correctness of any patches.

By using the patch you did you were doing the 'correct' thing in that
it is that latest one that I have publicly recommended.  However it is
not the latest patch... which is to say that I have not publicly
recommend any fixes that I have made, except by sending them to Alan.

But maybe I should....

On-top of that patch that you have, you could add:

http://www.cse.unsw.edu.au/~neilb/patches/linux-stable/2.4.19-pre3-ac1/patch-h-NfsdReserveFix
http://www.cse.unsw.edu.au/~neilb/patches/linux-stable/2.4.19-pre4-ac4/patch-A-NfsdSymolicSizes
http://www.cse.unsw.edu.au/~neilb/patches/linux-stable/2.4.19-pre7-ac1/patch-A-LockdTcpFix

plus this stanza:

--- ./fs/nfsd/export.c  2002/03/18 01:40:45     1.9
+++ ./fs/nfsd/export.c  2002/03/18 01:41:40     1.10
@@ -533,13 +506,15 @@
 
        for (hp = head; (tmp = *hp) != NULL; hp = &(tmp->h_next)) {
                if (tmp->h_addr.s_addr == addr) {
+#if 0
+/* this is not safe without BKL or some other spinlock */
                        /* Move client to the front */
                        if (head != hp) {
                                *hp = tmp->h_next;
                                tmp->h_next = *head;
                                *head = tmp;
                        }
-
+#endif
                        return tmp->h_client;
                }
        }

out of 
  http://www.cse.unsw.edu.au/~neilb/patches/linux-stable/2.4.19-pre3-ac1/patch-Q-NfsdLocksExplock

together these should bring you up-to-date...

Actually, there is another on.  I don't actually have a -stable patch
for it yet, but
http://www.cse.unsw.edu.au/~neilb/patches/linux-devel/2.5.15/patch-E-RpcDGRAMFix

should probably apply on-top of the others OK.

> 
> I am basically looking at test NFS-TCP support for a file server design
> for a mail server. We're using reiserfs throughout.

Well... Server support for NFS/TCP is very new, as you can probably
tell.  If you are happy to knit patches together, then go for it.

> 
> Would the -ac tree be stable enough to experiment with this? And are
> there any major advantages in using -ac tree or can I get away with
> using 2.4.18-stable.

I cannot comment on how stable -ac is, or what advantages it has,
other than the fact that all of the above are in it.

NeilBrown


> 
> thanks for your help,
> abhas.
> 
> -- 
> -------------------------------------------------------------------
> Abhas Abhinav		   | Free Software at its product-ive best.
> CEO, DeepRoot Linux
> http://www.deeproot.co.in           ---- Server Appliances ----
> Ph: +91 (80) 856 5624          ---- Linux Support and Services ----
> -------------------------------------------------------------------

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: RedHat 7.3 crashed NetApp?

[Cooke, Alan]

While running the 2.4.18-4smp kernel (RedHat), and testing NFS3, our =
NetApp filer running OnTap 6.1.1R2 experienced massive incoming traffic =
from the Linux client in question. The Linux client required rebooting. =
We have been able to repeat this behavior, and are suspicious of the =
network driver on the Linux client which is using the eth0 driver. A

Sound familiar?

Alan Cooke
Hewlett-Packard Company
ISS/EDA
Office: 281-518-9775
Fax    : 281-518-8015
Email  : acooke@hp.com

"If everything seems under control, you're just not going fast enough."
-- Mario Andretti








-----Original Message-----
From: H . J . Lu [mailto:hjl@lucon.org]
Sent: Thursday, May 16, 2002 5:39 PM
To: nfs@lists.sourceforge.net
Subject: [NFS] RedHat 7.3 crashed NetApp?


Has anyone seen

http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=3D65069


H.J.

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We =
supply
the hardware. You get the recognition. Email Us: =
bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[H . J . Lu]

On Fri, May 17, 2002 at 08:14:37AM -0500, Cooke, Alan wrote:
> While running the 2.4.18-4smp kernel (RedHat), and testing NFS3, our NetApp filer running OnTap 6.1.1R2 experienced massive incoming traffic from the Linux client in question. The Linux client required rebooting. We have been able to repeat this behavior, and are suspicious of the network driver on the Linux client which is using the eth0 driver. A
> 

I think it is the same as what happened to me. I had to reboot/unplug
my Linux client before the whole NFS network went down. I am using Intel
e100 driver on eepro100. But my Linux NFS severs have no problem with
that client.


H.J.

_______________________________________________________________

Have big pipes? SourceForge.net is looking for download mirrors. We supply
the hardware. You get the recognition. Email Us: bandwidth@sourceforge.net
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[Daryl Herzmann]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=US-ASCII, Size: 1473 bytes --]

Hi,

I reported the exact same thing with an IRIX server on Friday to this 
list.  I am glad that others are experiencing it too.  Using version 2 
made the troubles go away for me.

http://www.geocrawler.com/lists/3/SourceForge/789/75/8633160/

Time for a RH bugzilla entry or is there one?

Daryl

On Fri, 17 May 2002, H . J . Lu wrote:

>On Fri, May 17, 2002 at 08:14:37AM -0500, Cooke, Alan wrote:
>> While running the 2.4.18-4smp kernel (RedHat), and testing NFS3, our NetApp filer running OnTap 6.1.1R2 experienced massive incoming traffic from the Linux client in question. The Linux client required rebooting. We have been able to repeat this behavior, and are suspicious of the network driver on the Linux client which is using the eth0 driver. A
>> 
>
>I think it is the same as what happened to me. I had to reboot/unplug
>my Linux client before the whole NFS network went down. I am using Intel
>e100 driver on eepro100. But my Linux NFS severs have no problem with
>that client.
>
>
>H.J.
>
>

-- 
/**
 * Daryl Herzmann (akrherz@iastate.edu)
 * Program Assistant -- Iowa Environmental Mesonet
 * http://mesonet.agron.iastate.edu
 */




_______________________________________________________________

Hundreds of nodes, one monster rendering program.
Now that’s a super model! Visit http://clustering.foundries.sf.net/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[Benjamin LaHaise]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 564 bytes --]

On Fri, May 17, 2002 at 12:16:48PM -0500, Daryl Herzmann wrote:
> Time for a RH bugzilla entry or is there one?

There are several.  Some of the people reporting problems have verified 
that they are present in 2.4.19-pre8 as well.

		-ben

_______________________________________________________________

Hundreds of nodes, one monster rendering program.
Now that’s a super model! Visit http://clustering.foundries.sf.net/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: RedHat 7.3 crashed NetApp?

[Lever, Charles]

i'm wondering what are the common elements here:

is the EEpro 100 ethernet card common to all clients
that exhibit this problem?  NFSv3 over UDP?  what is
the nature of the network flood -- ICMP packets, or
UDP?  does "top" on the client show whether a particular
process or processes is spinning?

a tcpdump or at least "netstat -s" on the client after
the network is unplugged might be helpful.

> -----Original Message-----
> From: Cooke, Alan [mailto:acooke@hp.com]
> Sent: Friday, May 17, 2002 9:15 AM
> To: nfs@lists.sourceforge.net
> Subject: RE: [NFS] RedHat 7.3 crashed NetApp?
> 
> 
> While running the 2.4.18-4smp kernel (RedHat), and testing 
> NFS3, our NetApp filer running OnTap 6.1.1R2 experienced 
> massive incoming traffic from the Linux client in question. 
> The Linux client required rebooting. We have been able to 
> repeat this behavior, and are suspicious of the network 
> driver on the Linux client which is using the eth0 driver. A
> 
> Sound familiar?
> 
> Alan Cooke
> Hewlett-Packard Company
> ISS/EDA
> Office: 281-518-9775
> Fax    : 281-518-8015
> Email  : acooke@hp.com
> 
> "If everything seems under control, you're just not going 
> fast enough."
> -- Mario Andretti
> 
> 
> 
> 
> 
> 
> 
> 
> -----Original Message-----
> From: H . J . Lu [mailto:hjl@lucon.org]
> Sent: Thursday, May 16, 2002 5:39 PM
> To: nfs@lists.sourceforge.net
> Subject: [NFS] RedHat 7.3 crashed NetApp?
> 
> 
> Has anyone seen
> 
> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=65069
> 
> 
> H.J.
> 
> _______________________________________________________________
> 
> Have big pipes? SourceForge.net is looking for download 
> mirrors. We supply
> the hardware. You get the recognition. Email Us: 
> bandwidth@sourceforge.net
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 
> _______________________________________________________________
> 
> Have big pipes? SourceForge.net is looking for download 
> mirrors. We supply
> the hardware. You get the recognition. Email Us: 
> bandwidth@sourceforge.net
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 

_______________________________________________________________

Hundreds of nodes, one monster rendering program.
Now that’s a super model! Visit http://clustering.foundries.sf.net/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[Benjamin LaHaise]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 859 bytes --]

[a few data points from the bugzilla entries]

On Fri, May 17, 2002 at 03:17:53PM -0700, Lever, Charles wrote:
> i'm wondering what are the common elements here:
> 
> is the EEpro 100 ethernet card common to all clients
> that exhibit this problem?

eepro, tulip, 3com have all shown problems with 2.4.18 (at least 
serious performance issues).

>  NFSv3 over UDP?

Yes.  A number of people have now reported that forcing v2 operation 
does not result in slow performance, whereas in the 2.4.9-ac12 based 
kernel, v3 was fine.

		-ben

_______________________________________________________________

Hundreds of nodes, one monster rendering program.
Now that’s a super model! Visit http://clustering.foundries.sf.net/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[seth vidal]

[-- Attachment #1: Type: text/plain, Size: 1009 bytes --]

On Fri, 2002-05-17 at 20:17, Benjamin LaHaise wrote:
> [a few data points from the bugzilla entries]
> 
> On Fri, May 17, 2002 at 03:17:53PM -0700, Lever, Charles wrote:
> > i'm wondering what are the common elements here:
> > 
> > is the EEpro 100 ethernet card common to all clients
> > that exhibit this problem?
> 
> eepro, tulip, 3com have all shown problems with 2.4.18 (at least 
> serious performance issues).
> 
> >  NFSv3 over UDP?
> 
> Yes.  A number of people have now reported that forcing v2 operation 
> does not result in slow performance, whereas in the 2.4.9-ac12 based 
> kernel, v3 was fine.
> 

from the discussion in bugzilla it seemed like a fair number of people
were suffering from the nfs defaults going to nfsv3, upd and 32768 wsize
and rsize/ 32768 bytes. That size of packet is going to be hard for even
REALLY well maintained networks to manage.

It might be worthwhile to try out the nfsv3 client over tcp and see if
the same problems occur. 

-sv


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 232 bytes --]

RE: RedHat 7.3 crashed NetApp?

[Lever, Charles]

ben-

were any patches applied to RH's 2.4.18 that touched the NFS client?
for example, were any of Trond's experimental patches applied?

thanks!

> [a few data points from the bugzilla entries]
> 
> On Fri, May 17, 2002 at 03:17:53PM -0700, Lever, Charles wrote:
> > i'm wondering what are the common elements here:
> > 
> > is the EEpro 100 ethernet card common to all clients
> > that exhibit this problem?
> 
> eepro, tulip, 3com have all shown problems with 2.4.18 (at least 
> serious performance issues).
> 
> >  NFSv3 over UDP?
> 
> Yes.  A number of people have now reported that forcing v2 operation 
> does not result in slow performance, whereas in the 2.4.9-ac12 based 
> kernel, v3 was fine.
> 
> 		-ben
> 

_______________________________________________________________
Hundreds of nodes, one monster rendering program.
Now that's a super model! Visit http://clustering.foundries.sf.net/

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: RedHat 7.3 crashed NetApp?

[Benjamin LaHaise]

On Mon, May 20, 2002 at 07:19:11AM -0700, Lever, Charles wrote:
> ben-
> 
> were any patches applied to RH's 2.4.18 that touched the NFS client?
> for example, were any of Trond's experimental patches applied?

Just a few bits from -ac and the fix for irix nfs cookies.

		-ben
-- 
"You will be reincarnated as a toad; and you will be much happier."

_______________________________________________________________
Hundreds of nodes, one monster rendering program.
Now that's a super model! Visit http://clustering.foundries.sf.net/

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: RedHat 7.3 crashed NetApp?

[Vladimir Vuksan]

We have had a similar problem with NetApp F720 where couple times a day NetApp would stop responding. On one occassion we had an occurence where a single box was trying to write out a 2 GB core dump file to the NetApp which completely hosed it down. Even without misbehaving problems we had issues with it. We had a script from NetApp that was continuously doing sysstat, nfsstat but there were no indications on what was going on. This is all on a network of RedHat 7.2 clients with 2.4.9-31 RedHat stock kernel. What saved us was switching to NFS over TCP. I am not very fond of that particular solution since it is more like treating the symptoms and not the root cause.

Vladimir

--- "Lever, Charles" <Charles.Lever@netapp.com> wrote:
>i'm wondering what are the common elements here:
>
>is the EEpro 100 ethernet card common to all clients
>that exhibit this problem?  NFSv3 over UDP?  what is
>the nature of the network flood -- ICMP packets, or
>UDP?  does "top" on the client show whether a particular
>process or processes is spinning?
>
>a tcpdump or at least "netstat -s" on the client after
>the network is unplugged might be helpful.
>
>> -----Original Message-----
>> From: Cooke, Alan [mailto:acooke@hp.com]
>> Sent: Friday, May 17, 2002 9:15 AM
>> To: nfs@lists.sourceforge.net
>> Subject: RE: [NFS] RedHat 7.3 crashed NetApp?
>> 
>> 
>> While running the 2.4.18-4smp kernel (RedHat), and testing 
>> NFS3, our NetApp filer running OnTap 6.1.1R2 experienced 
>> massive incoming traffic from the Linux client in question. 
>> The Linux client required rebooting. We have been able to 
>> repeat this behavior, and are suspicious of the network 
>> driver on the Linux client which is using the eth0 driver. A
>> 
>> Sound familiar?
>> 
>> Alan Cooke
>> Hewlett-Packard Company
>> ISS/EDA
>> Office: 281-518-9775
>> Fax    : 281-518-8015
>> Email  : acooke@hp.com
>> 
>> "If everything seems under control, you're just not going 
>> fast enough."
>> -- Mario Andretti
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> -----Original Message-----
>> From: H . J . Lu [mailto:hjl@lucon.org]
>> Sent: Thursday, May 16, 2002 5:39 PM
>> To: nfs@lists.sourceforge.net
>> Subject: [NFS] RedHat 7.3 crashed NetApp?
>> 
>> 
>> Has anyone seen
>> 
>> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=65069
>> 
>> 
>> H.J.
>> 
>> _______________________________________________________________
>> 
>> Have big pipes? SourceForge.net is looking for download 
>> mirrors. We supply
>> the hardware. You get the recognition. Email Us: 
>> bandwidth@sourceforge.net
>> _______________________________________________
>> NFS maillist  -  NFS@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs
>> 
>> _______________________________________________________________
>> 
>> Have big pipes? SourceForge.net is looking for download 
>> mirrors. We supply
>> the hardware. You get the recognition. Email Us: 
>> bandwidth@sourceforge.net
>> _______________________________________________
>> NFS maillist  -  NFS@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs
>> 
>
>_______________________________________________________________
>
>Hundreds of nodes, one monster rendering program.
>Now that&#8217;s a super model! Visit http://clustering.foundries.sf.net/
>_______________________________________________
>NFS maillist  -  NFS@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/nfs

_____________________________________________________________
Want a new web-based email account ? ---> http://www.firstlinux.net

_____________________________________________________________
Promote your group and strengthen ties to your members with email@yourgroup.org by Everyone.net  http://www.everyone.net/?btn=tag

_______________________________________________________________

Don't miss the 2002 Sprint PCS Application Developer's Conference
August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs