Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Charles-Edouard Ruault]

On Mar 17, 2004, at 5:58 PM, Trond Myklebust wrote:

> P=E5 on , 17/03/2004 klokka 05:44, skreiv Charles-Edouard Ruault:
>
> [snip]

> Your Mac server is replying slooooooooowwwwwwwwllllllllllyyyyyyyyyyy.
> That either indicates that you have a dirty network which is losing
> packets (can happen if you are mixing 10Mbps and 100Mbps segments) or
> the Mac server is hanging.
>
> Cheers,
>   Trond
>
Hi Trond,
thanks for the quick reply.
As you pointed out i've got a network which mixes 100Mbps and 1Gbps (=20
the mac is on 1Gps and the linux on 100Mbps ).
However i've done a little troubleshooting on the network and i achieve=
=20
full 100Mbps speed scp'ing files from/to both machines.
I've also tried to use nfs over tcp instead of udp and it does not=20
change anything.
I've got a Gigabit card for the Linux box and i've scheduled to install=
=20
it within the next few days. I'll see if this helps or not.
In the meantime, any other hint is welcome.
Thanks.


Charles-Edouard Ruault
Idtect SA
tel: +33-1-42-81-81-84
fax: +33-1-42-81-82-21
http://www.idtect.com
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel=
" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Chris Croswhite]

I have seen similar issues, but with slowlaris NFS server to linux.=20
Never was able to resolve the issue and so moved to linux NFS server. =20

I am willing to spend sometime on this issue if Trond can give me some
assistance?  I can detail the issue if you have you are available.

TIA,
Chris

On Wed, 2004-03-17 at 09:41, Charles-Edouard Ruault wrote:
> On Mar 17, 2004, at 5:58 PM, Trond Myklebust wrote:
>=20
> > P=C3=A5 on , 17/03/2004 klokka 05:44, skreiv Charles-Edouard Ruault:
> >
> > [snip]
>=20
> > Your Mac server is replying slooooooooowwwwwwwwllllllllllyyyyyyyyyyy.
> > That either indicates that you have a dirty network which is losing
> > packets (can happen if you are mixing 10Mbps and 100Mbps segments) or
> > the Mac server is hanging.
> >
> > Cheers,
> >   Trond
> >
> Hi Trond,
> thanks for the quick reply.
> As you pointed out i've got a network which mixes 100Mbps and 1Gbps (=20
> the mac is on 1Gps and the linux on 100Mbps ).
> However i've done a little troubleshooting on the network and i achieve=
=20
> full 100Mbps speed scp'ing files from/to both machines.
> I've also tried to use nfs over tcp instead of udp and it does not=20
> change anything.
> I've got a Gigabit card for the Linux box and i've scheduled to install=
=20
> it within the next few days. I'll see if this helps or not.
> In the meantime, any other hint is welcome.
> Thanks.
>=20
>=20
> Charles-Edouard Ruault
> Idtect SA
> tel: +33-1-42-81-81-84
> fax: +33-1-42-81-82-21
> http://www.idtect.com
>=20
>=20
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id638&op=3Dclick
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Trond Myklebust]

P=E5 on , 17/03/2004 klokka 13:01, skreiv Chris Croswhite:
> I have seen similar issues, but with slowlaris NFS server to linux.=20
> Never was able to resolve the issue and so moved to linux NFS server. =20
>=20
> I am willing to spend sometime on this issue if Trond can give me some
> assistance?  I can detail the issue if you have you are available.

It's very simple: if you using UDP as the transport mechanism, then
packets can (and *will*) be lost.
This is particularly true if you are working across a net which mixes
network speeds, since the switches have to queue data when going from
the fast to the slower net: once this queue has built up to the point
where the switch runs out of memory, it will start dropping incoming
packets.

IOW: this is not an NFS client bug, it is a network design bug.

In these environments you *must* use TCP, since that has congestion
control capabilities baked into the protocol...

For more detailed info on how to deal with this sort of problem, I
suggest the NFS FAQ and HOWTO: see http://nfs.sourceforge.net/

Cheers,
  Trond



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Charles-Edouard Ruault]

Trond Myklebust wrote:

>P=E5 on , 17/03/2004 klokka 13:01, skreiv Chris Croswhite:
> =20
>
>>I have seen similar issues, but with slowlaris NFS server to linux.=20
>>Never was able to resolve the issue and so moved to linux NFS server. =20
>>
>>I am willing to spend sometime on this issue if Trond can give me some
>>assistance?  I can detail the issue if you have you are available.
>>   =20
>>
>
>It's very simple: if you using UDP as the transport mechanism, then
>packets can (and *will*) be lost.
> =20
>
yep, that's why i tried switching to TCP ... but with no better result.

>This is particularly true if you are working across a net which mixes
>network speeds, since the switches have to queue data when going from
>the fast to the slower net: once this queue has built up to the point
>where the switch runs out of memory, it will start dropping incoming
>packets.
>
>IOW: this is not an NFS client bug, it is a network design bug.
>
> =20
>
We'll see if this disappears after we've finished migrating the network=20
to 1Gbps

>In these environments you *must* use TCP, since that has congestion
>control capabilities baked into the protocol...
>
>For more detailed info on how to deal with this sort of problem, I
>suggest the NFS FAQ and HOWTO: see http://nfs.sourceforge.net/
>
> =20
>
I'll definitely have a look and try again with TCP just in case i can=20
capture a different error log.
Thanks again for the info.

>Cheers,
>  Trond
> =20
>



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Lever, Charles]

charles-

> >It's very simple: if you using UDP as the transport mechanism, then=20
> >packets can (and *will*) be lost.
> > =20
> >
> yep, that's why i tried switching to TCP ... but with no=20
> better result.
>=20
> >This is particularly true if you are working across a net=20
> which mixes=20
> >network speeds, since the switches have to queue data when=20
> going from=20
> >the fast to the slower net: once this queue has built up to=20
> the point=20
> >where the switch runs out of memory, it will start dropping incoming=20
> >packets.
> >
> >IOW: this is not an NFS client bug, it is a network design bug.
> >
> > =20
> >
> We'll see if this disappears after we've finished migrating=20
> the network=20
> to 1Gbps
>=20
> >In these environments you *must* use TCP, since that has congestion=20
> >control capabilities baked into the protocol...

what matters here is flow control.  UDP simply doesn't have it.

for gigabit Ethernet, you need flow control at the link level and
at the transport level.  so once you have gigabit infrastructure,
be sure you have enabled full gigabit flow control on your servers
and on your switches.  then you should use TCP and not UDP so you
have transport layer flow control.


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: Linux 2.4.25, nfs client hangs when talking to a MacOS nfs server.

[Charles-Edouard Ruault]

On Mar 17, 2004, at 8:56 PM, Lever, Charles wrote:

> charles-
>
>>> [ snip ]
>>>
>>>
>> We'll see if this disappears after we've finished migrating
>> the network
>> to 1Gbps
>>
>>> In these environments you *must* use TCP, since that has congestion
>>> control capabilities baked into the protocol...
>
> what matters here is flow control.  UDP simply doesn't have it.
>
> for gigabit Ethernet, you need flow control at the link level and
> at the transport level.  so once you have gigabit infrastructure,
> be sure you have enabled full gigabit flow control on your servers
> and on your switches.  then you should use TCP and not UDP so you
> have transport layer flow control.
>
Charles and Trond,
thanks for the advices !
I've upgraded the network cards of the linux machines to Gigabit 
ethernet cards and used tcp for my nfs mounts.
The problem disappeard so far so i'm pretty happy !
However i'm very disappointed by the throuhput i'm getting on the Gbps 
network :
When i scp a 473991492 bytes file from one machine to the other, i'm 
getting 14Mbps throughput, which is far less that 1Gpbs theoretical.
Looking at the network card info ( i'm using the sk98lin driver ) i've :
Mar 18 14:49:15 omnirot kernel: eth0: network connection up using port A
Mar 18 14:49:15 omnirot kernel:     speed:           1000
Mar 18 14:49:15 omnirot kernel:     autonegotiation: yes
Mar 18 14:49:15 omnirot kernel:     duplex mode:     full
Mar 18 14:49:15 omnirot kernel:     flowctrl:        symmetric
Mar 18 14:49:15 omnirot kernel:     role:            slave
Mar 18 14:49:15 omnirot kernel:     irq moderation:  disabled
Mar 18 14:49:15 omnirot kernel:     scatter-gather:  enabled

The switch i'm using is a Netgear GS516T

i've tried transferring to/from 2 linux machines connected to this 
switch using the exact same kernel and network cards and also to Mac 
Xserv using a Gbps card . They both yield the same low throuput.
Any hint of what could be causing this ?
Thanks in advance.


Charles-Edouard Ruault
Idtect SA
tel: +33-1-42-81-81-84
fax: +33-1-42-81-82-21
http://www.idtect.com



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs