Re: async vs. sync

[jehan procaccia <jehan.procaccia@int-evry.fr>]

jehan procaccia wrote:

> [root@arvouin tmp]# mount cobra3:/p2v5f1 -o 
> async,wsize=32768,rsize=32768,soft /mnt/cobra3
> [root@arvouin /mnt/cobra3/mci/test/Test-sync]
> $time tar xvfz /usr/src/redhat/SOURCES/httpd-2.0.51.tar.gz
>
> sorry I don't want to wait more than 10 minutes to send that mail, but 
> again seeing the files apperaing very slowly on the tty it seems not 
> to be the solution :-( .

For the record ... it finnaly ended, not very better than the original 
13 minutes !
real    12m53.520s
user    0m1.070s
sys     0m4.268s

>
>> Your test may also be somewhat misleading as it is performing
>> a lot of file creates and those take a lot of time no matter
>> how you do it, do the normal user applications do a lot of file 
>> creates/opens or do the normal users do a lot of large
>> file writes?   How many files are in that tar file? And what
>> is their average size?    File creates and file opens are one
>> place that NFS tends to have a large difference in speed.
>>  
>>
> 3000 files with often less than 100Ko or even 10Ko in size  in the 
> httpd source tarball. Sure it won't be the day to day usage of our 
> students, although they are teach computer science in our school and 
> might have that kind of usage , anyway "untar, gunzip, make, make 
> install" is one of Mine favorite usage ;-) , maybe I'am too selfish !.
>
>>                                 Roger
>>
>>  
>>
>>> -----Original Message-----
>>> From: jehan.procaccia [mailto:jehan.procaccia@int-evry.fr] Sent: 
>>> Monday, November 22, 2004 12:47 PM
>>> To: Roger Heflin
>>> Cc: 'Olaf Kirch'; nfs@lists.sourceforge.net; 
>>> trond.myklebust@fys.uio.no; mci-unix@int-evry.fr
>>> Subject: Re: [NFS] async vs. sync
>>>
>>> OK , thanks for the explanation . So now I try:
>>>
>>> exports          mount
>>> sync             async     -> Safe and fast
>>>
>>> [root@arvouin ~]
>>> $mount cobra3:/p2v5f1/ -o async /mnt/cobra3
>>>
>>> unfortunatly the async mount option  doesn't shows up in 
>>> /proc/mounts, so I am not sure my client is using async, how can I 
>>> check that ?
>>>
>>> [root@arvouin ~]
>>> $cat /proc/mounts
>>> cobra3:/p2v5f1/ /mnt/cobra3 nfs
>>> rw,v3,rsize=8192,wsize=8192,hard,tcp,lock,addr=cobra3 0 0
>>>
>>> Anyway, I untar again my httpd, I'am late so I didn't wait 13 minute 
>>> or so but it started to be as long as it was :-(
>>>
>>> Now on the server I removed ACL (nerver asked for it though !?) $ 
>>> cat /proc/fs/nfs/exports | grep arvouin
>>> /p2v5f1 arvouin.int-evry.fr(rw,no_root_squash,sync,wdelay,no_acl) #
>>> 157.159.21.55
>>> again I cannot check on the client that this option is removed  ?, 
>>> anyway again I untar my httpd,  I'am still late ... but seeing each 
>>> file (tar -v !) shouwing up every second or so in the tty tells me 
>>> that it will take at least 10 minutes ... :-(
>>>
>>> Any other idea ? what is wrong in my config ?
>>>
>>> thanks.
>>>
>>>
>>>
>>>
>>> Roger Heflin wrote:
>>>
>>>   
>>>
>>>> This might be the issue, (and someone correct this if I am     
>>>
>>> incorrect),   
>>>
>>>> I know I ran into it a couple of years ago, and it is not     
>>>
>>> the easiest   
>>>
>>>> to understand exactly what is actually going on.
>>>>
>>>> There are 2 places where you can put sync and async, one is     
>>>
>>> the exports   
>>>
>>>> and one is on the mount command.  They are different.
>>>>
>>>> You want sync on the exports, this will allow a client to     
>>>
>>> survive without
>>>   
>>>
>>>> data loss if the server reboots.   You want async on the     
>>>
>>> client mount end
>>>   
>>>
>>>> and
>>>> this will generally give you the speed.   With async on the     
>>>
>>> client end
>>>   
>>>
>>>> the client is keeping track of what is outstanding if the server 
>>>> crashes, so you won't lose data.  With async on both ends the 
>>>> server tells one that the data is safe when it is not, and if the 
>>>> server crashes the client things that the data was safe when it     
>>>
>>> really was not.
>>>   
>>>
>>>> If you put sync in both locations then your NFS disk is fully 
>>>> synced and the application won't even start another write until the 
>>>>     
>>>
>>> last one   
>>>
>>>> is confirmed
>>>> finished and on the actual disk.   With async on the client     
>>>
>>> end the next
>>>   
>>>
>>>> write
>>>> will start before the client has received an ack from the     
>>>
>>> server, and   
>>>
>>>> this will be reasonably fast.
>>>>
>>>> So basically:
>>>>
>>>> exports          mount
>>>> sync             sync      -> really safe and really slow
>>>> sync             async     -> Safe and fast
>>>> async            either    -> unsafe and fast.
>>>>
>>>> Running async exports and async mount did not appear (under     
>>>
>>> my testing)   
>>>
>>>> to be faster under a sustained load than did sync exports     
>>>
>>> and async mount.
>>>   
>>>
>>>> When
>>>> the initial test was started async/async was faster but that quick 
>>>> changed once the buffer cache filled up.
>>>>
>>>>                      Roger
>>>>
>>>>
>>>>
>>>>
>>>>     
>>>>
>>>>> -----Original Message-----
>>>>> From: nfs-admin@lists.sourceforge.net 
>>>>> [mailto:nfs-admin@lists.sourceforge.net] On Behalf Of       
>>>>
>>> jehan.procaccia
>>>   
>>>
>>>>> Sent: Monday, November 22, 2004 11:55 AM
>>>>> To: Olaf Kirch
>>>>> Cc: nfs@lists.sourceforge.net; trond.myklebust@fys.uio.no; 
>>>>> mci-unix@int-evry.fr
>>>>> Subject: Re: [NFS] async vs. sync
>>>>>
>>>>> Olaf Kirch wrote:
>>>>>
>>>>>  
>>>>>       
>>>>>
>>>>>> On Tue, Nov 16, 2004 at 10:48:12AM -0800, Lever, Charles wrote:
>>>>>>
>>>>>>
>>>>>>    
>>>>>>         
>>>>>>
>>>>>>> i'm just looking for clarification so i can provide a good
>>>>>>>      
>>>>>>>           
>>>>>>
>>>>> explanation
>>>>>  
>>>>>       
>>>>>
>>>>>>> in the Linux NFS FAQ about the evils of using "async."            
>>>>>>
>>> i'll cruise   
>>>
>>>>>>> through the server code.
>>>>>>>  
>>>>>>>
>>>>>>>      
>>>>>>>           
>>>>>>
>>>>>> Just about the only reason for async I can think of is if
>>>>>>    
>>>>>>         
>>>>>
>>>>> you have an
>>>>>  
>>>>>       
>>>>>
>>>>>> incoming data stream you need to write at a constant rate
>>>>>>    
>>>>>>         
>>>>>
>>>>> (think of a
>>>>>  
>>>>>       
>>>>>
>>>>>> diskless set top box writing an mpeg2 stream)
>>>>>>
>>>>>> Olaf
>>>>>>
>>>>>>
>>>>>>    
>>>>>>         
>>>>>
>>>>> OK, but using sync at my site is really really slow ... compare to 
>>>>> async ! here's a detailed (options printed)       
>>>>
>>> demonstration   
>>>
>>>>> for an untar operation that takes 13 minutes in async mode       
>>>>
>>> and only 14   
>>>
>>>>> secondes in sync mode !!
>>>>>
>>>>> 1) Export in sync mode
>>>>> NFS server  (RedHat ES3 kernel 2.4.21-4.ELsmp) options for that 
>>>>> export:
>>>>> $ cat /proc/fs/nfs/exports | grep arvouin
>>>>> /p2v5f1 arvouin.int-evry.fr(rw,no_root_squash,sync,wdelay,acl) #
>>>>> 157.159.21.55
>>>>> $ cat /var/lib/nfs/xtab  | grep arvouin
>>>>> /p2v5f1
>>>>> arvouin.int-evry.fr(rw,sync,wdelay,hide,nocrossmnt,secure,no_r
>>>>>  
>>>>>       
>>>>
>>>> oot_squash,no_all_squash,subtree_check,secure_locks,acl,mappi
>>>>     
>>>
>>> ng=identit
>>>   
>>>
>>>> y,ano
>>>> nuid=-2,anongid=-2)
>>>>
>>>>
>>>>     
>>>>
>>>>> Client running Fedora Core 2, kernel 2.6.8-1.521 [root@arvouin 
>>>>> /mnt/cobra3/mci/test/Test-sync] $cat /proc/mounts
>>>>> cobra3:/p2v5f1 /mnt/cobra3 nfs
>>>>> rw,v3,rsize=8192,wsize=8192,hard,tcp,lock,addr=cobra3 0 0 $time 
>>>>> tar xvfz /usr/src/redhat/SOURCES/httpd-2.0.51.tar.gz
>>>>> real    13m3.686s
>>>>> user    0m1.055s
>>>>> sys     0m4.354s
>>>>>
>>>>> 2) Export in async mode:
>>>>> Same NFS server, options for that export:
>>>>> $ cat /proc/fs/nfs/exports | grep arvouin
>>>>> /p2v5f1 arvouin.int-evry.fr(rw,no_root_squash,async,wdelay,acl) #
>>>>> 157.159.21.55
>>>>> $ cat /var/lib/nfs/xtab  | grep arvouin
>>>>> /p2v5f2
>>>>> arvouin.int-evry.fr(rw,async,wdelay,hide,nocrossmnt,secure,no_
>>>>>  
>>>>>       
>>>>
>>>> root_squash,no_all_squash,subtree_check,secure_locks,acl,mapp
>>>>     
>>>
>>> ing=identi
>>>   
>>>
>>>> ty,an
>>>> onuid=-2,anongid=-2)
>>>>
>>>>
>>>>     
>>>>
>>>>> Same client running Fedora Core 2, kernel 2.6.8-1.521
>>>>> cobra3:/p2v5f1 /mnt/cobra3 nfs
>>>>> rw,v3,rsize=8192,wsize=8192,hard,tcp,lock,addr=cobra3 0 0 
>>>>> [root@arvouin /mnt/cobra3/mci/test/Test-sync] $time tar xvfz 
>>>>> /usr/src/redhat/SOURCES/httpd-2.0.51.tar.gz
>>>>> real    0m14.802s
>>>>> user    0m0.867s
>>>>> sys     0m2.886s
>>>>>
>>>>> My users won't accept the sync performances ! . I have no       
>>>>
>>> choice, but   
>>>
>>>>> is running in async mode is really evil as you mentioned it       
>>>>
>>> ? is there   
>>>
>>>>> a way to have better performances in sync in my case ?  As       
>>>>
>>> anyone had   
>>>
>>>>> the same gap in performance as me ( here 55 times longer in       
>>>>
>>> sync mode   
>>>
>>>>> !)  ?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> SF email is sponsored by - The IT Product Guide Read honest       
>>>>
>>> & candid   
>>>
>>>>> reviews on hundreds of IT Products from real users.
>>>>> Discover which products truly live up to the hype. Start       
>>>>
>>> reading now.   
>>>
>>>>> http://productguide.itmanagersjournal.com/
>>>>> _______________________________________________
>>>>> NFS maillist  -  NFS@lists.sourceforge.net 
>>>>> https://lists.sourceforge.net/lists/listinfo/nfs
>>>>>
>>>>>  
>>>>>       
>>>>
>>>>
>>>>
>>>>     
>>>
>>
>>  
>>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs